JP2013012942A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus allowing visual recognition of proper tracking as being executed.SOLUTION: A face to be tracked is tracked by tracking processing using a face detection circuit 1307, tracking processing using luminance information by a tracking processing circuit 1305, and tracking processing using color information by a tracking processing circuit 1306. A face periphery to be tracked is tracked by estimation from the position of the face detected by the tracking processing using the face detection circuit 1307, and tracked by tracking processing using luminance information by the tracking processing circuit and tracking processing using color information by the tracking processing circuit 1306. The display position of a tracking frame is changed in accordance with the reliability of the detection result of the face to be tracked and the detection result of the face periphery to be tracked.

Description

  The present invention relates to an imaging apparatus having a function of tracking a subject.

  2. Description of the Related Art Conventionally, techniques for performing automatic focusing control (AF) or automatic exposure control (AE) so as to follow a specific subject when shooting a moving object or shooting a moving image are known. Tracking processing is used to follow such a specific subject. There are various types of tracking processes, such as those using luminance information, those using color information, and those using face detection. Patent Document 1 discloses an automatic tracking device that performs a tracking operation on a subject image based on an imaging signal output from an imaging unit.

  In the automatic tracking device disclosed in Patent Document 1, parameters representing the characteristics of the subject (for example, the color of the subject and the background) are extracted for a preset tracking field of view, and the extracted features and newly extracted Based on the characteristics of the subject, the presence / absence of movement of the subject and, if the subject moves, the moving direction or moving position thereof are detected. Then, the tracking frame is moved following the movement of the subject.

Japanese Patent No. 2605004

  By the way, when only one type of tracking process is used, if the tracking process fails, the correct tracking process may not be possible thereafter. For example, in the case of tracking processing using face detection, if a face is no longer detected while tracking a face on the screen, there is a possibility of switching to another face on the screen for tracking. There is. When such a transfer occurs, the tracking frame also moves to the other face. In this case, the user needs to reset the tracking target.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an imaging apparatus that can visually recognize that an appropriate tracking process is being performed.

  To achieve the above object, an imaging device according to a first aspect of the present invention includes an imaging unit that images a subject and outputs image data, and a face detection unit that detects a face portion of the subject from the image data A first subject tracking unit that tracks the face of the subject in the image data based on the image data, and a first tracking unit that tracks the periphery of the face of the subject in the image data based on the image data. While the two subject tracking units, the display unit displaying an image based on the image data, and the detection of the face by the face detection unit or the tracking of the face by the first subject tracking unit are effective The display of the position of the face is superimposed on the image displayed on the display unit, and the detection of the face by the face detection unit and the tracking of the face by the first subject tracking unit are invalid When The image displayed on the serial display unit, characterized by comprising a control unit that superimposes a display frame indicating the position of the face peripheral portion.

  In order to achieve the above object, an imaging apparatus according to a second aspect of the present invention includes an imaging unit that captures an image of a subject and outputs image data, and a face that detects the face of the subject from the image data. A detection unit; a face periphery tracking unit that tracks the face periphery of the subject in the image data based on the image data; a display unit that displays an image based on the image data; and the face by the face detection unit While the detection of the face is valid, a display frame indicating the position of the face is superimposed on the image displayed on the display, and the detection of the face by the face detection becomes invalid And a control unit that superimposes a display frame indicating the position of the peripheral portion of the face on the image displayed on the display unit.

  ADVANTAGE OF THE INVENTION According to this invention, the imaging device which a user can visually recognize that the appropriate tracking process is performed can be provided.

It is a figure which shows the structure as an example of the imaging device provided with the tracking apparatus which concerns on one Embodiment of this invention. It is a figure for demonstrating the tracking process using luminance information. It is a figure for demonstrating the tracking process using color information. It is a figure for demonstrating a face detection process. It is a flowchart which shows the imaging operation of an imaging device. It is a flowchart shown about the tracking process as a tracking method which concerns on one Embodiment of this invention. It is a flowchart shown about a periphery position estimation process. It is a flowchart shown about each part tracking process. It is a flowchart shown about a candidate tracking position calculation process. It is a flowchart shown about a tracking position update determination process. It is a figure shown about the specific example of a tracking position update determination process. It is a figure shown about the example of a display of a display frame.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram illustrating a configuration as an example of an imaging apparatus including a tracking device according to an embodiment of the present invention. An imaging apparatus 100 illustrated in FIG. 1 includes a photographing optical system 102, a focus adjustment mechanism 104, a diaphragm 106, a diaphragm driving mechanism 108, a shutter 110, a shutter driving mechanism 112, an imaging element 114, and an imaging element interface ( IF) circuit 116, RAM 118, display element 120, display element drive circuit 122, touch panel 124, touch panel drive circuit 126, recording medium 128, system controller 130, operation unit 132, and ROM 134. Have.

  The photographing optical system 102 is an optical system for condensing a light flux F from a subject (not shown) on the light receiving surface of the image sensor 114. The photographing optical system 102 has a plurality of lenses such as a focus lens. The focus adjustment mechanism 104 includes a motor and its drive circuit. The focus adjustment mechanism 104 drives the focus lens in the photographing optical system 102 in the optical axis direction (the one-dot chain line direction in the drawing) under the control of the CPU 1301 in the system controller 130.

  The diaphragm 106 is configured to be openable and closable, and adjusts the amount of the light beam F incident on the image sensor 114 via the photographing optical system 102. The aperture drive mechanism 108 has a drive mechanism for driving the aperture 106. The diaphragm driving mechanism 108 drives the diaphragm 106 according to the control of the CPU 1301 in the system controller 130.

  The shutter 110 is configured to place the light receiving surface of the image sensor 114 in a light shielding state or an exposure state. The exposure time of the image sensor 114 is adjusted by the shutter 110. The shutter drive mechanism 112 has a drive mechanism for driving the shutter 110, and drives the shutter 110 under the control of the CPU 1301 in the system controller 130.

  The imaging element 114 has a light receiving surface on which a light flux F from a subject condensed through the photographing optical system 102 is imaged. The light receiving surface of the image sensor 114 is configured by two-dimensionally arranging a plurality of pixels, and a color filter is provided on the light incident side of the light receiving surface. Such an image sensor 114 converts an image (subject image) corresponding to the light beam F formed on the light receiving surface into an electrical signal (hereinafter referred to as an image signal) corresponding to the light amount. Here, as the imaging device 114, imaging devices having various configurations such as a CCD system and a CMOS system are known. Various color filters such as a Bayer array are also known. In the present embodiment, the configuration of the image sensor 114 is not limited to a specific configuration, and image sensors having various configurations can be used.

  The image sensor IF circuit 116 drives the image sensor 114 under the control of the CPU 1301 in the system controller 130. The image sensor IF circuit 116 reads out the image signal obtained by the image sensor 114 under the control of the CPU 1301 in the system controller 130, and performs CDS (correlated double sampling) processing or AGC (automatic operation) on the read image signal. Analog processing such as gain control processing is performed. Further, the image sensor IF circuit 116 converts the analog processed image signal into a digital signal (hereinafter referred to as image data).

The RAM 118 is, for example, an SDRAM, and has a work area, an evaluation image area, a reference image area, a tracking position log area, and a tracking color area log area as storage areas.
The work area is a storage area provided in the RAM 118 for temporarily storing data generated in each part of the imaging apparatus 100 such as image data obtained by the imaging element IF circuit 116.
The evaluation image area is a storage area provided in the RAM 118 for temporarily storing evaluation image data. The evaluation image data is image data of a frame including a subject to be tracked in a tracking process described later. In the tracking process, processing is performed so as to track this tracking target.
The reference image area is a storage area provided in the RAM 118 for temporarily storing reference image data. The reference image data is image data of a frame to be searched for a tracking target in tracking processing described later. In the tracking process, a search is performed in the reference image data.
The tracking position log area is a storage area provided in the RAM 118 for temporarily storing the tracking position log. The tracking position log is a log in which the tracking position obtained as a result of the tracking process is recorded. In the present embodiment, a tracking target is tracked using a plurality of tracking processes in combination. Therefore, in the tracking position log, the tracking positions (candidate tracking positions) obtained by the respective tracking processes are individually recorded, for example, for the past 10 frames. The tracking position log also records the final tracking position adopted by the tracking process described later.
The tracking color area log area is a storage area provided in the RAM 118 for temporarily storing the tracking color area log. The tracking color area log is a log in which the tracking color area obtained by the tracking process is recorded. In this embodiment, the tracking color area is recorded for the past 10 frames, for example. Details of the tracking color area will be described later.

  The display element 120 is a liquid crystal display (LCD), for example, and displays various images such as an image for live view and an image recorded on the recording medium 128. The display element driving circuit 122 drives the display element 120 based on the image data input from the CPU 1301 of the system controller 130 and causes the display element 120 to display an image.

  The touch panel 124 is integrally formed on the display screen of the display element 120, and detects a contact position of a user's finger or the like on the display screen. The touch panel drive circuit 126 drives the touch panel 124 and outputs a contact detection signal from the touch panel 124 to the CPU 1301 of the system controller 130. The CPU 1301 detects a contact operation on the display screen of the user from the contact detection signal, and executes processing according to the contact operation.

  The recording medium 128 is a memory card, for example, and records an image file obtained by a shooting operation. The image file is a file configured by adding a predetermined header to image data. In the header, data indicating shooting conditions, data indicating a tracking position, and the like are recorded as tag data.

  The system controller 130 is a control circuit for controlling the operation of the imaging apparatus 100. The CPU 1301, the AF control circuit 1302, the AE control circuit 1303, the image processing circuit 1304, the tracking processing circuits 1305 and 1306, and face detection. A circuit 1307, a face orientation determination circuit 1308, a color information acquisition circuit 1309, and a memory control circuit 1310 are included.

  The CPU 1301 has a function as a control unit. Each block outside the system controller 130 such as the focus adjustment mechanism 104, the aperture driving mechanism 108, the shutter driving mechanism 112, the display element driving circuit 122, and the touch panel driving circuit 126, and the system The operation of each control circuit in the controller 130 is controlled. The CPU 1301 also has a function as a second subject tracking unit, and estimates the position of the face peripheral part from the position of the face part to be tracked detected by the face detection circuit 1307.

  The AF control circuit 1302 controls contrast AF processing. Specifically, the AF control circuit 1302 extracts the high frequency component of the image data obtained by the image sensor IF circuit 116, and acquires the focus evaluation value for AF by integrating the extracted high frequency component. . The CPU 1301 controls the focus adjustment mechanism 104 to bring the focus lens into focus while evaluating the contrast of the image data according to the focus evaluation value.

  The AE control circuit 1303 controls the AE operation. Specifically, the AE control circuit 1303 calculates subject luminance using the image data obtained by the image sensor IF circuit 116. The CPU 1301 calculates the aperture amount (aperture value) of the aperture 106 during exposure, the opening time of the shutter 110 (shutter speed value), the image sensor sensitivity, the ISO sensitivity, and the like according to the subject brightness.

  The image processing circuit 1304 performs various image processing on the image data. This image processing includes color correction processing, gamma (γ) correction processing, compression processing, and the like. The image processing circuit 1304 also performs decompression processing on the compressed image data.

The tracking processing circuit 1305 performs tracking processing using the luminance information of the image data as the second feature information. The tracking process using this luminance information will be briefly described. In the tracking process using luminance information, for example, when a tracking target is set in the (N-1) frame shown in FIG. 2A, the image data of the (N-1) frame is used as the evaluation image data in the RAM 118. Is stored in the evaluation image area. Image data in a predetermined range 202 including the tracking target of the evaluation image data is set as reference image data. In the subsequent tracking process, a portion corresponding to the reference image data 202 of the reference image data is searched.
As an example of N frame tracking processing, N frame image data is first stored in the reference image area of the RAM 118 as reference image data. A portion of the reference image data corresponding to the reference image data 202 is searched for by obtaining a correlation amount between the image data in the predetermined search range 204 and the reference image data 202. The correlation amount is determined from, for example, the sum of absolute differences between the standard image data and the reference image data (obtained by obtaining absolute values of luminance differences for each pixel and integrating them). For example, if the sum of absolute differences between the reference image data in the reference image data area 206 and the standard image data 202 shown in FIG. 2B is obtained, the reference image data area 206 and the standard image data 202 are clearly shown. It is different image data, and the sum of absolute differences is increased. On the other hand, if the sum of absolute differences between the reference image data area 208 and the standard image data 202 is obtained, the sum of absolute differences becomes small. Thus, the difference absolute value sum decreases as the correlation amount with the reference image data 202 increases. In the tracking process using luminance information, an area having the maximum correlation amount, that is, the minimum difference absolute value sum is searched from the reference image data. In the example of FIG. In the tracking position log area, the position with the highest degree of matching in the area 208 is recorded as the tracking position. When there are a plurality of such positions, for example, a position close to the center of the region 208 is set as the tracking position. In the next tracking process, this tracking position is preferably set as the start position of the tracking process. As a result, the time required for the tracking process can be reduced.

  The tracking processing circuit 1306 performs a tracking process using the color information of the image data as the first feature information. The tracking process using this color information will be briefly described. In the tracking process using color information, a tracking color area that is an area that can be determined to be the same color as the color set in the evaluation image data is searched. As shown in FIG. 3A, when the position 302 of the subject is designated in the (N-1) frame, the color information of the position 302 in the evaluation image data is acquired. Then, an area having the same color information as the position 302 is searched for using the position 302 as the start position of the tracking process. Specifically, the color information is sequentially acquired from the start position toward the periphery from the start position, and if it can be determined that the acquired color information is the same as the color information at the position 302, the acquired color information is included in the area. When it cannot be determined that the information is the same as the color information at the position 302, it is not included in the area. When the tracking color area is searched in this way, for example, in the case of a monochromatic subject as shown in FIG. 3A, a rectangular area 304 inscribed in the subject becomes the tracking color area. Further, the tracking position recorded in the tracking position log area is, for example, the barycentric position of the tracking color area 304 (same as the position 302 in the example of FIG. 3A). In the next tracking process, this tracking position is set as the starting position of the tracking process.

  As an example of the N frame tracking process, the tracking position 302 of the (N-1) frame is tracked within a predetermined search range of the N frame image data stored as the reference image data shown in FIG. An area that can be determined to be the same color as the color of the tracking color area 304 is sequentially searched from the periphery of the tracking position 302 as the processing start position. In the example of FIG. 3B, the area 306 is a tracking color area. In the example of FIG. 3B, the center of gravity position 308 is set as the tracking position, and the tracking position 308 is recorded in the tracking position log area. Also, information indicating the range of the tracking color area 306 (for example, the positions of the four corners) is recorded in the tracking color area log area.

  Here, the tracking processing circuits 1305 and 1306 function as an example of the first subject tracking unit, the second subject tracking unit, and the eyelid periphery tracking unit.

  The face detection circuit 1307 detects the face portion of the subject (person) in the image data. Here, the face detection process will be briefly described. In the face detection process, the correlation amount between the image data obtained in each frame and the face parts 402, 404, and 406 as shown in FIG. The face part 402 is image data corresponding to a shadow pattern around a person's nose, the face part 404 is image data corresponding to a shadow pattern around a person's eye, and the face part 406 is a person This is image data corresponding to the shadow pattern around the mouth. The amount of correlation between the image data and the face part parts 402, 404, and 406 is maximized when a predetermined arrangement representing a person's face is obtained as shown in FIG. At this time, it is assumed that a face exists in an area 408 including the face parts 402, 404, and 406. Note that the size of the face parts 402, 404, and 406 may be changed according to a preset search face size. Here, in FIG. 4B, the face area is a rectangular area, but it may be a circular area.

  The face orientation determination circuit 1308 determines the orientation of the face detected by the face detection circuit 1307 from the arrangement of the face parts 402, 404, and 406 in the area 408. For example, when the face parts 402, 404, and 406 are gathered on one of the left and right sides of the area 408, it is determined that the face is in the horizontal direction.

The color information acquisition circuit 1309 acquires the color information of the tracking position obtained by the tracking processing circuit 1305. This color information is used in a candidate tracking position calculation process described later.
The memory control circuit 1310 is an interface that performs control for the CPU 1301 and the like to access the RAM 118, the recording medium 128, and the ROM 134.

The operation unit 132 is various operation members operated by the user. The operation unit 132 includes, for example, a release button, a moving image button, a mode dial, a selection key, a power button, and the like.
The release button button has a 1st release switch and a 2nd release switch. The 1st release switch is a switch that is turned on when the user half-presses the release button. When the 1st release switch is turned on, a shooting preparation operation such as AF processing is performed. The 2nd release switch is a switch that is turned on when the user fully presses the release button. When the 2nd release switch is turned on, an exposure operation for still image shooting is performed.
The moving image button is an operation member for instructing the start or end of moving image shooting. When the moving image button is pressed by the user, the moving image shooting process is started. Further, when the moving image button is pressed during the moving image shooting process, the moving image shooting process is ended.
The mode dial is an operation member for selecting shooting settings of the imaging apparatus. In the present embodiment, for example, a still image shooting mode and a moving image shooting mode can be selected as shooting settings of the imaging apparatus. The still image shooting mode is a shooting setting for shooting a still image. The moving image shooting mode is a shooting setting for shooting a moving image.
The selection key is an operation member for selecting or determining an item on the menu screen, for example. When the selection key is operated by the user, an item is selected or determined on the menu screen.
The power button is an operation member for turning on or off the power of the imaging apparatus. When the power button is operated by the user, the imaging apparatus 100 is activated and becomes operable. If the power button is operated while the imaging apparatus is activated, the imaging apparatus 100 enters a power saving standby state.

  The ROM 134 stores program codes for the CPU 1301 to execute various processes. In addition, the ROM 134 stores various control parameters such as control parameters necessary for the operation of the photographing optical system 102, the aperture 106, the image sensor 114, and the like, and control parameters necessary for image processing in the image processing circuit 1304. ing. Further, the ROM 134 also stores face part data used for face detection in the face detection circuit 1307, data for displaying a tracking frame, and the like. The ROM 134 also stores statistical data indicating the distance from the face position to the face peripheral position on the image data, which is used in the peripheral position estimation process described later.

Next, the operation of the imaging apparatus according to the present embodiment will be described. FIG. 5 is a flowchart showing the shooting operation of the imaging apparatus 100. The CPU 1301 reads the necessary program code from the ROM 134 and controls the operation of FIG.
In step S <b> 100, the CPU 1301 determines whether the current shooting setting of the imaging apparatus 100 is the still image shooting mode. As described above, the shooting setting is set by the mode dial.

  When it is determined in S100 that the shooting setting is the still image shooting mode, in S102, the CPU 1301 starts a live view operation. As a live view operation, the CPU 1301 controls the shutter drive mechanism 112 to release the shutter 110, and then the CPU 1301 controls the image sensor IF circuit 116 to start imaging by the image sensor 114. Thereafter, the CPU 1301 inputs the image data stored in the work area of the RAM 118 as a result of imaging by the imaging device 114 to the image processing circuit 1304 and performs image processing for live view display. Subsequently, the CPU 1301 inputs image data that has been subjected to image processing for live view display to the display element driving circuit 122 and causes the display element 120 to display an image. By repeatedly executing such a display operation, the subject image is displayed as a moving image. By this moving image display, the user can observe the subject.

In step S104, the CPU 1301 determines whether the first release switch is turned on. The CPU 1301 continues the live view operation until it is determined in S104 that the 1st release switch is turned on.
If it is determined in S104 that the first release switch is turned on, in S106, the CPU 1301 performs a release AF process. In the release AF, the focus lens is driven to the in-focus position by scan driving. In the scan drive, the CPU 1301 controls the focus adjustment mechanism 104 to evaluate the focus evaluation values sequentially calculated by the AF control circuit 1302 while driving the focus lens in one direction within a predetermined scan range. Then, the CPU 1301 stops the driving of the focus lens at the lens position where the contrast is maximized as a result of the evaluation of the focus evaluation value. Such scan driving is performed when the difference between the position of the focus lens before AF and the in-focus position is large.

  In step S108, the CPU 1301 controls the display element driving circuit 122 to display a tracking frame on the display element 120. Here, the tracking frame is displayed at the position of the tracking target on the screen of the display element 120. For example, a subject focused by release AF may be set as a tracking target, and a tracking frame may be displayed on the subject. When a face portion is detected by the face detection circuit 1307, a tracking frame is set on the face. You may make it display. Furthermore, when a subject displayed on the screen of the display element 120 is designated by the touch panel 124, a tracking frame may be displayed on the subject. Thus, in the present embodiment, the CPU 1301, the AF control circuit 1302, the face detection circuit 1307, the touch panel 124, and the like function as an example of the tracking target setting unit.

In S110, the CPU 1301 performs a tracking process. Details of this tracking process will be described later.
In step S112, the CPU 1301 performs AF processing so that the subject at the tracking position is focused, and performs AE processing so that exposure of the subject at the tracking position is appropriate.
In the AF processing after the tracking processing, the focus lens is driven to the in-focus position by scan driving or wobbling driving. In wobbling driving, the CPU 1301 determines whether or not the focus evaluation value calculated by the AF control circuit 1302 when the focus lens is driven has increased with respect to the focus evaluation value at the previous lens position. The CPU 1301 finely drives the focus lens in the same direction as the previous time when the focus evaluation value increases, and finely drives the focus lens in the opposite direction to the previous time when the focus evaluation value decreases. Such an operation is repeated at high speed to gradually drive the focus lens to the in-focus position.

  In the AE process, the CPU 1301 sets the aperture of the aperture 106 (aperture) during main exposure to set the luminance of the subject at the tracking position calculated by the AE control circuit 1303 to a predetermined appropriate amount (appropriate exposure amount). Value), the shutter 110 opening time (shutter speed value) is calculated.

  In step S114, the CPU 1301 determines whether the 2nd release switch is turned on. If it is determined in S114 that the 2nd release switch is not turned on, the CPU 1301 executes processing after the tracking processing in S110. In this way, in the still image shooting mode, the tracking process is continued until the 2nd release switch is turned on.

In S114, when it is determined that the 2nd release switch is turned on, in S116, the CPU 1301 controls the display element driving circuit 122 to hide the tracking frame.
In step S <b> 118, the CPU 1301 performs processing for recording still image data on the recording medium 128. At this time, the CPU 1301 controls the shutter driving mechanism 112 to close the shutter 110. Thereafter, the CPU 1301 controls the aperture driving mechanism 108 to narrow the aperture 106 to the previously calculated aperture value. Subsequently, the CPU 1301 controls the shutter driving mechanism 112 to perform imaging (exposure) by the imaging element 114 while opening the shutter 110 for the previously calculated opening time. Thereafter, the CPU 1301 processes still image data obtained via the image sensor 114 in the image processing circuit 1304. The CPU 1301 adds a header to the still image data processed by the image processing circuit 1304 to generate a still image file, and records the generated still image file on the recording medium 128.
In S120, the CPU 1301 adds data indicating the tracking position obtained as a result of the tracking processing in S110 to the still image file previously recorded on the recording medium 128. Thereafter, the CPU 1301 ends the operation shown in FIG.

If it is determined in S100 that the shooting setting is the moving image shooting mode, in S122, the CPU 1301 starts a live view operation.
In step S124, the CPU 1301 determines whether the moving image button has been turned on. Until it is determined in S124 that the moving image button has been turned on, the CPU 1301 continues the live view operation.

In S124, when it is determined that the moving image button is turned on, in S126, the CPU 1301 controls the display element driving circuit 122 to display the tracking frame on the display element 120.
In S128, the CPU 1301 performs a tracking process. Details of the tracking process will be described later.

  In step S130, the CPU 1301 performs AF processing so as to focus on the subject at the tracking position, and performs AE processing so that the exposure of the subject at the tracking position is appropriate. In the AF process in S130, the focus lens is driven to the in-focus position by wobbling driving.

  In step S <b> 132, the CPU 1301 performs processing for recording moving image data on the recording medium 128. At this time, the CPU 1301 controls the aperture driving mechanism 108 to narrow the aperture 106 to the aperture value calculated in the AE process. Subsequently, the CPU 1301 causes the image sensor 114 to perform imaging (exposure) for a time corresponding to the shutter speed value calculated in the AE process. After the exposure is completed, the CPU 1301 generates a moving image file and records it on the recording medium 128. Further, the CPU 1301 processes the moving image data obtained via the image sensor 114 in the image processing circuit 1304, and records the moving image data processed in the image processing circuit 1304 in a moving image file.

In S134, the CPU 1301 simultaneously records data indicating the tracking position obtained as a result of the tracking process in S128 together with the moving image file previously recorded on the recording medium 128.
In step S136, the CPU 1301 determines whether the moving image button has been turned off. If it is determined in S136 that the moving image button is not turned off, the CPU 1301 executes processing after the tracking processing in S128. Thus, in the moving image shooting mode, the tracking process and the recording of moving image data are continued until the moving image button is turned off.

If it is determined in S136 that the moving image button has been turned off, in S138, the CPU 1301 controls the display element driving circuit 122 to hide the tracking frame. Thereafter, the CPU 1301 ends the operation shown in FIG.
Next, tracking processing as a tracking method according to the present embodiment will be described with reference to FIG. As described above, in the present embodiment, tracking of a tracking target is performed using a plurality of tracking processes in combination.
In step S <b> 200, the CPU 1301 controls the imaging element IF circuit 116 to execute imaging by the imaging element 114. In step S <b> 202, the CPU 1301 takes in the image data obtained in the image sensor IF circuit 116 into the RAM 118 by imaging with the image sensor 114. Here, the image data obtained by the first tracking process is assumed to be evaluation image data. For this reason, the CPU 1301 captures the image data obtained in the first tracking process into the evaluation image area of the RAM 118. Further, the image data obtained by the second and subsequent tracking processing is referred to as reference image data. For this reason, the CPU 1301 captures the image data obtained in the second and subsequent tracking processing into the reference image area of the RAM 118.

  In step S <b> 204, the CPU 1301 causes the face detection circuit 1307 to perform tracking processing using face detection. Since the tracking process using face detection has been described above, a description thereof will be omitted. In the face detection process, the face part is detected from the evaluation image data in the first tracking process, and the face part is detected from the reference image data in the second and subsequent tracking processes.

In S206, the CPU 1301 determines the orientation of the face detected in S204 by the face orientation determination circuit 1308. If no face is detected in S204, the processing in S206 and the next S208 is omitted.
In step S <b> 208, the CPU 1301 estimates the position of the face periphery from the position of the face detected by the face detection circuit 1307. Here, the face peripheral portion is a portion (torso, limbs, etc.) other than the face portion of the subject to be tracked. In the following description, the position of the tracking target chest is estimated as an example of the peripheral portion of the face. Details of the peripheral position estimation process will be described later.

In step S <b> 210, the CPU 1301 performs tracking of the face part to be tracked and the face peripheral part using the tracking processing circuits 1305 and 1306. Details of each part tracking process in S210 will be described later.
In step S212, the CPU 1301 performs tracking position update determination processing. This tracking position update determination process is a process of determining which tracking position is adopted as the tracking position in the next frame among candidate tracking positions obtained by a plurality of tracking processes. Details of the tracking position update determination process in S212 will be described later.

  In S214, the CPU 1301 determines the reliability of the tracking position adopted as a result of the tracking position update determination process. When the tracking result of the tracking process using face detection is adopted, the reliability is determined from the face detection results of a plurality of past frames. Specifically, if a face is detected over a plurality of past frames, it is determined that there is reliability. Further, when the result of the tracking process using the luminance information is adopted, the reliability is determined from the contrast of the reference image data, for example. Specifically, when the sum of differences between adjacent pixels in the tracking target area in the reference image data is equal to or less than a predetermined value, it is determined that there is reliability. Further, when the result of the tracking process using color information is adopted, the reliability is determined from the saturation of the reference image data, for example. Specifically, if the saturation at the tracking position of the reference image data is greater than or equal to a predetermined value, it is determined that the reference image data is reliable. These threshold values for determining reliability can be set as appropriate.

  In S <b> 216, the CPU 1301 records the final tracking position in the tracking log area of the RAM 118. Although details will be described later, in the present embodiment, since both the face tracking process and the face peripheral tracking process are performed, both the face position and the face peripheral area can be obtained. One of these is set as a tracking position used for AF control or AE control. For example, as a result of the reliability determination in S214, the detection result of the face tracking position is valid, that is, either the reliability of the eyelid detection in the eyelid detection circuit 1307 or the reliability of the eyelid tracking in each part tracking process. In some cases, the CPU 1301 employs the tracking position of the face. In addition, as a result of the reliability determination in S214, the detection result of the face tracking position is invalid, that is, either the reliability of the eyelid detection in the eyelid detection circuit 1307 or the reliability of the eyelid tracking in each part tracking process. In this case, the CPU 1301 adopts the tracking position of the face peripheral part. However, the RAM 118 stores the tracking positions of both the face part and the face peripheral part. If it is determined that neither of the tracking positions is reliable, the tracking position may not be recorded.

In S218, the CPU 1301 controls the display element driving circuit 122 to update the display position of the tracking frame to a position corresponding to the tracking position stored in S218. Thereafter, the CPU 1301 ends the tracking process of FIG.
Next, the peripheral position estimation process will be described with reference to FIG. In the peripheral position estimation process, the position of the tracking target face peripheral part (chest part) is estimated from the face part position detected in the face detection process.
In step S300, the CPU 1301 acquires information on the position of the face detected by the face detection circuit 1307 (for example, the position of the center of gravity of the face area). In step S <b> 302, the CPU 1301 acquires information on the size of the face (for example, the size of the face area) detected by the face detection circuit 1307. In step S <b> 304, the CPU 1301 acquires statistical data indicating the distance from the position of the face part to the position of the face peripheral part (chest part) from the ROM 134.

  In step S306, the CPU 1301 calculates the position of the peripheral portion of the face on the image data from the acquired statistical data. Specifically, the position of the face is used as a reference, and the position below the reference position by the distance obtained according to the statistical data is set as the position of the face periphery. Here, the statistical data stored in the ROM 134 is statistical data from the position of the face portion measured at a certain fixed distance to the position of the face peripheral portion. Therefore, when the distance from the imaging device to the tracking target is different from the distance when the statistical data is obtained, the position of the face peripheral portion is corrected according to the distance difference. The distance from the imaging device to the tracking target is calculated from, for example, the size of the face (ratio of the size of the face on the screen). Then, the value of the statistical data is corrected according to this distance, and the position of the peripheral portion of the face on the actual image data is calculated. After calculating the position of the face periphery, the CPU 1301 ends the process of FIG.

Here, in the example of FIG. 7, the position below the position of the face portion is set as the position of the face peripheral portion in order to simplify the description. In this case, a correct result is obtained when the person to be tracked is standing upright, but a correct result cannot be obtained, for example, when the person to be tracked is lying down. On the other hand, by detecting each organ of the face (eyes, nose, mouth, etc.) and determining in which direction each organ is aligned with respect to the screen, a person as a tracking target stands upright. It may be determined whether the person is sleeping or sleeping, and the position of the face periphery may be calculated according to the determination result. For example, when the eyes, nose, and mouth are lined up from the left to the right of the screen, it is estimated that there is a face peripheral portion at a position in the right direction with respect to the position of the face portion.
In the example of FIG. 7, the position of the heel periphery is estimated from the position of the heel detected by the heel detection circuit 1307. On the other hand, the position of the periphery of the heel may be estimated from the tracking position of the heel obtained in each part tracking process described later.

Next, each part tracking process will be described with reference to FIG. In each part tracking process, a tracking target face part and a face peripheral part are tracked in a tracking process other than the face detection process.
In step S <b> 400, the CPU 1301 causes the tracking processing circuit 1306 to perform face tracking processing using color information as the first feature information. More specifically, the tracking processing circuit 1306 tracks the tracking color area having the same color information as the face color information set in the evaluation image data in the reference image data. Then, the CPU 1301 stores the tracking position of the face obtained as a result of the tracking process using the color information in the tracking log area of the RAM 118. In addition, the CPU 1301 stores the tracking color area of the face obtained as a result of the tracking process using the color information in the tracking color area log area of the RAM 118.

Here, in the first part tracking process, since only the evaluation image data has been acquired, the processes after S400 are omitted. In the following description, the description will be continued on the assumption that both the evaluation image data and the reference image data have been acquired.
In step S <b> 402, the CPU 1301 causes the tracking processing circuit 1305 to perform face tracking processing using luminance information as the second feature information. Specifically, the tracking processing circuit 1305 tracks an area having the same pattern as the face set in the evaluation image data in the reference image data. Then, the CPU 1301 stores the tracking position of the face obtained as a result of the tracking process using the luminance information in the tracking log area of the RAM 118.

In step S <b> 404, the CPU 1301 sets one of the tracking result of the tracking processing circuit 1305 and the tracking result of the tracking processing circuit 1306 as the final face tracking position (face candidate tracking position) in each part tracking process. The face candidate tracking position calculation process will be described later.
In step S <b> 406, the CPU 1301 uses the tracking processing circuit 1306 to perform tracking processing for the peripheral portion of the face using color information. Specifically, the tracking processing circuit 1306 uses a tracking color area having the same color information as the color information (for example, the color information of clothes) of the face periphery (for example, the chest) set in the evaluation image data in the reference image data. To track. Then, the CPU 1301 stores the tracking position of the peripheral portion of the face obtained as a result of the tracking process using the color information in the tracking log area of the RAM 118. In addition, the CPU 1301 stores the tracking color area in the peripheral portion of the face obtained as a result of the tracking process using the color information in the tracking color area log area of the RAM 118.

  In step S <b> 408, the CPU 1301 uses the tracking processing circuit 1305 to perform tracking processing of the face peripheral portion using luminance information. Specifically, the tracking processing circuit 1305 tracks an area having the same pattern as the face periphery set in the evaluation image data in the reference image data. Then, the CPU 1301 stores the tracking position of the peripheral portion of the face obtained as a result of the tracking process using the luminance information in the tracking log area of the RAM 118.

  In S410, the CPU 1301 uses either the tracking result of the tracking processing circuit 1305 or the tracking result of the tracking processing circuit 1306 as the final tracking position (peripheral part candidate tracking position) of the face periphery in each part tracking process. This peripheral part candidate tracking position calculation process will be described later. After the peripheral part candidate tracking position calculation process, the CPU 1301 ends the process of FIG.

  Next, the face candidate tracking position calculation process and the peripheral part candidate tracking position calculation process will be described with reference to FIG. Here, the face candidate tracking position calculation process and the peripheral part candidate tracking position calculation process differ only in whether the tracking position used for each process is a face part or a face peripheral part. Therefore, in FIG. 9, these processes will be described together. In the processing of FIG. 9, the color information at the tracking position obtained from the color information and the color information at the tracking position obtained from the luminance information are compared, and the tracking position and luminance information obtained from the color information are obtained. The final candidate tracking position is determined according to the positional relationship with the tracking position.

In step S500, the CPU 1301 uses the color information acquisition circuit 1309 to acquire the color information of the tracking position obtained from the luminance information in the reference image data. Actually, the color information of the minute region including the tracking position is acquired.
In step S502, the CPU 1301 determines whether the color information of the tracking position (color information of the tracking color area) obtained from the color information matches the color information of the tracking position obtained from the luminance information. Here, “match” means that the color information of the tracking position obtained from the luminance information is within an arbitrary range set in advance based on the color information of the tracking position obtained from the color information. In S502, when the color information of the tracking position obtained from the color information matches the color information of the tracking position obtained from the luminance information, in S504, the CPU 1301 determines that the tracking color area is obtained from the luminance information. Determine if the location is included.

  If it is determined in S504 that the tracking color area does not include the tracking position obtained by the luminance information, in S506, the CPU 1301 employs the tracking position obtained by the luminance information as the final candidate tracking position. . Thereafter, the CPU 1301 ends the process of FIG.

  If the color information of the tracking position obtained from the color information does not match the color information of the tracking position obtained from the luminance information in S502, or the tracking position where the tracking color area is obtained from the luminance information in 5304 In step S508, the CPU 1301 employs the tracking position obtained from the color information as the final candidate tracking position. Thereafter, the CPU 1301 ends the process of FIG.

  By determining the final candidate tracking position as shown in FIG. 9, it is possible to track a tracking target with intense movement by tracking processing using color information, and in a scene including a plurality of same-color subjects. Can be tracked by tracking processing using luminance information. As a result, even if face detection fails, it is possible to continue tracking the tracking target.

Here, in each part tracking process of FIG. 8, the tracking process is performed by combining the luminance information and the color information. However, other tracking processes such as a tracking process based on feature amount detection may be combined.
Next, tracking position update determination processing will be described with reference to FIG. In the tracking position update determination process, the final tracking position is determined according to the orientation of the face to be tracked.

In step S <b> 600, the CPU 1301 determines whether a face is detected by the face detection circuit 1307.
If it is determined in S600 that the face is detected by the face detection circuit 1307, in S602, the CPU 1301 determines the orientation of the face using the face orientation determination circuit 1308. Then, the CPU 1301 determines whether the face orientation is the front orientation as a result of the face orientation determination by the face orientation determination circuit 1308.

  If it is determined in S602 that the face is facing the front, in S604, the CPU 1301 determines the position of the face detected by the face detection circuit 1307 (for example, the barycentric position of the region 408) as the final face. This is the tracking position. In the next frame, the CPU 1301 uses the final face tracking position as the tracking processing start position, and performs face detection processing, face tracking processing using color information, and face information using luminance information. The tracking process is performed. In step S606, the CPU 1301 sets the position of the face periphery calculated in the periphery position estimation process as the final tracking position of the face periphery. In the next frame, the CPU 1301 uses the final tracking position of the face periphery as a tracking process start position, and performs tracking processing of the face periphery using color information and tracking processing of the face periphery using luminance information. I do. In step S <b> 608, the CPU 1301 uses the color information acquisition circuit 1309 to acquire the color information of the position of the face periphery calculated in the periphery position estimation process. Then, the CPU 1301 updates the color information to be tracked in the tracking process of the face peripheral part using the color information in the next frame to the newly acquired color information of the position of the face peripheral part. Thereafter, the CPU 1301 ends the process shown in FIG.

  Examples of scenes corresponding to S604 to S608 are shown in FIGS. 11 (a) and 11 (b). When the face portion is detected, as shown in FIG. 11A, a region A1 including the face portion is obtained, and the position of the center of gravity of the region A1 becomes the position of the face portion. Further, the position B1 below the face by a predetermined distance is the position of the face periphery. On the other hand, each part tracking process is performed separately from the face detection process. For example, in FIG. 11B, the position C1 is the candidate tracking position of the face portion, and the position D1 is the candidate tracking position of the face peripheral portion.

Here, as shown in FIG. 11B, when the face portion is facing the front, it can be considered that both the detection accuracy of the face portion and the estimation accuracy of the peripheral portion of the face are high accuracy. . Therefore, in the present embodiment, when the tracking result as shown in FIG. 11B is obtained, the start position of the face tracking process of the tracking processing circuits 1305 and 1306 in the next frame is set to the position C1. Instead, the face position A1 detected by the face detection circuit 1307 is used. In this way, it is easy to track the same face portion in the next frame. Similarly, the start position of the tracking process of the face peripheral part of the tracking processing circuits 1305 and 1306 in the next frame is set to the position B1 of the face peripheral part estimated in the peripheral position estimation process instead of the position D1. In this way, it is easy to track the same face periphery in the next frame.
A person often wears clothes, and various patterns are often formed on the clothes. Therefore, in particular, in the tracking process using color information, it is desirable to perform the tracking process according to the color information of the position of the latest face peripheral part. Therefore, in S608, the tracking target color information is updated to the color information of the position B1.

  If it is determined in S602 that the face is not facing the front, in S610, the CPU 1301 sets the face position detected by the face detection circuit 1307 as the final face tracking position. In step S <b> 612, the CPU 1301 sets the candidate peripheral tracking position of the face periphery obtained in each part tracking process as the final tracking position of the face periphery. In step S614, the CPU 1301 does not update the color information to be tracked in the tracking process using the color information, and uses the past color information as it is. Thereafter, the CPU 1301 ends the process shown in FIG.

  An example of a scene corresponding to S610 to S614 is shown in FIG. When the face is detected, as shown in FIG. 11C, a region A2 including the face is obtained, and the position of the center of gravity of the region A2 becomes the position of the face. Further, a position B2 below the face by a predetermined distance is the position of the face periphery. On the other hand, each part tracking process is performed separately from the face detection process. For example, in FIG. 11C, the position C1 is the candidate tracking position of the face part, and the position D1 is the candidate tracking position of the face peripheral part.

Here, as shown in FIG. 11C, when the face can be detected and is not facing the front, in other words, when the face is sideways, a certain degree of detection accuracy is guaranteed for the face. The estimation accuracy is not guaranteed for the face periphery. This is because the peripheral part of the face is simply estimated as a position at a predetermined distance from the face part. For this reason, in this embodiment, when the tracking result as shown in FIG. 11C is obtained, the start position of the face tracking processing of the tracking processing circuits 1305 and 1306 in the next frame is set to the position C1. Instead, the face position A2 detected by the face detection circuit 1307 is used. On the other hand, the start position of the tracking process of the face peripheral part of the tracking processing circuits 1305 and 1306 in the next frame is set as the position D1 of the face peripheral part obtained by the current part tracking process. By doing so, it is possible to accurately track the peripheral portion of the face even in the next frame.
Note that when the face portion is in landscape orientation, the accuracy of estimation of the peripheral portion of the face is not guaranteed, and the tracking target color information is not updated.

  If it is determined in S600 that the face detection circuit 1307 has not detected a face, the CPU 1301 determines the face tracking candidate tracking position obtained in each part tracking process as the final tracking position in S616. And In step S <b> 618, the CPU 1301 sets the candidate peripheral tracking position of the face periphery obtained in each part tracking process as the final tracking position of the face periphery. In step S620, the CPU 1301 does not update the color information to be tracked in the tracking process using the color information, and uses the past color information as it is. Thereafter, the CPU 1301 ends the process shown in FIG.

  An example of a scene corresponding to S616 to S620 is shown in FIG. Since no face is detected, the result of the tracking process can be obtained only by the tracking process of each part. For example, in FIG. 11D, the position C2 is the face candidate tracking position, and the position D2 is the face periphery candidate tracking position.

As shown in FIG. 11D, when the face part cannot be detected because the face part is facing back or the face part is moved greatly, the face parts of the tracking processing circuits 1305 and 1306 in the next frame are detected. The start position of the tracking process is the face position C2 obtained by the respective part tracking processes. In addition, the start position of the tracking process of the face peripheral part of the tracking processing circuits 1305 and 1306 in the next frame is set as the position D2 of the face peripheral part obtained by the current part tracking process. By doing so, it is possible to accurately track the face portion and the face peripheral portion in the next frame.
Note that the tracking target color information is not updated even when the face portion cannot be detected.

Next, display of the tracking frame will be described. As described above in the description of S216, in the present embodiment, both the face tracking process and the face peripheral tracking process are performed, so that both the face position and the face peripheral area are obtained. In the present embodiment, any one of these tracking positions is set as a tracking position used for AF control or AE control. In the display of the tracking frame, the finally adopted tracking frame is displayed. For example, when the tracking position of the buttock is adopted, the tracking frame E1 is displayed at the position of the buttock to be tracked as shown in FIG. Further, when the tracking position in the vicinity of the eyelid is adopted, the tracking frame E2 is displayed at the position of the periphery of the eyelid to be tracked as shown in FIG.
By displaying the tracking frame as shown in FIG. 12A and FIG. 12B, the user can easily visually recognize the part that is currently subject to AF control or AE control.

  Regardless of whether the tracking position of the buttocks or the tracking position of the periphery of the heel is adopted, the tracking including both the heel part to be tracked and the periphery of the heel as shown in FIG. The frame E3 may be displayed. By displaying the tracking frame in this way, the user can easily identify the subject that is the current tracking target.

Further, the buttocks area and the tracking candidate position shown in FIG. 11 may be displayed as they are. In this case, for example, as shown in FIG. 11B and FIG. 11C, the size of the tracking frame indicating the buttock area obtained by the heel detection circuit 1307 is set by the tracking processing circuits 1305 and 1306. The size of the tracking frame indicating the obtained tracking position is made larger. Further, the size of the tracking frame indicating the tracking position obtained by the tracking processing circuit 1305 may be different from the size of the tracking frame indicating the tracking position obtained by the tracking processing circuit 1305. Thus, it is possible to distinguish a tracking frame by varying a magnitude | size.
Furthermore, the display color of the tracking frame indicating the hip region obtained by the eyelid detection circuit 1307, the display color of the tracking frame indicating the tracking position obtained by the tracking processing circuit 1305, and the tracking processing circuit 1305 The display color of the tracking frame indicating the tracking position may be different. The tracking frame can also be distinguished by making the colors different in this way.

  As described above, in the present embodiment, by performing the tracking process by combining the eyelid detection circuit 1307 and the tracking processing circuits 1305 and 1306, it is possible to continue tracking a specific tracking target desired by the user. is there. Further, since the display position of the tracking frame is switched between the heel part and the heel peripheral part according to the result of the tracking process, the user can easily recognize that the appropriate tracking process is being performed. .

  Here, in the above-described embodiment, a tracking process using luminance information and a tracking process using color information are performed in each part tracking process. This is to further improve the performance of the tracking process. On the other hand, although the performance of the tracking process is degraded, only one of the tracking process using luminance information and the tracking process using color information may be performed in each part tracking process. Further, since the heel tracking is performed by the heel detection circuit 1307, the tracking of the heel using the tracking processing circuits 1305 and 1306 may be omitted.

  Although the present invention has been described above based on the embodiments, the present invention is not limited to the above-described embodiments, and various modifications and applications are naturally possible within the scope of the gist of the present invention. Further, in the description of the operation described above, the operation is described using “first”, “next”, and the like for convenience. However, it does not mean that the operation is essential in this order.

  Further, the above-described embodiments include various stages of the invention, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some configuration requirements are deleted from all the configuration requirements shown in the embodiment, the above-described problem can be solved, and this configuration requirement is deleted when the above-described effects can be obtained. The configuration can also be extracted as an invention.

  DESCRIPTION OF SYMBOLS 100 ... Imaging device, 102 ... Imaging optical system, 104 ... Focus adjustment mechanism, 106 ... Aperture, 108 ... Aperture drive mechanism, 110 ... Shutter, 112 ... Shutter drive mechanism, 114 ... Imaging element, 116 ... Imaging element interface (IF) Circuit: 118 ... RAM, 120 ... Display element, 122 ... Display element drive circuit, 124 ... Touch panel, 126 ... Touch panel drive circuit, 128 ... Recording medium, 130 ... System controller, 132 ... Operation unit, 134 ... ROM, 1301 ... CPU 1302: AF control circuit, 1303 ... AE control circuit, 1304 ... Image processing circuit, 1305, 1306 ... Tracking processing circuit, 1307 ... Face detection circuit, 1308 ... Face orientation determination circuit, 1309 ... Color information acquisition circuit, 1310 ... Memory Control circuit

Claims (17)

An imaging unit for imaging a subject and outputting image data;
A face detector for detecting the face of the subject from the image data;
A first subject tracking unit that tracks the face of the subject in the image data based on the image data;
Based on the image data, a second subject tracking unit that tracks the periphery of the face of the subject in the image data;
A display unit for displaying an image based on the image data;
While detection of the face part by the face detection part or tracking of the face part by the first subject tracking part is effective, a display frame indicating the position of the face part on the image displayed on the display part When the detection of the face by the face detection unit and the tracking of the face by the first subject tracking unit are disabled, the peripheral portion of the face is displayed on the image displayed on the display unit. A control unit for superimposing a display frame indicating the position of
An imaging apparatus comprising:   The control unit detects the face by the face detection unit and detects the first from the state in which the detection of the face by the face detection unit or the tracking of the face by the first subject tracking unit is effective. The superposition of a display frame indicating the position of the face portion superimposed on the display portion is stopped when the tracking of the face portion by the subject tracking portion changes to an invalid state. The imaging device described in 1.   When the detection of the face by the face detection unit and the tracking of the face by the first subject tracking unit are valid, the control unit adds the face to the image displayed on the display unit. The imaging apparatus according to claim 2, wherein a display frame indicating the position of the face detected by the detection unit is superimposed.   When the detection of the face by the face detection unit and the tracking of the face by the first subject tracking unit are valid, the control unit adds the face to the image displayed on the display unit. The display frame indicating the position of the face detected by the detection unit and the display frame indicating the position of the face tracked by the first subject tracking unit are overlapped with each other. Imaging device.   When the face detection by the face detection unit is invalid and the tracking operation of the face by the first subject tracking unit is valid, the control unit displays the image displayed on the display unit. The imaging apparatus according to claim 2, wherein a display frame indicating the position of the face tracked by the first subject tracking section is superimposed on the image capturing apparatus.   The second subject tracking unit determines the position of the chest of the subject based on the position of the face detected by the face detection unit or the position of the face tracked by the first subject tracking unit. The imaging apparatus according to claim 1, wherein the tracking is performed by estimating the position of a peripheral portion. The first subject tracking section is
A first color tracking unit that detects color information of the image data and tracks the face of the subject;
A first luminance tracking unit that detects luminance information of the image data and tracks the face of the subject;
Have
The second subject tracking unit includes:
A second color tracking unit that detects color information of the image data and tracks a peripheral part of the face of the subject;
A second luminance tracking unit that detects luminance information of the image data and tracks the periphery of the face of the subject;
The imaging apparatus according to claim 1, wherein the imaging apparatus includes:   The control unit according to any one of claims 1 to 7, wherein a size of a display frame indicating the position of the face is different from a size of a display frame indicating the position of the periphery of the face. The imaging device described.   The said control part makes the color of the display frame which shows the position of the said face, and the color of the display frame which shows the position of the said periphery part of a face different from any one of Claim 1 thru | or 8 characterized by the above-mentioned. The imaging device described.   The imaging apparatus according to claim 1, wherein the control unit superimposes a display frame including the face part and the face peripheral part on an image displayed on the display unit. An imaging unit for imaging a subject and outputting image data;
A face detector for detecting the face of the subject from the image data;
Based on the image data, a face periphery tracking unit that tracks the face periphery of the subject in the image data;
A display unit for displaying an image based on the image data;
While the detection of the face by the face detection unit is effective, a display frame indicating the position of the face is superimposed on the image displayed on the display unit, and the detection of the face by the face detection unit is performed. A control unit that superimposes a display frame indicating the position of the peripheral portion of the face on the image displayed on the display unit when
An imaging apparatus comprising:   The control unit is superimposed on the display unit when the detection of the face by the face detection unit is changed to a state in which the detection of the face by the face detection unit is invalid. The imaging apparatus according to claim 11, wherein superimposition of a display frame indicating the position of the face is stopped.   The face periphery tracking unit estimates the position of the chest of the subject as the position of the face periphery based on the position of the face detected by the face detection unit, and performs the tracking. Item 13. The imaging device according to Item 11 or 12. The face periphery tracking unit detects color information of the image data and tracks the face of the subject;
A luminance tracking unit that detects luminance information of the image data and tracks the face of the subject;
The imaging apparatus according to claim 11, further comprising:   The control unit according to any one of claims 11 to 14, wherein a size of a display frame indicating the position of the face is different from a size of a display frame indicating the position of the peripheral portion of the face. The imaging device described.   The said control part makes the color of the display frame which shows the position of the said face, and the color of the display frame which shows the position of the said periphery part of a face different from any one of Claim 11 thru | or 15 characterized by the above-mentioned. The imaging device described.   The imaging apparatus according to claim 11, wherein the control unit superimposes a display frame including the face part and the face peripheral part on an image displayed on the display part.

Images