JP2006208558A - Imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the focusing operation time when the current shooting condition is almost the same as the previous shooting condition, so that the focusing operation time is shortened, but the time for performing the scanning again is still necessary. ing. For example, in continuous shooting mode in which continuous shooting is performed, shooting is performed with the focus lens position of the first continuous shooting, or shooting with a continuous shooting interval longer than the focusing operation time.
SOLUTION: Face detection means is provided, and when the same scene is determined by face detection information in previous shooting and current shooting, the focus lens position is controlled to the previous shooting position and the distance measuring operation is omitted. , To shorten the release time lag.
[Selection] Figure 7

Description

  The present invention relates to automatic focus adjustment using a face detection technique of an imaging apparatus.

  In digital cameras and digital video cameras, a contrast detection method (also called a scan method or a hill-climbing method) is used to adjust the focus with the lens position where the high-frequency component of the luminance signal of the subject image formed on the image sensor is maximized. However, it is becoming difficult to achieve both AF accuracy and speed with the increase in the magnification and the number of pixels.

Therefore, it is determined whether the conditions of the current shooting and the previous shooting are the same, and if it is determined that they are the same, the narrow range including the focus lens position at the time of the previous shooting is scanned. There is a method for shortening the focusing operation time. This method is particularly effective when a subject at the same distance is repeatedly photographed or when a focusing operation is performed while tracking the subject.
JP 2004-102135 A

  In the invention in the patent document shown above, when the current shooting condition is the same as the previous shooting condition, the scan range is set to be narrow, so the focusing operation time is shortened. Time to do is essential. Therefore, for example, in the continuous shooting mode in which continuous shooting is performed, shooting is performed with the focus lens position of the first continuous shooting, or shooting with a continuous shooting interval longer than the focusing operation time.

  Also, if there is no change in the subject distance in the same scene as the previous shooting, there is no problem with focus even if the focusing operation by scanning is omitted with the focus lens position at the previous shooting, but the brightness and shooting Since it is difficult to determine whether the omission of scanning is possible only with various imaging conditions such as the imaging mode setting, it is not sufficient for speeding up by omitting the scanning operation.

  Furthermore, when different shooting targets exist in the same focus adjustment area as the previous shooting, the main subject may be determined as the same main subject even though the main shooting is different from the previous shooting. In order to increase the accuracy of the determination of the shooting conditions, it is inevitably required to make the conditions for changes in the shooting conditions stricter. However, if even a slight change in the field of view determines that the shooting conditions have changed, it is difficult to achieve the effect of shortening the focusing operation time, which is not practical.

  Therefore, in the present invention, face detection is performed on the captured image, and it is determined whether the scene is the same by the face detection information in the previous shooting and the current shooting. Is automatically controlled to reduce the release time lag by controlling the camera to the previous shooting position, omitting the scan operation, and performing the scan operation when the scene changes and the scan is necessary. An object of the present invention is to provide a focusing device.

In order to solve the above problems and achieve the object, an image pickup apparatus of the present invention is photographed by a distance measuring means for measuring a distance to a subject, a storage means for storing shooting conditions at the time of distance measurement by the distance measuring means, A face detection information extraction means for extracting face detection information from the image signal, and a determination means for making a determination based on the face detection information and the shooting conditions,
According to the determination result of the determination unit, the imaging is switched between an operation of shooting using the shooting conditions at the previous shooting as it is and an operation of performing a new distance measurement operation and resetting the shooting conditions to perform shooting. apparatus.

  In order to solve the above-described problems and achieve the object, the control method of the imaging apparatus according to the present invention stores a distance measurement process for measuring a distance to a subject and a shooting condition at the time of distance measurement by the distance measurement process. A step, a face detection information extraction step for extracting face detection information from the captured image signal, and a determination step for making a determination based on the face detection information and the shooting condition, and depending on a determination result of the determination step A method for controlling an image pickup apparatus, wherein an operation for photographing using the photographing conditions at the time of the previous photographing as they are and a behavior for performing a new distance measuring operation and resetting the photographing conditions to perform photographing are switched.

  According to the electronic camera of the present invention, it is determined whether the scene is the same based on the face detection information and the shooting conditions. If the scene is determined to be the same, the focus lens position is controlled to the previous shooting position and the distance measuring operation is omitted. However, when the scene changes and distance measurement is necessary, the distance measurement operation is performed, so that unnecessary distance measurement operation can be omitted and the release time lag can be shortened.

  Especially during continuous shooting, the continuous shooting interval can be shortened in the case of the same scene where the focus lens position of the first continuous shooting can be kept, and when the scene changes, the distance measurement operation is performed and the in-focus state is maintained. It becomes possible.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
FIG. 1 is a block diagram showing the configuration of an electronic camera to which an embodiment of the present invention is applied.

  101 is a photographing lens including a zoom mechanism, 102 is an aperture and shutter for controlling the amount of light, 103 is an AE processing unit that performs automatic exposure (hereinafter referred to as AE) processing, and 104 is a focus for focusing on an image sensor described later. It is a lens. Reference numeral 105 denotes an AF processing unit that performs automatic focusing (hereinafter referred to as AF) processing, 106 denotes a strobe, and 107 denotes an EF processing unit that performs light emission control of the strobe 106. Reference numeral 108 denotes an image sensor such as a CCD or CMOS having a photoelectric conversion function, 109 denotes an A / D converter including a CDS circuit for removing output noise of the image sensor 108 and a non-linear amplifier circuit before A / D conversion, and 110 denotes an A An image processing unit that performs various image processing on the image data digitally converted by the / D conversion unit, 111 a WB processing unit that performs white balance (hereinafter referred to as WB) processing on the image data, and 112 an image processing A format conversion unit that converts the format of the image processed by the unit 110, 113 a DRAM that is a high-speed built-in memory such as a random access memory, and 114 an image recording unit that includes a recording medium such as a memory card and its interface It is. Reference numeral 115 denotes a system control unit for controlling the system such as a shooting sequence, 116 denotes a video RAM (hereinafter referred to as VRAM) which is an image display memory, and 117 denotes an image display as well as a display for assisting operations and a display of a camera state. , An operation display unit for displaying a shooting screen and a distance measurement area at the time of shooting, 118 an operation unit for operating the camera from the outside, 119 an attitude detection sensor as a detecting means for detecting the attitude of the camera, and 120 a power source for the system , A shooting mode switch 121 for setting a shooting mode such as program, landscape, portrait, pan focus, etc., and a switch 122 for performing a shooting standby operation such as AF or AE (hereinafter referred to as SW1). ), 123 is a photographing switch for photographing after the operation of SW1 (hereinafter referred to as SW2). 124 drive mode switch for setting such single shooting and continuous shooting, self timer, 125 is a face detection processing unit.

  The DRAM 113 is used as a high-speed buffer memory as a temporary image storage means, or a working memory for image compression / decompression. The operation unit 118 includes, for example, a menu switch for performing various settings such as a shooting function of the imaging apparatus and settings for image playback, a zoom lever for instructing a zoom operation of the shooting lens, an operation mode switching switch between a shooting mode and a playback mode, and the like. I mean.

  Next, the operation of the imaging apparatus according to the present embodiment will be described with reference to the flowchart shown in FIG.

  In step S201, the zoom operation is performed according to the flowchart of FIG.

  In step S202, the state of the switch SW1 is checked. If “ON”, the process proceeds to step S203. If not, the process returns to step S201 and the process is repeated until the SW1 is “ON”. Here, the function of SW1 is to perform a shooting standby operation such as AF or AE.

  In step S203, face detection processing is performed according to the flowchart of FIG.

  In step S204, the AE processing unit 103 performs AE processing so that the exposure of the face area is optimized based on the result of the face detection processing obtained in step S203.

  In step S205, an AF operation is performed according to the flowchart of FIG.

  In step S206, the state of SW2 is checked. If “ON”, the process returns to step S207, and if not, the process returns to step S203, and the process is repeated until SW2 is “ON”. Here, the function of SW2 is to perform photographing after operation of SW1.

  In step S207, a photographing operation is performed according to the flowchart of FIG.

  Next, the processing flow of the zoom operation in step S201 in the flowchart of FIG. 2 will be described with reference to the flowchart of FIG.

  In step S301, it is checked whether a zoom lever (not shown) in FIG. 1 is operated to the tele (telephoto) side. If “ON”, the process proceeds to step S302; otherwise, the process proceeds to step S306.

  In step S302, it is checked whether the zoom position is at the tele end. If it is not the tele end, the process proceeds to step S303, and if it is the tele end, the process proceeds to step S304.

  In step S303, the zoom position is moved to the Tele side, and the process returns to step S301.

  In step S304, it is checked whether the electronic zoom is used and the maximum magnification is reached. If it is the maximum magnification, the process proceeds to step S306, and if not, the process proceeds to step S305.

  In step S305, the electronic zoom magnification is increased and the process returns to step S301.

  In step S306, it is checked whether a zoom lever (not shown in FIG. 1) is operated to the Wide (wide angle) side. If “ON”, the process proceeds to step S307; otherwise, the process proceeds to step S311.

  In step S307, it is checked whether the zoom position is at the wide end. If it is not the wide end, the process proceeds to step S308, and if it is the wide end, the process proceeds to step S311.

  In step S308, it is checked whether the electronic zoom is used. If so, the process proceeds to step S309, and if not, the process proceeds to step S310.

  In step S309, the magnification of the electronic zoom is lowered and the process returns to step S306.

  In step S310, the zoom position is moved to the Wide side, and the process returns to step S306.

  In step S311, the current zoom position is stored in the DRAM 113.

  Next, face detection processing performed by the face detection processing unit 125 will be described. The face detection processing method is generally a method using learning typified by a neural network, a method using template matching from a region having a physical shape characteristic such as eyes and nose, skin color and eyes. Many methods have been proposed, such as methods that detect image features such as the shape of images and use statistical analysis. Generally, it is common to recognize multiple faces by combining them. is there.

  Here, a method of performing face detection by performing template matching from color information and a characteristic shape of eyes will be described.

  FIG. 4 is a flowchart for the face detection process in step S203 in the flowchart of FIG. 2, and FIG. 5 is a diagram for explaining an example of the face detection method and the eye shape determination method in this embodiment.

  The operation will now be described with reference to the flowchart of FIG.

  In step S401, the skin color region is extracted from the target image data. FIG. 6 is a chromaticity diagram showing representative colors in the CIELAB Lab color space, and an area indicated by an ellipse is likely to be a skin color. FIG. 5A is a diagram illustrating image data to be subjected to face detection, and FIG. 5B is a diagram illustrating a region in which the chromaticity of the skin color region is extracted from the image illustrated in FIG. 5A.

  In step S402, a high-pass filter is applied to the target image. FIG. 5c is an image obtained by applying a high-pass filter to the image shown in FIG. 5b.

  In step S403, eye detection in the image obtained in step S402 is performed based on a template image of an eye prepared in advance using template matching. FIG. 5d is an explanatory diagram of template matching.

  In step S404, face recognition is performed from the positional relationship of the eye regions detected in step S403, and feature quantities such as the number, position, and size of faces are extracted.

  In step S405, face detection information such as the number, position, and size of the face obtained in step S404 is stored in the DRAM 113.

  In the description of the face detection processing method according to the flowchart of FIG. 4 and the explanatory diagram of FIG. 5 described above, the face detection area for performing the face detection processing is the entire shooting screen, but the range of the distance measurement area set in the shooting screen The size and position of the face detection area may be changed according to the shooting mode at the time of shooting and the posture of the camera.

  For example, when shooting portrait in portrait mode, the main subject's face is often located above the center of the shooting screen, such as when shooting a person in portrait mode, and in pan focus mode. When taking a group photo, the number, position, and size of the faces to be captured are completely different, so it is effective to change the face detection area depending on the shooting mode and the camera posture.

  In face detection processing by template matching or the like, the detection area is longer and the release time lag is longer as the face detection area is wider, so it is desirable that the detection area be as narrow as possible. Therefore, the face detection time can be shortened by appropriately setting the size and position of the face detection area according to the distance measurement mode, the shooting mode, and the posture of the camera.

  Next, the AF operation in step S205 in the flowchart of FIG. 2 will be described based on the flowchart of FIG. The AF operation is performed by detecting a peak of a high frequency component (hereinafter referred to as a focus evaluation value) of a signal obtained from the image sensor.

  In step S <b> 601, face detection results and shooting condition information in the previous shooting and the current shooting are acquired from the DRAM 113. The photographing condition information here includes at least zoom position information.

  In step S602, based on the face detection result acquired in step S601 and the shooting condition information, it is determined whether the scene is the same as the previous shooting. Whether the scene is the same as the previous shooting performed here is determined based on the number of faces, information about the position of the face, information including at least one of the information about the size of the face, and the zoom position. Determined by information.

  If it is judged that the scene is the same as that at the previous shooting and the distance measurement result at the previous shooting is in focus, the process proceeds to step S603; otherwise, the process proceeds to step S604.

  In step S603, the focus lens 104 is set to the same position as in the previous shooting. Then, after the setting, the AF operation processing step ends.

  In step S604, settings relating to driving of the focus lens 104 are performed. Specifically, the scan start position, scan end position, scan direction, lens movement amount during sampling, and the like are set.

  In step S605, the focus lens 104 is moved to the scan start position set in step S604.

  In step S606, the focus evaluation value and the position of the focus lens 104 are stored in the DRAM 113. Note that the position of the focus lens 104 is detected as a relative position from the reset position by a photo interrupter (not shown) when driven by a stepping motor.

  In step S607, it is checked whether or not the lens position is at the scan end position. If it is the end position, the process proceeds to step S609, and if not, the process proceeds to step S608.

  In step S608, the focus lens 104 is moved by a predetermined amount in the predetermined direction set in step S604, and the process proceeds again to step S606.

  In step S609, if the focus evaluation value stored in step S606 has a focus evaluation value greater than or equal to a predetermined value, that is, if the focus evaluation value is in focus, the maximum value with the highest focus ratio is indicated. A value is obtained, and the position of the focus lens 104 at that time is extracted. If the focus evaluation value stored in step S606 does not have a focus evaluation value greater than or equal to a predetermined value, that is, if it is out of focus, the focus lens 104 is moved to a preset position called a fixed point.

  In step S610, the focus lens 104 is moved to a position where the focus evaluation value obtained in step S609 shows the maximum value.

  In step S611, the result of focusing / non-focusing in the current AF operation, camera setting information, and shooting condition information such as subject information are stored in the DRAM 113.

  Next, the photographing operation in step S207 in the flowchart of FIG. 2 will be described based on the flowchart of FIG.

  In step S701, a photometric operation, that is, measurement of subject brightness is performed.

  In step S702, the image sensor 108 is exposed according to the photometric result in step S701.

  In step S703, the image formed on the image pickup element surface in step S702 is photoelectrically converted into an analog signal, which is converted into a digital signal after preprocessing such as output noise removal and nonlinear processing by the A / D converter 109. Converted.

  In step S704, the output signal from the A / D conversion unit 109 is sent to the WB processing unit 111 to perform WB processing, and the processed signal is adjusted by the image processing unit 110 to obtain an appropriate output image signal. .

  In step S 705, the output image signal obtained in step S 704 is converted into an image format such as a JPEG format by the format converter 112 and temporarily stored in the DRAM 113.

  In step S706, the image data temporarily stored in the DRAM 113 in step S705 is transferred and stored by the image recording unit 114 to a memory in the camera or an external storage medium such as a memory card attached to the camera.

  Hereinafter, an example of determining whether the scene is the same as that at the time of the previous shooting in step S602 in the flowchart of FIG. 7 will be described with reference to FIGS. FIG. 9 to FIG. 11 are diagrams showing target scene images for determining whether the scene is the same as that at the previous shooting.

  As shown in FIG. 9, the face detection information detected in the previous shooting and the current shooting is both the number of faces and the position is the center of the screen, but the size of the face is compared with the previous shooting. In the case of a largely detected scene, it is determined that the scene has changed and is not the same scene, and distance measurement is performed. This is because the focus position of the focus lens 104 changes depending on the zoom position when zooming has been performed since the previous shooting, and because the distance to the subject is reduced when zooming is not performed, Judge that there is a change.

  As shown in FIG. 10, the face detection information detected in the previous shooting and the current shooting does not change in size for the face at the same position as in the previous shooting, but for a scene with an increased number of faces. State. In this case, if there is no change in the face detection information within the set distance measurement area and the shooting condition information such as the zoom position is the same, it is determined that the scene is the same, and the focus lens 104 is driven to the previous shooting position. On the other hand, when the number of faces increases in the distance measurement area, it is determined that there is a change in the scene and distance measurement is performed.

  As shown in FIG. 11, the face detection information detected in the previous shooting and the current shooting will be described with respect to a scene in which the number of faces and the size of the face are not changed, but the face position is moving. In this case, if the size of the detected face is small and the zoom position obtained from the shooting condition information is a zoom position closer to the wide depth of field, the change in the detected face distance is the subject. Since the change is within the depth of field and the influence on the focus is negligible, it is determined that the scene is the same, and the focus lens 104 is driven to the previous shooting position. On the other hand, if the zoom position is close to Tele with a shallow depth of field, it is determined that there is a change in the scene and distance measurement is performed.

  Although the AF operation shown in the embodiment of the present invention is based on the contrast detection method, the present invention is not limited to the contrast detection method. For example, a configuration in an electronic camera on which an AF sensor is separately mounted is also possible.

  As described above, in the present embodiment, by using the face detection unit in the imaging apparatus including the automatic focus adjustment device, the face detection information is obtained by the face detection unit from the previous captured image and the current captured image, and the same scene is obtained. If it is determined that the scene is the same, the focus lens position is controlled to the previous shooting position and the scanning operation is omitted.If the scene changes and scanning is necessary, scanning is performed. Necessary scanning can be omitted, and the release time lag can be shortened.

  In addition, by using the face detection means to determine whether or not the scene has changed, if there is no change in the person who is the main subject in focus, Even if there is some change in the world, it can be determined that there is almost no doubt that the main subject remains the person. Therefore, if there is no change in the camera settings and there is almost no change in the face detection result, it can be considered that there is no change in the situation of the main subject based only on these results.

(Second Embodiment)
The operation of this embodiment will be described with reference to the flowchart of FIG. Steps for performing the same processing as in the first embodiment are assigned the same numbers as in FIG.

  This embodiment is different from the first embodiment in that the first embodiment is a shooting process at the time of single shooting, whereas in the present embodiment, it is a shooting process at the time of continuous shooting. .

  Since the steps up to step S207 are the same in the flowchart, the processing after step S308 in FIG. 12 will be described.

  In step S308, it is checked whether the setting by the drive mode switch 124 is a continuous shooting mode in which continuous shooting is performed. If it is the continuous shooting mode, the process proceeds to step S309.

  In step S309, it is determined whether SW2 is kept pressed. If SW2 is kept pressed, the process returns to step S203 and is repeated. If SW2 is not kept pressed in step S309, shooting is terminated.

  As described above, in this embodiment, it is determined whether or not the shooting mode is the continuous shooting mode. If the shooting mode is the continuous shooting mode, the process of the first embodiment is repeated again in the same scene. In such a case, it is possible to reduce the release time lag by eliminating the trouble of scanning the image for each shot.

  In the above-described embodiment, the face detection process has been described with an example. However, the present invention is not limited to this. Any other known face detection process may be used as long as the position, size, and number of faces can be grasped. Furthermore, an individual may be specified from the information regarding the detected face, and a criterion for determining whether the individual matches the previous shooting may be added.

  Further, in the above-described embodiment, the description has been given by taking the example in which the photographing lens is incorporated in the electronic camera, but a configuration in which the photographing lens can be detached from the electronic camera body may be employed. Further, it may be configured such that imaging and a control unit including a face detection processing unit and a system control unit can communicate with each other by wire or wirelessly in order to convert a subject image from an electrical signal to image data. .

  Further, in the above-described embodiment, when a change in the shooting scene is detected according to the face detection information and the zoom information, but the shooting mode is set assuming that a moving subject is shot, these are set. A scan for the AF operation may be performed every time without detecting a change in the shooting scene.

It is a block diagram of the imaging device in a 1st embodiment. It is a flowchart figure of the process in 1st Embodiment. It is a flowchart figure of the zoom process in 1st Embodiment. It is a flowchart figure of the face detection process in 1st Embodiment. It is a figure explaining an example of the face detection method and the discrimination method of the shape of eyes. It is a chromaticity diagram showing representative colors in the CIELAB Lab color space. It is a flowchart figure of AF processing in a 1st embodiment. It is a flowchart figure of the process of imaging | photography operation | movement in 1st Embodiment. It is a figure which shows the case where the magnitude | size is changing although the number and position of a face do not change at the time of last imaging | photography. It is a figure which shows the case where the size of the face of the same position as the time of previous photography does not change, but the number of faces increases. It is a figure which shows the case where the whole position has changed, although the number and size of a face do not change at the time of last imaging | photography. It is a flowchart figure of the process in 2nd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Shooting lens 102 Aperture and shutter 103 AE processing unit 104 Focus lens 105 AF processing unit 106 Strobe 107 EF processing unit 108 Image sensor 109 A / D conversion unit 110 Image processing unit 111 WB processing unit 112 Format conversion unit 113 DRAM
114 Image recording unit 115 System control unit 116 VRAM
117 Operation display unit 118 Operation unit 119 Attitude detection sensor 120 Main SW
121 Mode SW
122 SW1
123 SW2
124 Drive mode SW
125 Face detection processing unit

Claims (10)

Ranging means for measuring the distance to the subject;
Storage means for storing photographing conditions at the time of distance measurement by the distance measuring means;
Face detection information extraction means for extracting face detection information from the captured image signal;
A determination unit configured to perform determination based on the face detection information and the shooting condition;
According to the determination result of the determination unit, the imaging is switched between an operation of shooting using the shooting conditions at the previous shooting as it is and an operation of performing a new distance measurement operation and resetting the shooting conditions to perform shooting. apparatus.   The imaging apparatus according to claim 1, wherein the face detection information includes at least one of the number, position, and size of faces.   The imaging apparatus according to claim 1, wherein the photographing condition includes at least a zoom position of a lens.   4. The operation according to claim 1, wherein the imaging device has a continuous shooting mode in which continuous shooting is possible, and the camera is in a predetermined setting including the continuous shooting mode during shooting. An imaging device.   The determination unit determines whether or not a scene has changed from a previously performed shooting based on the face detection information and the shooting conditions, and the determination unit has changed the scene from the previous shooting. If it is determined that the shooting has not been performed, shooting is performed using the shooting conditions at the time of the previous shooting as is. If it is determined that the shooting has changed, a new distance measurement operation is performed and the shooting conditions are reset and the shooting is performed. The imaging apparatus according to claim 1, wherein: A distance measurement process for measuring the distance to the subject,
A storage process for storing shooting conditions at the time of distance measurement by the distance measurement process;
A face detection information extraction step of extracting face detection information from the captured image signal;
A determination process for performing determination based on the face detection information and the shooting conditions;
According to the determination result of the determination process, imaging is performed by switching between an operation of shooting using the shooting conditions at the previous shooting as it is and an operation of performing a new distance measuring operation and resetting the shooting conditions to perform shooting. Control method of the device.   The method of controlling an imaging apparatus according to claim 6, wherein the face detection information includes at least one of the number, position, and size of faces.   8. The method according to claim 6, wherein the photographing condition includes at least a zoom position of a lens.   The operation according to any one of claims 6 to 8, wherein the imaging device has a continuous shooting mode in which continuous shooting is possible, and is in a predetermined setting including the continuous shooting mode at the time of shooting. A method for controlling the imaging apparatus. The determination process is to determine whether or not the scene has changed from the previously performed shooting based on the face detection information and the shooting condition, and the scene has changed from the previous shooting by the determination process. If it is determined that the shooting has not been performed, shooting is performed using the shooting conditions at the time of the previous shooting as is. If it is determined that the shooting has changed, a new distance measurement operation is performed and the shooting conditions are reset and the shooting is performed. The method for controlling an imaging apparatus according to claim 6, wherein:

Images