JP2008301355A - Imaging apparatus and program thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize an imaging apparatus capable of suppressing deterioration of image quality and obtaining a good image with a main subject not blurring and its program.
When a shutter button is half-pressed, face detection processing is performed (S3). When a plurality of faces are detected by the face detection process (Y in S5) and it is recognized that there is a registered person's face among the detected faces (Y in S7), the faces are recognized. The subject is determined as the main subject, and the process of detecting the recognized face movement is started (S8). On the other hand, when one face is detected by the face detection process (N in S5), when a plurality of faces are detected but it is not recognized that there is a registered person's face (N in S7), The detected face is determined as the main subject, and a process of detecting the detected face motion is started (S9). If the shutter button is fully pressed and the detected amount of motion is greater than or equal to a predetermined amount (Y in S12), sensitivity-type blur reduction processing is performed (S14).
[Selection] Figure 2

Description

  The present invention relates to an imaging device and a program thereof, and more particularly, to an imaging device that suppresses blurring of a subject and a program thereof.

  In recent years, in an imaging apparatus, for example, a digital camera, when a subject moves quickly, a technique has been developed that obtains image data without blurring by increasing the shooting sensitivity and increasing the shutter speed (Patent Literature). 1).

JP Patent Publication No. 2004-120576

  However, according to the above-mentioned patent document, it is possible to obtain image data without blurring. However, if any of the captured subjects moves, the imaging sensitivity is increased to increase the shutter speed. Due to the movement of the subject that has nothing to do with the main subject that the person really wants to shoot, the shooting sensitivity has increased and the image quality has deteriorated.

  Therefore, the present invention has been made in view of such conventional problems, and an object of the present invention is to provide an imaging apparatus capable of suppressing deterioration in image quality and obtaining a good image in which a main subject is not blurred, and a program therefor. And

In order to achieve the above object, an image pickup apparatus according to the first aspect of the present invention includes an image pickup means for picking up an image of a subject,
Face detection means for detecting whether or not there is a face in the image data imaged by the imaging means;
A face blur detection unit that detects a blur amount of the face detected by the face detection unit based on image data captured by the imaging unit;
Based on the amount of blur detected by the face blur detection means, blur reduction means for reducing blur,
It is provided with.

For example, as described in claim 2, the face detection means includes
You may make it detect whether a face exists in the predetermined area | region of the image data imaged by the said imaging means.

For example, as described in claim 3, the predetermined region is
You may make it be the center area | region of an angle of view, or the area | region arbitrarily designated by the user.

Further, for example, as described in claim 4, the blur reduction means is
Sensitivity-type blur reduction may be performed by increasing the shooting sensitivity and increasing the shutter speed.

Further, for example, as described in claim 5, the blur reduction means is
When the amount of blur detected by the face blur detector is larger than a predetermined amount, blur reduction may be performed.

Further, for example, as described in claim 6, the blur reduction means includes:
The degree of blur reduction may be increased according to the amount of blur detected by the face blur detection means.

For example, as described in claim 7, the face detection unit includes:
Means for detecting whether or not there is a face of a person registered in advance in the image data imaged by the imaging means;
The face blur detection means includes
Based on the image data picked up by the image pickup means, the blur amount of the face of the person registered in advance detected by the face detection means may be detected.

For example, as described in claim 8, the face detection unit includes:
Only when it is detected that there are a plurality of faces in the image data picked up by the image pickup means, it may be detected whether there is a face of a person registered in advance.

Further, for example, as described in claim 9, the face blur detection unit includes:
When a face of a person registered in advance is detected by the face detection means, the amount of blurring of the face of the person registered in advance is detected, and the face of the person registered in advance by the face detection means is detected. If no face is detected and a face is detected, the amount of shake of the detected face may be detected.

Further, for example, as described in claim 10, the image sensor includes a blur detection unit that detects a blur amount of a predetermined area of image data captured by the imaging unit,
The blur reducing means is
When a face is not detected by the face detection unit, blur reduction may be performed based on a blur amount of a predetermined area detected by the blur detection unit.

Further, for example, as described in claim 11, the blur detection unit includes:
Detect blur amount in multiple areas of image data,
The blur reducing means is
When a face is not detected by the face detection unit, blur reduction may be performed based on the blur amount of each area detected by the blur detection unit.

Further, for example, as described in claim 12, the face blur detection unit and the blur detection unit include:
You may make it detect blur amount by detecting a motion vector.

Further, for example, as described in claim 13, a still image shooting control unit that performs still image shooting using the imaging unit is provided,
The blur reducing means is
Blur reduction may be performed on still image data captured by the still image capturing control means.

In order to achieve the above object, a program according to the invention described in claim 14 includes an imaging process for imaging a subject,
Face detection processing for detecting whether or not there is a face in the image data captured by the imaging processing;
A face blur detection process for detecting a blur amount of the face detected by the face detection process based on image data captured by the imaging process;
Based on the blur amount detected by the face blur detection process, the blur reduction process for reducing the blur,
And each of the above processes is executed by a computer.

  According to the present invention, it is possible to suppress a deterioration in image quality and obtain a good image in which the main subject is not blurred.

Hereinafter, the present embodiment will be described in detail with reference to the drawings as an example in which the imaging apparatus of the present invention is applied to a digital camera.
[Embodiment]
A. Configuration of Digital Camera FIG. 1 is a block diagram showing a schematic electrical configuration of a digital camera 1 that implements the imaging apparatus of the present invention.
The digital camera 1 includes a photographic lens 2, a lens driving block 3, an aperture / shutter 4, a CCD 5, a vertical driver 6, a TG (timing generator) 7, a unit circuit 8, a DMA controller (hereinafter referred to as DMA) 9, a CPU 10, and a key input. 11, memory 12, DRAM 13, DMA 14, motion vector detection unit 15, DMA 16, image generation unit 17, DMA 18, DMA 19, display unit 20, DMA 21, compression / decompression unit 22, DMA 23, flash memory 24, and bus 25. .

  The photographic lens 2 includes a focus lens and a zoom lens that are composed of a plurality of lens groups (not shown). A lens driving block 3 is connected to the photographing lens 2. The lens driving block 3 moves the focus lens and the zoom lens in the optical axis direction in accordance with a focus motor and a zoom motor (not shown) that drive the focus lens and the zoom lens in the optical axis direction, respectively, and a control signal sent from the CPU 10. And a zoom motor driver (not shown).

The diaphragm / shutter 4 includes a drive circuit (not shown), and the drive circuit operates the diaphragm / shutter 4 in accordance with a control signal sent from the CPU 10. The aperture / shutter 4 functions as an aperture and shutter.
The diaphragm is a mechanism for controlling the amount of light incident on the CCD 5, and the shutter is a mechanism for controlling the time for which light is applied to the CCD 5, and the time for which light is applied to the CCD 5 (exposure time). Depends on the shutter speed.

  The CCD 5 is scanned and driven by the vertical driver 6, photoelectrically converts the intensity of light of each color of the RGB value of the subject image at a constant period, and outputs it to the unit circuit 8 as an imaging signal. The operation timing of the vertical driver 6 and the unit circuit 8 is controlled by the CPU 10 via the TG 7.

  A TG 7 is connected to the unit circuit 8, a CDS (Correlated Double Sampling) circuit that holds the imaged signal output from the CCD 5 by correlated double sampling, and an AGC that performs automatic gain adjustment of the imaged signal after the sampling. (Automatic Gain Control) circuit and an A / D converter that converts the analog signal after the automatic gain adjustment into a digital signal. The image pickup signal obtained by the CCD 5 passes through the unit circuit 8 and is then Bayered by the DMA 9 The data is stored in the buffer memory (DRAM 13).

The CPU 10 has a function of performing AE processing, AF processing, and the like, and is a one-chip microcomputer that controls each part of the digital camera 1.
In particular, the CPU 10 includes a face detection unit 101 that detects the face of the subject, and a sensitivity type blur reduction unit 102 that performs sensitivity type blur reduction according to the motion vector (blur amount) detected by the motion vector detection unit 15. The face detection unit 101 performs a face detection process for detecting whether a human face is in the image data, and a face recognition process for recognizing who the detected face is.
Here, the sensitivity-type blur reduction is to reduce blur by increasing the sensitivity of an image sensor such as a CCD and increasing the shutter speed. Here, “increased sensitivity” refers to the action of brightening the image. The method increases the sensitivity by increasing the gain of the imaging signal output from the CCD 5, or the CCD 5 is driven to add pixels. Alternatively, the sensitivity may be increased by executing pixel addition processing on the image pickup signal output from the CCD 5.

The key input unit 11 includes a plurality of operation keys such as a shutter button, a mode switching key, a cross key, and a SET key that can be pressed halfway down, and outputs an operation signal corresponding to a user key operation to the CPU 10.
In the memory 12, a control program (for example, a program necessary for AE and AF processing) necessary for the CPU 10 to control each part of the digital camera 1 and necessary data (AE program diagram, etc.) are recorded. The CPU 10 operates according to the program.

  The DRAM 13 is used as a buffer memory for temporarily storing image data picked up by the CCD 5 and also as a working memory for the CPU 10.

The DMA 14 reads Bayer data or luminance / chrominance signal image data stored in the buffer memory and outputs the data to the motion vector detection unit 15.
The motion vector detection unit 15 detects a motion vector in a certain region of the frame (image data), and detects a motion vector (motion amount) of the image data using a representative point matching method, a block matching method, or the like. To do. In order to detect this motion vector, since the motion vector is detected based on a frame including a certain region and a frame captured after that, a storage circuit that holds the captured frame for a certain time is also included. The detected motion vector is sent to the CPU 10 via the DMA 14. Thereby, the amount of blur in the certain area can be detected well.

The DMA 16 reads the image data of the Bayer data stored in the buffer memory (DRAM 13) and outputs it to the image generation unit 17.
The image generation unit 17 performs processing such as pixel interpolation processing, γ correction processing, and white balance processing on the image data sent from the DMA 16, and also generates a luminance color difference signal (YUV data). That is, it is a portion that performs image processing.
The DMA 18 stores the image data (YUV data) of the luminance / color difference signal subjected to image processing by the image generation unit 17 in a buffer memory.

The DMA 19 outputs image data of YUV data stored in the buffer memory to the display unit 20.
The display unit 20 includes a color LCD and its drive circuit, and displays an image of the image data output from the DMA 19.

The DMA 21 outputs the YUV data image data and compressed image data stored in the buffer memory to the compression / decompression unit 22, and buffers the image data compressed by the compression / decompression unit 22 and the decompressed image data. It is stored in memory.
The compression / decompression unit 22 is a part that performs compression / decompression of image data (for example, compression / decompression in JPEG or MPEG format).
The DMA 23 reads compressed image data stored in the buffer memory and records it in the flash memory 24, or stores the compressed image data recorded in the flash memory 24 in the buffer memory.

B. Operation of Digital Camera 1 The operation of the digital camera 1 in the first embodiment will be described with reference to the flowchart of FIG.

When the photographing mode is set by the user operating the mode switching key of the key input unit 11, in step S <b> 1, the CPU 10 starts imaging of the subject by the CCD 5, and is sequentially generated by the image generation unit 17 and buffer memory (DRAM 13 The so-called through image display is started in which the image data of the luminance color difference signal stored in () is displayed on the display unit 20.
Next, in step S2, the CPU 10 determines whether or not the shutter button is half-pressed by the user. This determination is made based on whether or not an operation signal corresponding to a half-press operation of the shutter button is sent from the key input unit 11.

If it is determined in step S2 that the shutter button is not half-pressed, the process stays in step S2 until the shutter button is half-pressed. If it is determined that the shutter button is half-pressed, the process proceeds to step S3. Face detection processing is performed based on image data captured immediately before or after the button is pressed halfway. That is, processing for detecting whether or not there is a human face in the image data is performed. This face detection process is performed on the image data in the central region of the angle of view of the captured image data. That is, even when the captured image data includes a face, a face that is not located in the central region is not detected.
At this time, when the shutter button is half-pressed, the CPU 10 outputs the frame image data imaged immediately before or immediately after the shutter button half-press to the motion vector detection unit 15 and then the frame image imaged by the CCD 5. We will also acquire data.

FIG. 3A is a diagram illustrating an example of the state of the central region of the field angle that is the target of the face detection process.
Reference numeral 21 denotes an angle of view of all image data picked up by the CCD 5, and reference numeral 22 denotes an angle of view of a central area to be subjected to face detection processing. Usually, the photographer brings the main subject to be photographed to the center of the angle of view, so the face near the edge is determined not to be the main subject, and the predetermined area located at the center of the angle of view is determined. Only face detection processing is performed. Thereby, it is possible to reduce the burden of the face detection process, and it is possible to suppress the occurrence of erroneous detection of detecting a face that is not the main subject.

  Since this face detection process is a well-known technique and will not be described in detail, for example, it is imaged with pre-stored general human face feature data (feature data of eyes, eyebrows, nose, mouth, ears, etc.). It may be a method for detecting whether or not there is image data of a human face in the captured image data by comparing and collating the obtained image data, or another method may be used. . In short, any method for detecting a human face may be used. This face detection process is such that it roughly recognizes a person's face, such as the outline of a person's face, eyes and mouth, and there is no need to specifically recognize who the face is. .

Next, in step S <b> 4, the CPU 10 determines whether or not a face has been detected by the face detection process by the face detection unit 101.
If it is determined in step S4 that a face has been detected by the face detection process, the process proceeds to step S5, and the CPU 10 determines whether or not a plurality of faces have been detected.
If it is determined in step S5 that a plurality of faces have been detected, the process proceeds to step S6, where the face detection unit 101 of the CPU 10 recognizes whether there is a registered person's face among the detected faces. Process.

This face recognition process not only detects whether a person's face is in the image, but also specifically recognizes who the subject's face is. For example, recognition of eye position, mouth position, nose position, etc., and their positional relationship, etc. are digitized to calculate numerical data (face feature data), and the face of a registered person Face recognition is performed by comparing with the face feature data. By this face detection process, the position and size of the detected face on the image data are also known. The face area is determined based on the position and size of the face.
Here, the user can register a person's face in the face registration mode or the like. In this face registration mode or the like, when the user designates a face image of a person to be registered, the CPU 10 registers numerical data (face feature data) calculated by face recognition from the face image of the person. The registered face feature data is recorded in the memory 12.

  Next, in step S7, the CPU 10 determines whether or not a registered face exists in the image data based on the recognition result obtained by the face recognition process. If no face is registered, it is determined that there is no registered face in the image data.

If it is determined in step S7 that the registered face is in the image data (if it is determined that there is a face recognized face), the face is determined to be the main subject, and the process proceeds to step S8. Based on the frame image data sequentially captured, the detection unit 15 starts processing for detecting the amount of movement of the registered face that has been face-recognized, and the process proceeds to step S11. At this time, the position and size (face area information of the detected face) of the face recognized for the face are also sent to the motion vector detection unit 15, and the amount of motion of the face is detected based on the position and size. The detected amount of motion is the amount of blurring of the main subject, and the amount of motion increases as the amount of motion increases. This blur amount may be caused by movement of the subject or may be caused by movement of the digital camera 1.
In addition, when there are a plurality of recognized faces, the motion vector detection unit 15 detects the motion amount (blur amount) of each recognized face in parallel.

  The operation of the motion vector detection unit 15 will be briefly described. Among the frame image data subjected to face recognition, the image data of the face area recognized by the face, the frame image data captured next by the CCD 5, and Based on the above, the recognized face motion amount (motion vector) is detected by a block matching method or the like. That is, the amount of motion of the recognized face is detected by detecting where the image data of the recognized face area is in the frame image data captured next by the block matching method or the like. When frame image data is newly captured, a motion vector is detected by block matching or the like based on the detected face region image data and the newly captured frame image data.

  The CCD 5 also performs face recognition processing on the next captured image of the foam image, thereby detecting where the recognized face is and detecting the amount of movement of the face. In short, any method may be used as long as the recognized face is located or the movement of the recognized face is detected.

On the other hand, if it is determined in step S5 that a plurality of faces are not detected, or if it is determined in step S7 that a plurality of faces are detected but no registered face is present in the image data, detection is performed. The determined face is determined to be the main subject, and the process proceeds to step S9. The CPU 10 causes the motion vector detection unit 15 to detect the face motion detected by the face detection process in step S3 based on the frame image data sequentially captured. The process of detecting the amount is started, and the process proceeds to step S11. At this time, the position and size of the detected face (face area information of the detected face) are sent to the motion vector detection unit 15, and the amount of motion of the face is detected based on the detected position and size. This detected amount of movement is the amount of blurring of the main subject.
In addition, when there are a plurality of detected faces (NO in step S7), the motion vector detection unit 15 detects the detected motion amount (blur amount) of each face in parallel.

  As described above, this motion detection processing is performed based on the image data of the face area detected in step S3 by the block matching method or the like and the frame image data captured next. Although the amount of motion is detected, when there is one detected face, the amount of motion of the face may be detected by performing face detection processing on the frame image data captured thereafter. . When there are a plurality of detected faces, each detected face is imaged by recognizing each detected face and performing face recognition on the frame image data captured thereafter. It is also possible to detect where the detected frame image data is and to detect the amount of movement of each detected face. In short, any method may be used as long as the detected face is located or the motion of the detected face is detected.

On the other hand, if it is determined in step S4 that no face has been detected by the face detection process, the process proceeds to step S10, where the CPU 10 sets a number of sample points in the frame image data that is the target of the face detection process, and the motion vector. Based on the frame image data sequentially captured, the detection unit 15 starts processing for detecting the amount of movement of the set sample points, and the process proceeds to step S11. At this time, the position and size (sample area information) of each set sample point are also sent to the motion vector detection unit 15, and based on this, the amount of motion of each sample point is detected.
In addition, the motion vector detection unit 15 detects the amount of motion (blur amount) of each sample point in parallel.
Thereby, it is possible to detect image blur.

  FIG. 3B is a diagram illustrating a state of sample points. Reference numeral 23 denotes each sample point, and this sample point has a certain area. Then, the amount of motion of each sample point 23 is detected by detecting where the image data of each sample point 23 is in the next captured frame image data by the block matching method or the like.

In step S11, the CPU 10 determines whether or not the shutter button has been fully pressed by the user.
If it is determined in step S11 that the shutter button is not fully pressed, the process stays in step S11 until the shutter button is fully pressed. If it is determined that the shutter button is fully pressed, the process proceeds to step S12, and the CPU 10 detects the detection. It is determined whether or not the amount of movement is greater than or equal to a predetermined amount. This determination is made as to whether or not the amount of motion detected immediately before the shutter button is fully pressed (the largest amount of motion when a plurality of motion amounts are detected) is equal to or greater than a predetermined amount.

If it is determined in step S12 that the detected motion amount is not greater than or equal to the predetermined amount, the process proceeds to step S13, and the CPU 10 performs a normal AE process and proceeds to step S15. That is, sensitivity-type blur reduction is not performed. Thereby, when the detected amount of motion is not equal to or greater than the predetermined amount, the image quality is not deteriorated. This is because if the detected amount of motion is not greater than or equal to the predetermined amount, it is not a blur that reduces blur even if the image quality is degraded.
This normal AE process is to set an appropriate shutter speed, aperture value, and sensitivity so that an appropriate exposure amount is obtained. A normal AE program diagram for taking a still image (considering the amount of motion). No AE program diagram).

On the other hand, if it is determined in step S12 that the detected movement is greater than or equal to the predetermined amount, the process proceeds to step S14, where the sensitivity-type blur reduction unit 102 of the CPU 10 sets the sensitivity to a higher value and increases the shutter speed. To go to step S15. Accordingly, it is possible to reduce the blur only when there is a blur that needs to be reduced, and it is possible to reduce the blur of the main subject while suppressing the deterioration of the image quality.
In this AE process, the sensitivity is set higher than the sensitivity set by the normal AE process and the shutter speed is set faster than the shutter speed set by the normal AE process so that the appropriate exposure amount is obtained. At this time, it is performed based on an AE program diagram in consideration of the amount of motion for still image shooting. That is, sensitivity-type blur reduction is performed. It should be noted that the shutter speed or the like may be set based on normal AE processing so that the set shutter speed is increased and the sensitivity is reset again.

In addition, this sensitivity-type blur reduction changes the intensity (degree) of blur reduction according to the amount of motion detected immediately before the shutter button is fully pressed (the largest amount of motion when there are a plurality of detected motion amounts). To do. When the difference between the detected motion amount and the predetermined amount is small, the sensitivity blur reduction strength is weakened. When the difference between the detected motion amount and the predetermined amount is large, the sensitivity blur reduction strength is increased. To do. This is because if the detected motion is much larger than the predetermined amount, the sensitivity and shutter speed are considerably high, and the blur cannot be eliminated unless it is set fast (the intensity of blur reduction is not increased). On the other hand, if the detected motion is slightly larger than the predetermined amount, blurring can be eliminated by setting the sensitivity and shutter speed slightly higher and faster (even if the blur reduction intensity is reduced). . Thereby, it is possible to reduce blurring of the main subject while suppressing deterioration in image quality.
If the detected amount of motion is equal to or greater than a predetermined amount, the blur reduction intensity may be uniform.

  As described above, in the above embodiment, when a human face is detected in the central area of the angle of view, the detected amount of movement of the face is detected, and the sensitivity is determined according to the amount of movement of the face. Since the motion blur reduction is performed, the movement amount of other subjects other than the main subject does not adversely affect the sensitivity motion blur reduction.

FIG. 4A is a diagram illustrating an example of image data captured immediately before or after the shutter button is half-pressed and a state of a detected face, taking as an example a case where the number of detected faces is one. Yes.
Reference numeral 22 in FIG. 4A shows the angle of view of the central area that is the target of the face detection process. As a subject, a person 31 posing and a car 32 running in the left direction are captured. I understand. In such a composition, it is understood that the photographer does not want to photograph the car 32 but wants to photograph the person 31. That is, the true photographer (main subject) is the person 31.

Conventionally, even in such a case, the amount of movement of the traveling car 32 is also detected, and sensitivity-type blur reduction is performed according to the detected amount of movement. That is, the photographing sensitivity is increased and the shutter speed is increased so that the car 32 does not blur. Therefore, despite the small or no movement of the person 31 as the main subject, the photographing sensitivity has increased more than necessary, and the image quality has deteriorated accordingly, so that a good image cannot be obtained.
On the other hand, when a face is detected in the center area of the angle of view, that is, when the face of the person 31 is detected, the sensitivity-type blur reduction process is performed according to the detected amount of movement of the face. There is no need to increase sensitivity unnecessarily, and blurring of the person 31 as the main subject can be reduced. Therefore, it is possible to obtain a good image without blurring of the main subject.

FIG. 4B is a diagram illustrating an example of image data captured immediately before or after the shutter button is half-pressed and a state of a face that is detected and recognized, and an example in which a plurality of faces are detected is illustrated. I have to.
Similarly, reference numeral 22 in FIG. 4B shows the angle of view of the central region that is the target of the face detection process. As subjects, a person 31 posing and a girl 33 running leftward You can see that the background was imaged. In such a composition, it can be seen that the photographer does not want to photograph the girl 33 but wants to photograph the person 31. That is, the true photographer is the person 31.

However, when there are a plurality of faces as described above, a plurality of faces are detected, and the movement of the girl 33 that is not the main subject adversely affects the blur reduction process. In other words, the sensitivity is increased and the shutter speed is increased so that the running girl 33 does not blur, so the sensitivity increases more than necessary, and the image quality deteriorates accordingly.
Therefore, by registering the face of the person 31 in advance, even if there are a plurality of detected faces (here, the face of the person 31 and the face of the girl 33), the face of the registered person (main subject) Since the sensitivity-type blur reduction process is performed based on the amount of movement, it is not necessary to increase the sensitivity unnecessarily, and the blur of the person 31 as the main subject can be reduced. Therefore, it is possible to obtain a good image without blurring of the main subject while suppressing deterioration in image quality.

  4A and 4B have been described as an example. For example, when the photographer pans the camera so that the person who is the moving main subject is located in the center, the movement of the main subject Conversely, the background movement of mountains, buildings, people, etc. increases, but even in such a case, the background movement does not affect the sensitivity-type blur reduction process.

  In step S15, the CPU 10 performs still image shooting processing under the exposure conditions set in the AE processing, is subjected to image processing by the image generation unit 17, is compressed by the compression / decompression unit 22, and is stored in the buffer memory. Still image data is recorded in the flash memory 24. As a result, it is possible to obtain a good still image in which the main subject is not blurred while suppressing deterioration in image quality.

As described above, in the embodiment, when a face is detected, it is determined that the detected face is the main subject, and sensitivity-type blur reduction is performed according to the detected amount of movement of the face. The blurring of the main subject can be reduced without unnecessarily increasing the sensitivity, and a good image without blurring of the main subject can be obtained while suppressing deterioration in image quality.
In addition, when the face of a registered person (main subject) is in the image data, sensitivity-type blur reduction is performed based on the amount of movement of the registered person's face, so that multiple faces are detected. Even in such a case, it is possible to reduce blurring of the main subject without unnecessarily increasing sensitivity, and it is possible to obtain a good image without blurring while suppressing deterioration in image quality.
Further, since the face recognition process is performed when there are a plurality of detected faces, the face recognition process is not performed unnecessarily, and the processing load can be reduced. That is, face recognition processing that is more burdensome than face detection processing is not performed wastefully.
In addition, since the target area for the face detection process is the center area of the angle of view, a face that is unlikely to be the main subject is not detected, and the burden of the face detection process can be reduced.
In addition, when a face is not detected, the amount of motion of a plurality of sample points is detected, and blurring is reduced based on the detected amount of motion, so that it is possible to detect image blurring, and image blurring is detected. Can be reduced.

[Modification]
The above-described embodiment can be modified as follows.

(01) In the above embodiment, the motion detection process in steps S8 and S9 detects the movement of the face wherever the face moves in the image data. In the case where the face that is out of the center area of the angle of view deviates, detection of the amount of movement of the face may be stopped thereafter.
This is because the user usually pans the camera so that the main subject is located in the center, and it is considered that the face outside the center area of the angle of view is not the main subject, and the movement of the face that is not the main subject is sensitive. It can prevent adversely affecting the expression blur reduction.

(02) In the above-described embodiment, the blurring of still image data obtained by still image shooting is reduced. However, the present invention is also applied during moving image shooting (during live image display and moving image shooting recording). It may be.
In this case, the sensitivity, shutter speed, and aperture are set using a normal AE program diagram for moving image capturing and an AE program diagram considering the amount of motion for moving image capturing.

In addition, when reducing blur during moving image capturing, blur may be corrected not by sensitivity-type blur reduction but by trimming. At this time, the blur may be reduced by correcting the blur by changing the trimming position according to the detected face motion vector.
Also, in the case of still image shooting, blur correction may be performed by trimming instead of sensitivity-type blur reduction. In this case, continuous shooting is performed (sequential still image shooting is performed a plurality of times), and the obtained still image data is trimmed and combined according to the amount of blur to correct the blur.

In either case of still image shooting or moving image shooting, if a correction lens is provided in front of the CCD 5 instead of the sensitivity-type blur reduction and the correction lens is moved (shifted), there is no correction lens. Blur may be corrected by moving (shifting) the CCD 5. Also at this time, the correction lens or the CCD 5 is moved in accordance with the detected face motion vector.
If a configuration for correcting blur by such trimming or a configuration for correcting by shifting the CCD 5 or the correction lens, it is possible to appropriately correct the blur of the main subject without degrading the image quality.

(03) In the above embodiment, the face recognition process is performed based on the face detected by the face detection process. Specifically, it is determined whether the registered person's face is in the image data. The face recognition process of recognizing and detecting may be performed at once. That is, the face recognition process may be performed without using the detection result of the face detection process. Further, only the face recognition process may be performed without performing the face detection process. This is because the face can naturally be detected also by the face recognition process.
In this case, the face recognition process is performed only on the image data in the center area of the angle of view. That is, it specifically recognizes and detects whether there is a registered person's face in the central area of the angle of view.

  (04) In the above embodiment, when at least the face detection mode and the face recognition mode are provided, and the face detection mode is selected by the user, the face detection process is performed and the face recognition mode is selected. May perform face recognition processing. That is, the face detection process and the face recognition process may be performed alternatively.

(05) In the above embodiment, when it is determined that a plurality of faces are detected (Y in step S5), and it is determined that there is no registered face (Y in step S7), the operation in step S9 In this example, the detected amount of movement of each face is simply detected in parallel, but the largest amount of movement among the detected amounts of movement of each face may be detected. Alternatively, the average value of the detected amount of movement of each face may be used as the amount of movement, and further, from among each detected face, the proximity from the center of the angle of view of the face, One face may be specified by comprehensively determining the size, and only the amount of movement of the specified face may be detected. This is because the face closer to the center of the angle of view and the larger face are considered as the main subject. This is because the user usually images the main subject so that it is near the center of the angle of view, and also images the main subject so that the face of the main subject becomes large to some extent.
Also, the degree of weighting of the amount of motion detected for each face is determined by comprehensively determining the proximity of the face angle of view from the center and the size of the face, and the average value of the amount of motion is calculated according to the weighting degree. You may make it do.

  Similarly, the operation in step S8 when there are a plurality of faces registered by the face recognition process in step S6 is to detect one of the largest amount of movement among the movement amounts of each face recognized. Alternatively, the average value of the amount of movement of each face recognized face may be used as the amount of movement, and further, the proximity from the center of the angle of view of the face among each face recognized face. Alternatively, one face may be specified by comprehensively determining the size of the face, and only the amount of movement of the specified face may be detected. Also, the degree of weighting of the amount of motion of each face recognized is determined by comprehensively judging the proximity from the center of the angle of view of the face and the size of the face, and the average value of the amount of motion is determined according to the degree of weighting. You may make it calculate.

  Even in the operation of step S10 in the case where there is no detected face, one of the largest motion amounts among the sample points may be detected, or the average motion amount of the detected sample points may be detected. The value may be the amount of movement.

  (06) In the above embodiment, the operation from step S3 to step S10 is performed after the shutter button is half-pressed. However, during the through image display before the shutter button is half-pressed, the operation from step S3 to step S10 is performed. When the operation is performed and the shutter button is half-pressed, the operations in steps S12 to S14 may be performed, and the operation in step S15 may be performed after the shutter button is fully pressed.

  (07) In the above embodiment, the face detection process and the face recognition process are performed once, but may be performed periodically until the shutter button is fully pressed. A new face may be captured over time, and the person who is the main subject may be out of the central area of the angle of view due to the lack of the photographer's shooting technology. Can no longer be determined as the main subject, so it is possible to correct such problems by periodically performing face detection processing and face recognition processing, and to reliably detect the movement of the face that is the main subject. Can do.

(08) In the above embodiment, the face detection process detects a face in the center area of the angle of view, but detects a face in an area other than the center of the angle of view. Alternatively, a face in the area designated by the user may be detected. In short, any face may be used as long as it detects a face in one area (predetermined area) out of the entire area of the image data. This is because the processing load of the face detection process can be reduced. Further, when face detection is performed on an area designated by the user, the face that is the main subject can be effectively detected.
Note that it may be detected whether or not there is a face within the entire range of the image data.

(09) In the above embodiment, when the amount of motion detected immediately before the shutter button is fully pressed is greater than or equal to the predetermined amount, sensitivity-type blur reduction is performed. However, the detected amount of motion is greater than or equal to the predetermined amount. Regardless of whether or not, blur reduction processing may be performed.
In this case, the sensitivity-type blur reduction process is performed according to the amount of motion detected immediately before the shutter button is fully pressed (the largest amount of motion when a plurality of motion amounts are detected). That is, when the detected amount of motion is small, the sensitivity-type blur reduction process is performed with a low intensity, and when the detected amount of motion is large, the sensitivity-type blur reduction process is performed with a high intensity. Thereby, it is possible to reduce blurring of the main subject while suppressing deterioration in image quality.

  (10) In the above embodiment, the face detection process is always performed, but only the face recognition process may be performed. In this case, when the shutter button is half-pressed in step S2, face recognition processing is performed. If a registered face is recognized by the face recognition processing, the process proceeds to step S8, and a registered face is not recognized. Advances to step S10.

  (11) In the above embodiment, when a face is not detected (NO in step S4), a large number of sample points are set. However, one sample point may be set. For example, an area located at the center of the angle of view or an area designated by the user may be set as the sample point. In short, it is only necessary to detect the amount of movement in a region that is not based on the detected and recognized face, and the number and position of sample points to be selected are not limited to the above embodiment. As a result, image blur can be detected, and image blur can be reduced.

  (12) In the above embodiment, it is determined whether or not the amount of motion detected immediately before the shutter button is fully pressed (the largest amount of motion when there are a plurality of motion amounts) is a predetermined amount or more. However, it may be determined whether the largest motion amount among the plurality of motion amounts detected from the start of the motion detection process to immediately before the shutter button is fully pressed is greater than or equal to a predetermined amount. . If the largest amount of movement is greater than or equal to a predetermined amount, blur reduction processing is performed according to the largest amount of movement.

  (13) Moreover, the aspect which combined the said embodiment and said modification (01) thru | or (12) arbitrarily may be sufficient.

(14) The above-described embodiments and modifications of the present invention are merely examples as the best embodiments, so that the principle and structure of the present invention can be better understood. And is not intended to limit the scope of the appended claims.
Therefore, it should be understood that all the various variations and modifications that can be made to the above-described embodiments of the present invention are included in the scope of the present invention and protected by the appended claims.

  Lastly, in each of the above embodiments, the case where the imaging apparatus of the present invention is applied to the digital camera 1 has been described. However, the present invention is not limited to the above embodiment, and the main point is to reduce subject blurring. Any device that can do this is applicable.

1 is a block diagram of a digital camera according to an embodiment of the present invention. It is a flowchart which shows operation | movement of the digital camera of embodiment. It is a figure which shows the center area | region of the angle of view used as the object of a face detection process, and the mode of a sample point. It is a figure which shows an example of the mode of the face of the image data imaged immediately before shutter button half-pressing, or immediately after detection and recognition.

Explanation of symbols

1 Digital Camera 2 Shooting Lens 3 Lens Drive Block 4 Shutter / Shutter 5 CCD
6 Vertical driver 7 TG
8 Unit circuit 9 DMA
10 CPU
11 Key input section 12 Memory 13 DRAM
14 DMA
15 motion vector detector 16 DMA
17 Image generator 18 DMA
19 DMA
20 Display unit 21 DMA
22 Compression / decompression unit 23 DMA
24 flash memory 25 bus

Claims (14)

Imaging means for imaging a subject;
Face detection means for detecting whether or not there is a face in the image data imaged by the imaging means;
A face blur detection unit that detects a blur amount of the face detected by the face detection unit based on image data captured by the imaging unit;
Based on the amount of blur detected by the face blur detection means, blur reduction means for reducing blur,
An imaging apparatus comprising: The face detection means includes
The image pickup apparatus according to claim 1, wherein the image pickup apparatus detects whether or not there is a face in a predetermined area of the image data picked up by the image pickup means. The predetermined area is:
The imaging apparatus according to claim 2, wherein the imaging apparatus is a central area of an angle of view or an area arbitrarily designated by a user. The blur reducing means is
4. The image pickup apparatus according to claim 1, wherein sensitivity-type blur reduction is performed such that shooting sensitivity is increased and shutter speed is increased. The blur reducing means is
5. The imaging apparatus according to claim 1, wherein blur reduction is performed when a blur amount detected by the face blur detection unit is larger than a predetermined amount. The blur reducing means is
6. The imaging apparatus according to claim 1, wherein a degree of blur reduction is increased in accordance with a blur amount detected by the face blur detection unit. The face detection means includes
Means for detecting whether or not there is a face of a person registered in advance in the image data imaged by the imaging means;
The face blur detection means includes
The blur amount of a face of a person registered in advance detected by the face detection unit is detected based on image data captured by the imaging unit. Imaging device. The face detection means includes
8. The imaging apparatus according to claim 7, wherein it is detected whether or not there is a face of a person registered in advance only when it is detected that there are a plurality of faces in the image data imaged by the imaging means. . The face blur detection means includes
When a face of a person registered in advance is detected by the face detection means, the amount of blurring of the face of the person registered in advance is detected, and the face of the person registered in advance by the face detection means is detected. 8. The imaging apparatus according to claim 7, wherein when the face is not detected and the face is detected, the amount of blurring of the detected face is detected. A blur detection unit that detects a blur amount of a predetermined area of the image data captured by the imaging unit;
The blur reducing means is
The blur reduction is performed based on a blur amount of a predetermined area detected by the blur detection unit when a face is not detected by the face detection unit. The imaging device described. The blur detection means includes
Detect blur amount in multiple areas of image data,
The blur reducing means is
The image pickup apparatus according to claim 10, wherein when a face is not detected by the face detection unit, blur reduction is performed based on a blur amount of each area detected by the blur detection unit. The face shake detection means and the shake detection means are:
The imaging apparatus according to claim 1, wherein a blur amount is detected by detecting a motion vector. Still image shooting control means for taking a still image using the imaging means,
The blur reducing means is
The imaging apparatus according to claim 1, wherein blur reduction is performed on still image data captured by the still image capturing control unit. Imaging processing for imaging a subject;
Face detection processing for detecting whether or not there is a face in the image data captured by the imaging processing;
A face blur detection process for detecting a blur amount of the face detected by the face detection process based on image data captured by the imaging process;
Based on the blur amount detected by the face blur detection process, the blur reduction process for reducing the blur,
A program characterized by causing a computer to execute the processes described above.

Images