JP2012060292A - Image processor, image selection method, and program - Google Patents

Abstract

An object of the present invention is to appropriately perform selection processing of an image in which a specific person exists in a plurality of images captured continuously.
An imaging apparatus 100 includes an image acquisition unit 6a that acquires a plurality of images including a plurality of persons including a specific person in succession, and a characteristic section (face image) of the specific person. ) And a person registration DB 7 (first registration table T1) stored for a plurality of persons in association with priorities, and a plurality of images acquired by the image acquisition unit based on the priority stored in the person registration The face detection unit 5 that detects each part (face image) and the feature parts (face images) detected by the face detection unit based on the feature parts (face images) stored in the first registration table Each includes an evaluation value calculation unit b1 that evaluates a smile level and calculates an evaluation value, and an image selection unit 6b that selects a captured image to be recorded based on the evaluation value calculated by the evaluation value calculation unit.
[Selection] Figure 1

Description

  The present invention relates to an image processing apparatus, an image selection method, and a program, and more particularly, to an image selection technique suitable for recording from continuously captured images.

  When attempting to capture an image having a plurality of faces, it is not easy to capture an image in which all of the faces do not blink and are smiling. Therefore, as a conventional technique, a technique is known in which the best face image is selected for each person from a plurality of images that are continuously captured, and these are combined into one sheet (for example, Patent Documents). 1 and 2).

JP 2008-198062 A JP 2002-199202 A

  However, in the techniques of Patent Documents 1 and 2, if an image with a high degree of satisfaction is obtained for all persons existing in one image, the number of consecutive images may increase as the number of persons increases. There is. In such a case, there is also a problem that a sufficient memory capacity must be ensured for temporarily storing continuously captured images.

  Accordingly, the present invention has been made in view of the above problems, and an image processing apparatus and an image that can appropriately perform selection processing of an image in which a specific person exists in a plurality of images captured continuously. It is an object to provide a selection method and a program.

In order to solve the above problem, an image processing apparatus according to claim 1 is provided.
Acquiring means for acquiring a plurality of images including a specific person by continuously capturing a plurality of images, and a characteristic unit that stores a plurality of persons in association with a priority of a characteristic unit of the specific person A storage unit; a detection unit that detects each of the feature parts based on the priority stored in the feature unit storage unit in the plurality of images acquired by the acquisition unit; and a feature unit that is detected by the detection unit. Calculating means for calculating the evaluation value based on the characteristic part stored in the storage means, and selecting means for selecting the captured image to be recorded based on the evaluation value calculated by the calculation means It is characterized by having.

The invention according to claim 2 is the image processing apparatus according to claim 1,
The calculating means calculates an evaluation value for each of a plurality of persons stored in the feature storage means, and the selecting means is an image including a person from each captured image based on the calculated evaluation values. It is characterized by selecting a captured image to be recorded by including an extracting means for extracting an area and an image synthesizing means for synthesizing the image areas extracted by the extracting means.

The image selection method of the invention described in claim 3 is:
An acquisition step of acquiring a plurality of images including a specific person by continuously capturing a plurality of images, and a memory that stores in advance a feature portion of the specific person associated with a priority and stored for the plurality of persons The detection step of detecting the feature portion based on the priority in the plurality of images acquired in the acquisition step, and the feature portion detected in the detection step respectively in the memory And a calculation step for calculating an evaluation value and a selection step for selecting a captured image to be recorded based on the evaluation value calculated in the calculation step. It is characterized by that.

The program of the invention according to claim 4 is:
A computer of an image processing apparatus including a memory in which a feature portion of the specific person is previously associated with a priority and stored for the plurality of persons is continuously picked up by a plurality of images including the plurality of persons including the specific person. Acquisition means for acquiring the detection means, detection means for detecting the characteristic portions based on the priority stored in the memory in the plurality of images acquired by the acquisition means, and the characteristic portions detected by the detection means, respectively. Are calculated based on the characteristic parts stored in the memory, and a calculation means for calculating an evaluation value, a selection means for selecting a captured image to be recorded based on the evaluation value calculated by the calculation means, It is characterized by functioning.

  ADVANTAGE OF THE INVENTION According to this invention, the selection process of the image which a specific person exists in the several image imaged continuously can be performed appropriately.

It is a block diagram which shows schematic structure of the imaging device of Embodiment 1 to which this invention is applied. FIG. 2 is a diagram schematically illustrating an example of a first registration table recorded in the imaging apparatus in FIG. 1. 3 is a flowchart illustrating an example of an operation related to continuous imaging processing by the imaging apparatus of FIG. 1. It is a figure which shows typically an example of the captured image which concerns on the continuous imaging process of FIG. It is a block diagram which shows schematic structure of the imaging device of Embodiment 2 to which this invention is applied. It is a figure which shows typically an example of the 2nd registration table currently recorded on the imaging device of FIG. 6 is a flowchart illustrating an example of an operation related to continuous imaging processing by the imaging apparatus of FIG. 5. It is a figure which shows typically an example of the captured image which concerns on the continuous imaging process of FIG. It is a figure which shows typically an example of the image which concerns on the continuous imaging process of FIG. It is a block diagram which shows schematic structure of the imaging device of the modification 1.

  Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.

[Embodiment 1]
The imaging apparatus 100 according to the first embodiment has a high evaluation value of a characteristic portion (face) of a specific person registered in a person registration database prepared in advance among a plurality of continuously captured images. An image is selected, and the selected captured image is recorded as a recorded image on a recording medium.
The above method will be specifically described below.
FIG. 1 is a block diagram illustrating a schematic configuration of an imaging apparatus 100 according to the first embodiment. As shown in the figure, the imaging apparatus 100 includes an imaging unit 1, an imaging control unit 2, an image data generation unit 3, a memory 4, a face detection unit 5, an image processing unit 6, and a person registration database ( DB) 7, recording control unit 8, display control unit 9, display unit 10, operation input unit 11, and central control unit 12.

The imaging unit 1 includes a lens unit 1a and an electronic imaging unit 1b.
The lens unit 1a includes a plurality of lenses such as a zoom lens and a focus lens.
The electronic imaging unit 1b is composed of, for example, an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-oxide Semiconductor), and converts an optical image that has passed through various lenses of the lens unit 1a into a two-dimensional image signal. To do.
In addition, although illustration is abbreviate | omitted, the imaging part 1 may be provided with the aperture_diaphragm | restriction which adjusts the quantity of the light which passes the lens part 1a.

The imaging control unit 2 controls the imaging of the subject by the imaging unit 1. That is, the imaging control unit 2 includes a timing generator, a driver, and the like, although not illustrated. Then, the imaging control unit 2 scans and drives the electronic imaging unit 1b with a timing generator and a driver, converts the optical image into a two-dimensional image signal with the electronic imaging unit 1b at a predetermined period, and the electronic imaging unit 1b. The image frame (captured image P) is read out from the imaging region for each screen and is output to the image data generation unit 3.
Here, the subject means a plurality of persons (see FIG. 4A, etc.) including a specific person existing in a predetermined background. Specifically, a person whose face registration flag is “ON” in the first registration table T1 (see FIG. 2) registered in the person registration DB 7 (for example, “A”, “B”, etc.) is specified. The subject is at least one of the specific persons and at least one other than the specific person (for example, “C”, “D”, etc.).

As described above, the imaging unit 1 and the imaging control unit 2 continuously capture images including a plurality of persons including a specific person in the background, and a plurality of (for example, n images; n is a natural number) captured images. Image frames of Pa1 to Pan (see FIGS. 4A to 4D, etc.) are sequentially generated.
Note that the captured images Pa1 to Pan may be a plurality of still images or a plurality of image frames constituting a moving image.

Further, the imaging control unit 2 may perform adjustment control of conditions when imaging an object such as AF (automatic focusing process), AE (automatic exposure process), AWB (automatic white balance), and the like.
Here, when a human face is detected from the image frame of the captured image P, the imaging control unit 2 performs AF (automatic focus processing), AE (automatic exposure processing) As a reference area for AWB (automatic white balance), AF (automatic focusing processing), AE (automatic exposure processing), AWB (automatic white balance), or the like may be performed.

The image data generation unit 3 appropriately adjusts the gain for each RGB color component with respect to the analog value signal of the image frame transferred from the electronic imaging unit 1b, and then performs sample holding by a sample hold circuit (not shown). The digital signal is converted into digital data by a / D converter (not shown), color processing including pixel interpolation processing and γ correction processing is performed by a color process circuit (not shown), and then a digital luminance signal Y and color difference signal Cb , Cr (YUV data).
The luminance signal Y and the color difference signals Cb and Cr output from the color process circuit are DMA-transferred to a memory 4 used as a buffer memory via a DMA controller (not shown).

  The memory 4 is constituted by, for example, a DRAM (Dynamic Random Access Memory) or the like, and temporarily stores data processed by the face detection unit 5, the image processing unit 6, the central control unit 12, and the like.

The face detection unit 5 detects a face of a specific person from each of a plurality of captured images P,... Continuously captured by the imaging unit 1 using a predetermined face detection method.
That is, the face detection unit 5 detects, for each of a plurality (for example, n) of image frames of the captured images Pa1 to Pan, a plurality of human faces including a specific person from the image frame as a feature portion. Specifically, for example, the face detection unit 5 sequentially acquires image frames of a plurality of captured images P,... Generated by capturing a live view image by the imaging unit 1 from the memory 4, and from within each image frame. A face image area is detected, and face parts corresponding to eyes, nose, mouth, and the like are sequentially detected from within each detected area. Since the face detection process is a known technique, detailed description thereof is omitted here.
Then, the face detection unit 5 detects a person's face detected from each captured image P (for example, the captured image Pa1) as a specific person whose face registration flag is registered in the first registration table T1 of the person registration DB 7. It is determined whether or not the face image matches (for example, “A” or the like). Specifically, the face detection unit 5 refers to, for example, the first registration table T1 and acquires a face image of a specific person with a registered face registration flag as a template. Then, the face detection unit 5 determines whether or not the face image acquired as a template of a specific person matches the detected face for each of the plurality of captured images P,. As a result of the determination, the face detection unit 5 recognizes the face determined to match the face image acquired as a template as the face of a specific person.
In this way, the face detection unit 5 detects the face of a specific person whose face registration flag (recording instruction information) is registered from each of the plurality of captured images P,... Continuously captured by the imaging unit 1. Detect as feature.

  The image processing unit 6 includes an image acquisition unit 6a and an image selection unit 6b.

The image acquisition unit 6a acquires a plurality of captured images P,.
That is, the image acquisition unit 6a acquires a plurality of captured images P,... Captured by the imaging unit 1 at a predetermined imaging frame rate. Specifically, the imaging unit 1 continuously captures images of a plurality of persons (see FIG. 4A, etc.) including a specific person (for example, “A”) existing in a predetermined background. YUV data of image frames of a plurality (for example, n) of captured images Pa1 to Pan is acquired.
In this way, the image acquisition unit 6a acquires a plurality of captured images P,... In which a plurality of persons including a specific person exist in the continuously captured background.

  The image selection unit 6b has a high evaluation value of a face (characteristic unit) for a specific person (for example, “A”) among the plurality of captured images P acquired by the image acquisition unit 6a. Select. Specifically, the image selection unit 6b includes an evaluation value calculation unit b1 that calculates the smile level of a face as an evaluation value for a specific person, and calculates an evaluation value among a plurality of captured images P,. A captured image P having a high smile level of the face calculated by the part b1 is selected.

The evaluation value calculation unit b1 calculates the smile level of the face as an evaluation value for a specific person existing in the plurality of captured images P acquired by the image acquisition unit 6a.
That is, the evaluation value calculation unit b1 selects the specific person with respect to the captured image P in which the face of the specific person (for example, “A”) is detected by the face detection unit 5 among the plurality of captured images P,. Is calculated as an evaluation value. Specifically, for example, the evaluation value calculation unit b1 detects eyes from the face of a specific person, and digitizes the smile degree according to the size of the entire face of the black eye. For example, the evaluation value calculation unit b1 detects the mouth from the face of a specific person, and quantifies the degree of smile according to how the mouth corners rise or how the mouth opens. Then, the evaluation value calculation unit b1 calculates the total smile level of the face of a specific person by integrating the smile levels digitized for the black eyes and the mouth corners.
Note that the method of calculating the evaluation value of the face (feature part) of a specific person by the evaluation value calculation unit b1 is an example, and is not limited thereto, and can be arbitrarily changed as appropriate. For example, the evaluation value of the face of a specific person may be calculated using the degree of inclination of eyebrows, the degree of eyebrows between the eyebrows and the corners of the corners of the eye in addition to the eyes and mouth. Further, the smile level of the face is calculated as the evaluation value. However, the evaluation content is an example and is not limited to this, and can be arbitrarily changed as appropriate.
As described above, the evaluation value calculation unit b1 calculates the smile level of the face for the specific person as the evaluation value.

  Then, based on the evaluation result by the evaluation value calculation unit b1, the image selection unit 6b selects a captured image P having a high facial smile calculated by the evaluation value calculation unit b1 among the plurality of captured images P,. select. Specifically, when one specific person is registered in the first registration table T1, the image selection unit 6b includes the captured image P in which the face of the specific person is detected by the face detection unit 5. The captured image P with the highest smile level calculated by the evaluation value calculation unit b1 is selected. That is, the image selection unit 6b selects the captured image P having the highest smile level calculated by the evaluation value calculation unit b1 as the recorded image recorded on the recording medium M.

In addition, the image selection unit 6b evaluates the face of any one of a plurality of specific persons based on the priority registered in the first registration table T1 in association with the face registration flag. A captured image P having a high value is selected. That is, when a plurality of specific persons are registered in the first registration table T1, the image selection unit 6b refers to the priority registered in association with the face registration flag, and specifies a plurality of specific persons. Among the persons, the captured image P having the highest face smile degree is selected for a specific person (for example, “A”) having the highest priority.
For a specific person with the second highest priority (for example, “B”), for example, for a specific person with the highest priority, a plurality of captured images P having a face smile degree of a predetermined value or more are selected. ... Among the plurality of captured images P,..., The captured image P having the highest face smile degree may be selected for a specific person having the second highest priority.

  Further, for a person other than a specific person (for example, “C” and “D”), for example, for a specific person, a plurality of captured images P having a face smile degree of a predetermined value or more are selected. Among the plurality of picked-up images P,..., A picked-up image P having the highest face smile degree may be selected for the person with the highest priority among persons other than the specific person. When a plurality of people other than the specific person are registered in the first registration table T1, the smile level of the face is greater than or equal to a predetermined value for the person with the highest priority among those other than the specific person. A plurality of captured images P are selected, and among the plurality of captured images P,..., The captured image P having the highest face smile degree for a person with the second highest priority among persons other than a specific person. May be selected.

  As described above, the image selection unit 6b has the evaluation value of the face (feature portion) detected by the face detection unit 5 for a specific person with the face registration flag registered among the plurality of captured images P,. Select a high captured image P.

The person registration DB 7 stores a first registration table T1 (see FIG. 2) for registering a face registration flag of a specific person.
Here, the first registration table T1 will be described with reference to FIG.

As shown in FIG. 2, the first registration table T <b> 1 includes names of persons (for example, “A”, “B”, “C”, “D”, etc.) and face images of each person (substance image data). f1 to f4, face registration flags, and priorities are stored in association with each other.
The face registration flag is recording instruction information related to an instruction to record a captured image P having a specific person in the background. That is, the face registration flag preferentially records a captured image P having a specific person in the background among other captured images P,... Is instruction information to be recorded. The face registration flag is registered in association with a plurality of specific persons. Specifically, for example, “A” and “B” face registration flags are set to “ON”, and the person becomes a specific person. Accordingly, preferential recording of the captured image P in which the face images f1 and f2 of the specific person “A” and “B” exist among the plurality of captured images P,. Is done.
For example, the face registration flags “C” and “D” are set to “OFF”, and the person is a person other than the specific person. As a result, preferential recording on the other captured image P of the captured image P in which the face images f3 and f4 of “C” and “D” that are people other than the specific person exist is not instructed.
In addition, although two persons of “A” and “B” are illustrated as specific persons, for example, the number of specific persons is not limited to this, and may be appropriately set if at least one person is set. It can be changed arbitrarily. Similarly, as the person other than the specific person, for example, two people of “C” and “D” are illustrated, but the number of persons other than the specific person is an example and is not limited to this. If one person is set, it can be arbitrarily changed.

The priority represents the priority for each person who records the captured image P on the recording medium M. For example, the priority represents the priority of recording on the recording medium M of the captured image P in which each person exists for all persons registered in the first registration table T1, that is, a specific person and other persons. ing. Specifically, for example, the higher the numerical value, the higher the priority. In FIG. 2, the highest priority is given to the recording of the captured image P in which the face image f1 of “A” exists, and then “B”. The captured image P in which the face image f2 exists, the captured image P in which the face image f4 “D” exists, and the captured image P in which the face image f3 “C” exists exist in this order.
In addition, although what registered the priority about all the persons registered into 1st registration table T1 was illustrated, it is an example and it is not restricted to this, If it is registered about at least a specific person good.

As described above, the person registration DB 7 registers the face registration flag (recording instruction information) related to the recording instruction of the captured image P in which the image data of the face image of a specific person exists.
The face images f1 to f4 of each person in the first registration table T1 may be used for face detection processing and face recognition processing.

The recording control unit 8 is configured so that the recording medium M is detachable, and controls reading of data from the loaded recording medium M and writing of data to the recording medium M.
Specifically, the recording control unit 8 selects the image selected by the image selection unit 6b among the plurality of captured images P1, P2, P3,... A specific person (for example, “A”) having a high facial smile (evaluation value) is selected as a recorded image and recorded on the recording medium M.
As described above, the recording control unit 8 records the captured image P selected by the image selection unit 6b on the recording medium M as a recording image.

  The recording medium M is composed of, for example, a non-volatile memory (flash memory) or the like. However, the recording medium M is an example and is not limited to this, and can be arbitrarily changed as appropriate.

The display control unit 9 performs control for reading display image data temporarily stored in the memory 4 and displaying the read image data on the display unit 10.
Specifically, the display control unit 9 includes a VRAM, a VRAM controller, a digital video encoder, and the like (all not shown). The digital video encoder periodically reads the luminance signal Y and the color difference signals Cb, Cr read from the memory 4 and recorded in the VRAM under the control of the central control unit 12 from the VRAM via the VRAM controller. It reads out, generates a video signal based on these data, and outputs it to the display unit 10.

  The display unit 10 is, for example, a liquid crystal display device, and displays an image captured by the electronic imaging unit 1b based on a video signal from the display control unit 9 on a display screen. Specifically, the display unit 10 updates the live view image while sequentially updating a plurality of image frames generated by imaging the subject by the imaging unit 1 at a predetermined frame rate in the still image capturing mode or the moving image capturing mode. indicate.

  The operation input unit 11 is for performing a predetermined operation of the imaging apparatus 100. Specifically, the operation input unit 11 includes a shutter button related to an imaging instruction of a subject, a selection determination button related to an instruction to select an imaging mode or a function, a zoom button related to an instruction to adjust the zoom amount, etc. A predetermined operation signal is output to the central control unit 12 in response to the operation of these buttons.

  The central control unit 12 controls each unit of the imaging apparatus 100. Specifically, although not shown, the central control unit 12 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like, and includes various processing programs for the imaging apparatus 100 ( Various control operations are performed according to (not shown).

Next, the continuous imaging process by the imaging device 100 will be described with reference to FIGS.
FIG. 3 is a flowchart illustrating an example of an operation related to continuous imaging processing. FIGS. 4A to 4D are diagrams schematically illustrating examples of captured images Pa1 to Pan related to the continuous imaging process.

The continuous imaging process is performed when a continuous imaging mode is selected from a plurality of imaging modes displayed on the menu screen based on a predetermined operation of a selection determination button (not shown) of the operation input unit 11 by the user. It is a process to be executed.
In the continuous imaging process described below, “A”, “B”, “C”, and “D” are imaged as subjects. In addition, “A” and “B” assume that the face registration flag of the first registration table T1 is set to “ON” (see FIG. 2).

As shown in FIG. 3, first, the display control unit 9 displays a live view image on the display screen of the display unit 10 based on a plurality of image frames generated by imaging by the imaging unit 1 and the imaging control unit 2 ( Step S1).
Next, the face detection unit 5 sequentially acquires image frames of a plurality of captured images P1, P2, P3,... Generated by capturing the live view image by the image capturing unit 1 from the memory 4, and from within each image frame. A human face image is detected using a predetermined face detection method (step S2). Subsequently, the face detection unit 5 detects a face image registered with the face registration flag “ON” in the first registration table T1 from the face image of the person detected from each captured image P, that is, as a specific person. It is determined whether or not a registered face image has been detected (step S3).

If it is determined in step S3 that a face image of a specific person has been detected (step S3; YES), the display control unit 9 refers to the first registration table T1 and is associated with this face image. The names of registered persons (for example, “A”, “B”, etc.) are displayed together with the live view image (step S4).
Thereafter, the central control unit 12 determines whether or not an imaging instruction is input based on a predetermined operation of a shutter button (not shown) of the operation input unit 11 by the user (step S5). Also, if it is determined in step S3 that a face image of a specific person has not been detected (step S3; NO), the central control unit 12 moves the process to step S5 and issues an imaging instruction. It is determined whether or not an input has been made (step S5).

If it is determined in step S5 that an imaging instruction has not been input (step S5; NO), the central control unit 12 shifts the process to step S2 and executes the subsequent processes.
On the other hand, if it is determined in step S5 that an imaging instruction has been input (step S5; YES), the imaging control unit 2 determines the focus lens focus position and exposure conditions (shutter speed, aperture, amplification factor, etc.) Conditions such as white balance are adjusted, and the optical image of the subject is continuously captured by the electronic imaging unit 1b at a predetermined imaging frame rate (step S6). Subsequently, the image data generation unit 3 generates YUV data for each of a plurality of (for example, n) captured images Pa1 to Pan sequentially transferred from the electronic imaging unit 1b by continuous imaging (step S7).

  Next, after the image acquisition unit 6a of the image processing unit 6 acquires the YUV data of the generated plurality of captured images Pa1 to Pan, the evaluation value calculation unit b1 of the image selection unit 6b includes the plurality of captured images Pa1 to Pa1. Of the Pans (a plurality) of which the face image of the specific person is detected by the face detection unit 5, the smile degree in the image data of the face image is calculated as an evaluation value (step S8). Specifically, the evaluation value calculation unit b1 specifies, for example, the plurality of captured images Pa1 to Pan in which face images of “A” and “B”, which are specific persons, are detected by the face detection unit 5. The smile level of each person's face is calculated.

  Next, the image selection unit 6b refers to the priority registered in association with the face registration flag in the first registration table T1, and determines the face of the specific person with the highest priority (for example, “A”). A plurality of captured images P having the highest smile level in the image are selected (step S9). Here, in the first embodiment, referring to FIG. 2, the number of people whose face registration flag is set to “ON” in the first registration table T1 is two (that is, a plurality of people). First, a plurality of captured images P having a smile level of an image equal to or greater than a predetermined value are selected as candidates, and the priority is the next among the plurality of captured images P1 to Pan among the selected captured images P. The captured image P having the highest smile level of the face image is selected for a specific person (for example, “B”) that is very high.

For example, in the first captured image Pa1 (see FIG. 4A), the smile degree of the face image “A” is evaluated to be greater than or equal to a predetermined value, but the specific person “B” Is evaluated to be less than a predetermined value. In the second captured image Pa2 (see FIG. 4B) and the nth captured image Pan (see FIG. 4D), “A” and “B” which are specific persons. It is evaluated that both the smile levels of the face images are greater than or equal to a predetermined value. Further, in the third captured image Pa3 (see FIG. 4C), the smile degree of the face image of the specific person “A” is less than a predetermined value, and the specific person “B” It is evaluated that the smile degree of the face image “” is greater than or equal to a predetermined value.
On the other hand, for a person other than a specific person (for example, “C”, “D”, etc.), the captured image P is selected considering only the priority. That is, the third captured image Pa3 is evaluated not to be selected because the smile level of the face image is both equal to or higher than a predetermined value, but the smile level is less than the predetermined value for a specific person.
As described above, in the above case, the smile level of the face “A” that is a specific person and has the highest priority is equal to or higher than a predetermined value, and the smile level of the face “B” that is the specific person is predetermined. The picked-up image Pa2 and the n-th picked-up image Pan that are greater than or equal to the value are selected as candidates, and among these candidates, the smile level of “D” having the highest priority among those other than the specific person The picked-up image Pa2 in which is equal to or greater than a predetermined value is selected by the image selection unit 6b.

Thereafter, the recording control unit 8 records the second captured image Pa2 selected by the image selection unit 6b as a recording image in a predetermined recording area of the recording medium M (step S12).
Thus, the continuous imaging process is finished.

  As described above, according to the imaging device 100 of the first embodiment, the feature portion (face) of a specific person whose face registration flag is registered in the person registration database 7 among a plurality of continuously captured images. ) Is selected, a good facial expression is obtained for at least a specific person (for example, “A”, etc.) among a plurality of persons existing in one captured image, and other persons are selected. The picked-up image P with a relatively good expression can be selected. In addition, since it is not necessary to evaluate the feature (face) for people other than a specific person who does not need to place strict emphasis on smiles (good expressions), it is possible to increase the number of consecutive images. There is no invitation.

[Embodiment 2]
Hereinafter, the imaging apparatus 200 according to the second embodiment will be described with reference to FIGS.
In the second embodiment, a captured image with a high evaluation value of a face (feature part) is selected for each of a plurality of specific persons, and an image region including each specific person is extracted from each of the captured images. Then, the plurality of extracted image areas are synthesized with each other to generate one synthesized image and record it on the recording medium.
Note that the imaging apparatus 200 according to the second embodiment has substantially the same configuration as the imaging apparatus 100 according to the first embodiment except for the points described below, and the description thereof is omitted.

FIG. 5 is a block diagram illustrating a schematic configuration of the imaging apparatus 200 according to the second embodiment to which the present invention is applied.
As illustrated in FIG. 5, in the imaging device 200 according to the second embodiment, a second registration table T2 (see FIG. 6) is stored in the person registration DB 7.

As shown in FIG. 6, the second registration table T2 includes names of persons (for example, “E”, “F”, “G”, etc.), face images (substances are image data) f5 to f7 of each person. The face registration flag is stored in association with each other.
The face registration flag is registered in association with face images f5 to f7 of a plurality of specific persons. Specifically, for example, “E”, “F”, and “G” face registration flags are set to “ON”, and the person becomes a specific person. As a result, among the plurality of captured images P,..., The captured images having the specific persons “E”, “F”, and “G” are preferentially recorded as compared with the other captured images. Is instructed.

  In addition to the image acquisition unit 6a and the image selection unit 6b, the image processing unit 6 includes a region extraction unit 6c and an image generation unit 6d.

For each of a plurality of specific persons whose face registration flags (recording instruction information) are registered in the second registration table T2, the image selection unit 6b receives a plurality of captured images P,... Acquired by the image acquisition unit 6a. Among them, each of the captured images P having a high evaluation value of the face (feature portion) is selected.
That is, the evaluation value calculation unit b1 of the image selection unit 6b calculates the smile level of a face as an evaluation value for a specific person existing in the plurality of captured images P,... Acquired by the image acquisition unit 6a. Note that the method of calculating the evaluation value of the face of a specific person by the evaluation value calculation unit b1 is the same as in the first embodiment, and a detailed description thereof will be omitted.
Then, based on the evaluation result by the evaluation value calculation unit b1, the image selection unit 6b includes a plurality of specific persons whose face registration flag is set to “ON” among the plurality of captured images P,. The picked-up image P with a high face smile degree calculated by the evaluation value calculation unit b1 is selected. For example, the image selection unit 6b selects the n-th captured image Pbn (see FIG. 8D) as the captured image P with the smile degree of the face “E” equal to or greater than a predetermined value, and “F”. The first captured image Pb1 (see FIG. 8A) is selected as the captured image P having a smile level of the face of “G”, and the captured image P having the smile level of “G” is greater than or equal to the predetermined value. As a result, the first captured image Pb1 (see FIG. 8A) and the second captured image Pb2 (see FIG. 8B) are selected.

For each of a plurality of specific persons, the area extracting unit 6c trims and extracts the image area A including the person from each of the captured images P having a high face evaluation value.
That is, for each of a plurality of specific persons, the area extraction unit 6c selects an image area A including each person from each of the captured images P having a high face evaluation value selected by the image selection unit 6b. Then, the image area A is trimmed and extracted. Specifically, for example, the region extraction unit 6c overlaps each other based on the coordinates of the center of the face (feature part) of each specific person (for example, “E”, “F”, “G”, etc.). A range to be cut out is set so as not to be extracted, and the set range (trimming region) is extracted. If each specific person overlaps in the vertical direction or the horizontal direction, the region extraction unit 6c uses the coordinates of the center of the face (characteristic part) as a reference to determine the intermediate position between the specific persons. A range (trimming region) to be cut out so as to be divided at the intermediate position may be extracted.
For example, in the case of “E”, the region extraction unit 6c includes “E” from the n-th captured image Pbn specified by the image selection unit 6b as the captured image P whose face smile degree is a predetermined value or more. The first image area A1 (see FIG. 9A) is trimmed and extracted. For example, in the case of “F”, the region extraction unit 6c determines “F” from the first captured image Pb1 that is specified as the captured image P having a face smile degree of a predetermined value or more by the image selection unit 6b. Is extracted by trimming the second image area A2 (see FIG. 9B). For example, in the case of “G”, the region extraction unit 6c captures the first captured image Pb1 or the second captured image specified by the image selection unit 6b as the captured image P having a face smile degree of a predetermined value or more. A third image region A3 (see FIG. 9C) including “G” is trimmed and extracted from the image Pb2.

Note that the trimming method of the image area A including the specific person by the area extracting unit 6c is an example, and is not limited to this, and can be arbitrarily changed as appropriate. For example, the image area A is trimmed for each person. For example, as shown in “F” and “G”, the first captured image Pb1 is a captured image P whose face smile degree is a predetermined value or more. May be extracted by trimming one image area A including both “F” and “G”.
As described above, the region extraction unit 6c includes an image including the person from each of the captured images P having a high evaluation value of the face (characteristic part) for each of the plurality of specific persons selected by the image selection unit 6b. Each area A is trimmed and extracted.

The image generation unit 6d generates a single composite image G by combining a plurality of image areas A,... Extracted for each person.
Specifically, for example, the image generation unit 6d includes a first image region A1 (see FIG. 9A) including “E” extracted by the region extraction unit 6c and a second image including “F”. The image area A2 (see FIG. 9B) and the third image area A3 including “G” (see FIG. 9C) are combined to form one composite image G (see FIG. 9D). ) Is generated.
In this way, the image generation unit 6d generates a single composite image G by combining a plurality of image regions A,... Extracted for each person by the region extraction unit 6c.

  The recording control unit 8 records one composite image G (see FIG. 9D) generated by the image generation unit 6d on the recording medium M as a recording image.

Next, continuous imaging processing by the imaging apparatus 200 will be described with reference to FIGS.
FIG. 7 is a flowchart illustrating an example of an operation related to continuous imaging processing. However, since the processing flow is the same as that in step S8 of FIG. 3 in the first embodiment, only the differences from the first embodiment will be described in the drawings, and the description will be omitted.
Fig.8 (a)-FIG.8 (d) are figures which show typically an example of the captured images Pb1-Pbn which concern on a continuous imaging process. FIGS. 9A to 9C schematically show the first to third image areas A1 to A3 extracted for a specific person, and FIG. 9D shows a synthesized composite image. It is a figure which shows G typically.
In the continuous imaging process described below, “E”, “F”, and “G” are specific persons, and the face registration flag of the second registration table T2 is set to “ON”. (Refer to FIG. 6), a person other than a specific person crossing in front of “E”, “F”, and “G” is simply represented as a passerby J.

When the smile level of the face image of a specific person is calculated as an evaluation value by the face detection unit 5 in step S8, the image selection unit 6b is configured for each of the plurality of specific persons in the plurality of captured images P,. A captured image P having a high smile level of the face calculated by the evaluation value calculation unit b1 is selected (step S21).
For example, in FIGS. 8A to 8D, when “E”, “F”, and “G” are imaged as subjects, a passerby J in front of “E”, “F”, and “G” is used. When “E” is a specific person, the first to third captured images Pb1 to Pb3 of the specific person “E” have a face smile level less than a predetermined value. Similarly, for “F”, which is a specific person, the second to n-th captured images Pb2 to Pbn have facial smiles less than a predetermined value. Similarly, for the specific person “G”, the third captured image Pb3 and the n-th captured image Pbn have face smiles less than a predetermined value.
Therefore, the image selection unit 6b selects the n-th captured image Pbn (see FIG. 8D) as the captured image with the smile degree of the face “E” being a predetermined value or more, and also the face “F”. The first captured image Pb1 (see FIG. 8A) is selected as a captured image having a smile level of “G” greater than or equal to a predetermined value, and the first image is selected as a captured image having a smile level of “G”. The captured image Pb1 (see FIG. 8A) and the second captured image Pb2 (see FIG. 8B) are selected.

Subsequently, the area extraction unit 6c selects, for each of a plurality of specific persons, an image area A including each person from each of the captured images selected by the image selection unit 6b and having a high degree of smile on the face. Then, each image area A is trimmed and extracted (step S22).
For example, the region extraction unit 6c calculates “E” from the n-th captured image Pbn selected as the captured image P having a face smile degree of a predetermined value or more by the image selection unit 6b for the specific person “E”. The first image area A1 including “” (see FIG. 9A) is selected and the image area A1 is trimmed. In addition, for example, for the specific person “F”, the region extraction unit 6c uses, from the first captured image Pb1 selected as the captured image P having a face smile degree of a predetermined value or more by the image selection unit 6b. The second image area A2 (see FIG. 9B) including “F” is selected and the image area A2 is trimmed. Further, for example, for the specific person “G”, the region extraction unit 6c selects the first captured image Pb1 or 2 selected as the captured image P having a face smile degree of a predetermined value or more by the image selection unit 6b. A third image area A3 (see FIG. 9C) including “G” is selected from the first captured image Pb2, and the image area A3 is trimmed.

Next, the image generation unit 6d generates a single combined image G by combining the plurality of image regions A extracted by the region extraction unit 6c (step S23). Specifically, the image generation unit 6d includes a first image area A1 including “Mr. E” (see FIG. 9A) and a second image area A2 including “F” (FIG. 9B). Reference) and the third image region A3 (see FIG. 9C) including “G” are combined to generate one composite image G (see FIG. 9D).
Thereafter, the recording control unit 8 records the single composite image G generated by the image generation unit 6d as a recording image in a predetermined recording area of the recording medium M (step S24).
Thus, the continuous imaging process is finished.

  As described above, according to the imaging apparatus 200 of the second embodiment, a captured image having a high face (characteristic) evaluation value is selected for each of a plurality of specific persons, and each specific image is selected from each of the captured images. Each image area including a person is trimmed and extracted, and a plurality of extracted image areas are combined to generate a single composite image. Therefore, an image with a high degree of satisfaction is obtained for each of a plurality of specific persons. It can be recorded on the recording medium M as a recorded image.

The present invention is not limited to the above-described embodiment, and various improvements and design changes may be made without departing from the spirit of the present invention.
For example, among the plurality of captured images P,... Captured by the continuous imaging process, the captured image P in which the face of a specific person is not detected may be excluded from the processing target of the image processing unit 6.

That is, as illustrated in FIG. 10, the imaging apparatus 100 according to the first modification has registered a face registration flag (recording instruction information) among a plurality of captured images P,. The image processing unit 6 includes a non-detected image designating unit 6 e that designates a captured image P in which a face of a specific person has not been detected by the face detecting unit 5. And the image acquisition part 6a does not acquire the captured image P from which the face of the specific person designated by the non-detection image designation | designated part 6e was not detected among the some captured images P ....
Note that the imaging apparatus 100 of the first modification has substantially the same configuration as that of the imaging apparatus 100 of the first embodiment except for the points described above, and a description thereof will be omitted. Moreover, although the said modification 1 was illustrated as a modification of the imaging device 100 of Embodiment 1, it is good also as a modification of the imaging device 200 of Embodiment 2. FIG.

  Therefore, according to the imaging apparatus 100 of the first modification, the captured image P in which the face of a specific person whose face registration flag is registered is not detected among the plurality of captured images P,. Since it is not acquired as a target, it is possible to reduce the load on the image processing and improve the processing speed.

Moreover, in the said modification 1, among the some captured images P, ..., the captured image in which the person (for example, passerby J etc.) to whom the face registration flag is not registered overlapped before the specific person. P may be specified so that the captured image P is not acquired by the image acquisition unit 6a. That is, for a captured image P in which a specific person with a registered face registration flag is not detected by the face detecting unit 5 and a person with no registered face registration flag is detected with the face detecting unit 5, image acquisition is performed. The unit 6a may be excluded from the processing target of the image processing unit 6.
Even with such a configuration, similar to the first modification, it is possible to reduce the load on the image processing and improve the processing speed.

  In the first and second embodiments and the first modification, the face is exemplified as the human characteristic portion. However, the face is an example, and the present invention is not limited thereto, and can be arbitrarily changed as appropriate. For example, a part of the face (for example, eyes and mouth) or the whole human body may be evaluated as the characteristic part.

  In addition, the configurations of the imaging devices 100 and 200 are merely examples illustrated in the first and second embodiments, and are not limited thereto. For example, the captured image P is generated by an external device (not shown) that is different from the imaging devices 100 and 200, and is transmitted from the external device via a predetermined communication line. The captured image P having a high evaluation value of the characteristic part (for example, face) may be selected and recorded.

In addition, in the above embodiment, the functions of the acquisition unit, the detection unit, the calculation unit, and the selection unit are driven by the face detection unit 5 and the image processing unit 6 under the control of the central control unit 12. However, the present invention is not limited to this, and may be realized by executing a predetermined program or the like by the central control unit 12.
That is, a program memory (not shown) that stores a program stores a program including an acquisition process routine, a detection process routine, a calculation process routine, and a selection process routine. Then, the CPU of the central control unit 12 may function as an acquisition unit that acquires by continuously capturing a plurality of images including a plurality of persons including a specific person by an acquisition process routine. In addition, the CPU of the central control unit 12 may be caused to function as a detection unit that detects each of the feature portions based on the priority stored in the memory in the plurality of images acquired by the acquisition unit by the detection processing routine. good. Further, the CPU of the central control unit 12 is caused to function as a calculation unit that evaluates the feature portions detected by the detection unit based on the feature units stored in the memory and calculates an evaluation value by the calculation processing routine. You may do it. In addition, the CPU of the central control unit 12 may function as a selection unit that selects a captured image to be recorded based on the evaluation value calculated by the calculation unit by a selection processing routine.

  Furthermore, as a computer-readable medium storing a program for executing each of the above processes, a non-volatile memory such as a flash memory or a portable recording medium such as a CD-ROM is applied in addition to a ROM or a hard disk. Is also possible. A carrier wave is also used as a medium for providing program data via a predetermined communication line.

100, 200 Imaging device 1 Imaging unit 2 Imaging control unit 5 Face detection unit 6 Image processing unit 6a Image acquisition unit 6b Image selection unit b1 Evaluation value calculation unit 6c Area extraction unit 6d Image generation unit 6e Non-detection image designation unit 7 Person registration Database 8 Recording control unit 12 Central control unit M Recording medium

Claims (4)

An acquisition means for acquiring a plurality of images including a specific person by continuously capturing a plurality of images;
A feature storage unit that stores a plurality of people in association with the priority of the feature of the specific person;
Detecting means for detecting each of the feature parts based on the priority stored in the feature part storage means in the plurality of images obtained by the obtaining means;
Calculating means for evaluating each of the characteristic portions detected by the detecting means based on the characteristic portions stored in the storage means and calculating an evaluation value;
Selection means for selecting a captured image to be recorded based on the evaluation value calculated by the calculation means;
An image processing apparatus comprising: The calculation means calculates an evaluation value for each of a plurality of persons stored in the feature storage means,
The selection means includes
Extraction means for extracting an image region including a person from each captured image based on the calculated evaluation value;
Image synthesizing means for synthesizing the image areas extracted by the extracting means;
The image processing apparatus according to claim 1, wherein a captured image to be recorded is selected. An acquisition step of acquiring a plurality of images including a specific person by continuously capturing a plurality of images;
The priority is read from a memory in which a feature portion of the specific person is associated with a priority in advance and stored for a plurality of people, and the feature portion is based on the priority in the plurality of images acquired in the acquisition step. Detecting step for detecting each of
A calculation step for evaluating each of the feature portions detected in the detection step based on the feature portions stored in the memory and calculating an evaluation value;
A selection step for selecting a captured image to be recorded based on the evaluation value calculated in the calculation step;
An image selection method comprising: A computer of an image processing apparatus including a memory that stores a plurality of persons in association with a priority, and a feature portion of the specific person in advance.
An acquisition means for acquiring a plurality of images including a specific person by continuously capturing a plurality of images;
In the plurality of images acquired by the acquisition means, detection means for detecting the characteristic portions based on the priorities stored in the memory,
Calculating means for evaluating each of the characteristic portions detected by the detecting means based on the characteristic portions stored in the memory and calculating an evaluation value;
Selection means for selecting a captured image to be recorded based on the evaluation value calculated by the calculation means;
A program characterized by functioning as

Images