JP2012015742A - Reproduction device and recording device - Google Patents

Abstract

An imaging device for a consumer enables high-speed processing and low memory consumption, so that a representative image with high visibility can be extracted.
Face recognition information acquisition means for acquiring face recognition information representing an area where a face portion corresponding to a subject of image data exists, and focus for acquiring focus information representing an area in which the image data is in focus An information acquisition unit, a region match detection unit that detects a match between the face recognition region acquired by the face recognition information acquisition unit, and the focus region acquired by the focus information acquisition unit, and the region match detection unit match Then, a control means is provided for controlling to display the determined image data on the display device as a reproduced image.
[Selection] Figure 2

Description

  The present invention relates to a reproducing apparatus and a recording apparatus, and more particularly to a technique for automatically selecting a specific frame image from a moving image or a plurality of still image data recorded on a recording medium.

  Conventionally, a method of performing face detection processing on image data recorded by an imaging device such as a digital camera or a video camera, registering the detected face image as a thumbnail representative image indicating an outline of the content, and displaying and outputting the image (Patent Document) 1-4) are disclosed.

In the method disclosed in Patent Document 1, a face image obtained by face recognition is extracted from a frame image in a moving image, and a feature amount is calculated from the face image. Then, the same person is determined based on the calculated feature amount, and the image having the highest appearance frequency is determined as the representative image.
In the method disclosed in Patent Literature 2, after a representative image candidate is determined from a predetermined section of a moving image, face recognition is performed by comparing the representative image with a preset face template, and based on the recognition result. Search for and output a representative image.
The method disclosed in Patent Document 3 performs person recognition from a frame image in a moving image, and extracts a region including the person as a partial image. Then, the extracted partial image, the source frame image, and the address information on the recording medium are recorded in association with each other. At the time of reproduction, the above-mentioned partial image is used as scene information to search for an arbitrary scene in the moving image.
The method disclosed in Patent Literature 4 calculates the presence / absence of a face region from a frame image in a moving image, and calculates a similarity with a face of a specific person image set in advance. Then, based on the calculated similarity determination result, the moving image is reproduced centering on the specific person.

JP 2007-174378 A JP 2006-331271 A JP 2004-297305 A JP 2005-018451 A

  Patent Documents 1 to 4 each provide a representative image extraction method by face recognition from a moving image for the purpose of improving searchability during reproduction. However, in Patent Documents 1 to 4, in the face recognition determination when the subject moves, subject blurring or blurring that does not match the desired area may occur. The face image may be determined.

In particular, when shooting with a consumer-oriented digital camera or video camera, the above problem occurs remarkably regardless of still images and moving images. For this reason, the conventional techniques proposed in Patent Documents 1 to 4 have a problem that high-definition representative images with high visibility cannot always be obtained. In particular, when a consumer imaging device is used, there is a problem that the processing time required for the determination process becomes long, a problem that a large amount of memory is consumed, and a problem that a high-definition representative image with high visibility cannot be obtained. There was a point.
In view of the above-described problems, an object of the present invention is to enable processing speed and memory consumption to be reduced, and to extract a representative image with high visibility.

  The reproduction apparatus according to the present invention is a reproduction apparatus that reproduces image data and displays the image data on a display device, face recognition information acquisition means for acquiring face recognition information representing an area where a face portion corresponding to the subject of the image data exists. A focus information acquisition unit that acquires focus information representing a region in focus of the image data, a face recognition region acquired by the face recognition information acquisition unit, and a focus region acquired by the focus information acquisition unit A region coincidence detecting unit for detecting coincidence with the control unit, and a control unit for controlling the display device to display the image data determined to be coincident by the region coincidence detecting unit as a reproduced image.

  According to the present invention, it is possible to increase the processing speed and reduce the amount of memory, and it is possible to extract only representative images with high visibility from a large number of recorded image data. Therefore, it becomes easy for the user to grasp intuitive contents, and the searchability can be improved.

It is a block diagram which shows the schematic structural example of the reproducing | regenerating apparatus of embodiment. It is a figure explaining the extraction process in which a face image area | region and a focus area correspond. It is a figure explaining the extraction process in which a face image area | region and a focus area correspond. It is a figure explaining the extraction process in which a face image area | region and a focus area correspond. It is a figure explaining an example of the index screen displayed on LCD. It is a block diagram explaining the system configuration | structure which connected the external connection apparatus. It is a figure explaining the narrowing-down process when two or more coincidence images are detected. It is a figure explaining the process which produces | generates face image recognition information as coordinate information. It is a figure explaining a mode that face image recognition information is processed as bitmap information. It is a flowchart explaining the extraction processing procedure of the reproduction image in which an area | region corresponds. It is a flowchart explaining the extraction process in case a some face image exists. It is a flowchart explaining the extraction process in case a some face image exists. It is a figure explaining the process which determines a match with a face image area | region and a focus area | region.

Next, the best mode for carrying out the invention will be described in detail with reference to the drawings.
"1. System configuration example of this embodiment"
FIG. 1 shows a schematic block diagram of a playback apparatus taking as an example a video camera system including means for detecting a coincidence image of a face recognition area and a focus area according to the present embodiment.
Hereinafter, the function of each block of the imaging reproduction system according to the present embodiment will be described.

The optical unit 101 includes a lens, an optical element, and a driving mechanism for the optical element. For example, a lens, an IRIS motor, an AF motor, and a flash (all not shown) are provided to form an optical image.
The imaging unit 102 includes an image sensor such as a CCD and a CMOS, an AGC, an AD converter, and the like. The imaging unit 102 converts an optical image formed by the optical unit 101 into an electrical signal and A / D converts the captured image data. Generate. In addition, in the optical unit 101 and the imaging unit 102, focus position information indicating where the captured image data is in focus and the focused area in the screen is obtained from the drive parameters used for focus control. Generate.

  Further, the image of which the focus position information is information-compressed by an encoding process, which will be described later, as EXIF information in still image data and as camera information metadata including focus position information in a moving image data using a known method. Embed in data.

The face image recognition unit 103 performs face image detection determination from the input frame image.
In the present embodiment, it is assumed that the above-described face detection determination is performed on a captured frame image at the time of capturing and recording, and the face image information is stored as metadata in the image data by a recording method described later. However, it may be a procedure of creating face image information after performing the aforementioned face detection determination on the decoded image frame at the time of reproduction read from the recording medium. For image data that does not have the face image information as metadata in advance Also, this embodiment can be applied. Note that the face image detection determination algorithm is detected by applying known techniques such as edge detection in the screen, specific hue detection such as skin color, and face shape pattern matching.

  The encoding unit 104 subjects the image data input from the imaging unit 102 to video compression that reduces spatial redundancy and temporal redundancy by, for example, DCT conversion and motion prediction such as MPEG. .

  The recording / reproducing unit 105 adds an error correction parity code such as ECC to the moving image data compressed by the encoding unit 104, and performs recording / writing on a large-capacity recording medium such as the magnetic disk 106 or the optical disk 107. Further, the frame currently being shot is displayed and output on the screen of the display output unit 109 such as an LCD panel.

Subsequently, a procedure for reproducing the moving image data recorded on the recording medium by each means described above will be described.
The recording / reproducing unit 105 reads the compressed moving image data written on the recording medium such as the magnetic disk 106 or the optical disk 107, and performs error correction using the parity code added at the time of recording. Then, the corrected moving image data is output to the decoding unit 108, and the decoding unit 108 decodes it as the same image data as input.

  The data decoded by the decoding unit 108 is output to the display output unit 109. Further, in order to control each block constituting the above-described imaging system, there is a control microcomputer 110 having a built-in program, which is performed via the processing instruction of the block and the system bus 111 of the data flow.

The system bus 111 performs data transfer between processing blocks and transmission / reception of control commands.
The semiconductor memory 112 accumulates data necessary for processing in each system block.
In addition, operations such as recording and reproduction requests from the user or power-on / stop are notified to the control microcomputer 110 via the external control 113, and the control microcomputer 110 is triggered by the notification content as a trigger. Control overall operation.

The external input / output interface 114 is connected by a wired cable such as USB or IEEE1394, or a wireless LAN such as Bluetooth, and transmits / receives image data in the imaging apparatus between external devices or networks.
The externally connected device 115 is connected to the imaging apparatus of the present embodiment, and is a device such as a television that displays and outputs image data and a printer that prints image data.
The above is the basic system configuration of the playback apparatus of this embodiment.

“2. Face image and focus area matching image extraction process”
Next, a method for extracting an image frame in which the face image area and the focus area match according to the present embodiment will be described with reference to FIGS. 2, 3, 4, and 10.
FIG. 2 is a diagram in which data of each scene is read from image data stored in a recording medium such as the optical disc 107, face image detection and focus detection are performed, and only matching images are extracted.

  As shown in Fig. 2, when moving image data with CLIP001.MOV and CLIP002.MOV file names is recorded as recorded data on the recording medium, CLIP002.MOV is read, and the playback image whose face image matches the focus area Shall be extracted.

  Assume that the image data CLIP002.MOV read from the recording medium has five representative scenes SCN01 to SCN05 as shown in FIG. The representative scene may be, for example, a first frame image at the start of recording by the imaging apparatus, or a frame image randomly extracted at intervals of several seconds. Of course, all frame images may be used.

  For each representative scene, the decoding process described with reference to FIG. 1 is performed, and both the focus area information recorded in the metadata format in the image data and the face recognition information created by a method described later are acquired. Focus information acquisition and face recognition information acquisition are performed. In the present embodiment, it is desirable that the focus information described in the metadata has an area in focus on the captured screen, a so-called in-focus area as a coordinate value. Alternatively, it is only necessary to have camera information (for example, focal length, angle of view, depth of field, etc.) that can derive area information having the same meaning as the in-focus area.

  In the example in FIG. 2, for the sake of convenience, it is easy to explain that each area information is different, the focus area detected in the representative scene is represented by a solid line with a thick edge portion, and the face recognition area is a square surrounded by a broken line. Respectively.

  In the present embodiment, in order to detect the degree of overlap between the face detection area and the focus area for each of the above-described representative scenes, an area coincidence degree detection process for detecting a ratio ratio of coincidence areas is performed. Then, when the detection result is equal to or greater than a predetermined focus area threshold value, it is determined that the images match.

Expressing the above judgment method as a mathematical expression, when the area of the focus area is Sf [pixel ² ] and the area of the overlap area of the face recognition area and the focus area is So [pixel ² ], the threshold value as a weighting factor is 80% Then,
So [pixel ² ] ≥ Sf [pixel ² ] x 0.8 (Formula 1)
A reproduced image having a value of So is determined to be a matching image.

FIG. 13 is a diagram for explaining in detail a specific example of the above-described coincidence determination method using overlapping regions.
FIG. 13A illustrates a reproduced image to be a certain match determination target, and FIG. 13B illustrates coordinate values as position information of the focus area and the face recognition area in the determination target reproduced image.

  It is assumed that the reproduced image to be determined is image data of 1920 pixels in the horizontal direction and 1080 lines of vertical lines. Then, with the focus area and face recognition area in the image data described above as the origin at the upper left vertex, the upper left and lower right diagonal components in each area are acquired when reading the reproduction data.

Since each coordinate value is As (100, 100), Ae (500, 500), Bs (200, 200), Be (550, 600), the area Sf of the focus area is
Sf = (550-200) * (600-200) = 350 * 400 = 14 * 10 ⁴ [pixel ² ]
The area So of the overlapping area of the face recognition area and the focus area indicated by the shaded area in FIG.
So = (500-200) * (500-200) = 300 * 300 = 9 * 10 ⁴ [pixel ² ]
It becomes.

Next, if the weighting coefficient for matching is 80%,
9 * 10 ⁴ [pixel ² ] <14 * 10 ⁴ * 0.8 = 11.2 * 10 ⁴ [pixel ² ]
Therefore, Formula (1) is not satisfied.
Therefore, it is determined that the reproduced image in FIG. 13 is not a coincident image.

  In the example in FIG. 2, SCN01, SCN03, and SCN05 are determined to be images in which the face recognition and the focus area are the same among the five representative scenes as the above-described determination results. It is possible to flexibly change the number of matching images to be extracted depending on what value is set as the threshold value for the above-mentioned matching determination, and whether to narrow down to a high-definition representative image or to select many candidate images. You can choose adaptively whether you can choose.

Depending on the scene to be photographed, not only the subject as shown in FIG. 2 but also cases as shown in FIGS. 3 and 4 are assumed.
FIG. 3A shows an example of a reproduced image in a case where a plurality of face recognition areas indicated by broken lines exist within a focus area indicated by solid lines at the time of pan-focus shooting or the like. In the example of FIG. 3A, five face recognition areas surrounded by a broken line are included in the focus area surrounded by a solid line.

  For the purpose of the present invention, it is desirable that such a reproduced image is determined as a coincidence image. However, the determination process according to (Equation 1) described above cannot be determined as a coincidence image. For this reason, in the present embodiment, in order to prevent detection failure even in such a case, even when all of the face recognition areas are included in the focus area, it is determined that they are coincident images.

  Specifically, a method of determining that the face recognition area is entirely included in the focus area is used. This determination method calculates from the coordinate information of the face recognition area and the coordinate information of the focus area whether or not the coordinate value of the face recognition area similarly exists within the coordinates of the focus area surrounded by the four vertices. Judge by.

  FIG. 3B shows a focus area and a face recognition area corresponding to FIG. 3A expressed as coordinate values. In the coordinate values, the upper left vertex of the rectangular area is the start point, and the lower right vertex is the end point. For example, it is assumed that a reproduced image having a horizontal resolution of 1280 pixels and a vertical line number of 960 lines has a value as shown in the legend of FIG.

From FIG. 3B, the focus area of the above-described reproduced image corresponds to a rectangular area in which FocusStart (10, 10) to FocusEnd (1270, 950) are connected as a diagonal line.
Similarly, for the face recognition area, from Face1Start (20, 10) to Face1End (300, 310) for each detected number of people. From Face2Start (320, 20) to Face2End (620, 320). From Face3Start (700, 30) to Face3End (1100, 400). From Face4Start (200, 310) to Face4End (350, 460). It is expressed as a rectangular area connected by diagonal lines from Face5Start (600, 300) to Face5End (760, 460).

It is determined whether the horizontal direction component and the vertical direction component value of each coordinate of the face recognition area are within the coordinate values of the focus area start point FocusStart and the focus area end point FocusEnd.
That is, when the start point or end point coordinate value of a face recognition area is FaceX (a, b), the following expressions (2) and (3) are satisfied if both the start point and end point of the face recognition area satisfy It is determined that the face detection area is included in the focus area.

Horizontal component value of FocusStart ≦ a ≦ Horizontal component value of FocusEnd (Equation 2)
Vertical component value of FocusStart ≦ b ≦ Vertical component value of FocusEnd (Equation 3)
In the above example, since there are face detection areas for five people, the above expression is all compared from the face recognition area coordinates Face1Start to Face5End for five people to determine whether the face detection area is included in the focus area. There is a need to.

As a judgment result by the above-mentioned formula,
Horizontal direction 10 ≤ 20/300/320/620/700/1100/200/350/600/760 ≤ 1270
Vertical 10 ≤ 10/310/20/320/30/400/310/460/300/460 ≤ 950
Therefore, it is determined as a matching image.

On the other hand, in the case of FIG. 4, when the depth of field is shallow and a person or the like is photographed up, the focus is focused on only a part of the face (only the nose part in FIG. 4).
When the focus area is included in the face recognition area as described above, the determination result of (Equation 1) is a coincidence image, but a blur image that cannot be identified depending on the size of the focus area and the focus position may be extracted. There is sex.

  However, there is also image data as a photographing effect such as soft focus. Therefore, in actuality, the pre-processing of the matching image data detection process of the face recognition area and the focus area according to (Equation 1) is performed in consideration of the setting by the user or the lens performance. For example, the minimum area of the focus area that occupies the reproduced image is derived in advance, and it is determined whether or not it is applicable. Accordingly, it is possible to prevent a detection omission that occurs when a person is photographed up.

In FIG. 10, the above-described matching image extraction process is expressed as a flowchart.
In S101, it is determined whether the extraction process has been completed up to the final frame of the target moving image data. If the result of this determination is that it is complete, the matching image extraction process is terminated.

  On the other hand, if the result of determination in S101 is that there are still frame images to be searched for matching images, the process proceeds to S102, where data reading from the recording medium described in FIG. 1 is performed, and compressed image data is decoded. . In step S103, the focus area information of the image data to be searched is acquired from the metadata in the image data or the management information file. In step S104, face recognition area information obtained by the face image recognition unit 103 is acquired from the image data.

  Next, in S105, the degree of coincidence between the focus area and the face recognition area is derived as a determination process based on the above-described equation (1) based on the two area information acquired from the above-described steps S103 and S104.

Next, in step S106, a determination is made as to whether or not the determination result in S105 is equal to or greater than a predetermined threshold value. If the result of this determination is that it is greater than or equal to the threshold value, it is determined as a matching image, and the process proceeds to S107 to display and output the reproduced image. On the other hand, if the result of determination in S106 is smaller than the threshold value, the next image frame extraction process is continued.
The flow of the matching image extraction process between the focus area and the face recognition area has been described above.

"3. Embodiments applying the present invention"
Next, an embodiment in which the face image and focus area matching image frame extraction processing according to the present invention is applied will be described with reference to FIGS. 5 and 6.
FIG. 5 shows an appearance of a disc camcorder on which a moving image is recorded and reproduced according to this embodiment, and an embodiment of an index screen displayed on the LCD of the disc camcorder.
As shown in FIG. 5, an outline of the content in the recording medium is visually expressed by configuring and outputting a menu screen in which a plurality of thumbnail images obtained by reducing the image data of the extracted matching images are arranged. It is possible to display and output as a thumbnail index. Thereby, it is not necessary to develop and hold all detected matching images on the memory, and moreover, it is possible to efficiently browse a plurality of image data on one screen.

  In the example in FIG. 5, seven thumbnail images from THM001 to THM007 are displayed, but up to nine images can be displayed on one screen. When there are more than nine thumbnail images, another thumbnail index screen that can be jumped by a feed button or the like may be created over a plurality of pages. However, the maximum number of thumbnails that can be displayed on one screen may be determined from the viewpoint of the resolution that does not hinder the visibility of the user, the size of the display device, and the thumbnail image data capacity that can be held in the memory. The number is not limited to nine.

  Furthermore, the user can select a desired thumbnail image from the thumbnail menu, and can quickly reproduce the moving image content corresponding to the thumbnail image without causing a waiting time. A method for reducing the extracted matching image may be a method using a known technique such as a bicubic method.

  In order to enable the above-described random access, simultaneously with the generation of the thumbnail images, each thumbnail image and the recording position on the moving image data where the matching image as the reduction source exists are associated and held. For example, the file name on the recording medium and the time position at which the coincident image is detected are stored in association with the logical address of the system memory in the table format in FIG.

  In the case of FIG. 5, three video files, clip001.mov, clip002.mov, and clip003.mov, have already been recorded. Thumbnail image names THM001 to THM007 of matching images in the focus area and face recognition area detected in each moving image file are associated with the recording time. By creating a random access table in which the matching image thumbnail and the recording position are associated, the user selects a desired thumbnail image, refers to the corresponding recording time of the corresponding moving image file, and starts from the desired position. Playback can be realized.

  FIG. 6 shows an embodiment in which an imaging device such as a video camera is connected to a printer or the like via the external interface of FIG. 1 and a matching image is displayed and output on the GUI of the printer as a recommended image in the video print mode. When a disk drive for an optical disk is mounted on the printer, the same function can be realized by taking out the optical disk from the imaging apparatus and loading it in the drive.

  In FIG. 6, as in the embodiment described with reference to FIG. 5, the coincidence image of the focus image and the face recognition area according to the present embodiment is acquired as the playback function of the printer body. The reduced thumbnail images are output side by side on an operation screen or display screen by a display device such as an LCD, and notified to the user as a recommended print image. In this embodiment, in addition to the thumbnail image, a window screen having a user interface for print settings in which the number of copies, print size, print execution, and a print candidate image feed button are arranged is output as the operation screen. To do. This saves the user from having to search for a frame image suitable for printing.

  The user operates the operation window with an input unit such as a button key, a touch panel, or a jog dial provided in the printer body, thereby selecting a candidate by displaying a corresponding thumbnail image or printing out the matching image. . In the example of the print selection screen in FIG. 6, thumbnail images of three matching images PRNT001 to PRNT003 are output as print candidates.

In this print selection screen, the currently selected thumbnail image PRNT002 has a thick border so that the user can intuitively understand which image is being selected. There is no limitation on how the image is highlighted.
Further, by displaying the previous candidate PRNT001 and the subsequent candidate PRNT003 side by side on the left and right, when a plurality of matching images having similar compositions are detected, the user can easily compare which one to print.
Further, as shown in FIGS. 5 and 6, the focus area and face recognition area information of the coincidence image detected by the coincidence image detection of FIG. 2 is drawn on the corresponding thumbnail image by OSD (On Screen Display). The user can be identified on a small display screen. Alternatively, a display mode of focus information and face recognition information may be optionally provided as a playback function. Thereby, even on a small screen with a low resolution, the user can intuitively understand where the face image exists and where the focus is focused. Therefore, it is possible to prevent a user operation failure or a redo operation.

  In this way, the focus area and the face detection area are output as the OSD function, so even if it is difficult to determine from which the head image is to be played back or printed from only a thumbnail image that is a small screen, the user can Judgment becomes easier.

In the case of the example in FIG. 5, whether the user wants to reproduce from the scene of the person who is focused on THM001 or the scene of the person who is focused on THM002 or THM003, the user intuitively only from the matching images It may be difficult to judge.
Thus, in the present embodiment, the thumbnail image and the focus area corresponding to the thumbnail image and the face recognition area are drawn in an overlapping manner, thereby further supporting an auxiliary function for improving searchability. This effect can be obtained similarly in the embodiment of FIG.

  Note that, as in the example illustrated in the present embodiment, the method of displaying the focus area by edge enhancement can be realized by various methods. For example, the image data in the focus area can be realized by outputting an image in which edge detection is performed using a known technique such as a differentiation method, a Laplacian method, a Sobel filter, or a Canny filter.

“4. Narrowing process when multiple matching images exist”
Next, a narrowing process when a plurality of matching image frames of the face image and the focus area are detected within a predetermined reproduction section according to the present embodiment will be described with reference to FIGS. 7, 11, and 12.
If the above-described face image recognition and focus area coincidence image detection are performed on moving image data, it is conceivable that coincident images are continuously detected in the case of a still picture.
In addition, in moving image data in which information compression is performed using motion prediction and DCT (discrete cosine transform), such as MPEG and H.264, a P picture obtained by encoding a picture in moving image data by performing inter-frame prediction It is known that a frame image obtained by performing intra-screen coding called an I picture has the highest definition and higher visibility than a B picture. Therefore, an encoding method determination process for determining a method for encoding a picture in moving image data is performed.

  In the present embodiment, using the above-described properties, a predetermined reproduction section for detecting the coincidence image shown in FIG. 7 is provided, and when a plurality of coincidence images are detected within this reproduction section, the I picture is preferentially used. Are selected as matching images. Note that the representative image detection / reproduction section is not particularly specified, but is preferably set to about 1 GOP (15 frame periods for NTSC video signals and 12 frame periods for PAL video signals).

  In the example of FIG. 7, it is assumed that there are eight reproduced images in the representative scene detection section. Subsequently, in the reproduced image, three frames from candidates 1 to 3 are detected as frame images in which the face recognition area and the focus area according to the present embodiment match.

  In the above-mentioned “2. Match image extraction process of face image and focus area” and the flowchart of FIG. 10, all reproduced images are displayed and output at this stage. On the other hand, in this method, an intra-frame encoded image detection process, a forward prediction encoded image detection process, and an intra macroblock number determination process are further performed as a narrowing process for narrowing the display output image. Then, the I picture in the encoded image data is displayed and output with the first priority. If an I picture is not found, the P picture is given the second priority. If neither an I picture nor a P picture can be detected in the candidate image, the most intra-played image among the candidate images is displayed. Frames with a large number of macroblocks are displayed and output.

  For example, in the case of an image frame of standard resolution (720 pixels × 480 lines), when encoded with MPEG-2, 1 macroblock corresponds to a raster image of 16 pixels × 16 lines, so 1350 macroblocks are included in one screen. Will be placed. Each macroblock has a coding mode (intra, inter, skip). For this reason, the above-described output image is narrowed down by setting as a threshold that more than half of 1350 and 675 macroblocks are intra-coded. In this way, when a stationary subject is photographed, a number of matching images with similar contents are displayed in succession, and conversely, the searchability is lowered for the user. It can be avoided.

In FIG. 7, since the reproduced image of candidate 2 is encoded with an I picture among the three candidate images, the reproduced image of candidate 2 is displayed and output as a matching image.
The above-described decoding result parameter of the compressed image is stored in the status register of the decoding unit 108 in FIG. It is also desirable from the viewpoint of speeding up that the program built in the control microcomputer 110 performs the narrowing determination process by reading the aforementioned register. By doing in this way, only the image data with the highest image quality can be provided to the user among the coincidence images detected in a plurality.

FIG. 11 and FIG. 12 show flowcharts of the matching image narrowing process according to the above-described present method.
The processing procedures from S101 to S106 in FIG. 11 are basically the same as the matching image extraction processing described with reference to FIG. .
In S117, a selection image narrowing process is performed. In step S118, the reproduced image is displayed and output on the external device 115.

Next, an example of the selection image narrowing process performed in S117 of FIG. 11 will be described with reference to the flowchart of FIG.
In the narrowing-down process of the present embodiment, first, in S121, the image data and image information extracted by the previous matching image extraction are temporarily stored in the memory until the narrowing-down process is completed. Subsequently, after the matching images for the playback section for narrowing are stored in the memory, the narrowing process is performed. At this time, it is determined whether or not a plurality of matching images exist continuously in the playback section. (S122).

  As a result of the determination in S122, if there is no matching image in the predetermined section, that is, if there is only one matching image, the narrowing process is completed with the frame image as the matching image, and the display is output (FIG. 11 S118).

  On the other hand, if it is determined in S122 that there are a plurality of matching images, the process proceeds to S123, where it is determined whether all matching images have been searched. By performing this determination processing, the image frame information stored in the memory is searched until the display image narrowing down is completed. As a result of the determination in S123, when all the search is performed, the process proceeds to S130, and the frame information other than the output image is discarded. On the other hand, as a result of the determination in S123, if not all have been searched, the process proceeds to S124.

  In S124, it is detected whether the image frame information being searched includes image data encoded with an I picture. If image data encoded with an I picture is detected as a result of this detection, the process proceeds to S125, and at that time, a reproduced image of the I picture is determined as an output image. Thereafter, the process proceeds to S130, and other image frame information is discarded, and then the reproduction image narrowing process is completed.

  If the I picture cannot be detected in the detection process of S124, the process proceeds to S126, and the image data encoded with the P picture is detected. As a result of this detection, if there is image data encoded with a P picture, the process proceeds to S127, and at that time, a reproduced image of the P picture is determined as an output image. Thereafter, the process proceeds to S130, and other image frame information is discarded, and then the narrowing process is completed.

  If the P picture cannot be detected as a result of the detection in S126, the process proceeds to S128, and the number of intra macroblocks in the encoded picture (usually a B picture subjected to bidirectional predictive encoding) is equal to or greater than a predetermined threshold. Is determined (S128). If the result of this determination is that it is greater than or equal to the threshold value, the process proceeds to S129, and at that time, the reproduced image of that picture is determined as the output image. Thereafter, the process proceeds to S130, and other image frame information is discarded, and then the narrowing process is completed.

On the other hand, if the result of determination in S128 is smaller than the predetermined threshold, the process returns to S123 to search for the next image frame information (S123).
The above is a flow showing an example of the matching image narrowing process between the focus area and the face recognition area.

"5. Face recognition information generation process"
Next, face image recognition information generation processing according to the present embodiment will be described with reference to FIGS.
In the present embodiment, there is no particular limitation on whether the coincidence image detection process between the face recognition area and the focus area is performed when image data is recorded or reproduced and read. However, when face recognition processing is performed in the face image recognition unit 103 in FIG. 1 during playback reading, face recognition is performed during recording from the viewpoint of memory usage efficiency and processing time from buffering the playback image once to display output. It is desirable to have information as metadata in image data.

In the present embodiment, two methods are proposed as face recognition information recording methods: a region information expression method using coordinate format data shown in FIG. 8 and a region information expression method using bitmap format data shown in FIG. .
First, as shown in FIG. 8, the face recognition area information creation method using coordinates is as follows. When face recognition is detected in two rectangular areas for two persons, the coordinates of two diagonal points for each rectangle are used. (Points A and B, Points C and D) are obtained. Then, the horizontal and vertical components are recorded in the image data as coordinate information in sets for the number of detected persons.

  In the example of FIG. 8, two sets of (xa, ya) and (xb, yb), (xc, yc) and (xd, yd) are recorded. These face recognition coordinate information is recorded as metadata in the image data at every recording time. For example, when the compressed video data and audio data in the figure are time-division multiplexed in units of access units, the recording interval may be any of a picture (frame) unit, a GOP unit, or a user-specified time interval. To do.

  As a metadata recording area, private stream packet data defined by the MPEG-2 system standard, a GOP header defined by the MPEG-2 video standard, or a method of storing in a user data area in a picture header can be considered. . Further, a method of converting the face recognition information shown in FIG. 9 into bitmap format data by the same method and recording it as auxiliary information in the image data is also conceivable.

  A bitmap is created with the face portion set to 1 and the other portion set to 0 in the screen of the reproduced image, and the face recognition area is expressed as a bitmap. In this way, expressing as a bitmap increases the amount of information to be stored compared to coordinate information, but can express a free shape that fits a person's face other than a rectangle, thus detecting the human face area The accuracy at the time of extracting the coincidence image is improved. Also, in the coincidence image extraction process of the face recognition area and the focus area, it is only necessary to count the number of bits with a bit value of 1, which has the advantage of being easy to calculate. As a result, it is only necessary to acquire the face image area data at the time of reproduction, and it is not necessary to perform face detection processing that may cause memory access to a large amount of image data in the reproduction apparatus, thereby realizing high speed. it can.

  If the recording interval of the recognition information is short, if the bitmap is recorded as it is, the overhead embedded in the image data increases. Therefore, data obtained by subjecting this bitmap data to run length compression by zero run may be recorded. In this way, it is possible to reduce the amount of face image recognition data that occupies the entire image data, and it is possible to add as much information to the image data itself as much as the amount of data. Therefore, it is possible to provide the user with higher quality image data with less information loss due to information compression.

  Further, although not shown in the present embodiment, an identifier such as flag information that can be identified as a reproduced image that has been subjected to area coincidence detection processing between the face recognition area and the focus area and determined as a coincidence image at the time of shooting and recording. May be recorded as metadata in the image data. In this way, only the flag information is judged and the coincidence image may be displayed and output, and further improvement in searchability of content can be expected.

  Note that the generation of face recognition information does not necessarily have to be performed during recording shooting. In other words, if the face recognition information is not present at the time of reproduction / reading of image data, face image detection processing may be performed from the decoded / decompressed image data to obtain a match image detection flag. When a coincidence image detection flag is detected, any one of a menu screen display process, a notification process for notifying the user that the image is a recommended print image, and a narrowing process for narrowing output data, Or do more.

  As an embodiment of the present invention, it is desirable that face recognition information is recorded as metadata in image data from the viewpoint of access efficiency to the recording medium. However, the attribute of the image data independent of the image data is described. You may have as management file data.

  Note that the control of the playback device may be performed by one control microcomputer 110 hardware, or the entire playback device may be controlled by a plurality of hardware sharing the processing. Although the present invention has been described in detail based on the preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms without departing from the gist of the present invention are also included in the present invention. included. Furthermore, each embodiment mentioned above only shows one embodiment of this invention, and it is also possible to combine each embodiment suitably.

  In the above-described embodiments, the case where the present invention is applied to a video camera has been described as an example. However, the present invention is not limited to this, and can be applied to any apparatus capable of reproducing moving images. That is, the present invention can be applied to devices capable of reproducing moving images, such as personal computers, PDAs, mobile phone terminals, portable image viewers, digital photo frames, game machines, and music players.

(Other embodiments)
The present invention is also realized by executing the following processing. That is, software (computer program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various computer-readable storage media. Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads out and executes the program code. In this case, the program and the storage medium storing the program constitute the present invention.

DESCRIPTION OF SYMBOLS 101 Optical part, 102 Imaging part, 103 Face image recognition part, 104 Encoding part, 105 Recording / reproducing part, 106 Magnetic disk, 107 Optical disk, 108 Decoding part, 109 Display output part, 110 Control microcomputer, 111 System bus, 112 Semiconductor memory, 113 External control, 114 External input / output interface, 115 External connection equipment

Claims (18)

In a playback device for playing back image data and displaying it on a display device,
Face recognition information acquisition means for acquiring face recognition information representing an area where a face portion corresponding to a subject of the image data exists;
Focus information acquisition means for acquiring focus information representing a focused area of the image data;
An area coincidence detection unit for detecting coincidence between the face recognition area acquired by the face recognition information acquisition unit and the focus area acquired by the focus information acquisition unit;
And a control unit configured to control the display device to display the image data determined to be matched by the region match detection unit as a playback image. The area coincidence detecting means detects the degree of overlap between the face recognition area and the focus area,
A coincidence determination unit that determines that the face recognition area and the focus area are reproduced images when the degree of coincidence detected by the coincidence degree detection unit exceeds a predetermined threshold with respect to the focus area; The playback apparatus according to claim 1, wherein:   The coincidence degree detection unit determines that the face recognition area and the focus area are reproduction images when all of the face recognition areas are included in the focus area. 2. The playback device according to 2. Matching image search means for searching for matching image data between the face recognition area and the focus area from video data or a plurality of still image data;
2. The playback apparatus according to claim 1, further comprising thumbnail image generation means for generating thumbnail images by reducing the match image data searched by the match image search means. An association means for associating the thumbnail image and the matching image data with a logical address on a recording medium;
5. The playback apparatus according to claim 4, wherein the control means controls to display the menu screen by arranging the thumbnail images on the display apparatus. Display output means for displaying and outputting the thumbnail image generated by the thumbnail image generation means on a print selection screen;
5. The reproducing apparatus according to claim 4, wherein the control unit controls the display output unit to display the thumbnail image on the print selection screen and notify the user that the image is a recommended print image. A drawing means for converting the face recognition area and the focus area into image data and superimposing the matching image on the matching image;
2. The reproducing apparatus according to claim 1, further comprising: OSD (On Screen Display) means for displaying and outputting an image superimposed by the drawing means on the display device. Encoding method determining means for determining an encoding method of a picture in the moving image data;
In the moving image data that has been subjected to motion prediction and information compression by DCT (discrete cosine transform), the control means matches if the coincidence degree detection means detects a plurality of coincident images within a predetermined reproduction section. The playback apparatus according to claim 1, wherein control is performed so as to perform a narrowing process of a playback image selected as an image. The reproduction image narrowing process includes an intra-frame encoded image detection process, a forward prediction encoded image detection process, and an intra macroblock number determination process,
The control means outputs the detected image data as a reproduced image when image data subjected to intra-frame coding is detected, and forward prediction without detecting the image data subjected to intra-frame coding. When the encoded image data is detected, the detected image data is output as a reproduced image, and when no image data of any of the encoding methods is detected, the encoded image data 9. The reproduction apparatus according to claim 8, wherein image data having the largest number of intra macroblocks or image data having the number of intra macroblocks equal to or greater than a predetermined threshold is output as a reproduction image.   The face recognition area information is converted into coordinate format data or bitmap format data based on the result of face image recognition for recognizing the face area of a person existing in the screen, and recorded as supplementary information in the image data. The reproducing apparatus according to claim 1, wherein   11. The reproducing apparatus according to claim 10, wherein the bitmap format data is run-length compressed. Flag acquisition means for acquiring a coincidence image detection flag at the time of reading and reproducing the image data;
The control means, when a matching image detection flag is detected by the flag acquisition means, a menu screen display process for displaying the thumbnail images side by side on the display device, a notification process for notifying the user that the image is a recommended print image, 2. The playback apparatus according to claim 1, wherein control is performed so as to perform any one or a plurality of narrowing-down processes for narrowing output image data. Face recognition information acquisition means for acquiring face recognition information corresponding to a subject;
Focus information acquisition means for acquiring focus information of captured image data;
A region match detection unit that performs a region match detection process between the face recognition region acquired by the face recognition information acquisition unit and the focus region acquired by the focus information acquisition unit when recording image data;
And a recording means for recording a coincidence image detection flag as incidental information in corresponding image data when the area coincidence detection means detects that the face recognition area and the focus area coincide with each other. A recording device. In a playback method for playing back image data and displaying it on a display device,
A face recognition information acquisition step of acquiring face recognition information representing an area where a face portion corresponding to a subject of the image data exists;
A focus information acquisition step of acquiring focus information representing a region in focus of the image data;
A region matching detection step for detecting a match between the face recognition region acquired in the face recognition information acquisition step and the focus region acquired in the focus information acquisition step;
And a control step of controlling the display device to display the image data determined to match in the region match detection step as a playback image. A face recognition information acquisition step of acquiring face recognition information corresponding to the subject;
A focus information acquisition step of acquiring focus information of the captured image data;
A region matching detection step of performing region matching detection processing between the face recognition region acquired in the face recognition information acquisition step and the focus region acquired in the focus information acquisition step when recording image data;
A recording step of recording a coincidence image detection flag in the corresponding image data as incidental information when it is detected in the region coincidence detection step that the face recognition region and the focus region coincide with each other. Recording method. In a computer program for causing a computer to execute a process of reproducing and displaying image data on a display device,
A face recognition information acquisition step of acquiring face recognition information representing an area where a face portion corresponding to a subject of the image data exists;
A focus information acquisition step of acquiring focus information representing a region in focus of the image data;
A region matching detection step for detecting a match between the face recognition region acquired in the face recognition information acquisition step and the focus region acquired in the focus information acquisition step;
A program causing a computer to execute a control step of controlling image data determined to match in the region match detection step to be displayed on the display device as a reproduced image. A face recognition information acquisition step of acquiring face recognition information corresponding to the subject;
A focus information acquisition step of acquiring focus information of the captured image data;
A region matching detection step of performing region matching detection processing between the face recognition region acquired in the face recognition information acquisition step and the focus region acquired in the focus information acquisition step when recording image data;
When the region coincidence detecting step detects that the face recognition region and the focus region are coincident with each other, the computer is caused to execute a recording step of recording the coincidence image detection flag in the corresponding image data as incidental information. A program characterized by that.   A computer-readable storage medium storing the program according to claim 16 or 17.

Images