JP2004185611A - Face position extraction method, program for causing computer to execute face position extraction method, and face position extraction device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method capable of extracting a face image from image information for real-time tracking of its position. <P>SOLUTION: The method prepares digital data of each pixel value within a target image region, including a human face region, and performs filtering processes (steps S102-110) step by step within the target image region using a between-the-eyes detecting filter, which is composed of six rectangles Si (1≤i≤6) connected with one another, to extract between-the-eyes candidate points. The method further sections object images by a predetermined size, centering the extracted between-the-eyes candidates, and selects a true candidate point from the between-the-eyes candidate points, according to pattern determination processing. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

  The present invention relates to image processing for processing an image from a camera or the like, and more particularly, to the field of image recognition for extracting a human face in an image.

  2. Description of the Related Art A TV conference system in which a plurality of people at remote locations hold a conference by communication has been put to practical use. However, in these systems, there is a problem that the amount of communication data increases when the video itself is transmitted. For this purpose, for example, a technique of extracting feature data on the gaze, face direction, facial expression, and the like of a target person in various places and transmitting only the extracted data to each other has been studied. The receiving side generates and displays a virtual image of the face of the person based on the data. Thereby, a TV conference can be efficiently performed while reducing the amount of communication data.

  Further, the technology of detecting a person from such an image has been actively studied as a technology indispensable for the development of fields such as human computer interaction, gesture recognition, and security.

  In the application of these person detection techniques, it is necessary to construct a stable system that satisfies the following conditions: 1) high detection rate, 2) resistance to changes in lighting environment, and 3) operation in real time. In the future, the need for real-time human detection for high-quality images (images with a large number of pixels constituting one screen) is expected to increase, and in the future, faster human detection algorithms will be developed. Will be needed.

  In order to detect a person, a method of first detecting a face is effective. The face has important information such as facial expressions, and if the face can be detected, it is easy to estimate the position of the limb and search.

  Up to now, many reports have been made on a face detection system using skin color information (for example, see Patent Document 1, Non-Patent Documents 1 to 4).

  These methods extract a skin color region from an image to obtain a face candidate region. Since the face candidate area can be limited, the processing range is limited and the amount of calculation can be significantly reduced, so that a high-speed system can be constructed. However, the method using color information is vulnerable to changes in the lighting environment, and cannot be expected to have stable performance when operated in a general environment.

  On the other hand, many face detection methods that do not use color information (use density information) use learning methods such as template matching and neural networks (for example, Non-Patent Documents 5 to 6). See). These techniques are characterized by high detection rates and robustness to lighting environments. For example, in the technology disclosed in Non-Patent Document 5, a very high detection rate is realized by applying a neural network.

  However, these methods need to match the template (model) over the entire image while changing the size, and have a problem that the amount of calculation is large. Therefore, when the pixel size increases, the amount of calculation increases dramatically, and it is very difficult to construct a real-time system.

On the other hand, in the technology disclosed in Non-Patent Document 7, a face is detected from the lightness / darkness relationship of the average brightness of the divided deposit area. However, the area is distributed from the forehead to the chin, and there are 16 divided areas. There is a problem of being influenced by style and beard.
Shinjiro Kawato and Shinji Tetsuya, "Real-time detection of eyebrows using a ring frequency filter", IEICE (D-II), vol. J84-D-II, no12, pp. 2577-2584, Dec. 2001. Shinjiro Kawato and Shinji Tetsuya, "Real-time detection and tracking of eyes", IEICE Technical Report, PRMU2000-63, pp. 15-22, Sept. 2000. Chai D, Gun K. N. "Segmentation of Faces Using Skin Color Maps in Videophone Applications", IEEE Transactions on Circuits and Systems for Video Technology, Vol. 4, pp. 551-564, 1990 (Chai, D. and Ngan, KN: "Face Segmentation Using Skin-Color Map in Videophone Application," IEEE Trans. On Circuits and Systems For Video Technology, Vol. 9, No., pp. 551- 564, 1990) J. Yang, A. Weibel, "Real-time face tracker", Proceedings 3rd IEEE Workshop on Application of Computer Vision, pp. 142-147, 1996 (J. Yang and A. Waibel, "A real-time face tracker," Proc. 3rd IEEE Workshop on Application of Computer Vision, pp. 142-147, 1996) H. Raleigh, S.M. Baruja, T .; Canada, "Face Detection by Neural Networks" IEEE Transaction Pattern Analysis and Machine Intelligence, Vol. 20, no. 1, pp. 23-38, January 1998 (H. Rowly, S. Baluja, and T. Kanada, "Neural-Network-Based Face Detection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 1, pp. .23-38, Jan.1998) E. FIG. Germouth, B.A. K. Low "Face Detection: Survey" Computer Vision and Image Understanding, 83 (3), pp. 139 236-274, 2001 (E. Hjelmas and BK Low, “Face Detection: A survey,” Computer Vision and Image Understanding, 83 (3), pp.236-274, 2001) Brian Scassellati "Eye Detection by Face Detection for Foveated Active Vision Systems" Procedings AAAI 1998 pp. 969-976 (Brian Scassellati, “Eye Finding via Face Detection for a Foveated, Active Vision System”, Proc. AAAI '98, pp.969-976) The technique disclosed in Japanese Patent Application Laid-Open No. 2001-52176 pays attention to a point between both eyes (hereinafter, referred to as a “between-the-Eyes”) as a stable facial feature point. ing. In other words, the forehead and nose are relatively bright around the area between the eyebrows, and the eyes and eyebrows on both sides have a dark pattern, and a ring frequency filter for detecting this is used.

  However, with the ring frequency filter, it is necessary to perform preprocessing to extract the skin color area and limit the area, and if the face with the bangs up to the eyebrows cannot be detected because the above pattern does not appear There was a problem that there is.

  Therefore, an object of the present invention is to provide a face position extraction device capable of extracting a face image from image information while suppressing the influence of lighting conditions and the hairstyle of a person, a method therefor, and the method using a computer. It is to provide a program to realize it.

  Still another object of the present invention is to provide a face position extracting device capable of identifying the position between eyebrows of a face and tracking the position in real time by suppressing the influence of lighting conditions and the hairstyle of a person, An object of the present invention is to provide a method for that and a program for realizing the method using a computer.

  According to an aspect of the present invention, there is provided a face position extracting method, comprising the steps of: preparing digital data of a value of each pixel in a target image region including a human face region; Extracting the position of the eyebrow candidate point by a filtering process using the eyebrow detection filter combined with the two rectangular shapes; cutting out a target image with a predetermined size around the position of the extracted eyebrow candidate point; Responsively, selecting a true candidate point from the eyebrow candidate points.

  Preferably, the eyebrows detection filter is obtained by dividing one rectangular shape into six parts.

  Preferably, the six rectangular shapes are two first rectangular shapes vertically adjacent to each other, and two second rectangular shapes vertically shifted from the first rectangular shape by a predetermined amount and vertically adjacent to each other. The shape and the second rectangular shape include two third rectangular shapes that are vertically displaced by a predetermined amount and that are adjacent in the vertical direction.

  Preferably, the step of selecting a true candidate point includes an eye pattern discriminating process for a target image corresponding to two predetermined rectangular shapes among rectangular shapes forming a eyebrow detection filter corresponding to the eyebrow candidate points. The step of detecting the position of the eye, the step of correcting the position of the eyebrow candidate point to the position of the middle point of the two eyes based on the detected position of the eye, and the step of correcting the corrected position of the eyebrow candidate point Rotating the input image so that the two eyes are horizontal at the center, and cutting out a target image of a predetermined size around the corrected position of the eyebrow candidate point in the rotated input image, and determining the pattern. Selecting a true candidate point from the eyebrows candidate points in accordance with the processing.

  Preferably, the step of preparing digital data includes the step of preparing the target image as a stereo image, and the step of selecting a true candidate point includes a step of: selecting a distance between the observation point of the eyebrow candidate point detected based on the stereo image. And selecting a true candidate point from among the eyebrow candidate points.

  According to another aspect of the present invention, there is provided a program for causing a computer to execute a method of extracting a face position in a target image region, the program comprising: Preparing digital data of pixel values, extracting the positions of the eyebrow candidate points by a filtering process using a combined eyebrow space detection filter of six rectangular shapes in the target image area, Cutting out a target image with a predetermined size around the position of the point, and selecting a true candidate point from eyebrow candidate points in accordance with the pattern determination processing.

  Preferably, the eyebrows detection filter is obtained by dividing one rectangular shape into six.

  Preferably, the six rectangular shapes are two first rectangular shapes vertically adjacent to each other, and two second rectangular shapes vertically shifted from the first rectangular shape by a predetermined amount and vertically adjacent to each other. The shape and the second rectangular shape include two third rectangular shapes that are vertically displaced by a predetermined amount and that are adjacent in the vertical direction.

  Preferably, the step of selecting a true candidate point includes an eye pattern discriminating process for a target image corresponding to two predetermined rectangular shapes among rectangular shapes forming a eyebrow detection filter corresponding to the eyebrow candidate points. The step of detecting the position of the eye, the step of correcting the position of the eyebrow candidate point to the position of the middle point of the two eyes based on the detected position of the eye, and the step of correcting the corrected position of the eyebrow candidate point Rotating the input image so that the two eyes are horizontal at the center, and cutting out a target image of a predetermined size around the corrected position of the eyebrow candidate point in the rotated input image, and determining the pattern. Selecting a true candidate point from the eyebrows candidate points in accordance with the processing.

  Preferably, the step of preparing digital data includes the step of preparing the target image as a stereo image, and the step of selecting a true candidate point includes a step of: selecting a distance between the observation point of the eyebrow candidate point detected based on the stereo image. And selecting a true candidate point from among the eyebrow candidate points.

  According to still another aspect of the present invention, there is provided a face position extracting apparatus, comprising: a photographing unit that prepares digital data of a value of each pixel in a target image region including a human face region; Means for extracting the positions of the eyebrow candidate points by a filtering process using the eyebrow space detection filter in which the six rectangular shapes are combined, and cutting out the target image with a predetermined size around the positions of the extracted eyebrow candidate points to determine the pattern. Selecting means for selecting a true candidate point from the eyebrow candidate points in accordance with the processing.

  Preferably, the eyebrows detection filter is obtained by dividing one rectangular shape into six.

  Preferably, the six rectangular shapes are two first rectangular shapes vertically adjacent to each other, and two second rectangular shapes vertically shifted from the first rectangular shape by a predetermined amount and vertically adjacent to each other. The shape and the second rectangular shape include two third rectangular shapes that are vertically displaced by a predetermined amount and that are adjacent in the vertical direction.

  Preferably, the selecting means determines an eye position by performing an eye pattern discriminating process on a target image corresponding to two predetermined rectangular shapes among the rectangular shapes forming the eyebrows gap detection filter corresponding to the eyebrows candidate points. Means for detecting, and means for correcting the position of the eyebrow candidate point to the position of the middle point of the two eyes based on the detected eye position, and two eyes centering on the corrected eyebrow candidate point position Means for rotating the input image so that is horizontal, and, for the rotated input image, a target image is cut out at a predetermined size around the position of the corrected eyebrow candidate point, and according to the pattern determination process, Means for selecting a true candidate point from the forehead candidate points.

  Preferably, the photographing unit includes a unit that prepares the target image as a stereo image, and the selecting unit selects one of the eyebrow candidate points according to a distance from the observation point of the eyebrow candidate point detected based on the stereo image. Means for selecting true candidate points.

  As described above, according to the present invention, it is possible to detect the position of a person's face, in particular, the position between the eyebrows or the eyes, from continuous screen information in real time.

[Embodiment 1]
[Hardware configuration]
Hereinafter, the face position extracting device according to the first embodiment of the present invention will be described. This face position extraction device is realized by software executed on a computer, such as a personal computer or a workstation, and extracts a person's face from a target image, and further extracts a human face image from a video of the person's face. It is for detecting the position and the position of the eyes. FIG. 1 shows the appearance of the face position extracting device.

  Referring to FIG. 1, a system 20 includes a computer main body 40 having a CD-ROM (Compact Disc Read-Only Memory) drive 50 and an FD (Flexible Disk) drive 52, and a display device connected to the computer main body 40. , A keyboard 46 and a mouse 48 as input devices also connected to the computer main body 40, and a camera 30 connected to the computer main body 40 for capturing images. In the apparatus of this embodiment, a video camera including a CCD (solid-state image sensor) is used as the camera 30, and processing for detecting the position of the eyebrows or eyes of a person who operates the system 20 in front of the camera 30 is performed. Shall be.

  That is, the camera 30 prepares digital data of the value of each pixel in the target image region, which is an image including the human face region.

  FIG. 2 is a block diagram showing the configuration of the system 20. As shown in FIG. 2, a computer main body 40 constituting the system 20 includes a CPU (Central Processing Unit) 56 connected to a bus 66 and a ROM (Read) in addition to a CD-ROM drive 50 and an FD drive 52. Only Memory) 58, a RAM (Random Access Memory) 60, a hard disk 54, and an image capturing device 68 for capturing an image from the camera 30. A CD-ROM 62 is mounted on the CD-ROM drive 50. An FD 64 is mounted on the FD drive 52.

  As described above, the main part of the face position extraction device is realized by computer hardware and software executed by the CPU 56. Generally, such software is stored and distributed in a storage medium such as a CD-ROM 62 or an FD 64, and is read from the storage medium by a CD-ROM drive 50 or an FD drive 52 and temporarily stored in a hard disk 54. Alternatively, when the device is connected to a network, the device is temporarily copied to a hard disk 54 from a server on the network. Then, the data is further read from the hard disk 54 to the RAM 60 and executed by the CPU 56. When a network connection is established, the program may be directly loaded into the RAM 60 and executed without being stored in the hard disk 54.

  The hardware itself and the operating principle of the computer shown in FIGS. 1 and 2 are general. Therefore, the most essential part of the present invention is the software stored in the storage medium such as the FD 64 and the hard disk 54.

  As a recent general tendency, various program modules are prepared as a part of a computer operating system, and an application program calls these modules in a predetermined arrangement when necessary and proceeds with processing. is there. In such a case, the software itself for realizing the face position extracting device does not include such a module, and the face position extracting device is realized only in cooperation with the operating system on the computer. However, as long as a general platform is used, it is not necessary to distribute software including such modules, and software itself not including those modules and a recording medium on which the software is recorded (and the software distributed on a network). Data signals in such a case) can be considered to constitute an embodiment.

[Basic principle of face image extraction]
First, in summary of the procedure of the present invention, in processing a video image in which a face is continuously photographed, a screen is scanned with a rectangular filter having a width of about a face and a height of about half of the width. The rectangle is, for example, divided into six parts of 3 × 2, and the average brightness of each divided region is calculated. When the condition of the relative light-dark relationship is satisfied, the center of the rectangle is set as the eyebrows candidate.

  When a continuous pixel is an eyebrow interval candidate, only the center candidate of the frame surrounding it is left as an eyebrow interval candidate. By comparing the remaining eyebrow interval candidates with the standard pattern and performing template matching or the like, a false eyebrow interval candidate is discarded from the eyebrow interval candidates obtained by the above-described procedure, and a true eyebrow interval is extracted.

  Hereinafter, the procedure of face detection according to the present invention will be described in more detail.

(6-segment rectangular filter)
FIG. 3 is a diagram illustrating the above-described rectangular filter divided into six by 3 × 2 (hereinafter, referred to as “six-part rectangular filter”).

  The six-segment rectangular filter is a filter that extracts a facial feature that 1) the nose muscle is brighter than the both-eye area and 2) the eye area is darker than the cheek, and obtains the position between the eyebrows of the face. A rectangular frame of i horizontal pixels and j vertical pixels (i, j: natural number) is provided around the point (x, y).

  As shown in FIG. 3, this rectangular frame is horizontally divided into three equal parts and vertically into two equal parts to be divided into six blocks S1 to S6.

  FIG. 4 is a conceptual diagram showing a case where such a six-segment rectangular filter is applied to a face image. FIG. 4A shows the shape of a six-divided rectangular filter, and FIG. 4B shows a state in which the six-divided rectangular filter is applied to both eyes and a cheek of a face image.

  Considering that the nose muscle is usually narrower than the eye region, it is more desirable that the width w2 of the blocks S2 and S5 is smaller than the width w1 of the blocks S1, S3, S4 and S6. Preferably, width w2 can be half of width w1. FIG. 5 is a conceptual diagram showing a configuration of a 6-divided rectangular filter in such a case.

  In the first embodiment, a six-divided rectangular filter as shown in FIG. 5 is used.

  Further, the vertical width h1 of the blocks S1, S2 and S3 and the vertical width h2 of the blocks S4, S5 and S6 do not necessarily have to be the same. However, in the following description, the vertical width h1 and the vertical width h2 are described as being equal.

  In the six-segment rectangular filter shown in FIG. 5, for each block Si (1.ltoreq.i.ltoreq.6), the average value of the pixel luminance "bar Si" (the superscript "-" is added to Si) is obtained.

  Assuming that one eye and eyebrows are present in the block S1 and the other eyes and eyebrows are present in the block S3, the following relational expression (1) holds.

  FIG. 6 is a conceptual diagram showing an image to be scanned by such a six-segment rectangular filter.

  As shown in FIG. 6, the target image for detecting a face image is composed of M pixels in the horizontal direction and M × N pixels in the vertical direction. In principle, the above-mentioned six-segment rectangular filter is applied while sequentially shifting one pixel in the horizontal and vertical directions from the pixel (0, 0) at the upper left corner to check the validity of the relational expression (1). Should be performed. However, it is inefficient to obtain the average value of the luminance in each block every time the six-part rectangular filter is shifted in this way.

  Therefore, in the present invention, a process for obtaining the sum of pixels in a rectangular frame is described in a known document (P. Viola and M. Jones, “Rapid Object Detection using a Boosted Cascade of Simple Features,” Proc. Of IEEE Conf. CVPR). , 1, pp. 511-518, 2001), which incorporates a technique for accelerating calculations using integral images.

  From the image i (x, y), the “integral image” is defined by the following equation (2).

  The integral image can be obtained by repeating the following.

  s (x, y) represents the sum of the pixels in the row. However, s (x, -1) = 0 and ii (-1, y) = 0. The important point is that an integral image can be obtained by scanning the entire image only once.

  Using the integral image makes it possible to easily obtain the sum of the luminance values of the pixels in the rectangular area. FIG. 7 is a diagram showing a rectangular area for which the sum is obtained using such an integral image.

  Using the integral image, the sum Sr of the luminance of the pixels in the frame of the rectangle D shown in FIG. 7 can be obtained by calculating the values of four points as follows.

  As described above, by using the integral image, the sum of the luminance values of the pixels in the rectangular area, and thus the average of the luminance values of the pixels, can be obtained at a high speed. It is possible.

(Extraction process of eyebrows candidate points)
In the following, a process of extracting candidate points between eyebrows using the above-described six-segment rectangular filter will be described.

  FIG. 8 is a flowchart illustrating a process of extracting candidate points between eyebrows.

  Referring to FIG. 8, first, as initialization processing, values of variables m and n are set to m = 0 and n = 0 (step S100).

  Subsequently, the upper left corner of the six-division filter is matched with the (m, n) pixel of the image (step S102). Further, the average density bar Si of the pixels in the area of the block Si is calculated (step S104).

  Next, it is tested whether or not the value of the average density bar Si satisfies the eyebrows candidate condition according to the equation (1) (step S106).

  If the test condition is satisfied (step S108), a pixel at the (m + i / 2, n + j / 2) position corresponding to the center point of the filter is marked with a forehead candidate mark (step S110). On the other hand, when the test condition is not satisfied (step S108), the process proceeds to step S112.

  In step S112, the value of the variable m is incremented by one. Next, it is determined whether the value of the variable m is within the range in which the filter can move in the horizontal direction in the target image (step S114). If it is within the range in which the filter can move, the process returns to step S102. On the other hand, if the filter has reached the limit in which it can move in the horizontal direction, the value of the variable m is reset to 0, and the value of the variable n is incremented by 1 (step S116).

  Next, it is determined whether the value of the variable n is within the range in which the filter can move in the vertical direction in the target image (step S118). If it is within the range in which the filter can move, the process returns to step S102. On the other hand, when the filter has reached the limit in which it can move in the vertical direction, the connectivity of the pixels is checked for the eyebrow candidate mark, and the center pixel of the outer shape frame of the connected element is determined as the eyebrow candidate point for each connected element ( Step S120). Here, the “center pixel” is not particularly limited, but may be, for example, the position of the center of gravity of each connected element.

  FIG. 9 is a diagram showing the result of extracting eyebrows candidate points by the above processing.

  FIG. 9A shows the shape and size of the applied six-segment rectangular filter, and FIG. 9B shows the connected element with the eyebrow interval mark as a hatched area.

  The size of the six-part rectangular filter to be applied to a given target image is determined by, for example, the size of the face image in the target image if the size is known in advance. It is also possible to set according to. Alternatively, in accordance with the size of the face when a person is present within the range to be photographed (distance from the camera 30), six-segment rectangular filters of various sizes are prepared in advance, and When detecting a face for the first time, from among the plurality of types of six-segment rectangular filters, ones having different sizes are sequentially selected and applied, and the degree of conformity of the face detection described below is the highest. You may choose a higher one.

(Extraction of eye candidate points and extraction of true eyebrows candidate points)
The eyebrows candidate points extracted as described above include false eyebrows candidate points in addition to the true eyebrows candidate points. Therefore, a true eyebrows candidate point is extracted by the procedure described below.

  First, a candidate point of the eye position is extracted based on the information of the eyebrow candidate point.

  For this purpose, a plurality of eye images are extracted from the face image database, and an average image is obtained.

  FIG. 10 is a diagram showing the template of the right eye obtained in this manner. The left-eye template may be obtained by horizontally inverting the right-eye template.

  By using the right-eye template and the left-eye template and performing template matching processing in the areas of the blocks S1 and S3 of the six-segment rectangular filter centered on the eyebrow candidate point shown in FIG. Points can be extracted.

  FIG. 11 is a flowchart for explaining a process of extracting a true eyebrow candidate point after extracting such eye candidate points.

  Referring to FIG. 11, first, in each area of blocks S1 and S3 of the eyebrow interval extraction filter, a point that best matches the eye template is searched for and set as candidate points for the left and right eyes (step S200).

  Next, the position of the eyebrow candidate point is corrected to the middle point of the left and right eye candidate points (step S202). Subsequently, the input image is rotated so that the candidate points of the left and right eyes are horizontally aligned around the corrected eyebrows candidate point position (step S204).

  The degree of similarity between the pattern around the corrected corrected eyebrows candidate point and the eyebrows template formed in advance by a procedure described later is calculated (step S206).

  It is determined whether or not the similarity is equal to or greater than a predetermined threshold (step S208). On the other hand, if it is less than the threshold value, it is set as a false eyebrows candidate point (step S212).

  Such processing is performed for all the eyebrow gap candidate points.

  FIG. 12 is a diagram for explaining the extraction process of eye candidate points in step S200 of FIG.

  In FIG. 12, a white circle is a candidate point between the eyebrows before correction, and a white cross indicates a candidate point of the eye.

(Browser template)
Next, a method of forming a forehead template used in step S206 of FIG. 11 will be described.

  FIG. 13 is a flowchart for explaining the procedure for forming the eyebrow template.

  Referring to FIG. 13, first, a plurality of face image data are prepared (step S300). Subsequently, the operator inputs the position of both eyes for each face image using a mouse or the like (step S302).

  Further, hereinafter, as processing inside the computer, the image is rotated about the midpoint of both eyes so as to be horizontal, and the orientation is normalized (step S304). The image is enlarged or reduced so that the distance between both eyes becomes a predetermined distance, and the size is normalized (step S306). Next, an i × j pixel between eyebrows is extracted about the center of both eyes (step S308).

  Further, the density is converted so that the variance becomes another predetermined value, for example, 1.0 so that the average density of the extracted eyebrow pattern becomes a predetermined value, for example, zero, and the density is normalized ( Step S310).

  The average pattern of many normalized eyebrow patterns is calculated (step S312), and the obtained average pattern is used as a template between eyebrows (step S314).

  However, according to the present invention, the eyebrow template obtained in step S314 is further processed as follows.

  That is, in the case of a person whose hair extends to the eyebrows, the forehead has a low luminance value, but the average template has a high density value. If the matching evaluation is performed as it is, the matching degree will be low. Therefore, a predetermined number of pixels from the top, for example, a portion corresponding to the sum of three pixels is not evaluated so as not to be affected by the hairstyle. For example, if the eyebrow template obtained in step S314 has a pattern of 32 × 16 pixels, template matching is eventually performed using a pattern of 32 × 13 pixels.

  FIG. 14 is a diagram for explaining the forehead template.

  FIG. 14A shows the eyebrow template obtained in step S314 of FIG. 13, and FIG. 14B shows the final eyebrow template for eliminating the effect of the forehead.

  In the template matching, evaluation can be performed independently for the left and right sides in consideration of the case where the lighting direction differs depending on the face direction. In this case, the above-mentioned eyebrow template may be divided into two parts on the left and right sides and template matching may be performed. For example, in the case of the eyebrow template having the size as in the above example, the template matching may be performed using a pattern of 16 × 13 pixels on each side on the left and right sides.

  Next, the template matching process in step S206 of FIG. 11 will be described in more detail.

  FIG. 15 is a flowchart for explaining the template matching procedure in step S206.

  Referring to FIG. 15, first, eyebrow interval candidate points are extracted (step S400), and rotation is performed about the eyebrow interval candidate points as necessary to perform scale correction (step S402).

  Next, an image having the same size as the template is cut out centering on the eyebrow candidate points (step S404). The correlation value between the cut out eyebrow interval pattern and the eyebrow interval template is calculated and set as a similarity (step S406).

  Note that the similarity is calculated by normalizing the density of the cut out eyebrow candidate pattern (mean zero, variance 1.0), calculating the square of the difference from the corresponding pixel of the template for each pixel, and summing them up. May be requested. That is, in this case, since the value of the sum can be regarded as the dissimilarity, the similarity may be evaluated by its reciprocal.

  FIG. 16 is a diagram illustrating an example of extracting the eyebrows and eye positions from the target image in this manner.

  Despite the state of wearing the hat and covering the mouth with the hand, the position of the eyebrows (the center of the rectangular frame in the figure) and the position of the eyes (cross) are detected well.

  In the present invention of the first embodiment, a candidate point between eyebrows is first extracted by a six-segment rectangular filter using grayscale information, and then the position of the eye is finally specified. And a high-speed face position extraction can be performed.

  Further, if the above-described processing is performed for each frame of a captured video image, it is possible to track a face image in a moving image.

  At this time, it is also possible to narrow down the area to be filtered in the original frame based on the information of the previous frame in which the face image has already been detected.

  In the above description, a six-segment rectangular filter obtained by dividing a rectangular shape into six parts of 3 × 2 is used as a filter used when searching for a candidate point between eyebrows.

  However, the shape of the filter is not limited to those shown in FIGS. 3 and 5 in order to be able to cope with a case where the face image is inclined from horizontal.

  FIG. 17 and FIG. 18 are diagrams for explaining other shapes of such a filter.

  That is, as shown in FIGS. 17 and 18, the blocks S1 and S4 and the blocks S3 and S5 can be shifted up and down by a predetermined amount in the opposite directions to the blocks S2 and S5 in FIG. It is possible.

  In this case, even if the face image is inclined by an angle corresponding to the amount of displacement, candidate points between eyebrows can be extracted well.

  In the present specification, the filters having the shapes shown in FIGS. 3 and 5 (six-segment rectangular filters) and the filters shown in FIGS. 17 and 18 are collectively referred to as “inter-brows detection filter”. To

[Embodiment 2]
As described with reference to FIG. 11 of the first embodiment, when a true candidate point is extracted from the eyebrow candidate points, it is generally necessary to correct the position of the eyebrow candidate points, rotate the input image, and the like. However, when the motion of the person in the image is relatively small as in a video conference, the process of extracting the true candidate points can be simplified.

  FIG. 19 is a flowchart for explaining processing for extracting a true eyebrow interval candidate point in the face position extracting apparatus according to the second embodiment.

  Referring to FIG. 19, first, the similarity between a pattern centered on the eyebrow candidate point in the input image and a previously formed eyebrow template is calculated (step S500).

  It is determined whether or not the similarity is equal to or greater than a predetermined threshold (step S502). If the similarity is equal to or greater than the threshold, it is determined as a true eyebrows candidate point (step S504). On the other hand, if it is less than the threshold value, it is set as a false eyebrows candidate point (step S506).

  Such processing is performed for all the eyebrow gap candidate points.

  Other processes and configurations are the same as those of the face position extraction device of the first embodiment, and therefore, description thereof will not be repeated.

  Even in such a configuration, when the change in the position of the person from the camera 30 or the direction of the face is small, the same effect as in the first embodiment can be obtained.

[Embodiment 3]
In the first and second embodiments, one camera 30 performs photographing.

  On the other hand, if two cameras 30 are used to form a two-lens stereo configuration, distance information on a person can be obtained.

  In other words, the method of detecting a true face candidate point from the candidate points extracted by the six-segment rectangular filter uses a method similar to that of the first or second embodiment in principle even in the third embodiment. Can be.

  However, in the face position extracting apparatus according to the third embodiment, the camera 30 has a twin-lens stereo configuration, and switches the size at which the face candidate region is cut out in accordance with the distance information in order to further expand the range of detectable face sizes.

  By switching the cutout size of the face candidate region, matching can be performed by scaling to the same face size as the average face template, and the detection range of the face size can be expanded.

  In the third embodiment, as described above, a two-lens stereo configuration is used, and parallax information of a candidate point is obtained. Since the size of the face is considered to be inversely proportional to the parallax, the size at which the candidate point is cut out from the parallax information is determined. Therefore, it is possible to cut out a face candidate area with an optimal size and to match the template with the template.

  In the following, a description will be given together with an evaluation of a face image database in which images of slightly different facial expressions, direction lighting conditions, and the like are stored for a total of 400 images for 40 people and 10 people each.

  The face images in this database are monochrome images having an image size of 92 × 112. The rectangle size was based on the number of pixels between the temples on the horizontal and the size of the number of pixels from the eyebrows to the tip of the nose on the vertical. As a result of manual measurement, the standard rectangular size was set to 60 × 30 for the face image (92 × 112).

  First, FIG. 20 is a diagram illustrating a range in which eyebrow gap candidate points can be detected for the same face image using six-segment rectangular filters having different sizes.

  In FIG. 20, the eyebrows are extracted while changing the rectangular size by 20% from the reference size. In the experiment, the true candidate point extraction rate and the number of candidate points were examined. Whether or not the candidate points include the true candidate points was visually determined whether or not the candidate points exist near the eyebrows.

  According to FIG. 20, the extraction rate in the standard rectangular size (60 × 30) is 92.0%, which is considered to be functioning effectively. On the other hand, when the rectangle size is 84 × 42, the extraction rate is extremely poor, and it is considered that the rectangle is too large to extract facial features.

  Referring to FIG. 20, it can be confirmed that the eyebrows candidate points can be extracted with a rectangle having a size of 0.6 to 1.2 times the size of the reference rectangle. It is considered that the face size and the rectangle size have a simple proportional relationship. Therefore, it is considered that the rectangular filter can extract eyebrow candidate points of the face having a size in the range of 0.83 to 1.67 times from the face having the reference size.

  Next, in order to obtain the relationship between the distance of the person and the size of the face candidate region to be cut out, the face of the person is photographed with the camera configuration used in the face position extraction device, and the distance between the Measure the parallax of the position and the size of cutting out the most suitable face for that face.

  For example, the parallax is obtained by manually measuring the difference in the number of pixels in the horizontal direction at the position between the eyebrows of the person captured by the left and right cameras 30. The size of the face is measured by manually measuring the number of pixels between the left and right temples. Although not particularly limited, the size of the six-segment rectangular filter in the vertical direction can be determined to be half of the horizontal direction.

  FIG. 21 is a diagram illustrating a relationship between parallax and a size at which an optimal face is cut out.

  Based on FIG. 21, the relationship between the size of the 6-divided rectangular filter, the size for cutting out face candidate points, and the relationship between parallax and the size for cutting out face candidate points is determined.

  FIG. 22 is a diagram showing the relationship among the 6-part rectangular filter size, the parallax, and the size at which a candidate point is cut out, set from FIG. Utilizing that the size of extracting a face candidate region that can be extracted by a six-segment rectangular filter of a certain size has a range of 0.83 to 1.67 times, for example, a filter size of two stages of 40 × 20 and 24 × 12 is used. It was set to cover the whole. The size for cutting out the face candidate area was set to be switched every 5 parallax pixels. It is considered that setting the cut-out size finely increases the accuracy. However, since the matching processing of the average face template has a certain degree of flexibility, switching within this range is sufficient. In FIG. 22, for example, when the rectangular filter size is 40 × 20, if the parallax is 20 as a result of stereo matching, this means that candidate points are cut out at a size of 48 × 24.

  If a disparity that does not fit in this table appears, or if no match is found, the candidate point is discarded as a false candidate point.

  Through the above processing, the face position extraction device of the third embodiment can extract candidate points between eyebrows from a target image.

  FIG. 23 is a flowchart illustrating a process of extracting a true inter-brows candidate point in the face position extracting device according to the third embodiment.

  Referring to FIG. 23, first, the distance of the candidate point from camera 30 is estimated by the twin-lens stereo method (step S600).

  Next, it is determined whether the distance is within a predetermined range (step S602). If there is no candidate point between the eyebrows within the predetermined range, it is determined that it is a false candidate point (step S612).

  On the other hand, when the distance is within the predetermined range, next, the eyebrow templates having different sizes prepared in advance are selected according to the distance (step S604).

  The similarity between the pattern around the eyebrow candidate point in the input image and the selected eyebrow template is calculated (step S606).

  It is determined whether or not the similarity is equal to or greater than a predetermined threshold (step S608). On the other hand, if it is less than the threshold value, it is set as a false eyebrow interval candidate point (step S612).

  Such processing is performed for all the eyebrow gap candidate points.

  Other processes and configurations are the same as those of the face position extraction device of the first embodiment, and therefore, description thereof will not be repeated.

  In such a configuration, since the true candidate point is extracted in consideration of the distance of the person from the camera 30, the position of the face image can be detected at a higher speed. Therefore, the face image can be tracked by performing the processing of the third embodiment on each frame of the moving image.

  In the third embodiment as well, as described in FIG. 11 of the first embodiment, when extracting a true candidate point from the eyebrow candidate points, the position of the eyebrow candidate point is corrected after detecting the eye position. It is also possible to rotate the input image.

[Embodiment 4]
In the third embodiment, the eyebrow template is selected according to the distance from the camera 30 to the eyebrow candidate point from the eyebrow templates of different sizes prepared in advance.

  However, it is also possible to perform template matching by reducing (or enlarging) the input image so as to match the size of the reference eyebrow template according to the distance of the eyebrow candidate point from the camera 30.

  FIG. 24 is a flowchart for explaining processing for extracting a true eyebrow interval candidate point in the face position extracting apparatus according to the fourth embodiment.

  Referring to FIG. 24, first, the distance of the candidate point from camera 30 is estimated by the twin-lens stereo method (step S700).

  Next, it is determined whether the distance is within a predetermined range (step S702). If there is no candidate point between the eyebrows within the predetermined range, it is determined that it is a false candidate point (step S712).

  On the other hand, if the distance is within the predetermined range, the input image is reduced according to the distance so that the eyebrow image matches the template size (step S704).

  The similarity between the reduced pattern centered on the candidate points of the input image and the eyebrow template is calculated (step S706).

  It is determined whether or not the similarity is equal to or greater than a predetermined threshold (step S708). On the other hand, if it is less than the threshold value, it is set as a false eyebrows candidate point (step S712).

  Such processing is performed for all the eyebrow gap candidate points.

  Other processes and configurations are the same as those of the face position extracting device of the third embodiment, and therefore, description thereof will not be repeated.

  In such a configuration, since the true candidate point is extracted in consideration of the distance of the person from the camera 30, the position of the face image can be detected at a higher speed. Therefore, by performing the processing of the fourth embodiment on each frame of a moving image, it is possible to track a face image.

  Also, in the fourth embodiment, as described in FIG. 11 of the first embodiment, when extracting a true candidate point from the eyebrow candidate points, the position of the eyebrow candidate point is corrected after the position of the eye is detected. It is also possible to rotate the input image.

  In the processing of each of the embodiments described above, the position of the eyebrows or the eyes can be detected in real time from screen information that is continuous at predetermined intervals on the time axis, for example, continuous frame images. Further, by continuously detecting the position between the eyebrows or the eyes in each of such continuous screen information, the position between the eyebrows or the eyes can be tracked.

[Modified Example of Processing for Selecting True Eyebrows Between Candidate Points Between Eyebrows]
In the embodiment described above, in the face position extraction processing, processing is performed such that a candidate point between the eyebrows is extracted from the image with the eyebrow detection filter, and the true eyebrows are selected from the candidate points.

  In other words, the “process of detecting a true eyebrow interval” corresponds to performing a pattern determination process for selecting a candidate point corresponding to a true eyebrow interval from a plurality of eyebrow interval candidate points. In the above-described embodiment, the pattern discrimination processing is performed based on the “similarity with the eyebrow template”. However, the pattern discrimination method is not necessarily limited to such a method.

  In the following, a modified example that can be used as such a pattern determination process, including a pattern determination process based on “similarity with the eyebrow template” will be described.

(1) Pattern discrimination processing based on the similarity with the pattern template Assuming that the template is f = {t _ij } and the evaluated pattern is f = {f _ij }, a simple similarity evaluation value (q) is as follows: There is a sum of the absolute values of the differences between the corresponding pixel values as in equation (5).

  Alternatively, the sum of squares of the absolute value of the difference as in the following equation (6) can be used.

  When equation (5) or (6) is used, it is determined that the smaller the value is, the higher the similarity is.

  On the other hand, a normalized correlation value represented by the following equation (7) can be used as another evaluation value.

In equation (7), if {t _ij } and {f _ij } completely match, the value of q is 1; if it is a completely inverted pattern (the opposite of light and dark), the value of q is − Becomes 1. Otherwise, the value of q is between 1 and -1. When equation (7) is used, the evaluation is such that the larger the value of q, the higher the similarity.

  Since the normalized correlation value is evaluated based on the difference from the average value, even if the brightness level is shifted as a whole, the evaluation is not affected. Further, for example, when the illumination becomes dark, not only does the average value of brightness decrease, but also the difference between light and dark decreases. Even in this case, the value of q is not affected by the normalization term of the denominator.

It is also possible to use the average pattern of the following as shown in equation (8), a number of sample pattern as a template _{^{(s n = {s n ij}} }).

  In this case, weighted similarity evaluation can be performed. For example, the upper right part of the right eye and the upper left part of the left eye may or may not have bangs depending on the person. Therefore, even if there is a difference from the template, it is considered that the part is not so important.

  Therefore, when there are many sample patterns, the variance indicating how much the brightness varies between samples at each pixel position is first calculated as shown in the following equation (9).

  Next, using the reciprocal of the variance for weighting, weighted similarity evaluation using an evaluation value q as shown in the following equation (10) can be performed.

  Or, there is a relationship between the pixels, such as "the right eye position should have the left eye at the target position and be black as well", and "the center should be bright at the nose line". Can be weighted in consideration of covariance, which is an index indicating how much the data varies. Expression (9), on the other hand, is for the case of self-dispersion.

  The similarity weighted in consideration of such a covariance is called “Maharanobis distance”.

That is, _assuming that t _ij is represented as a vector in a line, the Mahalanobis distance q is represented as the following equation (11).

Where Σ is the covariance matrix of s ^n. Even using this Mahalanobis distance q, a pattern discrimination process based on the similarity with the pattern template can be performed.

(2) Statistical pattern discriminating process The process of extracting candidate points between eyebrows from an image with an eyebrow detection filter and selecting a true eyebrow interval from the candidate points is, in other words, a process of extracting candidate points between eyebrows. It can be considered as a procedure of extracting a true eyebrows by determining whether the pattern corresponds to a face pattern or not a face pattern.

  In this case, a statistical pattern determination process can be applied to the “face” and “non-face” determination process.

  That is, when a large number of “face” and “non-face” samples are given, the statistical pattern discrimination process determines the “unknown” pattern as “face” or “non-face” based on those data. Is what you do. On the other hand, the concept of “non-face” is not necessary in the above-described similarity calculation.

(2-1) Linear Discriminant Method When a pattern f = {f _ij } is considered as an I × J-dimensional vector in which pixel values are arranged in a line, one pattern is considered as one point in an I × J-dimensional space.

  In the following description, since it is difficult to show three or more dimensions on a plane, a two-dimensional case will be described as an example.

  FIG. 25 is a conceptual diagram illustrating an example of the distribution of “face” samples and “non-face” samples.

  As shown in FIG. 25, if a sample of “face” (顔) and a sample of “non-face” (×) are distributed, a straight line separating “face” (顔) and “non-face” (×) L1 is obtained in advance, and it can be determined whether the input pattern of "unknown" is on the straight line L1 as "face" (o) or "non-face" (x).

  In two dimensions, it is a straight line ax + by, but in three dimensions, it is a plane represented by ax + by + cz. More generally, in a higher dimension, there is a hyperplane represented by a linear combination of each dimensional element. Such a discrimination based on a hyperplane is referred to as a “linear discrimination method”.

  In general, it is not always possible to completely separate “face” (○) and “non-face” (×) in one hyperplane, but “non-face” (×) is placed on the side of “face” (○). The hyperplane is determined so that the sum of the error that comes and the error that causes a “face” (○) on the side of the “non-face” (×) is minimized.

(2-2) Support Vector Machine Even if the hyperplane is determined so that the error is minimized by the above-described linear discriminant method, the error may be too large in practical use.

Even in such a case, for example, a point in a three-dimensional space of (x, y, z) is set to a higher dimension such as (x ² , y ² , z ² , xy, yz, zx) (in this case, It is known that, when mapping onto a (six-dimensional) space, the "face" (o) and "non-face" (x) can sometimes be separated on the hyperplane of the space, as described above. ing. Moreover, the support vector machine can calculate the hyperplane of the high-dimensional space to be mapped in the original space without actually mapping to the high-dimensional space.

  For a specific configuration for detecting a face using a support vector machine, see, for example, E. Osuna, R. Freund, and F. Girosi: "Training Support Vector Machines: an Application to Face Detection", Proc. Of International Conference on Computer Vision and Pattern Recognition, pp. 130-136 (1997).

  Hereinafter, an outline of the support vector machine will be described.

  FIG. 26 is a diagram illustrating a high-dimensional space of a mapping destination to which the support vector machine is applied.

  FIG. 26 also describes the high-dimensional space as a two-dimensional space.

  In the support vector machine, two parallel hyperplanes are assumed. One of these two hyperplanes is a hyperplane P1 which is in contact with a sample of "non-face" (x in the figure), and another is a hyperplane P2 which is in contact with a sample of "face" (in the figure). Pair.

  Other pairs of hyperplanes P3 and P4 are also conceivable. However, in the support vector machine, the pair having the largest interval among pairs of possible hyperplanes is adopted. This interval is considered to be a margin for determination, and a pair having the maximum margin is adopted.

  The determination of the “face” pattern and the “non-face” pattern by the hyperplane as shown in FIG. 26 is performed by determining the intermediate hyperplane equidistant from the hyperplane P1 and the hyperplane P2 in the above-described linear determination. It is performed as if it were a hyperplane.

(2-3) Discrimination by Bayesian Estimation When there are exclusion events H ₁ (which is a face) and H ₂ (which is a non-face), and A is an arbitrary event (an extracted light and shade pattern), Bayes' theorem Is represented by the following equation.

Here, P (H ₁ | A) is the posterior probability that when A occurs, it is H ₁ , and P (A | H ₁ ) is the prior probability that A occurs when H ₁ . After you find the A occurs, it compares both the posterior probability is the posterior probability or H ₂ is H _1, a determination that the pattern of the higher probability in the Bayesian decision. The ratio of the two posterior probabilities is expressed by the following equation.

  If the expression (9) is larger than 1, it is determined that the direction is 1. Equation (9) can be rewritten as equation (10) below.

Therefore, a large number of samples of the events H ₁ and H ₂ are collected, P (A | H ₁ ) and P (A | H ₂ ) are estimated, and λ is used as a threshold parameter to obtain an equation (10). When it is determined, it is possible to determine to determine whether events H ₂ determines an event a and event H _1.

  For a method of detecting a face by the Bayes determination method, for example, see H. Shneiderman and T. Kanade: "Probabilistic Modeling Of Local Appearance and Spatial Relationships for Object Recognition", Proc. Of International Conference on Computer Vision and Pattern Recognition, pp. 45-51 (1998).

  In addition, it is also possible to perform a “face” and “non-face” discrimination process by a discrimination using a neural network.

  The embodiments disclosed this time are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

It is an outline view of a system concerning one embodiment of the present invention. 1 is a block diagram illustrating a hardware configuration of a system according to an embodiment of the present invention. It is a figure showing a 6-partition rectangular filter. FIG. 9 is a conceptual diagram showing a case where a six-divided rectangular filter is applied to a face image. FIG. 11 is a conceptual diagram illustrating another configuration of the six-segment rectangular filter. FIG. 5 is a conceptual diagram illustrating an image to be scanned by a divided rectangular filter. FIG. 9 is a diagram illustrating a rectangular area for which a sum is obtained using an integral image. It is a flowchart for demonstrating the process which extracts the candidate point between eyebrows. It is a figure showing the extraction result of the eyebrows candidate point. It is a figure showing a template of the right eye. It is a flowchart for demonstrating the process of extracting the eyebrows candidate point after extracting the eye candidate point. FIG. 12 is a diagram for describing an eye candidate point extraction process in step S200 of FIG. 11. It is a flowchart for demonstrating the formation procedure of an eyebrow template. It is a figure for explaining a forehead template. It is a flowchart for demonstrating the procedure of template matching of step S206. FIG. 9 is a diagram illustrating an example in which the positions of the eyebrows and the eyes are extracted from the target image. It is a 1st figure for demonstrating the other shape of an eyebrows detection filter. FIG. 10 is a second diagram for explaining another shape of the eyebrow detection filter. 13 is a flowchart for describing processing for extracting a true eyebrow interval candidate point in the face position extracting device according to the second embodiment. FIG. 11 is a diagram showing in which range eyebrow candidate points can be detected with respect to the same face image using six-division rectangular filters of different sizes. FIG. 9 is a diagram illustrating a relationship between parallax and a size for cutting out an optimal face. FIG. 22 is a diagram illustrating a relationship among a 6-divided rectangular filter size, parallax, and a size at which a candidate point is cut out, set from FIG. 21. 15 is a flowchart for describing processing for extracting a true eyebrow interval candidate point in the face position extracting device according to the third embodiment. 15 is a flowchart illustrating a process of extracting a true inter-brows candidate point in the face position extracting device according to the fourth embodiment. It is a conceptual diagram which shows an example of distribution of a "face" sample and a "non-face" sample. FIG. 3 is a diagram illustrating a high-dimensional space of a mapping destination to which a support vector machine is applied.

Explanation of reference numerals

  20 face position extraction device, 30 camera, 40 computer body, 42 monitor.

Claims (15)

Preparing digital data of the value of each pixel in the target image region including the human face region;
Extracting a position of a candidate point between eyebrows by a filtering process using an eyebrow detection filter in which six rectangular shapes are combined in the target image area;
Centering on the position of the extracted eyebrow interval candidate point, cutting out the target image with a predetermined size, and selecting a true candidate point from the eyebrow interval points in accordance with a pattern determination process. Extraction method of face position. The face position extracting method according to claim 1, wherein the eyebrow gap detecting filter is obtained by dividing one rectangular shape into six. The six rectangular shapes are:
Two first rectangular shapes that are vertically adjacent,
Two first rectangular shapes that are shifted from the first rectangular shape by a predetermined amount in the vertical direction and that are adjacent in the vertical direction;
2. The face position extracting method according to claim 1, wherein the second rectangular shape is shifted by a predetermined amount in the vertical direction and includes two third rectangular shapes adjacent in the vertical direction. Selecting the true candidate points comprises:
Detecting a position of an eye by eye pattern discrimination processing for the target image corresponding to two predetermined rectangular shapes among the rectangular shapes forming the eyebrows detection filter corresponding to the eyebrows candidate point; ,
Correcting the position of the eyebrow candidate point to the position of the midpoint of two eyes based on the detected position of the eye;
Rotating the input image so that the two eyes are horizontal about the corrected eyebrows candidate point position;
With respect to the rotated input image, the target image is cut out at a predetermined size around the position of the corrected eyebrow candidate point, and a true candidate point is selected from the eyebrow candidate points according to a pattern determination process. Selecting the face position. 2. The method according to claim 1, further comprising: Preparing digital data includes preparing the target image as a stereo image,
Selecting the true candidate points comprises:
2. The face position extraction according to claim 1, further comprising the step of selecting a true candidate point from the eyebrow candidate points according to a distance from the observation point of the eyebrow candidate point detected based on the stereo image. Method. A program for causing a computer to execute a method of extracting a face position in a target image region, wherein the program includes:
Preparing digital data of the value of each pixel in the target image region including the human face region;
Extracting a position of a candidate point between eyebrows by a filtering process using an eyebrow detection filter in which six rectangular shapes are combined in the target image area;
Centering on the position of the extracted eyebrow interval candidate point, cutting out the target image with a predetermined size, and selecting a true candidate point from the eyebrow interval points in accordance with a pattern determination process. program.   The program according to claim 6, wherein the eyebrows detection filter is obtained by dividing one rectangular shape into six. The six rectangular shapes are:
Two first rectangular shapes that are vertically adjacent,
Two first rectangular shapes that are shifted from the first rectangular shape by a predetermined amount in the vertical direction and that are adjacent in the vertical direction;
7. The program according to claim 6, wherein the second rectangular shape is shifted from the second rectangular shape by a predetermined amount in the vertical direction, and includes two third rectangular shapes adjacent in the vertical direction. Selecting the true candidate points comprises:
Detecting a position of an eye by eye pattern discrimination processing for the target image corresponding to two predetermined rectangular shapes among the rectangular shapes forming the eyebrows detection filter corresponding to the eyebrows candidate point; ,
Correcting the position of the eyebrow candidate point to the position of the midpoint of two eyes based on the detected position of the eye;
Rotating the input image so that the two eyes are horizontal about the corrected eyebrows candidate point position;
With respect to the rotated input image, the target image is cut out at a predetermined size around the position of the corrected eyebrow candidate point, and a true candidate point is selected from the eyebrow candidate points according to a pattern determination process. The step of selecting. Preparing digital data includes preparing the target image as a stereo image,
Selecting the true candidate points comprises:
The program according to claim 6, further comprising a step of selecting a true candidate point from the eyebrow candidate points according to a distance from the observation point of the eyebrow candidate point detected based on the stereo image. Photographing means for preparing digital data of the value of each pixel in the target image area including the human face area,
Means for extracting the positions of the eyebrow candidate points by filtering processing by the eyebrow space detection filter in which the six rectangular shapes are combined in the target image area;
Selecting means for cutting out the target image at a predetermined size around the position of the extracted eyebrow interval candidate point, and selecting a true candidate point from the eyebrow interval candidate points in accordance with the pattern discrimination processing , Face position extraction device.   The face position extraction device according to claim 11, wherein the eyebrow gap detection filter is obtained by dividing one rectangular shape into six. The six rectangular shapes are:
Two first rectangular shapes that are vertically adjacent,
Two first rectangular shapes that are shifted from the first rectangular shape by a predetermined amount in the vertical direction and that are adjacent in the vertical direction;
12. The face position extracting device according to claim 11, wherein the second rectangular shape is shifted by a predetermined amount in the vertical direction and includes two third rectangular shapes adjacent in the vertical direction. The selecting means,
Means for detecting the position of the eyes by eye pattern discrimination processing for the target image corresponding to two predetermined rectangular shapes among the rectangular shapes forming the eyebrows space detection filter corresponding to the eyebrows space candidate points; ,
Means for correcting the position of the eyebrow candidate point to a position of a midpoint between two eyes based on the detected position of the eye;
Means for rotating the input image so that the two eyes are horizontal about the corrected eyebrows candidate point position;
With respect to the rotated input image, the target image is cut out at a predetermined size around the position of the corrected eyebrow candidate point, and a true candidate point is selected from the eyebrow candidate points according to a pattern determination process. Means for selecting a face position. The photographing unit includes a unit that prepares the target image as a stereo image,
12. The method according to claim 11, wherein the selecting unit includes a unit that selects a true candidate point from the eyebrow candidate points according to a distance from the observation point of the eyebrow candidate point detected based on the stereo image. Face position extraction device.

Images