JP2004013768A - Personal identification method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique (method) for identifying individual which stably gives a correct result by making effective use of colors. <P>SOLUTION: The individual identification method includes a step that defines a skin color by using a color system which is not affected by lightness and detects an area showing the skin color from an image taken in color, a step which discriminates whether the image detected of the area showing the skin color is a forward-facing image of the face of a person and a step which identifies an individual who meets the conditions of the forward-facing the image of the face of the person from among individuals registered in advance. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention belongs to the technical field of image recognition for identifying a predetermined individual from a color photographed image.
[0002]
[Prior art]
Techniques for identifying an individual using a captured image have been developed mainly in the field of security and robots. That is, in the field of security, it is important for a target person to identify a specific individual from among registered persons in terms of maintaining security. On the other hand, giving a robot the ability to identify an individual to make it an intelligent robot is an important technique for making the robot useful in human society beyond mere interest. However, technology for recognizing individuals is easy for humans, but not for machines. For this purpose, techniques for identifying individuals and related techniques have been developed, but no sufficiently reliable yet compact and practical device has been developed.
[0003]
For example, Japanese Unexamined Patent Publication No. Hei 11-339048 discloses a personal identification technique (method, device) for security. FIG. 10 shows the configuration of this personal identification device (hereinafter referred to as the conventional device 1), and FIG. 11 is a flowchart showing the procedure. In FIG. 10, a photographing device 110 includes a CCD camera and a lighting device for irradiating light from the lateral direction, and a photographed image irradiated with light and a photographed image not irradiated with light are recorded in an image memory 111. The imaging target determination unit 112 determines whether the captured image in the image memory 111 is an image of a human or an image of a human face. The face area extraction unit 113 extracts a face image of the identification target person using a conventional technique. For example, a difference between a background image and an image including a person is used. The feature extraction unit 114 obtains a face feature amount using a conventional technique. For example, a statistical method of performing principal component analysis of a difference from an average face and a method of performing template matching from a gray image such as an eye, a nose, and a mouth are known. The similarity determination unit 115 calculates the similarity by comparing the feature amount of the face of the identification target person with the feature amount of the registered face.
[0004]
The main point of this personal identification device (conventional device 1) is to determine whether the captured image is an image of a human or an image of a human face. That is, when photographing is performed by irradiating light from the side of the camera, a shadow appears in the photographed image of the human due to the contour of the face, the shape of the eyes, nose and mouth, and the unevenness of the face. Not appear in the image. Note that the flowchart of FIG. 11 can be understood by looking at the flowchart, and thus the description is omitted.
[0005]
JP-A-2002-56338 discloses a personal identification device applied to a robot. FIG. 12 shows a configuration diagram of a robot device using this personal identification device (hereinafter, referred to as a conventional device 2). FIG. 13 shows a person identification device in the robot apparatus, and FIG. 14 shows a processing procedure of the person identification means. The main purpose of this conventional device is to identify a person in an environment where lighting conditions are not constant, such as a home environment. Hereinafter, portions considered to be relevant to the present invention will be briefly described. In FIG. 13, the person identification device 214 includes a video acquisition unit 202, a facing distance sensor 205, a person detection unit 201, and the like. Here, the image acquisition unit 202 includes left and right cameras 203 and 204, and the facing distance sensor 205 measures the distance between the camera and the target object in the optical axis direction.
[0006]
The person detection / identification unit 201 detects the head of a person from the video image obtained by the video acquisition unit 202, performs head detection tracking unit 206 that outputs rectangular coordinates of the head, and performs face area search processing. A front face alignment unit 207 that outputs a rectangular image at the center of the face as a front face image; a face feature detection unit 218 that detects a feature amount by raster-scanning the front face image and outputs feature amount data; A face identification unit 209 for performing face identification processing with reference to the dictionary data in the dictionary storage unit 210, and an identification result correction unit 211 for performing integration processing with past identification results and outputting the result to a person detection identification unit. Have been.
[0007]
Above, the front face alignment means 207 is a head rectangle cutting means for cutting a partial image from the input image data with the head being a rectangle, and searching for a front face area from the head rectangle image, And a standard face dictionary, and a front face search means for outputting a pattern distance or a similarity to the standard face dictionary, and a front face likeness determination means for determining whether or not the front face image is really a front face. Specifically, the front face likeness determination means obtains the variance of the grayscale values of the front face image data, and determines that the face is not a front face when the variance is equal to or smaller than the threshold value.
[0008]
The conventional device 2 has the above configuration and performs processing according to the procedure shown in FIG. The description of the processing procedure is omitted. In the conventional device 2, a captured image is processed as a gray-scale image, and a complicated analysis and procedure are required because no color is used. But we live in a colored space and know that colors provide important information. Therefore, it is considered that individual identification can be more easily performed by using colors.
[0009]
[Problems to be solved by the invention]
The present invention has been made in view of the above background, and has as its object to provide a technique (method) for personal identification that makes effective use of colors and that can obtain a stable and correct result.
[0010]
[Means for Solving the Problems]
In order to solve the above problems, the present invention employs the following solutions. That is,
The invention according to claim 1 includes a step of detecting a region indicating a flesh color from a captured color image, a step of determining that the image of the detected flesh color region is a forward-looking face image of a person, Identifying an individual from among individuals registered in advance for the image.
[0011]
According to a second aspect of the present invention, in the first aspect of the present invention, the step of detecting the skin color region includes comparing and matching a parameter of a skin color component registered in advance with a color component of the color image to determine a human skin color. It is characterized by detecting a part.
[0012]
According to a third aspect of the present invention, in the first aspect of the present invention, the flesh-color area is detected by assigning a flesh-color probability to each pixel of the color image, and determining the area having the maximum average flesh-color probability. If the average value is detected and the average value is sufficiently larger than a predetermined value, it is set as a skin color area.
[0013]
According to a fourth aspect of the present invention, in the first to third aspects of the present invention, a skin color of the skin color portion is defined in a predetermined range using a color system that is not affected by lightness, and a photographing assistance during photographing is defined. It is characterized in that it can be adjusted in consideration of photographing conditions such as a light emission color of a lamp or conditions such as a difference in flesh color component depending on race.
[0014]
According to a fifth aspect of the present invention, in the first to fourth aspects of the present invention, the step of determining that the image is a forward-looking face image of the person represents the forward-looking face image of the person in a region where a flesh-colored portion is displayed. The method is characterized by including a preliminary step of correcting an area.
[0015]
According to a sixth aspect of the present invention, in the first aspect of the invention, the step of determining that the image is a forward-looking face image of the person includes: The degree of matching with the face database created by learning is obtained, and it is determined that the image is a forward-looking image of a person.
[0016]
According to a seventh aspect of the present invention, in the first to fourth aspects of the invention, the step of determining that the image is a forward-looking face image of the person includes moving a scanning window up and down with respect to the skin color area. In the case where a search window showing the features of the forward face such as eyes and nose is found, the skin color area is determined to be a forward image of a person or a candidate thereof.
[0017]
According to an eighth aspect of the present invention, in the invention of the seventh aspect, the determination as to whether or not the image is a forward-looking face image of the person is made by performing a wavelet transform on a large number of correct cases and wrong cases for the candidates. The determination is made using the created histogram.
[0018]
According to a ninth aspect of the present invention, in the first to eighth aspects of the present invention, in the personal identification step, the forward facing face database of the individual in which the forward facing face image of the person obtained in the step is registered in advance. And identifying the individual by comparing with.
[0019]
According to a tenth aspect of the present invention, in the ninth aspect of the present invention, the database is created by extracting features of a forward-looking face of an individual from a large number of images for one individual.
An eleventh aspect of the present invention is characterized in that, in the invention of the ninth aspect, the collation uses a technology of a support vector machine.
[0020]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 is a flowchart showing a procedure of a personal identification method according to an embodiment of the present invention. FIG. 2 is an apparatus diagram for performing the method of the present embodiment. In FIG. 2, the processing device 11 is constituted by a computer such as a personal computer. The processing device 11 includes a processing unit 12, a bus line 15, a working storage device 13, a data storage storage device 14, and input / output buffers 16 and 17. Note that a CCD camera 10, a display 20, an external device 21, and the like are connected as attached devices. Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0021]
In FIG. 1, in step S11, an image photographed by the photographing camera 10 is stored in the storage device 14 of the processing device 11. In step S12, the skin color is set. The color of the human skin not only varies depending on the gender, the age such as the elderly and children, but also greatly varies depending on races such as whites and blacks. Furthermore, it looks remarkably changed by the lighting in the room. Also, when creating a device, it is necessary to rationally determine the skin color. Here, it is defined by adopting the Munsell color system, and is corrected for a change in environmental conditions due to an illumination lamp or the like.
[0022]
The Munsell color system is a color display method that has been improved from the color arrangement method proposed by the American painter Munsell. The color of the object is displayed in three-dimensional coordinates using hue (hue), lightness, and chroma (which represents the amount of tint, hereinafter referred to as tint) as basic axes. Here, the hue means red, yellow, green, blue, and magenta. Lightness is the brightness of a color, with white (a color having a reflectance of 100%) being 10 and black (a color having a reflectance of 0) being 0. Color represents the vividness of a color, and is measured from the distance from an achromatic color. The color tone is different from the saturation, and the saturation changes depending on the value of the lightness, but the color does not change depending on the value of the lightness. Therefore, by defining a skin color in the Munsell color system, a skin color that does not change with lightness can be defined. That is, there is a feature that a skin color that is hardly affected by the brightness of the indoor lighting can be defined.
[0023]
FIG. 3 shows a diagram in the case where a certain value is taken in the Munsell color system. In FIG. 3, a portion surrounded by oblique lines is a portion indicating the flesh color of a person (white). In the circumferential direction of FIG. 3, the hues are defined in the order of R (red), Y (yellow), G (green), B (blue), P (purple), and the amount of color (chroma) in the radial direction. Is stipulated. FIG. 4 is a diagram showing the distribution of skin color in a shaded portion for a specific case (Mitsuo Ikeda, "Basic Color Engineering", Asakura Shoten, p. 16). Therefore, if this part is defined as a flesh color, the flesh color can be generally expressed correctly. In addition, it is practically sufficient to be able to distinguish from the color of the clothes and the background color. In addition, when there is a color in the illumination light or in a special case (such as a black person), the correction is made in consideration of the conditions. By doing so, practical skin color identification becomes possible.
[0024]
In step S13, the captured image is scanned using a skin color mask. For example, for each pixel, the color of a portion defined in the Munsell color system is set to a probability of 1, and a low probability value is assigned as the pixel moves away from the portion. Is detected as a skin color. As a result, a flesh color portion in the captured image is detected. By this detection, the size and position of the flesh color portion can be determined. A rectangular region including a flesh color portion is determined, and the partial image is output. In this case, the rectangular area may be a part of the face part, or may include other parts. FIG. 5 is an illustration of these cases. In FIG. 5A, the forehead portion is omitted because a hat is worn, and a part of the frontal face is displayed. FIG. 5B shows the entire front face correctly displayed. In FIG. 5C, since only underwear is worn, an unnecessary neck portion other than the front face is displayed. However, it is important that the operation described above can obtain the position and approximate size of the frontal face (more precisely, the size of the frontal face candidate).
[0025]
In step S14, it is determined whether the aforementioned rectangular image is a front image of a person. This determination step includes a preliminary step of detecting whether or not the face is a human face using some general features of the face (for example, eyes and nose), and a step of determining whether or not the face is facing front. It consists of a determination step for verification. In the former preliminary step, as shown in FIG. 6A, a search is performed on the rectangular image 20 obtained in step S13 while moving the window 21 from top to bottom (or from bottom to top). If at least one window including the eyes and the nose is found in the search, the probability that the captured image includes the frontal face of a person becomes very large.
[0026]
In the determination step, verification is performed on an image region including the eyes and the nose. As a verification method, for example, a Schneiderman method using a statistical method is used. For this purpose, we prepare hundreds of thousands of sample images of the correct frontal face including the eyes and nose and samples of the incorrect images that are not correct, and perform wavelet transform on these images to create histograms. The histogram is performed for a plurality of colors, and the results for each of these colors are superimposed and displayed in FIG. FIG. 7A is a histogram for a correct frontal face sample, and FIG. 7B is a histogram for an erroneous sample. In this figure, the vertical axis indicates "positive" when the frequency is correct, and "negative (-)" when the error is incorrect. The horizontal axis indicates frequency characteristics indicating factors (or attributes). As can be understood from FIG. 7, it can be understood that if the factors E, F, and I are verified, correct determination can be made.
[0027]
When it is determined that the candidate for the front image includes the front image of the person, the positions of the eyes and the nose can be determined in the preliminary step, and thus the front face of the person is included as shown in FIG. It is possible to cut out only the image area. In addition, it can be understood that the determination step (S14) described above can obtain a very stable and correct result.
[0028]
In step S15, data representing the characteristics of the face of the captured image is extracted. In order to easily extract features, the input image (A) is converted as shown in FIG. 8 as the output image (B). For example, the nose of the input image (A) is brought to the center, rotated and moved so as to be symmetrical, and the brightness and contrast are converted. In addition, the size is also converted so that the display size of the face becomes constant. The facial features are extracted from the output image thus obtained. The feature extraction of the face differs depending on the face identification method. As a face identification method, a method of obtaining an inner product of a light and shade pattern (feature vector) and an individual characteristic such as a face shape are determined in advance by considering a light and shade pattern of a frontal face image as a feature vector. There is a method of extracting from face images and comparing them.
[0029]
Specific features such as the shape of the face include, for example, the contour of the face (jaw shape, round face, thin face, etc.), the shape of the eyes, nose, mouth, eyebrows, etc. The degree of symmetry, the unevenness of the face (the unevenness of the cheek), and the color of the face (the color of the cheek, forehead, etc.) are extracted. Feature extraction is performed on features (elements) that are predetermined as feature vectors. However, performing such feature extraction directly has a problem that it is difficult to mechanize (for example, it is difficult to create a program). Therefore, in the present embodiment, several to about 20 face photograph images are prepared for each individual as a personal face database to be registered, and features are statistically extracted. That is, the face photograph image is subjected to luminance conversion, a histogram or the like is created, and the individual characteristics described above are indirectly grasped from the luminance distribution.
[0030]
In step S16, the feature extracted from the photographed image is compared with an already registered personal feature database to identify the person in the photographed image. The personal feature database extracts facial feature vectors using about 10 images per person and is registered in the memory 14. Reference comparison between the feature of the captured image and the feature database is performed by a method called a support vector machine. That is, an identification function (division plane) for identifying each individual is determined, and if a feature vector extracted from a captured image belongs to any individual in the feature database, that individual is determined. Further, when the person does not belong to any individual, it is determined that the person is other than the registered person. In step S17, the determined result is output.
[0031]
As described above, the embodiments and examples of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to the examples, and there are design changes and the like without departing from the gist of the present invention. Is also included in the present invention. For example, extraction of facial features, creation of a database, and identification of an individual are not limited to the methods described above, and other methods may be employed. In the determination step in step 14, "eyes and nose" are used, but only "eyes" may be used. In this case, it can be detected even when the face is masked.
[0032]
【The invention's effect】
As described above, according to the configuration of the present invention, since the frontal face of the target person is performed reliably and easily, there is an effect that stable individual identification means can be obtained.
[Brief description of the drawings]
FIG. 1 shows a flowchart illustrating a procedure according to a method of an embodiment of the present invention.
FIG. 2 shows a schematic of an apparatus for performing the method of the embodiment of the present invention.
FIG. 3 shows a Munsell color system.
FIG. 4 shows data contents of a flesh color portion.
FIG. 5 shows an image of a skin color portion.
FIG. 6 shows a partial face search method (A) and its result (B).
FIG. 7 shows histograms of an image sample of a frontal face (A) and a sample of an error image (B).
FIG. 8 shows an input example (A) and an output example (B) of the feature extraction conversion.
FIG. 9 shows a configuration of a conventional personal identification device.
FIG. 10 shows a flowchart of the conventional apparatus.
FIG. 11 shows an example in which another conventional apparatus is used for a robot.
FIG. 12 shows a configuration of another conventional apparatus.
FIG. 13 shows a flowchart of the other conventional apparatus.
[Explanation of symbols]
REFERENCE SIGNS LIST 10 camera 11 processing device 12 processing unit 13 working storage device 14 storage storage device 15 bus line 20 display 21 external device

Claims (11)

Detecting an area indicating a flesh color from the captured color image, determining that the image of the detected flesh color area is a forward-looking face image of a person, and pre-registering the forward-looking face image. Identifying the individual from the individuals. 2. The personal identification according to claim 1, wherein the step of detecting the skin color region detects and compares a skin color portion of a human by comparing and comparing a parameter of a skin color component registered in advance with a color component of the color image. Method. In the detection of the skin color region, a skin color probability is given to each pixel of the color image, a region where the average value of the skin color probability is maximum is detected, and the average value is sufficiently large compared to a predetermined value. The personal identification method according to claim 1, wherein is a flesh-tone area. The skin color of the skin color portion is defined in a predetermined range using a color system that is not affected by lightness, and shooting conditions such as the emission color of a shooting auxiliary lamp during shooting, or conditions such as differences in skin color components due to race are taken. The personal identification method according to any one of claims 1 to 3, wherein the personal identification method is adjusted in consideration of the situation. 5. The method according to claim 1, wherein the step of determining that the image is a forward-looking face image of a person includes a preliminary step of correcting an area in which a flesh-colored portion is displayed to an area representing a forward-looking face image of a person. The personal identification method according to any one of the above. The step of determining that the image is a forward-looking face image of the person, the degree of matching the image of the skin color region with a face database created by learning from a large number of human faces is determined. The personal identification method according to claim 1, wherein it is determined that there is a personal identification. The step of determining that the image is a forward-looking face image of the person is performed by moving a scanning window up and down with respect to the flesh-colored area, and a scanning window showing features of the forward-facing face such as eyes and nose is found. The personal identification method according to any one of claims 1 to 4, wherein the skin color area is determined to be a forward-looking image of a person or a candidate thereof. The determination as to whether the image is a forward-looking face image of the person is made by using a histogram created by performing a wavelet transform on a number of correct cases and wrong cases for the candidate. 7. The personal identification method according to 7. 9. The personal identification method according to claim 1, wherein the personal identification step identifies the individual by comparing the forward-looking face image of the person obtained in the step with a previously registered forward-looking face database of the individual. The personal identification method according to any one of the above. The personal identification method according to claim 9, wherein the database is created by extracting features of the individual's forward face from a large number of images for one individual. The personal identification method according to claim 9, wherein the matching is performed using a technology of a support vector machine.

Images