JP2003308530A - Image recognizer - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To precisely extract an area corresponding to the skin of a human. <P>SOLUTION: In an original image converted to L*a*b* color, an L*b* filter 2 changes to black all pixel values of pixels with a value of L* data within a range of 40 to 96 and a value of B* data outside a range of 0 to 16. A first a* filter 3 changes to black (L*=-100, a*=b*=0) all pixel values of pixels with a value of a* data outside a range of 4 to 16, and a second a* filter 4, whose characteristic is shown in Fig. 3c, changes to black (L*=-100, a*=b*=0) all pixel values of pixels with a value of a* data outside a range of -6 to 1. In a skin color range, pixels with a value of a* data in a range of 1 to 4 are excluded as they are possibly a pixel corresponding to a wall or a fluorescent lamp. With reference to nonblack areas included in images output by the first a* filter 3 and second a* filter 4, an area of the image corresponding to the skin of a human is extracted. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for extracting a region of an image portion of an object such as a human face included in an image such as a photograph.

[0002]

2. Description of the Related Art For example, as a technique for extracting an image portion of a person's face included in an image such as a photograph, there is Japanese Patent Laid-Open No. 2000-18772.
The technology described in Japanese Patent Publication No. 1 is known. In this technique, an image portion of a person's face is determined by cutting out an image portion whose pixel color is flesh color and performing a predetermined post-processing on the clipped image portion. That is, in this technique, the main feature amount of the image portion of the human face is used as the skin color to extract the image portion of the human face.

[0003]

In an image photographed under a normal living environment, the color distribution range of the image portion of the person's face and the color of the image portion of the wall imaged as the person's background are shown. The distribution range or the color distribution range of the image portion in which the fluorescent lamp is imprinted may partially overlap. Also, for this reason, when cutting out the pixel portion of the color within the skin color distribution range,
An image part on which a wall or a fluorescent lamp is imprinted may be cut out together with an image part of a person's face. That is, it may not be possible to accurately cut out only the area of the image portion of the person's face. And in this case,
In general, the accuracy of the post-processing for determining the image portion of the face of the person performed on the cut-out image portion also deteriorates, and the amount of processing increases.

Therefore, an object of the present invention is to extract an image portion of a target object included in an image with good accuracy.

[0005]

[Means for Solving the Problems] To achieve the above objects,
The present invention is, for example, an image recognition apparatus for extracting a region of an image portion of a person's face included in an image, and a plurality of colors obtained by dividing a person's skin color range into at least fluorescent lamp colors are excluded. A human skin region extracting unit that extracts a region of pixels including a color in any of the ranges, and a region of an image portion of a person's face included in an image is calculated based on the region extracted by the human skin region extracting unit. An image recognition apparatus is provided which comprises:

According to such an image recognition apparatus, it is possible to prevent the region of the image portion of the fluorescent light whose color is within the human skin color range from being extracted together with the region of the human skin image portion. It is possible to extract only the area of the image portion that reliably corresponds to the skin of the person. Therefore, if the area of the image portion of the person's face included in the image is extracted on the basis of the area thus extracted, the area of the image portion of the person's face can be accurately extracted. In addition, since unnecessary fluorescent lamp regions are excluded from the target of the face region calculation processing in advance, the processing amount is small, and the region of the image portion of the human face can be extracted at high speed.

In order to achieve the above object, the present invention provides
For example, in an image recognition device that extracts a region of an image portion of a person's face included in an image, the color range of a person's skin color is at least approximately 1 to 1 in the L * a * b * color model. Based on the human skin region extracting means for extracting a region of pixels including a color in any of a plurality of color ranges divided so as to exclude the color of 4, and the region extracted by the human skin region extracting means. An image recognition apparatus, comprising: a face area calculation unit that calculates an area of an image portion of a face of a person included in an image.

By doing so, it is possible to prevent the area of the image portion of the fluorescent lamp or the wall whose color is within the human skin color range from being extracted together with the area of the human skin image portion. It is possible to extract only the area of the image portion that surely corresponds to the skin. Therefore, similarly, if the area of the image portion of the person's face included in the image is extracted based on the area thus extracted, the area of the image portion of the person's face can be accurately extracted. Further, since unnecessary fluorescent lamps and wall regions are excluded in advance from the target of the face area calculation processing, the processing amount is small, and the area of the image portion of the human face can be extracted at high speed.

More specifically, in such an image recognition apparatus, the human skin area extracting means has an L * a * b * color model in which the L * data value is within a range of approximately 40 to 96. If the B * data value is in the range of approximately 0 to 16 and the a * data value is in the range of approximately −6 to 1 or approximately 4 to 16, the pixel area is extracted. It is possible to efficiently extract only the area of the image portion that surely corresponds to the skin of the person.

Further, in such an image recognition device, the face area calculation means is
By calculating the area of the skin-colored pixels that is continuous with the area extracted by the human skin area extraction means, the area of the color range estimated to be the image part of the fluorescent lamp or the wall is calculated as described above. Due to the elimination, even if a part of the area of the image portion corresponding to the skin of the person is no longer extracted by the person skin area extracting means, the image portion corresponding to the skin of the person that is not extracted is extracted. Can be obtained separately from the fluorescent lamp or the image portion of the wall, and as a result, the entire area of the image portion corresponding to the skin of the person can be obtained.

In order to achieve the above object, the present invention provides
For example, an image recognition method for extracting a region of an image portion of a target object included in an image, wherein a color range of the target object is excluded so that a color range of a predetermined foreign substance whose color range overlaps with the target object is excluded. A step of extracting a pixel area in which a color is included in any one of a plurality of divided color ranges; and a pixel area of a color within the color range of the object which is continuous with the extracted area on the image is calculated. An image recognition method is provided.

According to such an image recognition method, the region of the image portion of the foreign matter whose color is within the color range of the object is excluded from being extracted together with the image portion of the object,
It is possible to extract only the area of the image portion that reliably corresponds to the target object. Therefore, if the area of the image portion of the target object included in the image is extracted based on the area thus extracted, the area of the image portion of the target object can be accurately extracted. In addition, since the area of unnecessary foreign matter is excluded in advance from the processing target of the next step, the processing amount is small and the area of the image portion of the target object can be extracted at high speed. In addition, with the exclusion of the color range area estimated to be the image portion of the foreign matter as described above,
Even when a part of the image part area corresponding to the target object is not extracted, only the image part of the target object that is not extracted can be obtained separately from the foreign object image part. As a result, the entire area of the image portion of the target object can be obtained.

[0013]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below by taking an example of application to a case where an image portion of a human face is extracted from an image. FIG. 1 shows the configuration of the image recognition apparatus according to this embodiment. As shown in the figure, the image recognition apparatus includes an L * a * b * conversion unit 1, an L * b * filter 2, a first a * filter 3, a second a * filter 4, a first skin color region extraction unit 5, and a second skin color region extraction unit 5. It has a skin color area extraction unit 6, a skin color area synthesis unit 7, and a face area extraction unit 8. However, the image recognition device is a computer having a general configuration including a CPU, a main memory, an external storage device such as a hard disk device, an input device such as a keyboard and a pointing device, and a display device. Can be built into. In this case, each unit of the image recognition apparatus is realized as a process embodied on the electronic computer by the CPU executing the program loaded from the external storage device into the main memory.

The operation of the image recognition apparatus will be described below with reference to the flow chart shown in FIG. First, the L * a * b * conversion unit 1 sets the color model of the input original image to CI
E (Commission International de I'Eclariage) L * a * b
* Convert to color (step 21). That is, the color of each pixel of the original image is converted into an image represented by L * data, a * data, and b * data. In the L * a * b * color model, there are two elements: black to white (lightness = Lightness: L *), elements from green to red (a *), elements from blue to yellow (b *).
A color is represented by two color elements.

Next, the L * b * filter 2 performs a filtering process on the original image converted into the L * a * b * color by paying attention to the L * data and b * data of each pixel (step 22). That is, as shown in the characteristic of FIG.
* All pixel values of pixels whose data value is in the range of 40 to 96 and B * data value is not in the range of 0 to 16 are black (L * =-100, a * = b * = 0 ). Here, according to the experiments performed by the present inventors, human skin colors photographed by a digital camera have L * data values in the range of 40 to 96 and a * data values in the range of −6 to 16. Of which, the value of B * data is 0-16
Approximately fits within the range.

Next, the first a * filter 3 and the second a * filter 4 respectively perform the filtering process focusing on the a * data with respect to the image subjected to the filtering process by the L * b * filter 2 (step 23, 24). The first a * filter 3 has a * data value of 4 as shown in FIG. 3b.
All pixel values of pixels that are not within the range of ~ 16 are black (L * =-1
00, a * = b * = 0). On the other hand, the second a * filter 4
As shown in Fig. 3c, the value of a * data is -6 to 1
All pixel values of pixels that are not within the range are black (L * =-100,
Change to a * = b * = 0).

FIG. 4 shows an example of two images output by the first a * filter 3 and the second a * filter 4, respectively. Figure 4b
Shows the image output by the first a * filter 3 for the original image of FIG. 4a, and FIG. 4c shows the image output by the second a * filter 4 for the original image of FIG. 4a. Further, FIG. 4d shows an image obtained by combining the images of FIGS. 4b and 4c, and for the image subjected to the filtering process by the L * b * filter 2, the characteristic shown in FIG.
* All pixel values of pixels whose data values are not within the range of -6 to 1 or 4 to 16 are black (L * =-100, a * = b * =
It is the same as the image obtained by performing the filtering process for changing to 0).

Here, according to the experiments by the present inventors, generally, in an image taken by a digital camera, the colors of the inner wall of an office or a house and the fluorescent light which fall within the range of human skin color are , A * data is often in the range of 1 to 4.
Therefore, as shown in FIGS. 4b and 4c, according to the L * b * filter 2, the first a * filter 3 and the second a * filter 4, the image portion that truly corresponds to human skin is displayed on the inner wall of the office or house. It can be obtained after excluding the image part of the fluorescent lamp.

As can be understood from the image of FIG. 4d, the a * data is changed from a color in the range of 1 to 4 to black and falls within the range of human skin color. Since the color of the fluorescent lamp is excluded, the a * data of the image portion corresponding to human skin is within the range of 1 to 4 in both the images output by the first a * filter 3 and the second a * filter 4. The pixels in are no longer included.

Then, next, the first a * filter 3 and the second a *
From the image output by the filter 4, a process of obtaining an image portion corresponding to human skin is performed. That is, the first skin color region extraction unit 5 obtains the non-black region in the output image of the first a * filter 3 shown in FIG. 5a, and in FIG. 5c, the obtained non-black region (501 in FIG. , 50
The area of the skin color pixel which is spatially continuous with the area 2) is obtained (step 25). Here, as the skin color pixel,
L * data value is in the range of 40 to 96, a * data value is-
Pixels within the range of 6 to 16 and B * data values within the range of 0 to 16 are considered as skin color pixels. Further, preferably, even a flesh-colored pixel that is continuous with the non-black region directly or through another pixel is not smoothly continuous with the pixel value of the adjacent pixel (the difference between the pixel values is greater than a predetermined value). Pixels having (larger) pixel values should not be included in the area of the skin color pixel that is spatially continuous with the non-black area. As a result, in the example shown,
For the output image of the first a * filter 3 shown in FIG. 5a,
As shown in FIG. 5e, two skin color regions, that is, a region 504 corresponding to the face and a region 505 corresponding to the neck are obtained.
Even if the color of the inner wall of the office or the house or the pixel of the fluorescent lamp is within the range of human skin color, it is not included in the skin color region because it is not continuous with the non-black region.

On the other hand, the second skin color area extraction unit 6 obtains the non-black area in the output image of the second a * filter 4 shown in FIG. 5b, and in FIG. 5d, the non-black area obtained in the original image (FIG. 5d). The area of the skin color pixel which is continuous with the area indicated by 503) is obtained (step 26). As a result, in the illustrated example, for the output image of the second a * filter 4 shown in FIG. 5b, one skin color region 506 corresponding to the face is obtained as shown in FIG. 5f.

Next, the skin color area synthesizing section 7 extracts from the original image the image portion in the area where the skin color areas extracted by the first skin color area extracting section 5 and the second skin color area extracting section 6 are combined. 27). As a result, as shown in FIG. 5g, the extracted image portion becomes an image portion corresponding to the skin of the person.

Then, the face area extracting section 8 obtains an area corresponding to the human face in the original image based on the image portion extracted by the skin color area synthesizing section 7 (step 29). Various methods can be used to calculate the region corresponding to the human face, and for example, the following method can be used. That is, first, the contour line of the flesh color area in the image portion extracted by the flesh color area synthesis unit 7 is obtained. In the illustrated example, as shown in FIG. 5h, the contour line 507 of the skin color area corresponding to the face and the contour line 508 of the skin color area corresponding to the neck are obtained. Next, each of the obtained contour lines is regarded as a part of the face contour, and a virtual face contour line that matches the contour line is set. For example, for the contour line 507 of the skin color area corresponding to the face, the virtual contour line 509 of FIG. 5j is set, and the contour line 50 of the skin color area corresponding to the neck is set.
For 8, the virtual outline 510 of the face shown in FIG. 5k is set.

Then, as shown in FIGS. 5m and 5n, the image portion within the set contour line of each virtual face is extracted from the original image, and the extracted image portion is provided with the characteristics as a human face. Whether or not there is a pattern matching process for the face pattern information P prepared in advance is performed to check. Then, if the extracted image portion has a feature as a human face, the image portion is determined as a region corresponding to the human face (region shown in FIG. 5r). The face pattern information P does not need to be an image, and defines the features that a human face usually has (for example, two shapes corresponding to eyes, existing in an arrangement corresponding to two eyes, etc.). I wish I had it.

The face area extraction unit 8 outputs the information of the area corresponding to the human face obtained as described above, together with the original image of FIG. 5q, to a processing system which performs processing using these ( Step 30). Here, as a processing system that uses such an image and information of a region corresponding to a human face in the image, a monitoring system that monitors intrusion of a suspicious person or a face that does not hide other images or characters in the image There are an image processing system that performs processing such as synthesizing, a system that automatically lays out an image so that the face direction and size are appropriate, and prints it on a postcard or the like.

The embodiments of the present invention have been described above. As described above, according to the present embodiment, fluorescent lamps and walls having a color included in the color range of human skin color are excluded as much as possible, and an image portion corresponding to human skin is well extracted,
It becomes possible to accurately calculate the area of the image portion corresponding to the human face from the extracted image portion.

In the above embodiment, instead of the first a * filter 3, the second a * filter 4, the first skin color region extraction unit 5, and the second skin color region extraction unit and skin color region synthesis unit 7,
As shown in the characteristic of FIG. 3d, all the pixel values of the pixels whose a * data values are not within the range of -6 to 1 or 4 to 16 are black (L * =-100, a * = b * = It is also possible to provide an a * filter that performs a filtering process for changing to 0), and a skin color area extraction unit that extracts a skin color area from the output image of this a * filter.

Further, the characteristics of each filter described above are appropriately corrected according to the environment in which the image is captured, when the image to which the present embodiment is applied is not captured in a normal living environment. You can do it. In the above, the case of extracting the image portion of the face of the person from the image has been described.
The method of extracting the image portion of the target object by dividing the color range of the target object of this embodiment so that the color range of the foreign matter partially overlapping the color range of the target object is excluded is It can be similarly applied to extract image parts.

The present embodiment is also applicable to a moving image that is a sequence of still images.

[0030]

As described above, according to the present invention, the image portion of the target object included in the image can be extracted with good accuracy.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an image recognition device according to an embodiment of the present invention.

FIG. 2 is a flowchart showing an operation of the image recognition device according to the embodiment of the present invention.

FIG. 3 is a block diagram showing characteristics of a filter included in the image recognition apparatus according to the embodiment of the present invention.

FIG. 4 is a diagram showing an operation example of the image recognition apparatus according to the embodiment of the present invention.

FIG. 5 is a diagram showing an operation example of the image recognition apparatus according to the embodiment of the present invention.

[Explanation of symbols]

1: L * a * b * conversion unit, 2: L * b * filter, 3: first a * filter, 4: second a * filter, 5: first skin color region extraction unit,
6: Second skin color area extraction unit, 7: Skin color area synthesis unit, 8: Face area extraction unit.

Claims (6)

[Claims] 1. An image recognition apparatus for extracting an area of an image portion of a person's face included in an image, wherein a plurality of color ranges obtained by dividing a person's skin color range so that at least fluorescent colors are excluded. A human skin region extracting unit that extracts a region of a pixel whose color is included in any one of the above, and a region of the image portion of the person's face included in the image based on the region extracted by the human skin region extracting unit. An image recognition apparatus, comprising: a face area calculation means. 2. An image recognition apparatus for extracting an area of an image portion of a human face included in an image, wherein the skin tone color range of the person is at least a * data value in an L * a * b * color model. Human skin area extracting means for extracting an area of a pixel in which a color is included in any of a plurality of color ranges divided so that colors of approximately 1 to 4 are excluded, and an area extracted by the human skin area extracting means And a face area calculating unit that calculates an area of an image portion of a person's face included in the image based on the image recognition apparatus. 3. The image recognition apparatus according to claim 2, wherein the human skin area extracting unit has a L * a * b * color model in which the L * data value is within a range of approximately 40 to 96, and , B * data value is in the range of approximately 0 to 16 and a * data value is in the range of approximately -6 to 1 or approximately 4 to 16 apparatus. 4. The image recognition apparatus according to claim 2 or 3, wherein the face area calculation means is a skin color spatially continuous with the area extracted by the human skin area extraction means on the image. An image recognition apparatus comprising: calculating a pixel area of the pixel. 5. An image recognition method for extracting a region of an image portion of a target object included in an image, wherein a color range of the target object is excluded from a color range of a predetermined foreign substance whose color range overlaps with that of the target object. A step of extracting a pixel area in which a color is included in any of a plurality of color ranges divided as described above, and a pixel area of a color in the color range of the object that is continuous with the extracted area on the image An image recognition method comprising the step of: 6. A computer program read and executed by a computer, wherein the computer program causes the computer to exclude a color range of a target object and a color range of a predetermined foreign substance having a color range overlapping with the target object. Extracting a region of a pixel whose color is included in one of a plurality of color ranges divided as described above, and a pixel of a color within the color range of the object that is continuous with the extracted region on the image And a step of calculating a region of the computer program.