JP2000011144A - Eyebrow deformation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an eyebrow deformation system simple in the operation for deforming and displaying eyebrows and capable of expressing natural eyebrows in a facial image in a short time. SOLUTION: This system is provided with an eyebrow template storing means for preliminarily storing plural eyebrow templates expressing the shape of eyebrows, an eyebrow template correcting means for correcting the eyebrow template selected from the plural eyebrow templates according to a facial image, a deleting means for deleting the eyebrows from the facial image, and a composition means for composing the corrected eyebrow template to the facial image from which eyebrows are deleted. Thus, it is possible to deform and display the eyebrows in a short time by a simple operation only for selecting the desired eyebrow template.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an eyebrow deformation system, and more particularly to an eyebrow deformation system for deforming and displaying eyebrows in a human face image.

[0002]

2. Description of the Related Art An image production method using makeup applies a modeling theory, and understands the characteristics of a shape and how an impression is given when the image is seen. For example, straight lines create sharp or static images, and curves create soft or dynamic images. Therefore, by making the eyebrows linear or curved, it is possible to produce a sharp image or a soft image, respectively. Similarly, a descending line can be used to create a calm image, and a thick line can be used to produce a robust image.

[0003] It has been considered to realize such image effects by makeup using image processing technology of a computer system. In a conventional computer system image processing technique, a shape is simply deformed linearly or curvedly.

[0004]

However, in the image processing technology of the conventional computer system, the operator must specify the eyebrow part in the face image, perform a deformation operation, correct the color of the deformed part, and the like in detail. In addition, there is a problem that the operation requires specialized knowledge of image processing, and is troublesome and time-consuming.

SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and it is easy to operate an eyebrow in a face image and display it in a short time. The purpose is to provide a system.

[0006]

According to a first aspect of the present invention, there is provided an eyebrow deforming system for deforming and displaying eyebrows in a face image, wherein a plurality of eyebrow templates representing a shape of an eyebrow are stored in advance. Storage means, eyebrow template correction means for correcting an eyebrow template selected from the plurality of eyebrow templates according to the face image,
The image processing apparatus further includes an erasing unit that erases eyebrows from the face image, and a synthesizing unit that synthesizes the corrected eyebrow template with the face image from which the eyebrows have been erased.

As described above, a plurality of eyebrow templates representing the shape of the eyebrows are prepared, and the eyebrow template selected from the templates is corrected in accordance with the face image, and the eyebrow is deleted from the face image and corrected. By combining the eyebrow templates, it is possible to deform the eyebrows and display natural eyebrows without taking time by a simple operation of selecting a desired eyebrow template.

According to a second aspect of the present invention, in the eyebrow deformation system according to the first aspect, the eyebrow template correcting means includes an eyebrow head and an eyebrow of the selected eyebrow template according to a shape of a nose and an eye of a face image. The position of each buttocks is determined and the eyebrow template is modified. As described above, since the arrangement positions of the eyebrow head and the eyebrow tail of the eyebrow template selected according to the shape of the nose and eyes of the face image are determined, the eyebrow template can be synthesized in a natural form.

According to a third aspect of the present invention, in the eyebrow deformation system according to the first or second aspect, the erasing means combines a pixel near the eyebrow of the face image with a portion of the face image where the eyebrow is to be erased. . In this manner, since the pixels near the eyebrows are combined with the part where the eyebrows are to be erased, even if there is a difference in the shape between the eyebrow to be erased and the eyebrow template to be synthesized, the part where the eyebrows are erased becomes a skin color. It is possible to prevent the generated face image from being damaged.

According to a fourth aspect of the present invention, in the eyebrow deforming system according to the third aspect, there is provided a first smoothing means for smoothing a boundary of a portion where the eyebrow is erased by the erasing means. As described above, since the boundary of the portion where the eyebrows are removed is smoothed, the boundary is not noticeable, and the quality of the synthesized face image can be improved.

[0011] The invention according to claim 5 provides the invention according to claims 1 to 4.
The eyebrow deforming system according to any one of the above, further comprising a second smoothing unit that performs smoothing on an eyebrow edge in the face image in which the eyebrow template is combined by the combining unit. As described above, since the edge of the eyebrow in the synthesized face image is smoothed, the boundary of the synthesized eyebrow becomes inconspicuous, and the quality of the synthesized face image can be improved.

[0012]

FIG. 1 is a block diagram showing one embodiment of an eyebrow deformation system according to the present invention. In the figure, a central processing unit (CPU) 10 has an input device 20 via a bus 15,
Each of the storage device 30, the display device 40, the printing device 50, and the communication device 60 is connected. The input device 20 includes a keyboard 21, a mouse 22, a digital camera (or a CCD camera) 23, and the like.
RAM 31, ROM 32, hard disk device 33, flexible disk device 34, and the like. The CPU 10 executes various processing programs stored in the storage device 30, stores the results in the storage device 30, displays the result on the display device 40, prints the output by the printing device 50, and outputs the result. In addition to the various processing programs, a plurality of types of eyebrow templates representing the shape of the eyebrows are stored in the storage device 30 in advance.

FIG. 2 shows a main flowchart of one embodiment of the eyebrow deformation system of the present invention. In the figure, step S1
In the feature point extraction process of 0, the face of the subject is imaged, and feature points of the obtained face image are extracted. Next, in step S12, a desired eyebrow shape to be deformed is selected using the mouse 22 from a plurality of eyebrow templates displayed on the display device 40.

Thereafter, in step S14, the original left eyebrow of the face image is deleted, and in step S16, the left eyebrow of the eyebrow template selected is combined with the face image. Further, the original right eyebrow of the face image is deleted in step S18, and the right eyebrow of the eyebrow template selected in step S20 is combined with the face image. Step S2
In step 2, the face image combined with the eyebrows is displayed on the display device 40 and printed out by the printing device 50.

FIG. 3 shows a flowchart of one embodiment of the feature point extracting process executed in step S10. In the figure, the face of the subject is imaged by the digital camera 23 of the input device 20 in step S30, and the obtained face image is stored in the hard disk device 33 of the storage device 30. Next, step S3
In step 2, face position recognition processing is performed on the face image. By this face position recognition processing, labeling of the face area is performed, and feature points of the face outline are extracted. The face image is represented by XY coordinates with the origin at the upper left corner, the X axis in the horizontal direction, and the Y axis in the vertical direction.

Next, in step S34, an edge image creating process of the face area is performed to create an edge image (binary image) of the face. In step S36, the edge image is projected on the Y axis to obtain an edge histogram. The Y projection to be created is performed. Thereafter, in step S38, the Y coordinate of each part of the face, that is, each of the eyebrows, eyes, nose, and mouth is searched from the histogram by Y projection.

In step S40, at the Y coordinate position including the eyes and the nose in the face edge image, the characteristic points of the eyes and the nose and the characteristic points associated therewith are extracted. Step S4
In step 2, at the Y coordinate position including the mouth in the edge image of the face, a feature point of the mouth and a feature point attached thereto are extracted. Also,
In step S44, at the Y coordinate position including the eyebrows in the face edge image, the feature points of the eyebrows and the feature points attached thereto are extracted, and the process ends.

Here, the extraction of the characteristic points of the eyes, nose and eyebrows used for deformation of the eyebrows will be described in detail. FIG. 4 shows a flowchart of one embodiment of the eye and nose feature point extraction process executed in step S40. In the figure, step S46
Then, a face image in the Y coordinate area including the eyes and the nose is acquired. The threshold value TH is sequentially increased from the lowest value in the loop processing of the next step S47, and the face image is binarized using the threshold value TH in step S48 in the loop. The reason why binarization is performed by varying the threshold value TH is to accurately detect a pupil.

In the next step S49, in the edge image obtained by the binarization, a region surrounded by continuous edges is labeled. Then, it is determined whether or not each region labeled in step S51 in the loop processing of step S50 is a pupil. Here, it is determined whether or not the region is a pupil based on conditions such as whether the region is substantially circular and has an area within a predetermined range, and whether or not the region has a similar region forming a pair.

Next, in step S52, it is determined whether or not two pupils have been detected. If the pupils have been detected, in step S54, the face image is binarized using a fixed threshold value (edge image). I do. Thereafter, in step S56, the feature points of the left eye are extracted in the left eye region centered on the left pupil in the edge image. In step S58, the feature points of the right eye are extracted in the right eye region centered on the right pupil in the edge image. Extract points. Note that the pupils PL and PR shown in FIG.
1, ER1, eye corners EL4, ER4, and eye contours EL5, ER5, EL3, ER3 above and below the pupil, and these feature points are stored.

Thereafter, the process proceeds to step S60. Here, an equilateral triangle having two pupils as two points and one point in the direction of the mouth is set. Then, in step S62, the point having the largest Y coordinate in the above-described regular triangle in the edge image is extracted and stored as the feature point N2. Next, step S
At 64, a point having the smallest X coordinate between the feature points EL3 and ER3 and the feature point N2 is extracted and stored as the feature point N1, and a point having the largest X coordinate is extracted as the feature point N3. Remember. All the feature points of the eyes and the nose are extracted, and the process is terminated.

FIGS. 6 and 7 show a flowchart of one embodiment of the eyebrow feature point extraction process executed in step S44. In FIG. 6, in step S100, it is determined whether or not the Y coordinate of the eyebrow is appropriate. Here, for example, when the Y coordinate of the eyebrows is smaller than the average value of the Y coordinates of the feature points F1 and C2 of the face, it is determined to be appropriate. Here, if the Y coordinate of the eyebrow is not appropriate, it is set to an appropriate value in step S102.

In the next step S102, a loop process is performed by displacing the X coordinate from the inside to the outside of the left eyebrow in the face area, and in step S104 in this loop, the Y coordinate is shifted from the upper end to the lower end of the eyebrow area. Is displaced to perform a loop process. When a pixel which is an edge from the face edge image and is not a flesh color is first detected from the face image in step S106 in this loop, the process proceeds to step S108. In step S108, the Y coordinate of the detected pixel is stored as the upper end of the left eyebrow at the X coordinate at this time. Further, in step S110 in the loop of step S102, the Y coordinate is displaced from the lower end to the upper end of the eyebrow region to perform a loop process. In step S112 in this loop, the face is determined from the edge image and the face When a non-skin color pixel is first detected from the image, the Y coordinate of the pixel detected in step S114 is stored as the lower end of the left eyebrow at the X coordinate at this time.

In the next step S122 in FIG. 7, a loop process is performed by displacing the X coordinate from the inside to the outside of the right eyebrow in the face area, and a step S1 in this loop is performed.
At 24, a loop process is performed by displacing the Y coordinate from the upper end to the lower end of the eyebrow region, and step S126 in this loop is performed.
When a pixel that is an edge from an edge image of a face and is not a flesh color is first detected from the face image, a step S1 is performed.
Proceed to 28. In step S128, the Y coordinate of the detected pixel is stored as the upper end of the right eyebrow at the X coordinate at this time. Further, step S1 in the loop of step S122
At 30, the loop processing is performed by displacing the Y coordinate from the lower end to the upper end of the eyebrow region, and step S132 in this loop is performed.
When a pixel that is an edge from an edge image of a face and is not a flesh color is first detected from the face image, a step S1 is performed.
The Y coordinate of the pixel detected at 34 is stored as the lower end of the right eyebrow at the X coordinate at this time.

Then, in step S144, the highest position (minimum Y coordinate) at the upper end of each of the left and right eyebrows is extracted as a feature point of the mountain of eyebrows (points BL2 and BR2 shown in FIG. 5).
In step S146, the positions of the uppermost and lower ends of the left and right eyebrows, which are the innermost (center side) of the face, are extracted as the feature points of the eyebrows (points BL3 and BR3 shown in FIG. 5). In step S148, the outermost positions of the face at the upper and lower ends of the left and right eyebrows are extracted as feature points of the eyebrow tail (points BL1 and BR1 shown in FIG. 5). Step S1
At 49, the lowest position (maximum Y coordinate) at the lower end of each of the left and right eyebrows is determined as a feature point of the eyebrow bottom (points BL4, BR shown in FIG. 5).
Extract as 4). Then, this process ends.

FIG. 8 shows a flowchart of an embodiment of the eyebrow erasing process executed in steps S14 and S18. In the figure, in step S160, the eyebrow portion (portion surrounded by the eyebrow edge image) in the region including the eyebrows (the eyebrow region) of the face image is masked, and at this time, the eyebrow color information is stored. Next, in step S162, the loop processing is performed by displacing the Y coordinate from the upper end to the lower end of the eyebrow of the face image, and in step S164 in this loop, the X coordinate is displaced from the inside to the outside of the eyebrow to perform the loop processing. . Step S166 in this loop
It is determined whether or not the XY coordinates are a mask portion, and if it is a mask portion, the process proceeds to step S168.

In step S168, a flesh-colored pixel for copying is randomly selected from the outer peripheral position of the mask portion in the eyebrow region, and the pixel selected in step S170 is copied to the XY coordinate pixel of the mask portion. As a result, the skin color is combined with the eyebrow portion, and the eyebrow is erased. Then, in step S172, median filter processing is performed on the eyebrow erasure part to remove noise, and in step S174, smoothing near the eyebrow edge (by the edge image) of the face image is performed. Further, step S17
Smoothing near the eyebrows erased in step 6 is performed, and the process ends. As described above, since the boundary of the portion where the eyebrows are removed is smoothed, the boundary is not noticeable, and the quality of the synthesized face image can be improved.

FIG. 9 is a flowchart of an embodiment of the eyebrow synthesizing process executed in steps S16 and S20. In the drawing, the inclination of the eyebrow is calculated from the feature points of the eyebrow tip and the tail of the eyebrow extracted in step S180 (BR3, BR1 for the right eyebrow), and the eyebrow is set so that the inclination of the selected eyebrow template becomes the calculated eyebrow inclination. Rotate the template. Next step S
At 182, the color of the eyebrow template is changed to the original eyebrow color stored at the time of eyebrow erasure.

Then, in step S184, FIG.
As shown in the figure, the nostril feature point N3 and the inner corner feature point ER4
And the intersection of the Y coordinate (straight line L2) of the original eyebrow feature point BR3 is set as the eyebrow head TBR3 of the eyebrow template. Further, as shown in FIG. 10 (B), the intersection of a straight line L3 passing through the feature point N3 of the nostril and the feature point ER1 of the outer corner of the eye, and a straight line L4 passing through the eyebrow tip of the rotated eyebrow template and the outer edge of the eyebrow is defined as Set as hip TBR1.
Thereafter, the eyebrow template rotated at step S186 is enlarged or reduced so that the eyebrow tip and the eyebrow tail of the eyebrow template and the eyebrow tail coincide with the eyebrow tip TBR3 and the eyebrow tail TBR1, and the size is adjusted. As described above, since the arrangement positions of the eyebrow head and the eyebrow tail of the eyebrow template selected according to the shape of the nose and eyes of the face image are determined, the eyebrow template can be synthesized in a natural form.

Next, in step S188, the Y coordinate is displaced from the upper end to the lower end of the eyebrow of the face image to perform loop processing. In step S190 in this loop, the X coordinate is displaced from the inside to the outside of the eyebrow. Perform processing. In step S192 in this loop, the eyebrow template is copied to the face image while linearly interpolating to synthesize the eyebrows. Thereafter, in step S194, the Y coordinate is displaced from the upper end to the lower end of the eyebrow of the face image to perform loop processing, and in step S196 in this loop, the X coordinate is displaced from the inside to the outside of the eyebrow to perform loop processing. . Edge detection is performed in step S198 in this loop, and it is determined in step S200 whether the edge is an edge. If it is an edge, the process proceeds to step S202, in which smoothing is performed using nine pixels adjacent to the XY coordinates, and the synthesized eyebrows are blended with the face image so that a natural expression can be obtained.

As described above, a plurality of eyebrow templates representing the shape of the eyebrows are prepared, and the eyebrow template selected from the templates is corrected in accordance with the face image, and the eyebrow is deleted from the face image to correct the eyebrow. By combining the eyebrow templates, it is possible to deform the eyebrows and display natural eyebrows without taking time by a simple operation of selecting a desired eyebrow template. Further, since the edge of the eyebrow in the synthesized face image is smoothed, the boundary of the synthesized eyebrow can be prevented from unnaturally conspicuous, and the quality of the synthesized face image can be improved.

Note that the storage device 30 corresponds to the eyebrow template storage means, steps S180 to S186 correspond to the eyebrow template correction means, steps S160 to S170 correspond to the erasure means, step S153 corresponds to the synthesis means, Steps S172 to 176 correspond to the first smoothing means, and step S202 corresponds to the second smoothing means.

[0033]

As described above, according to the first aspect of the present invention, an eyebrow template storing means in which a plurality of eyebrow templates representing the shape of an eyebrow are stored in advance, and an eyebrow selected from the plurality of eyebrow templates An eyebrow template correcting unit that corrects a template according to the face image, an erasing unit that erases the eyebrows from the face image, and a synthesizing unit that synthesizes the corrected eyebrow template with the face image from which the eyebrows have been erased. .

As described above, a plurality of eyebrow templates representing the shape of the eyebrows are prepared, and the eyebrow template selected from the templates is corrected in accordance with the face image. By combining the eyebrow templates, it is possible to deform the eyebrows and display natural eyebrows without taking time by a simple operation of selecting a desired eyebrow template.

In the invention according to the second aspect, the eyebrow template correcting means determines an arrangement position of each of the eyebrow head and the eyebrow tail of the selected eyebrow template according to the shape of the nose and eyes of the face image. Modify the eyebrow template. As described above, since the arrangement positions of the eyebrow head and the eyebrow tail of the eyebrow template selected according to the shape of the nose and eyes of the face image are determined, the eyebrow template can be synthesized in a natural form.

[0036] In the third aspect of the present invention, the erasing means combines a pixel near the eyebrow of the face image with a portion of the face image where the eyebrows are to be erased. In this manner, since the pixels near the eyebrows are combined with the part where the eyebrows are to be erased, even if there is a difference in the shape between the eyebrow to be erased and the eyebrow template to be synthesized, the part where the eyebrows are erased becomes a skin color. It is possible to prevent the generated face image from being damaged.

Further, the invention according to claim 4 has a first smoothing means for performing smoothing on a boundary of a portion where an eyebrow is erased by the erasing means. As described above, since the boundary of the portion where the eyebrows are removed is smoothed, the boundary is not noticeable, and the quality of the synthesized face image can be improved. Further, the invention according to claim 5 has a second smoothing means for smoothing an eyebrow edge in the face image in which the eyebrow template is synthesized by the synthesizing means.

As described above, since the smoothing is performed on the edge of the eyebrow in the synthesized face image, the boundary of the synthesized eyebrow becomes inconspicuous, and the quality of the synthesized face image can be improved.

[Brief description of the drawings]

FIG. 1 is a block diagram of an embodiment of an eyebrow deformation system according to the present invention.

FIG. 2 is a main flowchart of an embodiment of an eyebrow deformation system according to the present invention.

FIG. 3 is a flowchart of one embodiment of a feature point extraction process executed in step S10.

FIG. 4 is a flowchart of an embodiment of eye and nose feature point extraction processing executed in step S40.

FIG. 5 is a diagram showing feature points of a face image.

FIG. 6 is a flowchart of an embodiment of eyebrow feature point extraction processing executed in step S44.

FIG. 7 is a flowchart of an embodiment of an eyebrow feature point extraction process executed in step S44.

FIG. 8 is a flowchart of an embodiment of eyebrow erasure processing executed in steps S14 and S18.

FIG. 9 is a flowchart of an embodiment of eyebrow synthesis processing executed in steps S16 and S20.

FIG. 10 is a diagram for explaining how to determine one of an eyebrow tip and an eyebrow tail of an eyebrow template.

[Explanation of symbols]

 Reference Signs List 10 Central processing unit (CPU) 20 Input device 21 Keyboard 22 Mouse 23 Digital camera 30 Storage device 31 RAM 32 ROM 33 Hard disk device 34 Flexible disk device 40 Display device 50 Printing device

 ──────────────────────────────────────────────────続 き Continued on the front page F-term (reference) PP23 RR02

Claims (5)

[Claims] 1. An eyebrow deformation system for deforming and displaying eyebrows in a face image, comprising: an eyebrow template storage unit in which a plurality of eyebrow templates representing an eyebrow shape are stored in advance; Eyebrow template correcting means for correcting an eyebrow template according to the face image, erasing means for erasing eyebrows from the face image, and synthesizing means for synthesizing the corrected eyebrow template with the face image from which the eyebrows have been erased. An eyebrow deformation system characterized by having. 2. The eyebrow deforming system according to claim 1, wherein said eyebrow template correcting means determines an arrangement position of an eyebrow head and an eyebrow tail of said selected eyebrow template according to a shape of a nose and an eye of a face image. And modifying the eyebrow template. 3. The eyebrow deformation system according to claim 1, wherein the erasing unit combines a pixel near the eyebrow of the face image with a portion of the face image where the eyebrow is to be erased. system. 4. The eyebrow deformation system according to claim 3, further comprising a first smoothing means for performing smoothing on a boundary of a portion where the eyebrow has been erased by said erasing means. 5. The eyebrow deformation system according to claim 1, further comprising a second smoothing unit that performs smoothing on an eyebrow edge in a face image in which the eyebrow template is synthesized by the synthesizing unit. Eyebrow deformation system.