JP2009053981A - Makeup simulation device - Google Patents

Abstract

The present invention provides a makeup simulation apparatus that is small, simple, lightweight, has a high processing speed, and has high reality.
Information on a three-dimensional shape of a face and information defining a standard application shape indicating a desired cosmetic application region corresponding to the three-dimensional shape is held for a standard face image. A two-dimensional face image is taken, feature points are extracted from the two-dimensional face image of the photographed subject, and the feature points are compared with the two-dimensional feature points projected on the standard application shape corresponding to the feature points. Then, the standard application shape is deformed so that the feature points projected in two dimensions of the standard application shape coincide with the feature points of the subject, and synthesized with the face image of the subject. A state in which a desired cosmetic is applied to the generated face image is virtually generated. For example, the feature points are provided on the outer periphery of the pupil, the center and the left and right of the lips, and the left and right of the contour of the face, and the center of gravity and end points inside the parts such as the eyes, lips and face are extracted, and detailed contour extraction is not performed.
[Selection] Figure 1

Description

  The present invention is used for an apparatus that displays on a screen a state in which makeup is virtually applied to a face image of a subject. In particular, the present invention relates to an apparatus used for selling cosmetics at a store.

  For cosmetic research and development or in-store makeup advice, a makeup simulation device is known that displays a virtual makeup on a face image of a subject on a screen (see, for example, Patent Document 1 or 2). ).

The device for research and development is devised so that a face image virtually put on makeup based on the three-dimensional shape of the subject can be displayed with high reality. In addition, the apparatus for over-the-counter sales of cosmetics is devised so that a face image of the customer himself / herself can be captured and the face image in a state of virtually applying makeup can be displayed in real time.
JP-A-2005-216131 JP 2006-23921 A

  Generally, in order to collect data on a subject's three-dimensional shape, it is necessary to photograph the subject from different angles using a plurality of camera devices. Further, complicated processing is required for processing of captured image data. Therefore, it is difficult to capture, process, and display at high speed with the above-described research and development apparatus that uses the three-dimensional shape of the subject.

  On the other hand, it is desirable that a device for use at a store is small, simple, and lightweight. In order to use it while facing a customer, the waiting time for processing is as short as possible, the processing speed is high, and complicated operations are required. It needs to be as small as possible. Therefore, it is difficult to use a plurality of camera devices or to use information processing of a three-dimensional shape that requires complicated calculations, and as a result, it is impossible to obtain high-level expressions such as gloss with an anisotropic makeup. There is a problem that the reality of the obtained face image is low.

  The present invention has been made to solve such problems, and provides a makeup simulation apparatus that is small, simple, lightweight, has a high processing speed, and is highly suitable for performing makeup simulation at a store. The purpose is to do.

  The present invention provides a standard face information holding means for holding, for a standard face image, information on the three-dimensional shape of the face and information on a standard application shape indicating a desired cosmetic application region for the three-dimensional shape; A face image photographing means for photographing a two-dimensional face image of the subject, a feature point extracting means for extracting a feature point from the two-dimensional face image of the subject photographed by the face image photographing means, and the feature point extracting means The feature points of the subject extracted by the above and the feature points projected in two dimensions of the standard application shape held in the standard face information holding means are compared, and the feature points projected in two dimensions of the standard application shape The three-dimensional shape and the standard application shape are deformed so that the characteristic points of the subject coincide with each other, and a deforming and synthesizing unit that combines with the face image of the subject, and the face image generated by the deformation and synthesizing unit, Desired Makeup simulation means for virtually generating a state in which cosmetics are applied, and this makeup simulation means is used to virtually generate a state in which a desired cosmetic is applied, based on the information on the three-dimensional shape. It is a makeup simulation apparatus provided with means for expressing the gloss and color of the coated surface.

  For example, the feature points can be provided on the outer periphery of the pupil, the center and left and right of the lips, and the left and right of the face contour. Such feature point extracting means is characterized in that it extracts the center of gravity and end points inside the parts such as eyes, lips, and face, and does not perform detailed contour extraction. Alternatively, the feature points are provided on the outer periphery of the pupil, the center and left and right of the lips, and the feature points to be provided on the left and right of the face outline are analogized based on the feature points provided on the outer periphery of the pupil, the center of the lips, and the left and right. May be provided.

  According to this, since it is possible to generate a three-dimensional face image of a subject simply by taking a two-dimensional image of the subject, there is no need for a plurality of camera devices and complicated calculations, and a small number of main images Therefore, it is possible to eliminate complicated operations and complicated calculations, and this is a small, simple, lightweight, high processing speed, requiring complicated operations. A makeup simulation device that does not perform can be realized.

  Further, means for extracting external face image data consisting of a surface reflected light component reflected on the face surface and internal face image data consisting of an internally reflected light component reflected internally from the face image of the subject, Means for holding cosmetic film optical data comprising a surface reflected light component reflected on the surface, a film color component scattered and reflected inside the decorative film, and a transmissive component that transmits the decorative film; The means for expressing is based on the information of the three-dimensional shape of the face image of the subject, based on the cosmetic film optical data, the external face image data, and the internal face image data according to the shooting environment including the position of illumination. The gloss and color of the face image can be expressed.

  Thereby, a makeup simulation result with high reality can be displayed.

  The present invention can also be viewed from the viewpoint of a program. That is, the present invention is a program that, when installed in a general-purpose information processing apparatus, causes the general-purpose information processing apparatus to realize a function corresponding to the function of the makeup simulation apparatus of the present invention.

  By recording the program of the present invention on a recording medium, the general-purpose information processing apparatus can install the program of the present invention using this recording medium. Alternatively, the program of the present invention can be directly installed on the general-purpose information processing apparatus via a network from a server that holds the program of the present invention.

  Thereby, the function corresponding to the function of the makeup | decoration simulation apparatus of this invention is realizable using a general purpose information processing apparatus.

  The program of the present invention includes not only a program that can be directly executed by a general-purpose information processing apparatus but also a program that can be executed by installing it on a hard disk or the like. Also included are those that are compressed or encrypted.

  ADVANTAGE OF THE INVENTION According to this invention, the makeup | simulation simulation apparatus suitable for performing makeup | decoration simulation at a shop front is small, simple and lightweight, has a high processing speed, high reality, and does not require complicated operations.

  A makeup simulation apparatus according to an embodiment of the present invention will be described with reference to the drawings. As shown in FIG. 1, the makeup simulation apparatus of the present embodiment, for a standard face image, information on the three-dimensional shape of the face and a standard application shape indicating a desired cosmetic application region for the three-dimensional shape. A standard face information holding unit 1 that holds the information of the subject, a face image photographing unit 2 that photographs a two-dimensional face image of the subject using the camera device 11, and two of the subjects photographed by the face image photographing unit 2 A feature point extracting unit 3 for extracting feature points from a three-dimensional face image, a feature point of a subject extracted by the feature point extracting unit 3, and the standard application held in the corresponding standard face information holding unit 1 The feature point comparison unit 4 for comparing the feature points projected in two dimensions of the shape, and the feature points projected in two dimensions of the standard application shape and the feature points of the subject match based on the comparison result Deform standard application shape A deformation composition unit 5 for combining with the face image of the subject, and a makeup simulation unit 13 for virtually generating a state in which a desired cosmetic is applied to the face image generated by the deformation composition unit 5. . The makeup simulation unit 13 includes means for expressing the gloss and color development state of the cosmetic application surface based on the information of the three-dimensional shape when virtually generating a state where a desired cosmetic is applied. More specifically, the skin image data extraction unit 7, the cosmetic film optical data holding unit 8, and the illumination information holding unit 9 are provided. For example, the feature points are provided on the outer periphery of the pupil, the center and left and right of the lips, and the left and right of the face outline. The standard application shape may be defined in a mesh state on the standard face shape or mathematically defined by a spline function or the like.

  The bare skin image data extraction unit 7 extracts external face image data composed of a surface reflected light component reflected from the face surface and internal face image data composed of an internally reflected light component reflected from the face image of the subject. .

  The decorative film optical data holding unit 8 is a decorative film optical component composed of a surface reflected light component reflected on the surface of the decorative film, a film color component scattered and reflected inside the decorative film, and a transmissive component transmitted through the decorative film. Retain data.

  Based on the information of the three-dimensional shape of the face image of the subject synthesized by the deformation synthesis unit 5, the makeup simulation unit 13 determines the skin according to the shooting environment including the position of the illumination held in the illumination information holding unit 9. The gloss and color development state of the face image of the subject is expressed from the image data and the cosmetic film optical data.

  As shown in FIG. 2, in the standard application shape, a region where a desired cosmetic is applied to the standard face is held together with information about the three-dimensional shape. The three-dimensional shape information is, for example, normal information with respect to the surface of the face image. The normal is a straight line perpendicular to the surface of the face image, and the state of surface irregularities can be recognized by examining the direction of the normal (vector direction).

  2 (a) shows a state where makeup is applied to the eyelid, FIG. 2 (b) shows a state where makeup is applied to the eyebrows, and FIG. 2 (c) shows a state where makeup is applied to the cheeks. 2 (d) shows a state where makeup is applied to the lips, FIG. 2 (e) shows a state where makeup is applied to the forehead, and FIG. 2 (f) shows a state where the entire face is applied. The state where makeup is applied as a base is shown.

  The operator virtually generates a state in which the desired cosmetic is applied to the face image of the subject by appropriately selecting and combining the standard application shape diagrams 2 (a) to 2 (f) in consultation with the subject. Are displayed on the display unit 10.

  For example, cosmetic A is used for the eyelids, cosmetic B is used for the eyebrows, cosmetic C is used for the cheeks, cosmetic D is used for the lips, cosmetic E is used for the forehead, and cosmetics are used for the base of the entire face. When executing the makeup simulation using F, the operator designates cosmetic A for the makeup part of the standard application shape in FIG. 2 (a), and cosmetics for the makeup part of the standard application shape in FIG. 2 (b). B is designated, cosmetic C is designated for the standard application shape makeup part of FIG. 2 (c), cosmetic D is designated for the standard application shape makeup part of FIG. 2 (d), and FIG. 2 (e). The cosmetic product E is designated for the makeup part of the standard application shape, and the cosmetic product F is designated for the makeup part of the standard application shape of FIG.

  Based on this designation, the gloss / coloration state adjusting unit 6 acquires cosmetic film optical data of the designated cosmetic from the cosmetic film optical data holding unit 8, and performs facial image for each makeup part in each standard application shape. Individually adjust the gloss and color development.

  A plurality of standard application shapes are prepared according to sex, age group, face type, and the like, and the operator can select any one at the start of the makeup simulation.

  The concept of the processing flow is shown in FIG. The feature of the present invention is that, for a subject, only a two-dimensional image is taken, but by synthesizing a standard application shape having three-dimensional shape information into the subject's two-dimensional image, A face image holding a coating shape having information on the three-dimensional shape of the subject can be generated. A realistic makeup simulation result can be displayed by adjusting the gloss and coloring state of the face image that holds the application shape having the information of the three-dimensional shape of the subject thus generated. it can.

  Here, adjustment of gloss and color development will be described. In the illumination information (imaging environment), information on the angle at which the subject is illuminated is recorded, and is information that is fixedly set when the installation of the apparatus is completed.

  The bare skin image data includes a surface reflected light component reflected from the surface of the bare skin and an internal reflected light component reflected from the inside of the bare skin. The surface reflected light component is a factor that represents the gloss of the face, and is internally reflected. The light component is a factor representing the color of the face.

  The cosmetic film optical data includes a surface reflected light component reflected on the surface of the decorative film, a film color component scattered and reflected inside the decorative film, and a transmissive component transmitted through the decorative film. The light component is a factor representing the gloss of the decorative film, the film color component is a factor representing the color of the decorative film, and the transmissive component is a factor representing the appearance of the bare skin seen through the decorative film.

  Based on these various data and the three-dimensional data of the synthesized face image, a face image of a makeup simulation result with high reality is generated. The details are described in Patent Document 2.

  Next, a method for synthesizing a standard application shape having three-dimensional information on a two-dimensional image of a subject, which is a feature of the present invention, will be described with reference to FIGS. As shown in FIG. 4, the feature points of the face image of the subject are extracted (S1), compared with the feature points of the standard application shape (S2), and if there is a deviation (S3), the standard application shape is obtained. Then, deformation is performed to correct this deviation (S4), and then, it is combined with the face image of the subject (S5).

  At this time, it is only necessary to extract a small number of main feature points, so that complicated calculations can be eliminated, and a camera device for taking a three-dimensional image is not required, and it is small, simple, and lightweight. Thus, a makeup simulation apparatus with a high processing speed can be realized. In addition, using feature points that are not easily affected by changes in the brightness of an image, the robustness of position detection is increased, and complicated position adjustment is not required.

  In the following explanation, the feature points of the left and right sides of the face contour, the periphery of the pupil, the center of the lips, and the left and right sides are obtained, but the feature points of the face contour can be inferred from the feature points of the pupil and lips. The procedure for obtaining may be omitted.

  In other words, the effect of the face outline is smaller than the impression that the eyes and lips give to the person, so the face's contours are based on uniform rules such as extrapolating from the positional relationship based on the feature points of the eyes and lips. There is almost no practical problem even if the feature points of the contour are inferred.

  As shown in FIGS. 5 and 6, feature points (black circles) are set in advance in the standard application shape. The setting locations are the face outline (F01 to F04), the periphery of the pupil (RE01 to RE04, LE01 to LE04), the center of the lips (L02), and the left and right (L10, L03). As shown in FIG. 7, feature points (double circles) are similarly extracted from the face image of the subject.

  5 and 7, feature points F01 and F03 are set not only on the left and right sides of the face outline but also on the top and bottom sides of the face outline. These are the feature points F02 and F04 on the left and right sides of the face. Since the center of the lips (L02) and the left and right (L10, L03) are set, it can be easily derived from the width of the face and the position of the lips.

  As shown in FIG. 8, basic points (RE11 to RE14) for determining an outline are set for feature points (RE01 to RE04) around the pupil. As a result, the outline of the eye is determined as shown in FIG. In addition, the setting positions of the basic points (RE11 to RE14) are fixedly set in advance to the positions that are generally most beautifully viewed in relation to the setting positions of the feature points (RE01 to RE04). Although there are some individual differences for each subject, there is no difference that is problematic in practice.

  FIG. 10 shows feature points (RE01 'to RE04') of the right eye part of the subject. On the other hand, as shown in FIG. 11, the contour shown in FIG. 9 is aligned, and as shown in FIG. 12, the standard application shape is deformed to match the right eye part of the subject. The broken line in FIG. 12 is the standard application shape after deformation, and the solid line is the image of the subject. As with the right eye, the standard application shape is also deformed for the left eye.

  FIG. 13 shows the characteristic points (L01 to L03) of the lips of the standard application shape. For these feature points (L01 to L03), basic points (L11 to L18) are set in advance. As a result, the lip contour is determined as shown in FIG. In addition, the setting positions of the basic points (L11 to L18) are fixedly set in advance to the positions that are generally most beautifully seen in relation to the setting positions of the feature points (L01 to L03). In this setting, as shown in FIG. 15, the ratio of the ideal width W1, height H1, and H2 to the width W of the lips is generally obtained, and is set according to this.

  FIG. 16 shows characteristic points (L01 'to L03') of the subject's lips. On the other hand, as shown in FIG. 17, the contour shown in FIG. 14 is aligned, and as shown in FIG. 18, the standard application shape is deformed to match the lip portion of the subject. The broken line in FIG. 18 is the standard application shape after deformation, and the solid line is the image of the subject. Note that the lip width W1, height H1, and H2 are the ideal shapes shown in FIG. 15 even if they differ from the shape of the subject.

  In this way, the three-dimensional image of the standard application shape is deformed and synthesized with the two-dimensional image of the subject. Thereby, a three-dimensional face image of the subject can be generated.

(Example of the program)
A description will be given of an embodiment of a program that, when installed in a general-purpose information processing apparatus, causes the general-purpose information processing apparatus to realize a function corresponding to the function of the makeup simulation apparatus of the present embodiment.

  By recording the program of this embodiment on a recording medium, the general-purpose information processing apparatus can install the program of this embodiment using this recording medium. Alternatively, the program of this embodiment can be installed directly on the general-purpose information processing apparatus via a network from a server holding the program of this embodiment.

  Thereby, the function corresponding to the function of the makeup | decoration simulation apparatus of a present Example is realizable using a general purpose information processing apparatus.

  The program of this embodiment includes not only a program that can be directly executed by a general-purpose information processing apparatus but also a program that can be executed by installing it on a hard disk or the like. Also included are those that are compressed or encrypted.

  INDUSTRIAL APPLICABILITY The present invention can be used to realize a makeup simulation apparatus that is suitable for performing a makeup simulation at a store, is small, simple, lightweight, has a high processing speed, and has high reality. Accordingly, it is possible to provide appropriate advice on cosmetics at a store and promote sales of cosmetics.

The block block diagram of the makeup | decoration simulation apparatus of a present Example. The figure which shows the example of a standard application | coating shape. The conceptual diagram of the flow of a process. The flowchart which shows the flow of the deformation | transformation synthesis | combination of the standard application shape using a feature point. The figure which shows the feature point of a standard application | coating shape. The figure which shows the feature point of the eyes of a standard application | coating shape. The figure which shows the feature point of a test subject's face image. The figure which shows the basic point of the right eye of a standard application | coating shape. The figure which shows the outline of the right eye of a standard application | coating shape. The figure which shows the feature point of the right eye of a test subject's face image. The figure which compares the feature point of the right eye of a standard application | coating shape with the feature point of a test subject's right eye. The figure which shows the state which match | combined the outline of the right eye of the standard application | coating shape with the outline of the test subject's right eye. The figure which shows the feature point and basic point of a lip of a standard application | coating shape. The figure which shows the outline of the lip of a standard application | coating shape. The figure which shows the ideal shape with respect to the width | variety of a lip. The figure which shows the feature point of a test subject's lip. The figure which compares the feature point of the lip of a standard application | coating shape with the feature point of a test subject's lip. The figure which shows the state which match | combined the outline of the lip of the standard application | coating shape with the outline of the test subject's lip.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Standard face information holding part 2 Face image photographing part 3 Feature point extraction part 4 Feature point comparison part 5 Deformation composition part 6 Gloss / coloration state adjustment part 7 Skin image data extraction part 8 Cosmetic film optical data holding part 9 Illumination information holding part DESCRIPTION OF SYMBOLS 10 Display part 11 Camera apparatus 12 Light source 13 Makeup simulation part

Claims (5)

Standard face information holding means for holding, for a standard face image, information on the three-dimensional shape of the face, and information on a standard application shape indicating a desired cosmetic application area for the three-dimensional shape,
A face image photographing means for photographing a two-dimensional face image of the subject;
Feature point extracting means for extracting feature points from the two-dimensional face image of the subject photographed by the face image photographing means;
The feature point of the subject extracted by the feature point extraction unit is compared with the feature point projected on the two-dimensional shape of the standard application shape held in the standard face information holding unit, and the two-dimensional Deformation and synthesizing means for deforming the three-dimensional shape and the standard application shape so that the feature points projected on the subject coincide with the feature points of the subject and synthesizing with the face image of the subject;
Makeup simulation means for virtually generating a state in which a desired cosmetic is applied to the face image generated by the deformation synthesis means,
The makeup simulation means includes a means for expressing the gloss and color development state of the cosmetic application surface based on the information of the three-dimensional shape when virtually generating a state where a desired cosmetic is applied. .   The makeup simulation apparatus according to claim 1, wherein the feature points are provided on the outer periphery of the pupil, the center and left and right of the lips, and the left and right of the contour of the face.   The feature points are provided on the outer periphery of the pupil, the center and left and right of the lips, and the feature points to be provided on the left and right of the face contour are analogized based on the feature points provided on the outer periphery of the pupil, the center of the lips and on the left and right. The makeup simulation apparatus according to claim 1 provided. Means for extracting external face image data comprising a surface reflected light component reflected from the face surface of the subject and internal face image data comprising an internally reflected light component reflected internally;
And means for holding cosmetic film optical data comprising a surface reflected light component reflected on the surface of the decorative film, a film color component scattered and reflected inside the decorative film, and a transmissive component transmitted through the decorative film. ,
The means for expressing is based on the three-dimensional shape information of the face image of the subject, the face of the subject from the cosmetic film optical data, the external face image data, and the internal face image data according to the shooting environment including the position of illumination. The makeup simulation apparatus according to any one of claims 1 to 3, which expresses a glossy and colored state of an image.   A program that, when installed in a general-purpose information processing apparatus, causes the general-purpose information processing apparatus to realize a function corresponding to the makeup simulation apparatus according to any one of claims 1 to 4.

Images