JP2010179012A - Complexion measuring device, complexion measuring program, makeup simulation device and makeup simulation program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately measure the completion of a user's face from an image of the user's face photographed while the influence of environmental light is eliminated. <P>SOLUTION: This complexion measuring device 1 for measuring the complexion of a user's face includes: a photographing means 14 for photographing a complexion measuring image at such a shutter speed as to eliminate the influence of environmental light and photographing a display image at such a shutter speed as to be influenced by the environmental light; a stroboscopic means 16 for emitting light interlocking with the shutter operation of the photographing means 14; a measuring means for measuring the complexion of the user's face from the complexion measuring image influenced by the stroboscopic light, which has been photographed by the photographing means 14; and a display means 13 for displaying measurement result of the measuring means together with the display image influenced by the stroboscopic light and environmental light, which is photographed by the photographing means 14. By this configuration, the above problem is solved. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a skin color measuring device, a skin color measuring program, a makeup simulation device, and a makeup simulation program, and in particular, a skin color measuring device, a skin color measuring program, a makeup simulation device, and a makeup simulation program for measuring the skin color of a user's face. About.

  Conventionally, in order to verify the effect of the foundation according to the skin color of the user's face, cosmetics stores and the like have measured the skin color of the user's face with a device. The measurement of the skin color of the user's face is, for example, a method in which the device is brought into contact with or close to the user's face (see, for example, Patent Document 1), or a method in which the user's face is photographed and obtained from the user's face image ) Is generally known.

JP 2008-286649 A JP 2005-34355 A

  At stores, skin counseling, makeup simulation, and the like are performed using images taken at stores. However, the measurement of the skin color of the user's face by a method obtained from the photographed user's face image cannot avoid the influence of a light source (environmental light) such as a fluorescent light in a cosmetics store or the like, and the skin color of the user's face is accurately determined. Could not be measured. The technique described in Patent Document 2 uses light from a light source whose polarization plane is adjusted by transmitting through a polarizing plate, and does not solve the problem of ambient light.

  The present invention has been made in view of the above points, and a skin color measuring apparatus and a skin color measuring program capable of accurately measuring the skin color of a user's face from an image of the user's face taken by removing the influence of ambient light An object of the present invention is to provide a makeup simulation apparatus and a makeup simulation program.

  In order to solve the above-described problems, the present invention is a skin color measurement device that measures the skin color of a user's face, and captures an image for skin color measurement at a shutter speed capable of removing the influence of ambient light, and the influence of ambient light. A photographing means for photographing a display image at a shutter speed, a strobe means for emitting light in conjunction with a shutter operation of the photographing means, and a skin color measurement image influenced by the strobe light photographed by the photographing means. Measuring means for measuring the skin color of the user's face, and display means for displaying the measurement result by the measuring means together with the display image influenced by the strobe light and the environmental light taken by the photographing means.

  In addition, what applied the component, expression, or arbitrary combination of the component of this invention to a method, an apparatus, a system, a computer program, a recording medium, a data structure, etc. is also effective as an aspect of this invention.

  ADVANTAGE OF THE INVENTION According to this invention, the skin color of a user's face can be measured correctly from the image of the user's face image | photographed remove | eliminating the influence of environmental light.

It is an external view of 1st Example of the skin color measuring apparatus by this invention. It is an external view of 2nd Example of the skin color measuring apparatus by this invention. It is an external view of 3rd Example of the skin color measuring apparatus by this invention. It is an external view of 4th Example of the skin color measuring apparatus by this invention. It is a hardware block diagram of one Example of the skin color measuring apparatus by this invention. It is a hardware block diagram of the other Example of the skin color measuring apparatus by this invention. It is a process block diagram of an example showing the process sequence of the skin color measuring apparatus. It is a flowchart of an example showing the process sequence of the skin color measuring apparatus. It is an image for skin color measurement of an example influenced only by strobe light. It is an example of a display image influenced by ambient light and strobe light. It is an image for a display showing an example of a skin color measurement area. It is an image figure of an example of a skin color classification map. It is a flowchart of one Example showing the outline of the process which a makeup simulation apparatus performs. It is a transition diagram of an example of a screen displayed on a monitor. It is an example processing block diagram showing the processing procedure of the makeup simulation apparatus. It is a screen image figure showing the process which a simulator main application performs. It is an image figure of an example which represented the tracking point in the image image of the user's face contained in the image before makeup. It is a block diagram of an example of a makeup process parameter file. It is an image figure which shows the tracking point referred by foundation process. It is a flowchart of an example which shows a foundation process. It is an example image image showing an outline. It is an image figure of an example showing the original image before a blurring process, and the original image after a blurring process.

  Next, modes for carrying out the present invention will be described based on the following embodiments with reference to the drawings. The skin color measuring apparatus and makeup simulation apparatus of the present embodiment are the same in the process of measuring the appearance and the skin color of the user's face. Although the skin color measuring device will be mainly described here, the makeup simulation device is also the same.

  FIG. 1 is an external view of a first embodiment of a skin color measuring apparatus according to the present invention. FIG. 1A is a front view of the skin color measuring apparatus 1. FIG. 1B is a plan view of the skin color measuring apparatus 1. FIG. 1C is a side view of the skin color measuring apparatus 1.

  The skin color measuring device 1 has a main body part 10, an arm part 11, and a base part 12. The skin color measuring apparatus 1 has a main body 10 attached to a base 12 so as to be movable by an arm 11. The main body 10 includes a monitor 13 as display means, a camera 14 as photographing means, illumination 15 as illumination means, and a strobe 16 as strobe means. The skin color measuring apparatus 1 has a built-in computer (not shown).

  The monitor 13 is a liquid crystal display, an organic EL (organic electroluminescence) display, or the like. The camera 14 is a CMOS (Complementary Metal-Oxide Semiconductor) camera, a CCD (Charge Coupled Device) camera, or the like. Note that the camera 14 has a mechanism that allows photographing through a polarizing filter (not shown). The illumination 15 is a cold cathode tube or a light emitting diode (LED).

  The monitor 13, camera 14, illumination 15, and strobe 16 are controlled by a computer as will be described later. The strobe 16 is installed at a position where the strobe light is effectively applied to the cheek portion, which is the main measurement site for measuring the skin color of the user's face. The monitor 13 is a touch panel monitor for accepting operations of a user, a cosmetic salesperson, and the like. The operation by the user is accepted from the touch panel monitor, and the monitor 13 outputs the operation information to the computer. A mechanism for accepting an operation by a user, a cosmetic salesperson or the like is not limited to the touch panel monitor, and an operation unit may be separately provided.

  FIG. 2 is an external view of a second embodiment of the skin color measuring apparatus according to the present invention. Note that the plan view and the side view of the external view of FIG. 2 are omitted because the plan view and the side view are the same as the external view of FIG. FIG. 2 is a front view of the skin color measuring apparatus 1. The skin color measuring apparatus 1 in FIG. 2 is different from the skin color measuring apparatus 1 in FIG.

  The skin color measuring apparatus 1 in FIG. 2 is, for example, an organic EL display in which the monitor 13 can emit light. The skin color measuring apparatus 1 in FIG. 2 causes both end portions 13a of the monitor 13 to emit light, and the both end portions 13a serve as the illumination 15 in FIG. Since the other structure of the skin color measuring apparatus 1 of FIG. 2 is the same as that of the skin color measuring apparatus 1 of FIG. 1, description is abbreviate | omitted.

  FIG. 3 is an external view of a third embodiment of the skin color measuring apparatus according to the present invention. FIG. 3 is a front view of the skin color measuring apparatus 1. In the skin color measuring apparatus 1 of FIG. 3, the main body 10 has a horizontally long shape and is arranged so that two monitors 13A and 13B are arranged, and a camera 14 is arranged between the two monitors 13A and 13B. This is different from the skin color measuring apparatus 1 shown in FIG.

  In the skin color measuring apparatus 1 of FIG. 3, the camera 14 is a CCD camera or the like. The camera 14 may be a CMOS camera of a micro camera. The skin color measuring device 1 in FIG. 3 has the camera 14 positioned between the monitors 13A and 13B, so that the line of sight in the photographed user's face image is natural compared to the skin color measuring device 1 in FIG. Become. 3 is the same as that of the flesh color measuring apparatus 1 in FIG. 1, and thus the description thereof is omitted.

  FIG. 4 is an external view of a fourth embodiment of the skin color measuring apparatus according to the present invention. FIG. 4 is a front view of the skin color measuring apparatus 1. The skin color measuring apparatus 1 in FIG. 4 has a main body 10 that is horizontally long and is arranged so that two monitors 13A and 13B are arranged, and a camera 14 is arranged between the two monitors 13A and 13B. 1 is different from the skin color measuring apparatus 1 of FIG.

  The skin color measuring apparatus 1 in FIG. 4 is, for example, an organic EL display in which the monitors 13A and 13B can emit light. The skin color measuring apparatus 1 in FIG. 4 causes both end portions 13a and 13b of the monitors 13A and 13B to emit light, so that both end portions 13a and 13b serve as the illumination 15 in FIG. 4 is a CMOS camera or the like in which the camera 14 is a microminiature camera.

  The skin color measuring apparatus 1 in FIG. 4 has the camera 14 positioned between the monitors 13A and 13B, so that the line of sight in the photographed user's face image is natural compared to the skin color measuring apparatus 1 in FIG. Become. 4 is the same as that of the flesh color measuring apparatus 1 in FIG. 1, and thus the description thereof is omitted.

  FIG. 5 is a hardware configuration diagram of an embodiment of the skin color measuring apparatus according to the present invention. The hardware configuration diagram shown in FIG. 5 represents an example of the skin color measuring apparatus 1 of FIG. In addition, the same code | symbol is attached | subjected about the same part as FIG.

  The skin color measuring apparatus 1 shown in FIG. 5 includes a monitor 13, a camera 14, an illumination 15, a strobe 16, a touch panel monitor 21, a memory card I / F 22, a main storage device 24, and an arithmetic processing device 25 that are connected to each other via a bus B or the like. Have The memory card I / F 22, the memory card 23, the main storage device 24, and the arithmetic processing device 25 in FIG. 5 are the above-described computers. The memory card 23 is connected to the bus B via the memory card I / F 22.

  The memory card 23 has a flash memory that does not lose its memory even if it is not energized, and a controller circuit that handles external input / output. Further, the memory card 23 conforms to, for example, the ATA standard, and looks like an auxiliary storage device such as a hard disk from the arithmetic processing unit 25. As the memory card 23, for example, Compact Flash (registered trademark), which is one of memory card standards, can be used.

  The skin color measurement program for controlling the skin color measuring device 1 is at least a part of various programs for controlling the skin color measuring device 1. The skin color measurement program is provided by distributing the memory card 23, for example. Since the skin color measuring apparatus 1 can remove the memory card 23, the memory card 23 can be replaced to cope with version upgrades and system changes.

  When the power is turned on after the memory card 23 storing the skin color measurement program is connected to the memory card I / F, the skin color measurement device 1 reads the skin color measurement program from the memory card 23 and stores it in the main storage device 24. . The skin color measuring apparatus 1 causes the arithmetic processing unit 25 to implement various processes as described below in accordance with the skin color measuring program stored in the main storage device 24. The monitor 13, the camera 14, the illumination 15, the strobe 16, and the touch panel monitor 21 are controlled by various processes by the arithmetic processing unit 25 as described later.

  FIG. 6 is a hardware configuration diagram of another embodiment of the skin color measuring apparatus according to the present invention. The hardware configuration diagram shown in FIG. 6 represents an example of the skin color measuring apparatus 1 of FIG. In addition, the same code | symbol is attached | subjected about the same part as FIG. The skin color measuring apparatus 1 in FIG. 6 is different from the skin color measuring apparatus 1 in FIG. This is because the skin color measuring apparatus 1 in FIG. 6 emits light at both ends of the monitor 13 so that both ends serve as the illumination 15 in FIG. Since the other structure of the skin color measuring apparatus 1 of FIG. 6 is the same as that of the skin color measuring apparatus 1 of FIG. 5, description is abbreviate | omitted.

  The hardware configuration diagram of the flesh color measuring apparatus 1 in FIGS. 3 and 4 differs from the hardware configuration diagram in FIGS. 5 and 6 in that it has two monitors 13A and 13B, but the other configurations are the same. Therefore, the description is omitted.

(Skin color measuring device)
Next, the processing procedure of the skin color measuring apparatus 1 will be described. FIG. 7 is a processing block diagram of an example showing the processing procedure of the skin color measuring apparatus. FIG. 8 is a flowchart illustrating an example of the processing procedure of the skin color measuring apparatus.

  The skin color measuring apparatus 1 shown in FIG. 7 is a dedicated API 31, a reduction unit 34, a trimming unit 35, a face recognition processing unit 38, a tracking point DB 39, and a detailed part extraction realized by executing a skin color measurement program on a computer 30. A unit 40, a trimming unit 41, a skin color analysis unit 43, a skin color classification map 44, a tracking point DB 45, and a display control unit 46.

  For example, when an operation for starting skin color measurement of the user's face is received from the touch panel monitor, a dedicated API (application programming interface) 31 proceeds to step S1 and performs still image shooting processing. In the still image shooting process in step S1, the dedicated API 31 controls the camera 14 and the strobe 16 to capture the skin color measurement image 32 and the display image 33.

  Specifically, the dedicated API 31 controls the strobe 16 to emit light when the camera 14 is released. In addition, when the dedicated API 31 captures the skin color measurement image 32, the shutter speed of the camera 14 is set to a shutter speed (for example, 1/1000 second) at which the influence of ambient light can be removed.

  When the shutter speed of the camera 14 is sufficiently high, for example, 1/1000 second, the photographed skin color measurement image 32 is affected by only the strobe light without the influence of the ambient light. FIG. 9 is an example of a skin color measurement image affected only by the strobe light.

  When the dedicated API 31 captures the display image 33, the shutter speed of the camera 14 is set to a shutter speed (for example, 1/300 second) that is affected by ambient light. When the shutter speed of the camera 14 is slow, such as 1/300 second, the captured display image 33 is affected by the ambient light and strobe light. FIG. 10 is an example of a display image affected by the ambient light and strobe light.

  As described above, the skin color measurement image 32 and the display image 33 affected by the strobe light can receive a merit that variations in the color and illuminance of the strobe light are small. The setting of the camera 14 can be fixed because there is little variation in the color and illuminance of the strobe light. For example, the setting of the camera 14 is fixed in a state where white balance is achieved with a gray card during flash photography. Further, the brightness in the setting of the camera 14 is fixed so that the iris and gain values do not saturate the pixels near the skin color.

  In step S 2, the reduction unit 34 performs a reduction process on the display image 33 to obtain a reduced image 36. The face recognition processing unit 38 obtains tracking points (to be described later) for recognizing the user's face from the reduced image 36 by FFT (Fast Fourier Transform) and stores the tracking points in the tracking point DB 39. Further, the face recognition processing unit 38 sets the skin color measurement area 50 shown in FIG. 11 from the tracking points of the recognized user's face.

  FIG. 11 is an example display image showing a skin color measurement area. The skin color measurement area 50 can be changed by an external setting file. For example, the skin color measurement area 50 may be the entire skin color part of the face excluding troubles or shadows by an external setting file. In particular, since the skin color of the cheek portion is stable, the skin color measurement area 50 in FIG. 11 can be used as a reference. The face recognition processing unit 38 adds the information of the set skin color measurement area 50 to the tracking points stored in the tracking point DB 39.

  The trimming unit 35 performs a trimming process on the display image 33 to obtain a display image 37. Based on the tracking points stored in the tracking point DB 39, the detailed part extracting unit 40 extracts a detailed part from the image of the user's face included in the trimmed display image 37, and uses the type used for the facial feature diagnosis from the detailed part. Get additional information.

  The detailed part extraction unit 40 stores a new tracking point in which additional information is added to the tracking point stored in the tracking point DB 39 in the tracking point DB 45.

  In step S <b> 3, the trimming unit 41 performs a trimming process on the skin color measurement image 32 to obtain a skin color measurement image 42. The skin color analysis unit 43 obtains the tracking point and skin color measurement area 50 stored in the tracking point DB 45.

  The skin color analysis unit 43 creates a histogram using the image of the skin color measurement area (for example, a cheek) 50 of the skin color measurement image 42 based on the tracking point and the skin color measurement area 50. Further, the skin color analysis unit 43 calculates an average value of pixels for n% of the entire color from frequently used colors based on the created histogram. Note that the HSV color space (HSV value) is used for the classification of the skin color.

  In step S4, the skin color analysis unit 43 obtains the skin color classification map shown in FIG. FIG. 12 is an image diagram of an example of a skin color classification map. The skin color classification map is created by monitor tests of representative HSV values of “standard”, “light”, “dark”, “yellow”, and “red”.

  The skin color analysis unit 43 determines the positional relationship between the representative HSV value of the skin color classification map and the HSV value obtained by the skin color calculation process in step S3, and classifies the skin color by plotting it on the skin color classification map of FIG.

  In step S5, the display control unit 46 uses the display image 37 and the information on the skin color classification result obtained by the skin color analysis unit 43 to display the information on the skin color classification result obtained by the skin color analysis unit 43 and the display image 37 on the monitor 13. Display.

  The display control unit 46 is configured to display information on the skin color classification result by the skin color analysis unit 43 based on the display image 37, additional information on the tracking points stored in the tracking point DB 45, and information on the skin color classification result by the skin color analysis unit 43. Further, the information on the facial diagnosis result and the display image 37 may be displayed on the monitor 13.

  As described above, the skin color measurement device 1 can easily capture an image of the user's face by removing the influence of the ambient light, and can accurately capture the user from the image of the user's face taken by removing the influence of the ambient light. The skin color of the face can be measured.

(Makeup simulation device)
The skin color measuring function of the skin color measuring apparatus 1 described above can be used in a makeup simulation apparatus described later. Therefore, hereinafter, a makeup simulation apparatus that uses the skin color measurement function of the skin color measurement apparatus 1 will be described.

  The makeup simulation apparatus according to the present invention is based on the concept of a mirror essential item for makeup, and has an interactive function with a user. In addition, the makeup simulation apparatus according to the present invention has a feature that makes the user feel as if they are doing makeup naturally while looking at the mirror.

  The makeup simulation apparatus performs image processing on a captured moving image by a computer, thereby applying makeup to the face image of the user included in the moving image and displaying the image on the monitor 13 which is a digital mirror. The makeup simulation apparatus can also display product information and beauty information on the monitor 13.

  Next, a processing procedure of the makeup simulation apparatus will be described. FIG. 13 is a flowchart of an embodiment showing an outline of processing performed by the makeup simulation apparatus. FIG. 14 is a transition diagram of an example of a screen displayed on the monitor.

  For example, when an operation for starting skin color measurement of the user's face is received from the touch panel monitor, the makeup simulation apparatus performs a face image capturing process in step S11. The face image capturing process in step S11 is the same as step S1 in FIG. After the skin color measurement image and the display image are captured by the face image capturing process, the makeup simulation apparatus displays a screen image 61 including, for example, a display image on the monitor 13.

  Further, the makeup simulation apparatus proceeds to step S12, and performs bare skin color analysis processing corresponding to steps S2 to S5 in FIG. The makeup simulation apparatus classifies the skin color of the user's face into “standard”, “lighter”, “darker”, “yellowish”, and “redness” by bare skin color analysis processing. The makeup simulation apparatus displays a screen image 62 including information on the skin color classification result and a display screen on the monitor 13.

  Proceeding to step S13, the makeup simulation apparatus performs a recommended FD proposal process for proposing a recommended FD (foundation) from the information of the skin color classification result of step S12. The recommended FD proposal process can be realized by associating FD product information with a database (DB) for each user face color classified by the bare skin color analysis process in step S12. That is, the makeup simulation apparatus obtains recommended FD product information and a display screen by acquiring from the DB FD product information associated with the classified face color of the user based on the skin color classification result information in step S12. Is displayed on the monitor 13.

  In step S14, the makeup simulation apparatus performs a stereoscopic effect analysis process using the skin color measurement image. A known technique can be used for the stereoscopic effect analysis process. The makeup simulation apparatus displays a screen image 64 including a stereoscopic effect analysis result and a display screen on the monitor 13.

  In step S15, the makeup simulation apparatus displays on the monitor 13 a screen image 64 for selecting which product effect simulation of foundation, pre-makeup, or face color is to be performed. The screen image 65 includes, for example, a moving image of the user's face and a device such as a button for selecting which product effect simulation of foundation, pre-makeup, or face color is to be performed.

  For example, when a button for selecting the product effect simulation of the foundation is selected from the touch panel monitor, the process proceeds to step S16, and the makeup simulation apparatus performs a foundation simulation process. In the foundation simulation process, foundation makeup is applied to the user's face included in the moving image, and the screen image 66 is displayed on the monitor 13. For the foundation simulation process, a known technique (for example, JP 2007-216000 A) can be used.

  For example, when a button for selecting pre-make product effect simulation is selected from the touch panel monitor, the process proceeds to step S17, and the makeup simulation apparatus performs pre-make simulation processing. In the pre-make simulation process, pre-make-up is performed on the user's face included in the moving image, and the screen image 67 is displayed on the monitor 13. Note that a known technology (for example, Japanese Patent Application Laid-Open No. 2007-216000) can be used for the pre-makeup simulation process.

  For example, when a button for selecting face color product effect simulation is selected from the touch panel monitor, the process proceeds to step S18, and the makeup simulation apparatus performs face color simulation processing. In the face color simulation process, face color makeup is applied to the face of the user included in the moving image, and the screen image 68 is displayed on the monitor 13. The face color simulation process can use a known technique (for example, Japanese Patent Application Laid-Open No. 2007-216000).

  FIG. 15 is a processing block diagram illustrating an example of a processing procedure of the makeup simulation apparatus. The makeup simulation apparatus 100 in FIG. 15 partially has the configuration of the skin color measurement apparatus 1 shown in FIG.

  The makeup simulation apparatus 100 shown in FIG. 15 includes an analog camera 151, a USB capture device 152, a moving image file 153, a makeup camera 154, a strobe 155, a still image system 156, a shared memory 157, a simulator main application 158, an interface application 159, and a moving image. It has a file 160. Makeup camera 154 corresponds to camera 14 in FIG. The strobe 155 corresponds to the strobe 16 of FIG.

  The analog camera 151 outputs, for example, a captured NTSC moving image via the USB capture device 152. The moving image output from the USB capture device 152 is input to the simulator main application 158 using, for example, DirectX 161 as an example of an API. The moving image file 153 is also input to the simulator main application 158 using the DirectX 161.

  The makeup camera 154 outputs the captured moving image via IEEE1394 as an example of a serial interface. The moving image output from the makeup camera 154 is input to the simulator main application 158 using the dedicated API 31. The simulator main application 158 obtains a moving image resolution original image from a moving image input using the dedicated API 31, and obtains the aforementioned skin color measurement image 32 and display image 33.

  The simulator main application 158 uses the moving image and moving image file 153 input using the DirectX 161 and the moving image having the moving image resolution obtained from the moving image input using the dedicated API 31 as an original image and trimming. And reduction processing.

  The simulator main application 158 performs trimming on the original image to obtain a pre-makeup image. Further, the simulator main application 158 performs a reduction process on the original image to obtain a face recognition processing image. The face recognition processing unit 163 obtains a tracking point 164 to be described later for recognizing the user's face from the face recognition processing image by FFT.

  Based on the tracking point 164, the makeup processing unit 165 performs makeup including foundation, eyebrows, shadows, lipstick, and teak on the user's face image included in the pre-makeup image to obtain a post-makeup image. The makeup processing unit 165 includes a foundation processing unit 166, an eyebrow processing unit 167, a shadow processing unit 168, a lipstick processing unit 169, and a cheek processing unit 170.

  The makeup processing unit 165 can include product information 171 for makeup as in the image after makeup in the image after makeup. The moving image server 172 writes the pre-makeup image and the post-makeup image in the shared memory 157. The moving image server 172 can also output the pre-makeup image and the post-makeup image as a moving image file 160.

  The interface application 159 includes a moving image control object 182, a moving image display object 183, and other controllers 184 that use the ActiveX controller 180 and the ActiveX viewer 181. The interface application 159 and the simulator main application 158 are linked by ActiveX & Socket.

  The interface application 159 can display the pre-makeup image and the post-makeup image written in the supply memory 157 on the monitor 13 using the ActiveX viewer 181.

  When the simulator main application 158 receives an operation for starting skin color measurement of the user's face from the touch panel monitor, the simulator main application 158 performs a still image shooting process using the dedicated API 31. In the still image shooting process, the dedicated API 31 controls the makeup camera 154 and the strobe 155 to capture the skin color measurement image 32 and the display image 33.

  The process of measuring the skin color of the user's face after capturing the skin color measurement image 32 and the display image 33 is the same as in FIG. The simulator main application 158 outputs the display image 37, additional tracking point information stored in the tracking point DB 45, and skin color classification result information by the skin color analysis unit 43 to the still image system 156.

  The still image system 156 is based on the display image 37, additional information on the tracking point stored in the tracking point DB 45, information on the skin color classification result by the skin color analysis unit 43, and information on the skin color classification result by the skin color analysis unit 43; Information on the face diagnosis result and the display image 37 are displayed on the monitor 13.

  FIG. 16 is a screen image diagram showing processing performed by the simulator main application. A screen image 400 represents an example of displaying a pre-makeup image. A screen image 401 represents an example in which tracking points 164 obtained from the face recognition processing image are displayed superimposed on the pre-makeup image. A screen image 402 represents an example in which a post-makeup image obtained by applying makeup to the user's face included in the pre-makeup image based on the tracking point 164 is displayed.

  Hereinafter, details of the face recognition process and the makeup process among the processes performed by the simulator main application 158 will be described with reference to the drawings. For makeup processes such as foundation processing, eyebrow processing, shadow processing, lipstick processing, and cheek processing, a known technique (for example, Japanese Patent Application Laid-Open No. 2007-216000) can be used. Therefore, in this embodiment, a foundation process will be described as an example of a makeup process.

(Face recognition processing)
FIG. 17 is an image diagram of an example in which tracking points are represented in the image of the user's face included in the pre-makeup image. The face recognition processing unit 163 of the simulator main application 158 obtains 45 tracking points 164 for recognizing the user's face from the image of the user's face. The tracking point 164 in FIG. 17 is an example, and may be adjusted according to the processing capability of the makeup simulation apparatus 100 and the definition of the monitor 13.

  In this way, by obtaining the tracking point 164 from the user's face included in the pre-makeup image, the makeup processing unit 165 associates the tracking point 164 with the makeup method and color as shown in FIG. It can be set in the processing parameter file.

FIG. 18 is a configuration diagram of an example of a makeup process parameter file. The makeup process parameter file in FIG. 18 is a file in which makeup methods and colors are set in association with tracking points 164 for each eye, mouth, cheek, and the like. The makeup process parameter file is set for each makeup pattern (image).

(Foundation treatment)
As shown in FIG. 19, the foundation processing unit 166 included in the simulator main application 158 refers to the right and left eye corners, one eye and one eyebrow, and the tracking points 164 between the eyes and the center of the nose to perform foundation processing. FIG. 19 is an image diagram showing tracking points referred to by the foundation process.

  FIG. 20 is a flowchart illustrating an example of the foundation process. The foundation process includes a contour generation process in step S101, a blurring process on the target image in step S102, and an image pasting process in step S103. Note that the processing of steps S101 to S103 is repeated for the target area.

  Proceeding to the contour generation processing in step S101, the foundation processing unit 166 generates three types (four locations) of forehead, nose, and cheek (left and right) shown in the image of FIG. FIG. 21 is an example of an image representing an outline. It should be noted that the size, position, etc. of each contour may be designated by a GUI.

  Proceeding to the blurring process on the target image in step S102, the foundation processing unit 166 performs the blurring process on the target image corresponding to the generated contour as shown in FIG. FIG. 22 is an image diagram illustrating an example of an original image before blurring processing and an original image after blurring processing. By the blurring process on the target image, the foundation processing unit 166 can smooth the fine roughness of the skin.

  Proceeding to the image pasting process in step S103, the foundation processing unit 166 pastes the target image after the blurring process on the three types (four locations) of the forehead, nose, and cheek (left and right) contours of the pre-makeup image, Get an image after makeup.

  As described above, according to the present invention, makeup can be accurately applied to a user's face included in a moving image with a small processing load. In this way, the makeup simulation apparatus 100 can apply makeup to the image of the user's face included in the captured moving image and display it on the monitor 13 which is a digital mirror, and removes the influence of ambient light. Thus, an image of the user's face can be easily taken, and the skin color of the user's face can be accurately measured from the image of the user's face taken by removing the influence of ambient light.

  Note that the present invention is not limited to the specifically disclosed embodiments, and various modifications and changes can be made without departing from the scope of the claims.

DESCRIPTION OF SYMBOLS 1 Skin color measuring apparatus 10 Main body part 11 Arm part 12 Base part 13, 13A, 13B Monitor 13a, 13b Both end part 14 Camera 15 Illumination 16 Strobe 21 Touch panel monitor 22 Memory card I / F
23 Memory Card 24 Main Storage Device 25 Arithmetic Processing Device 30 Computer 31 Dedicated API
34 Reduction unit 35, 41 Trimming unit 38 Face recognition processing unit 39, 45 Tracking point DB
40 Detailed Part Extraction Unit 43 Skin Color Analysis Unit 44 Skin Color Classification Map 46 Display Control Unit 100 Makeup Simulation Device

Claims (8)

A skin color measuring device for measuring the skin color of a user's face,
A photographing means for photographing a skin color measurement image at a shutter speed capable of removing the influence of ambient light, and photographing a display image at a shutter speed affected by ambient light;
Strobe means for emitting light in conjunction with the shutter operation of the photographing means;
Measuring means for measuring the skin color of the user's face from the skin color measurement image affected by the strobe light photographed by the photographing means;
A skin color measuring apparatus comprising: display means for displaying a measurement result by the measuring means together with the display image influenced by strobe light and environmental light taken by the photographing means.   The skin color measuring apparatus according to claim 1, wherein the photographing unit captures the skin color measuring image through a polarizing filter.   2. The skin color measuring device according to claim 1, further comprising illumination means including a cold cathode tube or a light emitting diode for illuminating the face of the user.   The skin color measuring apparatus according to claim 1, wherein the display means includes an organic EL (organic electroluminescence) display, and the display means causes a part of the organic EL display to function as an illumination means for illuminating a user's face. .   The skin color measuring device according to claim 1, wherein the display means is arranged with two displays arranged side by side, and the photographing means is arranged between the two displays. A skin color measuring device that includes a photographing unit, a strobe unit, a display unit, and a computer, and measures a skin color of a user's face.
A shooting control unit that causes the computer to shoot a skin color measurement image at a shutter speed capable of removing the influence of ambient light and to shoot a display image at a shutter speed affected by the ambient light;
Strobe control means for causing the computer to emit light in conjunction with the shutter operation of the photographing means;
Measuring means for measuring the skin color of the user's face from the skin color measurement image affected by the strobe light captured by the photographing means;
Skin color measurement for causing the computer to function as display control means for causing the display means to display the measurement result by the measurement means together with the display image influenced by the strobe light and the environmental light taken by the photographing means. program. A makeup simulation apparatus having a skin color measurement function for measuring a skin color of a user's face,
A photographing means for photographing a skin color measurement image at a shutter speed capable of removing the influence of ambient light, and photographing a display image at a shutter speed affected by ambient light;
Strobe means for emitting light in conjunction with the shutter operation of the photographing means;
Measuring means for measuring the skin color of the user's face from the skin color measurement image affected by the strobe light photographed by the photographing means;
Display means for displaying the measurement result by the measurement means together with the display image influenced by the strobe light and the environmental light photographed by the photographing means;
The user's face image included in the display image captured by the photographing unit is recognized based on a tracking point, and a makeup process is performed on the user's face image included in the display image and displayed on the display unit. A makeup simulation apparatus having makeup processing means. A makeup simulation apparatus having a skin color measuring function for measuring a skin color of a user's face, which includes a photographing unit, a strobe unit, a display unit, and a computer.
A shooting control unit that causes the computer to shoot a skin color measurement image at a shutter speed capable of removing the influence of ambient light and to shoot a display image at a shutter speed affected by the ambient light;
Strobe control means for causing the computer to emit light in conjunction with the shutter operation of the photographing means;
Measuring means for measuring the skin color of the user's face from the skin color measurement image affected by the strobe light captured by the photographing means;
Display control means for causing the computer to display the measurement result by the measuring means on the display means together with the display image influenced by the strobe light and the environmental light taken by the photographing means;
The computer recognizes a user's face image included in the display image captured by the photographing unit based on a tracking point, performs a makeup process on the user's face image included in the display image, A makeup simulation program for causing the display means to function as a makeup processing means.

Images