JP2013158481A - Electronic mirror and program of the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an electronic mirror or the like capable of objectively evaluating a makeup result and showing it to a user while suppressing power consumption.
A display unit 50 for displaying an image, an imaging unit 20 provided integrally with the display unit, and a user's face based on an original image including a user's face captured by the imaging unit. If the symmetry of the makeup performed by the user is judged by comparing at least one of the brightness and color of the right half and the left half of the image, the image based on the original image and the makeup are And control means 40 for causing the display means to display a warning indicating that it is asymmetric.
[Selection] Figure 1

Description

  The present invention relates to an electronic mirror and a program thereof.

  With the progress of an aging society, the number of elderly people is increasing, and it is desirable to provide auxiliary functions for elderly people for things used in daily life. For example, an elderly female with presbyopia may take off glasses during makeup, but makeup may fail if the face in the mirror appears blurred. Therefore, it is useful to give the mirror a function of objectively evaluating the makeup result.

  For example, the apparatus according to the invention of Patent Document 1 can perform objective evaluation of makeup results by extracting facial elements such as eyebrows, eyes, lips, and nose and evaluating the symmetry of the elements. it can.

JP 2009-213751 A JP 2004-180047 A

  Here, an electronic mirror has been proposed in place of a commonly used mirror (generally made of glass that reflects visible light). For example, in the electronic mirror according to the invention of Patent Document 2, an image photographed by an imaging device is reversed left and right and reflected on a liquid crystal display device in real time. Therefore, the electronic mirror can be used in the same way as a commonly used mirror.

  For example, if the device according to the invention of Patent Document 1 is incorporated in such an electronic mirror, an elderly person can immediately notice a makeup failure. However, the invention of Patent Document 1 evaluates the symmetry of facial elements and the like assuming a still image. Therefore, when the image processing of Patent Document 1 is performed in an electronic mirror that displays an image in real time, a CPU having a very high processing capability and a stable power supply are required.

  However, electronic mirrors are routinely used in makeup, etc., instead of commonly used mirrors. Considering the case where the makeup is retouched on the go, for example, a battery-driven electronic mirror incorporated in a compact case may be used. Then, incorporating the device according to the invention of Patent Document 1 into a battery-powered electronic mirror is not realistic from the viewpoint of, for example, the stability of power supply and power consumption.

  The present invention has been made in view of such problems. According to some aspects of the present invention, an electronic mirror or the like that can objectively evaluate a makeup result and show it to a user while suppressing power consumption is provided.

(1) The present invention is an electronic mirror that includes a display unit that displays an image, an imaging unit that is provided integrally with the display unit, and an original image that includes a user's face captured by the imaging unit. On the basis of the brightness and color of the right half and the left half of the user's face to determine the symmetry of the makeup performed by the user. Control means for causing the display means to display an image based on the image and a warning that the makeup is asymmetric.

  The electronic mirror of the present invention compares at least one of the brightness and color of the right half and the left half of the user's face based on the original image including the user's face. Here, makeup is applied symmetrically on the right half and the left half of the face. Therefore, the electronic mirror of the present invention determines that the makeup is asymmetric when the compared brightness, color, or both are significantly different between the right half and the left half of the face, and displays a warning on the display means. . At this time, since the electronic mirror of the present invention does not perform image processing for extracting facial elements such as eyebrows, eyes, lips, and nose, processing time can be shortened, and power consumption increases by image processing for extracting elements. Can be avoided. Then, an objective evaluation of the makeup result can be performed and shown to the user.

(2) The electronic mirror includes a storage unit that stores feature data that is a feature of a user's face, and the control unit corrects an image based on the original image based on the feature data, and May determine the symmetry of the makeup performed.

(3) In this electronic mirror, when the control unit determines that the makeup performed by the user is asymmetric based on an image before the correction based on the feature data is performed, the correction based on the feature data is performed. To determine the symmetry again.

  According to these inventions, the control unit reads the feature data that is the feature of the user's face from the storage unit and corrects the image. And a control means judges the symmetry of makeup based on the image after correction. At this time, the feature data is, for example, data representing the position, size, color, etc. of the user's moles, scratches, spots, etc., and is used by the control means so as not to affect the determination of symmetry. The For example, based on the feature data, image processing is performed such that a mole or scratch is replaced with a surrounding color. Therefore, it is possible to accurately detect only asymmetry due to uneven makeup or forgetting makeup.

  Here, only when it is determined that the control means is asymmetric, the control means may read the feature data that is the feature of the user's face from the storage means and correct the image. For example, when the foundation is applied to the entire face, small spots may not be visually recognized. At this time, if the control means corrects the image based on the feature data, an accurate detection result may not be obtained. Therefore, the influence of moles, scratches, stains and the like, which are the features of the user's face, is removed only when it is determined that the image is asymmetric once. Then, it becomes possible to suppress power consumption rather than always correcting an image based on feature data.

(4) In this electronic mirror, the control means acquires environmental information, which is information on the environment at the location where the electronic mirror is used, based on the original image, and performs color correction based on the environmental information on the original image. The symmetry of the makeup performed by the user may be determined.

  According to the present invention, since the image is corrected based on the environment information and then the symmetry of the makeup performed by the user is determined, a more accurate detection result can be obtained. The color correction of the image based on the environment information is, for example, correction for removing the shadow of the user's face. In this example, it can be avoided that the control means erroneously determines asymmetry due to the influence of the shadow.

(5) The electronic mirror includes an instruction input unit that receives an instruction from the user and generates an instruction signal. The control unit is configured to generate a right half and a left half of the user's face based on the instruction signal. The area of the image used to compare the two may be divided into an upper part and a lower part, and the symmetry of the makeup performed by the user on at least one of the upper part and the lower part may be determined.

  According to the present invention, the control means determines the left-right symmetry of the makeup by dividing the image area into two parts in the vertical direction in accordance with an instruction from the user. At this time, the left / right symmetry may be determined at the upper part and the left / right symmetry may be determined at the lower part, or the left / right symmetry may be determined only from the upper or lower region. At this time, for example, in the determination of the left-right symmetry, it is possible to determine whether there is makeup unevenness or forgetting makeup for the makeup of the eyes, eyebrows, and forehead. On the other hand, for example, in the determination of the left / right symmetry of the lower part, it is possible to determine whether or not makeup unevenness or forgetting to make up for the makeup on the cheeks and mouth.

  At this time, it can be shown more precisely to the user where the makeup unevenness is. For example, if a warning to the effect that there is uneven makeup at the mouth is displayed on the display means, the user can immediately correct the makeup and the convenience of the electronic mirror is enhanced. Further, the process of dividing the image area into two in the vertical direction may be executed a plurality of times in accordance with a user instruction. At this time, it is possible to specify a portion with more uneven makeup.

(6) The present invention is a program for an electronic mirror including display means for displaying an image and imaging means provided integrally with the display means, the user's face imaged by the imaging means. When judging the symmetry of the makeup performed by the user by comparing the brightness and color of the right and left halves of the user's face based on the included original image, and judging that it is asymmetric In addition, the computer is caused to function as control means for causing the display means to display an image based on the original image and a warning that makeup is asymmetric.

  According to the present invention, the symmetry of the makeup performed by the user is determined by the program. At this time, for example, an object (brightness, color) used for comparison can be easily adjusted by program change. Therefore, it is possible to provide an electronic mirror that can reduce design constraints and can be changed flexibly.

The block diagram of the electronic mirror in 1st Embodiment. 2A to 2B are external views of an electronic mirror. 3A to 3D are diagrams illustrating examples of image correction and adjustment in an electronic mirror. FIG. 4A to FIG. 4B are diagrams illustrating a left / right reversing function. FIG. 5A to FIG. 5B are diagrams illustrating regions used for determining symmetry. FIG. 6A to FIG. 6B are diagrams illustrating divided areas used for determining symmetry. The figure explaining the area | region further divided | segmented used for determination of symmetry. The flowchart in 1st Embodiment. The flowchart of a subroutine. The block diagram of the electronic mirror in 2nd Embodiment. The external view of the electronic mirror in 2nd Embodiment. The flowchart in 2nd Embodiment. The flowchart of a subroutine.

1. First embodiment 1.1. Configuration of Electron Mirror A first embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a diagram showing the configuration of the electronic mirror of this embodiment.

  The electronic mirror 10 of this embodiment includes a camera 20, a CPU 40, and an LCD 50. The camera 20 corresponds to an imaging unit, the CPU 40 corresponds to a control unit, and the LCD 50 corresponds to a display unit. The electronic mirror 10 may include the memory block 30 and the input device 70 as in the present embodiment. The memory block 30 corresponds to storage means, and the input device 70 corresponds to instruction input means.

  The camera 20 may include, for example, a CCD image sensor (Charge Coupled Device Image Sensor), a CIS (Complementary Metal Oxide Semiconductor Image Sensor), and other image sensors. Moreover, it may be comprised not only in one but in multiple.

  The camera 20 converts an image including the photographed user's face (hereinafter referred to as an original image) into an image signal 120 and outputs the image signal 120 to the CPU 40. The image signal 120 may be, for example, an RGB signal, a YCbCr (YPbPr) signal, or other formats.

  Here, the image photographed by the camera 20 of the present embodiment is a projection of the front of the LCD 50. As shown later, the camera 20 is integrally provided in or near the LCD 50 corresponding to the mirror of the electronic mirror 10. Since the user is present in front of the LCD 50 when the electronic mirror 10 is used, the user is reflected on the LCD 50 as if reflected.

  The memory block 30 may include, for example, a random access memory (RAM 32) and a read only memory (ROM 34). The RAM 32 may be a DRAM or an SRAM. The ROM 34 may be a mask ROM or a programmable ROM such as an EEPROM (Electronically Erasable and Programmable Read Only Memory) or a flash ROM.

  The memory block 30 stores at least feature data of the user of the electronic mirror 10. Feature data is information that collects feature points to identify individual users. For example, the relative position of eyes, nose, mouth, etc. on the face, the position and size of moles, scratches, and spots, It may be a color or the like. For example, the characteristic data may be stored in the ROM 34 or may be written in the RAM 32 when the electronic mirror 10 is activated.

  Instead of the memory block 30, a large capacity storage device such as a hard disk may be used. At this time, feature data of many people can be stored. Further, there is a database composed of a hard disk or the like that stores feature data, and the memory block 30 may read the feature data when the electronic mirror 10 is activated. At this time, the memory block 30 may read the feature data from the database via the Internet.

  The CPU 40 performs image processing on the original image to generate an output image signal 140 that is an original image or an image signal after image processing. The image processing performed by the CPU 40 may adjust at least one of brightness and color, for example. Further, for example, the CPU 40 may receive the user characteristic data as the read data 130 and execute image processing for removing moles and scratches. In addition, for example, the CPU 40 may grasp the luminance and color distribution of the original image and execute image processing for removing shadows. The CPU 40 may generate the output image signal 140 for an image for which all image processing has been executed, or may generate the output image signal 140 for an intermediate image for which some image processing has been executed. Further, image enlargement / reduction may be executed.

  The CPU 40 also compares at least one of the brightness and color of the right half and the left half of the user's face based on an image on which some image processing has been performed. Then, the symmetry of the makeup performed by the user is determined. If it is determined that the makeup is asymmetric, a message warning the makeup asymmetry displayed on the LCD 50 is generated and output together with the output image signal 140. In order to accurately determine the symmetry of the makeup, performing image processing on the image acquired by the CPU 40 to generate a new image is also expressed as correcting the image. Further, the method of warning the makeup asymmetry is not limited to the message on the LCD 50 as in the present embodiment, and may be, for example, lighting of a lamp, display or blinking of an icon, generation of a sound, or the like.

  Here, the CPU 40 may store, for example, intermediate data at the time of image processing, feature data updated using the input device 70, and the like as write data 142 in the RAM 32. In addition, data stored in the RAM 32 and ROM 34 may be received as read data 130 as necessary.

  The LCD 50 displays an image subjected to image processing by the CPU 40 based on the output image signal 140. The image displayed on the LCD 50 may be enlarged or reduced, for example, compared to the original image. Further, the image displayed on the LCD 50 may be adjusted in brightness, contrast, color temperature, and the like as compared with the original image. Then, the LCD 50 displays a message warning about the makeup asymmetry.

  Note that the image displayed on the LCD 50 may be displayed horizontally reversed so that the user can use it in the same manner as a normal mirror. At this time, as in the present embodiment, the CPU 40 may invert the left and right as image processing, or the LCD 50 may invert the right and left by the display adjustment function instead of the CPU 40.

  The electronic mirror 10 according to the present embodiment includes an input device 70. The input device 70 outputs an instruction signal 170 in accordance with an instruction from the user. The CPU 40 receives the instruction signal 170 and sets, for example, a specified magnification, or turns the electronic mirror 10 into an on state (operation state) or an off state (stop state). In the present embodiment, the instruction from the user based on the instruction signal 170 is preferentially executed. That is, the CPU 40 may display the automatically enlarged / reduced image on the LCD 50, but when the user gives a magnification change instruction, the CPU 40 changes the magnification according to the user's instruction.

  Here, the input device 70 may include, for example, a button 72 and a slide switch 74. For example, the button 72 may be a main power button for switching the electronic mirror 10 between an on state and an off state. The slide switch 74 may be used by the user to manually set the magnification. There may be a plurality of buttons 72 and slide switches 74. For example, there is another button outside the figure, which may be used to turn on / off the horizontally reversed display of the LCD 50, or used when the user wants to know detailed information on the symmetry of makeup. May be.

  Although not shown in the figure, the electronic mirror 10 includes a power supply device that supplies power. Although the electronic mirror 10 may be operated by a household power source, it is assumed that the electronic mirror 10 is operated by a battery in the present embodiment. The battery may be a primary battery or a secondary battery.

1.2. External Appearance of Electronic Mirror FIG. 2A is an external view of the electronic mirror 10 of the present embodiment. The electronic mirror 10 has an appearance in which an LCD 50 that displays an image is surrounded by a frame 3. The frame 3 is provided with a button 72A, a button 72B, and a slide switch 74 having a movable section 75. Here, the button 72A is a main power button for switching the electronic mirror 10 between an on state and an off state, and the button 72B is a function button used when the user wants to know detailed information on the symmetry of makeup.

  Here, C in FIG. 2A is the center of the LCD 50. The camera 20 may be embedded at this position, but processing such as making a hole in the LCD 50 is necessary. In the electronic mirror 10 of the present embodiment, a camera 20 is embedded in the upper part of the frame 3. Here, the camera 20 exists on the vertical center line 5 passing through the center C of the LCD 50. Therefore, the user's face can be grasped almost from the front. The position of the camera 20 is not limited to the example in FIG. Further, the camera 20 may be removable, and the user may be able to project the back of the head or the top of the head, for example.

  The electronic mirror 10 of the present embodiment is battery-driven, but has a shape suitable for being installed at a specific place in a room. Here, in consideration of the case where makeup is retouched on the go, for example, a battery-driven electronic mirror incorporated in a compact case is convenient. FIG. 2B is an external view of the electronic mirror 10A. The electronic mirror 10A is operated by a battery and includes the hinge 51 so that the LCD 50 can be folded and carried around. Note that the same elements as those in FIG. 2A are denoted by the same reference numerals, and description of other elements is omitted.

  In particular, in the case of a small shape such as the electronic mirror 10A, it is difficult to use a large battery, and for example, a button-type battery may be used. At this time, in order to make the battery last longer, it is effective to reduce the power consumption accompanying the image processing executed by the CPU 40. Therefore, when the electronic mirror is provided with a function for objectively evaluating the makeup result, it is important to simplify the image processing necessary for the evaluation as much as possible to reduce the burden on the CPU 40.

  Hereinafter, with respect to the electronic mirror 10 of the present embodiment, image processing (image correction), determination method, and flow executed to objectively evaluate the presence or absence of makeup failure such as makeup unevenness or forgetting makeup will be described. . Note that the block diagram of the electronic mirror 10A is the same as that of the electronic mirror 10 of the present embodiment, and the following processing of the CPU 40 is common to the electronic mirror 10A, so only the electronic mirror 10 will be described.

1.3. Image Correction and Display Adjustment in Electronic Mirror FIG. 3A to FIG. 4B are diagrams for explaining image correction performed by the electronic mirror 10 to objectively evaluate the makeup result. In addition, the same code | symbol is attached | subjected to the same element as FIGS. 1-2 (B), and description is abbreviate | omitted. 3A to 4B are diagrams for explaining the image processing, and these images are not always displayed on the LCD 50. FIG.

  FIG. 3A shows a state in which an image (original image) including the user's face imaged by the camera 20 is displayed on the LCD 50. At this time, a shadow (shadow 77) is generated on the face by the ceiling lamp 78. Here, due to the presence of the shadow 77, it may not be possible to correctly determine the presence or absence of makeup unevenness. The CPU 40 of the electronic mirror 10 of the present embodiment performs image processing for removing such a shadow 77, and generates a corrected image as shown in FIG.

  Here, to remove the shadow 77, the CPU may execute the following method. The CPU obtains a hue value and a luminance value indicating the color and brightness of the original image, respectively. Then, it is determined that the boundary where the hue value and the luminance value are changing simultaneously is a boundary between objects or parts, and the boundary where only the luminance value is changed is a boundary where the shadow 77 is created.

  Further, the luminance value distribution of the original image is statistically processed to recognize the position and direction of the light source (in this case, the lamp 78). Then, for example, a dark portion near the nose or the like in the direction opposite to the light source is determined to be the shadow 77. At this time, the shadow 77 can be detected more accurately by considering the positional relationship with the light source. The shadow 77 does not affect the determination of the symmetry of the makeup. At this time, as shown in FIG. 3B, the CPU 40 may execute a shadow removal that replaces the shadow 77 with a surrounding color, for example, and the hue value and the luminance value of the pixel included in the shadow 77 are the makeup values. It may not be used in the determination of symmetry.

  For the image from which the shadow 77 is removed as shown in FIG. 3B, the CPU 40 may adjust the brightness to generate a corrected image as shown in FIG. For example, the brightness of the image in FIG. 3C may be a brightness that is easy to see for the user to correct makeup, or to generate an image from which moles, scratches, etc. are removed based on the feature data. It may be a suitable brightness.

  Further, the CPU 40 may execute enlargement / reduction according to the instruction signal when the user manually adjusts the magnification. FIG. 3D shows a change in the image when the user gives an enlargement instruction by moving the movable section 75 of the slide switch 74.

  FIG. 4A to FIG. 4B are diagrams for explaining the left / right reversing function of the present embodiment. The electronic mirror 10 displays on the LCD 50 an image that is reversed left and right with respect to the original image for the purpose of being used in the same manner as a normal mirror. FIG. 4A is an image in which left and right are reversed (the left / right reversing function is in an on state), and the user can perform makeup and the like in the same manner as a normal mirror.

  On the other hand, FIG. 4B is an image in which the left and right are not reversed (the horizontal reversal function is in an off state), and the user can confirm how he / she can see from another person's viewpoint. it can. The CPU 40 may execute switching between the display in FIG. 4A and the display in FIG. 4B as one of the image processes.

1.4. Objective Evaluation of Makeup Result FIGS. 5A to 7 are diagrams for explaining a method in which the electronic mirror 10 of the present embodiment objectively evaluates the makeup result. In addition, before objectively evaluating the makeup result, it is assumed that removal of shadows and adjustment of magnification (enlargement / reduction) are appropriately performed by the image processing. In addition, the same code | symbol is attached | subjected to the same element as FIGS. 1-4 (B), and description is abbreviate | omitted.

  The electronic mirror 10 according to the present embodiment determines whether or not there is a failure in the makeup result by comparing at least one of hue and brightness between the right half and the left half of the face. In general, if the right half and the left half of the face are symmetrical, makeup is performed correctly. On the other hand, when only one eyebrow is put on (forgetting to make up) or uneven makeup is created in part, at least one of color and brightness differs greatly between the right half and the left half of the face. At this time, it can be determined that the makeup has failed.

  Here, if the area of the right half and the left half of the face used for comparison are made small, the facial elements such as eyebrows, eyes, lips, and nose are extracted, and the degree of symmetry of the elements is evaluated to the same extent. Processing time is required and power consumption increases. Therefore, as in the following example, in this embodiment, comparison is made in a large area, and only when there is a user request, the area is narrowed and detailed analysis is performed. Therefore, the makeup result can be objectively evaluated while suppressing power consumption.

  In FIG. 5A, the electronic mirror 10 determines that the makeup of the user is asymmetric on the left and right, and displays an image showing the user on the display means together with a warning display 98 that the makeup is asymmetric. Represents the situation. The user can recognize from the warning display 98 that the makeup of the right half and the left half of the face is not symmetrical. In this example, the left cheek makeup has unevenness (makeup unevenness 90), and since the hue is different from that of the right cheek, a warning display 98 indicating a left-right asymmetry is displayed.

  At this time, the CPU determines the symmetry of the makeup by the following process. First, the CPU reads feature data from a memory block (see FIG. 1), extracts feature points of the user's face in the acquired image, and identifies the user by comparing with feature data information. Here, the memory block may store characteristic data of one specific person, but it is assumed that the electronic mirror 10 of the present embodiment stores characteristic data of a plurality of persons.

  As a method for specifying the user, a currently known general face identification method may be adopted. For example, facial feature points such as eyebrows and eye endpoints are extracted from the acquired image, and the distance between them (the distance between facial feature points) is measured. Then, the matching data may be selected by comparing with the characteristic data of a plurality of users.

Thereafter CPU obtains the boundary L ₁ of the vertical direction through the nose of the identified user. Boundary L ₁ is the boundary line separating the right and left halves of the face of the user in the image. Then, a horizontal boundary line L _T and a boundary line L _B perpendicular to the boundary line L ₁ are obtained. Boundary L _T, for example 5 may be determined by the hairline hair as in the example of (A). Then, the boundary line L _B, for example 5 may be determined by the jaws of the previous as in the example of (A).

FIG. 5B shows the boundary line L ₁ , the boundary line L _T , and the boundary line L _B taken out from the LCD 50 shown in FIG. The display of images such as the user's face is omitted. In FIG. 5B, an area including the left half of the user's face surrounded by these boundary lines and the black frame on the left side of the LCD 50 is an area A1. In FIG. 5B, an area including the right half of the user's face surrounded by these boundary lines and the black frame on the right side of the LCD 50 is an area A2. The CPU averages the brightness and color of the pixels included in the area A1 (in this example, the luminance value and the hue value, respectively), and the average value of the luminance values and hue values of the pixels included in the area A2. And the symmetry of the makeup may be determined. The comparison may be performed for at least one of the luminance value and the hue value. Further, pixel values are represented by RGB, YCbCr, or the like, and comparison may be executed for some or all of the colors.

  Here, an element peculiar to the user's face such as the mole 99 may affect the determination of the symmetry of the makeup. Therefore, the CPU generates an image from which moles, scratches, etc. are removed based on the feature data, and then extracts makeup data that is a color and brightness distribution caused by makeup, and executes the above comparison. Good. In addition, not only moles and scratches, but also face data (face data) that has not been applied by the specified user may be stored in the memory block. The CPU may read the face data and extract makeup data based on the face data.

In the above description, it has been described that there is no influence of the background. However, in practice, the background (particularly a uniform color, a background that is not bright) may affect the determination of the symmetry of the makeup. For this reason, the user's face is enlarged to such an extent that the background cannot be seen on the LCD 50, or the background that does not include the user's face from the area (area A1 and area A2 in FIG. 5B) based on the face outline. Processing for excluding the portion may be performed. Specifically, a boundary line L _L (not shown) and a boundary line L _R (not shown) that are parallel to the boundary line L ₁ at the positions of the user's left ear and right ear are obtained. A region surrounded by the boundary line L _L , the boundary line L _T , the boundary line L ₁ , and the boundary line L _B is defined as a region A1, and the boundary line L ₁ , the boundary line L _T , the boundary line L _R , and the boundary line L _B The comparison may be executed with the region surrounded by the region A2.

1.5. Objective evaluation of makeup results (details)
As described above, the electronic mirror 10 of the present embodiment determines the left-right symmetry of the user's makeup, and displays a warning that there is a makeup failure when the left-right asymmetry is asymmetric. However, there are cases where the user does not know which part to fix by just displaying the warning of left-right asymmetry.

  FIG. 6A to FIG. 7 are diagrams for explaining processing for dividing an area more finely and displaying an appropriate warning to the user in order to specify a place where the user should correct makeup. . In addition, the same code | symbol is attached | subjected about the same element as FIG. 5 (A)-FIG. 5 (B), and description is abbreviate | omitted.

  FIG. 6A shows an example in which there is a makeup unevenness 92 in the eye shadow portion of the user's right eye. Here, the warning display 98 is displayed as “eye is asymmetrical”, and the user can notice the failure of makeup of the eye shadow.

5 Compared to (A), in the middle of the boundary line L _T and border L _B, the boundary line L ₂ in parallel thereto have been added. The boundary line L _2, further divided into the regions A1 and A2 in FIG. 5 (B), if there is asymmetry in left and right areas of the upper, and a warning display 98, such as in FIG. 6 (A), It is possible to more specifically indicate a part to be repaired to the user.

Also in this example, first, as shown in FIGS. 5A to 5B, the area A1 and the area A2 are compared with each other, and a warning display 98 “asymmetrical” is displayed. Thereafter, the user by pressing the button 72B, the CPU as shown in FIG. 6 (A) to add a border L _2, further 2 divides the area.

FIG. 6B shows a region in this case. The boundary line _{L 2,} the upper region A11 and the region A21 is made. In addition, lower regions A12 and A22 are also formed. For example, the CPU first compares at least one of brightness and color for the upper region A11 and region A21. When it is determined that the makeup is asymmetrical, a warning display 98 that “the eye is asymmetrical” is performed as shown in FIG.

  Thereafter, when the user presses the button 72B, the CPU compares at least one of brightness and color for the lower area A12 and area A22. When it is determined that the makeup is left-right asymmetric, for example, a warning display 98 that “the mouth is left-right asymmetric” is performed. In this manner, the user can know whether the left-right asymmetry of the makeup is in the eyes, in the mouth, or in both, by the warning display 98, and it is possible to specify the place where the makeup is to be repaired. is there.

  At this time, the CPU does not automatically determine the left-right symmetry of the upper and lower divided areas, but only divides the area when the user gives an instruction. Therefore, detailed determination of left-right symmetry is automatically executed, and as a result, there is no problem that power consumption increases. For example, when the user does not press the button 72B for a certain period of time, the warning display 98 may disappear and the detailed left-right symmetry determination may not be performed.

  Here, the detailed determination of left-right symmetry performed by dividing an area may be performed in multiple stages. For example, FIG. 7 shows a state in which the area is further divided according to a user instruction after being divided as shown in FIG.

At this time, a boundary line L ₃ and a boundary line L ₄ are further added and divided into, for example, eight regions A111 to A222. For example, if the comparison between the area A111 and the area A211 is asymmetrical, a warning display 98 “Forehead is asymmetrical” is displayed. Further, for example, if the comparison between the area A121 and the area A221 is asymmetrical, a warning display 98 “cheek is asymmetrical left and right” is displayed. Therefore, the user can more easily specify the place where the make-up is performed.

1.6. Flowchart FIGS. 8 to 9 show the control performed by the CPU included in the electronic mirror of this embodiment. For example, the CPU may read a program stored in the ROM and execute the control described below according to the program.

  As shown in FIG. 8, the CPU acquires a user image (original image) from the camera (S2). Then, the position of the light source is specified from the brightness distribution of the original image (S4). Then, the shadow on the user's face is identified and removed from the change in brightness and color (S6). At this time, by calculating the position with respect to the light source, it is possible to confirm whether it is a real shadow and carefully remove the shadow.

The CPU acquires feature data from the memory block (S8) and specifies the user of the original image (S10). Then, set the vertical center line passing through the nose of the user (e.g., corresponding to the boundary line L ₁ in FIG. 5 (A)), defines the right half and the left half area of the face (S12). Then, moles, scratches and the like are removed based on the specified feature data of the user (S14), and makeup data is extracted (S16).

  After that, based on makeup data that is a color and brightness distribution resulting from makeup, a determination process is performed to determine the symmetry between the right half region and the left half region of the face (S20).

  In the determination process, as shown in FIG. 9, the CPU selects the next area for determining symmetry (S202). For example, at the beginning, the area A1 and the area A2 in FIG. 5B are selected. Further, in the case of FIG. 6B, after the determination of the symmetry between the area A11 and the area A12 is completed, the areas for which the symmetry is determined next are the area A21 and the area A22.

  The CPU executes arithmetic processing for the pixel values included in each of the selected areas (S204). The pixel value may be at least one of brightness and color, and the arithmetic processing may be, for example, averaging.

  As a result of the comparison, it is determined whether the left and right sides of the vertical center line, that is, the right half and the left half of the face are symmetric (S206). If it is asymmetric (S206: yes), a warning is displayed (S208). If there is left-right symmetry and there is no makeup failure, no warning is displayed (S206: no). Here, regarding at least one of brightness and color, it may be determined that the difference between the average values of the left and right of the vertical center line is asymmetric when it is equal to or greater than a predetermined threshold.

  When the user presses the button within a predetermined time, a detailed left-right symmetry analysis is performed with the area narrowed (S210: yes). First, when there are already divided areas in the upper part and the lower part, it is confirmed whether or not there is an unselected area for which left / right symmetry is not determined (S212). If there is an unselected area, the process returns to S202 (S212: yes). If there is no unselected area (S212: no), each area is further divided into two parts, the upper part and the lower part (S214), and the process returns to S202.

  If the user does not desire detailed analysis of left-right symmetry and the button is not pressed within a certain time, the determination process ends.

  As described above, the electronic mirror according to the present embodiment determines whether brightness and color are symmetrical between the right half area and the left half area of the face, while suppressing power consumption. An objective evaluation of the makeup result can be made and shown to the user.

2. Second Embodiment 2.1. Configuration of Electron Mirror A second embodiment of the present invention will be described with reference to FIGS. FIG. 10 is a diagram showing a configuration of the electronic mirror 10B of the present embodiment. Explaining the difference from the first embodiment, the electronic mirror 10B includes a plurality of cameras (cameras 20A to 20D). Further, a human sensor 82 used to turn on or off the electronic mirror 10B is provided. The touch panel LCD 80 is also provided which omits the input device and also functions as an instruction input unit. In addition, the same code | symbol is attached | subjected to the same element as FIGS. 1-9, and only the element different from 1st Embodiment is demonstrated below.

  The cameras 20A to 20D in the electronic mirror 10B of the present embodiment create parallax by being installed in different places of the electronic mirror 10B. The distance to the detected object can be measured by the principle of triangulation. Therefore, for example, the distance and direction of the light source can be accurately grasped, and the shadow can be accurately removed. The number of cameras is not limited to four, but may be two or more.

  The CPU 40 receives each of the image signal 120A to the image signal 120D. The CPU 40 acquires an original image based on any one of the image signals 120A to 120D or based on an image signal obtained by combining the four.

  The human sensor 82 is a sensor that detects the presence of a human using infrared rays, ultrasonic waves, visible light, or the like. The electronic mirror 10B of this embodiment turns on the normal function of the electronic mirror 10B when the human sensor 82 detects the presence of the user and outputs a detection signal 182. On the contrary, when the human sensor 82 does not detect the presence of the user, the sleep state in which only the human sensor 82 and the CPU 40 are operated in the electronic mirror 10B is set. Therefore, power consumption can be suppressed, and the power button (see the button 72A in FIG. 2A) can be omitted.

  The touch panel LCD 80 functions not only as a display unit but also as an instruction input unit. For example, when the user wants to manually change the magnification, the user displays an image of a slide switch having a movable section on the touch panel LCD 80. When the user moves the movable section (image) on the touch panel LCD 80 with a finger, the instruction signal 180 is transmitted to the CPU 40, and the display image can be enlarged or reduced. Also, a function button (see button 72B in FIG. 2A) used when it is desired to know detailed information on the symmetry of makeup is displayed.

  At this time, slide switches and function buttons as parts can be omitted. Therefore, it is possible to increase the ratio of the touch panel LCD 80 to the entire electronic mirror 10B. That is, it is possible to make the display image easier to see without changing the overall size of the electronic mirror 10B. Note that the slide switch and the function button on the touch panel LCD 80 may be displayed when the user touches the touch panel LCD 80 and may not be displayed after a predetermined time has elapsed.

2.2. FIG. 11 is an external view of an electronic mirror 10B of the present embodiment. In the electronic mirror 10B, a slide switch 74 having a movable section and a button 72B are displayed on a touch panel LCD 80 for displaying an image. Then, the user can move the movable section with a finger or press the button 72B.

  Further, the camera 20A, the camera 20B, the camera 20C, and the camera 20D are provided in the upper, left, lower, and right portions of the frame 3, respectively. Therefore, for example, the CPU can calculate the exact distance to the light source by the parallax between the camera 20B and the camera 20D.

2.3. Flowchart FIGS. 12 to 13 show the control performed by the CPU included in the electronic mirror of the present embodiment. In addition, the same code | symbol is attached | subjected to the same process as FIGS. 8-9, and detailed description is abbreviate | omitted. Compared to the flowchart of the first embodiment, the difference is that the correction process for the user's moles and scratches based on the feature data is executed after determining that it is asymmetrical. This is because small moles and the like are not noticeable after applying the foundation, and have little influence on the judgment of symmetry. That is, it is more efficient to correct the image based on the user's facial features (such as moles and scratches) after determining left / right asymmetry.

  Further, the flowchart of the present embodiment is different from the flowchart of the first embodiment in that the determination of left-right symmetry is not performed unless the user presses a button. In the first embodiment, at least the first left / right symmetry determination is performed without the user pressing a button. However, in this embodiment, all left / right symmetry determinations are executed after the user presses a button. For this reason, the CPU does not make any unnecessary determination, so that power consumption can be further reduced.

  As shown in FIG. 12, the CPU acquires a user image (original image) from the camera (S2). Then, the position of the light source is specified from the brightness distribution of the original image (S4). Then, the shadow on the user's face is identified and removed from the change in brightness and color (S6). In the present embodiment, since the distance and direction to the light source can be accurately grasped based on the parallax, it is possible to accurately remove the shadow.

The CPU acquires feature data from the memory block (S8) and specifies the user of the original image (S10). Then, set the vertical center line passing through the nose of the user (e.g., corresponding to the boundary line L ₁ in FIG. 5 (A)), defines the right half and the left half area of the face (S12). Then, makeup data is extracted (S16).

  After that, based on makeup data that is a color and brightness distribution resulting from makeup, a determination process is performed to determine the symmetry between the right half region and the left half region of the face (S20A).

  In the determination process, as shown in FIG. 13, the CPU waits for the user to press a button and starts the process. That is, if the button is not pressed within a certain time, the determination process ends (S201: no).

  When the user presses the button (S201: yes), the CPU selects the next area for determining symmetry (S202). The method for selecting the next region is the same as in the first embodiment. Then, the CPU executes arithmetic processing for the pixel values included in each of the selected areas (S204).

  As a result of the comparison, it is determined whether the left and right sides of the vertical center line, that is, the right half and the left half of the face are symmetric (S206). If there is left-right symmetry and there is no failure in makeup (S206: no), a message to the effect of symmetry is displayed (S211B).

  However, if it is asymmetric (S206: yes), a correction process for removing the influence of moles, scratches and the like from the makeup data is executed based on the feature data of the user (S207). Then, it is determined again whether or not the vertical center line is symmetric (S209). Here, if there is left-right symmetry and there is no failure in makeup (S209: no), a display to the effect of symmetry is displayed (S211B). However, if the correction process is asymmetric even if it is executed (S209: yes), a warning is displayed (S211A).

  After the symmetry or warning is displayed, the CPU prepares for a detailed left-right symmetry analysis with a narrowed area. First, when there are already divided areas in the upper part and the lower part, it is confirmed whether or not there is an unselected area for which left / right symmetry is not determined (S212). If there is an unselected area, the process returns to S201 (S212: yes). If there is no unselected area (S212: no), each area is further divided into two parts, the upper part and the lower part (S214), and the process returns to S201.

  As described above, the electronic mirror according to the present embodiment determines whether brightness and color are symmetrical between the right half area and the left half area of the face, while suppressing power consumption. An objective evaluation of the makeup result can be made and shown to the user. Further, since the symmetry determination is executed after the user presses the button, the power consumption can be further reduced, and the cost can be reduced by reducing the number of parts by using the touch panel LCD.

3. Others The present invention is not limited to these examples, and the present invention includes configurations that are substantially the same as the configurations described in the embodiments (for example, configurations that have the same functions, methods, and results, or configurations that have the same purposes and effects). . In addition, the invention includes a configuration in which a non-essential part of the configuration described in the embodiment is replaced. In addition, the present invention includes a configuration that exhibits the same operational effects as the configuration described in the embodiment or a configuration that can achieve the same object. Further, the invention includes a configuration in which a known technique is added to the configuration described in the embodiment.

3 ... Frame, 10 ... Electronic mirror, 10A ... Electronic mirror, 10B ... Electronic mirror, 20 ... Camera, 20A ... Camera, 20B ... Camera, 20C ... Camera, 20D ... Camera, 30 ... Memory block, 32 ... RAM, 34 ... ROM, 40 ... CPU, 50 ... LCD, 51 ... hinge, 70 ... input device, 72 ... button, 72A ... button, 72B ... button, 74 ... slide switch, 75 ... movable section, 77 ... shadow, 78 ... electric light, 80 ... Touch panel LCD, 82 ... Human sensor, 90 ... Makeup unevenness, 92 ... Makeup unevenness, 98 ... Warning display, 99 ... Mole, 120 ... Image signal, 120A ... Image signal, 120B ... Image signal, 120C ... Image signal, 120D ... Image signal, 130 ... Read data, 140 ... Output image signal, 142 ... Write data, 170 ... Instruction signal, 180 ... Instruction signal , 182 ... detection signal

Claims (6)

Display means for displaying an image;
Imaging means provided integrally with the display means;
The symmetry of the makeup performed by the user by comparing at least one of the brightness and color of the right and left halves of the user's face based on the original image including the user's face imaged by the imaging means And a control unit that causes the display unit to display an image based on the original image and a warning that the makeup is asymmetric when the image is determined to be asymmetric. The electronic mirror according to claim 1,
Storage means for storing feature data that is a feature of the user's face;
The control means includes
An electronic mirror that corrects an image based on the original image based on the feature data and determines symmetry of makeup performed by a user. The electronic mirror according to claim 2,
The control means includes
An electronic mirror that performs correction based on the feature data and determines symmetry again when it is determined that makeup performed by the user is asymmetric based on an image before the correction based on the feature data is performed. The electronic mirror according to any one of claims 1 to 3,
The control means includes
Based on the original image, obtain environmental information which is information on the environment at the place where the electronic mirror is used,
The electronic mirror which judges the symmetry of the makeup | decoration which the user performed by performing the color correction based on the said environment information to the said original image. The electronic mirror according to any one of claims 1 to 4,
Including an instruction input means for receiving an instruction from the user and generating an instruction signal;
The control means includes
Based on the instruction signal, an image area used for comparing the right half and the left half of the user's face is divided into an upper part and a lower part, and the user performs at least one of the upper part and the lower part. An electronic mirror for judging the symmetry of makeup. An electronic mirror program comprising: display means for displaying an image; and imaging means provided integrally with the display means,
The symmetry of the makeup performed by the user by comparing at least one of the brightness and color of the right and left halves of the user's face based on the original image including the user's face imaged by the imaging means A program for causing a computer to function as control means for causing the display means to display an image based on the original image and a warning that makeup is asymmetric when the image is determined to be asymmetric.

Images