JP2010070889A - Virtual fitting system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a virtual fitting system simply displaying a virtual dressed condition in conformity to unspecified majority of users. <P>SOLUTION: The virtual fitting system 1 includes a half mirror 3, a display device 2 provided on the back side of the half mirror. A camera 4 for taking a picture of a user located in front of the half mirror 3 is mounted on the upper part of the display device 2. The camera 4 takes a picture of the user wearing a garment with position detecting marks, and a dress image is displayed to be superposed on an image in the half mirror 3 based on the detected position of the position detecting marks. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a virtual try-on device that virtually displays a state in which clothing or other clothing is worn on a half mirror.

  When a customer purchases clothes such as clothes to be worn in a store, the customer tries on a product to be purchased in a fitting room installed in the store in advance, and the size, comfort, or self It is common to confirm whether or not it looks good.

  In order for a customer to try on a plurality of clothes and find the customer's favorite clothes from among them, it is necessary to compare each of the clothes while trying on them, but the customer remembers when the comparison was tried on. If there are many clothes to be tried on, the memory at the time of first try-on will fade, and you will not be able to make a sufficient comparison, so you may accidentally select your favorite clothes. There is a risk that it will take a long time to try on again.

  In order to solve this problem, a predetermined shooting environment is provided in a fitting room where customers try on apparel products (clothes, etc.), and a plurality of try-on states are photographed, and the photographed images are stored. An apparel product sales support method for comparing and displaying stored images and allowing a customer to select an apparel product is known (see Patent Document 1).

  However, when trying on a plurality of clothes in this way, it is necessary to enter the fitting room multiple times and remove the clothes, so that there is a problem that the act of trying on takes time and is troublesome.

  In addition, when many customers try on clothes that are merchandise in stores, the clothes may be damaged, and depending on the degree of damage, the clothes may not be sold as merchandise.

  In order to solve this problem, a liquid crystal display device is placed on the back of the half mirror that displays the mirror image of the customer's user, and the image of the clothes that the user wants to try on is reflected in the mirror image on this liquid crystal display device. A dressing system capable of virtually trying on clothes by preventing them from being damaged by displaying them according to body parts such as the head, upper body, and lower body of the user, and reducing the time for fitting. It is known (see Patent Document 2).

  However, in this dressing system, in order to display clothes and the like according to the body part of the user, it is necessary to take an image of the user in advance and acquire this data. There is a problem that it is difficult to apply to a person.

  In addition, since clothes displayed on the liquid crystal display device are displayed in accordance with the user's pre-captured image, if the user moves, the displayed clothes and the user's body will be misaligned. The virtual try-on cannot be performed unless the posture is a predetermined posture, that is, a posture in a user's photographed image.

  By the way, as a virtual try-on according to the movement of the human body, a position detection mark for calculating a solid coordinate is pasted on a solid model in which each part of the human body is modeled in a predetermined solid shape. For example, a 3D (Dimension) data display method for projecting onto a screen or the like so that the clothing data is positioned at a position corresponding to the three-dimensional coordinates of the model is known. (See Patent Document 3).

However, although this 3D data display method can match the display position of the clothing data with a predetermined position of the three-dimensional model, the fitting person himself and the clothing data as described in Patent Document 2 are combined and virtually displayed. It is not something that will be tried on, but it cannot be virtually tried on in accordance with the image of a full-scale fitting person, and lacks reality.
Japanese Patent Laid-Open No. 2002-207802 JP 2006-331132 A JP 2005-256232 A

  The present invention has been made to solve the above-described conventional problems, and its purpose is to allow an unspecified number of try-ons to easily use a try-on of a customer such as a customer. Is to be able to do it.

  The virtual fitting device according to the present invention includes a display unit that projects and displays a clothing image from the back side of the half mirror, a position detection unit that detects a position of a predetermined part of the user located on the front side of the half mirror, A position calculating means for calculating the viewpoint position information of the user based on the detected position information of the predetermined part, and a mirror image of the user displayed on the half mirror based on the calculated position information of the viewpoint and the position information of the half mirror. On the other hand, a display control means for controlling the display means so as to display a clothing image on the half mirror is provided at a position overlapping when viewed from the user. Further, the position detecting means detects the position of the predetermined part every predetermined time. Further, a predetermined position detection mark is displayed at the predetermined portion, and the position detection means detects the position of the predetermined portion by the position detection mark. Furthermore, the position detection mark is displayed at the predetermined portion when a user wears clothing displaying the position detection mark. Further, the position detecting means is provided with a camera that is provided above the half mirror and photographs a user located on the front side of the half mirror.

  According to the present invention, when a try-on such as a customer virtually tries on a garment, an unspecified number of try-on users can be used, and the wearing state can be kept following the movement of the body of the try-on, The versatility and reality of the virtual fitting system can be improved.

  Hereinafter, embodiments according to the present invention will be described with reference to the accompanying drawings.

  FIG. 1 is an exploded perspective view of a part of an embodiment according to the present invention. The virtual try-on device 1 includes a half mirror 3 and a display device 2 provided on the back side thereof as display means. Above the display device 2, an object located in front of the half mirror 3, that is, the present device. And a camera 4 for photographing the user, and detects the position of a position detection mark on a position detection mark garment described later from an image photographed by the camera 4. In addition, you may make it provide the terminal device which inputs various information.

  The display device 2 is a planar casing and has a display screen on the entire front surface, and the half mirror 3 is configured to cover the display screen with the same size (area) as the display screen. The display device 2 may be a large screen display using a liquid crystal display, a PDP (plasma display panel), an LED (Light Emitting Diode), an EL (Electro Luminescence), or the like.

  The half mirror 3 reflects a part of incident light and transmits a part thereof. That is, when an image is not displayed on the display screen of the display device 2, a user located in front of the half mirror 3 (opposite to the display device 2) enters the front of the half mirror 3 and visually reflects the reflected light. can do. That is, the user can use the half mirror 3 as a normal mirror and can visually recognize a mirror image.

  On the other hand, when displaying an image on the display screen of the display device 2, a user located in front of the half mirror 3 can visually recognize the image of the display device 2 while being superimposed on a mirror image projected on the half mirror 3. .

  The camera 4 is attached to the upper part of the display device 2 and is used for photographing a user located in front of the half mirror 3 and photographing a calibration target serving as a reference. The camera 4 can be used as long as it is a photographing device that can obtain a still image. For example, a still image may be obtained by a digital still camera or a video camera.

  FIG. 2 is a schematic block configuration diagram relating to the control of the virtual try-on device 1. The control unit 10 performs overall display processing related to the clothing image on the display device 2, and includes a setting unit 101, a position calculation unit 102, and a display control unit 103. The image display unit 11 displays an image on the display screen of the display device 2 based on the image data transmitted from the display control unit 103. The position detection unit 12 detects the position of the predetermined part of the user M based on the captured image obtained from the camera 4. The storage unit 13 stores clothing images such as clothes to be displayed on the display device 2 and customer information (attribute information, past selection information, etc.) regarding the registered user. The clothing image is composed of three-dimensional image data, and the clothing image corresponding to the viewing direction can be displayed by using the three-dimensional image data. The operation input unit 14 inputs necessary selection information and the like when trying on.

  The setting unit 101 sets the positional relationship between the display device 2 and the camera 4. FIG. 3 is an explanatory diagram showing the positional relationship of each part with respect to the virtual fitting device 1. First, the positional relationship between the space coordinate Pm of the center position on the display screen of the display device 2 and the space coordinate Pc of the shooting position of the camera 4 is set. As a setting method, a calibration object serving as a reference is placed at a position where the user M stands, and a distance between a predetermined position of the calibration object and the center position of the display screen of the display device 2 is measured, and the calibration object is measured. The relative positional relationship between the reference space coordinates and the space coordinates Pm of the display device 2 is set.

  Next, the installed calibration object is photographed by the camera 4 and necessary calibration processing is performed, and the relative positional relationship between the reference space coordinates and the space coordinates Pc of the camera 4 is set based on the photographed image.

  In this way, the spatial coordinates Pm of the display device 2 and the spatial coordinates Pc of the camera 4 with respect to the reference spatial coordinates are set. Note that the distance measured between the calibration object and the display device 2 is set and registered as a spatial coordinate Pm based on the input data obtained by inputting a measured value from the operation input unit 4. Further, as for the set value relating to the spatial coordinate Pc of the camera 4, the spatial coordinate Pc is set and registered by analyzing image data obtained by photographing the calibration object with the camera 4.

  The position calculation unit 102 captures the user M standing at a predetermined position on the front side of the half mirror 3 with the camera 4 and calculates the spatial coordinates Pt related to the body of the user M and the spatial coordinates Pe related to the viewpoint from the image data.

  FIG. 4 is an explanatory diagram illustrating an example of a captured image of the user M captured by the camera 4. When the user M standing on the front side of the half mirror 3 is photographed by the camera 4 from obliquely above, the upper body close to the camera 4 is photographed largely.

  The user M wears the mark display clothes 5 on the upper body. In the mark display garment 5, one position detection mark is displayed at a position corresponding to the central portion of the chest of the user M in the wearing state, and the position corresponding to both sides on the abdominal side and both sides on the back side are corresponding to the torso. Two position detection marks 5a are displayed at each position. The mark display garment 5 is preferably one that can be easily worn on the garment, such as an apron. Also, it should be a solid and bright color so that the position detection mark can be read accurately.

  The position detection mark 5a is a rectangular mark of a predetermined size, and the read information is displayed in black and white and in a lattice shape. Each position detection mark has a different grid pattern so that the position to be displayed can be specified. By reading the position detection mark, the position of a predetermined part of the user can be specified. it can. Note that a combination of colors other than black and white may be used as long as a lattice pattern can be read.

  The upper body of the user M is photographed by the camera 4, and image data relating to the position detection mark 5 a is extracted from the photographed image by the position detection unit 12. Then, the relative position relationship of the position detection mark 5a with respect to the camera 4 is obtained from the position calculated by comparing the position of the extracted image data in the captured image and the outer shape of the position detection mark with the actual outer shape. Further, by analyzing the grid pattern of the image data of the position detection mark, it is specified in which part of the user M the position detection mark is displayed as described above. In this way, the position detection unit 12 detects the position of the predetermined part of the user. Then, by detecting the position of the position detection mark every predetermined time, the position of the predetermined part can be detected in real time according to the change in the user's posture.

  The position calculation unit 102 calculates the spatial coordinates of each position detection mark from the spatial coordinates Pc of the camera 4 based on the relative positional relationship thus obtained. With respect to the position detection mark displayed at the position corresponding to the central part of the chest, it is assumed that the viewpoint position of the user M exists at a predetermined position above the position detection mark. The coordinate Pe is calculated. The position detection marks displayed at the positions corresponding to both sides of the trunk are calculated as space coordinates Pt related to the trunk.

  In this way, the spatial coordinates Pe related to the user's viewpoint position and the spatial coordinates Pt related to the torso are calculated, and each spatial coordinate can be calculated in real time according to a change in the user's posture. Further, since the position detection mark is also displayed on the back side of the user, the spatial coordinates Pt related to the torso is calculated even when the user is turned to the half mirror, and the back side image data of the clothing image is obtained. Can be displayed.

  The display control unit 103 reads the three-dimensional image data of the clothing image stored in the storage unit 13 based on the clothing image selection information input from the operation input unit 14. Then, the display position and display size of the mirror image of the user M on the half mirror 3 are calculated based on the relative positional relationship between the spatial coordinates Pm of the display device 2 and the spatial coordinates Pe regarding the viewpoint position. Further, an inclination angle (an angle indicating whether the user M is facing directly or in an oblique direction) and a distance with respect to the half mirror 3 of the user M are calculated based on the spatial coordinates Pt related to the trunk. Since the space coordinates Pt regarding the trunk are set at the abdomen and back of the user M, the inclination angle can be calculated for the case where the user M rotates from 0 degrees to 360 degrees.

  Then, the read three-dimensional image data is displayed in accordance with the mirror image display position and display size, and the image data is rotated and enlarged / reduced in accordance with the tilt angle and distance. The processed image data is transmitted to the image display unit 11 and displayed on the display screen of the display device 2.

  FIG. 5 is an explanatory view of a state in which the clothing image R is displayed on the display screen as viewed from a viewpoint other than the user, and FIG. 6 is an explanatory view of a display state viewed from the viewpoint of the user. When the display state is viewed from a viewpoint other than the user, the mirror image N projected on the half mirror 3 does not overlap with the clothing image R displayed on the display screen, but when viewed from the user's viewpoint, it overlaps with the mirror image N. The clothing image R is displayed, and a state in which the user is wearing the clothing of the clothing image R virtually can be displayed.

  Also, the position detection mark is read every predetermined time, and the spatial coordinates Pe of the viewpoint position and the spatial coordinates Pt of the torso change according to the change of the user's posture, so the clothing image displayed correspondingly also changes. To come. Therefore, a realistic wearing state can be displayed.

  As the operation input unit 14, necessary operation information can be input using a known input device such as a keyboard or a remote controller. In addition, the operation information may be specified and input by reading the movement of the user's hand and the movement of the finger using a captured image from the camera 4. In addition, a clothing image may be selected while displaying the menu screen on the display screen and operating the menu screen.

  As described above, the virtual try-on device 1 allows the user to check and evaluate the wearing state of the clothes with the mirror image displayed on the half mirror 3 and the clothing image superimposed thereon without trying on the actual clothes. Can do. Moreover, by storing many types of clothing images in the storage unit 13, the user can easily change the clothing images displayed on the display device 2 according to his / her preference, and various clothes can be virtually displayed. It is possible to try on.

  The displayed clothing image may be a virtual clothing clothing image generated by a combination of a pattern and a color pattern arbitrarily selected by the user, in addition to the existing clothing clothing image.

  In the present embodiment, the case of trying on a single piece of clothing has been described. However, the present invention is not limited to this. For example, clothing images such as a hat to try on the head, pants to try on the lower body, and a skirt are also displayed on the display device 2 at the same time. It may be displayed. In this case, in order to calculate the spatial coordinates of the predetermined part, it is only necessary to wear clothes displaying position detection marks for the head and the lower body.

  FIG. 7 is a processing flow for displaying a clothing image on the display device 2. The user wears a mark display garment in advance and stands at a predetermined position on the front side of the half mirror 3. And a user is image | photographed with the camera 4 and a picked-up image is acquired (S100). Image data of a position detection mark is extracted from the acquired captured image (S101), and the extracted position detection mark is analyzed to detect the position of a predetermined part of the user (S102).

  Next, based on the position of the detected position detection mark and the spatial coordinate Pc of the camera 4, the spatial coordinates of each position detection mark are calculated (S103), and the space of the position detection mark corresponding to the torso of the user. The body space coordinates Pt are calculated from the coordinates (S104). Then, the spatial coordinate Pe of the user's viewpoint position is calculated from the spatial coordinate Pt of the trunk (S105).

  Then, it is checked whether the clothing image has already been selected and displayed (S106). If not, the clothing image is selected and displayed (S107), and a menu screen for making a selection is displayed on the display screen. To do. In the selection display screen, it is checked whether the clothing image has been selected by an operation input from a remote controller or the like (S108). If the clothing image is selected, the selected clothing image is read out, the spatial coordinates Pm of the display device 2, and the user's A clothing image is displayed based on the spatial coordinates Pe of the viewpoint position and the space coordinates Pt of the torso (S109), the clothing image is displayed on the display screen of the display device 2, and is superimposed on the mirror image of the user displayed on the half mirror 3. In addition, a clothing image is projected and displayed.

  If the clothing image has already been displayed in step S106, the process proceeds to step S109 to update the clothing image display process based on the newly calculated viewpoint position spatial coordinates Pe and torso spatial coordinates Pt. In this way, the clothing image can be changed following the change in the posture of the user.

  It is checked whether or not the change of the clothing image has been operated after the display of the clothing image (S110). If there is a changing operation, the process returns to step S109 to display the clothing image selected by the changing operation again. If there is no change operation, it is checked whether an end operation has been performed (S111), and if there is an end operation, an end process is performed. If there is no end operation, it is checked whether or not a predetermined time has passed (S112), and when the predetermined time has passed, the process returns to step S100, and processing such as position detection mark position detection and spatial coordinate calculation is repeated to display a clothing image. It will be updated.

  As described above, the position detection mark of the mark display garment worn by the user is photographed by the camera, and the clothing image is superimposed on the mirror image and displayed on the half mirror based on the detection position of the position detection mark. Therefore, the wearing state can be virtually displayed according to an unspecified number of users.

It is a disassembled perspective view regarding a part of embodiment which concerns on this invention. It is a schematic block block diagram regarding control of a virtual try-on device. It is explanatory drawing which shows the positional relationship of each site | part regarding a virtual fitting device. It is explanatory drawing which shows an example of the user's picked-up image image | photographed with the camera. It is explanatory drawing which looked at the state which displayed the clothing image on the display screen from viewpoints other than a user. It is explanatory drawing which shows the display state seen from a user's viewpoint. It is a processing flow which displays a clothing image on a display device.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Virtual fitting device, 2 ... Display device, 3 ... Half mirror, 4 ... Camera, 5 ... Mark display clothing, 5a ... Position detection mark.

Claims (5)

  Display means for projecting and displaying a clothing image from the back side of the half mirror, position detecting means for detecting the position of a predetermined part of the user located on the front side of the half mirror, and position information of the detected predetermined part Based on the calculated position information of the user and the mirror image of the user displayed on the half mirror based on the calculated position information of the viewpoint and the position information of the half mirror. A virtual try-on device comprising: display control means for controlling the display means so as to display a clothing image on the half mirror.   The virtual try-on device according to claim 1, wherein the position detection unit detects the position of the predetermined part at predetermined time intervals.   The predetermined position detection mark is displayed on the predetermined portion, and the position detection means detects the position of the predetermined portion by the position detection mark. Virtual try-on device.   The virtual try-on device according to claim 3, wherein the position detection mark is displayed at the predetermined portion when a user wears clothing displaying the position detection mark.   5. The virtual according to claim 1, wherein the position detection unit includes a camera that is provided above the half mirror and photographs a user located on a front side of the half mirror. Try-on device.

Images