WO2019134501A1 - Method and device for simulating fit of garment on user, storage medium, and mobile terminal - Google Patents

Abstract

A method and device for simulating fit of a garment on a user, a storage medium, and a mobile terminal. The method comprises: when visiting a shopping website, acquiring a garment fitting request (110); upon detection of the garment fitting request, acquiring a garment model corresponding to the garment fitting request and a corresponding user identification at the website (120); and acquiring a body model of a user according to the user identification, combining the garment model with the body model to obtain a garment fit model, and displaying the garment fit model (130).

Description

Method, device, storage medium and mobile terminal for simulating user trial installation

The present application claims priority to Chinese Patent Application No. 201810015953.3, filed on Jan.

Technical field

The embodiments of the present application relate to mobile terminal technologies, for example, to a method, an apparatus, a storage medium, and a mobile terminal for simulating user trial installation.

Background technique

With the rapid development of e-commerce, more and more users choose to buy clothing online.

However, current web platforms typically display merchandise in the form of two-dimensional photos. While providing buyers with a convenient shopping experience, the purchaser can not intuitively judge whether the clothes or accessories purchased by the purchaser fits through the two-dimensional photos, or whether they conform to their own image and temperament. As a result, the style or model of the clothes purchased by the purchaser does not meet their own requirements.

Summary of the invention

The embodiment of the present application provides a method, a device, a storage medium, and a mobile terminal for simulating user trial installation, which can simulate the effect of the user's own trial installation and enrich the functions of the mobile terminal.

In an embodiment, the embodiment of the present application provides a method for simulating user trial installation, including:

Get a trial installation request when visiting a shopping site;

When the trial installation request is detected, acquiring a clothing model corresponding to the trial installation request and a user identifier corresponding to the shopping website;

Acquiring a human body model of the user according to the user identifier, fusing the clothing model to the human body model to obtain a trial model, and displaying the trial model.

In an embodiment, the embodiment of the present application further provides a device for simulating user trial installation, the device comprising:

Requesting a detection module, configured to obtain a trial installation request when accessing the shopping website;

The information acquiring module is configured to acquire, when the trial installation request is detected, a clothing model corresponding to the trial installation request and a user identifier corresponding to the shopping website;

The trial model generation module is configured to acquire a human body model of the user according to the user identifier, fuse the clothing model to the human body model to obtain a trial model, and display the trial model.

In an embodiment, the embodiment of the present application further provides a computer readable storage medium, where the computer program is stored, and the program is implemented by the processor to implement the foregoing method.

In an embodiment, the embodiment of the present application further provides a mobile terminal, a 3-dimensional (3D) depth camera, a memory, a processor, and a computer program stored in the memory and operable on the processor, A 3D depth camera configured to capture a user image including depth of field information; the processor implementing the computer program to implement the method described above.

DRAWINGS

1 is a flowchart of a method for simulating user trial installation according to an embodiment of the present application;

2 is a flow chart of another method for simulating user trial installation according to an embodiment of the present application;

3 is a structural block diagram of an apparatus for simulating user trial installation according to an embodiment of the present application;

4 is a structural block diagram of a mobile terminal according to an embodiment of the present application;

FIG. 5 is a structural block diagram of a smart phone according to an embodiment of the present application.

Detailed ways

The present application will be described below in conjunction with the accompanying drawings and embodiments. The embodiments described herein are merely illustrative of the present application and are not intended to be limiting. For the convenience of description, only some but not all of the structures related to the present application are shown in the drawings.

In an embodiment, some example embodiments are described as a process or method depicted as a flowchart. Although the flowchart depicts multiple steps as a sequential process, many of the steps can be implemented in parallel, concurrently, or concurrently. Additionally, the order of one or more steps can be rearranged. The process may be terminated when its operation is completed, but may also have additional steps not included in the figures. The processing may correspond to methods, functions, procedures, subroutines, subroutines, and the like.

Embodiment 1

In an embodiment, a trial installation function is configured in advance to simulate the effect of the user wearing the garment himself. The trial function can be a stand-alone trial application with a separate trial application icon. The user can bookmark the URL of a frequently viewed shopping site in the trial application to link to the shopping site through the trial application. The trial application interface provides a trial function button, and when the trial function button is pressed, the trial function is considered to be enabled. There are many ways to attach a trial installation function to a mobile terminal, and it is not limited to the above-described methods. For example, the trial installation function can also be added to the Android control center to open the trial installation function when the user's operation indication of the icon corresponding to the trial installation function is detected. If the display operation of the control center is detected while browsing the shopping site, the control center is displayed and the user's operation instruction for the control center is obtained. If the operation indication is an operation on an icon corresponding to the trial function, the trial installation function is turned on.

In an embodiment, when it is detected that the installation of the trial application is completed, an initialization interface is displayed for prompting the user to initialize the trial installation function. When the trial installation function is initialized, the user image of the user (including photo shooting and video shooting) is acquired by the 3D depth camera according to the preset direction, the human body model is generated according to the user image, and the human body model is associated with the preset user identifier. Stored in the default whitelist. Wherein, the 3D depth camera can be set to capture an image including depth of field information. In an embodiment, the 3D depth camera includes a depth camera based on structured light depth ranging and a depth camera based on Time Of Flight (TOF) ranging. The preset user identifier may be a user identifier corresponding to the shopping website, such as a login name or a user name of the shopping website.

In an embodiment, the user is photographed by the preset direction to at least capture at least one frame of the user image including the depth information in the front, rear, left, and right directions of the user. Since the user image includes the pixel information and the depth of field information, the setting algorithm is used to construct a three-dimensional human body model of the user based on the user image, and the human body model is stored in the preset white list in association with the user identifier. The algorithm used in constructing the human body model in the embodiment of the present application is not limited.

In an embodiment, the shooting direction of the 3D depth camera is not limited to the four directions listed above, and the video shooting may be performed around the user one week. For example, acquiring a user video captured by a 3D depth camera around a user. Then, the user video is framed by using a set framing strategy to obtain a set number of user images with depth information, and a setting algorithm is used to generate a human body model according to the user image, and the human body model and the above User ID associations are stored in a preset whitelist. The framing strategy may be: extracting one frame of image every set time interval to obtain a set number of user images including depth of field information. The set time interval may be system default. In an embodiment, the shorter the time interval, the more user images are extracted, and the constructed human body model is closer to the real human body structure. FIG. 1 is a flowchart of a method for simulating user trial installation according to an embodiment of the present application. The method may be performed by a device that simulates a user's trial installation, where the device may be implemented by software and/or hardware, and may be integrated in In a mobile terminal, for example, a mobile terminal having a 3D depth camera. As shown in Figure 1, the method includes:

Step 110: When accessing the shopping website, obtain a trial installation request.

The trial installation request is a user instruction that triggers an operation of "wearing" the clothing model corresponding to the clothing selected by the user on the human body model. The trial request can be entered by user gesture or user voice. For example, when the trial installation function is initialized, a gesture representing a trial request is also predefined. When detecting the gesture information input by the user, the mobile terminal compares the gesture information with a preset gesture representing the trial request, and when the two match, determines that the trial installation request is detected. For another example, when the trial installation function is initialized, the user is prompted to input a control voice representing the trial installation request. Thereby, the user voice input by the voice assistant can be acquired, and the user voice is matched with the control voice. If the match is successful, it is determined that the trial installation request is detected.

In the embodiment of the present application, the mobile terminal determines whether the network resource is a shopping website according to a Uniform Resource Locator (URL) of the accessed network resource. For example, extracting a network resource name included in the URL, and querying a preset shopping website list according to the network resource name. If the network resource name belongs to the shopping website list, it is determined that the shopping website is accessed. In an embodiment, after determining to access the shopping website, it may be determined whether the trial installation function is enabled. If the trial installation function is turned on, the trial installation request entered by the user is obtained. If the trial installation function is not enabled, you can execute the original shopping website access process.

Step 120: When the trial installation request is detected, obtain a clothing model corresponding to the trial installation request and a user identifier corresponding to the shopping website.

The user identifier is a character corresponding to the shopping website for uniquely identifying the user, including but not limited to a login name or a username.

In the embodiment of the present application, when the trial installation request is detected, the login name or the user name corresponding to the shopping website is obtained as the user identifier.

In an embodiment, the garment model is a three-dimensional model of the garment. The clothes model can be obtained through an application programming interface (API) provided by the shopping website. Exemplarily, in the case that a trial installation request is detected, the clothing identifier corresponding to the trial installation request may be obtained by acquiring at least one frame screenshot. In an embodiment, the multi-frame screenshot image of the clothing corresponding to the trial request is obtained from a frame buffer. The clothes logo may be a clothing number, a clothes name, or a product number. The text content in the screenshot image may be extracted, and the clothing identification is determined according to the text content. The manner of obtaining the clothing identifier corresponding to the trial installation request is not limited to the manner of the screenshot, and may also obtain the product information (including the attribute information and the clothing image) of the clothing displayed by the current shopping website through the script file, and determine the clothing identifier according to the product information. Wait.

If the shopping website provides the trial service, the shopping website may be queried according to the clothing identification to obtain the clothing model corresponding to the clothing identification. In an embodiment, the shopping website providing the trial service has a stereo model of at least one product, and the mobile terminal can access the storage space of the stereo model through the API provided by the shopping website to obtain a clothing model corresponding to the clothing identifier.

If the shopping site does not provide the trial service, or if the corresponding clothing model does not exist in the clothing corresponding to the trial request, the set algorithm is used to construct the clothes model according to the attribute information of the clothes corresponding to the trial request. Exemplarily, if the shopping website does not provide the trial service, or the clothing corresponding to the trial request does not have a corresponding clothing model, the clothing attribute information corresponding to the trial installation request and the picture of the clothes are extracted from the screenshot, and the The configured conversion model generates a corresponding clothes model based on the attribute information of the clothes and the picture of the clothes. Among them, the conversion model is a deep neural network trained according to the clothing sample model and the clothes sample attributes.

For ease of understanding, the embodiment of the present application briefly describes the construction process of the conversion model. To construct the conversion model, the number of hidden layers of the deep neural network and the number of nodes of each layer in the input layer, the hidden layer, and the output layer, and the first parameter of the initialization deep neural network may be preset, wherein the first parameter includes each layer The offset value and the weight of the edge. Then, the deep neural network is trained by using a set number of clothes sample models and corresponding clothing attribute information to obtain a conversion model.

In one embodiment, the garment sample model includes a garment image and a garment frame model including, but not limited to, a top frame model, a skirt frame model, and a pants frame model. The sample properties of the clothes include neck circumference, chest circumference, waist circumference, shoulder width, arm length, hip circumference and length, and the length of the garment includes the length of the shirt, the length of the pants and the length of the skirt. The garment frame model is adjusted according to the clothes sample property, and a frame model matching the clothes sample is obtained, so that the clothing frame model can be reused for clothes of different sizes and models. In an embodiment, since the clothing frame model can be reused, only several clothing frame models such as tops, pants, dresses, and skirts can be preset to reduce the memory occupancy of the trial installation function. The three-dimensional texture mapping technique can be used to attach the clothes picture to the corresponding surface of the clothing frame model to obtain a clothes sample model. The picture of the garment includes an image of one or more pieces of clothing in the garment playing board. In order to simplify the training process of the conversion model, the front picture and the reverse picture of the clothes can be obtained as the clothes picture. For example, the front and back images of the clothes are extracted from the screenshot. In an embodiment, the outlines of the clothes in the front picture and the reverse picture are respectively extracted, and the shooting focus and the shooting angle are determined according to the outline of the clothes. Thereby, it can be determined whether the photographing focal length and the photographing angle of the front picture and the reverse picture are the same. When the shooting focal length is not the same, zoom in or out to make the front and back images the same. When the shooting angle is different, rotating the picture makes the front view and the reverse picture the same.

A set number of garment frame models are pre-built, and the garment frame model is adjusted according to the clothing sample attributes of the at least one garment sample to obtain a garment frame model corresponding to the at least one garment sample. Then, the front picture and the reverse picture of the clothes sample are respectively fused to the clothing frame model corresponding to the clothes sample, and the clothes sample model of each clothes sample is obtained. The clothes sample model is input into the above-mentioned preset depth neural network, and a conversion model is generated by using a set machine learning algorithm. Among them, the set machine learning algorithms include a forward propagation algorithm and a backward propagation algorithm. Exemplarily, the clothing sample model is used to train the preset deep neural network in two stages of forward propagation and backward propagation. When the error calculated by the backward propagation training reaches the expected error, the training ends and the conversion model is obtained.

In an embodiment, network parameters such as the number of layers of the deep neural network, the number of neurons, the convolution kernel, and/or the weight are not limited in the embodiment of the present application. The embodiment of the present application does not limit the execution body of the construction operation of the conversion model, and may be a server or a mobile terminal.

Step 130: Acquire a human body model of the user according to the user identifier, fuse the clothing model to the human body model to obtain a trial model, and display the trial model.

In one embodiment, the trial model refers to a three-dimensional model that presents the user "wearing" the garment obtained by blending the human body model with the clothing model. For example, a three-dimensional texture mapping technique can be used to fuse the clothing model and the human body model according to the correct surface correspondence to obtain a trial model, wherein the correct surface correspondence relationship includes the upper model corresponding to the upper part of the human trunk, and the pants model corresponding to the lower part of the human trunk. The front side of the clothes model corresponds to the front side of the human body model, and the reverse side of the clothes model corresponds to the reverse side of the human body model.

In the embodiment of the present application, when the shopping website is accessed, the trial installation request is detected, and an identifier that uniquely identifies the user identity, such as a login name or a user name corresponding to the shopping website, is obtained as the user identifier. The human body model is obtained by querying the preset whitelist according to the user identifier, wherein the preset whitelist associates the user identifier and the human body model. According to the spatial correspondence between the clothes model and the human body model, the image of the clothes model is attached to the surface of the human body model from different angles and directions to obtain a trial model, and the test model is displayed. In an embodiment, a model rotation function is provided. If a model rotation gesture input by the user is detected, the rotation angle and the rotation direction of the trial model are determined according to the duration of the rotation gesture of the model and the direction of the gesture sliding, so as to facilitate the user. Check the wearing effect of the clothes in one direction.

In the technical solution of the embodiment, after the trial installation function is enabled, if the access operation of the shopping website is detected, the trial installation request input by the user is acquired; when the trial installation request is detected, the clothing corresponding to the trial installation request is obtained. a model and a user identifier corresponding to the shopping website; acquiring a human body model of the user according to the user identifier, integrating the clothing model into the human body model to obtain a trial model, and displaying the trial model, thereby displaying the effect of the user trying on the clothes themselves . The above technical solution can solve the problem that the user can not intuitively determine whether the style or model of the clothing meets the requirements in the network shopping scenario in the related art, so that the mobile terminal can provide the function of simulating the real shopping scene, and realize the intuitive display of the user's own test. The effect of the device enriches the functions of the mobile terminal.

Buying case two

2 is a flow chart of another method for simulating user trial installation provided by an embodiment of the present application. As described in Figure 2, the method includes:

Step 210: When accessing the shopping website, obtain a trial installation request.

In an embodiment, the trial function can be turned on before visiting the shopping website. After the trial installation function is turned on, if it is detected that the user is browsing the shopping website, the trial installation request input by the user is obtained. For example, if the mobile terminal detects the opening operation of the trial application, it determines that the trial installation function is turned on, can be linked to the shopping website through the trial application, and after detecting the user's trial installation request, the trial application is applied. The trial model is displayed in the interface of the program. In an embodiment, the application corresponding to the shopping website may be opened by the trial application, so that the user browses the product in the application corresponding to the shopping website, and after detecting the user's trial request, jumps Go back to the trial application and display the trial model in the interface of the trial application.

Alternatively, the trial installation feature can be turned on during the visit to the shopping site. For example, when detecting the operation of the control center during the browsing of the shopping website by the user, the control center is displayed according to the operation, and if the trial function switch in the user opening the control center is obtained, it is determined that the trial installation function is turned on. If the user is still browsing the shopping website, the user receives the trial installation request.

There are many ways to open the trial installation function, and the embodiment of the present application is not specifically limited, and the scenario for obtaining the trial installation request is not limited to the above enumerated manner.

Step 220: Determine whether the shopping website provides a trial service. If the trial service is provided, step 230 is performed. If the trial service is not provided, step 250 is performed.

In an embodiment, the website identification of the shopping website providing the trial service is obtained in advance. For example, the website identifier of the shopping website that can provide the trial service is stored in a whitelist manner of the website, and is configured in the mobile terminal before the mobile terminal is shipped from the factory, wherein the website identifier may be a URL, or a key character included in the URL. . In an embodiment, when the website whitelist is updated, a new website whitelist is sent by the mobile manufacturer server to the mobile terminal for updating by the mobile terminal.

Obtaining the website identifier of the shopping website, including but not limited to obtaining the URL of the shopping website, using the URL as the website identifier, or obtaining the website identifier of the shopping website by analyzing the URL of the shopping website. For example, the URL of the shopping website on the mobile terminal is usually in the form of http://m.xx.cn, and the website identifier can be obtained by extracting key characters in the URL. Among them, the key character can be the character after //m., or directly extract multiple characters between the separator "." immediately adjacent to / / m to the next separator ".".

In the embodiment of the present application, the whitelist of the website is inquired according to the website identifier, and it is determined whether the website identifier belongs to the white list of the website. If the website identifier belongs to the white list of the website, the shopping website corresponding to the website identifier is determined to provide the trial installation service, and the execution is performed. In step 230, if the website identifier does not belong to the whitelist of the website, it is determined that the shopping website corresponding to the website identifier does not provide the trial service, and step 250 is performed.

Step 230: Obtain a screenshot, and extract, by the screenshot, a clothing identifier corresponding to the trial installation request.

Step 240: Acquire a clothing model corresponding to the clothing identifier.

The mobile terminal acquires the clothing model corresponding to the clothing identification through the API provided by the shopping website, and after obtaining the clothing model, jumps to step 270.

Step 250: Obtain a screenshot, and extract, by the screenshot, attribute information of the clothing corresponding to the trial installation request and a picture of the clothes corresponding to the trial installation request.

When the shopping website does not provide the trial service, the multi-frame page screenshot of the page corresponding to the trial request is obtained by the frame buffer Framebuffer. Thereby, the text information and the picture information in the screenshot of the page are respectively extracted, the attribute information of the clothes is obtained in the text information, and the front picture and the reverse picture of the clothes are obtained from the picture information. In one embodiment, the attribute information of the garment includes, but is not limited to, a neck circumference, a chest circumference, a waist circumference, a shoulder width, an arm length, a hip circumference, and an length of the garment, wherein the length of the garment includes a length of the upper garment, a length of the pants, and a length of the skirt.

Step 260: Input the attribute information of the picture and the clothing into a preset conversion model, and obtain a clothing model output by the conversion model.

In an embodiment, the conversion model is a deep neural network pre-configured in the mobile terminal and trained according to the clothing sample model and the clothing sample attributes. The front view picture, the reverse picture, and the attribute information of the clothes corresponding to the trial request are input into the conversion model, and a three-dimensional model corresponding to the clothes can be obtained and recorded as a clothes model.

Step 270: Query a preset white list according to the login name or the user name corresponding to the shopping website to obtain a human body model.

Step 280: Fusion the clothing model to the human body model to obtain a trial model, and display the trial model.

Step 290: Obtain a historical location record of the user, and determine a resident location and a time in the resident location according to the historical location record.

In an embodiment, the historical location record includes a historical location of the user and a historical time of staying at the historical location. For example, suppose that User A often appears in the xx building from 9:00 to 17:00 on weekdays, then the xx building is recorded as the historical location of User A, and 9:00 to 17:00 is the historical time of the user at a certain historical location.

Exemplarily, the mobile terminal determines the location of the user by using a Global Positioning System (GPS) and a Wireless Fidelity (WIFI) Service Set Identifier (SSID), and the location is The dwell time association at this location is stored in the location whitelist.

In the embodiment of the present application, the operation of acquiring the user historical location record is triggered by the trial installation request, and the operation of acquiring the user historical location record is performed in parallel with the operation of obtaining the trial installation model, and the parallel execution can improve the execution efficiency of the mobile terminal. In an embodiment, the operation of acquiring the user history location record may also be triggered by the obtained trial model. Therefore, after the trial model is obtained, the operation of acquiring the user history position record in this step is performed to avoid the occurrence of a jam phenomenon in the process of realizing the fitting function. The historical position record is analyzed, and the position where the number of occurrences exceeds the set number of times threshold and the dwell time exceeds the set time threshold is extracted, the position is taken as the resident position, and the dwell time at the resident position is recorded as the resident position time. .

Step 2100: predict a user occupation according to the resident location and time.

Using the means of big data analysis, the user occupation is predicted based on the above-mentioned resident location and the time at the resident location. For example, if a user stays at the Agricultural Bank xx branch between 8:30 and 17:30 every day and stays for more than 6 hours, the user is determined to be a bank employee.

Step 2110: Predict the user's consumption demand according to the user occupation and the clothing corresponding to the trial installation request.

The purchase preferences of users of different occupations are obtained by analyzing the purchase records of users of different occupations in advance. The current user's possible purchase preference is determined according to the current user occupation, and the reference purchase preference matching the current user is determined in conjunction with the clothing corresponding to the trial request. Then, the mobile terminal acquires feature information (eg, style, function, style, etc.) of the clothing corresponding to the trial request, and generates a current user's consumption demand according to the reference purchase preference and the feature information.

Step 2120: Query, by the shopping website, a target item that meets the consumption demand, and display the target item.

The mobile terminal queries the shopping website as the query condition by using the consumer demand, obtains the target product that meets the consumption demand, and displays the target product.

The embodiment of the present application does not limit the execution order of steps 290 to 2120.

It is assumed that the execution is performed in the order listed in the above steps, that is, after the trial model is displayed, the operation of determining the consumer demand is performed, and the occurrence of the jamming phenomenon can be avoided.

It is assumed that when the trial installation request is detected, the operation of predicting the consumption demand is performed in parallel with the operation of determining the trial model, and the target commodity can be displayed when the trial model is displayed. In an embodiment, when the trial installation model is displayed on the trial installation function interface, the thumbnail of the target product is displayed in a small window, and the thumbnail can be automatically replaced according to the operation instruction of the next batch displayed by the user. When it is detected that the user selects a thumbnail corresponding to a certain garment, the clothing model corresponding to the garment is obtained, the garment model is fused to the human body model to obtain a trial model, and the trial model is displayed. For example, the user's trial request selects a top garment, and according to the user's occupation and the characteristic information of the jacket, it is predicted that the user may purchase a pair of pants, and the pants that the professional crowd frequently buys are displayed in the form of thumbnails, and the user input is obtained. Choice of operation. When a selection operation for a certain trouser is detected, the clothing model corresponding to the trousers is acquired, and the clothing model corresponding to the trousers is fused to the human body model, and the user's own "wearing" the selected blouse and the fitting model of the trousers are displayed.

In an embodiment, the target item selected by the user is obtained, and it is determined whether the target item is available for sale online. If the item is available online, the user is prompted to select an online purchase or an offline purchase. When it is detected that the user selects the offline purchase, the offline store closest to the user is searched according to the current location of the user, and the information of the offline store is displayed on the screen. In an embodiment, the user clicks on the offline store, and can also provide a navigation route from the current location to the offline store, so that the user can quickly find the offline store.

In an embodiment, an online can be understood as an activity initiated on the network, and all or most of the activities on the network; offline can be understood as a real-life activity as opposed to network activity.

The technical solution of the embodiment obtains the historical location record of the user, determines the resident location and the time of the resident location according to the historical location record; predicts the user occupation according to the resident location and time; and according to the user occupation and the trial installation request The corresponding clothes predict the user's consumption demand; the target product that meets the consumer demand is searched by the shopping website, and the target product is displayed, and the target product can be pushed by the mobile terminal to the user in combination with the clothing corresponding to the user occupation and the trial installation request, so as to avoid the shopping website only based on the The purchase record pushes the product inaccurately, which improves the accuracy of the push product, thereby increasing the push purchase rate and enriching the functions of the mobile terminal.

Embodiment 3

FIG. 3 is a structural block diagram of an apparatus for simulating user trial installation according to an embodiment of the present application. The device may be implemented by software and/or hardware, and may be integrated into a mobile terminal, such as a mobile terminal having a 3D depth camera, and configured to perform the method of simulating user trial installation provided by the embodiment of the present application. As shown in Figure 3, the device comprises:

The request detection module 310 is configured to obtain a trial installation request when accessing the shopping website;

The information obtaining module 320 is configured to acquire, after detecting the trial installation request, a clothing model corresponding to the trial installation request and a user identifier corresponding to the shopping website;

The trial model generation module 330 is configured to acquire a human body model of the user according to the user identifier, fuse the clothing model to the human body model to obtain a trial model, and display the trial model.

The technical solution of the present embodiment provides a device for simulating a user's trial installation, which can solve the problem that the user cannot intuitively determine whether the style or model of the clothing meets the requirements in the network shopping scenario in the related art, so that the mobile terminal can provide simulated reality shopping. The function of the scene realizes the effect of displaying the user's own trial installation intuitively, enriching the functions of the mobile terminal.

In an embodiment, the apparatus further comprises:

a human body model generating module configured to: when the trial installation function is initialized, acquire a 3D depth camera to capture a user image obtained by the user according to a preset direction; and generate a human body model according to the user image, and the human body model and the user identifier The association is stored in a preset whitelist.

In an embodiment, the human body model generation module includes:

a video capture sub-module configured to obtain a user video captured by a 3D depth camera around the user;

The image acquisition sub-module is configured to perform frame processing on the user video by using a set framing strategy to obtain a set number of user images with depth information.

In an embodiment, the information acquisition module 320 is configured to:

In the case that the shopping website provides a trial service, a screenshot is obtained, and the clothing identifier corresponding to the trial installation request is extracted from the screenshot;

Obtaining a clothing model corresponding to the clothing identification.

In an embodiment, the information acquisition module 320 is configured to:

In the case that the shopping website does not provide the trial service, the screenshot is obtained, and the attribute information of the clothing corresponding to the trial request and the picture of the clothes corresponding to the trial request are extracted from the screenshot;

Entering the picture and the attribute information of the clothing into a preset conversion model to obtain a clothing model output by the conversion model, wherein the conversion model is a deep neural network trained according to a clothing sample model and a clothing sample attribute. The sample properties of the clothes include neck circumference, chest circumference, waist circumference, shoulder width, arm length, hip circumference and length.

In an embodiment, the trial model generation module 330 is configured to:

The preset whitelist is queried according to the login name or the user name corresponding to the shopping website to obtain a human body model.

In one embodiment, the garment model is fused to the mannequin to obtain a trial model and the trial model is displayed.

In an embodiment, the apparatus further comprises:

a pushing module, configured to acquire a historical location record of the user, determining, according to the historical location record, a resident location of the user and a time when the user is in the resident location; according to the resident location of the user and Predicting the occupation of the user when the user is in the resident position; predicting the consumption demand of the user according to the occupation of the user and the clothing corresponding to the trial installation request; querying the matching website in the shopping website The target commodity of the consumer demand, showing the target commodity.

In an embodiment, the push module is further configured to:

Obtaining the target item selected by the user;

In the case where it is determined that the target item selected by the user is available offline, the user is prompted to select to purchase the selected target item online or to purchase the selected target item offline.

In an embodiment, the push module is further configured to:

Providing, in the case that the selected target item is purchased under the user selection line, providing a sales position under the target item online;

Providing a navigation route from the current location of the user to the sales location.

Embodiment 4

The embodiment of the present application further provides a storage medium containing computer executable instructions for performing the method of any of the embodiments of the present application when executed by a computer processor.

In an embodiment, the computer-executable instructions, when executed by a computer processor, are used to perform the method of simulating user trial installation provided by the embodiments of the present application, the method comprising:

Get a trial installation request when visiting a shopping site;

When the trial installation request is detected, acquiring a clothing model corresponding to the trial installation request and a user identifier corresponding to the shopping website;

Acquiring a human body model of the user according to the user identifier, fusing the clothing model to the human body model to obtain a trial model, and displaying the trial model.

Storage medium Any one or more types of memory devices or storage devices. The term "storage medium" is intended to include: a mounting medium such as a Compact Disc Read-Only Memory (CD-ROM), a floppy disk or a tape device; a computer system memory or a random access memory such as a dynamic random access memory; (Dynamic Random Access Memory, DRAM), (Double Data Rate Random Access Memory, DDR RAM), Static Random Access Memory (SRAM), Extended Data Output Random Access Memory (Extended Data Output Random Access Memory) , EDO RAM), Rambus Random Access Memory (Rambus RAM), etc.; non-volatile memory such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory components Wait. The storage medium may also include other types of memory or a combination thereof. Additionally, the storage medium may be located in a first computer system in which the program is executed, or may be located in a different second computer system, the second computer system being coupled to the first computer system via a network, such as the Internet. The second computer system can provide program instructions to the first computer for execution. The term "storage medium" can include two or more storage media that can reside in different locations (eg, in different computer systems connected through a network). A storage medium may store program instructions (eg, embodied as a computer program) executable by one or more processors.

Certainly, a storage medium containing computer executable instructions provided by the embodiments of the present application, the computer executable instructions are not limited to the operation of simulating user trial installation as described above, and may also perform the simulation provided by any embodiment of the present application. Related operations in the user's method of trial installation.

Embodiment 5

The embodiment of the present application provides a mobile terminal, where the mobile terminal has an operating system, and the device for simulating user trial installation provided by the embodiment of the present application can be integrated into the mobile terminal. The mobile terminal can be a smart phone or a portable device (PAD). FIG. 4 is a structural block diagram of a mobile terminal according to an embodiment of the present application. As shown in FIG. 4, the mobile terminal can include a camera 410, a memory 420, and a processor 430. The camera 410 is a 3D depth camera, and the user image including the depth of field information can be captured by using a structured light scheme. The memory 420 is configured to store a computer program, a human body model, a trial model, and the like. The processor 430 reads and executes a computer program stored in the memory 420. The processor 430, when executing the computer program, implements the steps of: obtaining a trial installation request when accessing the shopping website; acquiring the clothing model corresponding to the trial installation request, and detecting the trial installation request; a user identifier corresponding to the shopping website; acquiring a human body model of the user according to the user identifier, fusing the clothing model to the human body model to obtain a trial model, and displaying the trial model.

The camera, memory and processor listed in the above examples are all components of the mobile terminal, and the mobile terminal may also include other components. Taking a smart phone as an example, the possible structure of the above mobile terminal will be described.

FIG. 5 is a structural block diagram of a smart phone according to an embodiment of the present application. As shown in FIG. 5, the smart phone may include: a memory 501, a central processing unit (CPU) 502 (also referred to as a processor, hereinafter referred to as a CPU), a peripheral interface 503, and a radio frequency (RF) circuit. 505, audio circuit 506, speaker 511, display 512, camera 513, power management chip 508, input/output (I/O) subsystem 509, other input/control devices 510, and external port 504, these components pass One or more communication buses or signal lines 507 are in communication.

The illustrated smartphone 500 is merely one example of a mobile terminal, and the smartphone 500 may have more or fewer components than those shown in the figures, two or more components may be combined, or may have different Component configuration. The various components shown in the figures can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

The following describes the smart phone of the device for simulating the user's trial installation provided by this embodiment.

The memory 501 can be accessed by the CPU 502, the peripheral interface 503, etc., and the memory 501 can include a high speed random access memory, and can also include a non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices. Or other volatile solid-state storage devices. The computer program is stored in the memory 511, and the human body model, the trial model, the preset white list, and the like may be stored, wherein the preset white list includes a plurality of white lists, and the first white list may store the human body model and the The user identification relationship, the second white list may be a website white list for storing the website identifier of the shopping website, and the third white list may be a location white list for storing the user location and the stay time at the user location.

Peripheral interface 503, which can connect the input and output peripherals of the device to CPU 502 and memory 501.

I/O subsystem 509, which can connect input and output peripherals on the device, such as display 512 and other input/control devices 510, to peripheral interface 503. The I/O subsystem 509 can include a display controller 5091 and one or more input controllers 5092 for controlling other input/control devices 510. Wherein, one or more input controllers 5092 receive electrical signals from other input/control devices 510 or transmit electrical signals to other input/control devices 510, and other input/control devices 510 may include physical buttons (press buttons, rocker buttons, etc.) ), dial, slide switch, joystick, click wheel. It is worth noting that the input controller 5092 can be connected to any of the following: a keyboard, an infrared port, a Universal Serial Bus (USB) interface, and a pointing device such as a mouse.

The display 512 can be a touch screen. The display 512 is an input interface and an output interface between the user terminal and the user, and displays the visual output to the user. The visual output can include graphics, text, icons, videos, and the like.

The camera 513 acquires an optical image of the user's human body using a structured light scheme, and converts the optical image into an electrical signal, which is stored in the memory 501 through the peripheral interface 503.

Display controller 5091 in I/O subsystem 509 receives an electrical signal from display 512 or transmits an electrical signal to display 512. Display 512 detects contact on the display, display controller 5091 converts the detected contact into interaction with a user interface object displayed on display 512, i.e., enables human-computer interaction, and the user interface object displayed on display 512 can be operational The icon of the game, the icon of the network to the corresponding network, and the like. In an embodiment, display 512 is a screen. In an embodiment, the device may also include a light mouse, which is a touch sensitive surface that does not display a visual output, or an extension of a touch sensitive surface formed by the screen.

The RF circuit 505 is configured to establish communication between the mobile phone and the wireless network (ie, the network side) to implement data reception and transmission between the mobile phone and the wireless network. For example, sending and receiving short messages, emails, and the like. In one embodiment, RF circuit 505 receives and transmits an RF signal, also referred to as an electromagnetic signal, and RF circuit 505 converts the electrical signal into an electromagnetic signal or converts the electromagnetic signal into an electrical signal, and through the electromagnetic signal and communication network And other devices to communicate. RF circuitry 505 may include known circuitry for performing these functions including, but not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a codec CODER-DECoder (CODEC) chipset, Subscriber Identity Module (SIM), etc.

The audio circuit 506 is arranged to receive audio data from the peripheral interface 503, convert the audio data into an electrical signal, and transmit the electrical signal to the speaker 511.

The speaker 511 is arranged to restore the voice signal received by the handset from the wireless network via the RF circuit 505 to sound and play the sound to the user.

The power management chip 508 is configured to provide power and power management for the hardware connected to the CPU 502, the I/O subsystem 509, and the peripheral interface 503.

The mobile terminal provided by the embodiment of the present invention can solve the problem that the user can not intuitively determine whether the style or model of the clothing meets the requirements in the network shopping scenario in the related art, so that the mobile terminal can provide a function of simulating a real shopping scene, and realize the intuitive function. Show the user's own trial installation effect, enrich the function of the mobile terminal.

The device for simulating user trial installation, the storage medium and the mobile terminal provided in the above embodiments can perform the method for simulating user trial installation provided by any embodiment of the present application, and have the corresponding functional modules and effects for executing the method. For technical details that are not described in detail in the above embodiments, reference may be made to the method of simulating user trial installation provided by any embodiment of the present application.

Claims (20)

A method of simulating a user's trial installation, including: Get a trial installation request when visiting a shopping site; When the trial installation request is detected, acquiring a clothing model corresponding to the trial installation request and a user identifier corresponding to the shopping website; Acquiring a human body model of the user according to the user identifier, fusing the clothing model to the human body model to obtain a trial model, and displaying the trial model. The method of claim 1 further comprising: When the trial installation function is initialized, acquiring a three-dimensional 3D depth camera to capture a user image obtained by the user according to a preset direction; Generating a human body model according to the user image, and storing the human body model in association with the user identifier in a preset white list. The method according to claim 2, wherein the acquiring the user image obtained by the user according to the preset direction by the 3D depth camera comprises: Obtaining a user video captured by a 3D depth camera around the user; The user video is subjected to framing processing by using a set framing strategy to obtain a set number of user images including depth information. The method of claim 1, wherein the obtaining the clothing model corresponding to the trial request comprises: In the case that the shopping website provides a trial service, a screenshot is obtained, and the clothing identifier corresponding to the trial installation request is extracted from the screenshot; Obtaining a clothing model corresponding to the clothing identification. The method of claim 1, wherein the obtaining the clothing model corresponding to the trial request comprises: In the case that the shopping website does not provide the trial service, the screenshot is obtained, and the attribute information of the clothing corresponding to the trial request and the picture of the clothes corresponding to the trial request are extracted from the screenshot; Entering the picture and the attribute information of the clothing into a preset conversion model to obtain a clothing model output by the conversion model, wherein the conversion model is a deep neural network trained according to a clothing sample model and a clothing sample attribute. The sample properties of the clothes include neck circumference, chest circumference, waist circumference, shoulder width, arm length, hip circumference and length. The method according to claim 1, wherein the acquiring the human body model of the user according to the user identifier comprises: The preset whitelist is queried according to the login name or the user name corresponding to the shopping website to obtain a human body model. The method of any of claims 1-6, further comprising: Obtaining a historical location record of the user, determining, according to the historical location record, a resident location of the user and a time when the user is in the resident location; Predicting the occupation of the user according to the resident location of the user and the time when the user is in the resident location; Predicting the user's consumption demand according to the occupation of the user and the clothing corresponding to the trial installation request; Querying the target product that meets the consumption demand in the shopping website, and displaying the target product. The method of claim 7, after displaying the target item, further comprising: Obtaining the target item selected by the user; In the case where it is determined that the target item selected by the user is available offline, the user is prompted to select to purchase the selected target item online or to purchase the selected target item offline. The method of claim 8 further comprising: Providing, in the case that the selected target item is purchased under the user selection line, providing a sales position under the target item online; Providing a navigation route from the current location of the user to the sales location. A device for simulating a user's trial installation, comprising: Requesting a detection module, configured to obtain a trial installation request when accessing the shopping website; The information acquiring module is configured to acquire, when the trial installation request is detected, a clothing model corresponding to the trial installation request and a user identifier corresponding to the shopping website; The trial model generating module is configured to acquire a human body model of the user according to the user identifier, fuse the clothing model to the human body model to obtain a trial model, and display the trial model. The apparatus of claim 10 further comprising: The human body model generating module is configured to: when the trial installation function is initialized, acquire a three-dimensional 3D depth camera to capture a user image obtained by the user according to a preset direction; and generate a human body model according to the user image, and the human body model and the user The identity association is stored in a preset whitelist. The apparatus of claim 11, wherein the human body model generation module comprises: a video capture sub-module configured to obtain a user video captured by a 3D depth camera around the user; The image acquisition sub-module is configured to perform frame processing on the user video by using a set framing strategy to obtain a set number of user images including depth information. The apparatus according to claim 10, wherein said information acquisition module is configured to: In the case that the shopping website provides a trial service, a screenshot is obtained, and the clothing identifier corresponding to the trial installation request is extracted from the screenshot; Obtaining a clothing model corresponding to the clothing identification. The apparatus according to claim 10, wherein said information acquisition module is configured to: In the case that the shopping website does not provide the trial service, the screenshot is obtained, and the attribute information of the clothing corresponding to the trial request and the picture of the clothes corresponding to the trial request are extracted from the screenshot; Entering the picture and the attribute information of the clothing into a preset conversion model to obtain a clothing model output by the conversion model, wherein the conversion model is a deep neural network trained according to a clothing sample model and a clothing sample attribute. The sample properties of the clothes include neck circumference, chest circumference, waist circumference, shoulder width, arm length, hip circumference and length. The apparatus of claim 10 wherein said trial model generation module is configured to: The preset whitelist is queried according to the login name or the user name corresponding to the shopping website to obtain a human body model. The apparatus of any of claims 10-15, further comprising: a pushing module, configured to acquire a historical location record of the user, determining, according to the historical location record, a resident location of the user and a time when the user is in the resident location; according to the resident location of the user and Predicting the occupation of the user when the user is in the resident position; predicting the consumption demand of the user according to the occupation of the user and the clothing corresponding to the trial installation request; and querying the matching in the shopping website The target item of the consumer demand displays the target item. The apparatus of claim 16 wherein said push module is further configured to: Obtaining the target item selected by the user; In the case where it is determined that the target item selected by the user is available offline, the user is prompted to select to purchase the selected target item online or to purchase the selected target item offline. The apparatus of claim 17, wherein the push module is further configured to: Providing, in the case that the selected target item is purchased under the user selection line, providing a sales position under the target item online; Providing a navigation route from the current location of the user to the sales location. A computer readable storage medium having stored thereon a computer program, the computer program being executed by a processor to implement the method of any of claims 1-9. A mobile terminal comprising a three-dimensional 3D depth camera, a memory, a processor and a computer program stored on the memory and operable on the processor, the 3D depth camera being arranged to capture a user image including depth of field information; the processor The method of any of claims 1-9 is implemented when the computer program is executed.

Images