KR20190082048A - Method and device to recommend customer item based on visual information - Google Patents

Abstract

The customer item recommendation apparatus according to an embodiment identifies a purchase intention of a customer based on a customer image, selects a purchase incentive model corresponding to the purchase intent to determine a recommendation item corresponding to the customer, And can provide association information.

Description

[0001] METHOD AND DEVICE TO RECOMMEND CUSTOMER ITEM BASED ON VISUAL INFORMATION [0002]

Hereinafter, techniques for recommending personalized goods and services are provided as technologies related to pattern recognition and computer vision.

Most of the existing product recommendation technologies recommend products using data related to purchase records of customers (for example, purchase records, product browsing records, etc.).

For example, based on a customer's online purchase record or online browse record, the product recommendation technology may recommend a product similar to a previously purchased or viewed product. However, since the above-described product recommendation technique can be applied only to an online purchase behavior of a customer, it can not be applied to a purchase behavior of a customer in a physical store.

When a customer is shopping in an offline store, an employee can provide a recommendation to the customer using the customer's purchase record. However, it is difficult for an employee to give an appropriate recommendation to a customer because the past purchase or browsing history is insufficient for a customer who visited an arbitrary offline store for the first time.

Augmented reality technology can be used to layer overlay related information for users in real life. Augmented reality technology can improve the quality of the user experience. In the field of existing applications, augmented reality technology can be applied in three areas: primarily display, content creation, and user interactions.

A method of recommending a customer item according to an exemplary embodiment includes: obtaining a customer image corresponding to a customer; Determining a recommended item for the customer based on the obtained customer image; And providing association information associated with the determined recommendation item.

Wherein the step of determining the recommendation item comprises: identifying a purchase tendency of the customer based on the acquired customer image; And determining the recommendation item for the customer based on the identified purchase propensity.

Wherein obtaining the customer image comprises obtaining at least one customer image using at least one camera, wherein the field of view of the at least one camera is stored in a physical store At least a portion of the purchase area can be covered.

The step of acquiring the customer image may include acquiring a customer image for at least one of the plurality of customers displayed at the user terminal.

Wherein the step of determining the recommendation item comprises: extracting an image feature for the customer from the customer image; And determining a recommendation item for the customer based on the image feature.

Wherein the step of determining the recommendation item comprises: retrieving a purchase propensity model corresponding to the customer from the purchase propensity model database based on the image characteristic; And determining the recommendation item for the customer based on the purchase propensity model corresponding to the customer.

Wherein the purchase propensity model database includes a plurality of purchase propensity models, wherein each of the plurality of purchase propensity models includes a feature vector of an item corresponding to a fashion category, a feature vector of a customer image, Wherein the step of retrieving the purchase propensity model comprises: determining, based on the image characteristics for the customer, a feature vector of the item corresponding to the fashion category, a feature vector of the customer image, Calculating a correlation level between the customer and each purchase propensity model according to the feature vector of the item; And determining a purchase propensity model having the highest correlation among the plurality of purchase propensity models as the purchase propensity model of the customer.

The customer item recommendation method may further include building the purchase propensity model database by building a purchase propensity model through an online purchase database.

The step of constructing the purchase propensity model database may include: learning statistics on online purchase records of a customer; Determining an eigenvalue of a customer from a result of disassembling the purchasing history matrix indicating the item purchase history of the customer; Classifying customers into a plurality of customer clusters based on the customer ' s unique value; And determining eigenvalues of the category based on the category-specific purchase data for each of the plurality of customer clusters.

The customer item recommendation method may further include building the purchase propensity model database by constructing the offline purchase propensity model through the offline shop purchase database.

The step of constructing the purchase propensity model database may include: extracting a customer ID, a customer image, and a customer purchase record from the offline store purchase database; Acquiring the customer's appearance information and the customer's purchase behavior information by analyzing the customer image; Classifying customers into a plurality of customer clusters based on information obtained from the customer image; And calculating a feature vector of the category based on the category-specific purchase data for each of the plurality of customer clusters.

The step of determining the recommendation item may include: matching an item image and a customer image of the item corresponding to each category in the purchase propensity model; And determining a recommendation item based on the fitting score calculated between the item image and the customer image.

Wherein the step of determining the recommendation item based on the calculated fitting score includes calculating an inner product between the feature vector of the individual item and the feature vector of the item worn by the customer in the purchase propensity model as the fitting score .

Wherein providing the association information associated with the recommendation item comprises: sending a push request to a customer terminal including an augmented reality display; And receiving information indicating whether to accept the push request from the customer terminal.

Providing association information associated with the recommendation item may include transmitting information associated with the recommendation item to the customer terminal in response to the push request being accepted by the customer terminal.

Providing association information associated with the recommendation item may include transmitting item information for a recommendation item from a location of the customer to a display disposed within a critical distance.

Providing association information associated with the recommendation item may include transmitting association information associated with the recommendation item to at least one of a business terminal and a customer terminal.

The step of determining the recommendation item may include determining the recommendation item based on the purchase area where the customer is located and the customer image.

A customer item recommendation apparatus according to an exemplary embodiment includes an image acquisition unit that acquires a customer image; And a processor for determining a recommendation item for the customer based on the acquired customer image and providing association information associated with the determined recommendation item.

1 is a flowchart illustrating a method of recommending a customer item according to an exemplary embodiment of the present invention.
2 is a flow chart illustrating a method for managing a customer based on image acquisition according to one embodiment.
FIG. 3 is a diagram illustrating a process of identifying a purchase behavior of a customer based on a customer image according to an exemplary embodiment.
4 is a diagram illustrating a process of building a purchase propensity model based on an online purchase database according to an embodiment.
5 is a flow diagram illustrating a method for building a purchase propensity model based on deep running in accordance with one embodiment.
FIG. 6 is a diagram illustrating a deep neural network designed to output a feature score from an item corresponding to a fashion category according to an embodiment.
7 is an explanatory diagram illustrating a method of constructing feature vectors corresponding to items corresponding to respective categories for each customer group according to an embodiment.
8 is a diagram illustrating a process of building a purchase propensity model based on an offline shop purchase database according to an embodiment.
9 is a diagram illustrating a neural network designed to segment a customer image according to one embodiment.
10 is a diagram illustrating a neural network designed to match an item image and an image segmented from a customer image according to one embodiment.
11 is a flow diagram illustrating a method for recommending an item according to an identified customer purchase preference according to one embodiment.
12 is a flowchart illustrating a method of recommending an item to an enhancement display device of a customer visiting an offline store according to an exemplary embodiment.
Figure 13 is a flow diagram illustrating a method for providing information associated with item recommendations to a display around a customer visiting an offline store in accordance with an embodiment.
14 is a flowchart illustrating a method for providing customer information and item recommendation to an augmented reality display device equipped with an in-store in-store sales assistant according to an exemplary embodiment.
15 is a flowchart illustrating a method for providing customer information and information associated with item recommendation on a face image displayed in a browser according to an exemplary embodiment.
16 is a block diagram illustrating a configuration of a customer item recommendation apparatus according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, the scope of the patent application is not limited or limited by these embodiments. Like reference symbols in the drawings denote like elements.

Various modifications may be made to the embodiments described below. It is to be understood that the embodiments described below are not intended to limit the embodiments, but include all modifications, equivalents, and alternatives to them.

The terms used in the examples are used only to illustrate specific embodiments and are not intended to limit the embodiments. The singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, the terms "comprises" or "having" and the like refer to the presence of stated features, integers, steps, operations, elements, components, or combinations thereof, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this embodiment belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the contextual meaning of the related art and are to be interpreted as either ideal or overly formal in the sense of the present application Do not.

In the following description of the present invention with reference to the accompanying drawings, the same components are denoted by the same reference numerals regardless of the reference numerals, and redundant explanations thereof will be omitted. In the following description of the embodiments, a detailed description of related arts will be omitted if it is determined that the gist of the embodiments may be unnecessarily blurred.

1 is a flowchart illustrating a method of recommending a customer item according to an exemplary embodiment of the present invention.

The customer item recommendation apparatus according to an exemplary embodiment can select information that a customer is interested in from a large amount of goods-related data for augmented reality shopping. The customer item recommendation device can identify the customer's purchase tendency to select information that the customer is interested in.

First, in step 110, the customer item recommendation apparatus can acquire a customer image corresponding to a customer. For example, the customer item recommendation device may acquire or receive at least one customer image. An agent of the physical store can input the customer image as a customer item recommendation device through the user interface.

Then, in step 120, the customer item recommendation apparatus can identify the purchase intention of the customer based on the customer image. For example, the customer item recommendation apparatus can search for a purchase propensity model corresponding to the customer based on the feature vector of the partial image segmented by the individual items from the customer image.

Then, in step 130, the customer item recommendation device may determine a recommendation item for the customer based on the acquired customer image. For example, the customer item recommendation device may determine a recommendation item based on the purchase propensity identified for the customer. The customer item recommendation apparatus can determine a recommendation item for the customer based on the purchase propensity model corresponding to the customer. The recommendation items may include, for example, recommended products and recommendation services.

In step 140, the customer item recommendation apparatus may provide association information related to the determined recommendation item. The association information associated with the recommendation item may include, for example, the category, class, and visual type of the recommendation item. For example, a customer item recommendation device may dispatch or visualize information associated with a recommended product or information associated with a recommended service to a customer.

2 is a flow chart illustrating a method for managing a customer based on image acquisition according to one embodiment.

According to one embodiment, the customer item recommendation device may include a plurality of cameras 209. [ The customer item recommendation apparatus can acquire a customer image using a plurality of cameras 209. [ A plurality of cameras 209 may be located within the area corresponding to the off-line store, and a view by the plurality of cameras 209 may cover at least some area within the off-line store. A plurality of cameras 209 may cover all zones within the off-line store. The plurality of cameras 209 may be mounted on a surveillance camera, a robot, a head-mounted display (HMD) device, a wearable glass, or the like.

First, in step 210, the customer item recommendation apparatus can search for a customer. According to one embodiment, the customer item recommendation device may detect a customer located in the purchase area of the offline store.

And in step 220 the customer item recommendation device may track the customer within the camera field of view. The customer item recommendation device can track a customer with a single camera, but not limited thereto, and can also track a customer using a plurality of cameras.

For example, in step 230, the customer item recommendation device may determine whether the customer being tracked is out of the camera field of view. In step 231, the customer item recommendation device may track a customer's location using another camera that provides a field of view close to the field of view of the camera in response to a customer's out of sight field of any camera. In addition, in response to a case where the customer is not available by any camera by at least one of the field of view, the resolution, the distance, and the lighting condition, the customer item recommendation device transmits the customer via another camera You can trace.

The customer item recommendation device may then determine in step 240 whether the customer has left the offline store area. The customer item recommendation device can estimate the customer's location from the tracking information obtained based on the camera and can determine whether the estimated customer's location is outside the offline store area. In step 241, the customer item recommendation device may terminate the trace for the customer and store the customer record information in response to the determination that the customer has already gone beyond the purchase area of the corresponding offline store. For example, the customer item recommendation device may store customer record information corresponding to the customer ID when the customer is out of the offline store area. The customer ID is described below.

And in step 250 the customer item recommendation device may track the customer and manage the customer ID while the customer is in the offline store area.

Then, in step 260, the customer item recommending device identifies the customer purchase propensity and recommends the item. For example, the customer item recommendation device may identify the detected customer in response to a customer located within the purchase area of the offline store being detected. The customer item recommendation apparatus can retrieve the customer ID matching the customer image corresponding to the detected customer. In response to the case where there is no matching result, the customer item recommendation apparatus can generate the customer ID for the customer. The customer ID is an identifier for distinguishing the customer, and the same customer ID may be assigned to the customer image indicating the same identification. The identity that the customer image points to may be identified, for example, through face recognition.

The customer item recommendation apparatus according to an exemplary embodiment may output a customer image and a customer ID to identify a customer's purchase tendency while searching and tracking the customer. The customer item recommending apparatus can recommend an item to a customer corresponding to the customer ID.

In addition, the customer item recommendation apparatus can receive a plurality of customer images in the course of searching and tracking a customer according to information required for each purchase incentive identification method. The customer item recommendation device may, for example, receive a plurality of customer images obtained at different times during tracking the customer.

The customer record information, which is retained when the customer is out of the purchase area of the off-line store, may include the customer ID and image data (e.g., customer image) indicating the customer, It is possible. The actual purchase record of the customer may include, for example, the type of product purchased by the customer, the product brand, the specification, the quantity, and the price. However, the purchase record is not limited to this, but may be changed according to the design.

FIG. 3 is a diagram illustrating a process of identifying a purchase behavior of a customer based on a customer image according to an exemplary embodiment.

The customer's purchase propensity identification process according to one embodiment may include two processes, that is, an offline procedure and an online procedure. The off-line process may include building a purchase propensity model database 340 from the purchase database 330. The online process may include searching for a purchase propensity model 350 suitable for any customer. For example, the on-line process may include a purchase propensity model database 340 based on the act of obtaining the customer image 310 and the visual features of the customer appearance identified from the customer image 310. [ May automatically determine a purchase propensity model 350 that is most appropriate for the customer.

According to one embodiment, the purchase propensity model database 340 may include a plurality of purchase propensity models 341. Each of the plurality of purchase propensity models 341 may be generated from the purchase database 330. Purchase database 330 may represent a database built on the basis of an individual customer's purchase action occurring in an offline store or online store. For example, the customer item recommendation device may acquire a plurality of purchase propensity models 341 through a statistical analysis (e.g., any algorithm) on the purchase database 330. In addition, the customer item recommendation device can statistically analyze purchase histories of customers exhibiting a similar purchasing tendency to generate a purchase history matrix. The customer item recommending device obtains a purchase tendency feature vector indicating the tendency to purchase items corresponding to different classes by decomposing the purchasing history matrix using collaborative filtering . Each purchase propensity model 350 may include a purchase propensity feature vector indicating the propensity to purchase an item.

Each of the plurality of purchase propensity models 341 according to one embodiment may represent one type of customer average purchase propensity that is representative. For example, the purchase propensity model 350 may be a model in which a plurality of customers represent a purchase propensity that represents a cluster of customers (customer clusters). The purchase propensity model 350 may include various feature vectors indicating purchase propensity of the customer cluster. For example, the purchase propensity model 350 may include a customer image feature vector, a feature vector 351 corresponding to a fashion category, and a feature vector 352 corresponding to a non-fashion category. In FIG. 3, for convenience of description, an image or an action that is the object of extraction of each feature vector of the purchase propensity model 350 is shown. However, the feature vector of the actual purchase propensity model 350 indicates an abstract value of the image or behavior As shown in FIG.

Items belonging to the fashion category in this specification can represent, for example, items that a person can wear. Items belonging to the non-fashion category can represent remaining items that do not belong to the fashion category. In addition, the fashion category and the non-fashion category can be classified into a plurality of classes, respectively. For example, the fashion category may include an upper class, a lower class, and an accessory class. Non-fashion categories may include brand classes, electronics classes, and the like. However, the categories included in each category are not limited to the above, and fashion category and non-fashion category may vary depending on the design.

The customer item recommending apparatus according to an exemplary embodiment of the present invention may be configured to receive a customer characteristic 310 from a corresponding customer image 310 in response to acquiring a customer image 310 for an arbitrary target customer A feature vector of a product corresponding to the customer image feature vector, and a customer image feature vector). In addition, the customer item recommendation device may generate a partial image 320 by segmenting the customer image 310 and extract additional features from the partial image 320.

For example, the customer item recommending apparatus can search for the purchase propensity model 350 most suitable for the target customer from the purchase propensity model database 340 based on the above-described characteristics. For example, the customer item recommendation apparatus may include a feature vector for the customer image extracted from the customer image 310 of the customer, a feature vector for the partial image 320, and a feature vector for the non- From the purchase incentive model database 340. The purchase incentive model database 340 may include a purchase incentive model 350, The customer item recommending apparatus can determine the retrieved purchase propensity model 350 as a purchase propensity model 350 for the target customer.

According to one embodiment, the customer item recommending apparatus can acquire personal information such as age, sex, race, etc. of the customer by analyzing the feature vector 361 for the customer image 310. For example, the customer item recommendation apparatus can extract the feature vector 361 based on the image recognition model from the customer image 310, and determine the age of the customer corresponding to the extracted feature vector 361 , Gender, race, and so on. The customer item recommending apparatus can extract the feature vector 361 based on the image recognition model from the face image among the customer image 310. [ The image recognition model may include a machine learning structure designed to output personal information (e.g., age and gender, etc.) corresponding thereto from an arbitrary human image.

According to one embodiment, the customer item recommending apparatus may collect statistical data 362 on a customer's purchasing behavior, etc., occurring in an offline store. For example, if a customer purchases a product sold in an off-line store, the customer item recommending apparatus stores a purchase count of the corresponding product in the customer ID corresponding to the customer, 362). In this specification, an offline store may represent a shop that occupies a physical area.

According to one embodiment, the customer item recommending apparatus can determine the customer's preference based on the feature vector 363 corresponding to the individual item corresponding to the fashion category included in the purchase propensity model 350. [ The customer item recommending device can determine the customer's preference by analyzing the customer's characteristics based on the determined purchase propensity model 350. [ For example, the customer item recommendation device may estimate the customer's preference from the partial image 320. The feature vector extracted from the partial image 320 indicating the item worn by the customer may represent a vector in which an image including an arbitrary item preferred by the customer cluster is abstracted. In addition, the feature vector 363 for an individual item may represent an abstract vector of images containing items preferred by customers belonging to the customer cluster. It may be highly likely that the customer would prefer an item having a feature vector similar to the feature vector for the partial image 320 in the purchase propensity model 350. [ Accordingly, the customer item recommendation apparatus can retrieve items having a feature vector similar to the feature vector extracted from the partial image 320 of the customer, from the items corresponding to the fashion category included in the purchase propensity model 350 .

For example, the customer item recommendation device may retrieve a garment item having a feature vector similar to the customer's partial image 320 of the garment items (e.g., top and bottom, etc.) included in the purchase propensity model 350 have. The customer item recommending device can provide recommendation information suitable for the customer's preference by recommending the retrieved item to the customer

The feature vector 352 corresponding to the non-fashion category may comprise a plurality of vector components. Each of the vector components included in the feature vector 352 corresponding to the non-fashion category may represent a non-fashion element corresponding to a different class. The non-fashion element may indicate, for example, the brand purchased by the customer, and the type of electronic product. However, the non-fashion category is not limited to this, and may include a class indicating various items other than the fashion item, and may be changed according to the design.

4 is a diagram illustrating a process of building a purchase propensity model based on an online purchase database according to an embodiment.

According to one embodiment, the customer item recommending apparatus can construct the purchase propensity model database 420 based on the online purchase database 410. [ The online purchase database 410 may include a large number of records from which multiple customers have purchased items (e.g., merchandise). The online purchase database 410 may include, but is not limited to, a customer ID, items purchased by the customer, browsing history of the customer, and the like. For example, an item purchased by a customer may include an item corresponding to a fashion category and an item corresponding to a non-fashion category.

First, the customer item recommendation apparatus can learn statistics on the customer's online purchase record from the online purchase database 410. [ Statistics for a customer's online purchase history may include, for example, M items and a purchase history matrix for N customers. The purchasing history matrix may indicate the purchase history of the customer. Here, N and M may be an integer of 1 or more. The i-th row of the purchase history matrix and the element of the j-th column may represent the purchase record of the i-th customer for the j-th item. i is an integer of 1 or more and N or less, and j is an integer of 1 or more and M or less.

The customer item recommendation device may determine a unique value of the customer from a result of decomposing the purchase history matrix (e.g., customer-item association vector). The eigenvalue can be expressed in the form of a customer feature vector. The calculation of the customer feature vector is described below with reference to FIG.

The customer item recommendation device can classify customers into a plurality of customer clusters by clustering customers based on the customer characteristic vector of the customer. The customer item recommending apparatus can calculate a similarity level between customers based on an inner product between customer characteristic vectors of individual customers. The customer item recommendation device can classify customers into individual customer clusters based on the degree of similarity between customers. An individual customer cluster may represent a cluster that includes customers with similar purchasing histories.

The customer item recommendation device can determine the eigenvalues of the category based on the purchase data for each category of the plurality of customer clusters (e.g., fashion category and non-fashion category, etc.). The eigenvalue of the category may be, for example, a purchase bias feature vector for that category.

The customer item recommending apparatus according to an exemplary embodiment may generate a purchase propensity model 430 corresponding to each customer cluster for each customer cluster. The purchase propensity model 430 may include various feature vectors representing the cluster. For example, the customer item recommendation device can decompose the purchasing history mattress to determine the purchasing propensity feature vector of each customer.

In this specification, the customer feature vector may represent a feature vector in which the purchase attributes of actual customers are abstracted. In addition, the purchase bias feature vector may represent a feature vector indicating a purchase tendency of a virtual customer, in which actual customers represent cluster clusters. The calculation of the customer feature vector and the purchase propensity feature vector is described in FIG.

For example, the purchase propensity model 430 mapped to an arbitrary customer cluster includes a purchase propensity model ID 431 indicating the customer cluster, a feature vector representing the customer cluster (for example, The feature vector 432 of the corresponding item, and the feature vector 433 corresponding to the non-fashion item).

For example, the customer item recommendation apparatus may include a purchase propensity model ID 431, a feature vector 432 of an item corresponding to a fashion category, and a feature vector 433 of an item corresponding to a non-fashion category, A purchase propensity model 430 may be assigned. The customer item recommendation apparatus can generate the purchase propensity model 430 by determining the feature vector of the customer cluster for each category based on the purchasing history matrix of each customer cluster.

5 is a flow diagram illustrating a method for building a purchase propensity model based on deep running in accordance with one embodiment.

First, in step 510, the customer item recommendation device may extract the customer-item association vector from the online purchase database 501. [ The customer-item association vector is a vector representing items purchased by the customer. For example, the length of the customer-item association vector may correspond to the number of kinds of items. The customer-item association vector may be generated based on one-hot encoding. For example, when an arbitrary customer purchases an arbitrary item, the customer-item association vector for the customer may have a value of 1 corresponding to the item. Conversely, if the customer does not purchase the item, the value of the element corresponding to the item in the corresponding customer-item association vector may be zero.

In step 520, the customer item recommendation apparatus can acquire an item image of an item corresponding to the fashion category from the online purchase database 501. [

Then, in step 530, the customer item recommending device may calculate the output based on the fashion item recommendation model. According to one embodiment, the fashion item recommendation model may represent a model (e.g., a neural network) designed to output the result of predicting whether any customer will purchase any item corresponding to a fashion category. The item item recommendation device may train a fashion item recommendation model based on an image of an item corresponding to a fashion category and a large amount of customer-item association vector for an item corresponding to the fashion category.

For example, the customer item recommending device may input an item image of an item corresponding to the customer-item association vector and the fashion category of the customer into the fashion item recommendation model. The fashion item recommendation model can generate output in the form of a prime number between 0 and 1. The output of the fashion item recommendation model can indicate the probability that the customer will purchase the item corresponding to the fashion category, and the closer to 0, the lower the probability, and the closer to 1, the higher the probability.

In step 540, the customer item recommendation device may acquire the customer feature vector. For example, the customer item recommendation apparatus can determine the feature vector output from the hidden layer disposed before the output layer of the fashion item recommendation model as the customer feature vector. The fashion item recommendation model may include a plurality of hidden layers, and the output of the hidden layer may represent information that an arbitrary customer's purchase propensity is abstracted. For example, the customer feature vector may represent, from the customer-item association vector, a feature vector extracted based on the hidden layer of the fashion item recommendation model. The extraction of the customer feature vector is described below with reference to FIG.

Then, in step 550, the customer item recommendation device may perform customer clustering. For example, the customer item recommendation device may obtain each customer characteristic vector of the various customers of the online purchase database 501, as in step 540 described above, and clusters customers based on each customer ' can do.

And in step 560 the customer item recommendation device may acquire a plurality of customer clusters. For example, a customer item recommender may classify customers having similar customer feature vectors into the same customer cluster. Customers belonging to each customer cluster may have a similar tendency to purchase items. Each customer cluster can be treated as a representative virtual customer.

Then, in step 570, the customer item recommending apparatus can construct a purchase propensity model database. For example, the customer item recommendation apparatus can acquire a purchase record of a virtual customer having a representative from the online purchase database 501. [ From the purchase record of the online purchase database 501, the customer item recommendation device can build a customer-item association matrix for a virtual customer having representative of each customer cluster. The customer item recommendation device obtains a purchase bias feature vector for items corresponding to different categories (e.g., fashion category and non-fashion category) by decomposing a customer-item association matrix for a customer representing each customer cluster . The purchase propensity model may include a purchase propensity feature vector for an item corresponding to another category.

FIG. 6 is a diagram illustrating a deep neural network designed to output a feature score from an item corresponding to a fashion category according to an embodiment.

The customer item recommendation apparatus can input two kinds of vectors to the fashion item recommendation model 690. [

For example, the customer item recommendation device may enter a customer-item association vector 611. The item item recommendation device may apply a reducing dimension mapping to the customer-item association vector 611. [ The dimension reduction mapping may be, for example, an operation by a single matrix, but is not limited thereto and may be an operation by a neural network. For example, if the customer-item association vector 611 is a K dimension before the dimension is reduced and the vector after the dimension reduction is an L dimension, then Matrix Factorization Techniques for Recommender Systems (Y. Koren et al., August 2009, IEEE Computer) technique may be applied to the customer-item association vector 611. Here, K and L may be integers greater than or equal to 1, and K may be greater than L. In another example, the customer item recommender may reduce the dimension of the customer-item association vector 611 through a neural network operation of a multi-layer perceptron (MLP) structure. If the dimension reduction mapping P is implemented in the first hidden layer 621, the customer item recommendation device may determine the reduced dimensioned customer-item association vector 611 as a customer feature vector. For example, the customer-item association vector 611 for the u-th customer may be denoted by v _u and the customer feature vector may be denoted by p = P ^T v _u . Here, u may be an integer of 1 or more.

In addition, the customer item recommendation apparatus can input the item image 612. [ The item item recommendation device may extract the item image feature 632 by entering the item image 612 into the convolutional neural network 631. The item image feature 632 may represent a vector in which the visual characteristics of the item are abstracted. The customer item recommendation apparatus can obtain the item feature vector by applying a dimension reduction mapping Q to the vector in which the visual characteristics of the item are abstracted through the second hidden layer 633. [ For example, the image feature 632 extracted from the item image 612 of the i-th fashion category through the convolutional neural network 631 may be denoted as v _i , and the item feature vector may be represented as q = Q ^T v _i . Here, i may be an integer of 1 or more.

The customer item recommendation device may integrate the customer feature vector and the item feature vector, via a cascade layer 640. The customer item recommendation device may input the integrated feature vector into a coding neural network. The number of nodes of the intermediate layer 650 of the coding neural network can be reduced as the upper layer. The intermediate layer 650 is a neural collaborative filtering layer, for example, that learns the interaction (e.g., correlation) between the customer feature vector p and the item feature vector q One layer can be represented.

The output layer 660 of the coding neural network may finally output a value between 0 and 1 as the output y _ui . The output y _ui may indicate the probability that the u-th customer will purchase items corresponding to the i-th fashion category. The closer the output y _ui is to 1, the higher the probability that the u-th customer will purchase items in the i-th fashion category.

According to one embodiment, the customer item recommender may generate a large amount of training data for the fashion item recommendation model 690 from an online purchase database. In addition, the customer item recommendation apparatus can train the fashion item recommendation model 690 through a back-propagation algorithm.

7 is an explanatory diagram illustrating a method of constructing feature vectors corresponding to items corresponding to respective categories for each customer group according to an embodiment.

First, in step 710, the customer item recommendation device can identify the category of items purchased by the representative customer of each customer cluster. The representative customer of each customer cluster may, for example, represent a virtual customer representing the cluster. The customer item recommendation apparatus can retrieve items purchased by each customer belonging to the customer cluster from the online purchase database 701. [ Assuming that each customer cluster is a virtual customer having a representative, the customer item recommending device can search for items purchased by a virtual customer having the representative.

In step 721, the customer item recommending device may select an item corresponding to the non-fashion category. For example, the customer item recommending device may select an item corresponding to the non-fashion category among the items purchased by the representative customer representing the customer cluster.

At step 722 and 725, the customer item recommendation device may then construct a purchase history matrix based on the customer-item association vector for the item corresponding to class A and the item corresponding to class B in the non-fashion category.

At step 723 and 726, the customer item recommendation apparatus can decompose the purchasing history matrix described above using a collaborative filtering method.

Then, in steps 724 and 727, the customer item recommendation apparatus can calculate the purchase bias characteristic vector of the item corresponding to class A and the purchase bias characteristic vector of the item corresponding to class B, through decomposition of the purchase history matrix.

And in step 728 the customer item recommendation device may determine a purchase bias feature vector of the item corresponding to the non-fashion category, based on the purchase bias feature vector corresponding to each of the classes belonging to the non-fashion category.

Also, in step 731, the customer item recommendation device may select an item corresponding to the fashion category and the item image.

In step 732, the customer item recommendation apparatus can extract an image feature of the corresponding item image.

At step 733, the customer item recommendation device may then obtain an average image characteristic of the item belonging to the individual class in the fashion category. For example, a customer item recommender may obtain an average image feature by averaging image features for multiple item images belonging to individual classes (e.g., an image, etc.) within a fashion category.

In step 734, the customer item recommendation apparatus may determine a purchase bias characteristic vector of the item corresponding to the fashion category.

Then, in step 740, the customer item recommendation apparatus determines a purchase propensity model for the customer cluster based on the purchase propensity feature vector of the item corresponding to the non-fashion category and the purchase propensity feature vector of the item corresponding to the fashion category Can be generated.

8 is a diagram illustrating a process of building a purchase propensity model based on an offline shop purchase database according to an embodiment.

FIG. 8 illustrates the construction of a purchase propensity model 830 through the offline store purchase database 810. FIG. The offline store purchase database 810 according to an exemplary embodiment may include a customer ID, an image of a customer, and a purchase history of a customer.

Information that can be obtained through image analysis of the customer image may include customer's appearance information, customer's purchase behavior information, and the like. The customer's appearance information may include the customer's age, gender, hairstyle, clothing features, hair accessory features, brand features of the accessory, race, gaze direction, and facial expression. The customer's purchase behavior information may include the customer's footprint in the offline store and the time the customer stays in different areas within the offline store, the route within the store, the conversation, and interaction with the item.

The customer item recommending apparatus can classify customers into a plurality of clusters based on information obtained from the customer image. For example, a customer item recommender may extract customer feature vectors from purchase behavior information (e.g., customer-item association vectors) and may cluster customers based on customer feature vectors.

The customer item recommendation apparatus can calculate the feature vector of the category based on the purchase data per category for each of the plurality of customer clusters. For example, the purchase propensity model database 820 may include a purchase propensity model 830 corresponding to a customer cluster classified according to a customer feature vector. For each customer cluster, the customer item recommender may obtain statistics of purchase data for items of different categories. The customer item recommendation apparatus can calculate feature vectors for items of different categories.

The purchase propensity model 830 constructed for each customer cluster includes a feature vector 832 of an item corresponding to a fashion category, a feature vector 833 of a customer image, a feature vector 834 of an item corresponding to a non-fashion category, And the like. The purchase propensity model ID 831 may be mapped to each purchase propensity model 830.

According to one embodiment, the customer item recommendation device may obtain a customer image for a target customer. The customer item recommendation device can extract the image characteristic of the customer from the customer image. The image characteristics of the customer may include, for example, the image characteristics of the customer's face, and the image characteristics of the item image for the individual items worn by the customer. Based on the extracted image characteristics, the customer item recommendation apparatus extracts, from the purchase propensity model database 820, a feature vector of an item corresponding to a fashion category included in each purchase propensity model 830, a feature vector of a customer image, Based on the feature vector of the item corresponding to the purchase order model 830, and the like. The customer item recommendation apparatus determines a correlation between a customer and each purchase propensity model according to a feature vector of an item corresponding to a fashion category, a feature vector of a customer image, and a feature vector of an item corresponding to a non-fashion category, A correlation level can be calculated. The customer item recommendation apparatus can select a purchase propensity model having the highest degree of correlation with respect to the customer among a plurality of purchase propensity models. The degree of correlation may indicate the degree to which any customer and any purchase propensity model are related to each other. The degree of correlation may be, for example, but is not limited to the average of the inner products calculated between the feature vector of the purchase propensity model and the feature vector of the item worn by the customer.

For example, the customer item recommendation apparatus can calculate the dot product between the image feature vector of the customer and the visual feature related vector of each purchase propensity model. The customer item recommending apparatus can determine the purchase propensity model 830 corresponding to the largest internal product among the acquired internal propensity for each purchase propensity model as a purchase propensity model 830 suitable for the customer.

9 is a diagram illustrating a neural network designed to segment a customer image according to one embodiment.

The customer item recommendation apparatus can divide the customer image 901 based on the image segmentation model 900. [ For example, the customer item recommendation device may divide the customer image 901 into a background and an item image based on the image segmentation model 900. The image segmentation model 900 may be implemented in a machine learning structure as shown in FIG. 9, but is not limited thereto. For example, the image segmentation model 900 may include a neural network.

The image segmentation model 900 may include a convolution network 910 that receives a customer image 901. The extracted feature data based on the convolution network 910 may be propagated to the pooling layer 920. [ The pooling layer may be coupled to a plurality of convolution networks 931, 932, 933, 934. Each of the plurality of convolution networks 931, 932, 933, and 934 may be composed of kernels of different sizes. The image segmentation model 900 may be designed to upsample 940 the output of a plurality of convolutional networks 931, 932, 933, 934.

Further, the extracted characteristic data based on the convolution network 910 can be propagated to the other convolution network 960. The output of convolution network 960 may propagate to fully connected layers 971 and 972. The output of the full connection layer 972 can be used to calculate the auxiliary loss function in the training process.

The output of the full connection layer 972 and the output of the upsampling 940 may be propagated to the output convolution network 950. The output convolution network 950 can generate a customer image 901 with a partitioned image 909. For example, the divided image 909 of the customer image 901 may represent the category and class to which each of the pixels constituting the customer image 901 belong. The image 909 in which the customer image 901 is divided may include a background, a hat, a hair, a face, a sunglasses, a shawl, a top, an over skirt, a coat, a glove, a pants, a dress, a mini skirt, Legs, right legs, left legs, right legs, socks, bags, hair accessories, and wrist watches.

Based on the pixel classification result, the customer item recommendation apparatus can extract, from the customer image 901, an image window corresponding to an item corresponding to a different fashion category. In addition, the customer item recommendation device can remove obstructions (e.g., the color and pattern of the background object) by removing pixels classified as background from the image window corresponding to the individual item. An image window may be referred to as a partial image.

10 is a diagram illustrating a neural network designed to match an item image and an image segmented from a customer image according to one embodiment.

The customer item recommending apparatus can extract a partial image 1001 corresponding to an item (for example, clothes) worn by the target customer in response to acquiring the customer image of the target customer. For example, as described above with reference to FIG. 9, the customer item recommendation apparatus can extract an image window corresponding to an individual item as a partial image 1001. [ The customer item recommending apparatus according to an embodiment retrieves, from the item images 1002 corresponding to the fashion category managed by the purchase propensity model selected for the target customer, an image most suitable for the partial image 1001 .

For reference, the characteristics of the garment image in the worn state and the characteristics of the garment image in the state of being displayed on the online purchase site can exhibit a remarkable difference even if they have the same clothing style. A customer item recommendation device according to one embodiment may include a client image of a customer cluster (e.g., partial image 1001) and a customer image of a customer cluster (e.g., a customer image) based on a style comparison model (e.g., And the item image 1002 corresponding to the fashion category of FIG. The style comparison model may represent a model trained to output fitting scores indicating style similarity 1090 between two item images.

10, the customer item recommendation apparatus may include a neural network having a multitasking twin convolutional network structure 1010. The multi- At least a portion of the network 1011 of the neural network includes a first branch convolution network that extracts features from a customer image (e.g., partial image 1001) and features from an item image 1002 that corresponds to any fashion category And a second branch convolution network for extracting the second branch convolution network. The first branch convolution network and the second branch convolution network may share the same parameters with each other. The parameters of the neural network may be, for example, connection weights.

In the output layer 1012 of the neural network, the first branch convolution network receives classes (e.g., top, bottom, back, and sunglasses, etc.) 1081 corresponding to the fashion category from the partial image 1001 Can be output. The second branch convolution network may output, at the output layer 1012, a class 1082 of items corresponding to the fashion category from the item image 1002. The class 1081 of the item image 1002 and the class 1082 of the partial image 1001 may be the same or different from each other.

The item item recommendation device may also output the embedding vectors 1071 and 1072, respectively, of the output layer 1012 of the first branch convolution network and the second branch convolution network. A customer item recommendation apparatus according to an embodiment can calculate style similarity 1090 between a partial image 1001 and an item image 1002 based on two embedding vectors 1071 and 1072. [ For example, the customer item recommendation apparatus can calculate an inner product between two embedding vectors 1071 and 1072. [ The dot product between the two embedding vectors 1071 and 1072 may indicate style similarity 1090 between the two images. For example, style similarity 1090 between two images may be expressed as a fitting score.

In addition, the customer item recommendation apparatus can train the style comparison model based on the loss function according to Equation (1). For example, the customer item recommendation device may train the style comparison model by updating the parameters of the style comparison model based on the loss according to Equation (1).

[Equation 1]

In the above-described equation (1), L _pos and L _neg can be expressed by the following equations (2) and (3).

&Quot; (2) "

&Quot; (3) "

In the above-described equations (1) to (3), it can be assumed that image pairs of (Iq, Ip) have the same style and image pairs of (Iq, In) have different styles. The customer item recommendation apparatus can acquire the embedding vectors fq, fp, and fn by mapping each of the images Iq, Ip, and In as an embedding layer. The loss function Loss described above in Equation (1) can have two penalties. Lpos represents a penalize function for an excessively positive pair, and Lneg may represent a penalty for a negative pair that is too close to the margin m. StyleSim (fp, fq) may represent a function for calculating similarity (e.g., vector dot product) between fp and fq and StyleSim (fn, fq) may represent a function for calculating the similarity between fn and fq .

11 is a flow diagram illustrating a method for recommending an item according to an identified customer purchase preference according to one embodiment.

According to one embodiment, the customer item recommending apparatus can match an item image and a customer image of the item corresponding to each category in the purchase propensity model. For example, as described above with reference to FIG. 10, the customer item recommendation device may calculate a fitting score between the item image and the head image based on the neural network. The customer item recommendation apparatus can determine a recommendation item based on the calculated fitting score.

First, in step 1110, the customer item recommendation apparatus can determine a category of an item to be recommended. For example, the customer item recommendation device may obtain a user input specifying an item category from a target customer, and may determine an item category based on user input. As another example, the customer item recommendation device can calculate a confidence level for an individual category based on the frequency of purchase for the individual category, the purchase area where the customer is located, and the sales strategy of the store, You can determine which category you have. As another example, the customer item recommendation device can construct a category model for category recommendation. The customer item recommending device can predict a category to be recommended to the customer using the category model based on the neural network.

In step 1120, the customer item recommendation apparatus can retrieve item information. The item item recommendation apparatus according to an embodiment may retrieve, from the item database, item information corresponding to the category determined in the step 1110 described above. The item information stored in the item database may include a feature vector that abstractly describes the attribute of the item. The customer item recommendation device can obtain the feature vector of the item by decomposing the matrix according to the cooperative filtering algorithm. A calculation method for obtaining the feature vector of an item can be found in "Matrix Factorization Techniques for Recommender Systems ".

In step 1130, the customer item recommendation device may identify a purchase bias feature vector corresponding to the item category from the customer's purchase bias model. According to one embodiment, the customer item recommendation device may identify a purchase propensity model corresponding to a target customer. For example, the customer item recommendation apparatus may calculate an inner product between a feature vector of a purchase propensity model and a feature vector of an item worn by a customer, and determine an average of the calculated inner products. The customer item recommendation apparatus can determine the purchase propensity model having the highest average of the dot products as the purchase propensity model corresponding to the customer. The customer item recommendation device may determine, from the purchase propensity model, a purchase propensity feature vector corresponding to the category determined in step 1110. [

In step 1140, the customer item recommendation device may calculate the fitting score of the item with the customer. For example, the customer item recommendation device may calculate an inner product between the purchase bias feature vector identified in step 1130 and the feature vector of the item obtained in step 1120, as a fitting score.

Then, in step 1150, the customer item recommendation device may recommend item k of the item with the highest fitting score to the customer. For example, the customer item recommendation apparatus can determine an item having the highest fitting score from the item having the kth highest fitting score as the recommendation item. k may be an integer of 1 or more.

The customer item recommendation device may provide recommendation information in response to the determination of the item to be recommended, for example, through the example illustrated in FIGS. 12 to 15 below.

12 is a flowchart illustrating a method of recommending an item to an enhancement display device of a customer visiting an offline store according to an exemplary embodiment.

In FIG. 12, the customer item recommendation apparatus may be implemented in the server 1210.

First, at step 1211, the server 1210 can detect and track the customer. For example, the server 1210 may detect a customer through a camera disposed in an off-line store.

And in step 1212 the server 1210 may send a push request to the customer terminal 1220 (customer terminal 1220). The customer terminal 1220 may be, for example, an augmented reality device. The augmented reality apparatus is an apparatus for providing augmented reality, for example, a helmet including an augmented reality display, glasses, a mobile phone, and the like.

In step 1221, the augmented reality application of the customer terminal 1220 may detect a push request. For example, customer terminal 1220 may receive push requests sent from server 1210 via wireless communications.

And in step 1222 the customer terminal 1220 may determine whether to accept the push request by the augmented reality management application. In response to accepting the push request, the customer terminal 1220 may send information indicating acceptance to the server 1210. The server 1210 may receive information indicating whether to accept the push request from the customer terminal 1220.

The server 1210 may then in step 1213 respond to the push request accepted by the customer terminal 1220 to predict the purchase propensity based on the customer image. For example, the server 1210 may determine a purchase propensity model corresponding to the customer identified from the customer image as described above.

And in step 1214 the server 1210 may recommend items. For example, the server 1210 can determine the categories and classes of items to recommend to the customer based on the purchase propensity model. The server 1210 can retrieve, from the item database 1280, items with a higher fitting score for the purchase propensity model. The server 1210 may determine an item with a higher fitting score as a recommendation item and transmit the information associated with the recommendation item to the customer terminal 1220. [

At step 1215, the server 1210 may then continue customer tracking. Then, in step 1216, the server 1210 may update the recommendation information based on the status of the customer while tracking the customer. For example, when a customer is located in a fruit section in an in-store store, an item corresponding to a fruit category can be recommended. As another example, when a customer is located in a document area in an offline store, an item corresponding to a document category may be recommended.

In step 1223, the customer terminal 1220 can receive item information from the server 1210. [ The item information may indicate information related to the item recommended by the server 1210. The customer terminal 1220 can visualize the item information through the augmented reality to provide the item information to the customer.

Figure 13 is a flow diagram illustrating a method for providing information associated with item recommendations to a display around a customer visiting an offline store in accordance with an embodiment.

In the example shown in FIG. 13, the customer item recommendation apparatus can be implemented as a server.

First, at step 1311, the server 1310 can detect and track the customer. For example, a camera connected to the server 1310 may be installed near an article shelf in an offline store.

In step 1312, the server 1310 can predict the purchase propensity. For example, the server 1310 may determine, in step 1311, a purchase propensity model corresponding to the customer based on the customer image, in response to the customer being detected.

In step 1313, the server 1310 may then recommend the item. For example, the server 1310 may retrieve an item corresponding to the purchase propensity model from the item database 1380 and determine the retrieved item as a recommendation item. The server 1310 may send the item information for the recommendation item to the additional display 1320. The server 1310 may, for example, estimate the location of the customer in the offline store and may transfer the item information for the recommendation item to the additional display 1320 located within the threshold distance from the estimated customer location.

And in step 1314 the server 1310 may continue customer tracking. The server 1310 may then update the item recommendation in step 1315. [ For example, the server 1310 may update an item recommendation for a customer based on a tracked customer status change (e.g., a customer's location change in an offline store, etc.).

Further, in step 1321, the additional display 1320 may visualize the item information. For example, the additional display 1320 may be connected to the server 1310 as a display installed in an offline store. The additional display 1320 may visualize item information for a recommendation item, as determined by the server 1310.

14 is a flowchart illustrating a method for providing customer information and item recommendation to an augmented reality display device equipped with an in-store in-store sales assistant according to an exemplary embodiment.

In FIG. 14, the customer item recommendation apparatus may be implemented in the server 1410.

First, at step 1411, the server 1410 can detect and track the customer.

Then, in step 1412, the server 1410 may predict the customer's purchase propensity. For example, the server 1410 may determine a purchase propensity model corresponding to the customer based on the customer image, in response to the customer being detected by the camera installed in the off-line store.

In step 1421, the salesperson terminal 1420 may provide customer information. The salesperson terminal 1420 can be implemented as an augmented reality device, and is described as an augmented reality eyeglass in Fig. The salesperson terminal 1420 can assist the salesperson by visualizing the customer information identified based on the customer image to the salesperson. The salesperson terminal 1420 may be referred to as a business terminal.

In step 1413, the server 1410 may then recommend the item. For example, the server 1410 may retrieve an item corresponding to a purchase propensity model selected for the customer from the item database 1480 and determine the retrieved item as a recommendation item.

In step 1422, the salesperson terminal 1420 may provide item information. For example, the server 1410 may provide the item information related to the recommendation item to the salesman terminal 1420. [ The salesperson terminal 1420 can visualize the item information related to the recommendation item provided from the server 1410. [

The server 1410 can then continue to track the customer in step 1414. And in step 1415 the server 1410 may update the item recommendation. For example, in response to a change in the status of a customer located in front of a salesperson, the server 1410 may update the item recommendation.

Instead of the salesperson wearing the salesperson terminal 1420, the robot may be placed in the off-line store. The robot can visualize the image or perform the function of communicating with the customer. Similar to the method described in Fig. 14, the robot can provide item information related to a recommendation item to a customer.

Accordingly, the customer item recommendation apparatus according to an exemplary embodiment can provide a convenient shopping service to a customer.

15 is a flowchart illustrating a method for providing customer information and information associated with item recommendation on a face image displayed in a browser according to an exemplary embodiment.

In FIG. 15, the customer item recommendation apparatus may be implemented as a user terminal 1510.

First, in step 1511, the user terminal 1510 may obtain a customer image. The user terminal 1510 may obtain a customer image for at least one customer of the plurality of customers displayed in the browser of the user terminal 1510. [ For example, the user terminal 1510 may select at least some customers in response to user input that specifies at least some of the plurality of customers. The user terminal 1510 may obtain a customer image for at least some selected customers. The user terminal 1510 may be, but is not limited to, a customer device or a sales agent device.

And in step 1512 the user terminal 1510 may identify the customer. For example, the user terminal 1510 may identify the customer represented in the customer image from the acquired customer image.

The user terminal 1510 may then identify the purchase propensity in step 1513. For example, the user terminal 1510 may determine a purchase propensity model corresponding to the customer based on the customer image.

And in step 1514 the user terminal 1510 may visualize the purchase propensity. For example, the user terminal 1510 may display the purchase propensity model corresponding to the customer in order in the browser 1590. In FIG. 15, the user terminal 1510 can visualize that the propensity A is 95%, propensity B is 85%, propensity C is 83%, which is suitable for the customer.

The user terminal 1510 can then recommend the item in step 1515. [ For example, the user terminal 1510 may determine a recommendation item from the databases 1521, 1522, 1523 stored in the service provider server 1520 in accordance with a customer's purchase propensity model.

And in step 1516 the user terminal 1510 may visualize the recommendation item. The user terminal 1510 may display at least one recommendation item in accordance with the customer's purchase propensity model in the browser 1590 of the user terminal 1510 in order according to the fitting score. For example, in FIG. 15, the user terminal 1510 can visualize the information fitted to the customer as 98% for product A, 87% for product B, and 54% for product C, respectively.

16 is a block diagram illustrating a configuration of a customer item recommendation apparatus according to an embodiment.

The customer item recommendation apparatus 1600 may include an image acquisition unit 1610, a processor 1620, and a memory 1630.

The image acquiring unit 1610 can acquire a customer image. For example, the customer image can be an image sensor (e.g., a camera) installed in an off-line store, and can continuously capture a customer image corresponding to a target customer while tracking the target customer.

The processor 1620 may determine a recommendation item for the customer based on the obtained customer image and provide association information associated with the determined recommendation item. The processor 1620 may perform at least some of the operations of the method described above with respect to Figures 1-15.

Memory 1630 may store the data required to perform the customer item recommendation method temporarily or permanently. For example, the memory 1630 may include a purchase propensity model, an image recognition model, a fashion item recommendation model, an image segmentation model, a style comparison model, and a category model.

The customer item recommendation device 1600 according to one embodiment can build a purchase propensity model for any customer cluster based on a large amount of customer purchase history. The customer item recommendation apparatus 1600 can capture an arbitrary target customer with a camera and analyze the visual characteristics of the target customer using the deep learning technique to search for a purchase propensity model most suitable for the target customer. The customer item recommendation apparatus 1600 can determine an item to be recommended to the target customer based on the retrieved purchase propensity model.

Also, the customer item recommendation apparatus 1600 can transmit the association information associated with the recommendation item to the augmented reality apparatus worn by the customer in the offline store. In addition, the customer item recommendation apparatus 1600 can assist the salesperson by providing information on the recommendation item to the salesperson of the offline store. In addition, the customer item recommendation apparatus 1600 may provide an augmented reality based on the customer's purchase intention information to an image visualized in the browser of the user apparatus.

Therefore, the customer item recommendation apparatus 1600 does not request the purchase data recording of the customer in order to provide the information on the recommendation item to any customer. In addition, the customer item recommendation device 1600 does not require the customer to enter data or select a product. The customer item recommendation apparatus 1600 can recommend an item including a visual feature such as an item corresponding to a fashion category and can also recommend an item corresponding to a non-fashion category that does not have a visual feature.

According to one embodiment, the customer item recommendation apparatus 1600 may provide personalized recommendation information on goods and services related to high quality to the customer in the augmented reality shopping application, according to the purchasing tendency identified for the individual customer.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may be implemented within a computer system, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA) A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute one or more software applications that are executed on an operating system (OS) and an operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer readable medium may include program instructions, data files, data structures, and the like, alone or in combination. Program instructions to be recorded on the medium may be those specially designed and constructed for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described with reference to the drawings, various technical modifications and variations may be applied to those skilled in the art. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

1600: Customer item recommendation device
1610:
1620: Processor
1630: Memory

Claims (20)

In a customer item recommendation method,
Obtaining a customer image corresponding to a customer;
Determining a recommended item for the customer based on the obtained customer image; And
Providing association information associated with the determined recommendation item
And a customer item recommendation method. The method according to claim 1,
Wherein the step of determining the recommendation item comprises:
Identifying a customer's purchase tendency based on the acquired customer image;
Determining the recommendation item for the customer based on the identified purchase propensity
And a customer item recommendation method. The method according to claim 1,
Wherein the acquiring of the customer image comprises:
Using at least one camera, acquiring at least one customer image
Lt; / RTI >
Wherein the field of view of the at least one camera covers at least a portion of the purchase area in a physical store,
Customer item recommendation method. The method according to claim 1,
Wherein the acquiring of the customer image comprises:
Obtaining a customer image for at least one of a plurality of customers displayed at a user terminal
And a customer item recommendation method. The method according to claim 1,
Wherein the step of determining the recommendation item comprises:
Extracting an image feature for the customer from the customer image; And
Determining, based on the image feature, a recommendation item for the customer
And a customer item recommendation method. 6. The method of claim 5,
Wherein the step of determining the recommendation item comprises:
Retrieving a purchase propensity model corresponding to the customer from a purchase propensity model database based on the image characteristics; And
Determining the recommendation item for the customer based on the purchase propensity model corresponding to the customer
And a customer item recommendation method. The method according to claim 6,
Wherein the purchase propensity model database comprises:
Multiple purchase propensity models
/ RTI >
Wherein each of the plurality of purchase propensity models comprises:
The feature vector of the item corresponding to the fashion category, the feature vector of the customer image, and the feature vector of the item corresponding to the non-fashion category
/ RTI >
The step of searching for the purchase propensity model comprises:
A correlation degree between the customer and each purchase propensity model according to the feature vector of the item corresponding to the fashion category, the feature vector of the customer image, and the feature vector of the item corresponding to the non-fashion category, based on the image characteristic for the customer calculating a correlation level; And
Determining a purchasing propensity model having the highest correlation among a plurality of purchase propensity models as the purchase propensity model of the customer
And a customer item recommendation method. The method according to claim 6,
Constructing the purchase propensity model database by constructing the purchase propensity model through the online purchase database,
And a customer item recommendation method. 9. The method of claim 8,
The step of constructing the purchase propensity model database comprises:
Learning statistics on a customer's online purchase history;
Determining an eigenvalue of a customer from a result of disassembling the purchasing history matrix indicating the item purchase history of the customer;
Classifying customers into a plurality of customer clusters based on the customer ' s unique value; And
Determining eigenvalues of the category based on the category-specific purchase data for each of the plurality of customer clusters
And a customer item recommendation method. The method according to claim 6,
Establishing the purchase propensity model database by constructing the offline purchase propensity model through the offline shop purchase database,
And a customer item recommendation method. 11. The method of claim 10,
The step of constructing the purchase propensity model database comprises:
Extracting a customer ID, a customer image, and a purchase record of a customer from the offline store purchase database;
Acquiring the customer's appearance information and the customer's purchase behavior information by analyzing the customer image;
Classifying customers into a plurality of customer clusters based on information obtained from the customer image; And
Calculating feature vectors of the category based on the purchase data for each category for each of the plurality of customer clusters
And a customer item recommendation method. The method according to claim 6,
Wherein the step of determining the recommendation item comprises:
Matching an item image and a customer image of the item corresponding to each category in the purchase propensity model; And
Determining a recommendation item based on the fitting score calculated between the item image and the customer image
And a customer item recommendation method. 13. The method of claim 12,
Wherein the step of determining the recommended item based on the calculated fitting score comprises:
Calculating an inner product between the feature vector of the individual item and the feature vector of the item worn by the customer in the purchase propensity model as the fitting score
And a customer item recommendation method. The method according to claim 1,
Wherein providing the association information associated with the recommendation item comprises:
Transmitting a push request to a customer terminal including an augmented reality display; And
Receiving information indicating whether to accept the push request from the customer terminal
And a customer item recommendation method. The method according to claim 1,
Wherein providing the association information associated with the recommendation item comprises:
Transmitting information associated with the recommendation item to the customer terminal in response to the push request being accepted by the customer terminal
How to recommend customer items, including The method according to claim 1,
Wherein providing the association information associated with the recommendation item comprises:
Transmitting item information on a recommendation item from a customer's location to a display disposed within a critical distance
How to recommend customer items, including The method according to claim 1,
Wherein providing the association information associated with the recommendation item comprises:
Transmitting associated information associated with the recommendation item to at least one of a business terminal and a customer terminal
And a customer item recommendation method. The method according to claim 1,
Wherein the step of determining the recommendation item comprises:
Determining the recommendation item based on the purchase area where the customer is located and the customer image
And a customer item recommendation method. 18. A computer-readable medium having stored thereon one or more computer programs comprising instructions for performing the method of any one of claims 1 to 18. In a customer item recommendation apparatus,
An image obtaining unit for obtaining a customer image; And
A processor for determining a recommendation item for the customer based on the acquired customer image and providing association information associated with the determined recommendation item,
And a customer item recommendation device.

Images