CN106557489A - A kind of dress ornament searching method based on mobile terminal - Google Patents

Abstract

The present invention provides a mobile terminal-based clothing search method. The user takes a photo through the mobile terminal or selects a clothing picture from the gallery of the mobile terminal and uploads it to the server for clothing search. Obtain clothing product pictures and product information, store them in the clothing database, perform image feature extraction to form an image feature pool; respond to the clothing pictures taken or selected by the user through the mobile terminal, cut the clothing pictures, and perform matching queries on the server; The searched clothing results are displayed to the user through the mobile terminal with similar clothing pictures and corresponding clothing information. The clothing information includes the brand, price and material of the clothing. The method of the present invention combines the characteristics of the visualization of clothing products to provide users with a search solution for clothing products in the manner of "searching images with pictures", thereby improving the user's experience in searching for clothing products.

Description

A clothing search method based on mobile terminal

technical field

The present invention relates to the technical field of mobile Internet, in particular to the field of image search based on mobile Internet, in particular to a mobile terminal-based clothing search method.

Background technique

With the rapid development of the mobile Internet and the popularity of mobile terminals, e-commerce transactions are changing from traditional PC terminals to mobile terminals. Mobile terminals have the advantages of small size and easy portability, and more and more users prefer mobile terminals to search and purchase apparel products. Most of the traditional clothing search methods search for clothing products through plain text keywords. This method is too simple and ignores the important image information in e-commerce information. Moreover, users cannot accurately describe the keywords of products. Clothing products contain rich content, and it is often difficult to fully describe them with a small amount of text. Users with good expressive ability describe the characteristics of clothing, and it is difficult to search for clothing products similar to the clothing, and the search results are difficult to satisfy users.

Based on the above problems, it is necessary for those skilled in the art to propose a new clothing search method.

Contents of the invention

The purpose of the present invention is to provide a search method for clothing products based on a mobile terminal, combining the characteristics of visualization of clothing products, to provide users with a way of "searching for pictures with pictures" to search for clothing products, and to improve the user's experience in searching for clothing products.

In order to achieve the above object, the present invention proposes a mobile terminal-based clothing search method. The client takes a photo or selects a clothing picture from the client gallery and uploads it to the server for clothing search, including the following steps:

(1) Establishment of clothing image feature pool

Obtain clothing product pictures and product information from e-commerce websites through web crawlers, store them in the clothing database, and perform image feature extraction on the clothing pictures in the clothing database to form an image feature pool. The clothing database is stored in the server

(2) Search for similar products

In response to the clothing picture taken or selected by the user through the mobile terminal, after cutting the clothing picture, upload it to the aforementioned server, perform feature extraction on the clothing picture uploaded by the user on the server, and then match it with the image features stored in the image feature pool Inquire

(3) Display of search results

According to the inquired clothing results, the queried similar clothing pictures and corresponding clothing information are displayed to the user through the mobile terminal, and the clothing information includes the brand, price and material of the clothing.

Compared with the prior art, the clothing search method proposed by the present invention adopts the image search method for clothing search, which changes the current method of plain text keyword search, realizes "searching pictures by pictures", and can be used anytime, anywhere and is convenient to carry The mobile terminal takes photos of clothing, which has a more direct and accurate description of the actual clothing. Using image-based clothing search effectively solves the difficulty that users cannot or does not want to use text to search, and reduces the difficulty of shopping. The visualization feature meets the higher demand of users for search functions, and facilitates users to quickly find clothing information, which greatly improves Improved user experience.

It should be understood that all combinations of the foregoing concepts, as well as additional concepts described in more detail below, may be considered part of the inventive subject matter of the present disclosure, provided such concepts are not mutually inconsistent. Additionally, all combinations of claimed subject matter are considered a part of the inventive subject matter of this disclosure.

The foregoing and other aspects, embodiments and features of the present teachings can be more fully understood from the following description when taken in conjunction with the accompanying drawings. Other additional aspects of the invention, such as the features and/or advantages of the exemplary embodiments, will be apparent from the description below, or learned by practice of specific embodiments in accordance with the teachings of the invention.

Description of drawings

The figures are not intended to be drawn to scale. In the drawings, each identical or nearly identical component that is illustrated in various figures may be represented by a like reference numeral. For purposes of clarity, not every component may be labeled in every drawing. Embodiments of the various aspects of the invention will now be described by way of example with reference to the accompanying drawings, in which:

Fig. 1 is a schematic diagram of a specific embodiment of the mobile terminal-based clothing search method of the present invention.

Fig. 2 is a specific embodiment of the clothing picture cropping method of the present invention.

Fig. 3 is a schematic diagram of cutting clothing pictures by the smearing method shown in Fig. 2 .

Fig. 4 is a flow chart of a specific embodiment of the smearing method of the present invention.

Fig. 5 is a flow chart of a specific embodiment of query matching in the present invention.

Fig. 6 is a structural diagram of a specific embodiment of the clothing search device based on the mobile terminal of the present invention.

detailed description

In order to better understand the technical content of the present invention, specific embodiments are given together with the attached drawings for description as follows.

Aspects of the invention are described in this disclosure with reference to the accompanying drawings, which show a number of illustrated embodiments. Embodiments of the present disclosure are not necessarily intended to include all aspects of the invention. It should be appreciated that the various concepts and embodiments described above, as well as those described in more detail below, can be implemented in any of numerous ways, since the concepts and embodiments disclosed herein are not limited to any implementation. In addition, some aspects of the present disclosure may be used alone or in any suitable combination with other aspects of the present disclosure.

As shown in FIG. 1, according to an embodiment of the present invention, a mobile terminal-based clothing product search method, the client takes a photo or selects a clothing picture from the client gallery and uploads it to the server for clothing search, including the following steps :

(1) Establishment of clothing image feature pool

Obtain clothing product pictures and product information from major e-commerce websites through web crawler technology, store them in the clothing database, and extract image features from the clothing pictures in the clothing database to form an image feature pool. The clothing database exists in the server.

(2) Search for similar products

The user uploads the required clothing picture to the server through the mobile terminal. When uploading the clothing picture, the clothing picture needs to be cropped, and the feature extraction of the clothing picture uploaded by the user is performed on the server, and then the matching query is performed with the image features stored in the image feature pool.

Uploading clothing pictures through the mobile terminal includes: taking a photo and uploading or selecting a clothing photo from the mobile terminal to upload.

Clothing image clipping methods include a rectangular frame method and a smear method for image clipping.

The mobile terminal includes a mobile phone and a Pad.

(3) Display of search results

According to the inquired clothing results, the queried similar clothing pictures and corresponding clothing information are displayed to the user through the mobile terminal, and the clothing information includes the brand, price and material of the clothing.

As shown in FIG. 1, in the aforementioned clothing search method of the present invention, a photo of clothing is uploaded through the mobile terminal or a photo of clothing is selected from the mobile terminal and uploaded to the server, and then feature extraction is performed on the server, and according to the extracted image features in the image feature The matching query is performed in the pool, and the similar clothing pictures found in the query are returned to the user.

The image feature pool is to obtain clothing pictures and corresponding clothing information (such as material, price, brand, etc.) from major e-commerce websites (such as TAOBAO, Jingdong Mall, Tmall Mall, etc.) Then perform feature extraction on clothing pictures to form an image feature pool. When a user searches for clothing pictures, the image feature pool used is the image feature pool established above.

Under normal circumstances, clothing is displayed by models, and users may not like all parts of a piece of clothing. When users upload clothing pictures, they need to crop the pictures to extract the parts that users want most and upload them. .

Fig. 2 is a specific implementation of a method for cropping clothing pictures. In this example, a method of drawing a rectangular frame is adopted, and the part that the user is interested in is selected for uploading. The method of drawing a rectangular frame is that the user selects a part of the favorite clothing through the mobile terminal, and then performs a clothing search on the selected part of the frame, and returns the search result to the user.

Fig. 3 is a schematic diagram of cutting clothing pictures by smearing method. With reference to the illustrated example, this method is a kind of smear-style cropping. The user uses the mobile terminal to smear the clothing area of interest with his fingers, uploads it, and returns the search results to the user. Of course, if the result of the first search is unsatisfactory, the user can cut it again and perform a second search until a satisfactory clothing is found.

Users need to crop clothing pictures to search for the most satisfactory clothing. Although the method of drawing a rectangular frame can achieve the effect of cropping, the method of drawing a rectangular frame may frame irrelevant areas, such as selecting the background of a clothing picture to In the search area, the results of this search are not the most satisfactory clothing for users. Another example is that the user may go through the frame several times to select a relatively satisfactory area, which wastes the user's time and patience. The smear method provides a more precise and convenient cutting method, users only need to use their fingers to smear the favorite clothing part to search. For example, as shown in Figure 3, the clothing picture on the left side of Figure 3 is the original picture, and the right side is the clothing picture after painting. Smearing can be done in three steps:

Apply the collar first, the body second, and the arms last. As shown in the right picture of Figure 3, green represents the collar part, light blue represents the body part, and yellow represents the arm part. Then upload the smeared part to the server for feature extraction, and perform matching query in the image feature pool according to the extracted image features, and return the queried similar clothing pictures to the user.

The smearing method divides a piece of clothing into three areas for smearing and cutting, which can be the collar part, body part and arm part in turn. The example in Figure 4 shows a specific implementation process of clothing search given the smearing method Figure, as shown in Figure 4, the specific steps include:

In step 401, local descriptors are extracted for the three regions painted on the clothing picture, and further, in order to process the local information of clothing features more accurately, the feature points are processed using bone coordinates. The bone coordinates are converted from standard coordinates.

Step 402, mix and encode the local descriptors of each region. The local descriptors of the present invention are represented by adjacent visual words. The respective positions of the local descriptors of the three parts of the smeared clothing picture are x1,...xa,xa+ 1,...,xa+b,xa+b+1,...,xa+b+c, convert it to bone coordinates: x' ₁ ,..., x' _a , x' _a+1 , ..., x' _a+b , x' _a+b+1 , ..., x' _a+b+c , where a, b, and c are the three local descriptors of the clothing pictures Quantity, feature extraction is performed using bone coordinates. The visual words are obtained by clustering the local descriptors in the image feature pool using the K-means algorithm.

Step 403 , use the mixed descriptors to search for clothing, and calculate the similarity between the local descriptors of the searched clothing picture and the local descriptors of the image feature pool. The feature vector of the clothing is matched with the feature vector of each image in the image feature pool to query.

Step 404, return the clothing picture and corresponding clothing information obtained by matching the query to the user.

Figure 5 is a specific implementation of query matching, and the specific steps include:

Step 501, upload a clothing picture from the mobile terminal to the clothing device, the clothing picture is a cropped picture, and the cropping method uses smearing to extract local descriptors from the uploaded clothing picture.

Step 502, calculate the similarity between the uploaded clothing picture and each picture in the clothing database, the similarity is calculated by a similarity measurement method with space constraints. The method explores locations using a grid voting space, which is a voting map, which is based on the location relationship of local descriptors, has the same visual word and can calculate the similarity at the same time. Specific steps include:

1) Vote and superimpose the combination of feature points with the same visual word in the two clothing pictures for similarity calculation. The voting point is the position point where the two feature points have the same value as the motion vector.

2) The region with the highest voting point represents the projection, and the combination of voting points in general represents the similarity of two clothing pictures.

Step 503: Perform similarity calculation between the uploaded clothing picture and each clothing picture in the clothing database, and return the clothing picture with higher similarity (for example, meeting the set threshold standard) to the user.

Corresponding to the specific implementation of the aforementioned method, the present invention also discloses a mobile terminal-based apparel search device, as shown in Figure 6, which may specifically include:

Clothing picture acquisition module 601, this module acquires a piece of clothing picture from the mobile terminal, and the specific method can be to take a photo or obtain a piece of clothing picture from the mobile terminal and upload it to the server. In addition, the user can cut clothing pictures multiple times and select the part that the user likes to upload until the clothing that the user is satisfied with is found. The clipping method includes a method of drawing a rectangular frame and a method of smearing.

The local descriptor extraction module 602 is used to extract the corresponding local descriptors according to the received clothing pictures, and also extract the local descriptors on the server side to calculate the similarity with the received clothing pictures.

The clothing picture retrieval module 603 is used to perform matching query of local descriptors in the server-side image feature pool according to the received local descriptors of clothing pictures, and calculate the similarity between two clothing pictures. The image feature pool stores local descriptor information of clothing pictures obtained from e-commerce websites.

Clothing picture returning module 604 returns the retrieved clothing picture to the user and displays it on the user's mobile terminal.

Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Those skilled in the art of the present invention can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be defined by the claims.

Claims (4)

1. a kind of dress ornament searching method based on mobile terminal, user are taken pictures by mobile terminal or from movement A dress ornament picture is chosen in the picture library of terminal, the end that uploads onto the server carries out dress ornament search, it is characterised in that The method is comprised the following steps：

Step 1, the foundation in dress ornament picture feature pond

Dress ornament commodity picture and merchandise news are obtained from electric business website by web crawlers, dress ornament data are stored in Storehouse, carries out image characteristics extraction and forms characteristics of image pond, the dress ornament to the dress ornament picture in dress ornament data base In data base's presence server；

The search of step 2, similar commodity

The dress ornament picture for being shot by mobile terminal in response to user or being selected, carries out cutting to dress ornament picture Afterwards, aforementioned server is uploaded to, feature extraction is carried out to the dress ornament picture that user uploads on the server, so Matching inquiry is carried out with the characteristics of image that stores in characteristics of image pond afterwards；

Step 3, Search Results show

According to the dress ornament result for inquiring, the similar dress ornament picture for inquiring and corresponding dress ornament are believed Breath shows user, the dress ornament information to include brand, price and the material of dress ornament by mobile terminal.

2. the dress ornament searching method based on mobile terminal according to claim 1, it is characterised in that front State in step 2, dress ornament picture method of cutting out includes at least one in the method for rectangle frame and smearing method, rectangle Frame method is by electing dress ornament frame segment user interested, the part that frame is selected being uploaded to clothes then Business device carries out dress ornament search, and smearing method is smeared on dress ornament picture by user, will be interested Dress ornament region smears to choose to upload onto the server carries out dress ornament search.

3. the dress ornament searching method based on mobile terminal according to claim 1, it is characterised in that In the step of preceding method 2, based in the dress ornament search procedure for smearing method of cutting out, by following sides Formula carries out feature extraction, hybrid coding and matching inquiry：

1) three regions smeared to dress ornament picture carry out the extraction of local description respectively, three areas Domain is collar portion, body part and the arm segment of clothes respectively, the local description of dress ornament During extraction process, using skeleton coordinate processing feature point, skeleton coordinate is changed by standard coordinate Obtain；

2）The local description in each region is mixed and encoded, each local description uses adjacent vision Word represents that the respective position of three position local descriptions of the dress ornament picture smeared is： X1 ... xa, xa+1 ..., xa+b, xa+b+1 ..., xa+b+c is converted into skeleton coordinate： x′₁..., x '_a, x '_a+1..., x '_a+b, x '_a+b+1..., x '_a+b+c, wherein a, b, c be respectively smear three of dress ornament picture The quantity of part local description, feature extraction are carried out using skeleton coordinate, and the visual word is used K-means algorithms carry out cluster to the local description in characteristics of image pond and obtain；

3）Dress ornament search is carried out using mixed description, the local description of searched dress ornament picture is calculated With the similarity of the local description in characteristics of image pond, by the characteristic vector of dress ornament and characteristics of image pond per width The characteristic vector of image carries out matching inquiry.

4. the dress ornament searching method based on mobile terminal according to claim 1, it is characterised in that front In the step of stating method 2, the calculating of the similarity passes through a kind of method for measuring similarity meter of space constraint Calculate, the method explores position using grid voting space, the network voting space is a ballot map, It is the position relationship accorded with based on partial descriptions, can calculate with identical visual word and simultaneously similarity, Concrete steps include：

1）The characteristic point combination that the two width dress ornament pictures to carrying out Similarity Measure have identical visual word is thrown Ticket is superimposed, and polling place is two characteristic points and motion vector value identical location point；

2）Regional Representative with highest polling place projection, the combination of generally polling place represent two width dress ornament figures The similarity of piece；

Also, the dress ornament picture to uploading is entered in the manner described above with every dress ornament picture in dress ornament data base The dress ornament picture that similarity meets given threshold is returned to user by row Similarity Measure in step 3.