HK1242023B - Method and apparatus for determining order information - Google Patents

Description

Method and device for determining order information

Technical Field

The present disclosure relates to the field of Internet technology, and in particular to a method and device for determining order information.

Background Art

After selecting items at a traditional brick-and-mortar store, consumers need to queue up at the checkout counter to pay, where a checkout agent assists them in settling their bill using a computer. However, this checkout process is often inefficient and can result in long wait times. To improve checkout efficiency, existing technologies can analyze consumers' shopping process in real time using technologies such as computer vision, determining which items they have taken and which have been returned, and thereby modifying and confirming the order information, i.e., the consumer's bill list. However, many factors can lead to inaccurate judgments in this analysis process. For example, a consumer may mistakenly place a highly similar item in the wrong location, or the recognition device may be obscured when multiple people simultaneously take items from a lower shelf, resulting in misjudgment. These factors can lead to inaccurate order information and affect product settlement.

Summary of the Invention

In view of this, the present disclosure provides an order information determination method and apparatus to determine the order information of goods more quickly and accurately, so that the goods on the order are associated with consumers.

Specifically, the present disclosure is achieved through the following technical solutions:

In a first aspect, a method for determining order information is provided, wherein the method is used to determine an association between a user and the items they purchased, the method comprising:

Performing human body motion posture recognition on the user to obtain posture data of the user;

Locating the product to obtain location information of the product;

Determining, based on the posture data and location information, whether the user's posture is to pick up the product;

If the judgment result is that the user takes the product, the product is added to the user's order.

In a second aspect, a device for determining order information is provided, the device comprising:

A posture recognition module is used to recognize the user's human body posture and obtain the user's posture data;

A product positioning module is used to locate the product and obtain the location information of the product;

An information processing module, configured to determine, based on the posture data and location information, whether the user's posture is to pick up the product;

The order processing module is used to add the product to the user's order if the judgment result is that the user has taken the product.

The order information determination method and device disclosed in the present invention can quickly and accurately determine the consumer's order information and automatically complete the settlement and payment of the order. After completing shopping, the consumer can leave directly without having to queue up at the checkout counter, which improves shopping efficiency and provides a good shopping experience.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a regional layout of a convenience store provided by an embodiment of the present disclosure;

FIG2 is a more detailed schematic diagram of the layout of a convenience store provided by an embodiment of the present disclosure;

FIG3 is a schematic diagram of the structure of a computing device provided by an embodiment of the present disclosure;

FIG4 is a schematic diagram of a process for determining order information provided by an embodiment of the present disclosure;

FIG5 is a schematic structural diagram of an order information determination device provided by an embodiment of the present disclosure.

DETAILED DESCRIPTION

Shopping at supermarkets, shopping malls, convenience stores, and other places is a common practice in people's daily lives. After purchasing items, consumers must queue up at the checkout counter, which is very time-consuming and inconvenient. To improve shopping efficiency, a shopping management system can automatically identify the items purchased by a consumer, determine the relationship between the consumer and the items purchased, and automatically push the corresponding bill to the consumer, allowing them to pay by themselves. This eliminates the need for consumers to queue at the checkout counter, significantly improving shopping efficiency.

A key factor in implementing the above method is determining the relationship between consumers and the products they purchased. Only after this relationship is determined can the corresponding consumer's bill be obtained. This disclosure may refer to this relationship as "order information," specifically, the products selected by a consumer. The order information determination method provided in this disclosure is designed to simply and accurately determine order information to facilitate rapid settlement.

The following will take consumers shopping in convenience stores as an example to illustrate the processing process of the order information determination method disclosed in the present invention. However, this method can also be applied to other scenarios, such as supermarket shopping, shopping in shopping malls, warehouse inventory monitoring (who has taken which inventory in the warehouse), library book management (which books a user has borrowed), and other similar scenarios that require determining the relationship between people and items.

Figure 1 illustrates the layout of a convenience store. As shown in Figure 1 , the convenience store 100 may include an entrance area 101, a storage area 102, and an exit area 103. Consumers can enter the convenience store 100 through the entrance area 101, purchase items in the storage area 102, and exit the convenience store through the exit area 103 after completing their purchases. The storage area 102 may store a wide variety of items. For example, items 1021 to 1024 in Figure 1 illustrate items in the convenience store, including fruit, beverages, milk, bread, and the like. Figure 1 only illustrates a portion of the items; the actual storage area 102 may contain many more items. The convenience store layout shown in Figure 1 is a functional division, not an actual physical division. For example, in one example, multiple entrance areas 101, storage areas 102, and exit areas 103 may be integrated rather than arranged separately.

The convenience store 100 in Figure 1 may also include a shopping management system 104, which can communicate and interact with the aforementioned entrance area 101, storage area 102, and exit area 103. Please refer to Figure 2 for a more detailed illustration of the convenience store layout. For example, facial recognition devices 201 may be placed at the entrance area 101 within the convenience store 100. When a customer 202 enters the store, the facial recognition devices 201 may automatically collect facial recognition data of the customer 202. For example, the facial recognition devices 201 may capture an image of the customer at the entrance area, search it using a specific strategy to determine if it contains a face, and if so, return the location, size, and pose of the face. The facial recognition devices 201 may also perform pre-processing and feature extraction on the facial image to generate facial recognition data. Within the storage area 102, shelves 203 may be placed, which may hold a variety of products, such as bananas 2031 and milk 2031 in the example shown in Figure 2. When shopping, consumers can hold the products in their hands or with their arms, that is, they can use a part of their body to carry the products. After shopping, consumers can go to the exit area of the convenience store and walk out of the convenience store through the exit channel 204 without queuing to check out. Generally, one exit channel 204 can only allow one person to pass through in sequence.

In one example, the present disclosure can attach an RFID (Radio Frequency Identification) tag to each product, such as the RFID tag 205 attached to the product on shelf 203 in Figure 2. The information contained in the tags on different products is also different, including the identification information of the product.

As shown in FIG2 , a convenience store may also have multiple readers installed on walls, roofs, and other locations for receiving RFID tag signals, such as reader 206 installed in the entrance area, reader 207 installed in the storage area, and reader 208 installed in the exit area. The readers can transmit the received RFID tag information to the shopping management system 104, which can store and process the information. Furthermore, the convenience store may also be equipped with monitoring devices such as cameras 209. These monitoring devices can be used to conduct video surveillance of the store, and the monitoring information can also be transmitted to the shopping management system 104. The shopping management system 104 can also transmit the information within the system to other devices for display through network devices installed in the store, such as wireless antennas. For example, it can be transmitted to the smartphones carried by consumers, allowing consumers to conveniently view the information obtained by the shopping management system on their phones.

The shopping management system 104 can be a local or remote server system, which can include many computing and processing devices. For example, Figure 2 illustrates two computing devices, but in actual implementation, there can be more. As shown in Figure 3, a computing device can include a processor 301, an input/output interface 302, a network interface 303, and a memory 304. The above-mentioned processor 301, input/output interface 302, network interface 303, and memory 304 can be connected and communicate with each other via a bus 305. Figure 3 only illustrates some components, and an actual computing device can include more or fewer components. The memory 304 can also include a data management module 3041 and a shopping management module 3042. These modules can be in the form of hardware or software. When they are in the form of software, they can be computer executable programs.

For example, the computing device can receive information transmitted by RFID tags, cameras, and other devices in the convenience store through the network interface 303 and process the information (detailed in subsequent examples), such as the location information of the product through the RFID tag, or the membership ID information transmitted by the consumer. The processor 301 can process the received information by executing the instructions of the shopping management module 3042 to obtain some of the latest data, such as information about consumers who have just entered the convenience store, product location information, user posture data, or consumer order information; and the processor 301 can update this data to the database 305 by executing the instructions of the data management module 3041.

In one example, the database 305 can store data, for example, user information, location information, order information, and posture data. Among them, user information can be some member IDs of registered users of the shopping management system, location information can be the information obtained by positioning the goods as described in the subsequent examples, and order information can be which goods the consumer has purchased, and posture data can be the data obtained by performing dynamic human posture recognition on the user as described in the subsequent examples. In addition, the computing device can update this information based on the latest received data, for example, when there is a new registered user, user information can be updated, the location information of the goods can be updated in the real-time positioning of the goods, and order information can be updated according to changes in the goods in the order. In addition, the computing device can also output the data in the database, for example, the order information can be taken out from the database and sent to other devices such as the consumer's mobile phone, so that the consumer can view this information.

Figure 4 illustrates the process of determining order information. When a consumer enters a convenience store to purchase items, they enter through the entrance area, select items from the shelves, and use their hands or arms to pick up the items. After completing their purchases, they leave the store directly through the exit area. During this process, the shopping management system can execute the process in Figure 4 to complete the order information, that is, to determine which items the consumer has purchased from the convenience store.

In step 401, when a consumer enters a convenience store to purchase merchandise, they can enter their membership ID at the entrance. For example, the user can generate a QR code identifying their identity using a shopping app installed on their smartphone. This shopping app can be a client of the shopping management system, and the user has logged in their membership ID registered on the client on their smartphone. This membership ID can be referred to as the user ID. Alternatively, the user can enter their membership ID using NFC via a smart device, such as a wristband, which is already associated with the membership ID. The client can then upload the obtained membership ID to the shopping management system.

In addition, in the entrance area, the facial recognition device 201 can also perform facial recognition on the user and transmit the obtained facial recognition data to the shopping management system, so that the shopping management system can receive the binding information between the member ID as the user identifier and the facial recognition data, which is equivalent to knowing what the member ID corresponding to the consumer with a certain facial feature is, and the corresponding relationship can be stored in the database 305.

Users enter the store area to purchase items. RFID readers 208 and other devices installed throughout the convenience store can locate the RFID tags attached to the items. For example, reader 208 can receive tag signals transmitted by RFID tags attached to items. This tag signal can include product information, such as a product code that uniquely identifies the item. In step 402, reader 208 can transmit the tag signal to a shopping management system, which can perform positioning calculations based on the tag signal to obtain the location information of the location tag attached to the item. For example, referring to Figure 3, processor 301 on the computing device can execute executable code in shopping management module 3042 to perform positioning calculations based on the tag signal, and execute executable code in data management module 3041 to store the calculated location information in database 305. This step can be performed using conventional RFID positioning technology. Furthermore, facial recognition devices can be installed in the shopping area of the convenience store. These devices can not only detect facial features but also locate people, obtaining location information of consumers within the store.

In step 403, the shopping management system may determine, based on the location information, that the merchandise and the user are located in the exit area of the convenience store.

In step 404, the shopping management system can use the facial recognition device 210 installed at the exit area to identify the user's body posture and obtain the user's posture data. Furthermore, the reader 208 at the exit area can locate the product tag and obtain the product's location information. Human body posture can be identified by analyzing the input video, such as whether a person is placing their hands on their chest, holding their hands down, or folding their arms tightly. The user's posture data corresponds to the positional relationship between various body parts.

In this example, the posture data and position information can be used to determine whether the user's posture is to pick up the product. For example, the posture data can be used to determine whether the posture of the user's body part corresponding to the position information of the positioning tag is capable of picking up the product. Suppose the user's body part corresponding to the position of the positioning tag is the user's chest, but the user's posture data shows that there is nothing on the chest, and the user's hands are hanging down, then the product is not being held by the user. For the opposite example, suppose the user's body part corresponding to the position of the positioning tag is on the user's side thigh, and the posture data determines that the user's hands are hanging down at this time, and the user's hands happen to be in this area, then it can be determined that the user is holding the product with his hands.

If the user is determined to have taken the product, the product information can be added to the user's order in step 405. Furthermore, at the exit, the facial recognition device will again capture the facial recognition data and obtain the member ID corresponding to that data, adding the product to the order corresponding to that member ID. If facial recognition fails to obtain the corresponding member ID, other methods can be attempted, such as using a QR code or NFC. Product labels not from this convenience store can be excluded during product detection.

The disclosed method for determining order information is based on identifying user posture data and locating items. The relationship between the posture and the position of the items is then used to determine the order ownership of the items. This method provides more accurate order determination. For example, even if multiple people simultaneously pick up items from a shelf, the positional relationship between the items and the user's posture is still determined, and only the items that match the user's posture are assigned to the order. Furthermore, even if a user places a highly similar item in the wrong location, this will not affect the final determination of the item and user's posture. This method is unaffected by many misleading factors and is more accurate in determining the order ownership of items.

The above-mentioned process of determining the ownership of a product order based on the distance between the product and the shopping cart can be performed at any time after the user enters the convenience store. Alternatively, after obtaining the member ID and bound facial recognition data in the entrance area, no recognition is performed during the product selection process in the storage area, and only the user's posture and the positional relationship of the product are recognized at the exit area to determine the ownership of the product order.

In addition, when the shopping management system adds the product to the user's order, the order information data in the database 305 can be updated, that is, the products selected in the user's order are updated. In addition, the shopping management system can also send the order information to the user's smartphone. As shown in Figure 3, the client installed on the user's own smartphone can have a shopping cart information interface, which can display a list of product order information. The user can see the following information: "You purchased the following products: bananas, apples", so that the user can understand the changes in the order at any time. In addition, the quantity of the products purchased by the user can also be displayed, or other more product information can be displayed, such as the origin of the product. The present disclosure does not exclude the use of other auxiliary discrimination technologies to assist in determining order information; for example, multiple sensors can be used to assist.

After confirming the order information, the shopping management system can generate a bill to be paid based on the order and push it to the user. The user can then pay from the account associated with their membership ID. The user's account must have sufficient funds or be connected to other payment channels that can cover the order. Otherwise, the user may be restricted from leaving with items that exceed their payment capacity or trigger an alert device to warn them.

In the examples disclosed herein, an RFID tag is attached to a product, and the product is located by the RFID tag. However, the specific implementation is not limited to this, and positioning can also be performed by other positioning methods. Other positioning tags can be attached to the product, and corresponding other positioning technologies can be used for positioning.

The method disclosed herein can quickly and accurately determine the consumer's order information and automatically complete the settlement and payment of the order. After completing shopping, the consumer can leave directly without having to queue up at the checkout counter, thereby improving shopping efficiency and providing a good shopping experience.

The present disclosure also provides an order information determination device, which may be the shopping management module in Figure 3 or the shopping management system in Figure 1. As shown in Figure 5, the device may include: a gesture recognition module 51, a product positioning module 52, an information processing module 53, and an order processing module 54.

A gesture recognition module 51 is used to recognize the user's body gestures and obtain the user's gesture data;

The product positioning module 52 is used to locate the product and obtain the location information of the product;

An information processing module 53 is used to determine whether the user's posture is to pick up the product based on the posture data and position information;

The order processing module 54 is configured to add the product to the user's order if the judgment result is that the user has taken the product.

In one example, the information processing module 53 is specifically used to locate the positioning tag on the product based on the tag signal sent by the positioning tag to obtain the position information of the positioning tag, and the tag signal includes the product information of the product; based on the user's posture data, determine whether the posture of the user part corresponding to the position information of the positioning tag can pick up the product.

In one example, the positioning tag is a radio frequency identification (RFID) tag.

In one example, the information processing module 53 is specifically configured to determine whether the user is in a posture of picking up the product when the product and the user are located in an exit area based on the location information of the product and the user.

In one example, the order processing module 54 is also used to: obtain facial recognition data of a user who selects and purchases goods in the entrance area, wherein the facial recognition data corresponds to the user ID of the user; when it is determined that the goods and the user are located in the exit area, obtain facial recognition data again to obtain the user ID corresponding to the facial recognition data, and the order is an order for the user corresponding to the user ID.

The devices or modules described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

For the convenience of description, the above devices are described as being functionally divided into various modules. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and/or hardware.

The technical carriers involved in the payment described in the embodiments of the present application may include, for example, Near Field Communication (NFC), WIFI, 3G/4G/5G, POS card swiping technology, QR code scanning technology, barcode scanning technology, Bluetooth, infrared, Short Message Service (SMS), Multimedia Message Service (MMS), etc.

The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure should be included in the scope of protection of the present disclosure.

Claims (8)

1. A method for determining order information, characterized in that the method is used to determine the association between a user and the goods they have selected, the method comprising: Human body movement and posture recognition is performed on the user to obtain the user's posture data; The product is located to obtain its location information; Based on the posture data and location information, it is determined whether the user's posture indicates that they are picking up the product; If the determination result is that the user takes the product, then the product is added to the user's order; The step of determining whether the user's posture indicates they are picking up the product based on the posture data and location information specifically involves: Based on the location information of the goods and the user, when it is determined that the goods and the user are located in the exit area, it is determined whether the user's posture indicates that the user is picking up the goods. 2. The method according to claim 1, wherein determining whether the user's posture has taken the product based on the posture data and location information includes: Based on the tag signal sent by the positioning tag on the product, the positioning tag is located to obtain the location information of the positioning tag, and the tag signal includes the product information of the product; Based on the user's posture data, determine the posture of the user's body part corresponding to the location information of the positioning tag, and whether the user can pick up the product. 3. The method according to claim 2, wherein the positioning tag is a radio frequency identification (RFID) tag. 4. The method according to claim 1, characterized in that the method further comprises: In the entrance area, facial recognition data of users selecting goods is obtained, and the facial recognition data corresponds to the user's user identifier; When it is determined that the goods and the user are located in the exit area, facial recognition data is acquired again to obtain the user identifier corresponding to the facial recognition data. The order is the order of the user corresponding to the user identifier. 5. An order information confirmation device, characterized in that the device comprises: The posture recognition module is used to recognize the user's human body posture and obtain the user's posture data. The product positioning module is used to locate the product and obtain its location information; The information processing module is used to determine whether the user's posture is to pick up the product based on the posture data and location information; specifically, the information processing module is used to determine whether the user's posture is to pick up the product when the product and the user are located in the exit area based on the location information of the product and the user. The order processing module is used to add the product to the user's order if the determination result is that the user has taken the product. 6. The apparatus according to claim 5, characterized in that, The information processing module is specifically used to locate the positioning tag based on the tag signal sent by the positioning tag on the product, and obtain the location information of the positioning tag, wherein the tag signal includes product information of the product; and to determine the posture of the user's body part corresponding to the location information of the positioning tag, and whether the user can pick up the product, based on the user's posture data. 7. The apparatus according to claim 6, wherein the positioning tag is a radio frequency identification (RFID) tag. 8. The apparatus according to claim 5, characterized in that, The order processing module is further configured to: in the entrance area, acquire the facial recognition data of the user selecting goods, wherein the facial recognition data corresponds to the user's user identifier; when it is determined that the goods and the user are located in the exit area, acquire the facial recognition data again to obtain the user identifier corresponding to the facial recognition data, wherein the order is the order of the user corresponding to the user identifier.