JP2024069458A - SUPPORT SYSTEM, SUPPORT PROCESSING DEVICE, AND SUPPORT METHOD - Google Patents

Abstract

[Problem] To provide a support system with high usability. [Solution] A support system comprising: a storage unit that stores information on a customer's behavior pattern in a store, information on an in-store behavior classification that indicates a classification of the customer's behavior in the store, and support information including at least information regarding guidance for the in-store system; a sensor unit that acquires behavior data that is information on the customer's behavior in the store and location data that is information on the location where the behavior was performed; a calculation unit that identifies the behavior pattern of the customer from the behavior data, identifies the in-store behavior classification of the customer based on the behavior pattern and the location data, and identifies the support information intended for the customer based on the in-store behavior classification of the customer; and a display unit that displays the support information identified by the calculation unit to the customer. [Selected Figure] Figure 3

Description

The present invention relates to a customer support system within a store.

Labor shortages, social needs for non-contact shopping, and technological advances in various sensor devices and the use of information obtained from them are driving the introduction of labor-saving and unmanned stores. In one example of such a store, a customer can enter the store, pick up products, pay, and complete their shopping journey without coming into contact with a store clerk until they leave.

In order to manage such stores, it is necessary to understand customer behavior within the store based on information acquired from sensors, etc. For example, Patent Literature 1 discloses a technology for analyzing customer behavior within a store.

JP 2019-139321 A

In store formats such as labor-saving stores and unmanned stores, it is necessary to be able to respond concisely when there is an unclear point or a problem. As a method for this, AI, RPA (Robotic Process Automation), chatbots, etc. are being worked on. In that case, an answer is presented based on the problem that the customer inputs into the terminal. However, the inventor found that in labor-saving stores and unmanned stores, there are various actions and their purposes for each customer, and in reality, the necessary answer is rarely presented only based on the problem that the administrator has assumed in advance. On the other hand, in a customer support system in a store, it is not realistic for the customer to input the background and situation in detail due to the complexity of the system and ease of use for the customer. Therefore, a support system with high usability is required that can accurately predict the problem that the customer is looking for and present a solution without requiring the customer to perform complicated operations.

The present invention provides a method for providing a service that includes: a storage unit that stores information on customer movement patterns in a store, information on in-store behavior classifications that indicate classifications of customer behavior in a store, and support information including at least information on guidance for an in-store system;
A sensor unit that acquires action data that is information on actions of customers in the store and location data that is information on a location where the actions are performed;
a calculation unit that identifies the behavior pattern of the customer from the behavior data, identifies the in-store behavior classification of the customer based on the behavior pattern and the location data, and identifies the support information for the customer based on the in-store behavior classification of the customer;
a display unit that displays the support information identified by the calculation unit to the customer;
It is a support system that includes:

The present invention makes it possible to realize a support system with high usability.

Problems, configurations, and advantages other than those described above will become clear from the description of the embodiment of the invention below.

FIG. 1 shows an example of a store configuration. FIG. 1 is a diagram showing an example of a schematic diagram of a support system according to an embodiment. FIG. 2 is a diagram showing an example of the configuration of a store management device in an embodiment. FIG. 4 is a diagram showing an example of map information of a store. FIG. 1 shows a table of customer behavior patterns. FIG. 13 is a diagram showing a table of classifications of customer in-store behavior. FIG. 11 is a diagram showing a first support information table. FIG. 13 is a diagram showing a second support information table. A diagram showing a table of personal data. Flowchart of data acquisition. 1 is a flow chart for identifying supporting information. FIG.

The following describes the embodiments with reference to the drawings. The embodiments are illustrative for explaining the present invention, and some parts have been omitted or simplified as appropriate for clarity of explanation. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural.

The position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc., in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings.

As examples of various types of information, expressions such as "table," "list," and "queue" may be used, but the various types of information may be expressed in other data structures. For example, various types of information such as "XX table," "XX list," and "XX queue" may be expressed as "XX information." When explaining identification information, expressions such as "identification information," "identifier," "name," "ID," and "number" are used, but these are interchangeable.

When there are multiple components with the same or similar functions, they may be described using the same reference numerals with different subscripts. Also, when there is no need to distinguish between these multiple components, the subscripts may be omitted.

In the embodiments, the processing performed by executing a program may be described. Here, the computer executes the program using a processor (e.g., CPU, GPU), and performs the processing defined by the program using storage resources (e.g., memory) and interface devices (e.g., communication ports). Therefore, the subject of the processing performed by executing the program may be the processor. Similarly, the subject of the processing performed by executing the program may be a controller, device, system, computer, or node having a processor. The subject of the processing performed by executing the program may be a calculation unit, and may include a dedicated circuit that performs specific processing. Here, the dedicated circuit is, for example, an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a CPLD (Complex Programmable Logic Device).

The program may be installed on the computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and a storage resource that stores the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. In addition, in the embodiments, two or more programs may be realized as one program, and one program may be realized as two or more programs.

In the following embodiments, the store will be described as a labor-saving store or an unmanned store. However, the present invention is not limited to this, and can be applied to any store that has the function of acquiring data on the position and movement of customers within the store, identifying the support content that the customer requires from that data, and informing the customer of that support content within the store.

Figure 1 shows an example of a store configuration in this embodiment. There is a gate for entering and exiting the store on the left side of Figure 1, and a terminal device for personal authentication is installed near the gate on the store entrance side. Near the gate on the store exit side, there is a counter and a settlement confirmation display for customers to confirm their purchase details when leaving the store. Note that, although an example is shown here in which the store entrance and exit are via the same gate, a gate dedicated to the entrance and a gate dedicated to the exit may also be provided separately.

In the payment area in FIG. 1, a cash register terminal is installed for customers to make payments for products purchased. In addition, multiple product shelves are provided for displaying products. Weight sensors and cameras are installed on the shelves to detect the movement of products. Cameras and TOF (Time Of Flight) sensors, not shown in FIG. 1, are installed on the walls and ceilings of the store to detect the behavior of customers in the store. In addition, display terminals are installed in multiple locations in the store, such as each product shelf, authentication device, cash register terminal, payment confirmation display, and sales floor walls, and communication with customers is possible by operating the display terminals. Basically, in unmanned stores and labor-saving stores, it is difficult to expect support from nearby store clerks, but customers can request necessary support information or support from store clerks through the display terminal. In addition, although not shown in FIG. 1, a manager's booth may be installed in the store, and a display terminal may be installed for the store manager. Note that the manager's terminal does not necessarily have to be installed in each store, and may be configured to be installed in a management center that manages multiple stores.

Figure 2 shows an example of a schematic diagram of a support system that supports customers. The support system has a store management device 1 for managing the purchasing behavior of customers in a store and managing the store. The store management device 1 is connected to various terminals and sensors, and is connected via a network N1 to an electronic commerce server 2, which is a server with which the store communicates when electronically settling transactions in the store, and a customer information server 3, which holds customer information obtained at one or more stores. When tracking a specific customer in the store, the store management device 1 can obtain customer information about that customer from the customer information server 3. In addition, when authenticating a customer, the store management device 1 may communicate with the customer information server 3 to refer to the authentication information of the target customer.

In the support system of FIG. 2, various sensors 4 are communicatively connected to the store management device 1 via a network. Information acquired by the various sensors 4 is transmitted to the store management device 1 via the network. Examples of sensors 4 shown in FIG. 2 include a weight sensor that is installed on a shelf and detects the weight of the shelf, and a TOF sensor that detects the movement of people in the store, the movement of parts such as hands, and the movement of objects. The sensor information from this weight sensor makes it possible to detect a change in the weight of the shelf when a customer picks up an item. Furthermore, by combining the information from the weight sensor with the information from the TOF sensor, it becomes possible to detect when a customer picks up an item and information about the item picked up.

Other examples of sensors 4 include sensors that detect storage information and operating status information of refrigerators that display and cool products in the store, sensors that detect operating status information of microwave ovens that heat products in the store, AI speakers that acquire voice information of customers in the store, and thermometers that detect the temperature in the store. It is also possible that information entered into a smartphone held by a customer who enters the store is transmitted to the store management device 1 and used for analyzing customer behavior. In this case, the smartphone may function as sensor 4. So far, examples of sensors 4 connected to the store management device 1 have been described, but sensors 4 are not limited to these and may include various sensors for acquiring information about the environment in the store and the situation of customers.

In the support system of FIG. 2, the authentication terminal 5, the payment terminal 6, and the display terminal 7 are communicatively connected to the store management device 1 via a network. The authentication terminal 5 is a terminal for performing personal authentication when a customer enters a store. The authentication terminal 5 has an input unit for the customer to input authentication information. Here, the input unit may be a display that the customer can touch or a scanner that reads a barcode presented by the customer. The authentication information input to the authentication terminal 5 is transmitted to the store management device 1, and the store management device 1 authenticates the customer based on the authentication information. The payment terminal 6 is a terminal for the customer to make a payment. After the customer selects the product to be purchased in the store, the customer makes a payment at the payment terminal 6. The payment terminal 6 can perform a payment process based on transaction information linked to each customer. The payment terminal 6 may have a screen for displaying transaction information to the customer or a touch-operable display for selecting a payment method. The payment terminal 6 may have an authentication unit for authenticating the customer when making a payment. The payment terminal is the cash register terminal in FIG. 1. One or more display terminals 7 are provided in a store, and are used to display specific information to customers and store clerks. The display terminal 7 is the terminal with a display in FIG. 1. The specific information includes support information for solving problems customers may have in the store. The display terminal 7 may also be a terminal that displays a management screen that displays notification information for store clerks. The display terminal 7 may have an input unit that allows customers and store clerks to input information. The input unit may be a touch-operable display. Although an example is shown here in which the display terminal 7 is installed in advance in the store, it may also be a device terminal carried by a customer on which a store guide app is installed.

Communications between the customer information server 3, the e-commerce server 2 and the store management device 1, and between these servers and mobile terminals such as smartphones, can be carried out using a general public line network, whether wired or wireless, such as the fifth generation mobile communication system, known as 5G (5th Generation), which allows for "multiple simultaneous connections" and "ultra-low latency".

The store management device 1 can obtain customer purchasing data and customer situations based on data sent from various sensors 4 and various terminals. Furthermore, when using a smartphone as the sensor 4, it is possible to read code information for store entry from the smartphone, or obtain location information of the owner from the smartphone and add this to the information analysis of the location of actions within the store and the speed of movement. Furthermore, an AI speaker can be used to obtain information about situations in which customers are in difficulty or the situation the customer is in from conversations with the customer. This allows for more accurate analysis of the customer's situation.

Figure 3 shows an example of the configuration of a store management device 1 for supporting customers. The store management device 1 is composed of a general-purpose server device equipped with, for example, a data storage unit 10, an AI data accumulation unit 20 for improving accuracy, an authentication unit 30, a payment unit 40, an input unit 50, an output unit 60, a CPU 70, and a communication interface 80. The store management device 1 is not limited to the configuration shown in Figure 3. The store management device 1 may be a single server device, or may be composed of multiple servers. In addition, each device of the store management device 1 may be arranged in a single device, or may be arranged in multiple devices so as to be distributed. Each program and each piece of information held by the data storage unit 10 may be stored in a single storage device, or may be stored separately in multiple storage devices so as to be distributed. In addition, the store management device 1 may be configured by combining the customer information server 3 and the electronic commerce server 2 in Figure 2.

The store management device 1 may have one or more components. For example, the store management device 1 may be a computer system made up of one or more computers (electronic calculators). In addition, some of the components of the store management device 1 may be missing, and other components may be added.

The data storage unit 10 stores at least the data and programs necessary for tracking and analyzing customer behavior in the store. The data storage unit 10 is composed of a large-capacity non-volatile storage device such as a non-volatile semiconductor memory, a hard disk device, or an SSD (Solid State Drive), and is used to retain necessary information for a long period of time.

The action data 101 includes, for example, information on actions performed by a customer measured by a TOF sensor installed in a store. The TOF sensor can recognize a three-dimensional space by, for example, measuring the time it takes for infrared rays to reflect and return and calculating the distance. Specifically, the TOF sensor acquires information on objects in a space as a point cloud, and can recognize actions by using the combination of the points and the distribution of the point cloud as teacher data. For example, information on the movement of a customer, such as a customer stopping in front of a shelf, can be acquired. Furthermore, information on the action of a customer, such as a customer reaching out for a shelf or a customer turning their face in a specific direction, can be acquired. The sensor that acquires the action data 101 is not limited to a TOF sensor. In addition to the TOF sensor, data can also be acquired from a camera. In that case, as with the TOF sensor, in addition to information on the movement of the customer, information on the action of the customer, such as body movement and face direction, can be acquired. The action data is stored in association with information on the action performed by the customer and location information, which is information on the location in the store where the action was performed.

Sensor data 102 is data acquired from various sensors 4 installed in the store. For example, it includes information detected by a weight sensor installed on a shelf.

Product information 103 includes inventory information of products placed in the store and information about products that customers have picked up and plan to purchase. Note that information about products that customers have picked up and plan to purchase can be obtained by combining information contained in the action data 101 or the sensor data 102, for example, information from a weight sensor and information from a TOF sensor.

The in-store map information 104 is information showing map information within the store. The in-store map information 104 includes information regarding the division of areas within the store. The information regarding the division of areas within the store may include at least the types of areas within the store, such as an entrance area, an exit area, an area near shelves, an aisle area, and a payment area.

The movement pattern data 105 is a database that contains log information indicating what pattern of movement the customer performed. The movement pattern data 105 may also store in advance list information of movement patterns to be referenced in order to determine the pattern of movement performed by the customer. The movement pattern data 105 is identified by the movement pattern analysis program 110, described below, by determining from the movement pattern list the data with the highest matching rate for the data obtained from the movement data 101. The movement pattern data 105 is stored in association with location information regarding the location where the movement of the movement pattern was performed.

The in-store behavior classification data 106 stores behaviors performed by customers in a store in association with the purpose or motive of the behavior. Here, the classification of various behaviors performed by customers in a store for a certain purpose or motive by type of purpose or motive or type of behavior is called an in-store behavior classification. The in-store behavior classification can be estimated from the customer's behavior pattern and information about the location where the behavior pattern was performed. The in-store behavior classification data 106 is a database that contains log information of the in-store behavior classification of customers. The in-store behavior classification data 106 may also store in advance list information of expected in-store behavior classifications that should be referenced to determine the in-store behavior classification of the customer.

The support information data 107 includes support information presented to customers to solve problems that customers have in the store. Examples of support information include information for presenting a method of entering a store for customers who do not know how to enter the store, and information for presenting sales floor information for customers who want to find a specific product. The support information data 107 may be created in advance according to problems that customers are expected to have in the store. The support information data 107 may include information showing the correspondence between in-store behavior classification data and support information to be provided to customers. The support information data 107 may be created by accumulating matters that customers may have problems with in advance. In addition, the content of the problem entered by the customer and support information that is a solution to that problem may be automatically added from the performance data when the customer uses the system of the present invention. Data requiring support may be obtained from information on a questionnaire given to customers and automatically expanded.

The personal data 108 is log data related to each customer, which is linked to an individual customer identified by personal authentication and stored. The personal data 108 stores information on the behavior patterns performed by each customer when they visited the store currently or in the past, information on in-store behavior classification, and inquiry history. Here, the inquiry history may be a reference history of support information. The personal data 108 may also include information including customer authentication information and authentication history, payment history including information on payment method and amount, and information on the type and quantity of purchased products. When identifying the in-store behavior classification and support information of each customer, the personal data 108 of each customer may also be referenced. As a result, support information can be accurately identified according to the attributes of the individual customer. The personal data 108 may be obtained and stored as log data of each customer at other stores. Here, an example is shown in which the personal data 108 is log data at a specific store, but the present invention is not limited to this, and the personal data 108 may be processed data indicating individual attributes obtained from a third party such as an information provider.

The data acquisition program 109 is a program that has the function of exchanging necessary commands and information with various external sensors 4, authentication terminal 5, payment terminal 6, display terminal 7, electronic commerce server 2, and customer information server 3 via the communication interface 80. The data acquisition program 109 stores information acquired from various external sensors 4, etc. in operation data 101, sensor data 102, product information 103, personal data 108, etc.

The movement pattern analysis program 110 is a program that identifies a customer's movement pattern based on the information in the movement data 101. For example, it is possible to identify the pattern of change in the customer's center of gravity and posture, and the movement pattern of the hands and feet, from information on time-series changes in three-dimensional coordinates indicating the positions of characteristic points of the customer's body obtained from a TOF sensor. The movement pattern analysis program 110 may also identify the corresponding movement pattern of the customer by referring to list information of preset movement patterns included in the movement data 101. The identified movement pattern of the customer is stored in the movement pattern data 105 in association with the identification information of the customer.

The in-store behavior analysis program 111 is a program that identifies a customer's in-store behavior classification based on at least the information of the behavior data 101, the in-store map information 104, and the behavior pattern data 105. For example, by referring to a list of preset in-store behavior classifications, the program may identify the corresponding in-store behavior classification of the customer by using information on the customer's behavior pattern and the area category to which the location where the behavior pattern was performed as a search key. In addition, the personal data 108 may be referenced when classifying the in-store behavior. The identified in-store behavior classification of the customer is linked to the customer's identification information and stored in the in-store behavior classification data 106.

The support information identification program 112 is a program that identifies support information to present to a customer based on the customer's in-store behavior classification information contained in the in-store behavior classification data 106.

In the AI data accumulation unit 20 for improving accuracy, for example, as learning data 201, past inquiry content 204, which is information on inquiries from customers, location at time of inquiry 205, which is information on the location of past customers, behavior data at time of inquiry 206, which is information on the behavior data or behavior pattern of past customers, and evaluation data 207, which indicates whether the support information presented to the customer was appropriate, are stored. The past inquiry content 204 is information indicating the support information referred to by the customer. The evaluation data 207 may be a response to a questionnaire that the customer answers after providing support to the customer. As an example, the questionnaire includes questions regarding whether the displayed support information was appropriate or whether the support response was satisfactory. In addition, the AI data accumulation unit 20 for improving accuracy stores an inquiry behavior analysis program 202 and an output calculation program 203.

The inquiry behavior analysis program 202 updates the list information of behavior patterns included in the behavior pattern data 105 and the list information of in-store behavior categories included in the in-store behavior category data 106 based on past inquiry content 204, location at the time of inquiry 205, and inquiry time behavior data 206, which is information on past customer movements. As an example, when behavior data when an inquiry was made in the past is analyzed and it is determined that the behavior pattern when the inquiry was made is not in the list information, it is stored as a new behavior pattern in the behavior pattern data 105. As another example, when information on the behavior pattern and location when an inquiry was made in the past is analyzed and it is determined that the in-store behavior category when the inquiry was made is not in the list information, it is stored as a new in-store behavior category in the in-store behavior category data 106.

The output calculation program 203 evaluates the validity of the in-store behavior classification of the customer inferred from the customer's movement pattern and location, based on past inquiry content 204, location at the time of inquiry 205, behavior data at the time of inquiry 206 which is information on the customer's past movements, and evaluation data 207. As an example, there are cases where multiple in-store behavior classifications correspond to the same movement pattern and location data, and a reliability level, which is an index showing the accuracy of the estimation, is set for the multiple in-store behavior classifications. In this case, based on the evaluation data 207, an evaluation is performed to lower the reliability of the in-store behavior classification corresponding to support information that has been given feedback from the customer as being inappropriate.

The authentication unit 30 is a device having a function of performing a process of personal authentication of a customer. The authentication unit 30 can perform personal authentication of a customer based on the customer's authentication information input to the authentication terminal in FIG. 2. The customer authentication process may be configured to be performed by an external server via the communication interface 80.

The settlement unit 40 is a device that performs settlement processing for customer transactions. The settlement unit 40 can perform settlement processing based on transaction information linked to the customer's identification information and the payment method specified by the customer. The transaction information may be created by the customer reading and registering products at the cash register terminal. The transaction information may be generated from information on a list of products to be purchased that is automatically created and managed by obtaining information on products placed in a basket by the customer from information from a TOF sensor and a weight sensor. The customer may be able to select a payment method at the cash register terminal, or a predetermined payment method linked to the customer's identification information may be automatically selected. The authentication process may also be configured to be performed by communicating with an external server such as the electronic commerce server 2 via the communication interface 80.

Information entered by customers or store managers is input to the input unit 50. For example, a customer may request support information required by that customer via the input unit 50. In this case, the display terminal 7 in FIG. 2 may be connected to the input unit 50 to exchange information. As another example, when a manager updates information in the database stored in the data storage unit 10, the manager may input information via the input unit 50. When a manager views information regarding support responses to customers in the store, the manager can request information regarding data on in-store behavior of a specific customer and support response history via the input unit 50. In this case, a management terminal for the manager and the input unit 50 may be connected to enable information exchange.

The output unit 60 outputs information to be presented to customers and store managers. For example, support information may be output in response to an input by a customer requesting support information. In this case, the display terminal 7 in FIG. 2 may be connected to the output unit 60 to exchange information. As another example, the output unit 60 may be configured to output data on customer behavior within the store and support response history for managers. In this case, the output unit 60 may be connected to a management terminal for managers and configured to exchange information. Alarm information may also be output to alert customers and managers to certain precautions.

The CPU 70 is a processor that controls the overall operation of the store management device 1. For example, a memory composed of a volatile semiconductor memory is used as the work memory of the CPU 70.

The communication interface 80 is a communication device for communicating with the various sensors 4, authentication terminal 5, payment terminal 6, display terminal 7, electronic commerce server 2, and customer information server 3 in FIG. 2 using a predetermined wireless communication method.

Figure 4 shows an example of map information for a store. Figure 4 clearly shows an example of the division of areas within a store. In Figure 4, the entrance area, exit area, area near the shelves, payment area, and aisle area are shown as representative examples of areas within a store. Note that examples of map information within a store are not limited to these. Other examples may include a backyard area, toilet area, and eat-in space area, and these may be added depending on the operation style and internal environment of the store.

The entrance area is an area where authentication is performed before a customer can enter the store. In the entrance area, the customer may enter authentication information into a terminal to perform personal authentication. The store management device 1 can obtain authentication information that personally identifies the customer from the information entered into the authentication terminal. In addition, information on the customer's behavior when entering the store can be analyzed from the customer's behavior data obtained from the TOF sensor in the entrance area.

The exit area is an area where customers undergo authentication in order to leave the store. In the exit area, customers may be able to confirm the details of their payment using a payment confirmation display. The store management device 1 can analyze information about the customer's behavior when leaving the store from the customer's behavior data obtained from the TOF sensor in the exit area.

The shelf vicinity area is the area near the product shelves in the store. The store management device 1 can analyze information on customer movements and behavior near the shelves from information on customer movement data obtained from TOF sensors, cameras, etc. in the shelf vicinity area. For example, information on the speed at which a customer moves along the shelves, information on the direction their face is facing, information on whether they have stopped, and information on the length of time they have stopped can be obtained. Furthermore, information entered into a terminal with a display installed on the product shelves can also be used as information on customers near the shelves.

An aisle area is an area that is primarily used as a passageway for moving around within a store. An aisle area may be an area within a store that does not fit into other area categories. The store management device 1 can analyze information on customer movements and behavior in the aisle from customer movement data information obtained from a TOF sensor, camera, etc. in the aisle area. For example, information on the speed at which a customer moves through the aisle, the direction their face is facing, whether they have stopped, the length of time they have stopped and the number of times they have stopped can be obtained.

The payment area is the area near the product shelves in a store. The store management device 1 can analyze information on the customer's movements and behavior near the shelves from information on customer movement data obtained from TOF sensors, cameras, etc. in the area near the shelves. For example, information on the speed at which the customer moves along the shelves, the direction their face is facing, whether they have stopped, and the length of time they have stopped can be obtained. Furthermore, information entered into a cash register terminal or a terminal with a display installed in the payment area can also be used as customer information in the payment area.

Figure 5 shows an example of a table of customer motion patterns. The table in Figure 5 is included in the motion pattern data 105 in Figure 3. The table in Figure 3 includes a motion pattern CD501 and an acquired motion pattern 502. The motion pattern CD501 is code information for identifying each motion pattern. For example, by referring to the table in Figure 5, it is determined whether the characteristics of the customer's motion data obtained from a TOF sensor, a camera, etc. correspond to one of the acquired motion patterns 502 in Figure 5. As an example, it is possible to determine whether the speed at which the customer moved is faster or slower than a reference (20 m/min) as a different motion pattern. The determined customer motion pattern is stored in the motion pattern data 105 for each customer. Note that the acquired motion pattern 502 shown in Figure 5 is an example, and may be specified in more detail for each item, such as direction, speed, frequency, number of times, and duration. In addition, without using the table information shown in Figure 5, it is also possible to identify what the customer's motion data is intended to be based on the results of AI behavior analysis and use it as a motion pattern.

Figure 6 shows a table of in-store customer behavior classification. The table in Figure 5 is included in the in-store behavior classification data 106 in Figure 3. In-store behavior classification is a classification of various behaviors performed by customers in a store for a certain purpose or motive, classified by type of purpose or motive and type of behavior, and can be identified from information on the customer's behavior pattern and the location where the behavior pattern was performed. For example, if the location is the entrance and the movement is "turning the face toward the shelf and looking at multiple aisles for more than 2 seconds," the in-store behavior classification is identified as "looking for a store clerk/wanting to know whether a specific product is handled." The table of in-store customer behavior classification in Figure 6 includes location CD601, location 602, behavior CD603, behavior pattern 604, in-store behavior CD605, type of in-store behavior classification 606, in-store behavior classification 607, and reliability 608. The location 602 may be described as the above-mentioned area division in the store. The action CD603 and the action pattern 604 may correspond to the action pattern CD501 and the acquired action pattern 502 in FIG. 5, respectively. Here, each in-store action classification may be set in advance for a combination of a location and an action pattern. Furthermore, one or more inferred in-store action classifications may correspond to one combination of a location and an action pattern. For example, when the location is an entrance and the action is "staying for more than two seconds without moving coordinates", the in-store action classification may be configured as a table corresponding to two cases, "searching for a store clerk, because I can't enter the store (I authenticated but the door won't open)" and "searching for a store clerk, I don't know how to enter the store for the first time".

Next, the reliability 608 will be described. The reliability 608 is an index showing the likelihood of a prediction regarding the in-store behavior classification 607 predicted from a combination of the location 602 and the behavior pattern 604. The reliability 608 is set for each combination of the location 602, the behavior pattern 604, and the in-store behavior classification 607. For example, a reliability value of 70 indicates that there is a 70% chance that the in-store behavior classification linked to the corresponding location and behavior pattern matches the actual in-store behavior of the customer.

Here, the percentage is displayed, but it is not limited to a probability display, and may simply be a score value. The reliability may be a preset value, or may be updated based on past performance data or learning. As an example, the AI data accumulation unit 20 for improving accuracy described above can update the information of the reliability 608 based on the learning data 201. The information of the reliability 608 is used when identifying the support information to be displayed to the customer, as described below.

As an example shown in FIG. 6, if the behavior pattern at the entrance is "staying for more than 2 seconds without moving the coordinates" and the store behavior classification is "searching for a store clerk, because I was authenticated but the door won't open" then the reliability value is set to 30 in the customer's in-store behavior classification table. Also, if the behavior pattern at the entrance is "staying for more than 2 seconds without moving the coordinates" and the store behavior classification is "searching for a store clerk, because I don't know how to enter the store for the first time", then the reliability value is set to 50 in the customer's in-store behavior classification table. If the behavior pattern in the aisle is "turning the face toward an aisle with no shelves and looking through multiple aisles for more than 2 seconds" and the store behavior classification is "searching for a store clerk, because I want to know if a specific product is being sold", then the reliability value is set to 10. These specific behavior patterns and their times vary depending on the store format, products handled, store structure, etc.

Figures 7A and 7B show examples of support information tables. The tables of Figures 7A and 7B are included in the support information data 107 in Figure 3. The support information table shows the correspondence between in-store behavior classifications such as those shown in Figure 6 and support information to be provided to customers. Figure 7A is a first support information table. The first support information table consists of in-store behavior CD (code) 701, concerns/uncertainties/questions 702, and support information ID 703. In-store behavior CD 701 corresponds to in-store behavior CD 605 in Figure 6.

Figure 7B is a second support information table. The second support information table consists of a support information ID 704, a display command 705, and support content 706. Display commands and support content expected to be necessary when providing customer support are displayed on a terminal with a display placed on a shelf or the like. For example, when a customer touches the display screen of the terminal with a display, support content related to the service the customer requires can be displayed on the terminal with a display. Support information ID 704 in Figure 7B is the data of support information ID 703 in Figure 7A.

Figure 8 shows a table of personal data linked to personal authentication. The table of personal data consists of a first personal data table 801, a shopping information table 802, a shopping information child table 803, and an inquiry information table 804. The table in Figure 8 is included in the personal data 108 in Figure 3. The first personal data table 801 is an example of storing a member CD, a finger vein/two-dimensional barcode that stores two-dimensional barcode and finger vein information, a name, a payment method, shopping information, and support system usage. The finger vein/two-dimensional barcode is information used to identify an individual customer. The payment method is information related to payment such as cashless payment. The shopping information stores a shopping information ID. The shopping information ID can be used to link information in the shopping information table 802 and the shopping information child table 803. The support system usage status stores information on whether the member has used the support system, and if the member has used the support system in the past, an inquiry number is stored. The inquiry number can be used to link information in the inquiry information table 804 from support system usage status.

The shopping information table 802 stores information such as the store of purchase, purchase date, and payment information. The shopping information child table 803 stores the purchase date, purchased item, purchase price, and purchase quantity. The inquiry information table 804 stores performance information related to the customer's past behavior and inquiry history, such as the content of the inquiry, location, and behavioral data. The content of the inquiry may indicate the support information referenced.

Based on these pieces of information, it is possible to analyze the correspondence between the information on the past behavioral results of a certain individual and the inquiry information at that time, as to which shopping information was used when a customer returns to the store after purchasing an item, for example. Furthermore, information such as the reliability 608 is updated based on the information on the correspondence. For example, there are cases where the same movement pattern and position data correspond to multiple in-store behavior classifications. In that case, by analyzing the past history of the inquiry information table 804, if the frequency with which the first support information corresponding to the first in-store behavior classification among the multiple in-store behavior classifications is inquired is greater than the frequency with which the support information corresponding to the other in-store behavior classifications included in the multiple in-store behavior classifications is inquired, the reliability of the first in-store behavior classification may be updated to be greater than the reliability of the other in-store behavior classifications. Here, when the reliability 608 is updated based on personal data linked to personal authentication, information on the past performance of each customer may be obtained, and the reliability 608 may be set for each customer based on the information. In this way, it is possible to estimate the in-store behavior classification based on the characteristics of each customer, which contributes to improving accuracy. Also, instead of acquiring information on past performance for each customer, similar customers may be grouped based on age, gender, tendency of purchased product types, etc., and information on past performance for each customer group including the similar customers may be acquired. In this case, a reliability 608 is set for each customer group based on the information. Then, when predicting the in-store behavior classification of a certain customer, a determination is made as to which customer group the customer is classified based on the personal authentication information of the customer, and the in-store behavior classification of the customer is predicted using the reliability 608 set for the customer group to which the customer belongs. This makes it easier to acquire a sufficient amount of information on past performance to set the reliability 608, compared to when reliability 608 is set for each customer, which contributes to improved accuracy.

Figure 9A is an example of a flowchart for acquiring data. This processing flow starts when the authentication terminal 5 in Figure 2 detects that a customer is standing in front of the authentication terminal 5. The customer then inputs information into the authentication terminal, which causes the authentication terminal 5 to detect the customer (S10). Note that the customer may also be detected by having the customer hold a two-dimensional code displayed on the customer's mobile terminal over the reading section of the authentication terminal 5. Here, the two-dimensional code is linked to identification information that uniquely identifies the individual customer.

After detecting the customer, the store management device 1 reads out the personal information. The authentication unit 30 then authenticates the customer (S20).

By executing the data acquisition program 109, data on the movements of the target customer is acquired using a TOF sensor, camera, etc., and in-store behavior is tracked (S30). For example, it is possible to detect when a customer reaches out towards a shelf on which products are displayed. By using information that the customer has reached out towards the shelf and data from a weight sensor installed on the shelf, the product acquired by the customer can be identified and data on that product can be acquired from the app. In addition, it is possible to detect the movement of the customer placing the product determined to have been acquired by the customer into a basket using a TOF sensor, camera, etc. In this case, the product is added to a list of products that the customer plans to purchase, which is managed by the in-store data server 95.

Furthermore, in S30, the action of returning an item may be detected by a TOF sensor, a camera, or the like. In this case, the item that the customer recently picked up from the basket is identified, and the identified item is deleted from the customer's list of items to be purchased. As described above, data on items that the customer has picked up or returned can be obtained in chronological order from information obtained from sensors such as a TOF sensor, a camera, and a weight sensor. This chronological information can be used when analyzing customer behavior within the store.

Next, when it is detected that the customer has arrived in the payment area, the payment process begins. The payment unit 40 makes payment for the products on the product list that the customer plans to purchase using the payment method registered for that customer. At that time, the customer may be configured to be able to check the product list to be purchased and select the payment method on a terminal installed in the payment area. When the payment is completed, transaction details are sent to the customer's mobile terminal, and payment information including the details is sent to the electronic commerce server 2 of the support system, and the payment process ends (S40). Thereafter, when the customer passes through the store exit area, the data acquisition program 109 is executed to determine that the customer has left the store, and the customer is logged out (S50), and the data acquisition process ends.

Figure 9B is an example of a flowchart for identifying support information. In this embodiment, customer behavior within the store is constantly detected from the time the customer enters the store to the time the customer leaves. Then, the process of identifying support information is initiated, for example, when a person enters a specific area, when a person is detected as touching or approaching an inquiry terminal, or when a specific motion of a person is detected. (T00).

By executing the data acquisition program 109, customer information for the support recipient is acquired (T01). By executing the data acquisition program 109, information from the TOF sensor, camera, etc. is used to acquire customer behavior data and location information at the time the customer starts a support inquiry and immediately before that (T02). By executing the data acquisition program 109, personal data 108 is acquired from the data storage unit 10 (T03). By executing the motion pattern analysis program 110, the customer's motion pattern is identified from a table of customer motion patterns and motion data from the TOF sensor, camera, etc. (T04).

The in-store behavior analysis program is executed to identify the in-store behavior classification of the customer from the data on the behavior pattern in the table of in-store behavior classification of the customer and the location data, which is information on the location where the behavior of the behavior pattern was performed (T05). Note that the location data may be information on area divisions within the store included in the in-store map information 104.

Next, the in-store behavior analysis program is executed to determine whether the in-store behavior classification satisfies a predetermined criterion (T06). The predetermined criterion may be when customer movements are acquired from a TOF sensor or a camera, and a product is handed over between two or more customers. When a product is handed over, it may not be possible to acquire information about the product handed over by the customer, and as a result, there is a risk that the information in the list of products to be purchased may be erroneously linked to the customer's intention. In addition, cases where an abnormality is found in the analysis of sensor information or in-store behavior may be included as cases where the predetermined criterion is satisfied. Other behavior patterns that are prohibited by the store may be included in the predetermined criterion as error behavior. An example of such a prohibited behavior pattern is putting a picked up product away in a location other than the basket.

If the in-store behavior classification meets a predetermined criterion in T06 (YES in T06), an in-store behavior analysis program is executed, and a display unit such as a display placed on a shelf or the like displays a notification for the customer and the manager (T07). If it is detected that a product has been handed over between two or more customers, the customer may be notified to manually check the contents of the list of products to be purchased, or to avoid handing over products that may contain incorrect information on the list of products to be purchased. In addition, a notification may be sent to the manager urging them to check the information of the customer who made the handover.

In addition, customers may visit a store in groups, such as a parent and child, and the store in this embodiment may be configured to process payment for products purchased by customers in the same group together for each group. In this case, if it is detected that a product has been handed over between two or more customers in a group, the customers may be notified when making payment to have the group's bill processed simultaneously in the same payment area.

In the case where the predetermined criteria are not met in T06 and the result is normal (NO in T06), the support information identification program is executed to identify the support information for the customer from the in-store behavior classification (T08). Here, in addition to the in-store behavior classification, information on the past inquiry records of the target customer may be referenced. This makes it possible to identify support information tailored to the target customer, and improves the accuracy of support information selection. As an example, support information provided once or more within a specified period may be excluded, or support information that is frequently referenced by the customer may be identified and preferentially displayed regardless of its reliability. As another example, support information that has been presented once in the past may also be displayed to indicate that it has been referenced in the past. This helps the customer intuitively select support information. Then, the display unit displays the identified support information (T09). The data storage unit stores the customer's inquiry content, inquiry location, and behavior data at the time of inquiry in the inquiry information table 804 (T10). Then, the process of FIG. 9B ends (T11).

Here, if the support information displayed on the display unit does not contain the support information the customer desires, and the customer searches for support information via a display in the store, the combination of the behavior classification at that time, the inappropriate support information, and the support information the customer desires that the customer has searched for and referenced themselves may be stored separately in a table separate from the inquiry information table in the personal data table. The combination information of the inappropriate support information and the support information desired by the customer is analyzed. Then, the support information desired by the customer is added as information on the in-store behavior classification associated with the location and movement information inquired about by the customer in Figure 6. Alternatively, the inappropriate support information is corrected. Updating the table of in-store behavior classification of customers in Figure 6 in this way contributes to improving accuracy.

By authenticating the individual, inquiry information such as the customer's past support response history, past behavioral records of the individual, purchasing data, and other behavioral information are referenced, improving usability in identifying support information. As an example, support information provided more than once within a specified period of time may be excluded, or support information that is frequently referenced by the customer may be identified and displayed preferentially regardless of its reliability. As another example, support information that has been presented once in the past may also be displayed to indicate that it has been referenced in the past. As another example, when an in-store behavior classification indicating that the customer is looking for a certain product is identified by utilizing past purchasing data, information on products that the customer has purchased in the past may be displayed as candidates along with support information corresponding to the in-store behavior classification.

The support information required by a customer varies depending on the type of in-store behavior classification of the customer. Therefore, the classification categories of in-store behavior classification may include at least the types of behavior related to entering and leaving the store, behavior related to product selection, behavior related to searching the store, and behavior related to payment. By categorizing support information by type of customer behavior classification, it is possible to efficiently identify support information corresponding to the type of in-store behavior classification of the customer to be supported. In particular, the above-mentioned behavior related to entering and leaving the store, behavior related to product selection, behavior related to searching the store, and behavior related to payment are broad classifications of customer behavior flows within the store, and by identifying support information based on these classifications, the possibility of irrelevant support information being presented to the customer is reduced. Note that the types of in-store behavior classification are not limited to the above examples, and any type may be added.

The action data 101 includes information on the first location where the first action was performed. Here, the information on the first location may be coordinates within the store. The store management device 1 can refer to the information on the area divisions within the store included in the in-store map information 104, identify the area division to which the first location belongs, and use it to identify the in-store behavior classification. The information on the area divisions within the store may include at least the entrance/exit area, the area near the shelf, the aisle area, and the payment area as types of areas within the store. By taking into account the area in the store where the customer performed the behavior, it is possible to accurately identify the in-store behavior classification corresponding to each problem of the customer. In particular, the entrance/exit area, the area near the shelf, the aisle area, and the payment area described above are related to behavior related to entering and leaving an unmanned store, behavior related to product selection, behavior related to searching within the store, and behavior related to payment, respectively. Therefore, classifying the areas into the types contributes to improving the accuracy of identifying support information suitable for the customer. The types of areas within the store are not limited to the above examples, and any type may be added.

Figure 10 shows a display screen. Figure 10 shows an example of a display screen that the store manager can see. In this example, the store manager can see the display of the "shopping process" consisting of entering the store, wandering around the store, selecting a product, purchasing, paying, and leaving the store, the customer's current location in the store, and "display commands" such as "Are you looking for the location of the product you are looking for?", which is a question to the customer. The customer can see the "Can I help you with something?" part in the lower right of the display in Figure 10 through a display arranged on a shelf in the store. For example, the customer can touch the display area of "Can I help you with something?" to display support information related to the desired service on the display. In addition, if there are multiple pieces of support information for a certain customer, the multiple pieces of support information may be displayed from the top in order of their relevance. As an example, information with high reliability may be displayed from the top based on the reliability score in Figure 6. In addition, information on the reliability of the customer may be updated from information on the customer's past behavioral performance or past inquiry information, and the information may be displayed in the order of the updated reliability. For example, support information provided more than once within a specified period may be excluded from the display. Alternatively, support information that is frequently referenced by customers can be identified and displayed preferentially regardless of its reliability.

Referring to Figure 6, when analyzing the table of in-store customer behavior classifications, there may be multiple candidates that fit the in-store behavior classification.

Specifically, in Figure 6, the location is "Entrance" and there are two hits for the in-store behavior CD for the movement "staying for more than 2 seconds without moving coordinates." In other words, there may be multiple hits for candidates that correspond to the first location and the first movement pattern.

Furthermore, a case where the location in Figure 6 is "Entrance" and the movement is "Turn face toward shelves and look at multiple aisles for more than 2 seconds" may be hit at the same time. For example, in Figure 6, three hits, 1110, 2222, and 1111, may be hit as in-store behavior CD from the location and movement. In other words, along with a candidate in-store behavior classification corresponding to the first location and first movement pattern, a candidate in-store behavior classification corresponding to the first location and second movement pattern may also be hit at the same time.

In such a case, the in-store behavior analysis program is executed to identify multiple in-store behavior categories corresponding to each of the hit behavior patterns. The display unit may then display multiple pieces of support information corresponding to the identified multiple in-store behavior categories on the same screen.

The in-store behavior analysis program may also be executed to obtain information on the number of inquiries for each in-store behavior category, and the display unit may display support information corresponding to the in-store behavior category with the most inquiries in order.

In addition, an input unit may be provided that allows the customer to select and input information relating to the current "shopping process" in FIG. 10, and support information may be identified based on the information about the shopping process inputted into the input unit. By receiving information about the "shopping process" from the customer, it becomes easier to narrow down the type of support the customer requires, and appropriate support information can be provided to the customer.

According to this embodiment, it is possible to accurately predict the problem the customer is having and present a solution to that problem without requiring the customer to perform complex operations.

1 Store management device, 4 Sensor, 10 Data storage unit, 60 Output unit

The present invention provides a storage unit that stores information on customer movement patterns in a store, information on in-store behavior classifications including at least behaviors related to entering and leaving the store, behaviors related to product selection, behaviors related to searching the store, and behaviors related to payment as classifications of customer behavior in the store, and support information including at least information related to guidance of an in-store system;
A sensor unit that acquires action data that is information on actions of customers in the store and location data that is information on a location where the actions are performed;
a calculation unit that identifies the behavior pattern of the customer from the behavior data, identifies the in-store behavior classification of the customer based on the behavior pattern and the location data, and identifies the support information corresponding to the in-store behavior classification of the customer;
a display unit that displays the support information identified by the calculation unit;
It is a support system that includes :

Claims (13)

a storage unit that stores information on customer movement patterns in a store, information on in-store behavior classifications indicating classifications of customer behavior in a store, and support information including at least information on guidance for an in-store system;
A sensor unit that acquires action data that is information on actions of customers in the store and location data that is information on a location where the actions are performed;
a calculation unit that identifies the behavior pattern of the customer from the behavior data, identifies the in-store behavior classification of the customer based on the behavior pattern and the location data, and identifies the support information for the customer based on the in-store behavior classification of the customer;
a display unit that displays the support information identified by the calculation unit to the customer;
A support system comprising: 2. The support system of claim 1,
The in-store behavior classification is classified into a plurality of types,
A support system in which the multiple types of actions include at least actions related to entering and leaving a store, actions related to product selection, actions related to searching the store, and actions related to payment. 2. The support system of claim 1,
The location data includes information about area divisions in the store,
A support system in which the area divisions include at least the store's entrance area, shelf vicinity area, aisle area, and payment area. 2. The support system of claim 1,
The storage unit stores authentication information for identifying the customer and performance information regarding the customer's past behavior and inquiry performance in the store,
The calculation unit is
Acquire the performance information of the customer based on the authentication information;
A support system that identifies the support information based on the performance information in addition to the in-store behavior classification of the customer. 2. The support system of claim 1,
Furthermore, a weight sensor is provided on the shelf on which the product is placed,
The calculation unit is
A support system that identifies the movement pattern based on the movement data as well as data obtained from the weight sensor. 2. The support system of claim 1,
The calculation unit is
determining whether the in-store behavior classification of the customer satisfies a predetermined criterion;
This support system notifies the store manager of information indicating that the specified criteria are met when the specified criteria are met. 7. The support system according to claim 6,
The calculation unit is
A support system which, when it is detected that a product has been handed over between two or more customers, determines whether the in-store behavior classification of the customers satisfies a predetermined criterion. 2. The support system of claim 1,
The calculation unit is
If there are a plurality of pieces of support information for the customer, determining an index indicating a relevance of each of the plurality of pieces of support information;
The display unit is
The support system displays the plurality of pieces of support information in order of the index of relevance determined to be high. 2. The support system of claim 1,
The calculation unit is
This support system stores in the memory unit a combination of the corresponding in-store behavior classification, the support information identified by the calculation unit and displayed on the display unit, and the support information searched and referenced by the customer on the display unit, if the support information displayed on the display unit does not contain the support information the customer is looking for. 2. The support system of claim 1,
an input unit that allows the customer to select and input information regarding the current shopping process;
The calculation unit is
A support system which identifies the support information based on the shopping process information inputted to the input section. 2. The support system of claim 1,
The calculation unit is
When a plurality of behavior patterns are identified as the behavior pattern of the customer, a plurality of in-store behavior classifications corresponding to the identified behavior patterns are identified;
The display unit is
A support system that displays a plurality of pieces of support information corresponding to the plurality of in-store behavior classifications on the same screen. a storage unit that stores information on customer movement patterns in a store, information on in-store behavior classifications indicating classifications of customer behavior in a store, and support information including at least information on guidance for an in-store system;
acquiring action data, which is information on actions of a customer in the store detected by a sensor unit installed in the store, and location data, which is information on a location where the actions were performed, identifying the action pattern of the customer from the action data, identifying the in-store behavior classification of the customer based on the action pattern and the location data, and identifying the support information for the customer based on the in-store behavior classification of the customer;
and a calculation unit that controls a display unit in the store to display the identified support information. acquiring action data, which is information on actions of customers in the store detected by a sensor unit installed in the store, and location data, which is information on the location where the actions were performed;
identifying a behavior pattern of the customer from the behavior data;
Identifying an in-store behavior classification indicating a classification of the customer's behavior in the store based on the movement pattern and the position data;
Identifying support information including at least information regarding guidance of an in-store system for the customer based on the in-store behavior classification of the customer;
and displaying the support information to the customer.