JP7779967B2 - Information processing device, control method and program for information processing device - Google Patents

Description

The present invention relates to an information processing device that tracks people in stores, etc., a control method for the information processing device, and a program.

Traditionally, in order to digitize customer behavior within a store, there has been technology that tracks the movement of customers inside the store as seen on surveillance cameras installed on the ceiling. Recently, this technology is being considered for use in walk-through unmanned stores, where customers can pick up the products they want and pay automatically, as a technology that links customers in the store with the products they purchase.

Patent Document 1 discloses technology that uses images taken inside a store to associate information about customers entering the store at the entrance with information about the customer's movement within the store, and also to associate information about the customer's movement from the payment area to the entrance with information about purchases made at the store.

Japanese Patent Application Laid-Open No. 2016-162256

However, the technology in Patent Document 1 tracks the in-store movements of all customers who enter the store, starting from the entrance and exit. As a result, when a large number of customers enter the store and the store becomes crowded with many customers, the tracking load increases, making it difficult to accurately track the movement of each customer as they move around the store. For example, there are issues such as the person being tracked switching places, making it easier for incorrect tracking to occur.

The present invention has been made to solve the above-mentioned problems. Its purpose is to provide a system that appropriately reduces the number of people who need to be tracked within a store and reduces the tracking load.

The present invention is characterized by comprising a tracking means that performs a person tracking process based on an image captured by a photographing means that photographs the inside of a store, and when a product payment is made by the person who is the subject of the tracking process, controls to terminate the tracking process of that person, and a control means that manages the number of people who are the subject of the tracking process and controls to reduce the number of people in response to a product payment being made by the person who is the subject of the tracking process.

This invention makes it possible to appropriately reduce the number of people who need to be tracked within a store, thereby reducing the tracking load. As a result, even when the number of customers increases and the store becomes crowded with many customers, it is possible to accurately track the movement of each customer within the store. For example, it is possible to prevent incorrect tracking from occurring when the person being tracked changes places.

FIG. 1 is a diagram illustrating an overview of an information processing system according to an embodiment of the present invention. FIG. 1 is a diagram illustrating an overview of an information processing system according to an embodiment of the present invention. FIG. 1 is a diagram showing an example of the configuration of an information processing system according to an embodiment of the present invention. FIG. 1 is a block diagram showing an example of the configuration of a tracking device according to an embodiment of the present invention. 4 is a flowchart showing an example of the operation of the information processing system according to the present embodiment. 4 is a flowchart showing an example of the operation of the information processing system according to the present embodiment. 3A to 3C are diagrams illustrating various tables used in the information processing system of the present embodiment. FIG. 10 is a diagram showing an example of a product payment screen displayed on the product payment terminal.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
1 and 2 are diagrams for explaining an overview of an information processing system according to an embodiment of the present invention.
The information processing system of this embodiment is a system that tracks (also called "pursuit" or "tracking") customers within the store 300 using images captured by a TOF camera 308 installed on the ceiling or other part of the store 300.

In this system, even if a customer, such as customer 303, enters the store from store entrance/exit 302 through passing gate 315, tracking of that customer does not begin at that point (the customer is not yet considered a tracking target). Many customers simply enter the store and leave without purchasing any products, so by not tracking such customers, the number of customers to be tracked can be reduced and the tracking load can be lightened.

When a customer, like customer 304, opens the door of a showcase 309 (309A, 309B, ...), the system starts tracking that customer (the customer is the tracking target). This is because a customer who opens the door of showcase 309 is considered to have a high possibility of purchasing product 310 displayed in showcase 309.
It should be noted that customer 305 corresponds to a customer who is already being tracked by this system.

Furthermore, when a customer, like customer 306, makes a payment using the product payment terminal 312, the system will end tracking of that customer (remove them from the tracking target). This is because customers who have completed product payment are considered to no longer need to be tracked. In this way, by removing customers who have made a payment or other payment and no longer need to be tracked from the tracking target, the number of customers to be tracked can be reduced, and the tracking load can be lightened.

In this way, the system of this embodiment reduces the number of customers who need to be tracked when they enter the store, and as a result, makes it possible to accurately track the movement of each customer as they move around the store, even when a large number of customers enter and the store becomes crowded with many customers.

Furthermore, when a customer being tracked, such as customers 304 and 305, picks up an item from showcase 309, the item is identified using information obtained from the TOF sensor 308 installed in showcase 309, associated with the customer, and made available for use at checkout.

Furthermore, if a customer being tracked who is holding a product (i.e., a customer who has not yet paid) passes through a passing gate 315 installed at an internal entrance/exit 302 of the store, the image captured by a surveillance camera 317 installed on the ceiling or other location of the store 300 is correlated with the time when the customer passed through the passing gate 315. This makes it possible to identify customers who have taken products out of the store without paying. Note that there may be multiple internal entrances/exits 302, each of which may be equipped with a passing gate 315.

FIG. 3 is a diagram showing an example of the configuration of an information processing system according to this embodiment, and the same components as those in FIGS. 1 and 2 are denoted by the same reference numerals.
In the information processing system of this embodiment, tracking device 100 is an information processing device that tracks customers in store 300 using video of the inside of store 300 captured by TOF camera 301. Note that tracking device 100 and TOF camera 301 may be separate entities, but tracking device 100 may be incorporated inside TOF camera 301, and an edge processing type TOF camera 301 that determines the start and end of tracking inside TOF camera 301 may be used.

Each showcase 309 (e.g., 309A, 309B, ... in Figures 1 and 2) is a showcase with a door, and opening the door allows products displayed in the showcase 309 to be removed. The door of each showcase 309 is equipped with a sensor that can detect when the door of the showcase 309 is opened by a customer. When the door of the showcase 309 is opened, that information is immediately (in real time) notified to the tracking device 100 via a LAN or the like. Note that the door of the showcase 309 is not limited to one that allows the interior of the showcase to be seen through, such as glass, and may also be one that does not allow the interior of the showcase to be seen through. Furthermore, in this embodiment, the "showcase" is not limited to a so-called commercial showcase (display shelf), and may be any type that can store products for sale, such as a simple shelf (a shelf with a door in the example of this embodiment).

Furthermore, each showcase 309 is provided with a TOF camera 308 (e.g., 308A, 308B, ... in Figures 1 and 2). The TOF camera 308 includes a TOF camera body (corresponding to the TOF camera device described below) and a control unit that can identify the type and number of products removed from the showcase 309 by a customer (or picked up by a customer) from information acquired by the TOF camera body. The TOF camera body is attached to the showcase 309 at an angle that allows it to capture images of products being removed from the showcase 309 in which it is installed. The control unit may be realized by a control board or the like connected to the TOF camera body, or by a personal computer or the like connected to the TOF camera body. Information regarding the products identified by the TOF camera 308 is notified to the tracking device 100 via a LAN or the like at the time of identification (in real time). That is, the TOF camera 308 functions as an identification device that identifies the product type and number of products removed from the showcase 309 from the video image capturing the state of products being removed from the showcase 309. Note that the product type and number of products removed from the showcase 309 are identified using, for example, a trained image recognition model, but other methods may also be used.

TOF cameras 301 and 308 are equipped with TOF (Time of Flight) camera devices (also called TOF sensors). TOF camera devices are capable of measuring three-dimensional information using the time of flight of light, and can simultaneously capture distance images based on the three-dimensional information and, for example, color visible images (but are not limited to color visible images). Images captured by such TOF camera devices can be used to track customers and identify products that customers have picked up.

The product payment terminal 312 is a payment device that allows customers to pay for products themselves, and there may be multiple terminals. A two-dimensional code reader 311 is connected to the product payment terminal 312, and information about the customer making the payment can be obtained by using the two-dimensional code reader 311 to read a two-dimensional code (e.g., a QR code (registered trademark)) displayed on the display of the customer terminal 314. In this embodiment, a QR code (registered trademark) is used as an example of a two-dimensional code, but this is not limited to this. When paying for a product, the product payment terminal 312 obtains and uses information about the product that the customer is holding from the tracking device 100. Note that when the customer removes a product from the showcase 309, the information about the product that the customer is holding is identified by the TOF camera 308 installed in the showcase 309 and provided to the tracking device 100.

When the passing gate 315 detects (in real time) that a customer has notified the passing gate 315, it notifies the tracking device 100 via a LAN or the like. When a customer with unpaid merchandise passes through the passing gate 315, the tracking device 100 notifies the store manager device (wearable device) 316. At this time, the tracking device 100 uses a timestamp to associate the time with the video captured by the surveillance camera 317 (surveillance video that includes at least the area around the store entrance/exit 302). In other words, it associates the customer attempting to take unpaid merchandise out of the store with the video in which that customer appears. The surveillance video of the inside of the store 300 captured by the surveillance camera 317 is transmitted to and stored in a data server (not shown) via a LAN or the like (hereinafter referred to as the "surveillance camera video recording server").

Note that communication between the devices shown in Figure 3 and the tracking device 100 is not limited to LAN, and various other communication methods may also be used.

FIG. 4 is a block diagram showing an example of the configuration of the tracking device 100.
4, a CPU 201 performs various controls by loading programs stored in a ROM 203 or an external memory 212, etc., into a RAM 202 as needed and executing the programs. The RAM 202 is used as a work area for the CPU 201. The ROM 203 stores various programs and various data.

The input controller 205 controls input from a keyboard (KB) 209 and a pointing device (mouse, touch panel, etc.) not shown. The video controller 206 controls display on the display 211. Note that in the tracking device 100, the camera device 210 does not have to be connected to the input controller 205. Furthermore, the keyboard 209 and display 211 do not have to be provided in the tracking device 100.

The memory controller 207 controls access to an external memory 212 such as a hard disk (HD), a solid state drive (SSD), or a flash memory.
The communication I/F controller 208 controls communication with a wired or wireless network.
The system bus 204 connects the devices 201 to 208 together.

The hardware configuration of the product payment terminal 312 is basically the same as that of the tracking device 100 , but a two-dimensional code reader 311 is connected to the input controller 205 .
The hardware configuration of customer terminal 313 is also basically the same as that of tracking device 100, but communication I/F controller 208 is realized by, for example, a mobile phone line of a mobile communication system or the like and a communication module of a wireless LAN or the like. Customer terminal 313 can be realized by a so-called smartphone or the like.

The TOF cameras 301 and 308 can also be basically realized with the same hardware configuration as the tracking device 100, but instead of the keyboard 209, display 211, etc., they are equipped with the above-mentioned TOF camera device as the camera device 210. The TOF cameras 301 and 308 do not necessarily have to be equipped with the video controller 206 or display 211. The TOF camera 308 performs processes such as identifying the type and number of products that a customer has removed from the showcase 309, as described above, by having the CPU 201 of a control board or the like connected to the camera device 210 execute a program stored in the ROM 203.

The surveillance camera 317 can also be realized with a similar configuration, but instead of the keyboard 209, display 211, etc., it has a camera device 210 that captures visible images. Furthermore, the surveillance camera 317 does not necessarily have to have the video controller 206 or display 211.

The pass gate 315 can also be realized with a similar configuration, but includes a gate mechanism and the like.
The store manager device (wearable device) 316 can also be realized with a similar configuration.

The operation of the information processing system of this embodiment will be described below using a flowchart.
5 and 6 are flowcharts showing an example of the operation of the information processing system of this embodiment. In these figures, the operation of tracking device 100 is realized by CPU 201 of tracking device 100 loading a program stored in external memory 212 into RAM 202 as needed and executing it. Furthermore, the operation of TOF camera 308 is executed by the control unit provided in TOF camera 308 described above. Furthermore, the operation of payment terminal 312 is realized by the CPU of payment terminal 312 loading a program stored in external memory into RAM as needed and executing it. Note that the "customer's behavior within the store" written on the left side of FIG. 5 is written for ease of explanation and is not included in the operation of this system.

FIG. 7 is a diagram illustrating various tables used in the information processing system of this embodiment. Each table shown in FIG. 7 is stored, for example, in the external memory 212 of the tracking device 100.

As shown in FIG. 5, when a customer (visitor) opens the door of showcase 309, this action is detected by a sensor (not shown) provided in showcase 309. In response, the showcase number and information indicating that the door has been opened are notified to tracking device 100 via the LAN from a communication interface (not shown) provided in showcase 309 (S101).

When the tracking device 100 receives information from the showcase 309 indicating that the showcase number and the door have been opened, it determines whether the person in front of the showcase 309 corresponding to the showcase number is the person being tracked (S102). While the system is operating, the TOF camera 301 constantly transmits captured images (range images and color images) to the tracking device 100. The tracking device 100 uses this image to track customers, and manages the routes of those being tracked (hereinafter referred to as "tracked persons") and the number of people being tracked, using an in-store tracked person table such as that shown in Figure 7(a), for example.

If the tracking device 100 determines that the person in front of the showcase 309 corresponding to the showcase number is the person being tracked (Yes in S102), the process proceeds to S104. The person being tracked corresponds to a customer who has previously opened the doors of showcases A and B and whose tracking has begun, such as customer 305 shown in Fig. 2. The process of S104 will be described later.
On the other hand, if it is determined that the person in front of the showcase 309 corresponding to the showcase number is not the person being tracked (No in S102), the tracking device 100 proceeds to the process of S103.

In S103, the tracking device 100 starts tracking the untracked person in front of the showcase 309 corresponding to the showcase number (determines the person as the tracking target), and proceeds to S104.
In S104, the tracking device 100 updates the route management of the tracked person and the number of people being tracked. For example, it updates the tracked person and location information in the in-store tracked person table shown in FIG.

Furthermore, when a customer takes an item from the showcase 309, the TOF camera 308 installed in this showcase 309 identifies the type and number of items taken from the showcase and notifies the tracking device 100 of this together with the showcase number (S105).

In response to this notification, the tracking device 100 links the tracked person in front of the showcase 309 corresponding to the showcase number included in the notification with the product and quantity included in the notification (S106). Furthermore, in S107, the tracking device 100 updates the product information (information identifying the product, quantity) stored in the purchase list of all tracked people (for example, the on-hand product table shown in Figure 7(b)) based on the information linked in S106 above.

In addition, a predetermined application (e.g., a store-specific application) is launched on the customer terminal 313, and a QR code 314 with embedded customer information is displayed (S108). When the QR code 314 is read by the two-dimensional code reader 311 connected to the product payment terminal 312, the product payment terminal 312 transmits to the tracking device 100 the customer information (e.g., customer ID) read from the QR code 314 and information indicating that the QR code has been accepted (S109). Note that if there are multiple payment terminals 312, the payment terminal number, etc. for identifying the payment terminal 312 is also included in the transmitted information.

When the tracking device 100 receives information from the product payment terminal 312 indicating that it has accepted the customer information and QR code, it identifies the tracker in front of the product payment terminal 312 from among all trackers and links the tracker to the QR code information. For example, it registers the customer ID received from the product payment terminal 312 as the customer ID of the record corresponding to the identified tracker in the payment method table such as that shown in Figure 7(c). In addition, the tracking device 100 transmits product information (information identifying the product, quantity) stored in the purchase list of the identified tracker (e.g., Figure 7(b)) to the product payment terminal 312 (S110).

When the product payment terminal 312 receives product information (information identifying the product, quantity) from the tracking device 100, it displays a product purchase list based on the product information, for example, as shown in 1001 in Figure 8 (S111).

FIG. 8 is a diagram showing an example of a merchandise payment screen 1000 displayed on the display of the merchandise payment terminal 312 by the merchandise payment terminal 312.
The product payment screen 1000 displays a product purchase list 1001, a purchase total amount 1005, and payment method selection buttons 1002 to 1004. By selecting one of the payment method selection buttons 1002 to 1004, the customer can make payment using the selected payment method.

When the product payment terminal 312 accepts the customer's selection of a product payment method (S112), it performs the product payment process using the selected payment method (S113) and sends payment completion information (including customer information and payment method) to the tracking device 100.

When the tracking device 100 receives payment completion information from the product payment terminal 312, it registers the payment method included in the payment information as the payment method for the tracked person whose payment has been completed in, for example, a payment method table (e.g., Figure 7(c)) based on the payment completion information (S114). Furthermore, the tracking device 100 ends tracking of the tracked person whose payment has been completed (removes them from the tracking target), and updates the route management and number of people being tracked for the latest tracked person (S104).

As described above, in the system of this embodiment, tracking of a customer does not begin when the customer simply enters the store; tracking of the customer begins when the customer opens the showcase door, and ends when payment is completed. This makes it possible to appropriately reduce the number of customers who need to be tracked. As a result, even when the number of customers increases and the store becomes crowded with many customers, it is possible to accurately track the movement of each customer as they move around the store.

In this flowchart, a configuration has been described in which tracking of a tracked person is terminated once payment has been completed. However, tracking of the tracked person may be terminated on the condition that the tracked person lines up in line at product payment terminal 312 (or passes in front of payment terminal 311). For example, a tracked person who lines up in line at payment terminal 311 may be determined to be a customer intending to make payment, and tracking of the tracked person may be terminated.
By doing this, the number of pursuers can be reduced more quickly.

In addition, if a customer whose tracking has begun after opening the showcase door closes the door without taking any products from the showcase, tracking of the customer whose tracking has begun may be terminated.
Furthermore, tracking may not start just by opening the showcase door, but may start when the showcase door is opened and an item is taken from the showcase.

In addition, this system is configured so that the product removed by the tracker from the showcase is read by a TOF sensor installed in the showcase and used at the time of payment. This eliminates the need for customers to manually scan the barcodes of purchased products into a payment terminal, which is a cumbersome process. This makes it possible for customers to easily make payments in unmanned stores, etc.

Also, as shown in FIG. 6, when the passing gate 315 detects that a customer is passing through the passing gate 315, it notifies the tracking device 100 of the passing information (e.g., the passing gate number and information indicating that the customer has passed through) (S201).

When the tracking device 100 receives passage information from the passage gate 315, it determines whether or not a person among the tracked people who is carrying a product has passed through the passage gate 315 corresponding to the passage information. If the tracking device 100 determines that a person who is not a tracked person or a tracked person who is not carrying a product has passed through the passage gate 315 (No in S202), it determines that the person left the store without purchasing a product, and ends the processing due to the passage information in S201 above. In this case, if the tracked person who is passing through the passage gate 315 is not carrying a product, the tracking device 100 removes that tracked person from the tracking targets.
On the other hand, if the tracking device 100 determines that the tracked person carrying the product has passed through the passing gate 315 (Yes in S202), it proceeds to S203.

In S203, the tracking device 100 determines whether the payment method stored in the payment method table (e.g., Figure 7(c)) of the tracked person passing through the gate with the above-mentioned product is "Null." Note that "Null" is stored as the payment method in the payment method table until the product payment is completed. In other words, a tracked person whose payment method in the payment method table is set to "Null" corresponds to a tracked person who has not yet completed the product payment.

If the payment method stored in the payment method table (e.g., FIG. 7(c)) of the tracked person passing through the gate with the above-mentioned product is not "Null" (No in S203), the tracking device 100 determines that the tracked person has left the store after paying, and ends the processing based on the passage information in S201. In this case, the tracking device 100 also removes the tracked person passing through the passage gate 315 from the tracking targets.
On the other hand, if the payment method stored in the payment method table (e.g., Figure 7 (c)) of the tracked person passing through the gate with the above-mentioned product is "Null" (Yes in S203), the tracking device 100 proceeds to S204.

In S204, the tracking device 100 notifies the store manager device 316 of the detection of an unpaid person (a tracked person passing through the gate with the above-mentioned product).
The store manager device 316, which has received the notification of the detection of the non-payment person from the tracking device 100, notifies the manager of the detection of the non-payment person by vibrating, sounding an alarm, displaying a warning, etc. (S205).

In addition, in S206, the tracking device 100 transmits the detection time of the unsettled person (the tracked person passing through the gate with the above-mentioned product) to the surveillance camera video recording server (not shown).
The surveillance camera video recording server receives the notification of the time when the unpaid customer was detected from the tracking device 100, and records the detection time as a timestamp linked to the time of the image capture (S207). This makes it possible to record image information of customers who pass through the gate with unpaid merchandise.

As described above, the system of this embodiment can associate a customer who is being tracked and is holding a product (i.e., an unpaid customer) with an image captured by a surveillance camera mounted on the ceiling or other location of the store when the customer passes through a gate at the store entrance/exit. This makes it possible to identify the customer who has taken an unpaid product outside the store.
In addition, by notifying the store manager when an unpaid customer passes through the gate, it becomes possible to deal with customers who are trying to take unpaid merchandise out of the store.

In the above embodiment, a configuration has been described in which a sensor installed in a showcase detects that the showcase door has been opened, and the opening of the showcase door serves as a trigger for starting tracking of a customer standing in front of the showcase. However, the trigger for starting tracking of a customer is not limited to the detection of the showcase door being opened. Any configuration may be used as long as it detects some kind of customer action (a specific action taken by the customer toward the showcase or a specific change in the showcase that occurs in response to that action) to remove an item from the showcase, and that detection serves as a trigger for starting tracking of the customer.

For example, a showcase may be provided with a button or the like for opening the door and a sensor for detecting the operation of the button or the like, and the sensor's detection of the button or the like may be used as a trigger for starting customer tracking. Alternatively, a sensor may be provided with a sensor for detecting a hand touching the showcase door, and the sensor's detection of the hand touching the door may be used as a trigger for starting customer tracking. Furthermore, the showcase may be provided with an infrared sensor or the like, and the detection of a customer's hand inserting (entering) the showcase or a commodity being taken out of the showcase may be used as a trigger for starting customer tracking. Alternatively, a weight sensor or the like for detecting (measuring) the weight of the commodities contained in the showcase may be provided, and the detection of a commodity being removed from the showcase and the weight sensor's detection of the commodity being lighter may be used as a trigger for starting customer tracking.
Furthermore, a TOF camera or the like may be provided in the showcase, and the detection of a commodity being taken out of the showcase may be used as a trigger to start tracking the customer.
Furthermore, when a door sensor for each showcase detects that the showcase door has been opened, and then a TOF camera for a showcase with the same identification number detects that an item has been taken out of the showcase, this may be used as a trigger to start tracking a customer when multiple conditions for starting tracking are met. In this way, by setting multiple conditions, it is possible to further narrow down the customers in the store to be tracked.

Furthermore, when a store employee is replenishing merchandise in a showcase, there is no need to track that employee. For this reason, the employee may wear a specific hat, uniform, or other clothing, or an identifier such as a sash or badge, and the system may be configured to not track people whose identifier is recognized from the video captured by the TOF camera 301. The TOF camera 301 outputs, for example, a color image along with a distance image. Therefore, the tracking device 100, which performs tracking processing using the output from the TOF camera, can easily determine the identifier of the employee and exclude them from tracking targets. This makes it possible to easily exclude people who do not need to be tracked, such as employees, from tracking targets, reducing the tracking load.

Furthermore, in this embodiment, store 300 is provided with entrance/exit 302, but entrances and exits may be separated. That is, one or more entrances that only allow entry into store 300 and one or more exits that only allow exit from store 300 may be provided, and no entrances that allow entry and exit from store 300 may be provided. In this configuration, the passage information is transmitted in S201 of FIG. 6 to passage gate 315 provided at the exit.
Furthermore, the store may be configured to detect a customer's exit from the image captured by the TOF camera 301 without providing the passage gate 315.

Furthermore, the functions of the tracking device 100 may be realized by a single computer or by multiple computers. For example, the functions of the tracking device 100 may be realized using a cloud service or the like.

The functions of the tracking device 100 may also be realized by a control unit provided in the TOF camera 301. That is, the control unit provided in the TOF camera 301 may perform the functions of the tracking device 100 described above, such as tracking people inside the store using video captured by the camera device provided in the TOF camera 301. In this configuration, the TOF camera 301 is configured to be able to communicate with each device in the information processing system, such as the TOF camera 308, showcase 309, store payment terminal 312, passing gate 315, and store manager device 316, via a LAN or the like.

As explained above, it is possible to appropriately reduce the number of customers who need to be tracked among customers in the store. As a result, even when the number of customers increases and the store becomes crowded with many customers, it is possible to accurately track the movement of each customer as they move around in the store. This makes it possible to prevent incorrect tracking due to switching of tracked customers.
Furthermore, by controlling the start of customer tracking using a sensor that can detect the opening and closing of the door on the showcase, it is possible to prevent the system from increasing in cost and realize a system with an inexpensive configuration that appropriately reduces the number of people who need to be tracked.

It goes without saying that the configurations and contents of the various data described above are not limited to those described above, and that the data may be configured in various configurations and contents depending on the application and purpose.
Although one embodiment has been described above, the present invention can be embodied as, for example, a system, an apparatus, a method, a program, a storage medium, etc. Specifically, the present invention may be applied to a system made up of multiple devices, or may be applied to an apparatus made up of a single device.
Furthermore, the present invention also includes any combination of the above embodiments.

(Other embodiments)
The present invention can also be realized by supplying a program that realizes one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., an ASIC) that realizes one or more of the functions.
The present invention may be applied to a system made up of a plurality of devices, or to an apparatus made up of a single device.
The present invention is not limited to the above-described embodiments, and various modifications (including organic combinations of the embodiments) are possible based on the spirit of the present invention, and these modifications are not excluded from the scope of the present invention. In other words, all configurations that combine the above-described embodiments and their modifications are included in the present invention.

100 Tracking device 300 Store 301 TOF camera (installed on the ceiling)
308 TOF camera (installed in the showcase)
309 Showcase 312 Product payment terminal 315 Passing gate 317 Surveillance camera

Claims (17)

a tracking means for performing a person tracking process based on an image captured by an image capturing means for capturing images of the inside of the store, and for controlling the tracking process to end when the person being the target of the tracking process makes a payment for a product;
A control means for managing the number of people who are the targets of the tracking process and controlling the number of people to be reduced in response to a product payment being made by a person who is the target of the tracking process;
An information processing system comprising: The information processing system described in claim 1, characterized in that the tracking means does not target a person as a tracking target if a predetermined characteristic is recognized in the person. The information processing system described in claim 2, wherein the case where a predetermined characteristic is recognized in the person is the case where the person is recognized as wearing a predetermined identifier. The information processing system of any one of claims 1 to 3, characterized in that it has a notification means for sending a notification to a predetermined notification destination when it is recognized that the person being tracked is about to pass through the store exit when the person being tracked has not yet made a purchase payment. The information processing system described in claim 4, characterized in that the notification destination is a wearable device worn by the store manager. An information processing system as described in any one of claims 1 to 5, characterized in that it includes a timestamp means that, when the tracking target person has not yet paid for a product and it is recognized that the tracking target person is about to pass through the store's exit, it is possible to associate the time of the person's attempt to pass through with an image taken inside the store. The information processing system described in any one of claims 1 to 6, characterized in that the tracking means controls the person tracking process to end when it acquires information indicating the completion of product payment. The method further includes acquiring means for acquiring information indicating completion of the product payment,
The information processing system according to claim 7, characterized in that the acquisition means acquires information indicating that the product payment has been completed on the condition that the person is recognized as being in front of the payment terminal performing the product payment. The information processing system described in any one of claims 1 to 8, characterized in that the tracking process includes route management. 10. The information processing system according to claim 1, wherein the tracking means controls the photographing means so as to track a person in the store. The control means stores information related to the person who is the target of the tracking process in a storage unit,
An information processing system as described in any one of claims 1 to 10, characterized in that when a product payment is made by a person who is the subject of the tracking processing, information related to that person is deleted from the memory unit. A detection means is provided for detecting an action of removing a product from a display area,
12. The information processing system according to claim 1, wherein the tracking process is initiated in response to the detection of the behavior by the detection means. The information processing system described in claim 12, characterized in that the tracking process is not initiated if the behavior of an individual attempting to remove an item from the display area is not detected, even if the individual is recognized as having entered the store. 14. The information processing system according to claim 12, wherein the detection means detects the behavior based on an image captured by a photographing means provided in the display location. the display area has a door, and by opening the door, the products displayed in the display area can be taken out;
The information processing system according to claim 12 or 13, characterized in that the detection means detects an action of attempting to remove a product from the display area by detecting at least one of the following: the door being opened, a person touching the door, or a button for opening the door being operated. The tracking means of the information processing system is
a tracking step of performing a person tracking process based on an image captured by an image capturing means for capturing images of the inside of the store, and controlling to end the tracking process of the person when the person being the target of the tracking process makes a payment for a product;
A control step in which a control means of the information processing system manages the number of people who are the targets of the tracking process, and controls to reduce the number of people in response to a product payment being made by the person who is the target of the tracking process;
An information processing method comprising: A program for causing at least one computer to function as each means of the information processing system described in any one of claims 1 to 15.