WO2016052229A1 - Information processing device, information processing method, and program - Google Patents

Abstract

This information processing device (10) is provided with an object detecting unit (110) that detects objects inside a first specified area in the vicinity of a merchandise payment device, and a movement detecting unit (120) that detects if an object detected in the first specified area did not move to a second specified area in the vicinity of the merchandise settling device or if the object detected in the first specified area moved to an area other than the second specified area.

Description

Information processing apparatus, information processing method, and program

The present invention relates to an information processing apparatus, an information processing method, and a program.

In the store, a merchandise settlement apparatus (a so-called self POS (Point of） Sale) terminal) in which a customer himself performs a settlement operation (for example, reading a barcode attached to a commodity to be settled) is installed. According to such a product checkout apparatus, it is possible to expect an effect such as cost reduction for the personnel involved in checkout work. On the other hand, since the customer does not face the store clerk directly in the product checkout apparatus as described above, there is a problem that the risk of fraud in the product checkout process increases.

An example of a technique for preventing fraud is disclosed in, for example, Patent Document 1 below. The self-POS disclosed in Patent Literature 1 monitors that a barcode reading operation by a customer is properly performed.

JP 2009-289222 A

Since various patterns can be considered for cheating at the self-POS terminal, a technology for accurately preventing such various cheating is desired.

The present invention provides a technique for preventing fraudulent acts with high accuracy in a product checkout device such as a self-POS terminal.

According to the present invention,
Object detection means for detecting an object in a first predetermined area in the vicinity of the commodity checkout device;
The object detected in the first predetermined area has not moved to the second predetermined area in the vicinity of the commodity settlement apparatus, or the object detected in the first predetermined area is an area other than the second predetermined area. A movement detection means for detecting that the movement to
An information processing apparatus is provided.

According to the present invention,
Computer
Detect an object in the first predetermined area near the product checkout device,
The object detected in the first predetermined area has not moved to the second predetermined area in the vicinity of the commodity settlement apparatus, or the object detected in the first predetermined area is an area other than the second predetermined area. Detect that you moved to
An information processing method including the above is provided.

According to the present invention,
Computer
An object detection means for detecting an object in a first predetermined area in the vicinity of the commodity settlement apparatus;
The object detected in the first predetermined area has not moved to the second predetermined area in the vicinity of the commodity settlement apparatus, or the object detected in the first predetermined area is an area other than the second predetermined area. A movement detecting means for detecting that it has moved to
A program for functioning as a server is provided.

According to the present invention, it is possible to prevent fraudulent acts in a product checkout device such as a self POS terminal with higher accuracy.

The above-described object and other objects, features, and advantages will be further clarified by a preferred embodiment described below and the following drawings attached thereto.

It is a figure which shows notionally the process structure of the information processing apparatus in 1st Embodiment. It is a figure for demonstrating a self POS terminal. It is a figure which shows notionally the hardware structural example of the information processing apparatus in 1st Embodiment. It is a flowchart which shows the flow of a process of the information processing apparatus in 1st Embodiment. It is a flowchart explaining the flow of a process in case a goods scan area is a 2nd predetermined area. It is a flowchart explaining the flow of a process when a goods scan area is a 1st predetermined area. It is a figure which shows notionally the process structure of the information processing apparatus in 3rd Embodiment. It is a flowchart explaining the flow of a process in case a goods scan area is a 2nd predetermined area. It is a flowchart explaining the flow of a process when a goods scan area is a 1st predetermined area. It is a figure which shows notionally the process structure of the information processing apparatus in 4th Embodiment. It is a flowchart which shows the flow of a process of the information processing apparatus in 4th Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same reference numerals are given to the same components, and the description will be omitted as appropriate.

[First Embodiment]
[Processing configuration]
FIG. 1 is a diagram conceptually showing the processing configuration of the information processing apparatus 10 in the first embodiment. The information processing apparatus 10 may be an apparatus that is communicably connected to a commodity settlement apparatus (hereinafter referred to as a self POS terminal, not shown), or may be a self POS terminal itself. As illustrated in FIG. 1, the information processing apparatus 10 according to the present embodiment includes an object detection unit 110 and a movement detection unit 120.

The object detection unit 110 detects an object in the first predetermined area near the self-POS terminal. For example, the object detection unit 110 can acquire an image including a first predetermined area from an imaging unit provided for monitoring the self-POS terminal, and can detect an object existing in the first predetermined area from the image. Further, the object detection unit 110 acquires, for example, a plurality of images having different imaging times from an imaging unit provided for monitoring of the self-POS terminal, and an object in the first predetermined area based on a difference between the plurality of images. May be configured to detect. The detection of an object from an image may be realized, for example, by matching with the feature information of the appearance of the object such as shape and color, or the recognition of the object itself such as a product type using a known image recognition process. May be realized.

Specifically, the object detection unit 110 can detect an object in the first predetermined area as follows. The object detection unit 110 holds, in advance, a parameter indicating a position corresponding to the first predetermined area on an image captured by each imaging unit provided for monitoring of the self-POS terminal. The object detection unit 110 can determine whether or not the detected object exists in the first predetermined area using this parameter. In addition, when a predetermined mark (for example, a unique color) is attached to the first predetermined area in the real space, the object detection unit 110 recognizes the predetermined mark, so that the first predetermined area is recognized. The first predetermined area can also be specified from an image obtained by imaging the area. Further, the object detection unit 110 may be configured to detect an object in the first predetermined area based on information that may indicate that some object exists in the first predetermined area without using an image. For example, product identification information acquired by reading a product with a self-POS terminal is information that can indicate that a certain object (product) exists in the product scan area.

In the present specification, the first predetermined area is a certain space in a three-dimensional space, and is an area including a movement start position of an object to be processed by the movement detection unit 120 described later. The first predetermined area is determined based on a series of operations that the customer performs at the time of product check-out at the self-POS terminal, for example, as shown below. Generally, a customer temporarily places a product (unregistered product) that is not registered as a payment target in an area (unregistered product area) for placing an unregistered product. Then, the customer takes out the unregistered product to be registered as the settlement target from the unregistered product area, moves it to the area (product scan area) for reading the product to be settled with the self-POS terminal, and reads the unregistered product into the self-POS terminal. Make it. Since the product read by the self-POS terminal is a product registered as a payment target (registered product), the customer moves the registered product to the registered product area. The first predetermined area is set to the unregistered product area in a scene where the unregistered product is moved to the product scan area. Further, in a scene where the registered product is moved to the registered product area, the first predetermined area is set as the product scan area.

The movement detection unit 120 detects that the object detected in the first predetermined area has not moved to the second predetermined area in the vicinity of the self-POS terminal, or the object detected in the first predetermined area is the second predetermined area. Detects movement to other areas.

The movement detection unit 120 determines, for example, whether or not the object detected in the first predetermined area has moved to the second predetermined area within a predetermined time after being moved outside the first predetermined area. In addition, when an area other than the second predetermined area is defined in advance, the movement detection unit 120 may determine whether or not an object detected in the first predetermined area has moved to the defined area. . In addition, the movement detection unit 120 estimates the moving direction of the object using, for example, a known algorithm for tracking the object detected on the image over a plurality of images, and monitors the movement destination of the object. Also good. In this case, for example, when the estimated moving direction deviates from the direction toward the second predetermined area by a predetermined threshold or more, the movement detection unit 120 determines that the object has been moved to another area other than the second predetermined area. I can judge. Further, it may be determined from the difference between the plurality of images where the object detected in the first predetermined area has moved. Specifically, the movement detection unit 120 determines that there is an object outside the first predetermined area from the difference between the images in the first predetermined area, and determines the first predetermined area from the difference between the images in the second predetermined area. It is determined whether the object moved outside has moved to the second predetermined area.

In the present specification, the second predetermined area is a certain space in the three-dimensional space, and is an area in which an object detected in the first predetermined area is estimated to move next in a legal accounting act. For example, if the first predetermined area is an unregistered merchandise area, an object (unregistered merchandise) detected in the area should move to the merchandise scan area in order to register as a merchandise to be settled in a legitimate accounting act It is. That is, when the first predetermined area is an unregistered product area, the second predetermined area is set as a product scan area. Further, when the first predetermined area is the product scan area, the object (registered product) detected in the area should move to the registered product area by a valid accounting act. That is, when the first predetermined area is a product scan area, the second predetermined area is set as a registered product area.

Specifically, the movement detection unit 120 can perform the above-described detection as follows. The movement detection unit 120 holds, in advance, a parameter indicating a position corresponding to the second predetermined area on the image with respect to an image captured by each imaging unit provided for monitoring the self-POS terminal. The movement detection unit 120 specifies the second predetermined area on the image using this parameter. In addition, when a predetermined mark (for example, a unique color) is attached to the second predetermined area in the real space, the movement detection unit 120 recognizes the predetermined mark, so that the first predetermined area is recognized. The second predetermined area can also be specified from an image obtained by imaging the area.

The self-POS terminal will be described with reference to FIG. FIG. 2 is a diagram for explaining the self-POS terminal. FIG. 2A shows the self-POS terminal 20 and its surroundings when viewed from the front. FIG. 2B shows the self-POS terminal 20 and its vicinity when viewed from above.

As shown in FIG. 2A, the self-POS terminal 20 places a product reading unit 22 for reading unregistered products to be registered as a checkout target, an unregistered product storage place 24 for placing unregistered products, and a registered product. At least a registered product storage place 26 for the purpose.

Further, as shown in FIG. 2A, at least one surveillance camera 30 (30a to 30f) is provided around the self-POS terminal 20. For example, when one surveillance camera 30 is provided, the angle of view of the surveillance camera 30 is such that the imaging range of the surveillance camera 30 includes an unregistered product area, a product scan area, and a registered product area. Adjusted. When a plurality of surveillance cameras 30 are provided, each surveillance camera is arranged such that an unregistered product area, a product scan area, and a registered product area are included in a range in which the imaging ranges of the surveillance cameras 30 are combined. The angle of view of 30 is adjusted. The monitoring camera 30 may be provided on a ceiling, a wall or the like (for example, 30a to 30c), or may be provided on the main body of the self-POS terminal 20 (for example, 30d to 30f). In addition, in the example of FIG. 2A, the product reading unit 22 is configured to be able to image the appearance of the product, and further functions as an imaging unit for monitoring as described as “30g”. It may be configured. The number of surveillance cameras 30 provided in the self-POS terminal 20 and their positions are not limited to the example of FIG.

FIG. 2B illustrates an imaging range of each monitoring camera 30 illustrated in FIG. A range A indicates an imaging range of the monitoring camera 30a. The monitoring camera 30a is provided on a ceiling, a wall, or the like, and images an area including the periphery of the self-POS terminal 20 from above. A range B indicates an imaging range of the monitoring camera 30b. The monitoring camera 30b is provided on a ceiling, a wall, or the like, and images the unregistered product storage area 24 (unregistered product area) of the self-POS terminal 20 from above. A range C indicates an imaging range of the monitoring camera 30c. The monitoring camera 30c is provided on a ceiling, a wall, or the like, and images the registered product storage place 26 (registered product area) of the self-POS terminal 20 from above. A range D indicates an imaging range of the monitoring camera 30d. The monitoring camera 30d is provided in the self POS terminal 20, and images the user of the self POS terminal 20 from the upper front. A range G indicates an imaging range of the monitoring camera 30g. The monitoring camera 30g is the product reading unit 22 of the self-POS terminal 20, and images a user holding the product. In FIG. 2B, the imaging ranges of the monitoring camera 30e and the monitoring camera 30f are not shown from the viewpoint of easy viewing. The monitoring camera 30e and the monitoring camera 30f mainly capture the user standing in front from the upper right portion and the upper left portion of the self-POS terminal 20, respectively.

[Hardware configuration]
FIG. 3 is a diagram conceptually illustrating a hardware configuration example of the information processing apparatus according to the first embodiment. As illustrated in FIG. 3, the information processing apparatus 10 includes a CPU (Central Processing Unit) 101, a memory 102, a storage 103, an input / output interface 104, and the like. The bus 105 is a data transmission path for transmitting / receiving data among the CPU 101, the memory 102, the storage 103, and the input / output interface 104. However, the method of connecting the CPUs 101 and the like is not limited to bus connection. The CPU 101 is an arithmetic processing device such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The memory 102 is a memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The storage 103 is a storage device such as a hard disk, an SSD (Solid State Drive), or a memory card. The storage 103 may be a memory such as a RAM or a ROM.

The input / output interface 104 is used for transmitting and receiving data between the information processing apparatus 10 and an external apparatus. For example, when the information processing apparatus 10 acquires an image near the self-POS terminal from an external apparatus (for example, an imaging apparatus such as a security camera), the information processing apparatus 10 is connected to the external apparatus via the input / output interface 104. Is done. There are various methods for connecting the information processing apparatus 10 and an external apparatus via the input / output interface 104. For example, this connection is a bus connection via a bus line (for example, a USB (Universal Serial Bus) line), a network connection via a network line, or the like. The network line may be a wireless line or a wired line.

The storage 103 stores a program for realizing the functions of the information processing apparatus 10. Specifically, program modules for realizing the functions of the object detection unit 110 and the movement detection unit 120 are stored. The CPU 101 implements the functions of the object detection unit 110 and the movement detection unit 120 by executing these program modules. Here, when executing the above modules, the CPU 101 may execute the modules after reading them onto the memory 102 or without reading them onto the memory 102.

Note that the hardware configuration of the information processing apparatus 10 is not limited to the configuration shown in FIG. For example, each program module may be stored in the memory 102. In this case, the information processing apparatus 10 may not include the storage 103. When the information processing apparatus 10 is a self POS terminal, a display device, an input / output device, a reading device, and the like are further connected via the input / output interface 104. The display device is a device that displays a screen corresponding to drawing data processed by the CPU 101, GPU (not shown), or the like, such as an LCD (Liquid Crystal Display) or CRT (Cathode Ray Tube) display. The input device is a device that receives an input by a user operation, and is realized as, for example, a hardware button unit, a touch sensor, or the like. The display device and the input device may be integrated and realized as a touch panel. The reading device is a camera including a lens, an image sensor, and the like, and captures a still image or a moving image of a product to be paid. The reading device may be a symbol reading device such as a barcode reader having a light source, a light receiving element, and the like. When the information processing apparatus 10 is a self POS terminal, at least one imaging apparatus may be further connected via the input / output interface 104. The imaging device is a monitoring camera that includes a lens, an imaging element, and the like and images the vicinity of the self-POS terminal.

[Operation example]
A processing flow of the information processing apparatus 10 according to this embodiment will be described with reference to FIG. FIG. 4 is a flowchart showing the flow of processing of the information processing apparatus 10 in the first embodiment.

The object detection unit 110 detects an object present in the first predetermined area (S101). The object detection unit 110 detects an object existing in the first predetermined area from the captured image of the imaging means provided for monitoring of the self-POS terminal, for example, by matching with the feature information of the appearance of the object such as shape and color. To do. Further, when there is a movement of an object in the first predetermined area, the object detection unit 110 can also detect an object in the first predetermined area based on a difference between a plurality of images having different imaging times.

The movement detection unit 120 determines the movement destination of the object detected by the object detection unit 110 (S102). Specifically, when the object detected in the first predetermined area moves outside the first predetermined area, the movement detection unit 120 determines whether the object has not been moved to the second predetermined area, or the object Is moved to another area other than the second predetermined area.

When the movement destination of the object detected by the moving object detection unit 110 is not the second predetermined area (S102: other than the second predetermined area), the movement detection unit 120 determines that the movement is different from the normal movement of the product, and determines the abnormality. A notification (detection notification) indicating that it has been detected is output to an external device such as a terminal for a supervisor (store clerk) (S103). The movement detection unit 120 is not limited to this, and warning information (a warning message or warning message) for the user of the self-POS terminal is displayed via a display unit (not shown) or a voice output unit (not shown) of the self-POS terminal. Warning sound or the like may be output.

On the other hand, when the movement destination of the object detected by the moving object detection unit 110 is the second predetermined area (S102: second predetermined area), the movement detection unit 120 determines that the product is moving normally and outputs a detection notification. do not do. Each process described above is repeated until it is determined that the merchandise payment process has been completed (S104). “End of merchandise checkout process” means a certain timing at the end of a series of actions in which a customer performs checkout of merchandise at a self-POS terminal. However, there may be a certain amount of time for the timing of “completion of merchandise settlement”. Examples of the timing include, for example, a timing when a customer presses a button for confirming each settlement target product read by the self-POS terminal, a receipt issuance timing, a bag containing a product purchased after paying the product price. An example is the timing of lifting from the table. The timing of “completion of the product checkout process” is not limited to these examples.

The information processing apparatus 10 performs the above-described processing independently at least one of the unregistered product area and the product scan area and / or the product scan area and the registered product area.

[Operation and Effect of First Embodiment]
As described above, the information processing apparatus 10 according to the present embodiment indicates that the object detected in the first predetermined area has not been moved to the second predetermined area, or that the object has moved to an area other than the second predetermined area. Detect and output detection notification and warning information. Here, the first predetermined area and the second predetermined area are determined based on a series of operations that the customer performs at the product check-out at the self-POS terminal. In particular, the second predetermined area indicates an area in which an object (product) detected in the first predetermined area moves next in a valid accounting act. In other words, the information processing apparatus 10 according to the present embodiment can detect a motion different from the motion normally performed in a legitimate merchandise check-out act. When this “movement different from the normal movement” is detected, there is a high possibility that the customer using the self-POS terminal is performing some kind of operation (injustice) different from the normal operation. Therefore, according to the present embodiment, in accordance with the detection result of the movement detection unit 120, the monitoring staff is notified of fraud at the self-POS terminal, or warning information is sent to the customer using the self-POS terminal. By notifying, it can be expected to prevent fraud accurately.

In this embodiment, when the first predetermined area is an unregistered product area and the second predetermined area is a product scan area, for example, a fraudulent act in which a customer puts a product into a bag or the like that is not registered as a checkout target Can be detected. In the present embodiment, when the first predetermined area is a product scan area and the second predetermined area is a registered product area, for example, a product registered as a settlement target by a customer is replaced with a different product that is not registered as a settlement target. Can be detected. The first predetermined area and the second predetermined area are not limited to these examples.

[Second Embodiment]
This embodiment has the same configuration as that of the first embodiment except for the following points.

[Processing configuration]
The information processing apparatus 10 according to this embodiment further uses the product identification information read by the product reading unit 22 of the self-POS terminal, and performs an operation as described below. Here, the product identification information is information for uniquely identifying the product (for example, information that can be read from a barcode or the like).

[Operation example]
The movement detection unit 120 according to the present embodiment executes processes as shown in FIGS. 5 and 6 depending on whether the product scan area is the first predetermined area or the second predetermined area. FIG. 5 is a flowchart for explaining the flow of processing when the product scan area is the second predetermined area. FIG. 6 is a flowchart for explaining the flow of processing when the product scan area is the first predetermined area.

<When the product scan area is the second predetermined area>
The flow of processing when the product scan area is the second predetermined area will be described with reference to FIG. In this case, the unregistered product area is the first predetermined area.

The object detection unit 110 detects an object in the unregistered product area (S201). Then, the movement detection unit 120 monitors whether or not the detected object has moved from the unregistered merchandise area (S202).

When the detected object has moved from the unregistered product area (S202: YES), the movement detection unit 120 activates a timer or the like, and time information indicating the elapsed time since the object has moved from the unregistered product area Are managed (S203). Further, the movement detection unit 120 monitors whether or not the product identification information has been read by the self-POS terminal (S204).

When the product identification information is not read by the self-POS terminal (S204: NO), the movement detection unit 120 passes a predetermined time after the object moves from the unregistered product area based on the managed time information. It is determined whether or not it has been done (S205). When a predetermined time has passed since the object moved from the unregistered product area (S205: YES), the movement detection unit 120 outputs a detection notification to, for example, a terminal for a monitor (S206). However, the movement detection unit 120 may output warning information to the self-POS terminal.

On the other hand, when the merchandise identification information is read by the self-POS terminal (S204: YES), the movement detection unit 120 determines that the movement normally performed in the legitimate merchandise settlement act is executed, and the processing target is the unregistered merchandise area. Move to other objects that exist in Note that the managed time information is reset in response to the product identification information being read in S204. Then, the above-described process is repeated until the merchandise settlement process at the self-POS terminal is completed (S207).

<When the product scan area is the first predetermined area>
With reference to FIG. 6, the flow of processing when the product scan area is the first predetermined area will be described. In this case, the registered product area is the second predetermined area.

The object detection unit 110 monitors whether or not the product identification information has been read by the self-POS terminal (S301). When the product identification information is read by the self-POS terminal (S301: YES), the object detection unit 110 detects that an object (product) exists in the product scan area, and activates a timer or the like to detect the object. Time information indicating the elapsed time from the beginning is managed (S302).

Then, the movement detection unit 120 monitors whether or not the detected object has moved from the product scan area to the registered product area (S303). The movement detection unit 120 determines whether an object has moved from the product scan area to the registered product area based on, for example, an image including the registered product area, based on whether any object has moved to the registered product area after reading the product identification information. It can be determined whether or not it has moved.

When the detected object has not moved from the registered product area from the product scan area (S303: NO), the movement detection unit 120 detects the object in the product scan area based on the managed time information. It is determined whether or not a predetermined time has passed (S304). When a predetermined time has elapsed since the object was detected (S304: YES), the movement detection unit 120 outputs a detection notification to, for example, a terminal for a supervisor (S305). However, the movement detection unit 120 may output warning information to the self-POS terminal.

On the other hand, when the detected object has moved from the product scan area to the registered product area (S303: YES), the movement detection unit 120 determines that a motion normally performed in a legitimate product checkout operation has been executed, and is subject to processing. Move to another object in the unregistered product area. At this time, the managed time information is reset. Then, the above-described processing is repeated until the merchandise settlement processing at the self-POS terminal is completed (S306).

[Operation and Effect of Second Embodiment]
As described above, in this embodiment, after a certain object moves from the unregistered product area, detection notification or warning information is output depending on whether or not the product identification information is read in the product scan area. Here, when the product identification information of a certain product is read after an object has moved from the unregistered product area, the product from which the product identification information has been read may have been moved from the unregistered product area. high. Thereby, according to this embodiment, it can judge more reliably that it moved to the goods scanning area from the unregistered goods area, and it can control that a detection notice and warning information are outputted accidentally.

In the present embodiment, whether or not the object has moved to the registered product area is determined according to whether or not the product identification information has been read. Here, when the product identification information of a certain product is read, the product is normally moved to the registered product area. Thereby, according to this embodiment, it can suppress that the process regarding the movement of the object between a goods scan area and a registered goods area is performed accidentally.

[Third Embodiment]
This embodiment has the same configuration as the other embodiments except for the following points.

[Processing configuration]
FIG. 7 is a diagram conceptually showing the processing configuration of the information processing apparatus 10 in the third embodiment. As illustrated in FIG. 7, the information processing apparatus 10 according to the present embodiment further includes a product recognition unit 130. As in the first embodiment, the warning output unit 130 is realized by the CPU 101 executing a program module for realizing the function of the warning output unit 130 stored in the storage 103.

[Operation example]
The product recognizing unit 130 executes processes as shown in FIGS. 8 and 9 depending on whether the product scan area is the first predetermined area or the second predetermined area. FIG. 8 is a flowchart for explaining the flow of processing when the product scan area is the second predetermined area. FIG. 9 is a flowchart for explaining the flow of processing when the product scan area is the first predetermined area.

<When the product scan area is the second predetermined area>
The flow of processing when the product scan area is the second predetermined area will be described with reference to FIG. In this case, the unregistered product area is the first predetermined area.

The object detection unit 110 detects an object in the unregistered product area (S401). At this time, the product recognition unit 130 recognizes which product is the object in the unregistered product area based on the image including the unregistered product area (S402). The product recognition unit 130 recognizes which product the object detected by the object detection unit 110 is, for example, by pattern matching with a feature amount of each product stored in advance. Further, the movement detection unit 120 monitors whether or not the recognized product has moved from the unregistered product area (S403).

When the recognized product has moved from the unregistered product area (S403: YES), the movement detection unit 120 activates a timer or the like to indicate the elapsed time since the product has moved from the unregistered product area. Information is managed (S404). Further, the movement detection unit 120 monitors whether or not the merchandise identification information has been read by the self-POS terminal (S405).

When the product identification information is not read by the self-POS terminal (S405: NO), the movement detection unit 120 performs predetermined processing after the recognized product moves from the unregistered product area based on the managed time information. It is determined whether time has passed (S406). When a predetermined time has elapsed since the recognized product has moved from the unregistered product area (S406: YES), the movement detection unit 120 outputs a detection notification to, for example, a terminal for a supervisor (S407). However, the movement detection unit 120 may output warning information to the self-POS terminal.

On the other hand, when the product identification information is read by the self-POS terminal (S405: YES), the movement detection unit 120 determines whether the recognized product and the product specified by the product identification information are the same. (S408). For example, the movement detection unit 120 refers to a storage unit that stores information related to each product (a product name, a feature amount indicating a color, a shape, or the like) in association with the product identification information using the read product identification information. Product can be specified.

If the recognized product and the product specified by the product identification information are not the same (S408: NO), the movement detection unit 120 determines that an illegal act such as replacement of the product has been performed, and sends a detection notification or warning information. It outputs (S407).

On the other hand, when the recognized product and the product specified by the product identification information are the same (S408: YES), the movement detection unit 120 determines that the movement normally performed in the legitimate product check-out is performed, The object of processing moves to another object existing in the unregistered product area. The managed time information is reset when it is determined in S405 that the recognized product and the product specified by the product identification information are the same. Then, the above-described process is repeated until the merchandise settlement process at the self-POS terminal is completed (S409).

<When the product scan area is the first predetermined area>
The flow of processing when the product scan area is the first predetermined area will be described with reference to FIG. In this case, the registered product area is the second predetermined area.

The object detection unit 110 monitors whether or not the product identification information has been read by the self-POS terminal (S501). When the product identification information is read by the self-POS terminal (S501: YES), the object detection unit 110 detects that an object (product) exists in the product scan area, and activates a timer or the like to detect the object. Time information indicating the elapsed time from the beginning is managed (S502).

Then, the movement detection unit 120 monitors whether or not the detected object has moved from the product scan area to the registered product area (S503). The movement detection unit 120 determines whether an object has moved from the product scan area to the registered product area based on, for example, an image including the registered product area, based on whether any object has moved to the registered product area after reading the product identification information. It can be determined whether or not it has moved.

When the detected object has not moved from the registered product area from the product scan area (S503: NO), the movement detection unit 120 detects an object in the product scan area based on the managed time information. It is determined whether or not a predetermined time has passed (S504). When a predetermined time has elapsed since the object was detected (S504: YES), the movement detection unit 120 outputs a detection notification to, for example, a terminal for a supervisor (S505). However, the movement detection unit 120 may output warning information to the self-POS terminal.

On the other hand, when the detected object moves from the product scan area to the registered product area (S503: YES), the product recognition unit 130 determines which product in the registered product area is based on the image of the registered product area. Is recognized (S506). The merchandise recognition unit 130 recognizes which merchandise the object in the registered merchandise area corresponds to, for example, by pattern matching with feature quantities of each merchandise stored in advance. The movement detection unit 120 determines whether the recognized product and the product specified by the product identification information are the same (S507). For example, the movement detection unit 120 refers to a storage unit that stores information related to each product (a product name, a feature amount indicating a color, a shape, or the like) in association with the product identification information using the read product identification information. Product can be specified.

If the recognized product and the product specified by the product identification information are not the same (S507: NO), the movement detection unit 120 determines that an illegal act such as replacement of the product has been performed, and sends a detection notification or warning information. It outputs (S505).

On the other hand, when the recognized product and the product specified by the product identification information are the same (S507: YES), the movement detection unit 120 determines that a movement normally performed in a legitimate product check-out is performed, The object of processing moves to another object existing in the unregistered product area. The managed time information is reset when it is determined in S506 that the recognized product and the product specified by the product identification information are the same. Then, the above-described processing is repeated until the merchandise settlement processing at the self-POS terminal is completed (S508).

[Operation and effect of the third embodiment]
As described above, in this embodiment, it is determined whether the product recognized from the images of the unregistered product area and the registered product area is the same as the product specified from the product identification information read in the product scan area. . Then, according to the determination result, detection notification and warning information are output. Accordingly, it is possible to more accurately detect that an illegal act such as replacement of goods has been performed, and to prevent an illegal act at the self-POS terminal with high accuracy.

[Fourth Embodiment]
This embodiment has the same configuration as that of the first embodiment except for the following points.

[Processing configuration]
FIG. 10 is a diagram conceptually showing the processing configuration of the information processing apparatus 10 in the fourth embodiment. As illustrated in FIG. 10, the information processing apparatus 10 according to the present embodiment further includes an appearance information acquisition unit 140. Similar to the first embodiment, the processing interruption unit 140 is realized by the CPU 101 executing a program module for realizing the function of the processing interruption unit 140 stored in the storage 103.

Appearance information acquisition unit 140 acquires appearance information of an object detected in the first predetermined area. The appearance information of an object is information indicating characteristics that can be grasped from the appearance of the object. The appearance information of the object is, for example, the shape of the object, the color of the object, the size of the object, and the like. However, the appearance information of the object is not limited to these examples.

The movement detection unit 120 of the present embodiment uses the appearance information acquired by the appearance information acquisition unit 140 to indicate that the object having the appearance information has not moved to the second predetermined area, or the object having the appearance information. Is detected to have moved to an area other than the second predetermined area.

The movement detection unit 120 according to the present embodiment performs the process of determining the movement destination of the object exemplified in the first embodiment on the object having the appearance information acquired by the appearance information acquisition unit 140.

[Operation example]
An example of the operation of the information processing apparatus 10 according to this embodiment will be described with reference to FIG. FIG. 11 is a flowchart showing the flow of processing of the information processing apparatus 10 in the fourth embodiment. In the following, processing different from the first embodiment will be mainly described.

The appearance information acquisition unit 140 acquires the appearance information of the object detected in S101 (S601). The appearance information acquisition unit 140 acquires the edge feature amount, the color feature amount, the size on the image, and the like of the detected object using a known image processing algorithm. Then, the movement detection unit 120 determines the movement destination of the object having the same appearance information as the appearance information acquired in S601 in the image obtained from each imaging unit provided for the self-POS terminal ( S602). Here, “identical” does not indicate only completely identical, but allows a certain amount of error.

[Operation and effect of the fourth embodiment]
As described above, in the present embodiment, when an object is detected in the first predetermined area, the movement destination of the object having the same appearance information as the appearance information of the object is monitored. Thereby, according to this embodiment, the precision which determines the moving destination of the object detected in the 1st predetermined area can be improved. For example, the accuracy in tracking an object over a plurality of images is improved. In addition, even when an object is in some kind of shadow in some of the plurality of images, the object can be identified from the subsequent image using appearance information.

In the plurality of flowcharts used in the above description, a plurality of steps (processes) are described in order, but the execution order of the steps executed in each embodiment is not limited to the description order. In each embodiment, the order of the illustrated steps can be changed within a range that does not hinder the contents. Moreover, each above-mentioned embodiment can be combined in the range in which the content does not conflict.

Hereinafter, examples of the reference form will be added.
1.
Object detection means for detecting an object in a first predetermined area in the vicinity of the commodity checkout device;
The object detected in the first predetermined area has not moved to the second predetermined area in the vicinity of the commodity settlement apparatus, or the object detected in the first predetermined area is an area other than the second predetermined area. A movement detection means for detecting that the movement to
An information processing apparatus comprising:
2.
The movement detecting means includes
Based on a plurality of images, estimating the moving direction of the object detected in the first predetermined area,
Based on the estimated moving direction, the object detected in the first predetermined area has not moved to the second predetermined area, or the object detected in the first predetermined area is the second predetermined area. Detects moving to an area other than the area,
1. The information processing apparatus described in 1.
3.
The first predetermined area is an unregistered product area in which an unregistered product that is not registered as a payment target product is placed,
The second predetermined area is a product scan area for scanning the unregistered product,
The movement detecting means includes
Based on the image including the unregistered product area, product identification is performed by the product checkout apparatus until a predetermined time elapses after an object detected in the unregistered product area moves outside the first predetermined area. Based on whether information is detected, the object detected in the unregistered product area has not moved to the product scan area, or the object detected in the unregistered product area is the product scan area. Detect that you have moved to an area other than
1. Or 2. The information processing apparatus described in 1.
4).
The first predetermined area is an unregistered product area in which an unregistered product that is not registered as a payment target product is placed,
The second predetermined area is a product scan area for scanning the unregistered product,
Further comprising product recognition means for recognizing products existing in the unregistered product area based on the image including the unregistered product area;
The movement detecting means includes
Based on whether the recognized product and the product specified by the product identification information detected by the product checkout device indicate the same product, the object detected in the unregistered product area is the product scan. Detecting that it has not moved to the area, or that the object detected in the unregistered product area has moved to an area other than the product scan area,
1. Or 2. The information processing apparatus described in 1.
5.
The first predetermined area is a product scan area that scans unregistered products that are not registered as products for payment.
The second predetermined area is a registered product area in which a registered product registered as a payment target product is placed,
The movement detecting means includes
Whether or not an object that has been moved to the registered product area is detected based on an image including the registered product area, after a predetermined time elapses after the product identification information is detected by the product checkout device The object detected in the product scan area did not move to the registered product area, or the object detected in the product scan area moved to an area other than the registered product area Detect
1. To 4. The information processing apparatus according to any one of the above.
6).
The first predetermined area is a product scan area that scans unregistered products that are not registered as products for payment.
The second predetermined area is a registered product area in which a registered product registered as a payment target product is placed,
Further comprising product recognition means for recognizing products existing in the registered product area based on the image including the registered product area;
The movement detecting means includes
Based on whether the recognized product and the product specified by the product identification information detected by the product checkout device indicate the same product, the object detected in the product scan area is the registered product. Detecting that it has not moved to the area, or that the object detected in the product scan area has moved to an area other than the registered product area,
1. To 4. The information processing apparatus according to any one of the above.
7).
Further comprising appearance information acquisition means for acquiring appearance information of an object detected in the first predetermined area based on an image including the first predetermined area;
The movement detecting means includes
Detecting that the object having the acquired appearance information has not moved to the second predetermined area, or that the object having the acquired appearance information has moved to an area other than the second predetermined area;
1. Or 2. The information processing apparatus described in 1.
8).
In accordance with the detection result of the movement detection means, further comprising warning output means for outputting warning information,
1. To 7. The information processing apparatus according to any one of the above.
9.
Computer
Detect an object in the first predetermined area near the product checkout device,
The object detected in the first predetermined area has not moved to the second predetermined area in the vicinity of the commodity settlement apparatus, or the object detected in the first predetermined area is an area other than the second predetermined area. Detect that you moved to
Information processing method including the above.
10.
The computer is
Based on a plurality of images, estimating the moving direction of the object detected in the first predetermined area,
Based on the estimated moving direction, the object detected in the first predetermined area has not moved to the second predetermined area, or the object detected in the first predetermined area is the second predetermined area. Detects moving to an area other than the area,
Including Information processing method described in 1.
11.
The first predetermined area is an unregistered product area in which an unregistered product that is not registered as a payment target product is placed,
The second predetermined area is a product scan area for scanning the unregistered product,
The computer is
Based on the image including the unregistered product area, product identification is performed by the product checkout apparatus until a predetermined time elapses after an object detected in the unregistered product area moves outside the first predetermined area. Based on whether information is detected, the object detected in the unregistered product area has not moved to the product scan area, or the object detected in the unregistered product area is the product scan area. Detect that you have moved to an area other than
Including Or 10. Information processing method described in 1.
12
The first predetermined area is an unregistered product area in which an unregistered product that is not registered as a payment target product is placed,
The second predetermined area is a product scan area for scanning the unregistered product,
The computer is
Based on the image including the unregistered product area, recognize the product existing in the unregistered product area,
Based on whether the recognized product and the product specified by the product identification information detected by the product checkout device indicate the same product, the object detected in the unregistered product area is the product scan. Detecting that it has not moved to the area, or that the object detected in the unregistered product area has moved to an area other than the product scan area,
Including Or 10. Information processing method described in 1.
13
The first predetermined area is a product scan area that scans unregistered products that are not registered as products for payment.
The second predetermined area is a registered product area in which a registered product registered as a payment target product is placed,
The computer is
Whether or not an object that has been moved to the registered product area is detected based on an image including the registered product area, after a predetermined time elapses after the product identification information is detected by the product checkout device The object detected in the product scan area did not move to the registered product area, or the object detected in the product scan area moved to an area other than the registered product area Detect
Including 13. The information processing method according to any one of 1 to 12.
14
The first predetermined area is a product scan area that scans unregistered products that are not registered as products for payment.
The second predetermined area is a registered product area in which a registered product registered as a payment target product is placed,
The computer is
Based on the image including the registered product area, recognize the product existing in the registered product area,
Based on whether the recognized product and the product specified by the product identification information detected by the product checkout device indicate the same product, the object detected in the product scan area is the registered product. Detecting that it has not moved to the area, or that the object detected in the product scan area has moved to an area other than the registered product area,
Including To 12. The information processing method as described in any one of these.
15.
The computer is
Based on the image including the first predetermined area, obtain appearance information of the object detected in the first predetermined area,
Detecting that the object having the acquired appearance information has not moved to the second predetermined area, or that the object having the acquired appearance information has moved to an area other than the second predetermined area;
Including Or 10. Information processing method described in 1.
16.
The computer is
Outputting warning information according to the detection result relating to the movement of the object;
Further includes: To 15. The information processing method as described in any one of these.
17.
Computer
An object detection means for detecting an object in a first predetermined area in the vicinity of the commodity settlement apparatus;
The object detected in the first predetermined area has not moved to the second predetermined area in the vicinity of the commodity settlement apparatus, or the object detected in the first predetermined area is an area other than the second predetermined area. A movement detecting means for detecting that it has moved to
Program to function as.
18.
The movement detecting means includes
Based on a plurality of images, estimating the moving direction of the object detected in the first predetermined area,
Based on the estimated moving direction, the object detected in the first predetermined area has not moved to the second predetermined area, or the object detected in the first predetermined area is the second predetermined area. Detects moving to an area other than the area,
17. The program described in.
19.
The first predetermined area is an unregistered product area in which an unregistered product that is not registered as a payment target product is placed,
The second predetermined area is a product scan area for scanning the unregistered product,
The movement detecting means includes
Based on the image including the unregistered product area, product identification is performed by the product checkout apparatus until a predetermined time elapses after an object detected in the unregistered product area moves outside the first predetermined area. Based on whether information is detected, the object detected in the unregistered product area has not moved to the product scan area, or the object detected in the unregistered product area is the product scan area. Detect that you have moved to an area other than
17. Or 18. The program described in.
20.
The first predetermined area is an unregistered product area in which an unregistered product that is not registered as a payment target product is placed,
The second predetermined area is a product scan area for scanning the unregistered product,
The computer,
Based on the image including the unregistered product area, further function as product recognition means for recognizing products existing in the unregistered product area,
The movement detecting means includes
Based on whether the recognized product and the product specified by the product identification information detected by the product checkout device indicate the same product, the object detected in the unregistered product area is the product scan. Detecting that it has not moved to the area, or that the object detected in the unregistered product area has moved to an area other than the product scan area,
17. Or 18. The program described in.
21.
The first predetermined area is a product scan area that scans unregistered products that are not registered as products for payment.
The second predetermined area is a registered product area in which a registered product registered as a payment target product is placed,
The movement detecting means includes
Whether or not an object that has been moved to the registered product area is detected based on an image including the registered product area, after a predetermined time elapses after the product identification information is detected by the product checkout device The object detected in the product scan area did not move to the registered product area, or the object detected in the product scan area moved to an area other than the registered product area Detect
17. To 20. The program as described in any one of these.
22.
The first predetermined area is a product scan area that scans unregistered products that are not registered as products for payment.
The second predetermined area is a registered product area in which a registered product registered as a payment target product is placed,
The computer,
Based on the image including the registered product area, further function as product recognition means for recognizing products existing in the registered product area,
The movement detecting means includes
Based on whether the recognized product and the product specified by the product identification information detected by the product checkout device indicate the same product, the object detected in the product scan area is the registered product. Detecting that it has not moved to the area, or that the object detected in the product scan area has moved to an area other than the registered product area,
17. To 20. The program as described in any one of these.
23.
The computer,
Based on an image including the first predetermined area, further function as appearance information acquisition means for acquiring appearance information of an object detected in the first predetermined area;
The movement detecting means includes
Detecting that the object having the acquired appearance information has not moved to the second predetermined area, or that the object having the acquired appearance information has moved to an area other than the second predetermined area;
17. Or 18. The program described in.
24.
The computer,
Warning output means for outputting warning information according to the detection result of the movement detection means;
For further functioning. To 23. The program as described in any one of these.

This application claims priority based on Japanese Patent Application No. 2014-199009 filed on September 29, 2014, the entire disclosure of which is incorporated herein.

Claims (10)

Object detection means for detecting an object in a first predetermined area in the vicinity of the commodity checkout device;
The object detected in the first predetermined area has not moved to the second predetermined area in the vicinity of the commodity settlement apparatus, or the object detected in the first predetermined area is an area other than the second predetermined area. A movement detection means for detecting that the movement to
An information processing apparatus comprising: The movement detecting means includes
Based on a plurality of images, estimating the moving direction of the object detected in the first predetermined area,
Based on the estimated moving direction, the object detected in the first predetermined area has not moved to the second predetermined area, or the object detected in the first predetermined area is the second predetermined area. Detects moving to an area other than the area,
The information processing apparatus according to claim 1. The first predetermined area is an unregistered product area in which an unregistered product that is not registered as a payment target product is placed,
The second predetermined area is a product scan area for scanning the unregistered product,
The movement detecting means includes
Based on the image including the unregistered product area, product identification is performed by the product checkout apparatus until a predetermined time elapses after an object detected in the unregistered product area moves outside the first predetermined area. Based on whether information is detected, the object detected in the unregistered product area has not moved to the product scan area, or the object detected in the unregistered product area is the product scan area. Detect that you have moved to an area other than
The information processing apparatus according to claim 1 or 2. The first predetermined area is an unregistered product area in which an unregistered product that is not registered as a payment target product is placed,
The second predetermined area is a product scan area for scanning the unregistered product,
Further comprising product recognition means for recognizing products existing in the unregistered product area based on the image including the unregistered product area;
The movement detecting means includes
Based on whether the recognized product and the product specified by the product identification information detected by the product checkout device indicate the same product, the object detected in the unregistered product area is the product scan. Detecting that it has not moved to the area, or that the object detected in the unregistered product area has moved to an area other than the product scan area,
The information processing apparatus according to claim 1 or 2. The first predetermined area is a product scan area that scans unregistered products that are not registered as products for payment.
The second predetermined area is a registered product area in which a registered product registered as a payment target product is placed,
The movement detecting means includes
Whether or not an object that has been moved to the registered product area is detected based on an image including the registered product area, after a predetermined time elapses after the product identification information is detected by the product checkout device The object detected in the product scan area did not move to the registered product area, or the object detected in the product scan area moved to an area other than the registered product area Detect
The information processing apparatus according to any one of claims 1 to 4. The first predetermined area is a product scan area that scans unregistered products that are not registered as products for payment.
The second predetermined area is a registered product area in which a registered product registered as a payment target product is placed,
Further comprising product recognition means for recognizing products existing in the registered product area based on the image including the registered product area;
The movement detecting means includes
Based on whether the recognized product and the product specified by the product identification information detected by the product checkout device indicate the same product, the object detected in the product scan area is the registered product. Detecting that it has not moved to an area, or that an object detected in the product scan area has moved to an area other than the registered product area;
The information processing apparatus according to any one of claims 1 to 4. Further comprising appearance information acquisition means for acquiring appearance information of an object detected in the first predetermined area based on an image including the first predetermined area;
The movement detecting means includes
Detecting that the object having the acquired appearance information has not moved to the second predetermined area, or that the object having the acquired appearance information has moved to an area other than the second predetermined area;
The information processing apparatus according to claim 1 or 2. In accordance with the detection result of the movement detection means, further comprising warning output means for outputting warning information,
The information processing apparatus according to any one of claims 1 to 7. Computer
Detect an object in the first predetermined area near the product checkout device,
The object detected in the first predetermined area has not moved to the second predetermined area in the vicinity of the commodity settlement apparatus, or the object detected in the first predetermined area is an area other than the second predetermined area. Detect that you moved to
Information processing method including the above. Computer
An object detection means for detecting an object in a first predetermined area in the vicinity of the commodity settlement apparatus;
The object detected in the first predetermined area has not moved to the second predetermined area in the vicinity of the commodity settlement apparatus, or the object detected in the first predetermined area is an area other than the second predetermined area. A movement detecting means for detecting that it has moved to
Program to function as.

Images