WO2018198376A1

WO2018198376A1 - Digital smart security system and method, and program

Info

Publication number: WO2018198376A1
Application number: PCT/JP2017/017096
Authority: WO
Inventors: 和夫三輪
Original assignee: 株式会社テクノミライ
Priority date: 2017-04-28
Filing date: 2017-04-28
Publication date: 2018-11-01
Also published as: WO2018198385A1; JP6195331B1; JPWO2018198376A1

Abstract

Provided is a digital smart security system and method, and a program which make it possible to prevent shoplifting even when the tag is removed from a product or the tag has been disabled. This digital smart security system (1000) is provided with: a tag movement determination unit (111) which, when it is detected that a product has been removed from a display shelf, determines whether or not said product has begun moving; a suspicious person determination unit (115) which, when a monitoring target P has stopped for a predetermined time or longer at a location other than a POS register (300), or when a tag composing the monitoring target P has become undetectable, registers an image of the face of the relevant customer as that of a suspicious person; and a face recognition function unit (116) which performs face recognition comparing the facial image of a person captured by a surveillance camera (11) and the facial image of a suspicious person registered in a face information DB (160).

Description

Digital smart security system, method and program

The present invention relates to a digital smart security system, method, and program for performing crime prevention by photographing a store with a surveillance camera.

Patent Document 1 states that “the presence or absence of a mismatch is determined based on a product existing in a store and a product confirmed to be sold by the POS server, and when it is determined that there is a mismatch, The face information of the store clerk photographed by the store closing camera before the start-up of this time is recorded in the face information recording unit, and the suspicious person is specified.

Japanese Patent Application Laid-Open No. 2004-228867 describes that “when the adjusted imaging data from the adjustment imaging device and the moving imaging data from the commodity shelf imaging device match, the adjusted imaging data and the moving imaging data that are matched from the recording device are deleted.” It is described.

Patent Document 3 states that “the face information acquired by the face detection unit from the image captured by the camera in the vicinity of the timing at which the gate detects the tag is captured before the timing by the monitoring camera. In other words, an image including a face that is determined to be the same as the face represented by the face information is searched.

Patent Document 4 states that “It is activated by an artificial external command after closing a store, and a shoplifter's visual verification screen is automatically generated. The video taken during business hours is read from the customer review site video file unit 510. Originally, the customer video that settled the price of the product and the video of the shoplifter are included, but the video related to the product that was cashed out at this point is deleted and only the video equivalent to the shoplifting site video is displayed. Has been.

JP 2013-134644 A JP 2012-242912 A JP 2011-233133 A JP 2008-257487 A

In such a conventional security system, shoplifting is prevented by attaching a tag to a product in a store such as a convenience store. However, shoplifting cannot be prevented if the tag is removed from the product or the tag is wrapped in an electromagnetic shield sheet.

An object of the present invention is to provide a digital smart security system, method, and program capable of preventing shoplifting even when a tag is removed from a product or the tag is invalidated.

The digital smart security system according to the present invention includes a customer movement detecting means for detecting a customer moving with a tag-attached product, and the tag-attached product stops moving for a predetermined time except at a predetermined location. And tag abnormality determination means for determining that the tag is undetectable, and when the tag abnormality determination means determines that the tag has stopped moving or cannot be detected, an image of the face of the customer taken by the surveillance camera And a suspicious person registration means for registering as a suspicious person's face.

Further, the face authentication means for photographing the face of the customer by the monitoring camera, and authenticating whether or not the photographed customer's face matches the suspicious person's face registered in the suspicious person registration means, By providing a control means for reporting when the face authentication means recognizes the match, it is possible to immediately determine the suspicious person by performing face recognition after registration.

Further, the control means transmits information including an image of the suspicious person's face to a portable terminal device carried by the employee closest to the suspicious person, so that the clerk closest to the position of the suspicious person (employee) Can be rushed to the scene in real time.

Further, the digital smart security method of the present invention includes a step of detecting customer movement for detecting a customer who moves together with a tagged product, and the movement of the tagged product is stopped for a predetermined time at a location other than a predetermined location. And a tag abnormality determination step for determining that the tag is undetectable, and a face of the customer photographed by the monitoring camera when the tag abnormality determination step determines that the tag is stopped or undetectable. And a suspicious person registration step of registering the image as a suspicious person's face.

Further, the present invention provides a customer movement detecting means for detecting a customer who moves with a product with a tag, and stopping the movement of the computer with the tag for a predetermined time at a location other than a predetermined location; Tag abnormality determining means for determining that the tag cannot be detected, and when the tag abnormality determining means determines that the tag is stopped or undetectable, an image of the face of the customer taken by the monitoring camera is displayed. A program for causing a digital smart security system to function as a suspicious person registration means for registering as a suspicious person's face.

According to the present invention, it is possible to provide a digital smart security system, method and program capable of preventing shoplifting even when the tag is removed from the product or the tag is invalidated.

It is a block diagram which shows the structure of the digital smart security system which concerns on embodiment of this invention. It is a block diagram of the control part of the monitoring apparatus of the digital smart security system which concerns on embodiment of this invention. It is a flowchart which shows the registration process of a suspicious person's face of the control part of the monitoring apparatus of the digital smart security system which concerns on embodiment of this invention. It is a flowchart which shows the crime prevention control process of the control part of the monitoring apparatus of the digital smart security system which concerns on embodiment of this invention. It is a figure explaining the example of application of the digital smart security system concerning an embodiment of the invention.

DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings.
(Embodiment)
FIG. 1 is a block diagram showing a configuration of a digital smart security system according to an embodiment of the present invention.

This system can be used in department stores, supermarkets, convenience stores, shopping malls, airports, airplanes, stations, trains, ships, restaurants, underground malls, specialty stores, precious metal stores, luxury boutiques, complex facilities, offices, theaters, gathering venues, It can be applied to schools, hospitals, factories, financial institutions, government offices, hotels, museums, art galleries, stadiums, event venues, new product announcement venues, and skyscrapers.

As shown in FIG. 1, the digital smart security system 1000 includes a plurality of surveillance cameras 11 installed on the ceiling of a product sales floor and an RFID (Radio Frequency） IDentification) reader 20 installed on a product shelf of the product sales floor. An authentication tag 21 carried by a store clerk (employee) and authenticated by the RFID reader 20, a Wi-Fi (Wireless Fidelity) terminal (hereinafter referred to as Wi-Fi master) 30 installed in a business facility, an i-beacon ( iBeacon) (registered trademark) base unit 40, a portable terminal device 50 carried by a store clerk (employee), a monitoring device 100 for controlling the entire system, a POS (Point Of Sale) server 200, and a POS A cash register 300 and an EOS (Electronic Ordering System) 400 are provided.

The digital smart security system 1000 monitors the video from the surveillance camera 11 installed on the product sales floor, notifies the mobile terminal device 50 carried by the store clerk (employee) of the suspicious person, and stores the clerk (employee). ) Is a system that prevents shoplifting by suspicious individuals.

<Monitoring camera 11>
Part or all of the surveillance camera 11 is a PTZ camera having a PTZ (pan / tilt / zoom) function, and is remotely operated by the surveillance device 100. Surveillance cameras 11 are installed at various places on the product sales floor such as in a store and take pictures of a person's face. An image captured by the monitoring camera 11 is output to the monitoring device 100. The surveillance camera 11 may shoot a moving image at all times, or may shoot a still image every certain time (for example, several seconds).

A plurality of surveillance cameras 11 are installed in a store or the like, and a predetermined surveillance area can be photographed. In addition, there is one that causes a specific person to be tracked by the monitoring camera 11. Even if the surveillance camera does not track a specific person, a plurality of surveillance cameras 11 are installed, so that a person or the like can be photographed from various angles.

<RFID reader 20>
RFID readers 20 are installed at predetermined intervals on the display shelf of the store. The RFID reader 20 detects that the product attached with the RF tag has left the display shelf depending on whether or not there is a response to transmission to the RF tag attached to the product.

In addition, the RFID reader 20 installed in the POS cash register 300 detects whether there is a response to the transmission to the RF tag attached to the product, thereby managing the settlement of the product with the RF tag attached. To do. The RFID reader 20 is an RFID tag compatible handy terminal or reader / writer. The communicable distance with the RFID tag is, for example, 3 to 7 m. Moreover, the specification which makes a communicable distance about 400 m is also possible by using the active tag which incorporates a battery.

The RFID reader 20 can also be installed in a shopping cart. In this case, the I / F unit 180 is connected wirelessly. The position of the RFID reader 20 installed in the shopping cart is detected and tracked by the monitoring device 100.

The RF tag is removed or invalidated from an article that can be properly taken out of the store by purchasing it.

In this embodiment, the RFID reader 20 authenticates the authentication tag 21 possessed by the store clerk (employee). The RFID reader 20 captures the position of the store clerk (employee) carrying the authentication tag 21 by authenticating the authentication tag 21 possessed by the store clerk (employee).

<Wi-Fi master unit 30>
The Wi-Fi master device 30 exchanges information with the Wi-Fi slave device 51 of the mobile terminal device 50 using Wi-Fi. In addition, the Wi-Fi master device 30 can acquire position information acquisition by Wi-Fi positioning, that is, position information acquisition using a Wi-Fi access point and a predetermined position information service.

<I-beacon base unit 40>
iBeacon is a wireless technology that uses Bluetooth Low Energy. An i-beacon is a combination of an i-beacon parent device 40 that is a beacon device on the transmission side and an application (corresponding to an i-beacon child device 52 described later) of the mobile terminal device 50 that supports radio wave reception from the i-beacon parent device 40. Composed. The i-beacon transmits unique ID information necessary for identification, and reacts only to an application associated with the ID information of the mobile terminal device 50. The application of the portable terminal device 50 registers the same identifier as that of the i-beacon parent device 40. The application (i-beacon handset 52) of the mobile terminal device 50 stands by in the background by executing an application equipped with the i-beacon function, and excites a predetermined action when approaching the beacon of the i-beacon base unit 40. By using the i-beacon, it is possible to accurately measure position information such as “position” and “direction” of the mobile terminal device 50 from Wi-Fi or GPS.

<POS server 200>
The POS server 200 collects and analyzes information on the best-selling products based on sales information at the time of sale of the POS system.

The POS server 200 records and holds information on the time of sale (sales time) of the product registered by the POS register 20 as POS data. The POS server 200 varies depending on the corresponding POS system and model, etc., but records and manages functions (sales management function) of product sales performance information (sales information), records and manages merchandise inventory information (stock information), etc. Function (inventory management function), and the function (attribute management function) of inputting and recording the purchaser's attribute information (gender, age group, etc.) by the store clerk.

The POS server 200 may be an information processing system including a store computer, a network communication device, and the like, or is connected and linked to an external inventory management system or an electronic money system (for example, a system using an IC card). It may be a system. For example, point-of-sale information may be recorded as POS data in accordance with accounting operations and processing using an IC card.

<POS cash register 300>
The POS cash register 300 is a cash register device having a function corresponding to the POS server 200. The POS register 300 includes a handy scanner and an input device (not shown). The handy scanner reads a product code attached to a product, that is, a bar code, and transmits it to the POS register 300. Also, key input data is input from this input device. The key input data is data by key button input. Similarly, a plurality of key input data are input according to the sales of a plurality of products.

<EOS400>
The EOS inputs the product category, type, product name, quantity, etc., and sends the order data to the manufacturer and wholesaler via EDI (Eietric Date Interchhange: electronic data exchange) to place an order. EDI is a mechanism that unifies information related to commerce into a standard format and exchanges it electronically between companies. Data related to ordering is digitized according to a predetermined format and sent and received via communication lines, etc. To do.

In addition, although the monitoring apparatus 100 is installed in the store, you may install outside via the network which is not shown in figure. When the monitoring device 100 is installed on a server on a network, a plurality of stores can be set as security targets.

[Portable terminal device 50]
The portable terminal device 50 is carried by each of a plurality of shop assistants (employees) in the store. The mobile terminal device 50 is, for example, a smartphone 50a, a tablet 50b, or a notebook computer 50c. In addition, the mobile terminal device 50 includes a mobile phone, PHS (Personal Handy-Phone System), PDA (Personal
Digital Assistants) or dedicated terminals. In the present embodiment, the mobile terminal device 50 can be used by each store clerk (employee) at various places (that is, the current position), and includes a mail or a video from the monitoring device 100 via a telephone line (not shown). Video and the like can be received.

In the present embodiment, the mobile terminal device 50 is assumed to be used by the smartphone 50a and the tablet 50b, and each individual can use the device at various locations (that is, the current location). One of the portable terminal devices 50 is arranged at a headquarter or a head office (not shown).

The mobile terminal device 50 includes a Wi-Fi individual identification device (hereinafter referred to as Wi-Fi slave device) 51, an i-beacon slave device 52, and a GPS (Global Positioning System) that captures the position of a store clerk (employee) on the product sales floor. 53).

<Wi-Fi handset 51>
The Wi-Fi slave device 51 receives and individually identifies the radio waves of the Wi-Fi master device 30 installed in the business facility. The monitoring device 100 stores arrangement information of the Wi-Fi master device 30 installed in the facility as safety related information. When the Wi-Fi slave device 51 comes close to the Wi-Fi master device 30, the ID and position of the person carrying the portable device 50 can be determined.

<I-beacon handset 52>
The Wi-Fi slave device 51 is an application of the mobile terminal device 50 that supports radio wave reception from the i-beacon master device 40. The i-beacon parent device 40 transmits a beacon (unique ID information necessary for identification), and the application of the mobile terminal device 50 (i-beacon child device 52) is predetermined when the beacon of the i-beacon parent device 40 approaches. Excites action. As described above, by using the i-beacon, the “position”, “direction”, and the like of the mobile terminal device 50 can be accurately transmitted to the monitoring device 100 from Wi-Fi or GPS.

<GPS53>
The GPS 53 receives a radio wave of position information from a GPS satellite or the like. The GPS 53 calculates the current position information from the information received via the GPS antenna as three parameters of latitude, longitude, and altitude, and acquires the position information. The acquired position information is transmitted to the monitoring device 100 in a timely manner.

In this embodiment, an example using GPS satellites as means for acquiring position information has been shown, but a system using a positional relationship with a base station other than GPS may be used. For example, when an Android (registered trademark) smartphone or a high-performance mobile phone with a camera is used as the mobile terminal device 50 that is a mobile terminal, a base station and a mobile phone communication network (not shown) are used instead of or in combination with the GPS 53. It is also possible to send / receive information to / from the mobile phone company server via the mobile phone and acquire the current location information of the terminal from the approach confirmation.

Further, position information acquisition by Wi-Fi positioning, that is, position information acquisition using a Wi-Fi access point and a predetermined position information service may be used.

[Monitoring device 100]
The monitoring device 100 is installed, for example, in a store management room, and centrally manages the product sales floor in the store. The monitoring device 100 may be a general server computer or a personal computer.

The monitoring apparatus 100 includes a control unit 110 (control means), an input unit 120, a storage unit 130, a display unit 140, an output unit 150, a face information database (DB) 153, an image processing unit 170, and an interface. An (I / F) unit 180 and a communication unit 190 are provided, and each unit is connected by a bus 195.

The monitoring device 100 monitors whether a suspicious person who performs shoplifting acts appears in a store.

When it is detected that the suspicious person has appeared in the store, the monitoring device 100 notifies the headquarters, the head office, or the security company system of the person information and the store information through the network.

Hereinafter, when the subject is described as “XX section”, it is assumed that the control section 110 reads each program from the ROM as necessary, loads it into the RAM, and executes each function (described later). Each program may be stored in the storage unit 130 in advance, or may be taken into the monitoring apparatus 100 when necessary via another storage medium or communication medium.

The control unit 110 is configured by a CPU (Central Processing Unit) or the like, and controls the entire monitoring apparatus 100 and executes a control program to function as a digital smart security system. The detailed configuration of the control unit 110 will be described later.

The input unit 120 is an input device for the user of the monitoring apparatus 100 to input instructions and the like, such as a keyboard, a mouse, a touch panel provided on the screen of the display unit 140, and a microphone.

The storage unit 130 stores still images or moving images received from the monitoring camera 11, various data and programs used by the control unit 110, and the like.

The display unit 140 displays an operation status of the monitoring apparatus 100, an image received from the monitoring camera 11, a GUI (Graphical User Interface) for operating the monitoring apparatus 100, and the like.

The output unit 150 is an audio interface, for example, and outputs an audio signal from the monitoring device 100 to the acoustic system 158 in the store. As an audio signal output from the monitoring apparatus 100 to the acoustic system 158, for example, an audio signal input from an audio input device such as a microphone provided in the input unit 120 or music data stored in the storage unit 130 is controlled. 110 may be a reproduced audio signal. The acoustic system 158 includes an amplifier and a plurality of speakers arranged in the store, and broadcasts an audio signal input from the monitoring device 100 into the store.

The face information DB 160 accumulates face images (face information) of suspicious persons and store personnel. Basic data such as a person's face registered in the face information DB 160 is obtained from the headquarters, head office, or security company (not shown) via the I / F unit 180, and the face information DB 160 is constructed. In addition, the face information DB can be updated to the latest face image (face information) by exchanging information with the headquarters, the head office or the security company (not shown).

The image processing unit 170 performs a predetermined process on the received image. Examples of the predetermined processing include contour extraction, image resizing, and resolution conversion processing.

The I / F unit 180 connects each monitoring camera 11 and the monitoring device 100 arranged in the product sales floor. In addition, the I / F unit 180 connects each RFID reader 20 arranged in the product sales floor, and monitors the result of authentication of the authentication tag 21 possessed by a store clerk (employee) close to the RFID reader 20 To 100. The I / F unit 180 is connected to a headquarter, head office, security company, or the like (not shown) via a network or a dedicated line. The security company is not an essential component of the digital smart security system according to the present embodiment.

The communication unit 190 transmits / receives data to / from the mobile terminal device 50 via the base station. In this Embodiment, the communication part 190 receives the positional information on each portable terminal device 50 for every fixed period via a base station, and at the time of recognition of VIP, a Kramer, or a shoplifting addict, VIP, a Kramer, or a shoplifting addict Send information about emails (emails, videos including videos, etc.).

[Control unit 110]
FIG. 2 is a block diagram of the control unit 110 of the monitoring device 100 of the digital smart security system according to the embodiment of the present invention.

As shown in FIG. 2, the control unit 110 includes a tag movement determination unit 111 (customer movement detection unit), a customer position information acquisition unit 112 (customer movement detection unit), and a monitoring target setting unit 113 (customer movement detection unit). A monitoring target tracking unit 114 (customer movement detection unit), a suspicious person determination unit 115 (tag abnormality determination unit, suspicious person registration unit), a face authentication function unit 116 (face authentication unit), and a position determination unit 117 ( A control unit), a suspicious person tracking unit 118 (control unit), and a transmission control unit 119 (control unit).

When the product with the RF tag is moved away from the display shelf and starts to move, the control unit 110 uses the monitoring camera 11 to move the product with the product and the customer's face image and the customer's scene image (still image or moving image). Is temporarily stored in the storage unit 130.

The tag movement determination unit 111 determines whether or not the tag (RF tag) attached to the product has moved away from the display shelf. RFID readers 20 are installed on the display shelf at predetermined intervals. The RFID reader 20 detects that a product to which an RF tag is attached (hereinafter referred to as a product as appropriate) has left the display shelf depending on whether or not there is a response to transmission to an RF tag attached to the product. When the RFID reader 20 detects that the product has left the display shelf, the RFID reader 20 transmits detection information to the control unit 110 of the monitoring apparatus 100 after a predetermined time (for example, 10 seconds) has elapsed.

When the tag movement determination unit 111 detects that the product has left the display shelf, the tag movement determination unit 111 determines whether the product has started moving. The start of the movement of the product is detected by another adjacent RFID reader 20 when the product leaves the display shelf.

The customer location information acquisition unit 112 acquires location information of customers who move with this product. The customer location information acquisition unit 112, for example, captures the surroundings of the corresponding display shelf with the monitoring camera 11, and acquires the customer location information by superimposing the customer captured by the monitoring camera 11 with the store layout information.

The monitoring target setting unit 113 sets the monitoring target P by combining the product with the RF tag and the customer position information. When a customer picks up a plurality of products, the monitoring target P is a combination of customer position information and a plurality of RF tags. Further, the store clerk (employee) is excluded from the customer and does not form the monitoring target P.

The monitoring target tracking unit 114 tracks the monitoring target P set by the monitoring target setting unit 113.

The tag movement determination unit 111, the customer position information acquisition unit 112, the monitoring target setting unit 113, and the monitoring target tracking unit 114, as a whole, take a product with a tag from the display shelf and move with the customer. The customer movement detection means to detect is comprised. The display shelf may be anything such as a display case or a product storage box as long as it displays or stores products.

The suspicious person determination unit 115 determines whether the monitoring target P has been stopped at a place other than the POS cash register 300 for a predetermined time or more, and whether the tag forming the monitoring target P has become undetectable. The suspicious person determination unit 115 displays the image of the customer's face when the monitoring target P stops for a predetermined time or longer in a place other than the POS register 300 or when the tag forming the monitoring target P becomes undetectable. Register as a thing.

Further, in the present embodiment, the suspicious person determination unit 115 does not perform the determination at the monitoring exclusion spot even if the monitoring target P is the location of the POS cash register 300.

The face authentication function unit 116 performs face authentication by comparing a person's face image captured by the monitoring camera 11 with a suspicious person's face image registered in the face information DB 160. The face authentication function unit 116 performs face authentication to determine whether the face of a person included in the image received from the monitoring camera 11 is a suspicious person's face registered in the face information DB 160. Face authentication can be performed based on the similarity between the face information extracted from the received image and the suspicious person's face information registered in the face information DB 160.

The position determination unit 117 determines the position of the mobile terminal device 50 closest to the suspicious person. The position determination unit 117 acquires the position information of the plurality of mobile terminal devices 50 in the product sales floor using the GPS position information and the RFID reader 20. Specifically, the position determination unit 117 obtains the position information of the store clerk (employee) on the product sales floor by authenticating the authentication tag carried by the store clerk (employee) by the plurality of RFID readers 20 installed on the product sales floor. get. In this case, the position determination unit 117 determines the position of the suspicious person against the installation position of the monitoring camera 11. That is, when the store clerk (employee) comes close to each RFID reader 20, the position determination unit 117 corresponds to the store clerk (employee) by authenticating each RFID reader 20 of the authentication tag 21 possessed by the store clerk (employee). It is determined that the RFID reader 20 is in proximity. Since the installation position of the RFID reader 20 is registered in advance, the position of the store clerk (employee) can be determined.

Further, the position determination unit 117 acquires position information from the GPS function unit included in the mobile terminal device 50.

The suspicious person tracking unit 118 tracks the movement of the suspicious person on the product sales floor. The suspicious person tracking unit 118 tracks and detects a person who is recognized as a suspicious person by the plurality of monitoring cameras 11 and confirms the movement history in the product sales floor. The suspicious person tracking unit 118 tracks the movement of the suspicious person on the product sales floor. When the suspicious person moves away from the position of the nearest mobile terminal device 50 by a predetermined distance, the suspicious person tracking unit 118 moves the suspicious person against the other mobile terminal devices 50. The information regarding the suspicious person including the face image is transmitted.

The transmission control unit 119 transmits information related to the suspicious person (an image or a moving image of the suspicious person's scene) to a store clerk (employee) close to the position of the suspicious person. For example, the transmission control unit 119 transmits voice, video, and suspicious person location information (for example, information marked on the sales floor layout) by e-mail. The transmission control unit 119 determines the position of the mobile terminal device 50 closest to the suspicious person, and transmits information related to the suspicious person to the mobile terminal device 50. If there is no answerback signal even after a predetermined time (for example, 5 seconds) has elapsed from the transmitting store clerk (employee), the transmission control unit 119 selects a second store clerk (employee) related person and performs retransmission. . The second clerk (employee) related person is the clerk second closest to the suspicious person.

The operation of the digital smart security system configured as described above will be described below.
[registration process]
First, the process of registering a suspicious person's face in the digital smart security system will be described.

FIG. 3 is a flowchart showing the suspicious person's face registration process of the control unit 110 of the monitoring device 100 of the digital smart security system. This flow is executed by the control unit 110 (see FIG. 2) of the monitoring apparatus 100.

In step S1, the tag movement determination unit 111 of the control unit 110 determines whether or not the tag (RF tag) attached to the product has started to move away from the display shelf. RFID readers 20 are installed on the display shelf at predetermined intervals. The RFID reader 20 detects that a product to which an RF tag is attached (hereinafter referred to as a product as appropriate) has left the display shelf depending on whether or not there is a response to transmission to an RF tag attached to the product. When the RFID reader 20 detects that the product has left the display shelf, the RFID reader 20 transmits detection information to the control unit 110 of the monitoring apparatus 100 after a predetermined time (for example, 10 seconds) has elapsed. When the tag movement determination unit 111 of the control unit 110 detects that the product has left the display shelf, the tag movement determination unit 111 determines whether the product has started to move. The start of the movement of the product is detected by another adjacent RFID reader 20 when the product leaves the display shelf.

If the tag has moved away from the display shelf but has not started moving (step S1: No), it is determined that the customer is picking up the product and the process of this flow is terminated.

When the product with the RF tag is moved away from the display shelf and starts moving (step S1: Yes), the customer location information acquisition unit 112 of the control unit 110 in step S2 obtains the location information of the customer moving with the product. get. The customer location information acquisition unit 112, for example, captures the surroundings of the corresponding display shelf with the monitoring camera 11, and acquires the customer location information by superimposing the customer captured by the monitoring camera 11 with the store layout information. If the customer is a suspicious person, it is considered a rare case (the shoplifter does not usually notify his / her location information). However, if a suspicious person posing as a customer does not know the existence of the digital smart security system, the Wi-Fi slave device 51 and the i-beacon slave device 52 may be carried. As described above, when position information from the Wi-Fi handset 51 or i-beacon handset 52 possessed by the customer can be acquired, the position information of the customer may be acquired from these pieces of information.

If the product with the RF tag is moved away from the display shelf and starts to move, in step S3, the control unit 110 causes the surveillance camera 11 to move with the product and the customer's face image and the customer's scene image. (Still image or moving image) is temporarily stored in the storage unit 130 (see FIG. 1). The monitoring camera 11 is a PTZ camera, for example, and is remotely operated by the monitoring device 100. The surveillance camera 11 is an authentication camera that photographs a person's face. An image captured by the monitoring camera 11 is output to the monitoring device 100.

In step S4, the monitoring target setting unit 113 of the control unit 110 sets the monitoring target P by combining the product with the RF tag and the customer position information. A customer may pick up multiple items. In this case, the monitoring target P is a combination of customer location information and a plurality of RF tags. Further, the store clerk (employee) is excluded from the customer and does not form the monitoring target P.

In step S5, the monitoring target tracking unit 114 of the control unit 110 tracks the monitoring target P set by the monitoring target setting unit 113.

In step S6, the monitoring target tracking unit 114 determines whether or not the monitoring target P has moved to the POS cash register 300, and when the monitoring target P has moved to the POS cash register 300 (step S6: Yes) (the arrow in FIG. 1) a), it is determined that the customer is not a suspicious person, and the processing of this flow is terminated.

When the monitoring target P has not moved to the POS register 300 (step S6: No) (see arrow b in FIG. 1), the process proceeds to step S7. If a customer (monitoring target P) having a tag attached to a product does not move to the POS register 300 for a long time, it is determined as a caution or abnormal state.

In step S7, the suspicious person determination unit 115 of the control unit 110 determines whether or not the monitoring target P has been stopped at a place other than the POS cash register 300 for a predetermined time (for example, 10 minutes or 1 hour). If the monitoring target P has not been stopped for a predetermined time or longer in a place other than the POS register 300 (step S7: No), the process returns to step S5 and the tracking of the monitoring target P is continued.

When the monitoring target P is stopped at a place other than the cash register for a predetermined time or more (step S7: Yes), the suspicious person determination unit 115 at step S8, the place is a fitting room or a tasting corner where the eyes of the store clerk (employee) reach. It is determined whether the spot is a spot excluded from monitoring. Being a monitoring exclusion spot is not limited to the fact that the monitoring target P has moved to the monitoring exclusion spot, but is a condition that there is a store clerk (employee) at the monitoring exclusion spot.

When the monitoring target P is in the monitoring exclusion spot (step S8: Yes), the process returns to step S8.

When the monitoring target P is not in the monitoring exclusion spot (step S8: No), the process proceeds to step S9.

In step S9, the suspicious person determination unit 115 determines whether or not the tag constituting the monitoring target P has become undetectable. When a suspicious person removes a tag from a product or covers the tag with an electromagnetic shield sheet, radio waves responding to the tag that has been detected until now cannot be detected. If it is not the location of the POS register 300 but the response of the tag that has been detected so far stops, it is determined as a caution or abnormal state.

If the RF tag forming the monitoring target P is not undetectable (step S9: No), the process returns to step S5 and the tracking of the monitoring target P is continued.

If the RF tag forming the monitoring target P cannot be detected (step S9: Yes), the process proceeds to step S10.

In step S <b> 10, the suspicious person determination unit 115 determines that the customer constituting the monitoring target P is suspected of being a suspicious person who performs shoplifting, product replacement, damage, and the like, and stores it in the storage unit 130. The customer's face image is read out and registered as a suspicious person's face, and this flow ends.

Here, when the RF tag is removed, even if it is determined that the RF tag has stopped moving for a predetermined time or more and an attempt is made to register a face image, the customer has already left the place when the determination is made. Yes. Therefore, when the product tagged with the RF tag starts moving away from the display shelf in the above step S3, or when the movement is stopped for the predetermined time or longer, the image of the customer's face is stored. If it is determined in step S10 that the person is a suspicious person, the stored face image of the customer is registered as a suspicious person's face image.
With this flow, the face image of the suspicious person is registered in the face information DB 160.

[Security control of monitoring device 100]
FIG. 4 is a flowchart showing a crime prevention process of the control unit 110 of the monitoring device 100 of the digital smart security system. This flow is mainly executed by the control unit 110 (see FIG. 4) of the monitoring apparatus 100.

First, in step S21, the control unit 110 captures the video of the monitoring camera 11. The surveillance camera 11 can shoot the entire store with a plurality of wide-angle lens cameras.

In step S22, the face authentication function unit 116 of the control unit 110 performs face authentication by comparing the face image of the person photographed by the monitoring camera 11 with the face image of the suspicious person registered in the face information DB 160.

In step S23, it is determined whether or not the face image of the person photographed by the monitoring camera 11 matches the face image of the suspicious person registered in the face information DB 160 by the face authentication. When the customer visits the store again, the face image of the person photographed by the monitoring camera 11 matches the face image of the suspicious person registered in the face information DB 160. In addition to the case of visiting the store again, for example, if the face image of a suspicious person is registered on a certain floor (registration by the registration process of the suspicious person's face in FIG. 3), Authentication is possible, and face authentication is also possible when a customer leaves the store. Thus, in the case of an unmanned store, the exit gate can be closed when a suspicious person tries to leave the store. When the face image of the person photographed by the monitoring camera 11 does not match the face image of the suspicious person registered in the face information DB 160 (step S23: No), the processing of this flow ends.

As described above, when the customer visits the store again, the face image of the person photographed by the monitoring camera 11 matches the face image of the suspicious person registered in the face information DB 160 (step S23: Yes), in step S24, the position determination unit 117 of the control unit 110 determines the position of the mobile terminal device 50 closest to the suspicious person.

In step S25, the suspicious person tracking unit 118 of the control unit 110 tracks the movement of the product sales floor of the suspicious person.

In step S <b> 26, the transmission control unit 119 of the control unit 110 transmits information about the suspicious person (an image of the suspicious person's scene or a moving image) to the clerk (employee) close to the position of the suspicious person, and ends this flow. To do. For example, the transmission control unit 119 transmits voice, video, and suspicious person location information (for example, information marked on the sales floor layout) by e-mail.

Here, it may not be appropriate for the store clerk (employee) to take the above measures at the sales floor. It is up to the store clerk (employee) to decide what kind of response the clerk (employee) will take to the suspicious person, but it is significant to provide information as a judgment material.

If the face authentication is executed when the customer (suspicious person) leaves the store, information on the suspicious person is transmitted to the store clerk (employee) when the customer leaves the store.

In addition, when the suspicious person is a shoplifting addict, information related to shoplifting addicts may be transmitted to the mobile terminal device 50 of the headquarters, the head office, or a security company in addition to the response to the store clerk (employee).

[Application example]
FIG. 5 is a diagram for explaining an application example of the digital smart security system, and shows a store visitor and a sales floor layout (commodity sales floor). FIG. 5 shows a sales floor of a store such as a convenience store.

In FIG. 5, the RFID reader 20 of FIG. 1 is the electronic tag reader of FIG. 5, and the electronic tag is an RF tag. The Wi-Fi slave device 51 of FIG. 1 is connected to the Wi-Fi master device 30 via the Wi-Fi repeater 51A installed for each fixed distance unit or sales department in FIG. For customers in the store, position information and customer face images are detected by an electronic tag, i-beacon, Wi-Fi, smartphone, surveillance camera 11, electronic tag gate terminal, and the like.

When a customer picks up an electronic tag product (a product with a tag) and moves with the product, the behavior of the customer is detected by the movement signal of the electronic tag product possessed by the customer. In the example of FIG. 5, the smartphone (i.e., the application of the portable terminal device 50) possessed by the customer receives the radio wave transmitted by the i-beacon parent device 40 installed in the upper part of the display case, and responds with the ID, whereby the i-beacon. The parent device acquires the location information of the smartphone. The Wi-Fi handset 51 of the smartphone (mobile terminal device 50) possessed by the customer is transferred to the Wi-Fi master 30 of the monitoring device 100 via the Wi-Fi repeater 51A installed in the store. It is input, and the sales location corner or display case where the visitor is located is detected. A surveillance camera 11 is installed in each area such as a sales department, a sales corner, a passage, and a cash register, detects radio waves transmitted by the electronic tag reader 20, and installs (or shoots) the surveillance camera 11. In combination with information, the customer's location is identified. Further, the radio wave transmitted by the electronic tag reader 20 is detected to identify the year, month, day, hour, and minute of the location. Furthermore, images taken by the monitoring camera 11 installed in the corresponding area can be extracted in chronological order, and it can be detected that the customer is in the hall in front of the sales corner or the display case.

When the customer enters the shopping basket and moves the electronic tag product, the electronic tag of the product responds to the radio wave transmitted by the electronic tag reader 20, and the electronic tag reader 20 can receive the responded radio wave. .

By the way, in the case of high-priced products sold at stores, an electronic tag gate 12 is installed before the entrance, and when it is not settled, a warning is issued and the store clerk rushes. Further, single item management, ordering, sales, etc. of the product can be handled daily by product management by the handy scanner 250, the POS cash register 300, and the EOS 400. As measures against shoplifting and merchandise loss in such store sales activities, a surveillance camera 11 and an electronic tag gate 12 are installed, and a human guardman is introduced.

However, the actual situation is that suspicious persons such as shoplifting acts are not registered only by monitoring with the surveillance camera 11 or the like, and even if a shopkeeper who is not registered carries out shoplifting actions, it is often not detected. In addition, it is a current situation that it is still not sufficient for a store-related person to register as a suspicious person such as a shoplifting act because it requires a lot of labor and there is a risk of misidentification.

Furthermore, there is a problem that shoplifting cannot be prevented if the electronic tag is removed from the product or the electronic tag is wrapped in an electromagnetic shield sheet.

For example, if the customer A in FIG. 5 takes out a product from a display case (display shelf), puts it in his / her clothes or bag, and moves out of the store, in the case of the conventional example, unless it is cleared at the POS cash register 300 An alarm is generated at the electronic tag gate 12. However, if a suspicious person such as a shoplifting act removes the electronic tag from the product or wraps the electronic tag in an electromagnetic shield sheet, the electronic charge is not settled at the POS cash register 300. The tag gate 12 cannot detect it and cannot prevent shoplifting. Since shoplifting cannot be prevented, the suspicious person repeats shoplifting. In particular, the problem is that the suspicious person has taken out the electronic tag from the display case (display shelf) for the purpose of shoplifting and has taken it away. This is because the electronic tag gate 12 cannot be detected. Therefore, a suspicious person will not be identified indefinitely.

On the other hand, the digital smart security system 1000 detects that the product with the electronic tag is taken from the display case and moves with the customer, and the product with the electronic tag is located at a place other than the POS register 300. It is determined that the electronic tag has stopped for a predetermined time, or has disappeared because the electronic tag cannot be detected. When this determination is made, an image of the customer's face is taken by the surveillance camera 11 and registered as a suspicious person's face. Thus, the suspicious person's face can be registered at the time when the shoplifting action is being formed. Then, when the suspicious person visits the store again, face authentication can be performed and the mobile terminal device 50 of the clerk (employee) can be notified.

As described above in detail, according to the present embodiment, when the digital smart security system 1000 (see FIG. 1) detects that a product has left the display shelf, whether or not the product has started moving. The tag movement determination unit 111 for determining whether or not, the customer position information acquisition unit 112 for acquiring the position information of the customer who moves with the product, and the product to which the RF tag is attached and the position information of the customer are combined to determine the monitoring target P When the monitoring target setting unit 113 to be set, the monitoring target tracking unit 114 that tracks the monitoring target P set by the monitoring target setting unit 113, and the monitoring target P are stopped at a place other than the POS register 300 for a predetermined time or more, Or when the tag which forms the monitoring object P becomes undetectable, the suspicious person determination unit 115 which registers the image of the customer's face as that of the suspicious person is provided. In addition, a face authentication function unit 116 that performs face authentication for comparing a face image of a person photographed by the monitoring camera 11 with a face image of a suspicious person registered in the face information DB 160, and a salesclerk (employee) close to the position of the suspicious person A transmission control unit 119 for transmitting information on the suspicious person.

This configuration allows the customer's face image to be registered as a suspicious person even when the tag is removed from the product or the tag is invalidated. When the customer (suspicious person) visits the store again, leaves the store, or moves from the product area registered in the image to another product area, the clerk (employee) close to the position of the suspicious person Information on the suspicious person such as the suspicious person's face image is transmitted. A nearby store clerk (employee) can rush to the site in real time.

Especially, since the video of the suspicious person's face is registered at the time when the shoplifting action is being formed, it is possible to immediately check the suspicious person by performing face recognition after registration. Future shoplifting by the suspicious person, which was difficult with the prior art, can be prevented.

In this embodiment, the monitoring target P is set by combining the product with the RF tag and the location information of the customer. Tags can be attached to any product. For this reason, by setting the monitoring target P, it is possible to monitor a number of products in a lump, and it is possible to construct a security system that can be easily adopted even in a store with a large number of products sold or sold.

The above description is an illustration of a preferred embodiment of the present invention, and the scope of the present invention is not limited to this. For example, any face authentication technique of the control unit 110 using the monitoring camera 11 may be used. Moreover, it can replace with detecting with the RFID reader | leader 20 that the goods left | separated from the display shelf, and the monitoring camera 11 can also be used. That is, it may be detected that the product has left the display shelf by detecting that the product disappears from the display shelf by the monitoring camera 11. In addition to image recognition, customer tracking includes electronic tags, i-beacons, Wi-Fi, etc., and these may be combined as in the application example (see FIG. 5). When a customer carries a mobile terminal device, the mobile terminal device is installed by installing Wi-Fi equipment in the store, and the mobile terminal device carried by the customer makes a Wi-Fi connection. The customer can be identified. Similarly, since the mobile terminal device can be identified by the equipment of the i-beacon, the customer carrying the mobile terminal device can be identified. In this way, when a customer is identified by the mobile terminal device, the position of the mobile terminal device can be detected by the response of the radio wave from the mobile terminal device, so that the customer carrying the mobile terminal device The position can be detected. To this end, suspicious behavior of customers (e.g. (1) .repetitively reciprocating round trips in specific areas, (2) .pausing unnaturally long before a cashier or display case (3). It can also be marked as a person to watch out for, such as moving at an unnatural speed in the store and (4). In addition, in the case where the facility for detecting the position of the customer by such a portable terminal device is provided, not only the suspicious person but also the position of a general customer (who carries the portable terminal device) is detected. Therefore, it is possible to grasp the distribution and movement of customers in the store accompanying the shoplifting prevention function. This information can be used for store clerk placement. Also, if a burglar comes in, the customers who have moved up to that point will stop moving all at once, so this can be detected and used for reporting a burglary. Furthermore, since the surveillance camera 11 is provided, it is possible to capture a suspicious appearance of a customer (for example, wearing a mask, sunglasses, or a lookout hat) and mark it as a person requiring attention. Further, since it is detected by the RFID reader 20 that the product to which the RF tag is attached has left the display shelf, if it is detected that the product has unnaturally separated from the display shelf in large quantities, A customer can be marked as a sensitive person. This function can also be realized by the monitoring camera 11. That is, if it is detected by the monitoring camera 11 that the product has disappeared from the display shelf, and it is detected that the product has unnaturally left a large amount, the customer who is present is marked as a person requiring attention. You can also.

In this embodiment, the name “digital smart security system and method” is used. However, this is for convenience of explanation, and may be a prevent system, a sales system, a security method, or the like.

The digital smart security system and method of the present invention can also be realized by a program for causing a computer to function as the digital smart security system or method. This program may be stored in a computer-readable recording medium.

The recording medium on which this program is recorded may be the ROM itself of the digital smart security system, or a program reading device such as a CD-ROM drive is provided as an external storage device, and the recording medium is inserted therein. It may be a readable CD-ROM or the like.

Further, the recording medium may be a magnetic tape, a flexible disk, a hard disk, an MO / MD / DVD, or a semiconductor memory.

The digital smart security system, method and program according to the present invention have a great effect when applied to a sales management system capable of effectively reducing shoplifting damage in a commercial facility. Installation in various stores is expected.

DESCRIPTION OF SYMBOLS 11 Surveillance camera 12 Electronic tag gate 20 RFID reader 30 Wi-Fi parent machine 40 i beacon parent machine 50 Portable terminal device (terminal device)
51 Wi-Fi handset 51A Wi-Fi repeater 52 i-beacon handset 53 GPS
100 monitoring device 110 control unit 111 tag movement determination unit (customer movement detection means)
112 Customer location information acquisition unit (customer movement detection means)
113 Monitoring object setting unit (customer movement detection means)
114 Monitoring target tracking unit (customer movement detection means)
115 Suspicious person judging section (tag abnormality judging means, suspicious person registering means)
116 Face authentication function unit (face authentication means)
117 Position determination unit (control means)
118 Suspicious person tracking part (control means)
119 Transmission control unit (control means)
160 face information DB
200 POS server 250 Handy scanner 300 POS register 400 EOS
1000 Digital Smart Security System

Claims

Customer movement detection means for detecting a customer moving with the tagged product,
Tag abnormality determination means for determining that the product to which the tag is attached stops moving for a predetermined time at a place other than a predetermined location and that the tag cannot be detected;
Suspicious person registration means for registering the image of the customer's face imaged by the surveillance camera as the suspicious person's face when the tag abnormality determination means determines that the tag has stopped moving or cannot be detected. A featured digital smart security system.
Face authentication means for photographing a customer's face with a surveillance camera and authenticating whether the photographed customer's face matches the suspicious person's face registered in the suspicious person registration means;
2. The digital smart security system according to claim 1, further comprising a control means for notifying that the face authentication means is matched.
The digital smart security system according to claim 1, wherein the control means transmits information including an image of the suspicious person's face to a portable terminal device carried by an employee closest to the suspicious person.
Detecting customer movement detecting a customer moving with the tagged product;
A tag abnormality determination step for determining that the product with the tag attached is not moved to a predetermined place and stops moving for a predetermined time and that the tag cannot be detected;
A suspicious person registration step of registering, as a suspicious person's face, an image of the customer's face imaged by the surveillance camera when the tag abnormality determination step determines that the tag has stopped moving or cannot be detected. A digital smart security method characterized.
Computer
Customer movement detecting means for detecting a customer who moves with the product with the tag, the product with the tag being stopped moving for a predetermined time at a place other than the predetermined location, and the tag being undetectable. A tag abnormality determination unit for determining, and a suspicious person who registers the image of the customer's face captured by the monitoring camera as the suspicious person's face when the tag abnormality determination unit determines that the tag has stopped moving or cannot be detected For functioning as a digital smart security system provided with a user registration means.