JP2003169320A - Monitoring method and system thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a monitoring system for effectively finding an illegal act while reducing a monitoring burden. <P>SOLUTION: The monitoring system comprises a plurality of cameras 10, 15 and 20 that photograph a client and have transmission means for transmitting the photographed image signal; a reception means 31 for receiving an image signal that is transmitted by the transmission means; a personage-specifying means 33 for specifying a personage who is conscious of the plurality of cameras 10, 15 and 20 based on an image signal that is received by the reception means 31; a judging means 39 that monitors the behavior state of a personage who is specified by the personage-specifying means 33, performs a judgment based on the behavior condition, at the same time performs a judgment based on the image of the personage who is specified by the personage-specifying means 33; a recording means 37 for recording the image of the specified personage based on the judging means 39; and an alarm means 40 for giving an alarm when a personage is specified by the personage-specifying means 33. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a surveillance system for monitoring a person who commits illegal activities using a security camera.

[0002]

2. Description of the Related Art Generally, in supermarkets, convenience stores, bookstores, etc., security cameras are used to monitor illegal activities such as shoplifting. In addition, the security camera is installed in a blind spot, which is invisible to the clerk. A plurality of security cameras are installed in the store. The provided security camera is connected to a terminal such as an office. The terminal displays the image taken by the security camera. The displayed image is used to monitor the situation in the store, the movement of customers, etc. by the store owner and security guards.

[0003]

[Problems to be Solved by the Invention] In order to carry out perfect crime prevention, it is necessary to equip a plurality of cameras so that blind spots do not occur everywhere in a store, and to display a plurality of images transmitted from the cameras. Since it is necessary to monitor the display device all the time, the monitoring load is heavy.

Further, if the number of cameras is increased, many customers dislike the act and move to another store, causing a problem that the sales of the store are reduced.

Further, as an indispensable element for a person who commits an illegal act, a customer usually searches for a product when entering a store such as a supermarket. Therefore, the eyes and face are oriented in the direction in which the products are displayed (downward). However, customers who try to cheat are aware of security cameras. Therefore, there is an element that the direction of eyes and face should face toward the security camera.

In view of the above point, the present invention aims to provide a monitoring system capable of effectively detecting fraudulent activity while reducing the monitoring load.

[0007]

The present invention will be described below. In addition, in order to facilitate understanding of the present invention, reference numerals in the accompanying drawings are added in parentheses, but the present invention is not limited to the illustrated forms.

According to the first aspect of the present invention, a plurality of cameras (10, 15, 20) for monitoring fraudulent activities are provided, and the plurality of cameras are conscious of in the store (1) where the surveillance is performed by the monitor. It is characterized by including a step for determining whether or not the determination has been made, and a step for specifying a person who is conscious of the plurality of cameras based on the determination.

According to the present invention, by the plurality of cameras,
The person who seems to be suspicious of behavior (performing cheating) can be narrowed down from a large number of customers entering the store, so the supervisor (shopkeeper or security guard) is narrowed down rather than monitoring the entire customer. Since it is only necessary to monitor the customer, it is possible to reduce the monitoring burden on the monitor.

The invention of claim 2 is characterized by comprising a step for notifying the supervisor when the person is specified.

According to the present invention, the observer does not need to watch the image transmitted from the camera much more carefully, and after he / she is warned, he / she should watch it particularly carefully.
It is possible to reduce the surveillance burden on the observer.

The invention of claim 3 is characterized by comprising a step for recording the action of the specified person.

According to the present invention, the narrowed down person is tracked, and the behavior of the specified person is stored (recorded) in the storage section (37) one by one, so that an evidence photograph can be obtained if an illegal act is performed. can do.

According to a fourth aspect of the present invention, a plurality of cameras having a transmitting means for photographing the customer and transmitting the photographed image signal, and a receiving means for receiving the image signal transmitted by the transmitting means ( 31), and a person identifying means (33) for identifying a person who is conscious of the plurality of cameras based on the image signal received by the receiving means.

According to the present invention, it is possible to specify or narrow down the persons who are considered to be suspicious of behavior (perform misconduct) from a large number of customers who enter the store.

The invention of claim 5 is characterized by comprising a judging means (39) for monitoring the behavior of the person identified by the person identifying means and making a determination based on the behavior.

According to the present invention, the action state of the specified person is collated with a predetermined standard (for example, an action of prowling, moving to and from the same place, etc.),
Since it is determined whether or not this is the case, it is possible to further identify or narrow down the person who takes a suspicious action from a plurality of customers.

According to a sixth aspect of the present invention, there is provided a determining means (39) for making a determination based on the image of the person specified by the person specifying means.

According to the present invention, the image of the identified person is collated with the image (the image of the misconduct addict or the wanted criminal) stored in the storage unit in advance, and it is determined whether or not they match. Therefore, the person taking the suspicious behavior can be further specified or narrowed down from the plurality of customers.

The invention of claim 7 is characterized by comprising recording means (20) for recording the image of the specified person based on the result of the judging means.

According to the present invention, a specific person narrowed down by the person specifying means 33 or the like from a plurality of customers can be tracked and the action of the person can be stored (recorded) in the storage unit 37 one by one. Therefore, if an illegal act is committed, an evidence photograph can be obtained.

The invention of claim 8 is characterized by comprising warning means (40) for issuing a warning when a person is specified by the person specifying means and / or the judging means.

According to the present invention, by issuing a warning to the monitor when the person is specified, the monitor can be strengthened in that case, and the burden on the monitor can be reduced.

[0024]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the monitoring system according to the present invention will be described in detail below.

First, the concrete configuration of each device of the monitoring system and its operation will be described in order.

FIG. 1 is a schematic view of a store equipped with a monitoring system, FIG. 1 (a) is a plan view of the inside of the store, and FIG. 1 (b).
Is a sectional view taken along the line AA, FIG. 2 is a system diagram of the surveillance system, FIG. 3 is a configuration diagram of the surveillance system of the present invention, FIG. 4 is a state diagram when the customer is conscious of the camera, and FIG. It is a pattern diagram shown.

The store 1 shown in the figure is, for example, a convenience store or a supermarket. The store 1 includes products, display shelves 2 and 2 for displaying the products, and a cash register 3 for making a payment for the products. A doorway 4 is provided at a predetermined position of the store 1. For security reasons, there is only one doorway 4. The cash register 3 is provided near the doorway, and a plurality of display shelves 2 are provided so as to divide the inside of the store. Further, as the display shelf 2, a display shelf 2 having a height that allows a customer's face to be seen is used for security purposes.

A plurality of cameras 1 are installed at predetermined positions in the store 1.
0, 15, 20 are provided. This camera 10, 15,
20 is divided into a first camera 10, a second camera 15, and a third camera 20. The cameras 10, 15 and 20 are preferably digital video cameras capable of electronically recording captured images.

The first and second cameras 10 and 15 are cameras capable of photographing in all directions in a store (paving). For example, a fisheye type camera. The first camera 10 is mainly for photographing the situation of the customer and the entire store. The camera 10 is provided at a predetermined ceiling above the cash register 3 so that the entire store 1 can be viewed. On the other hand, the second camera 15 is mainly for photographing the customer. The camera 15 is provided at a predetermined position on the display shelf 2 so that the customer's face can be photographed.

The third camera 20 is for photographing a predetermined area in the store. It mainly functions as a recording unit for recording the situation of a customer's cheating. The camera 20 photographs each aisle divided by the display shelf 2. Further, the camera 20 is provided at a predetermined position in a ceiling corner (a predetermined upper corner in the drawing) for each passage.

Further, the first to third cameras 10, 15, 2
0 has a zoom function for enlarging a predetermined range and a direction changing function for changing the shooting direction.

Further, the first to third cameras 10, 15, 2
0 is provided with a transmission means for transmitting a captured image signal. Specifically, the first to third cameras 10, 1
5, 5 and 20 are connected to a host computer 30 via a transmission line 25, as shown in FIG. Multiple cameras 1
0, 15, and 20 are controlled by this host computer 30. For example, the shooting direction, enlargement (zoom), etc. are controlled as well as the shooting by each camera. The host computer 30 is, for example, an office 1 adjacent to the store 1.
Prepared for a. In addition, the transmission line 25 includes a plurality of cameras 1
It is for transmitting the image signals photographed by 0, 15, and 20 to the host computer 30. For example, a cable or the like is used, but it may be performed by wireless communication. The transmission path 25 may be a wireless or wired circuit.

The first to third cameras 10, 15, 20
The image signal captured by is transmitted to the host computer 30 via the transmission path 25. Host computer 3
0 activates various programs to identify the person who commits the misconduct based on the received image signal and controls the storage (recording) of the identified person.

The host computer 30 receives the image signals photographed by the first to third cameras (10, 15, 20) and performs control for storing (recording) the received image in the storage unit 37. A reception controller 31 as a recording and recording means, an image processor 32 that performs a predetermined image processing on the received image signal, and a person who is conscious of the first and second cameras 10 and 15 based on the received image signal. A person specifying unit 33 as a person specifying unit for specifying
An extraction unit 34 as an extraction unit for extracting the image and / or action pattern of the identified person, and a storage area for storing the image and the like, and a reference pattern of action of the dishonest person or the dishonesty in advance. The storage unit 37 in which images of addicts and wanted criminals are stored, the action pattern extracted by the extraction means and the reference pattern stored in the storage unit 37 are collated, and the identified person and storage unit 37 are compared. The collating unit 38 as a collating unit for collating with the image stored in, and the determining unit for determining whether or not there is suspicion based on the collation result by the collating unit. Based on the result determined by the determination unit 39, and the warning unit 40 as a warning unit for issuing a warning when a person is identified by the person identification unit or the determination unit. Extracting location information of the identified person by extracting means Te, the position information generating unit 35 for generating position information data, the position information the second and / or third
A transmission control unit 36 as a transmission control unit that transmits to the cameras 15 and 20 and controls the second and / or third cameras 15 and 20 to capture the action of the identified person.
And are equipped with.

Specifically, first, when a customer enters the store 1, the first camera 10 photographs the image signal of the customer and the photographed image signal of the customer and the action state signal of the customer are received and controlled via the transmission path 25. It is transmitted to the unit 31. The transmitted image signal and the behavioral signal of the customer are temporarily recorded in the storage unit 37. The reception control unit 31 receives the image signal and transmits it to the image processing unit 32. The image processing unit 32 performs predetermined image processing on the image signal. The image processing is, for example, a process of extracting each image of a customer entering the store. In the image processed image, the person who is conscious of the first camera 10 is specified by the person specifying unit 33. Further, the image-processed image may be temporarily recorded in the storage unit 37. The person is specified by the direction of the face. Specifically, the first
The positions of the eyes and mouth, the skin color, and the like of the image captured by the camera 10 and subjected to the image processing are extracted, and as a result, it is recognized whether or not the camera 10 is conscious by the direction of the face, and the person is specified.

FIG. 4 shows a specific example of the process for identifying a person. As illustrated, the image of the customer 50 is continuously extracted for a certain period of time by the first camera 10, and the extracted image is subjected to two-dimensional discrete cosine transform (2-D DCT),
The low-frequency energy changes are learned by a neural network, and the person is recognized by recognizing whether or not the camera is conscious (expression) based on the movement of the characteristic parts of the face (eyes, eyebrows or the entire face) and the skin color. Identify. Specifically, person 5
The person is specified by recognizing the facial expression image 50b when the 0 face is in the normal state 50a (the line of sight 51 is the product direction) and the state 50b in which the first camera 10 is conscious (the line of sight 52 is the camera direction). Further, the behavior state of the specified person is monitored by the first camera 10. Further, the data of the person (the customer who did not pay attention to the first camera 10) not specified by the person specifying unit 33 is deleted.

Next, the position of the specified person is extracted by the extraction unit 34, and the position information generation unit 35 generates position information. The generated position information is transmitted by the transmission control unit 36 to the second camera 15 via the transmission path 25. The second camera 15 uses the position information to
A person is specified, and images of the specified person are continuously extracted for a certain period of time. As described above, the person identifying unit 33 identifies the person by recognizing whether or not the second camera is conscious of (the facial expression) from the extracted image. Further, data of a person who is specified by the first camera 10 by the person specifying unit 33 but not specified by the second camera 15 (such as the image and action state of the specified person recorded in the storage unit 37). Data) is erased. in this way,
Suspicious persons performing fraud are narrowed down depending on whether or not they are aware of the cameras 10 and 15.

The image of the specified person is stored in the storage unit 37.
The collation unit 38 collates the images (the images of the misconduct addict and the wanted criminal, etc.) stored in advance, and the determination unit 39 determines whether the images match based on the collation result.

Specifically, the image of the specified person and the image previously stored in the storage unit 37 are collated by, for example, extracting a low frequency component. More specifically, the specified person image and the image stored in the storage unit 37 in advance are extracted, and the two-dimensional discrete cosine transform (2-D DCT) is performed.
Then, the energy change of the low frequency component is learned by the neural network, and it is recognized whether or not the person is the same person.

Further, the action pattern (moving line) after a lapse of a certain time from the entrance of the specified person is extracted. The action pattern of the identified person is determined by the extraction unit 34 by the first camera 1
It is extracted from the data (action status) transmitted from 0 and stored in the storage unit 37. The action patterns are extracted, for example, as shown in FIG. 5, the movement patterns of the customers A and B who have entered the store 1. The collating unit 38 collates whether or not the extracted action pattern is applicable to a reference pattern (for example, a unique reference pattern for wandering around or going to and from the same place) stored in advance in the storage unit 37. Based on the collation result, the determination unit 39 determines whether or not the result is true. In this way, the behavior pattern of the specified person is extracted for a certain period of time, and the extracted behavior pattern is collated with a predetermined standard to determine whether or not it is applicable.

Based on the determination of the person image and the action pattern, it is further comprehensively determined whether or not to commit fraud.

When the judgment section 39 judges that the specified person is suspicious, the warning section 40 gives a warning based on the judgment result, and the current position information of the specified person is displayed. The position information generated by the information generation unit 35 is transmitted to the third camera 20 by the transmission control unit 36 via the transmission path 25. Third camera 20
Identifies a person based on the transmitted position information and photographs the action of the identified person. The captured image is transmitted to the host computer 30 via the transmission path 25 and stored (recorded) in the storage unit 37.

Further, the warning is given to the store owner or the guard by some device such as an optical sensor or a display device.

As described above, the first camera 10 provided in the store 1 transmits the image signal obtained by photographing the customer entering the store to the host computer 30. The host computer 30 identifies the person who is conscious of the first camera 10 by the person identifying unit 33, and stores (monitors) the action state of the identified person. Further, the position information of the specified person is extracted from the stored action state, and the second camera 15 is controlled based on the position information to photograph the specified person. The image signal obtained by photographing is the host computer 3
0, and the person identifying unit 33 determines whether or not the second camera 15 is conscious. When the person is identified by the determination (when the second camera 15 is recognized), the pattern extraction unit 34 extracts the action pattern from the action state of the identified person stored in the storage unit 37 by the first camera 10. The extracted behavior pattern is collated by the collation unit 38 with the reference pattern stored in the storage unit 37 in advance. Based on this collation result, the determination unit 39 determines whether or not the result is true.

Further, the image extraction unit 34 extracts the image of the specified person stored in the storage unit 37. The extracted image is collated by the collating unit 38 with the person image (the image of the misconduct addict, the image of the wanted criminal, etc.) stored in the storage unit 37 in advance. Based on the result of this collation, the person identified by the determination unit 39 and the storage unit 37 in advance.
It is determined whether or not the person stored in is matched.
A comprehensive determination is made based on the above determination result.

The judgment is divided into stages according to the degree of risk.
For example, it is divided into A to C stages. In this case, A represents the highest risk and C represents the lowest risk.

Based on the result determined by the determination unit 39, the warning unit 40 warns the store owner or the guard. Also,
The position information is extracted from the specified person's action state stored in the storage unit 37 by the pattern extraction unit 34, and the third camera 20 is controlled by the host computer 30 based on the position information to specify the person, The specified person is photographed (recorded). The photographed data (action state) is stored (recorded) in the storage unit 37.

[0048]

According to the present invention, since a person who makes a suspicious behavior is monitored and a warning is given, a fraudulent activity can be effectively detected, and a monitoring load on a shopkeeper or a security guard can be reduced.

[Brief description of drawings]

FIG. 1 is a schematic view of a store equipped with a monitoring system of the present invention, FIG. 1 (a) is a plan view of the inside of the store, and FIG. 1 (b) is a sectional view taken along line AA.

FIG. 2 is a system diagram of FIG.

FIG. 3 is a configuration diagram of a monitoring system of the present invention.

FIG. 4 is a state diagram when a customer is aware of a camera.

FIG. 5 is a pattern diagram showing a behavior of a customer.

[Explanation of symbols]

1 store 10 First camera 15 Second camera 20 third camera 31 Reception control unit 33 Person Identification Department 37 Memory 39 Judgment section 40 Warning section

   ─────────────────────────────────────────────────── ─── Continued front page    F-term (reference) 5C022 AA01 AB61 AB65 AC11 AC27                       AC69 CA00                 5C054 AA01 CA04 CC05 CF06 CG06                       CH02 EA05 FA09 FC12 FE14                       FE28 FF02 FF06 GB02 HA19                 5C084 AA02 AA09 BB11 BB31 CC19                       DD11 EE10 GG54 GG78                 5C087 AA02 AA03 AA07 AA31 BB74                       DD05 DD20 EE08 EE11 EE14                       GG02 GG06 GG19

Claims (8)

[Claims] 1. A step for determining whether or not a plurality of cameras for monitoring a fraudulent activity are provided, and in which a plurality of cameras are monitored by a surveillance person by the cameras, A step of identifying a person who is conscious of the plurality of cameras based on the monitoring method. 2. The monitoring method according to claim 1, further comprising a step of notifying the monitoring person when the person is identified. 3. The monitoring method according to claim 1, further comprising a step of recording the behavior of the specified person. 4. A plurality of cameras provided with a transmission means for photographing a customer and transmitting the photographed image signal, a receiving means for receiving the image signal transmitted by the transmitting means, and a receiving means. A person identifying means for identifying a person who is conscious of the plurality of cameras, based on the received image signal, and a monitoring system. 5. A monitoring system comprising: a determination means for monitoring the behavior of the person identified by the person identification means and making a determination based on the behavior. 6. A monitoring system, comprising: a determination unit for performing determination based on an image of the person identified by the person identification unit. 7. The monitoring system according to claim 4, further comprising a recording unit for recording the image of the identified person based on the result of the determination unit. 8. The monitoring system according to claim 4, further comprising warning means for issuing a warning when a person is specified by the person specifying means and / or the judging means.