JPH09330415A - Image monitoring method and image monitoring system - Google Patents

Abstract

(57) [Abstract] [PROBLEMS] To provide an image monitoring system for selecting a determination algorithm adapted to a monitoring condition. A person extraction unit 300 analyzes images of a plurality of monitoring areas captured by a camera 100 in a predetermined cycle, and extracts a person image for each area. The person monitoring unit 600 sequentially takes in a person image for each monitoring area and calculates a movement locus. Furthermore, the monitoring environment of the monitoring target is obtained from the current time (year, month, day, time). When the target is a store, the monitoring environment is open or closed. The processing mode set in advance in the monitoring mode table 705 is read for this combination of the monitoring environment and the monitoring area. For example, a movement trajectory is
In processing mode 1 (monitoring area including the entrance / exit, while the store is open), it is determined to be a person entering the store, and in processing mode 4 (monitoring area not including the entrance / exit, while the store is closed), it is determined to be an intruder.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image monitoring system, and more particularly to a method for monitoring an intruder by a determination process adapted to a monitoring condition for a movement locus extracted by image processing.

[0002]

2. Description of the Related Art As a safety management system for a space where an unspecified number of people gather by image processing, Japanese Patent Application Laid-Open No. 4-60880 discloses that a person can be detected from a locus depending on the position of image data of a surveillance area and moving speed. It describes how to understand behavior and monitor intruders in the prohibited areas. Also,
It is also known to change the threshold of image processing day and night.

[0003]

There is a case where it is desired to change the monitoring mode depending on the time zone and the season even in the same monitoring area.
For example, in the vicinity of the entrance / exit of a store, not only is there a difference in the behavior of the person when the store is opened and when the store is closed, but the algorithms for determining an intruder are different.

However, in the conventional image monitoring method, since the analysis method for grasping the behavior of a person is fixed and dedicated for each monitoring target, a monitoring device is installed according to the monitoring target and the monitoring purpose. However, there is a problem that not only the system cost rises but also the safety management becomes complicated.

In view of the state of the art, the object of the present invention is to
An object of the present invention is to provide an image monitoring method and an image monitoring system that make a determination based on the same movement trajectory using an algorithm adapted to the monitoring conditions. This makes it possible to centralize the management of visitors and the monitoring of illegal intruders at places where an unspecified number of people enter and leave.

[0006]

[Means for Solving the Problems] The above-described object is an image for monitoring the behavior of a person with respect to the monitoring target from the movement trajectory of the person, which is obtained by time-sequentially capturing an image of the periphery of the monitoring target such as a building and performing predetermined image processing. In the monitoring method, the environment depending on the time of activity or pause of the monitored object is obtained from the current time, and a processing mode adapted to the environment is selected, or a processing mode adapted for each of the monitoring areas is selected, This is achieved by determining the behavior of the person on the movement trajectory. Alternatively, it can be achieved by switching the processing mode set corresponding to the monitoring area and the monitoring condition and determining the action of the person on the movement locus online.

Here, when the monitoring target is a store, the time-dependent environment is a store open state or a store closed state. Further, the monitoring area includes the vicinity of the entrance / exit of the monitoring target, and when an open / closed environment of the monitoring target is set in the monitoring condition, the "entry person" or The movement trajectory determined as "exiter" is determined as "intruder" in the processing mode corresponding to the closed environment.

That is, when the monitored area includes the vicinity of the entrance / exit of the object to be monitored, when the movement locus moves toward the entrance / exit and the end position thereof is near the entrance / exit,
In the processing mode corresponding to the open environment, it is determined as "entry person" and in the processing mode corresponding to the closed environment, "intruder", while the monitoring area includes the periphery other than the vicinity of the entrance and exit of the monitoring target In addition, when the movement locus moves toward the monitoring target and the end point position is near the preset monitoring line of the monitoring target, the processing modes corresponding to the open and closed environments are both “intruder”. Is determined.

Further, when the monitoring area covers a region other than the vicinity of the entrance / exit of the object to be monitored, the movement locus determined as "resident" in the processing mode corresponding to the open environment corresponds to the closed environment. It is determined that the candidate is an “intruder candidate” in the processing mode.

That is, when the monitored area includes a periphery other than the vicinity of the entrance / exit of the object to be monitored, and the movement locus is near the monitoring line preset in the monitored area and does not indicate the end point position, the opening is performed. In the processing mode corresponding to the environment (1), it is determined as "retained person", and in the processing mode corresponding to the closed environment, it is determined as "intruder candidate".

If the number of "intruder candidates" detected continuously exceeds the predetermined number of times, it is determined to be "intruder".

An image monitoring system to which the image monitoring method of the present invention is applied includes a television camera which periodically captures an image of the periphery of an object to be monitored, such as a building, and the captured images in time series for each monitoring area, and a predetermined image processing. Image processing means for performing
A person extracting means for extracting a person from a predetermined feature amount calculated through the image processing; and an image monitoring apparatus having a person monitoring means for determining a person's behavior with respect to a monitoring target from the movement trajectory of the person, and There is a monitoring mode table that presets a processing mode adapted to a combination of a monitoring condition that depends on time and the monitoring area, and the person monitoring means sets the monitoring condition obtained from the current time and the monitoring area of the movement locus. Based on the processing mode selected from the monitoring mode table, the behavior of the person on the movement trajectory is determined.

Further, it is characterized in that a surveillance area setting means for setting one or a plurality of surveillance areas is provided for each of the one or a plurality of the television cameras.

Further, if the judgment by the person monitoring means is “intrusion” or “intrusion candidate” with respect to the monitoring target,
A display means for displaying the determination result and the corresponding movement trajectory is provided.

According to the present invention, the process of grasping a person in each surveillance area and extracting the movement trajectory is performed in the same procedure, and the processing mode for determining an intruder or the like with respect to the movement trajectory of the person is switched according to the monitoring condition.

According to this, it is possible to flexibly apply a desired processing mode according to a monitoring target and a monitoring purpose, and it is possible to provide an image monitoring apparatus having a low system cost. Further, since online monitoring can be performed while automatically switching the processing mode according to the monitoring condition, safety management at a place where a large number of unspecified persons come in and out can be unified, and manpower for monitoring can be reduced.

[0017]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described in detail below with reference to the drawings. FIG. 1 is a configuration diagram of an image monitoring system according to an embodiment. In this system,
The ITV camera 100 photographs the area to be monitored, and the image signal is captured by the image input unit 200 and A / D converted.

The person extraction unit 300 creates a difference image from the inter-frame difference from the A / D converted input image signal or the background difference between the input image and a reference image (for example, a background image captured at the start of sampling). To calculate the concentration frequency distribution. Further, from the density information obtained by removing the noise of the density frequency distribution, a change area is extracted from a binary image binarized by a predetermined threshold value or a difference image thereof, and a feature amount of the change area described later is calculated to extract a person. To do.

The monitoring area setting unit 500 sets one or more monitoring areas to the monitoring mode table 7 via the monitoring condition setting unit 700.
Set to 05. Monitoring area 7 in monitoring mode table 705
For each 20th, the environmental condition 730 to be monitored, for example, if the store is a store open / closed, whether it is an active state in which humans are active in the target, or a dormant state in which humans are absent on holidays or after work hours. The processing mode 740 is set by the monitoring condition setting unit 700.

The person monitoring unit 600 refers to the monitoring mode table 705 based on the current monitoring time and monitoring area, and the person extracted by the person extracting unit 300 according to the processing mode is
When the movement trajectory is extracted and the behavior is determined to determine what kind of behavior in which area, and when it is determined that the intruder or the intruder candidate, the image is accumulated in time series in the data storage unit 1200, and the display control unit 1300 Is displayed in real time on the display device 1400 via.

FIG. 2 shows a functional block diagram of the image input section. When the image input unit 200 captures the video signal of the target scene from the ITV camera 100, one image capturing unit F
The image signal captured from k210 is used as a reference (for example, the first sampled image at the start of capturing) is used as a background image. Alternatively, as the image immediately before the inter-frame difference is performed, the other image capturing unit Fi22
The image signal captured by 0 is set as the input image to be processed. The image of the reference signal (background image or immediately preceding image) converted by the A / D conversion unit 230 and the input image to be processed are transmitted to the person extraction unit 300.

FIG. 3 is an explanatory diagram showing an example of area setting by the monitoring area setting unit. For example, with respect to the monitored facility 510, an intrusion line 550 is set at the boundary between the facility outer wall and the outdoor, a monitoring area 560 is set near the facility outer wall, and a monitoring area 570 is set near the facility entrance / exit by using a mouse or the like. . A plurality of monitoring areas can be set at arbitrary positions, but it is preferable to provide one monitoring area corresponding to the same monitoring mode. Also, it goes without saying that it is within the field of view of the ITV camera 100 and within the illuminatable range of the illumination device 800 (for example, LED infrared illumination) used when the illuminance is insufficient at night.

FIG. 4 shows a functional block diagram of the person extracting section. The difference image calculation unit 310 of the person extraction unit 300 takes in the image signal after A / D conversion, calculates the density difference for each pixel between the images, creates a difference image, and calculates the density frequency distribution. The unit 320 converts the density frequency distribution of the difference image into the density frequency distribution Hj (j = 0,
1 ,. . . , N) Calculate. After that, the noise removing unit 330
The density frequency distribution Hj is smoothed by the equation 1.

[0024]

## EQU1 ## Hj = (Hj-1 + 2Hj + Hj + 1) / (4-1) H0 = (2H0 + H1) / (3-1) Hn = (2Hn + Hn-1) / (3-1) where j = 1 ,. . . , N−1. Further, n is the maximum value of the density.

The binarization threshold calculation unit 360 automatically calculates the binarization threshold of the difference image from the noise-removed density frequency distribution Hj. The binarization processing unit 370 uses the binarization threshold value to convert the difference image from the difference image calculation unit 310 into two.
Quantize. At this time, an image having an area larger than a predetermined value which is not suitable for human detection is excluded as noise and a binary image is created.

The feature amount calculation unit 380 calculates the feature of person detection from the difference image obtained by the difference image calculation unit 310 and the binarized image obtained by the binarization processing unit 370 to identify the person. Here, when the monitoring place is dark and the difference image calculation unit 310 cannot effectively extract the difference image, the illumination device 800 is turned on. If an LED infrared lighting (near infrared) device or the like is used for the lighting device 800, no lighting pollution will occur and no suspicious person will notice.

FIG. 5 is an explanatory diagram showing an example for automatically calculating the binarization threshold value. Binarization threshold value calculation unit 36
0 represents the maximum density (max) 362 obtained from the density frequency distribution Hj smoothed by the noise removal unit 330, and the binarization threshold value (t
h) is calculated automatically.

First, when there are many areas of change, the smoothed density frequency distribution Hj becomes as shown in FIG. Here, when the density is searched from the maximum density 362 to the smaller one, there is at least one local maximum 363. Based on this maximum value 363, searching for a smaller concentration,
There is a local minimum 364 that is closest to the maximum concentration. This minimum 3
From the density value of 64 to the maximum density 362 is the density distribution of the changed portion. Therefore, the density value of the minimum value 364 is set as the binarization threshold value (th) 364. In image processing, this is the determination of the binarization threshold value by the modal method.

However, when the area of the changed portion is small, that is, when the frequency of the density of the changed portion is small, there is no maximum value 363 as shown in (a) and the density frequency as shown in FIG. Distribution. Binarization threshold (th) 36 in this case
8 is set to a value obtained by subtracting a constant C from the maximum density (max) 362 according to the equation 2.

[0030]

## EQU00002 ## th = max-C (th.gtoreq.thmin) where thmin is a preset lower limit threshold value 366, and when the calculated threshold value (th) 368 is smaller than thmin,
The lower threshold value is set to 366.

The constant C is an upper threshold (thma) of white noise in a normal state calculated in advance from the normal image of the target image.
x) and the lower threshold (thmin). Alternatively, any value may be used as long as it indicates the maximum fluctuation range of noise in the normal state of the target image. For example, in a scene where the outdoor environment is monitored in the daytime, the upper limit threshold thmax is about 12 to 14 gradations and the lower limit threshold thmin is about 4 to 5 gradations. Therefore, the constant C is about 7 to 10 gradations. Even when the maximum value of the changed portion includes white noise, the white noise is reduced by the smoothing process of Equation 1, so that even a slight change in the density difference can be set accurately.

FIG. 6 shows a functional block diagram of the binarization processing section. The binarization unit 374 creates a binary image with the binarization threshold th. The small area removing unit 376 removes an area having a small area unsuitable for extraction as noise, and changes only the change area to 2
Create a value image. The binarization processing unit 370 binarizes the changed area as an area having an area of a size suitable for a person. For example, when the resolution accuracy of an image is 512 × 512, several tens to 100 pixels or less. The area of is excluded as noise.

In many cases, in the binary image of only the change area, the object to be extracted is separated. The image correction unit 378 expands and connects the separated regions and then contracts them to correct the binary image. The image correction unit 378 has the same number of times of expansion and contraction, which is about two to three times when extracting a person.

FIG. 7 shows a functional block diagram of the feature amount calculation section. The calculated variance of the density is calculated using the difference image, and the area, the aspect ratio, the moving distance, and the moving direction are calculated in time series using the binary image, i−1 time, i time, and i + 1 time.

In this case, even if the density variance calculated by the density variance calculator 381 is the density variance of the difference image calculated by the difference image calculator 310, the difference image is binarized by the binarization processor 370.
It may be a variance value of the densities of the difference image in the masked area limited to the part that is digitized and extracted.

The area calculation unit 383 calculates the area by calculating the number of pixels of the binary image binarized by the binarization processing unit 370. The aspect ratio calculation unit 385 projects the binary image binarized by the binarization processing unit 370 in the X direction and the X projection distribution obtained by accumulating the number of pixels at each position of the X axis, and the Y direction. Then, the ratio of the Y projection distribution obtained by accumulating the number of pixels for each position on the Y axis is calculated as the aspect ratio.

FIG. 8 is an explanatory view showing the first method of calculating the moving direction in the moving direction calculating section. The moving direction calculating unit 388 calculates the moving direction from the center of gravity of the binary image extracted by the person extracting unit 300 at the first time (i = 1).

That is, the rectangular side of the monitoring area closest to the center of gravity of the person extracted for the first time is calculated. For example, in the case of the position of the extracted person 478, the closest monitoring area 570
The rectangular edge of is 472. A line 47 parallel to this rectangular edge 472
4 is obtained, and the movement of the extracted person 478 is forward in the direction of the front semicircle 476.

Similarly, in the case of the extracted person 486, the rectangular end side is 480, a line 482 parallel to the rectangular end side 480 is obtained, and the direction of the front semicircle 484 is set as the forward direction. In the case of the extracted person 492, the rectangular end side is 488, a line 490 parallel to the rectangular end side 488 is obtained, and the direction of the front semicircle 491 is set to the forward direction. In the case of the extracted person 496, the rectangle edge is 493, and the line 49 parallel to the rectangle edge 493 is
Find 4 and make the forward semicircle 495 the forward direction.

FIG. 9 is an explanatory diagram showing the second and subsequent movement direction calculation methods in the movement direction calculation unit. The moving direction calculation unit 388 sets the vector ai-1 calculated corresponding to the barycentric coordinate 485 at the closest position advanced from the i-1th barycentric position 475 as the moving direction. Here, to move forward is
When the angle of the vector ai-1 shown in the figure is θ, this is a case where the angle of the vector ai is within the range of θ−π / 2 to π / 2 + θ.

The moving distance calculation unit 387 calculates the distance between the center of gravity of the binary image extracted by the person extracting unit 300 at the (i-1) th time and the length between the center of gravity of the binary image extracted by the person extracting unit 300 at the ith time by i.
Calculated as the distance for the second time. However, the first moving distance =
Set to 0.

FIG. 10 is a flow chart showing the extraction procedure of the person extracting section. At the i-th time, it is first checked whether or not the shape is vertically long (step 405), and if the shape is vertically long (Y), large, medium, and small areas are checked (step 405).
410). If the area is medium, the density dispersion value is checked (step 415), and if the density dispersion value is large (Y), it is determined to be a person (step 420). If the dispersion value of the density is not large, it is determined to be a disturbance (step 425).

Here, in the case of an adult whole-body image, the aspect ratio is about 4/1 to 6/1. The area follows the visual field range of the camera, but for example, the resolution accuracy of the image is 512 × 512.
In the case of the visual field range in which the person's head to feet are reflected on the screen, about 25,000 to 35,000 pixels are medium.
It is determined in step 410 that the area is large (about 3 under the above conditions).
If it is 5000 pixels or more), it is determined to be a disturbance.

Next, when it is determined in step 410 that the area is small (about 25,000 pixels or less under the above conditions), the number of consecutive detections is counted each time this procedure is passed, and it is checked whether or not n consecutive times ( Step 430). In the case of n consecutive passages, the moving direction is checked (step 435), and if the moving direction is the same direction (Y), it is determined to be a person (step 440).

The number of consecutive detections n in step 430 is, for example, about 2 to 4 times when a person walks normally (i-2).
In the case of continuous passage at the 1st, i−1th, and ith times, n =
3). If the passage in step 430 is one-time or not continuous, it is determined to be a disturbance (step 425). Step 4
Even if the movement directions are not the same in 35, it is determined to be a disturbance.
The moving directions in step 435 are the same when moving forward as described in FIG.

Finally, if the shape is not vertically long in step 405, the moving distance is checked (step 445), and if the moving distance is small (Y), the moving direction is checked (step 450), and the moving direction is not regular. If (N), the area is checked (step 455). When the area is small (Y), the number of detections within a predetermined time is counted (step
460), if it is detected n times (Y), it is determined to be a person (step 465). Since this person has a small area even after being detected n times, it is a resident person who does not move much and stays in the vicinity of the same place, and is usually likely to be a worker in the vicinity.

If the movement distance is large in step 445, it is determined to be a disturbance (step 470). The moving distance in step 445 follows the visual field range of the camera. For example, when the resolution of the image is 512 × 512, the visual field range in which the whole body image of the person appears on the screen. The travel distance is less than the degree. If the moving direction in step 450 is regular, it is determined to be a disturbance. The moving direction in step 450 is the same as that in step 435, and it is assumed that the vector is not regular when compared with the immediately preceding vector or when the moving direction has no periodicity.

Even if the area checked in step 455 is not small, it is determined to be a disturbance. The area in step 455 is about 25,000 pixels or more when the area is not small, and is less than this area when it is small. In step 460, the number of times of detection within a predetermined time is counted, and if it is not detected n times, it is determined as a disturbance in step 470. The predetermined time in step 460 follows the monitoring target, but for example, in the case of suspicious worker detection, it may be within a few minutes, and the number of times the procedure of step 460 is passed within this time is one processing time. It is only necessary to be able to count several times in the case of about 500 milliseconds.

FIG. 11 is a functional block diagram showing the structure of the person monitoring section. Movement locus calculation unit 603 of the person monitoring unit 600
Is calculated by the person extraction unit 300, and the movement trajectory of the person is calculated while accumulating the coordinate data of the person for each monitoring area in time series. The calculated movement locus is numbered for each person and passed to the monitoring mode switching unit 613 in the order of monitoring areas.
When the processing of all the monitoring areas at the monitoring timing is completed, the end signal is output.

The monitoring mode switching unit 613 switches the monitoring mode according to the monitoring area according to the processing mode set in the monitoring mode table 705. Monitoring processing unit 623
Tracks the movement locus of the person passed from the movement locus calculation unit 603 by an action analysis algorithm for each processing mode, and determines a visitor, a moving person, an intruder, or the like. Data from the start of analysis to the determination is accumulated in the data storage unit 1200, and when an intruder or intrusion candidate is detected, the display control unit
Output to 1300.

FIG. 12 shows the format of the monitoring mode table. Depending on the input mode of the monitoring condition setting unit 700, the input format is displayed on the screen of the display device 1400, and the conditions can be set from the keyboard or the monitoring area setting unit 500. The monitoring level 710 is a determination level of an intruder in the gray zone, and in this example, the continuous detection number m when determining an intruder candidate as an intruder is set.

The monitoring area 720 is set by the monitoring area setting unit 500.
It is set on the screen of the monitoring target, and in this example, the monitoring area 570 including the entrance / exit of the building and the monitoring area 560 around the building not including the entrance / exit are set. An entrance area 540 is set near the entrance including the entrance, and a monitoring line 550 is set along the outer wall of the building.

In addition to this, the monitoring area can be arbitrarily set depending on the object to be monitored and the purpose of monitoring, such as whether the monitoring area is on the front side or the back side or near the placement of dangerous or important objects.

The monitoring environment 730 is set from the viewpoint of the operating state / stationary state or the open state / closed state of the monitored object, and is usually a condition determined by time zone, due date, time and the like. In this example, the monitoring target is a store, and two types of opening and closing are set for each monitoring area.

The processing mode 740 is the monitoring area 720 and the monitoring environment.
It is set in a person monitoring processing mode corresponding to 730 from among a plurality of behavior analysis algorithms prepared in advance.
In this example, modes 1 to 4 are set, which differ in the severity of monitoring and the correspondence.

FIG. 13 is a flowchart showing the monitoring mode switching processing. The monitoring mode switching unit 613 receives the monitoring area number and the movement locus of the person from the movement locus extraction unit 603, is activated, and repeats the following processing until the end signal is received (801).

First, the corresponding monitoring area is confirmed from the received monitoring area number (802). Next, the monitoring environment 730 opens from the current year / month / day / time by a timer (store)
/ It is calculated whether it is closed (store) (804). As a result, the processing mode number corresponding to the monitoring area and the open / closed state is selected from the monitoring mode table 705 (805). This processing mode number and the movement locus (one or more) received from the movement locus calculation unit 603 are transmitted to the monitoring processing unit 623 (806), and the step
Return to 801.

In the above, the same processing mode is set for the same area during the day and at night depending on the open / closed state of the store. However, some areas may require more stringent monitoring during the night. In such a case, it is possible to add day / night conditions to the opening / closing of the monitoring environment 730 to set various processing modes.

FIG. 14 is a flow chart showing a person action determination process according to the processing mode. A case where the monitoring target is a store and the monitoring condition of FIG. 12 is taken as an example will be described.

The monitoring processing unit 623 of the person monitoring unit is started by receiving the data of the processing mode number and the movement locus from the monitoring mode switching unit 613, and executes the processing by the determination program of any one of Mode 1 to Mode 4. . First, the processing mode is determined, and the process branches to any one of modes 1 to 4 (15
00). The start point position and the end point position of the movement trajectory are the position that first appears in the monitoring area and the position immediately before the position disappears from the monitoring area.

Mode 1 is a monitoring area 570 including an entrance / exit area 540, which is a case of opening a store. First, check whether the movement path is toward the entrance area 540 (151
0). If it is in that direction (Y), the end point position of the movement trajectory is checked at the entrance / exit area 540 (1520), and if so, it is determined to be a visitor (1530).

On the other hand, if the movement locus does not go to the entrance / exit area 540, the check is made in a direction away from the entrance / exit area 540 (1540). If so (Y), check whether the viewpoint position of the movement trajectory is in the entrance / exit area 540 (1550),
If so, it is determined that the player has exited (1560). This allows
The number of people entering and leaving the store can be managed. Note that step 1154
If the check of 0 or 1550 is not possible, it is determined that the person is a person outside the store (1560).

FIG. 15 shows a locus of movement determined as an attendant or an exiter in the mode 1. The movement locus in the monitoring area 570, which is the starting point position 920 and the ending point position 930, is determined to be an attendee. Further, it is determined that the moving track from the starting point position 960 to the ending point position 970 is an exiting person.

In mode 2, the monitoring area 570 including the entrance and exit
And that's the case of closing. First, a check is made to see if the movement trajectory has an end point position (1610). If there is an end point position (Y), check that position at the entrance area 540 (162
0), if so, it is determined to be an intruder (1640). This movement locus corresponds to a mode 1 visitor.

On the other hand, if there is no end point position on the movement locus (N), it is checked whether the current movement position is at the entrance / exit area 540 or its vicinity (1650). If so (Y), check whether the movement state is detected m times within a predetermined time (1660). If detected m times, it is determined to be an intruder (1640). If it is less than m times, it is determined as a candidate for an intruder (1670). The predetermined time in step 1660 is set to a value of, for example, several tens of seconds to 1 to 2 minutes depending on the importance of the facility to be monitored. In addition, m times are set, for example, about 2 to 4 times in facilities such as stores and buildings, and about 1 to 2 times in important facilities.

Mode 3 is a monitoring area that does not include an entrance / exit
At 560, it is the case of opening a store. First, it is checked whether the movement locus has an end point position (1700). If there is an end point position (Y), a check is made near the monitoring line 550 (1710). If it is near, it is determined to be an intruder (1720), and if not, it is determined to be a mover (1750).

On the other hand, if there is no end point position on the movement track (N), check whether the current movement position is close to the monitoring line 550 (1730). If it is close (Y), it is determined that the person is a stagnation (for example, a worker) around the building (1740). The worker may be determined when the detection of the stagnant person continues for a certain time or longer. The locus of movement of the stagnate or worker is similar to the intruder candidate in mode 2. However, in the opened state, the stagnation person around the building is not judged as an intruder or a candidate.

FIG. 16 shows a locus of movement of a person in Mode 3. In the case where there is an end point position 1150 apart from the monitoring line 550 in the monitoring area 560, if the starting point position is not in the vicinity of the entrance / exit area 540 or the monitoring line 550, it is determined to be a mover. This is the same in other modes.

Mode 4 is a monitoring area that does not include a doorway.
At 560, it was the case of closing the store. First, check whether the movement trajectory has an end point position (1800). If there is an end point position (Y), the check is made near the monitoring line 550 (1810), if it is near, it is determined to be an intruder (1820), and if not, it is determined to be a mover (1830).

On the other hand, if there is no end point position on the movement locus (N), check whether the current movement position is close to the monitoring line 550 (1840). If it is close (Y), check whether the moving state is detected m / 2 times within a predetermined time (185).
0). If detected m / 2 times, it is determined to be an intruder. m /
If it is less than twice, it is determined to be an intruder candidate (1860).

The mode 4 algorithm in this example is
The judgment reference position is basically changed from the entrance area to the monitoring line, and is basically the same as in the case of mode 2. However, due to the fact that there are cases in which people involved with the business after closing the store may stagnate near the entrance and exit, and generally the area near the entrance and exit is most severely protected, it is better to switch the decision from an intruder candidate to an intruder. The algorithm is used in a short time.

FIG. 17 shows the locus of movement of an intruder or an intruder in mode 4. The locus of movement in the monitoring area 560 goes from the starting point position 1020 to the position 1030, etc.
When the position 1040 near 550 is reached, it is determined to be an intruder candidate. Further, when the trajectory disappears from the area 560 after passing through the end point position 1050, it is determined that the intruder. However, the locus does not disappear, the position 1060, 1070, 1080 etc. and the monitoring line
If there is stagnation or prowling around 550, it is judged as an intruder candidate each time, and if it exceeds the predetermined number of times, it is judged as an intruder.

In the above processing, the data such as the movement trajectory during the analysis and the number of detections, and the final determination result are stored in the data storage unit 1200.
Is stored. If the determination result is an intruder or a candidate, it is displayed on the display device 1400 via the display control unit 1300.

FIG. 18 shows an example of an intruder monitoring screen displayed by the display controller. The display device 1400 displays user information 1480 such as user ID, abnormality notification 1490 such as abnormality information and alarm, and abnormality message 1495 such as presence of intruder. further,
The display control unit 1300 for image data displays on the screen the locus of movement of the intruder from the start point position 1020 to the end point position 1050 before disappearance. In this way, since the monitoring is performed together with the real-time screen display, it becomes easy for the monitoring staff to judge.

As described above, the image monitoring system according to the present embodiment extracts a person from the feature amount extracted through the image processing such as binarization by sequentially taking in the grayscale images of the area to be monitored, The behavior of a person is determined by a predetermined algorithm from the movement locus of, and the entrance / exiters are managed and illegal intruders are monitored. In this behavior judgment, the processing mode (judgment algorithm) adapted to the monitoring environment and the monitoring area
However, it is automatically switched according to the monitoring time. The monitoring environment can be automatically selected based on the current time, for example, the operating state (open store) / suspended state (closed) of the monitoring target.

According to this, it is possible to determine different actions for the same movement trajectory depending on the monitoring area and the monitoring environment. By the way, when the monitoring target is a store, the locus of the entrance / exiter from the entrance / exit when the store is opened is determined to be an intruder when the store is closed. A stagnation or stagnant person around the building is determined to be a worker or the like when the store is opened, but is determined to be an intruder candidate or an intruder when the store is closed.

[0077]

According to the present invention, based on the movement trajectory of a person extracted from the image processing apparatus, only the determination processing mode is automatically switched according to the monitoring condition, whereby the attendee management and the intruder monitoring are performed. Since it is possible to adopt a determination algorithm for various purposes such as various monitoring levels, it is possible to further improve the functionality of the image monitoring system and facilitate centralized management.

Furthermore, since a plurality of monitoring areas under different conditions can be imaged by one or a plurality of cameras and thereafter the same system can perform image processing and monitoring processing, there is an effect that the system configuration is simple and inexpensive.

[Brief description of drawings]

FIG. 1 is a functional block diagram of an image monitoring system according to an embodiment of the present invention.

FIG. 2 is a functional block diagram showing the configuration of an image input unit.

FIG. 3 is an explanatory diagram showing an example of setting a monitoring area.

FIG. 4 is a functional block diagram showing a configuration of a person extraction unit.

FIG. 5 is an explanatory diagram showing a method of calculating a binarization threshold value.

FIG. 6 is a functional block diagram showing the configuration of a binarization processing unit.

FIG. 7 is a functional block diagram showing the configuration of a feature amount calculation unit.

FIG. 8 is an explanatory diagram showing how to calculate a moving direction for the first time.

FIG. 9 is an explanatory diagram showing how to calculate a moving direction after the second time.

FIG. 10 is a flowchart showing a procedure of person extraction.

FIG. 11 is a functional block diagram showing the configuration of a person monitoring unit.

FIG. 12 is an explanatory diagram showing a format and setting contents of a monitoring mode table.

FIG. 13 is a flowchart showing a processing procedure for switching monitoring modes.

FIG. 14 is a flowchart showing a plurality of processing modes for determining a person's action.

FIG. 15 is an explanatory diagram showing movement trajectories of the visitors and the exiters.

FIG. 16 is an explanatory diagram showing a movement trajectory of a moving person.

FIG. 17 is an explanatory diagram showing an intruder candidate and a locus of movement of the intruder.

FIG. 18 is an explanatory diagram showing an example of a screen display when an intruder is detected.

[Explanation of symbols]

100 ... ITV camera, 200 ... image input section, 300 ... person extraction section,
500 ... monitoring area setting unit, 540 ... entry area, 550 ... monitoring line, 560 ... monitoring area (around the building that does not include the doorway), 570 ... monitoring area (including doorway), 600 ... person monitoring unit, 603 ... moving locus Calculation unit, 613 ... Monitoring mode switching unit,
623 ... Monitoring processing unit, 700 ... Monitoring condition setting unit, 705 ... Monitoring mode table, 720 ... Monitoring area, 730 ... Monitoring environment, 740 ...
Processing mode, 1200 ... Data storage unit, 1300 ... Display control unit,
1400 ... Display device.

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical indication H04N 7/18 G06F 15/62 380

Claims (12)

[Claims] 1. An image monitoring method for monitoring a person's behavior with respect to a monitoring target based on a movement trajectory of the person, which is obtained by time-sequentially capturing images of the surroundings of the monitoring target such as a building, and subjected to predetermined image processing. Alternatively, the image monitoring method is characterized in that a time-dependent environment such as a pause is obtained from the current time, a processing mode suitable for the environment is selected, and the action of the person on the movement locus is determined. 2. An image monitoring method for monitoring the behavior of a person with respect to a monitoring target from a moving locus of the person, which is obtained by time-sequentially capturing images of the surroundings of the monitoring target such as a building for each monitoring area, and extracted through predetermined image processing, An image monitoring method, characterized by selecting a processing mode adapted for each monitoring area to determine the behavior of a person on the movement trajectory. 3. A person is extracted from a predetermined feature amount calculated by time-sequentially capturing images of the surroundings of a monitoring target such as a building for each monitoring area, and from a movement locus of this person. In an image monitoring method for monitoring the behavior of a person with respect to a monitoring target, the behavior of the person on the movement locus is determined online while switching the processing mode set corresponding to the monitoring area and the time-dependent monitoring condition. An image monitoring method characterized by the above. 4. The open environment according to claim 3, wherein the monitored area includes the vicinity of an entrance / exit of the monitored object, and the open / closed environment of the monitored object is set in the monitoring condition. An image monitoring method, wherein a movement locus determined to be an "entry person" or an "exiter" in the processing mode is determined to be an "intruder" in the processing mode corresponding to the closed environment. 5. The moving trajectory according to claim 3, wherein an open / closed environment of a monitoring target is set in the monitoring condition, and the movement locus is set to the entrance / exit when the monitoring area includes a vicinity of the entrance / exit of the monitoring target. When moving toward the end point position near the doorway, it is determined as "entrancer" in the processing mode corresponding to the open environment, "intruder" in the processing mode corresponding to the closed environment, When the monitored area includes a periphery other than the vicinity of the entrance / exit of the monitoring target, the movement locus moves toward the monitoring target,
When the end point position is near the preset monitoring line of the monitoring target, the processing modes corresponding to the open and closed environments are both determined as "intruder", and the intruder monitoring method is characterized. . 6. The monitoring area according to claim 3, wherein the monitoring area includes a periphery other than the vicinity of the entrance / exit of the monitoring target, and an open / close environment of the monitoring target is set in the monitoring condition, The image monitoring method, wherein the movement locus determined to be "resident" in the processing mode corresponding to the open environment is determined to be "intruder candidate" in the processing mode corresponding to the closed environment. 7. The open / closed environment of a monitored object is set in the monitoring condition according to claim 3, 4 or 5, and when the monitored area includes a periphery other than the vicinity of the entrance / exit of the monitored object, When the movement locus is close to the monitoring line preset in the monitoring area and does not indicate the end point position, it is determined to be “resident” in the processing mode corresponding to the open environment, and corresponds to the closed environment. In the processing mode, the image monitoring method is characterized in that it is determined to be an “intruder candidate”. 8. The image monitoring method according to claim 6, wherein when the detection of the “intruder candidate” continuously exceeds a predetermined number of times, the “intruder” is determined. 9. The image monitoring method according to claim 3, wherein when the monitoring target is a store, the open environment is an open state and the closed environment is a closed state. 10. A television camera that periodically captures an image of a surrounding area of a monitored object such as a building, an image processing unit that captures the captured image in time series for each monitoring area, and performs predetermined image processing, and calculation through the image processing. In a system including a person extracting unit that extracts a person from the predetermined characteristic amount, and an image monitoring apparatus that has a person monitoring unit that determines the action of the person with respect to the monitoring target from the movement trajectory of the person, a monitoring condition depending on time And a monitoring mode table that presets a processing mode adapted to the combination of the monitoring areas, and the person monitoring means monitors the monitoring conditions based on the monitoring conditions and the movement area of the movement locus obtained from the current time. An image monitoring system, characterized in that the behavior of a person on the movement locus is determined according to a processing mode selected from a mode table. 11. The image monitoring system according to claim 10, further comprising monitoring area setting means for setting one or more monitoring areas for each of the one or more television cameras. 12. The display device according to claim 10 or 11, further comprising a display unit that displays a determination result and a corresponding movement trajectory when the person monitoring unit determines that the target is “intrusion” or “intrusion candidate”. Image surveillance system.