JP2000268173A - Object recognition image processing method - Google Patents

Abstract

(57) [Problem] To improve the detection accuracy while simplifying the setting of an object to be recognized in a method and apparatus for processing an image obtained using a television camera and recognizing the object. Aim. According to the present invention, in order to solve such a problem, the size of the detected object is changed to the size of the object in front of the screen using the angle of the foot position of the object captured by the TV camera. By performing the conversion, the reference size of the recognition object can be recognized by one size of the recognition object standing in front of the screen.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

TECHNICAL FIELD The technical field of the present invention relates to a monitoring CCTV device for detecting and monitoring an intruder or an intruder, and an intruder recognition image processing apparatus for detecting an intruder or an intruder in a monitoring area. For example, the present invention relates to an improvement in a method for extracting only an object to be recognized (for example, an intruder) from objects detected by the difference method.

[0002]

2. Description of the Related Art Video surveillance apparatuses using television cameras (hereinafter referred to as TV cameras) have been widely used. However, recently, in such a system,
A system in which the detection of a person or object that enters the monitoring field of view is automatically detected from the image signal instead of visually monitoring the person on the image monitor surface, and a predetermined notification and alarm processing can be obtained. Is being required. In order to realize such a system, first, an input image obtained from a TV camera is compared with a background image, that is, an image in which an intruder or an intruder to be detected is not reflected.
Extract the image of the object. The extraction of the image is performed by obtaining a difference in luminance value for each pixel and detecting an area having a large difference value as an object. This method is called a difference method and has been widely used conventionally. The image of the object extracted in this way is compared with the image of the intruder to be captured registered in advance by pattern recognition,
It is determined whether or not the object is an intruding object for which an alarm should be issued. If the object is an intruder, an alarm is issued.

[0005] Such a conventional apparatus will be described with reference to FIG. FIG. 5 shows only a pure hardware configuration, and is common to the present invention. In FIG. 1, a TV camera 22 captures an image of a fixed monitoring target area. The image input I / F 23 performs A / D conversion of the video signal of the TV camera 22 and converts the video signal from 0 to 2
It is converted into a luminance signal of 55 256 gradations. The image memory 24 stores the luminance signal obtained by the image input I / F 23 and performs an operation between images. The image memory 24 can store a plurality of images at the same time. The image output I / F 31 performs D / A conversion of a luminance signal of an image obtained by the image processing,
It converts a luminance signal into a video signal. The video monitor 32 displays a video signal obtained by the image output I / F 31. The data bus 28 is used for data transfer. The CPU 25 performs image analysis in the work memory 29 according to a program stored in the program memory 30. When an event is detected in the image analysis, a warning is output to a warning display monitor 27 such as a lamp using the output I / F 26.

Next, the flow of object recognition will be described with reference to FIGS.
This will be described with reference to FIG. First, in an initial background image creation step 1, a background image is created to obtain a background image 33.
The background image can be obtained by capturing an image of the monitoring range when no object exists. An example of an improved method of generating a background image is described in Japanese Patent Application No. 9-291910 according to the invention of the present inventors.

In the image input step 2 (see FIG. 6), the TV
Input image 34 (see FIG. 7) from camera 22 (see FIG. 5)
Get. In the difference step 3, the initial background creation step 1
Or, the background image 33 created in the background image update step 8
, And a difference for each pixel from the input image 34 obtained in the image input step 2, to obtain a difference image 35. In the binarization step 4, the difference image 35 obtained in the difference step 3 is subjected to threshold processing, and a pixel having a difference value equal to or less than a predetermined threshold value has a luminance value of 0, and a pixel having a difference value of not less than a luminance value has a luminance value of 255 (The luminance value of one pixel is represented by 256 tones from 0 to 255), and a binary image 36 is obtained. In the noise removal step 5, the noise region 37 included in the binary image 36 obtained in the binarization step 4
And a noise-removed image 38 is obtained. Here, the noise area 37 indicates an area composed of several pixels. In the labeling step 6, numbering of objects is performed using the noise-removed image 38 obtained in the noise removal step 5, and a labeling image 40 is obtained. By labeling step 6,
N for the object area 39 obtained in the noise-removed image 38
It is numbered 1. In the object recognition step 7-2, the labeling image 4 obtained in the labeling step 6
The size of the object is calculated using 0, and only the object to be recognized is extracted by comparing the calculated size of the object with the size of the object to be recognized. Background image update step 8
Then, the background image 33 is updated by a well-known weighted average calculation or the like. At this time, the above-mentioned Japanese Patent Application No. 9-2919 is disclosed.
The background image updating method described in No. 10 can also be used. In the object presence / absence determination step 9, the presence / absence of a recognition object is determined using the determination result of the object recognition step 7-2.
Here, in the object recognition step 7-2, it is determined whether or not the detected object is an object that needs to issue a warning (that is, an intruder). In the warning step 10, when it is determined that there is an intruding object in the object presence / absence determination step 9, a warning is displayed on the warning display monitor 27.

Next, an object recognition process 7-2 will be described with reference to FIG.
This will be described in more detail with reference to FIGS. 8, 9, 10, and 11.

In the following description, as an example, an object recognition image processing apparatus that issues an alarm when a person enters a place where only a car is permitted and no one is allowed to pass is described. In order to perform the object recognition process 7-2,
First, it is necessary to set the size of the object to be recognized as an intruder. The setting of the size of the recognition object will be described with reference to FIGS. An image of a fixed screen for performing object recognition is captured from the TV camera 22 (see FIG. 5),
The object recognition image 12 (see FIG. 9) is stored in one of the image memories 24. In the step 41 for setting the size of the recognition object at the back of the screen (see FIG. 8), the recognition object 42 is positioned so as to be located at the back of the object recognition image 12, and the width 43 of the recognition object 13 The height 44 is set by a pixel value.
The foot position 45 of the recognition object 13 is also set. In the step 11 for setting the size of the recognition object at the front of the screen, the recognition object 13 is positioned so as to be located at the front of the object recognition image 12, and the width 14 and the height 15 of the recognition object 13 are determined in pixels. Set by value. The foot position 16 of the recognition object 13 is also set. These two recognition objects 13 serve as recognition reference patterns. Next, regarding the object recognition processing,
explain. In the object position calculation step 17 (see FIG. 10), the center of gravity GN1, the width XN1, and the height YN1 of the object N1 obtained in the labeling image 40 (see FIGS. 7 and 10)
Is calculated. In the step 18-2 for calculating the size of the recognized object with respect to the foot position, the minimum width Xmin and the minimum height Ym of the object to be recognized are determined based on the foot position EN1 of the object N1.
in, the maximum width Xmax, and the maximum height Ymax are calculated. Minimum width Xmin, minimum height Ymi depending on foot position
n, the maximum width Xmax, and the maximum height Ymax are calculated by:
Equations 1 to 4 below are used. Formula 1 Xmin = X43 + ((X14−X43) / (E16
-E45) * (EN1-E45))-αx, where Xmin: minimum width at an arbitrary position, X43: width 43 of the deepest recognition object 42, X14: width 14 of the frontmost recognition object 13, E16: foot position 16 of the frontmost recognition object 13, E45: foot position 45 of the innermost recognition object 42, EN1: foot position of the detected object N _1, αx: constant (5) 2 Ymin = Y44 + ( (Y15-Y44) / (E16
-E45) * (EN1-E45))-αY where Ymin: minimum height at an arbitrary position, Y44: height 44 of the deepest recognition object 42, X15: height of the frontmost recognition object 13 E15: foot position 16 of the foreground recognition object 13; E45: foot position 45 of the farthest recognition object 42; EN1: foot position of the detection object N1; αY: constant (5) Equation 3 Xmax = X43 + ((X14-X43) / (E16
−E45) * (EN1-E45)) + αx where Xmax: maximum width at an arbitrary position, X43: width 43 of the deepest recognition object 42, X14: width 14 of the frontmost recognition object 13, E16. : Foot position 16 of the foreground recognition object 13, E45: foot position 45 of the backmost recognition object 42, foot position of the EN1 detected object N1, αx: constant (5) Equation 4 Ymax = Y44 + ((Y15- Y44) / (E16
−E45) * (EN1-E45)) + αY, where Ymax: maximum height at an arbitrary position, Y44: height 44 of the deepest recognition object 42, Y15: height of the frontmost recognition object 13 15, E16: foot position 16 of the foremost recognized object 13; E45: foot position 45 of the farthest recognized object 42; EN1: foot position of the detected object N1, αY: constant (5) size of the detected object In the determination step 19 (see FIG. 10), the horizontal width XN1 and the vertical width YN1 of the object N1 obtained in the labeling image 40 (see FIG. 11) and the size of the recognized object with respect to the foot position are obtained in the step 18-2. Minimum width Xmin, minimum height Ymin, maximum width X of the recognition object
By comparing max and the maximum height Ymax, it is determined whether the subject is a recognition target object. The processing of the size determination step 19 of the detected object is expressed by Expressions (7) and (8).
Becomes Equation 7 Minimum width Xmin of the recognized object ≦ width XN1 of the object N1 ≦
Maximum width of recognition object Xmax Equation 8 Minimum height of recognition object Ymin ≦ Height of object N1 YN1 ≦
The maximum height Ymax of the recognition object The recognition target object 21, that is, an intruder (in this case, a person) is determined when both of the conditions of Expressions 7 and 8 are satisfied by the object N1, and the other cases are the recognition targets. Outer object 20
(For example, an automobile).

[0008]

In the prior art described above, the size of one recognition target object, that is, the comparison reference, must be set at both the innermost and the lowermost positions on the screen. There is a problem that setting takes time. Since the reference is set at the innermost position and the lowermost position on the screen, if the object is located at an intermediate position between the two, the object to be recognized based on the two comparison standards is displayed on the screen. From the position, a comparison reference of the intermediate point is obtained, and a comparison operation with this comparison reference is performed. This comparison reference is obtained by linearly changing the image between the foreground image and the reference size of the innermost image, and obtaining from the position on the screen of the object to be recognized. In other words, for objects in the middle,
The position is approximately determined by the linear characteristic. As described above, the position of the object calculated in the object position calculation step in FIG. 10 is a position on the display screen, and is not always accurate. That is, the distance on the screen is the actual TV
May not match the distance between the camera and the object. To correct this, it is necessary to correct the linear characteristic in consideration of the angle-of-view characteristics of the lens of the TV camera, which requires a very laborious operation. That is, the conventional object recognition image processing apparatus has a problem in size conversion on the screen corresponding to the position of the object, and has a problem in that the accuracy is reduced depending on the distance from the TV camera. The purpose of the present invention is
The purpose is to simplify the setting of a reference pattern of one object to be recognized. Also, by using the foot position of the object and the angle from the camera to determine the size of the detected object, the distance is obtained, and converted to the size of the object at the foreground on the screen,
The purpose is to improve the accuracy of the size of the object.

[0009]

According to the present invention, in order to solve such a problem, an angle of a foot position of an object is used.
By converting the size of the detected object to the size of the object in front of the screen, the reference size of the recognition object is recognized using the size of the recognition object standing in front of the screen with one size Is made possible.

That is, the present invention firstly has a step of capturing a reference pattern of an object to be detected in advance and storing it in the image memory. Next, at the time of monitoring, the center of gravity of the object, which is the subject, is obtained by the step of obtaining the center of gravity of the object from the object captured by the TV camera, and the distance to the object is measured from the position of the center of gravity of the object and the elevation angle of the TV camera. . Next, the image of the object as the subject is converted into a size corresponding to the distance of the subject obtained in the step of measuring the distance. Next, the image of the subject converted into a size corresponding to this distance is compared with a pre-recorded reference pattern of a detection target to determine whether or not the image of the subject is a detection target object. Things.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In the following description of an embodiment, as in the prior art, an object recognition image which gives an alarm when a person enters a place where only a car is permitted and no person is allowed to pass. The processing device will be described. The hardware block configuration of the intruder recognition apparatus is shown in FIG. 5 as an example, which is common to the conventional example, and the entire flow of intruder recognition excluding the features of the present invention is shown in FIG. The operations from setting, updating, difference extraction, binarization, and labeling of the background image are common to those in the related art, and thus will be briefly described. In FIG. 5, the TV camera 22 captures an image of a fixed monitoring target area as described above. CPU 25, work memory 2
When an intruder is detected by the object recognition image processing in the computer 9, a warning is output to the warning display monitor 27 using the output I / F 26.

Next, the flow of object recognition will be described with reference to FIGS.
This will be described with reference to FIG. First, the initial background image creation step 1
Then, a background image is created and a background image 33 is obtained. The background image can be obtained by capturing an image of the monitoring range when no object exists. In image input step 2,
An input image 34 is obtained from the TV camera 22. In the difference step 3, the background image 33 created in the initial background creation step 1 or the background image update step 8 and the image input step 2
Then, a difference for each pixel from the input image 34 obtained in the above is calculated, and a difference image 35 is obtained. In the binarization step 4, the difference image 35 obtained in the difference step 3 is subjected to threshold processing, and the binary image 36
Get. In the noise removal step 5, the noise region is removed,
Obtain a labeling image. Hereinafter, the characteristic operation of the present invention will be described. Next, details of the object recognition step 7-1 will be described. In order to perform the object recognition step 7-1, it is necessary to set the size of the object to be recognized in advance. Regarding the setting of the size of the object to be recognized, FIG.
This will be described with reference to FIGS. An image of the reference screen for performing object recognition is fetched from the TV camera 22,
Is stored with the object recognition reference image 12 (see FIG. 3).
At this time, since the object to be recognized is a person, the object recognition image 12, that is, the person is positioned so as to be at the frontmost position on the screen, and the width 14 and the height 15 of the recognition object 13 (person) are set.
Is set as a pixel value. The above is the setting of the object recognition image performed before the monitoring.

The size of an object such as a person photographed on the screen differs depending on the position on the screen. Therefore, according to the present invention, the size of the object photographed during monitoring is converted into a size when photographed in the foreground on the screen, that is, a size when the reference image is captured. , Comparing the width 14 and the height 15 of the reference recognition object 13 with the size of the object being monitored. At this time, in order to fit the size of the monitored object to the reference image (that is, to fit the size of the object being shot at the foreground on the screen)
First, it is necessary to measure the distance to the object, which can be approximately obtained by obtaining the elevation angle of the TV camera and the center of gravity of the object.

A method for converting the size of a photographed object into the size when photographing is performed in front of the screen will be described with reference to FIGS. 1, 2, 3, 13, and 14. FIG.

The determination of whether or not an object imaged during monitoring is an object to be recognized will be described mainly with reference to FIG. In the object position calculation step 17, the labeling image 40 (FIG.
Center position GN1, width XN of the object N1 obtained by
1. Calculate the height YN1. In the step 18-1 for calculating the size of the recognized object with respect to the foot position, the width XN of the object is determined.
1. The height YN1 is converted into a width XN1 and a height YN1 of the object when the object is photographed in front of the screen. This conversion method will be described with reference to FIG. As shown in the figure, the vertical viewing angle Δθ of the TV camera 22 is obtained.
When the distance LA from the TV camera 22 to the point A and the distance LB from the camera to the point B are known, the angle θA to the point A and the angle θB to the point B are calculated by Expressions 9 and 1.
0 is obtained.

[0016]

(Equation 1)

Accordingly, the vertical viewing angle Δθ of the camera can be obtained by equation (11).

Equation 11 Δθ = θA−θB Next, the size of the object photographed on the screen is inversely proportional to the distance R from the camera to the object.

Here, the horizontal distance L to the object, the distance R to the object, and the angle .theta.

Equation 12 R = L / cos θ Since the width of the object photographed on the screen is inversely proportional to the distance R to the object, the width XN1 of the object on the screen and the image XN1 are photographed in front of the screen. Equation 13 has a relationship with the horizontal width XN1 'in this case.

Equation 13 XN1 ′ = XN1 × (cos θ / cos θA) Similarly, the height of the object photographed on the screen is inversely proportional to the distance R to the object, and therefore, the height YN1 of the object on the screen
And the height Y when it is photographed in front of the screen
N1 ′ has a relationship as shown in Expression 14.

Equation 14 YN1 ′ = YN1 × (cos θ / cos θA) In this manner, in step 18-1, the imaged object is moved to the front of the screen in accordance with the distance between the TV camera and the object. The size is converted into the size of the object when captured.

In the step 19 for determining the size of the detected object,
The width XN1 'and height YN1' of the object obtained in step 18-1 for converting the photographed object to the size when photographed in front of the screen and the recognized object in front of the screen Object 13 set in size setting step 11
By comparing the horizontal width 14 and the vertical width 15 of the (that is, the reference image), it is determined whether or not the object is a recognition object. The processing of the detection object size determination step 19 is expressed by Expressions 15 and 16.

Equation 15 Width of recognition object 14−αX ≦ Width of object XN1 ′ ≦ Width of recognition object 14 + αX Equation 16 Height of recognition object 15−αY ≦ Height of object YN1 ′ ≦ Height of recognition object 15 + αY When both of the conditions of Expression 15 and Expression 16 are satisfied by the object N1, the object is determined to be the recognition target object 20, and otherwise, it is determined to be the non-recognition object 21.

That is, according to Equations 15 and 16, FIG.
As shown in FIG. 7, in this embodiment, the object N1 is determined as an intruder (in this case, a person), and the others are determined as non-recognition-target objects 20 (in this case, automobiles). That is, when the recognition target object 21 is determined, the recognition target object condition 46 is established between the detection object N1 and the recognition object. Further, when it is determined that the object is a non-recognition object 20, a non-recognition object (automobile) condition 47 is established between the detection object N1 and the recognition object.

According to the above-described method, it is possible to perform the recognition process using the reference size of the recognized object as the size of the object located at the foreground on the screen.

[0027]

As described above, according to the present invention, the reference size of one object to be recognized only needs to be set to one size of the object located at the foreground on the screen. Is simple and can be performed in a short time.

Further, since the distance between the TV camera and the object is obtained, a reference image corresponding to the distance is obtained, and the judgment is made. Therefore, the intruding object can be judged with high accuracy.

[Brief description of the drawings]

FIG. 1 is a flowchart showing the principle of one embodiment of the present invention.

FIG. 2 is a flowchart showing the principle of initial setting of image recognition processing according to one embodiment of the present invention.

FIG. 3 is an explanatory diagram of the principle of initial setting of image recognition processing according to one embodiment of the present invention.

FIG. 4 is a flowchart illustrating the principle of image recognition processing according to one embodiment of the present invention.

FIG. 5 is a block diagram showing a hardware configuration.

FIG. 6 is a flowchart showing the principle of the conventional method.

FIG. 7 is an explanatory diagram of the principle of the conventional method.

FIG. 8 is a flowchart showing the principle of the initial setting of the conventional image recognition processing.

FIG. 9 is an explanatory diagram of the principle of the initial setting of the conventional image recognition processing.

FIG. 10 is a flowchart showing the principle of the conventional image recognition processing.

FIG. 11 is an explanatory diagram of the principle of calculating an object position.

FIG. 12 is an explanatory diagram of a recognition target object and a non-recognition object determination example.

FIG. 13 is an explanatory diagram of calculating the size of a recognized object with respect to a foot position according to the present invention.

FIG. 14 is an explanatory diagram of size conversion of a recognition object with respect to a foot position according to the present invention.

[Explanation of symbols]

1: Initial background creation step, 2: Image input step,
3: difference step, 4: binarization step, 5: noise removal step, 6: labeling step, 7-1, 7-
2: object recognition step, 8: background image update step,
9: object presence / absence determination step, 10: alarm processing step, 11: size setting step of the recognition object at the front of the image, 12: object recognition image, 13: recognition object at the front of the image , 14: the width of the foreground object on the screen, 15: the height of the foreground object on the screen,
16: Foot position of the recognition object at the foreground on the screen, 1
7: object position calculation step; 18-1: step of converting the photographed object into a size when photographed at the foreground on the screen; 18-2: object size calculation step with respect to the foot position; 19 ... Detection object size determination step,
20: object not to be recognized (judgment result), 21: object to be recognized (judgment result), 22: TV camera, 23: image input I
/ F, 24: image memory, 25: CPU, 26: output I
/ F, 27: alarm display monitor, 28: data bus, 29
... Work memory, 30 ... Program memory, 31 ... Image output I / F, 32: Video monitor, 33 ... Background image, 3
4: input image, 35: difference image, 36: binarized image, 3
7: noise area, 38: noise-removed image, 39: detection target object, 40: labeling image, N ₁ : detected object number, 41: size setting step of the recognized object at the innermost part of the screen, 42: on the image , 43: the width of the deepest recognized object on the screen, 44: the height of the deepest recognized object on the screen, 45: the foot of the deepest recognized object on the screen Position, XN ₁ : width of object N ₁ , YN ₁ : height of object N ₁ , EN ₁ : foot position of object N ₁ , GN ₁ : center of gravity of object N ₁ , 46: object condition diagram for recognition, 47 : Condition diagram of non-recognition object, A, B: field of view of camera, C: existing position of object, LA, LB: distance from camera to each point, θA: angle at point A, θB: angle at point B Angle, Δθ: horizontal viewing angle of camera, θ: angle between camera and object, R: distance from camera Separation, XN ₁ ′: horizontal width after conversion, YN ₁ ′: height after conversion, GN ₁ ′: position of center of gravity of converted object, 48: converted image of object size.

 ──────────────────────────────────────────────────続 き Continued on the front page F-term (reference) 2F065 AA03 AA07 AA17 AA23 CC16 FF04 FF09 JJ03 JJ26 QQ00 QQ04 QQ24 QQ25 QQ34 RR02 RR05 SS02 SS09 UU05 5C054 FC01 FC05 FC12 FC15 FD07 GA04 GB01 GB12 GG11 AGG5A08 GG78 5L096 BA02 CA04 DA03 EA03 EA05 EA43 FA60 FA66 GA08 GA34 HA03 HA08 JA09 9A001 BZ03 BZ04 EE05 HH21 HH24 HH27 JJ71 KK37

Claims (4)

[Claims] 1. A video signal obtained from a television camera that captures an image of an area to be monitored is recorded in an image memory, subjected to image processing using a CPU, and automatically determined whether or not the subject is a detection target object. In the monitoring image processing for recognition, in advance, a reference pattern of the detection target object is imaged, held in the image memory, and the center of gravity of the object is obtained from an image of the object captured by the television camera during monitoring,
The distance to the subject is measured from the position of the center of gravity of the subject and the elevation angle of the television camera, and based on the result, the image of the subject is converted into a size corresponding to the distance to the subject. An object recognition image processing method comprising: comparing an image of the object with a reference pattern of a detection target stored in a memory in advance to determine whether or not the image of the subject is the detection target object. 2. A video signal obtained from a television camera that captures an image of an area to be monitored is recorded in a plurality of image memories, and processed between the image memories using a CPU. A monitoring image processing method for automatically recognizing whether or not the object is a detection target object, such as a person, comprising: previously capturing a reference pattern of the detection target object and storing the reference pattern in the image memory; Obtaining the center of gravity of the subject from the image of the subject captured in the step, measuring the distance to the subject from the position of the center of gravity of the subject and the elevation angle of the television camera, and measuring the distance from the image of the subject Converting the size of the subject to the size corresponding to the distance obtained in the step Object recognition image processing method characterized by having a determining to whether or not the image of the object is a detection target object compared with the reference pattern of the detection target. 3. A television camera that captures an image of an area to be monitored, a plurality of image memories that hold video signals obtained from the television camera, and a CPU that controls at least the plurality of memories, In a monitoring image processing apparatus for automatically recognizing whether a subject is a detection target object, a step of previously capturing a reference pattern of the detection target object and storing the reference pattern in the image memory; Determining the center of gravity of the subject from the captured image of the subject, measuring the distance to the subject from the position of the center of gravity of the subject and the elevation angle of the television camera, and measuring the distance from the image of the subject Converting the object into a size corresponding to the distance of the subject obtained in the step; and an image of the subject converted into a size corresponding to the distance. Object recognition image processing method characterized by serial prerecorded comparing the magnitude of the reference of the detection object image of the subject and a step of determining whether or not a detection target object. 4. The object recognition image processing method according to claim 3, wherein the reference of the size of the detection target is defined by a minimum value and a maximum value of a width and a minimum value and a maximum value of a height.