JP2000175174A - Surveillance image processing device - Google Patents

Abstract

(57) [Summary] In a conventional monitoring image processing apparatus, a recognition processing unit performs image recognition using a normal difference processing unit as a background difference, and detects an event to be notified. In addition, the recognition processing unit that has detected the event changes the difference processing unit to the inter-frame difference, performs image recognition again, and determines whether the event is true,
It was fired when it was true. That is, a delay occurs between the video signal used for the background difference processing and the inter-frame difference processing, and the recognition accuracy is reduced. A difference processing unit performs background subtraction on a video signal output from an A / D conversion unit.
Then, an inter-frame difference is performed. Further, in the recognition processing unit 10a, when an event is detected on the difference processing unit 3 side, the difference processing unit 20 determines whether or not the event is true, and outputs an event detection when the event is true. Accordingly, the recognition processing unit 10a can perform the recognition process using the difference data generated from the same video frame, and thus the recognition accuracy is improved.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a surveillance image processing apparatus, and more particularly, to a method for reducing a delay difference between a video signal used in a process of recognizing a notification target and a process of confirming the notification target recognized in the recognition process. And a monitoring image processing apparatus for improving recognition accuracy.

[0002]

2. Description of the Related Art FIG. 6 is an overall block diagram of a conventional monitoring image processing apparatus. In the figure, 1 is a monitoring target (not shown)
A video input unit for inputting a video signal obtained by capturing an image or the like, an A / D converter 2 for quantizing the video signal from the video input unit 1 and outputting a digital video signal, and a digital video signal 3 from the A / D converter 2 A difference processing unit that receives a signal, compares two images of a current image and a background image for comparison (calculated and updated according to a certain rule), and outputs difference data representing a difference between the two images;
Reference numeral 4 denotes a threshold value calculation unit that outputs a threshold value for binarizing the difference data from the difference processing unit 3, and generally calculates an optimum threshold value based on the difference data.

[0005] Reference numeral 5 denotes a threshold value calculated by the threshold value calculation unit 4 for binarizing the difference data and outputting binary data.
The binarizing unit 6 performs a contraction / expansion process to remove noise.
A filter section for removing isolated points, lines, holes, etc., 7 is a labeling section for individually numbering all the constituent pixels of the change area serving as difference data, and 8 is an input of labeled binary data from the labeling section 7. A feature value calculation unit 9 for calculating feature values by calculation; a report feature value recording unit 9 for recording feature values of a report target (eg, a person, a vehicle, etc.); 10 a feature output from the feature value calculation unit 8 The recognition processing unit 11 compares the amount with the feature amount in the notification target feature amount recording unit 9, recognizes an event to be emitted when approximated, and outputs an event notification, and 11 is an event from the recognition processing unit 10. Issued in response to notification (eg: BEE
(A P sound or the like).

FIG. 7 is an internal configuration diagram of the difference processing section 3 shown in FIG. In the figure, reference numeral 15 denotes a current image storage unit for inputting and recording a digital video signal from the A / D conversion unit 2,
Reference numeral 16 denotes a background image storage unit for storing a background image for comparison, which is created by performing a corresponding operation on the digital video signal input from the A / D conversion unit 2. Is a background updating unit that calculates and updates the background image, and generally updates the image according to a simple rule (for example, an average of the past five current images is used as the background image). A difference unit 18 compares the two images of the current image storage unit 15 and the background image storage unit 16 and calculates a difference between the two images. The difference unit 18 outputs the difference data to the threshold value calculation unit 4 shown in FIG. .

Next, the operation will be described with reference to FIGS. A video signal obtained by capturing an image of a monitoring target or the like is output to an A / D converter 2 via a video input unit 1. The A / D converter 2 digitizes the video signal and converts the digital video signal into a difference processing unit 3
Output to 8 per pixel as digital video signal
It is common to convert to bit multi-valued data.

In the difference processing unit 3 to which the digital video signal is input, as shown in FIG. 7, the current image storage unit 15 stores the input digital video signal as the current image data and prepares for the subsequent processing. The background image storage unit 16 stores a background image created based on the input current image data and prepares for subsequent processing. In addition, the background update unit 17 includes the background image storage unit 1
6 is periodically updated according to a certain rule. This update processing is generally performed on the basis of the current image that is periodically sampled. For example, a method of using the average value of the current images of the past five scenes as a background image, a method of setting the average value of the current image of It is often a frame image. The difference unit 18 includes the current image storage unit 15 and the background image storage unit 1.
6 is compared, a difference is obtained for each pixel, and the difference data as the difference is output as a frame image.

[0007] The difference data used here generally represents a portion of the video signal where the luminance has changed. Normally, moving objects such as people and vehicles are mainly used, but in rare cases, the brightness of non-reporting objects such as "appearance of shadows due to sunlight shading" and "reflection of headlights of car at night". The change may be output as difference data. Further, the threshold value calculation unit 4
A threshold value for binarizing the difference data output by the difference processing unit 3 is obtained, and the difference data and the threshold value are output to the binarization unit 5. Normally, the difference data output from the difference processing unit 3 is multi-valued data, which is inappropriate for the recognition processing as it is. In general, binarization and conversion into binary data are also performed in order to make subsequent determinations at high speed. However, there is a case where the difference data is directly recognized and processed without performing the binarization depending on the device, so this is a reference.

That is, the binarizing section 5 outputs binary data obtained by binarizing the input difference data with the above threshold value. As a result, in the subsequent processing, the difference data serving as the change area becomes binary data. However, there is a case where the difference data is directly recognized and processed without performing the binarization depending on the device, so this is a reference.

Here, a method of detecting a change area serving as difference data will be described with reference to FIGS. FIG. 8 is an explanatory diagram showing an example of a video signal stored in the original image storing unit 15 shown in FIG. 7, and FIG. 9 is an explanatory diagram in a case where a difference data area output by the difference processing unit 3 is superimposed. That is, the change area in FIG. 8 is the area of the moving person, and becomes superimposed as shown in FIG. In FIG.
The portion stained black is a change area output as difference data. The circumscribed rectangle of the change area is the Feret diameter.

The filter unit 6 performs a contraction / expansion process for removing noise on the input binary data.
The labeling unit 7 assigns individual numbers to all the constituent pixels of the change area. The binary data processed by the filter unit 6 and the labeling unit 7 is output to the feature value calculation unit 8.

The feature value calculator 8 calculates the feature value of binary data indicating the input change area. Now, as an example, this feature amount is assumed to be “area”, “length and width”, and “speed”. Also,
The feature amount calculation unit 8 calculates the “area” and “length and width” calculated above.
A feature amount such as “speed” is output to the recognition processing unit 10. In addition, the feature quantity recording unit 9 records feature quantities of an object (e.g., a person or a vehicle) to which the apparatus is to issue a notice in advance.

The recognition processing section 10 includes a feature amount calculating section 8
Is compared with the feature amount recorded in the notification target feature amount recording unit 9, and when they are close to each other, it is recognized as an event to be emitted, and an event notification is output to the alerting unit 11. . In addition, when the event notification is input, the notification unit 11 performs an external notification process (not shown). Specifically, it is a process of sending a specific signal to a higher-level PC or the like or sounding a buzzer of the device.

That is, in the conventional monitoring image processing apparatus,
The feature amount calculation unit 8 inputs the difference data indicating the change area output by the difference processing unit 3, calculates the feature amount, and outputs the result to the recognition processing unit 10, and the recognition processing unit 10 receives the input from the feature amount calculation unit 8. When the approximated feature amount and the feature amount recorded in the alert target feature amount recording unit 9 are compared and approximated to each other, the event is recognized as an event to be emitted, and an event notification is output to the alerting unit 11 to perform an alerting process. Is done.

However, in the above-described processing, for example, when the difference processing unit 3 captures the shadow of the building due to the shade of sunlight as a change area and outputs it as difference data,
When the recognition processing unit 10 recognizes that the shadow feature amount and the feature amount recorded in the notification target feature amount recording unit 9 are similar, an event notification may be output to the notification unit 11. Hereinafter, such an event is referred to as a misrecognition event.

In the conventional monitoring image processing apparatus, the following operation is performed before the recognition processing unit 10 outputs the event notification to the alarm unit 11 in order to suppress the output of the erroneous recognition event. The above event is held as a provisional event, the parameter of the difference processing unit 3 is changed, for example, for inter-frame difference, the process is performed, and the output result of the feature value calculation unit 8 is waited. For example, when the tentative event is caused by a movement of an object or the like, the feature amount output from the feature amount calculation unit 8 is substantially the same as the feature amount output as the background difference. If the temporary event is a shadow of a building due to the shading of sunlight, the feature amount output from the feature amount calculation unit 8 is considerably smaller than the feature amount output as the background difference because no movement is involved. .

That is, the recognition processing unit 10 compares the feature amount extracted by the difference data based on the background difference with the feature amount extracted by the difference data based on the inter-frame difference. And outputs an event notification to the alerting unit 11. If the feature value extracted by the difference data based on the background difference is larger than the feature value extracted by the difference data based on the inter-frame difference by a predetermined amount, the operation of recognizing the event as a misrecognition event and outputting the event notification to the notification unit 11 is stopped. I do.

Thus, in this monitoring image apparatus, the recognition accuracy is improved by detecting an event of erroneously recognizing the shadow of a building caused by, for example, sunlight.

[0018]

The conventional monitoring image processing apparatus is configured as described above, and has the following problems. (1) In order to prevent the output of an event notification due to an erroneous recognition event, the recognition processing unit 10 changes the processing of the difference processing unit 3 to inter-frame difference processing when detecting an event with a feature amount due to a background difference, and again performs the feature amount. And the output of the event notification is stopped as a misrecognition event when the difference is a predetermined amount. However, in this method, a video signal used for the inter-frame difference uses a video signal delayed by a predetermined frame from the video signal used for the background difference, and there is a problem that recognition accuracy is reduced.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and a first object of the present invention is to provide a delay difference between a video signal used as a background difference and a video signal used as an inter-frame difference. It is intended to provide a monitoring image apparatus in which recognition accuracy is improved by reducing the number of images.

[0020]

A monitoring image apparatus according to the present invention includes a video signal input section for inputting a video signal obtained by capturing an image of a monitoring target, and performing a first difference process on the input video signal. A first difference processing unit that outputs the first difference data, and a second difference processing unit that is arranged in parallel with the first difference processing unit and outputs the second difference data to the same frame as the video signal frame processed by the first difference processing unit at the same time. A second difference processing unit that performs difference processing and outputs second difference data, a first feature amount extraction unit that receives the first difference data and outputs a first feature amount, and receives the second difference data A second feature value extraction unit that outputs a second feature value, and a recognition processing unit that recognizes an alarm target based on the first feature value and the second feature value.

Further, a monitoring image apparatus according to the following invention is provided.
A video signal input unit for inputting a video signal obtained by capturing an image of a monitoring target; and a difference for alternately performing different processing on the input video signal for each frame and alternately outputting first difference data and second difference data. A processing unit, a feature amount extraction unit that outputs a first feature amount when the first difference data is input, and outputs a second feature amount when the second difference data is input, and the first feature amount And a recognition processing unit for recognizing the alarm target based on the second feature amount.

Further, a monitoring image apparatus according to the next invention is provided with:
The difference processing unit includes a first difference processing unit that outputs the first difference data and a second difference processing unit that outputs the second difference data in parallel, and the first difference processing unit and the second difference processing unit output the second difference data. A selector is provided at the output destination, and the selector alternately outputs the first difference data and the second difference data.

Further, the monitoring image apparatus according to the following invention is provided.
The recognition processing unit recognizes the notification target using the first feature amount, and
The notification target recognized by the feature amount is confirmed by the second feature amount.

[0024]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiment 1 Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is an overall configuration diagram of a monitoring image processing apparatus according to Embodiment 1 of the present invention. The same reference numerals as in FIG. 6 denote the same or equivalent functions, and a description thereof will be omitted. In the figure, 20 is a difference processing unit, 21 is a threshold value calculation unit, 22 is a binarization unit, 23 is a filter unit,
4 is a labeling unit. 8a is a feature amount calculation unit,
10a is a recognition processing unit.

Next, the operation of the monitoring image processing apparatus shown in FIG. 1 will be described. The operations of the video input unit 1 to the labeling unit 7, the notification target feature amount recording unit 9 and the notification unit 11 are the same as those of FIG. It is the same and the description is omitted. In the monitoring image processing apparatus shown in FIG. 1, the difference processing unit 20 is arranged in parallel with the difference processing unit 3. The difference processing unit 20 performs an inter-frame difference process on the digital video signal input from the A / D conversion unit 2 and outputs difference data to the threshold value calculation unit 21.

That is, the difference processing unit 3 and the difference processing unit 20
Processes the same-frame digital video signal at the same time and outputs respective difference data. In addition, the processes of the threshold value calculation units 21 to the labeling unit 24 are the same as the processes of the threshold value calculation units 4 to the labeling unit 7, and the description is omitted. In addition, the feature amount calculation unit 8a outputs the 2
A feature amount of difference data serving as value data (hereinafter referred to as a background difference feature amount) and a feature amount of difference data serving as binary data output by the labeling unit 24 (hereinafter referred to as an inter-frame difference feature amount). The calculated value is output to the recognition processing unit 10a.

The recognition processing section 10 a normally compares the background difference feature quantity with the feature quantity to be reported which is recorded in the report feature quantity recording section 9. If they are similar, the background difference feature amount and the inter-frame difference feature amount are further compared, and if they are similar, the event is recognized as an event to be emitted, and an event notification is output to the alerting unit 11. . This operation will be described in more detail with reference to FIG. FIG. 2 is an operation comparison diagram of the monitoring image processing apparatus of FIG. 1 and FIG. 6, and FIG.
2 explains the operation of the monitoring image processing apparatus shown in FIG.
(2) describes the operation of the monitoring image processing apparatus shown in FIG. A indicates background difference data output in the background difference processing, and B indicates inter-frame difference data output in the inter-frame difference processing. A1 is the background difference feature amount for the background difference data A, and B1 is the inter-frame difference data B
Shows the inter-frame difference feature amount with respect to.

In FIG. 2A, the difference processing section 3 outputs the inter-frame difference data B x frames after outputting the background difference data A. Also, the inter-frame difference feature amount B1 is output x frames after the feature amount calculation unit 8 outputs the background difference feature amount A1. On the other hand, in FIG. 2B, the difference processing unit 20 outputs the background difference data A, and the difference processing unit 20 outputs the inter-frame difference data B at the same time. In addition, the feature amount calculation unit 8a simultaneously outputs the background difference feature amount A1 and the inter-frame difference feature amount B1 after x frames.

That is, in the conventional monitoring image processing apparatus shown in FIG. 6, as shown in FIG. 2A, the difference between the background difference data A for event recognition and the inter-frame difference data B used for confirmation is determined. A delay corresponding to x frames occurred during that time, and the recognition accuracy was low because image recognition processing was performed using different video signals. In addition, when the recognition processing unit 8 outputs an event notification, the event difference is output after the input of the background difference data A 2 × frames, and the processing time until the event notification is output has been long.

On the other hand, in the monitoring image processing apparatus according to the first embodiment shown in FIG. 1, as shown in FIG.
Since the difference processing unit 20 outputs the inter-frame difference data B, there is no delay in the video signal in which the background difference data A and the inter-frame difference data B are created, so that the recognition accuracy is improved. Further, when the recognition processing unit 8 outputs an event notification, it becomes possible to output the event difference target background difference data A x frames after the input, and the recognition time can be reduced to half.

As described above, in the monitoring image processing apparatus shown in FIG. 1, the recognition target can be recognized and confirmed by the same video signal, so that the recognition accuracy is improved. Further, the processing time can be reduced because of the parallel processing. In the above description, the difference processing unit 3 uses the background difference processing, and the difference processing unit 20
Has been described using the inter-frame difference processing. However, the recognition processing unit 10a can recognize the notification target by the difference feature amount obtained from the difference processing unit 3, and It goes without saying that any difference processing method may be used as long as the object can be recognized.

Embodiment 2 FIG. FIG. 3 is an overall configuration diagram of a monitoring image processing apparatus according to Embodiment 2 of the present invention. The same reference numerals as in FIG. 6 denote the same or equivalent functions, and a description thereof will be omitted. In the figure, 30 is a difference processing unit, 8b is a feature amount calculation unit, and 10b is a recognition processing unit. FIG. 4 is a block diagram of the difference processing unit 30 shown in FIG. 3, and the same reference numerals as those in FIG. 7 denote the same or equivalent functions, and a description thereof will be omitted. In the figure,
31 is a current image storage unit, 32 is a background image storage unit, 33 is a background update unit, 34 is a difference unit, and 35 is a selector. The current image storage unit 15 to the difference unit 18 are referred to as a first difference block, and the current image storage unit 31 to the difference unit 34 are referred to as a second difference block.
It is called a difference block.

Next, the operation of the monitoring image processing apparatus shown in FIG. 1 will be described. The video input unit 1, the A / D conversion unit 2, the threshold value calculation unit 4 to the labeling unit 7, The operations of the recording unit 9 and the alarm unit 11 are the same as those in FIG. The difference processing unit 30 to which the digital video signal is input from the A / D conversion unit 2 is output to each of the first difference block and the second difference block as shown in FIG. Also, the first
The operation of the difference block is the same as the operation with the difference processing unit 3 shown in FIG. 7, and the background difference data A is output to the selector 35. In the second difference block, the background update unit 33 does not update the background data if the difference is an inter-frame difference. That is, the digital video signal is directly stored in the background image storage unit 32.

Thus, the inter-frame difference processing is performed in the second difference block, and the inter-frame difference data B is output to the selector 35. The selector 35 alternately selects the input background difference data A and the inter-frame difference data B for each frame and outputs the data to the threshold value calculation unit 4. That is, the background difference data A and the inter-frame difference data B are alternately output to the threshold value calculation unit 4 for each frame. In addition, the processes of the threshold value calculation unit 4 to the labeling unit 7 are the same as those in FIG.

The feature calculating unit 8b converts the background difference feature A1 and the inter-frame difference feature B1 into one frame based on the background difference data A and the inter-frame difference data B which are alternately input and labeled. Output alternately every time.
In addition, the recognition processing unit 10a normally uses the background difference feature amount A1.
And the feature amount to be reported, which is recorded in the report target feature amount recording unit 9. If they are similar, the background difference feature value A1 and the inter-frame difference feature value B1 are further compared. If they are similar, they are recognized as an event and an event notification is output to the alert unit 11. This operation will be described in more detail with reference to FIG.

FIG. 5 is a diagram for explaining the operation of the monitoring image processing apparatus shown in FIG. 3. As shown in FIG. 5, the difference processing unit 30 outputs one background difference data A and one interframe difference data B.
It is output alternately for each frame. In addition, the feature value calculation unit 8
From b, the background difference feature amount A1 and the inter-frame difference feature amount B1 are alternately output for each frame. That is, the delay of the video signal for generating the background difference data A and the inter-frame difference data B is only one frame, and the recognition accuracy is improved compared to the monitoring image processing apparatus shown in FIG. Further, the period from when the difference processing unit 30 outputs the background difference data A to when the feature value calculation unit 8b outputs the inter-frame difference feature value B1 is (x + 1) frames, and the monitoring image processing shown in FIG. The processing time is shorter than that of the apparatus.

In addition, as compared with the monitoring image processing apparatus shown in FIG. 1, there is no need to provide the threshold value calculation section 21 to the labeling section 24, so that the configuration can be made inexpensively and the power consumption can be reduced.

[0038]

According to the present invention, the surveillance image processing apparatus performs the first difference processing on the video signal input by the first difference processing unit and outputs the first difference data, and outputs the first difference data. A second difference processing unit arranged in parallel with the processing unit performs second difference processing on the same frame as the video signal frame processed by the first difference processing unit at the same time and outputs second difference data A first feature value extraction unit inputs the first difference data and outputs a first feature value, and a second feature value extraction unit inputs the second difference data and outputs a second feature value, Since the recognition processing unit recognizes the notification target based on the first feature amount and the second feature amount, the first feature amount and the second feature amount can be generated from the same frame, so that the recognition accuracy is improved.

According to the next invention, the surveillance image processing device alternately performs different difference processing on the video signal input by the difference processing section for each frame to obtain the first difference data and the second difference data. Are output alternately, and the feature extraction unit outputs the first feature when the first difference data is input, and outputs the second feature when the second difference data is input. Recognizes an alarm target based on the first feature amount and the second feature amount, so that the first feature amount and the second feature amount can be generated from alternate frames, and thus has an effect of improving recognition accuracy.

According to the next invention, in the monitoring image processing apparatus, the difference processing unit includes a first difference processing unit that outputs first difference data and a second difference processing unit that outputs second difference data. Exist in parallel, and a selector is provided at an output destination of the first difference processing unit and the second difference processing unit. The selector alternately outputs the first difference data and the second difference data, and outputs the data. There is an effect that the first difference data and the second difference data can be output every other frame.

Further, according to the next invention, in the monitoring image processing apparatus, the recognition processing unit recognizes the notification target using the first feature amount, and recognizes the notification target recognized using the first feature amount as the second notification object. Since the confirmation is performed using the feature amount, there is an effect that the processing speed is improved.

[Brief description of the drawings]

FIG. 1 is an overall configuration diagram of a monitoring image processing apparatus according to a first embodiment.

2 is an operation comparison diagram of the monitoring image processing apparatus shown in FIG. 6 and FIG. 1; FIG. 2 (1) explains the operation of the monitoring image processing apparatus shown in FIG. 6; The operation of the monitoring image processing apparatus shown in FIG. 1 will be described.

FIG. 3 is an overall configuration diagram of a monitoring image processing apparatus according to a second embodiment;

FIG. 4 is a configuration diagram of a difference processing unit shown in FIG. 3;

FIG. 5 is an operation explanatory diagram of the monitoring image processing apparatus shown in FIG. 3;

FIG. 6 is an overall configuration diagram of a conventional monitoring image processing apparatus.

7 is an internal configuration diagram of the difference processing unit shown in FIG.

8 is an explanatory diagram showing an example of a video signal stored in an original image storage unit 15 shown in FIG.

9 is an explanatory diagram in a case where a region of difference data output by the difference processing unit illustrated in FIG. 6 is superimposed.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Video input part 2 A / D conversion part 3, 20, 30 Difference processing part 4, 21 Threshold calculation part 5, 22 Binarization part 6, 23 Filter part 7, 24 Labeling part 8a, 8b Feature amount calculation part 9 Reporting feature amount recording unit 10a, 10b Recognition processing unit 15, 31 Current image storage unit 16, 32 Background image storage unit 17, 33 Background update unit 18, 34 Difference unit 35 Selector

Claims (4)

[Claims] A video signal input unit that inputs a video signal obtained by capturing an image of a monitoring target; a first difference processing unit that performs first difference processing on the input video signal and outputs first difference data; A second difference processing unit that is arranged in parallel with the first difference processing unit, performs the second difference processing on the same frame as the video signal frame processed by the first difference processing unit at the same time, and outputs second difference data. A difference processing unit, a first feature amount extraction unit that receives the first difference data and outputs a first feature amount, and a second feature amount extraction that receives the second difference data and outputs a second feature amount And a recognition processing unit for recognizing an alarm target based on the first feature amount and the second feature amount. 2. A video signal input unit for inputting a video signal obtained by capturing an image of a monitoring target, and performing a different difference process on the input video signal alternately for each frame to generate first difference data and second difference data. A difference processing unit for alternately outputting, and a first feature value when the first difference data is input, and a second feature value when the second difference data is input.
A monitoring image processing apparatus, comprising: a feature amount extraction unit that outputs a feature amount; and a recognition processing unit that recognizes an alarm target based on the first feature amount and the second feature amount. 3. The difference processing unit includes a first difference processing unit that outputs first difference data and a second difference processing unit that outputs second difference data in parallel. 3. The monitoring image processing according to claim 2, wherein a selector is provided at an output destination of the two difference processing unit, and the selector alternately outputs the first difference data and the second difference data by the selector. apparatus. 4. The apparatus according to claim 1, wherein the recognition processing unit recognizes the notification target using the first feature amount, and confirms the notification target recognized using the first feature amount using the second feature amount. Item 4. The monitoring image processing device according to any one of Items 3 to 3.