JP2000113158A - Object tracking apparatus and method - Google Patents

Abstract

(57) [Problem] To provide an object tracking device capable of suppressing a template shift without losing the effect of sequentially updating a template when performing object tracking using a template matching method while sequentially updating a template. I do. SOLUTION: The position at the time of cutting out the template is stored in a template position storage means 22, and a correlation processing means 2 is provided.
3, the moving amount calculating means 24 obtains the moving amount of the template by comparing with the position obtained as a result of the correlation processing, and updates the template if the moving amount of the object is equal to or more than a predetermined condition. However, if the condition is less than a predetermined condition, the template is not updated.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing technique for a traffic flow monitoring system that automates a monitoring camera installed on a road by image processing, performs automatic measurement of a traffic flow, and automatically detects an unexpected event. The present invention relates to an object tracking device and an object tracking method for tracking an area of a car running on a road by image processing.

Further, an object tracking apparatus and object tracking by image processing which can be widely applied to tracking of a person in a monitoring camera in a department store yard, a monitoring camera in a station yard, or the like, a face or a hand used in a human interface, and the like. About the method.

[0003]

2. Description of the Related Art In recent years, there has been a demand for the development of a monitoring system that automates a surveillance camera installed on a highway, a highway, or the like by image processing to automatically measure a traffic flow or automatically detect an unexpected event. .

A particularly important image processing technique for traffic flow monitoring is the technique of accurately tracking vehicles in an image. If the individual vehicles can be accurately tracked in the image, a great deal of information such as the basic information such as the speed and running trajectory of each vehicle, the difference in traffic volume for each lane, the frequency of lane change, the traffic congestion status, etc. can be obtained.

[0005] To track a vehicle in an image, a template matching technique is often used. Template matching is a method of cutting out an image area of a vehicle at a certain time and using this as a basis, that is, as a template, finding a position that best fits the template in an image at a different time by a correlation operation or the like, and applying the position of the vehicle. It is.

There are two methods for setting a template.

A first method is a method in which when a vehicle enters an image, a vehicle region is cut out, and the same template is used as a template for all input images to perform matching. This can be called the “fixed template method”.

The second method is a method of updating a template every time an image is input, that is, a method of matching in a next image using a previous image extraction result as a template. This can be said to be the "update template method".

When a vehicle is tracked in an image having a depth as in traffic flow monitoring, the apparent size or shape of the vehicle changes depending on the depth position of the vehicle, and the vehicle moves from a sunlit area to a shadow area of a building. May move and the apparent brightness of the vehicle may change. In addition, the lighting environment may change due to the movement of the sun and clouds, which also changes the appearance of the vehicle. Therefore, since the appearance of the vehicle gradually changes from the time of intrusion, the update template method is often used in traffic flow monitoring as shown in the following literature.

Document 1: Kitamura: "Image monitoring method, image monitoring device and road monitoring device" in JP-A-9-282455, Document 2: Morita: "Image feature tracking device and image feature tracking" in JP-A-9-214945. Method ", Document 3: Takahashi, Kitamura, Sato: JP-A-10-154292
No. "Traffic flow monitoring device".

However, the update template method has a disadvantage that it is weak for stationary objects. Even when the vehicle is stopped, the image characteristics of the input image slightly fluctuate due to noise in the video signal, a change in the lighting environment, a change in the image due to the operation of the automatic aperture mechanism of the camera, and the like. Because of this variation,
The result of template matching of a stationary vehicle may move small (for example, about one pixel) in calculation.

When the movement of the template with respect to the stationary vehicle is repeated as described above, the position of the template gradually shifts from the vehicle. However, when the template is updated, only the position having the highest correlation with the previous position is output at any point in time, and it cannot be recognized that the position is gradually shifted in the calculation.

This phenomenon often occurs when the calculated image feature amount of the vehicle is small, such as when the vehicle body is smooth and has no pattern, or when the vehicle is apparently small. This is because the change in the image feature amount due to a change in the environment or noise becomes larger than the image feature amount in the calculation of the vehicle. And such a situation occurs frequently. This is because the vehicle often stops due to traffic jams, the vehicle body is often smooth, and the vehicle is often far away.

[0014] Although such a small deviation exists with respect to a moving vehicle, the ratio of the small deviation is smaller than the moving amount in the image of the vehicle, and the position of the vehicle in the image changes. Since the direction in which the minute shift occurs is also dispersed with the shift, the problem is small because the accumulated shift gradually approaches 0.

The fixed template method does not cause such a phenomenon that the vehicle gradually deviates, but it is not suitable when the apparent size or shape of the vehicle changes or the environment changes. It is on the street.

The problem of the update template method described above is not limited to vehicle tracking for traffic flow monitoring, but is also applicable to various applications such as tracking of persons, tracking of faces and hands, tracking of flying objects, and tracking of animals. A common problem that applies.

[0017] Of course, this is a common problem that also applies to situations in which the surrounding environment is photographed from a vehicle-mounted camera and objects in the environment are tracked by the updated template method.

[0018]

As described above, when the vehicle is tracked by the updated template matching, there is a problem that the template may be gradually shifted with respect to the stationary vehicle.

Therefore, an object of the present invention is as follows.
It is an object of the present invention to provide an object tracking device and an object tracking method capable of suppressing a phenomenon in which a template is gradually shifted without impairing the advantage of the updated template matching.

[0020]

According to a first aspect of the present invention, there is provided an object tracking apparatus for tracking an object by image processing, wherein an image area including an object to be tracked is cut out from an input image as a reference template, and the reference area is included in the reference template. A template image storage unit for storing an image of the reference template, a template position storage unit for storing a cutout position of the reference template in the input image as a reference position of the reference template, and a new input including new information of the object to be tracked. A correlation processing unit for obtaining a reference position of the reference template in the new input image based on the image and the reference template; and the reference position stored in the template position storage unit and the reference position obtained by the correlation processing unit. From the position, the movement amount of the collation position with respect to the reference position is calculated. A moving amount calculating unit that updates the reference position to the matching position if the moving amount is equal to or greater than a predetermined condition, and sets the reference position to the matching position if the moving amount is less than a predetermined condition. An object tracking device, comprising: an updating unit that does not update;

According to a second aspect of the present invention, when the moving amount is equal to or more than a predetermined condition, the updating unit updates the reference position to the collation position, and updates the reference position from the new input image to the collation position. A new template is cut out based on the reference template, the image in the reference template is updated to an image in the new template, and if the moving amount is less than a predetermined condition, the reference position is not updated and the reference is not updated. An object tracking device characterized by not updating an image in a template.

According to a third aspect of the present invention, the update section updates the reference position to the collation position and updates the reference position from the new input image to the collation position if the movement amount is equal to or greater than a predetermined condition. A new template is cut out based on the reference template, and the image in the reference template is updated to an image in the new template. If the moving amount is less than a predetermined condition, the reference position is not updated and the new position is not updated. An object tracking apparatus characterized in that a new template is cut out from an input image based on the collation position, and an image in the reference template is updated to an image in the new template.

According to a fourth aspect of the present invention, there is provided an object tracking method for tracking an object by image processing, wherein an image area including an object to be tracked is cut out from an input image as a reference template, and a template for storing an image in the reference template is stored. An image storing step, a template position storing step of storing a cutout position of the reference template in the input image as a reference position of the reference template, and a new input image including the new information of the object to be tracked and the reference template. And a correlation processing step for obtaining a reference position of the reference template in the new input image based on the reference position, and the reference position stored in the template position storage step and the reference position obtained by the correlation processing step. The collation position for the position A moving amount calculating step of calculating the moving amount of the reference position, and updating the reference position to the collation position if the moving amount is equal to or more than a predetermined condition, and updating the reference position if the moving amount is less than a predetermined condition. And updating the object to the collation position.

According to a fifth aspect of the present invention, there is provided a recording medium storing a program for realizing an object tracking method for tracking an object by image processing, wherein an image area including an object to be tracked is cut out from an input image as a reference template. A template image storage function for storing an image in a reference template, a template position storage function for storing a cutout position of the reference template in the input image as a reference position of the reference template, and new information of the object to be tracked. A correlation processing function for determining a collation position of the reference template in the new input image based on the new input image and the reference template;
A movement amount calculation function for calculating a movement amount of the comparison position with respect to the reference position from the comparison position obtained by the correlation processing function and the reference position stored in the template position storage function; A program for updating the reference position to the collation position if the condition is equal to or more than a predetermined condition, and an update function for not updating the reference position to the collation position if the movement amount is less than a predetermined condition, is recorded. A recording medium for an object tracking method characterized by the following.

According to the present invention, it is possible to determine that the minute movement of the template is a calculated movement amount due to image noise or environmental fluctuation, so that updating of the template can be suppressed. It is possible to prevent the minute movement of each of them from accumulating and deviating from the position of an object such as a vehicle. Further, when the moving amount of the template is large, the template can be updated, so that the advantage of the updated template method is not lost.

Thus, the phenomenon that the template is gradually shifted while performing the update template matching can be suppressed, and the way to realize the traffic flow monitoring system by the image processing is opened.

[0027]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, specific embodiments of the present invention will be described with reference to the drawings.

[First Embodiment] FIG. 1 is a block diagram showing a first embodiment of the present invention. FIG. 1 shows an object monitoring apparatus 10 that detects and tracks an object by image processing as a whole. The object monitoring device 10 is an image processing technology for a traffic flow monitoring system that automates a monitoring camera installed on a road by image processing, performs automatic measurement of a traffic flow, and automatically detects an unexpected event. Is a device for tracking an area of a car traveling on a vehicle. The object monitoring device 10 can also be used for tracking a person using a monitoring camera in a department store premises or a monitoring camera in a station premises, or tracking a face or a hand used in a human interface.

The object monitoring device 10 includes an image input unit 11,
It comprises an image memory 12, an initial template extracting means 13, a result output means 14, and an object tracking device 20.

The object monitoring apparatus 10 can be realized by connecting a personal computer to a TV camera for inputting an image, and a program for realizing the functions of the respective units is provided.
What is necessary is just to record on recording media, such as FD, CD-ROM, MO, and DVD.

The combination of the means in the frame of FIG. 1 corresponds to the object tracking device 20. Note that the configuration of the object monitoring device 10 is not limited to this, and a device or a connection method connected to the object tracking device 20 may take another form.

The object tracking device 20 includes a template image storage unit 21, a template position storage unit 22, a correlation processing unit 23, a movement amount calculation unit 24, and a template image cutout unit 25.

The image input means 11 performs A / D conversion on images input in a time-series manner and sequentially transmits the images to an image memory. The image memory 12 has a function of storing images and transmitting an image of a corresponding frame in response to a request signal from the initial template extracting unit 13 and the template image extracting unit 25.

When an object enters the image, the initial template extracting means 13 determines the information (position, shape, size, etc.) of the template in the initial state and transmits it to the template image cutting means 25.

The template image extracting means 25 reads out the image of the corresponding frame number from the image memory 12 in accordance with the transmitted template information, cuts out the template image from the image, and stores the template image storing means 21
Send to The extracted image is stored in the template image storage unit 21.

The template image cutting means 25
Transmits the transmitted template information to the template position storage unit 22. The template position information is stored in the template position storage unit 22.

As described herein, the template information includes the size and shape (generally a rectangle) in addition to the position. This is because the shape and size of the updated template may change. The amount of movement of the template is a function including information on the size and shape of the template (for example, the distance between the positions of the centers of gravity and the distance between the centers of the templates).
However, in the following description, information necessary for calculating the movement amount will be collectively expressed by the term “template position”.

The above is the initial operation of the object tracking device 20, and the subsequent operations are repeatedly performed every frame to track the object.

FIG. 3 shows the procedure of the process repeatedly executed.

First, when a new image (the image at the next time, the image of the next frame) is input to the image memory 12, the correlation processing means 23 operates using the new image as a reference image (step 301).

The correlation processing means 23 uses the some matching measure from an appropriate search range, and the template transmitted from the template image storage means 21 corresponds best among the reference images transmitted from the image memory 12. A process for obtaining a position is performed (step 302).

In performing the correlation processing, there are known various methods for setting a search range and applying a measure, and the difference between these methods is not essential to the present invention. As an invention of this system, for example, Japanese Patent Application No. Hei 10-6
No. 6657, “Object region tracking device and object region tracking method”, and Japanese Patent Application No. 9-345094, “Object region tracking device and object region tracking method”.

The processing result of the correlation processing means 23 is transmitted to the result output means 14 (step 303). The processing result is also transmitted to the movement amount calculating means 24.

The moving amount calculating means 24 calculates the moving amount of the template from the template position at the previous time stored in the template position storing means 22 and the new template position (the result transmitted from the result of the correlation processing means 23) (step 304). ).

As a result, for example, when the moving amount is (△ X, △ Y)
And

Here, if ΔX ≦ Tx and ΔY ≦ Ty, the moving amount calculating means 24 determines that “the template has not moved”. If ΔX> Tx or ΔY> Ty, it is determined that “the template has moved” (step 305).

As the threshold values Tx and Tx used here, it is easy to use 1 (pixel) for each general digital image size (VGA size, 640 pixels × 480 pixels). If the size of the input image is large, a small value of about 1 to 3 may be used. In addition, if it is desired to make the movement amount of the Y axis sensitive depending on the traveling direction in the image of the vehicle, Tx = 1 and Ty = 0 can be set.

When the moving amount calculating means 24 determines that "the template has moved", the position of the new template is transmitted to the template position storing means 22, and the template position storing means 22 stores it. That is, the template position is updated (step 306).

The moving amount calculating means 24 transmits the new template position to the template image extracting means 25, and the template image extracting means 25 fetches the image used as the reference image at the current time from the image memory 12, and reads the template position. The image area is cut out as a template image at the next time. The template image is updated by being transmitted to and stored in the template image storage unit 21 (step 30).
7).

If the moving amount calculating means 24 determines that "the template has not moved", the template updating procedure is not performed.

Thus, the processing for one input image is completed, and a state of waiting for the next input image is entered (step 30).
1).

[Second Embodiment] In the second embodiment, a new embodiment of the template update procedure will be described.

In the first embodiment, when it is determined in step 305 that “the template has not moved”,
Neither the update of the template position (step 306) nor the update of the template image (step 307) was performed.

However, in the present invention, when the template is not moved, the template position storage means 2
The point is that the information on the template position 2 is not updated, and a method of updating only the template image can be considered. The second embodiment is based on that concept.

FIG. 4 shows the procedure of the repetitive processing of the object tracking device 20 configured according to the second embodiment.

The concept is as follows.

The case where the movement of a minute template of about one pixel is detected is a calculation movement which appears due to the influence of noise. In this case, if the template image is cut out at a new position and updated, a phenomenon of a gradual shift may occur. Therefore, it is possible to cut out the template from the new image at the “old position” before being updated. Therefore, the case where the second embodiment in which only the template image is updated from the new image to the template without updating the template position information is effective is effective, for example, when the weather changes or the camera's automatic aperture mechanism is used. When the entire screen becomes brighter or darker due to the operation, updating the template image with the brightness of the new image can be expected to improve the subsequent results.

[Third Embodiment] The point of the present invention is that if the template is not moved, the information of the template position in the template position storage means 22 is not updated. It is also possible to execute the output of the template position executed by the above only when the template is moved. The third embodiment is based on that concept.

FIG. 5 shows the procedure of the repetitive processing of the object tracking device 20 configured according to the third embodiment.

The case where the method of outputting a result only when an object moves as in the third embodiment is effective is that valuable objects are automatically monitored by a surveillance camera at a museum or a store, for example, and the target object is monitored. It is conceivable to configure a device that sends an alarm only when moving. If there is no change, the signal is not transmitted, so that even if many places are monitored at once, there is no fear that the traffic on the communication path is unnecessarily increased.

[0061]

As described above, according to the present invention,
While performing the update template matching, the phenomenon that the template is gradually shifted can be suppressed. Therefore, the moving vehicle and the stopped vehicle in the image can be similarly stably tracked. As a result, it can play an important role in realizing a monitoring system that automatically measures a traffic flow from a surveillance camera on a highway or detects an unexpected event automatically. This practical effect is truly significant.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of an object monitoring device according to an embodiment of the present invention.

FIG. 2 is a schematic diagram of a tracking result.

FIG. 3 is a flowchart of a repetition process of the object monitoring device 10.

FIG. 4 is a flowchart of a repetition process according to a second embodiment.

FIG. 5 is a flowchart of a repetitive process according to a third embodiment.

[Explanation of symbols]

 REFERENCE SIGNS LIST 10 object monitoring device 11 image input means 12 image memory 13 initial template extraction means 14 result output means 20 object tracking device 21 template image storage means 22 template position storage means 23 correlation processing means 24 movement amount calculation means 25 template image cutout means

 ──────────────────────────────────────────────────続 き Continuing from the front page (72) Inventor Kazunori Onoguchi 8-6-26 Motoyama Minami-cho, Higashinada-ku, Kobe City, Hyogo Prefecture F-term (reference) 5B057 AA16 CE09 CH11 DA07 DB02 DB09 DC02 DC34 DC40

Claims (5)

[Claims] An object tracking apparatus for tracking an object by image processing, comprising: a template image storage unit that cuts out an image area including an object to be tracked from an input image as a reference template, and stores an image in the reference template; A template position storage unit that stores a cutout position of the reference template in the input image as a reference position of the reference template, based on the new input image including the new information of the object to be tracked and the reference template, A correlation processing unit for determining a reference position of the reference template in the new input image, and the reference position stored in the template position storage unit and the comparison position determined by the correlation processing unit,
A movement amount calculation unit that calculates a movement amount of the collation position with respect to the reference position; and if the movement amount is equal to or more than a predetermined condition, the reference position is updated to the collation position, and the movement amount is a predetermined condition. An object tracking device, comprising: an update unit that does not update the reference position to the collation position if the value is less than the reference position. 2. The method according to claim 1, wherein the updating unit updates the reference position to the collation position if the moving amount is equal to or greater than a predetermined condition, and updates a new template based on the collation position from the new input image. And updating the image in the reference template to an image in the new template.If the movement amount is less than a predetermined condition, the reference position is not updated, and the image in the reference template is updated. An object tracking device characterized by not updating. 3. The updating section updates the reference position to the collation position if the movement amount is equal to or greater than a predetermined condition, and updates a new template from the new input image based on the collation position. And updates the image in the reference template to an image in the new template.If the moving amount is less than a predetermined condition, the reference position is not updated and the new input image is used to update the reference position. An object tracking apparatus, wherein a new template is cut out based on a collation position, and an image in the reference template is updated to an image in the new template. 4. An object tracking method for tracking an object by image processing, comprising: a template image storing step of cutting out an image area including an object to be tracked from an input image as a reference template, and storing an image in the reference template; A template position storing step of storing a cutout position of the reference template in the input image as a reference position of the reference template, based on the new input image including new information of the object to be tracked and the reference template, A correlation processing step of determining a reference position of the reference template in the new input image; and the reference position stored in the template position storage step, based on the reference position stored in the template position storage step. Of travel distance A moving amount calculating step to update the reference position to the matching position if the moving amount is equal to or greater than a predetermined condition, and to change the reference position to the matching position if the moving amount is less than a predetermined condition. An object tracking method, comprising: an updating step that does not update; 5. A recording medium recording a program for realizing an object tracking method for tracking an object by image processing, wherein an image area including an object to be tracked is cut out from an input image as a reference template, and an image in the reference template is cut out. A template image storage function for storing a reference position of the reference template in the input image, a template position storage function for storing the reference template as a reference position, and a new input image including new information of the object to be tracked. Based on the reference template, a correlation processing function for obtaining a reference position of the reference template in the new input image, and the reference position stored in the template position storage function and the comparison position obtained by the correlation processing function From the reference to the reference position A movement amount calculation function for calculating the movement amount of the position; and if the movement amount is equal to or greater than a predetermined condition, the reference position is updated to the collation position; if the movement amount is less than a predetermined condition, the reference amount is updated. A recording medium for an object tracking method, wherein a program for realizing an updating function of not updating a position to the collation position is recorded.