JP2005114588A - Tracking device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a tracking device capable of more surely detecting a tracking object. <P>SOLUTION: This tracking device is equipped with a distance image collection part 14 for collecting distance images showing a three-dimensional shape in a visual field of a distance image sensor 2, a correlation calculation part 17 for extracting a distance image part having the largest correlation with distance image kernels 41, 42 from among the distance images, and a subject position calculating part 18 for calculating the subject position shown by the distance image part. The distance image kernels 41, 42 show the rough three-dimensional shape of an object whose center is projected to the front side. A moving body which is the tracking object usually has the three-dimensional shape, wherein the center is projected to the front side, as viewed from the distance image sensor 2. Hereby, the tracking object is detected, by comparing the distance image photographed this time with the distance image photographed the last time, the tracking object is detected more surely. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a tracking device, and more particularly to a tracking device that tracks the position of a moving object.

  A distance image sensor that captures a distance image showing a three-dimensional shape of a subject is known. The distance image sensor includes a light projecting unit that intermittently irradiates a plurality of different areas on the surface of the subject with a laser, and receives reflected light from the laser, and the reflected light is received after the laser is emitted by the light projecting unit. A light-receiving unit that measures the flight time until it is generated, and a distance image generation unit that generates a distance image indicating the three-dimensional shape of the subject based on the flight time.

  A tracking system that tracks the position of a moving object is known. In the tracking system, the distance image sensor is applied, and an operation of detecting the tracking target and an operation of tracking the tracking target are executed.

  FIG. 10 shows an operation for detecting a tracking target. The tracking system repeatedly executes an operation for detecting a tracking target. The tracking system first collects a distance image from the distance image sensor (step S101). The tracking system compares the distance image with the distance image captured last time and detects a region having a change (step S102). When the tracking system detects a region with a change between the distance image and the distance image captured last time (step S102; YES), the tracking system has a region with the change among the distance images captured this time. Is recorded as a template (step S103).

  FIG. 11 shows an operation for tracking the tracking target. The tracking system performs an operation of tracking the tracking target after an area having a change is detected by the operation of detecting the tracking target. The tracking system first collects a distance image from the distance image sensor (step S111). The tracking system detects the position of the region having the highest similarity with the template from the distance image (step S112).

  Such a tracking system may detect and track a moving body that does not require tracking. Such a tracking system may detect an area where there is no detected moving object when the template includes a background, when the moving object is deformed, or when the moving object changes its direction. . There is a need for a tracking device that can more reliably detect a tracking target and track the tracking target more reliably.

  Japanese Patent Application Laid-Open No. 8-55294 discloses a vehicle type discriminating apparatus that reliably discriminates a bicycle and a motorcycle traveling on a road or a vehicle such as an automobile. The vehicle type discriminating apparatus is provided with a gantry across the width of the road, and a plurality of loop antennas and laser radars are disposed at appropriate positions of the gantry, that is, above the road surface. In addition, light pulses are emitted from the laser radar toward the road surface at regular intervals. When a vehicle such as an automobile, motorcycle, or bicycle passes under the gantry on the road surface, a spark noise radio wave generated from the automobile or motorcycle is received by the loop antenna. Further, since the return time of the light pulse in the laser radar changes due to the passage of the vehicle, the vehicle height and width of the vehicle are detected by this, and the vehicle type of the vehicle is discriminated based on the presence or absence of the spark noise radio wave. .

  Japanese Patent Application Laid-Open No. 11-175883 relates to a traffic volume measuring device that measures traffic volume with an imaging device installed on a road, and a traffic volume measuring device that measures traffic volume in real time and controls traffic lights based on the traffic time without delay. And a traffic light control device are disclosed. The traffic measuring device includes an image pickup means having a plurality of image pickup devices for picking up the same direction on the road, an image input means for inputting an image signal output from the image pickup means, and an image signal entering a crossing store. A feature extracting means for extracting the characteristics of the vehicle to be detected, a three-dimensional measuring means for performing three-dimensional measurement by associating the extracted features, a model matching means for matching a pre-registered vehicle model, and a matching result And a presence number measuring means for determining the number of vehicles.

JP-A-8-55294 Japanese Patent Application Laid-Open No. 11-175883

The subject of this invention is providing the tracking apparatus which detects a tracking object more reliably.
Another object of the present invention is to provide a tracking device that prevents the background from being erroneously recognized as a tracking target.
Still another object of the present invention is to provide a tracking device that more reliably tracks a tracking target that moves.
Still another object of the present invention is to provide a tracking device having a small storage capacity of a recording device necessary for identifying a tracking target.
Still another object of the present invention is to provide a tracking device that identifies a tracking target in more detail.
Still another object of the present invention is to provide a tracking device having a wide field of view.

  In the following, means for solving the problems will be described using the reference numerals used in the best modes and embodiments for carrying out the invention in parentheses. This reference numeral is added to clarify the correspondence between the description of the claims and the description of the best mode for carrying out the invention / example, and is described in the claims. It should not be used to interpret the technical scope of the invention.

  The tracking device (1) according to the present invention includes a distance image collection unit (14) that collects a distance image indicating the three-dimensional shape of the field of view of the distance image sensor (2), and a distance image kernel (41, 42) from among the distance images. ) And a distance calculation part (17) for extracting the distance image part having the largest correlation, and a subject position calculation part (18) for calculating the subject position indicated by the distance image part. The distance image kernel (41, 42) shows a rough three-dimensional shape of an object whose center projects forward. The tracking target moving body generally has a three-dimensional shape with the center protruding forward, as viewed from the distance image sensor (2). The tracking device (1) can detect the tracking target more reliably by detecting the tracking target by comparing the distance image captured this time with the distance image captured last time.

  The tracking device (1) according to the present invention further includes a difference calculation unit (16) that calculates a difference distance image by excluding a portion of the distance image that matches the background distance image indicating the three-dimensional shape of the background. At this time, the correlation calculation unit (17) extracts a distance image portion from the difference distance image. At this time, the tracking device (1) can more reliably detect the tracking target without erroneously recognizing the background as the tracking target.

  The tracking device (1) according to the present invention uses a turning device (3) to change the direction in which the distance image sensor (2) is directed so that the subject position is located at the center of the distance image (19). ). At this time, the tracking device (1) can more reliably capture the moving tracking target in the distance image.

  The tracking device (1) according to the present invention includes a background database (11) that associates a plurality of directions with a plurality of background distance images, and a direction collection unit (15) that collects directions from direction sensors. At this time, the difference calculation unit (16) refers to the background database (11), searches for the background image corresponding to the direction among the plurality of background distance images, and matches the background distance image from the distance images. The difference distance image is calculated except for. At this time, the tracking device (1) can detect the tracking target more reliably without erroneously recognizing the background as the tracking target even when the distance image sensor (2) turns.

  There are a plurality of distance image kernels (41, 42), and it is preferable to show different three-dimensional shapes. The distance image kernel (41, 42) preferably includes a kernel indicating the three-dimensional shape of the tracking target. For example, the kernel indicating the three-dimensional shape of the whole human body or the three-dimensional shape of the human head. It is preferable that the kernel and the kernel which shows the three-dimensional shape of a vehicle are included.

  The correlation calculation unit (17) matches the sizes of the comparison target portion and the distance image kernel based on the position indicated by the comparison target portion in the distance image, and calculates the comparison target portion and the distance image kernel. Calculate the correlation. At this time, the tracking device (1) can more reliably detect the tracking target without erroneously recognizing an object whose size is extremely different from the tracking target.

  The tracking device (1) according to the present invention includes a normal vector database (12) for associating a target with a set of vectors, and a normal vector calculation unit (20) for calculating a set of normal vectors on the surface of the subject indicated by the distance image portion. ) And a normal vector identification unit (21) that searches the normal vector database (12) for a corresponding target corresponding to the set of normal vectors from the target. The set of normal vectors on the surface of the identification target generally has a smaller amount of information than the distance image of the identification target. Such identification is faster and requires less storage capacity of the recording device than identification using a distance image to be identified. For this reason, the tracking device (1) is preferable because of low manufacturing cost.

  The tracking device (1) according to the present invention includes a temperature distribution database (13) for associating a target with a temperature distribution, and an infrared image collection unit (5) that collects the subject temperature distribution of the subject indicated by the distance image portion from the infrared camera (5). 22) and an infrared image identification unit (23) for searching a corresponding target corresponding to the subject temperature distribution from the target with reference to the temperature distribution database (13). There are two objects (for example, a bicycle and a motorcycle) that have a substantially equal three-dimensional shape and different temperature distributions. At this time, the tracking device (1) can distinguish two such objects, which is preferable.

  A tracking device (1) according to the present invention includes a distance image sensor (2) that generates a distance image indicating a three-dimensional shape of a field of view, a control device (6) that calculates a direction based on the distance image, and a distance image sensor ( And a turning device (3) for changing the direction of 2) to a direction. At this time, the tracking device (1) can expand the field of view of the distance image sensor (2).

  The tracking device according to the present invention can detect the tracking target more reliably.

  An embodiment of a tracking device according to the present invention will be described with reference to the drawings. As shown in FIG. 1, the tracking device 1 includes a distance image sensor 2, a turning measure 3, a direction sensor 4, an infrared camera 5, and a control device 6. The distance image sensor 2 captures a distance image indicating the three-dimensional shape of the subject. That is, the distance image sensor 2 intermittently irradiates a plurality of different areas on the surface of the subject with laser, receives the reflected light of the laser, and the reflected light is received after the laser is emitted by the light projecting unit. Is measured, and a distance image indicating the three-dimensional shape of the subject is generated based on the flight time. The distance image sensor 2 transmits the distance image to the control device 6.

  The turning measure 3 changes the direction in which the distance image sensor 2 is directed according to an instruction from the control device 6. The direction sensor 4 measures the direction in which the distance image sensor 2 is facing and transmits the direction to the control device 6. The infrared camera 5 measures the temperature distribution of the subject by photographing the heat radiation emitted from the subject with infrared rays according to an instruction from the control device 6. The infrared camera 5 transmits the temperature distribution to the control device 6. The control device 6 is an information processing device exemplified by a personal computer, and controls the distance image sensor 2, the turning measure 3, the direction sensor 4, and the infrared camera 5 by executing an installed computer program. Such a tracking device 1 is preferable in that the field of view for tracking the tracking target can be expanded by using the distance image sensor 2 that cannot turn.

  FIG. 2 shows the control device 6 in more detail. The control device 6 includes a background database 11, a normal vector database 12, a temperature distribution database 13, a distance image collection unit 14, a direction collection unit 15, a difference calculation unit 16, a correlation calculation unit 17, and a subject position calculation unit 18 which are computer programs. A turning unit 19, a normal vector calculating unit 20, a normal vector identifying unit 21, an infrared image collecting unit 22, and an infrared image identifying unit 23.

  The background database 11 records a background distance image captured by the distance image sensor 2. The normal vector database 12 records a set of normal vectors of the surface to be identified. The temperature distribution database 13 records the temperature distribution to be identified.

  The distance image collection unit 14 collects distance images captured by the distance image sensor 2. The direction collection unit 15 collects the direction in which the distance image sensor 2 is facing from the direction sensor 4. The difference calculation unit 16 calculates a difference distance image obtained by removing a portion that matches the background distance image recorded in the background database 11 from the distance images collected by the distance image collection unit 14. That is, only the subject that has entered the field of view of the distance image sensor 2 is captured in the difference distance image.

  The correlation calculation unit 17 calculates the correlation between the difference distance image created by the difference calculation unit 16 and the distance image kernel, and extracts the distance image portion having the largest correlation from the difference distance image. That is, the correlation calculation unit 17 is a distance image showing a rough three-dimensional shape to be tracked, and has previously recorded a plurality of distance image kernels showing the three-dimensional shape of an object with the center protruding forward. . The correlation calculation unit 17 first calculates the distance to the subject imaged in a part of the difference distance image, scales the distance image kernel based on the distance, and determines the size of the part and the distance image kernel. By making them coincide with each other, the correlation between the part and the distance image kernel is calculated. The correlation calculation unit 17 calculates the correlation in this way for all the parts of the difference distance image where the subject is imaged, and selects the distance image part having the largest correlation with the distance image kernel from among all the parts. Extract. Note that the correlation calculation unit 17 can also prevent the distance image portion from being output when the correlation of the extracted distance image portion is smaller than a predetermined threshold.

  The subject position calculation unit 18 calculates the position of the subject indicated by the distance image portion extracted by the correlation calculation unit 17. The turning unit 19 changes the direction in which the distance image sensor 2 faces by driving the turning device 3 so that the position calculated by the subject position calculating unit 18 is arranged at the center of the distance image.

  When the distance image portion is extracted by the correlation calculation unit 17, the normal vector calculation unit 20 calculates a set of normal vectors on the surface of the subject indicated by the distance image portion. The normal vector identification unit 21 refers to the normal vector database 12 and identifies the subject based on the set of normal vectors calculated by the normal vector calculation unit 20. When the distance image portion extracted by the correlation calculation unit 17 is extracted, the infrared image collection unit 22 collects the temperature distribution of the subject indicated by the distance image portion from the infrared camera 5. The infrared image identification unit 23 refers to the temperature distribution database 13 and identifies the subject based on the temperature distribution collected by the infrared image collection unit 22.

  FIG. 3 shows the background database 11. The background database 11 records a background 32 in association with the direction 31. A direction 31 indicates the direction of the distance image sensor 2. The background 32 indicates a background distance image captured by the distance image sensor 2 facing the direction. At this time, the difference calculation unit 16 refers to the background database 11 to search for a background distance image corresponding to the direction collected by the direction collection unit 15, and searches the distance image for a portion that matches the background distance image. Excluding the difference image.

  FIG. 4 shows the normal vector database 12. The normal vector database 12 records the target 34 in association with the normal vector group 33. A target 34 indicates a target identified by the tracking device 1. The normal vector group 33 indicates a set of normal vectors of the target surface. At this time, the normal vector identification unit 21 refers to the normal vector database 12 and searches for an object corresponding to the set of normal vectors calculated by the normal vector calculation unit 20 to identify the subject.

  FIG. 5 shows the temperature distribution database 13. The temperature distribution database 13 records the temperature distribution 35 in association with the object 36. A target 36 indicates a target identified by the tracking device 1. The temperature distribution 35 indicates the temperature distribution of the target. The infrared image identification unit 23 refers to the temperature distribution database 13 and searches for an object corresponding to the temperature distribution collected by the infrared image collection unit 22 to identify the subject.

  FIG. 6 shows an example of a distance image kernel used by the correlation calculation unit 17. The distance image kernel 41 is formed from a plurality of pixels. Each of the pixels indicates the degree of protrusion in front of the object to be imaged. That is, the larger the pixel value, the closer to the distance image sensor 2. That is, the distance image kernel 41 shows a three-dimensional shape (for example, a cone) that has a larger amount protruding toward the front as it is closer to the center. FIG. 7 shows another example of the distance image kernel used by the correlation calculation unit 17. The distance image kernel 41 shows a square object. The tracking target of the tracking device 1 usually has a three-dimensional shape with the center protruding forward as viewed from the distance image sensor 2. The tracking device 1 can more reliably detect the tracking target by using such a distance image kernel.

  It is also useful for the distance image kernel to show a long protrusion or a long protrusion. The range image kernel is also useful to set the size of the object. At this time, the correlation calculation unit 17 extracts a distance image of a person who enters the field of view of the distance image sensor 2 by using, for example, a distance image kernel that shows a long protrusion whose length is about the height of the person. Can do. The correlation calculation unit 17 can extract a distance image of a person's head that enters the field of view of the distance image sensor 2 by using, for example, a distance image kernel indicating a sphere whose diameter is about the head of the person. The correlation calculation unit 17 can extract a distance image of a car that enters the field of view of the distance image sensor 2 by using, for example, a distance image kernel indicating the shape of the car.

  The tracking device 1 performs an operation for tracking a tracking target and an operation for identifying the tracking target. FIG. 8 shows an operation of tracking the tracking target. The distance image sensor 2 always captures a distance image. The tracking device 1 first collects a distance image captured by the distance image sensor 2 (step S1), and collects a direction in which the distance image sensor 2 is facing from the direction sensor 4 (step S2). The tracking device 1 refers to the background database 11 to search for a background distance image corresponding to the collected direction, and removes a portion that matches the background distance image from the distance image to calculate a differential distance image ( Step S3). That is, only the subject that has entered the field of view of the distance image sensor 2 is captured in the differential distance image, and only the subject is captured even when the distance image sensor 2 turns.

  When the subject other than the background appears in the difference distance image (step S4, YES), the tracking device 1 calculates the distance to the subject imaged in a part of the difference distance image, and based on the distance The range image kernel is enlarged / reduced to match the size of the portion with the range image kernel (step S5), and the correlation between the portion and the range image kernel is calculated (step S6). The tracking device 1 calculates the correlation in this way for all the parts of the difference distance image where the subject is imaged, and extracts the distance image part having the largest correlation with the distance image kernel from all the parts. (Step S7).

  At this time, the tracking device 1 can detect the tracking target faster than the identification using the distance image of the identification target. For this reason, the tracking device 1 can detect the tracking target moving at high speed more reliably.

  The tracking device 1 calculates the position of the subject indicated by the extracted distance image portion, and drives the swivel device 3 so that the position is located at the center of the distance image so that the distance image sensor 2 is directed. Is changed (step S8). The tracking device 1 can also prevent the direction in which the distance image sensor 2 is facing from changing when the correlation of the extracted distance image portion is smaller than a predetermined threshold.

  FIG. 9 shows an operation for identifying the tracking target. This operation is executed when the tracking device 1 detects a subject other than the background. The tracking device 1 first extracts a distance image portion where the subject is imaged from the distance image captured by the distance image sensor 2 (step S11). The tracking device 1 calculates a set of normal vectors on the surface of the subject indicated by the extracted distance image portion (step S12). The tracking device 1 refers to the normal vector database 12 and searches for an object corresponding to the set of normal vectors to identify the subject (step S13).

  The set of normal vectors on the surface of the identification target generally has a smaller amount of information than the distance image of the identification target. Such identification is preferable, faster than the identification using the distance image to be identified, and requires a smaller storage capacity of the recording apparatus.

  The tracking device 1 collects the temperature distribution of the subject indicated by the distance image portion from the infrared camera 5 (step S14). The tracking device 1 refers to the temperature distribution database 13 and searches for an object corresponding to the temperature distribution to identify the subject (step S15). The tracking device 1 outputs a target that matches the target identified by the temperature distribution among the targets identified by the normal vector as a final identification result. There are two tracking objects (for example, a bicycle and a motorcycle) that have a substantially equal three-dimensional shape and different temperature distributions. At this time, the tracking device 1 can distinguish two such objects, which is preferable.

FIG. 1 is a block diagram showing an embodiment of a tracking device according to the present invention. FIG. 2 is a block diagram showing the control device. FIG. 3 shows a background database. FIG. 4 is a diagram showing a normal vector database. FIG. 5 is a diagram showing a temperature distribution database. FIG. 6 is a diagram illustrating an example of a distance image kernel. FIG. 7 is a diagram illustrating another example of the distance image kernel. FIG. 8 is a flowchart showing an operation for tracking a tracking target. FIG. 9 is a flowchart showing an operation for identifying a tracking target. FIG. 10 is a flowchart showing an operation for detecting a known tracking target. FIG. 11 is a flowchart showing an operation for tracking a known tracking target.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1: Tracking apparatus 2: Distance image sensor 3: Turning apparatus 4: Direction sensor 5: Infrared camera 6: Control apparatus 11: Background database 12: Normal vector database 13: Temperature distribution database 14: Distance image collection part 15: Direction Collection unit 16: Difference calculation unit 17: Correlation calculation unit 18: Subject position calculation unit 19: Swivel unit 20: Normal vector calculation unit 21: Normal vector identification unit 22: Infrared image collection unit 23: Infrared image identification unit 31: Direction 32: Background 33: Normal vector group 34: Object 35: Temperature distribution 36: Object 41: Distance image kernel 42: Distance image kernel

Claims (9)

A distance image collection unit for collecting a distance image indicating a three-dimensional shape of the field of view of the distance image sensor;
A correlation calculation unit that extracts a distance image portion having the largest correlation with the distance image kernel from the distance image;
A subject position calculation unit that calculates a subject position indicated by the distance image portion,
The distance image kernel is a tracking device that shows a three-dimensional shape of an object whose center projects forward. In claim 1,
A difference calculating unit that calculates a difference distance image by excluding a portion that matches a background distance image indicating a three-dimensional background shape from the distance image;
The correlation calculation unit extracts the distance image portion from the difference distance image. In either claim 1 or claim 2,
A tracking device further comprising: a turning unit that changes a direction in which the distance image sensor faces such that the subject position is arranged at the center of the distance image. In claim 3,
A background database that maps multiple directions to multiple background distance images;
A direction collection unit for collecting the direction from a direction sensor;
The difference calculating unit refers to the background database, searches for the background image corresponding to the direction among the plurality of background distance images, and removes a portion that matches the background distance image from the distance images. And a tracking device for calculating the difference distance image. In any one of Claims 1-4,
A tracking device that includes a plurality of distance image kernels and exhibits different three-dimensional shapes. In any one of Claims 1-5,
The correlation calculation unit matches the size of the comparison target part and the distance image kernel based on the position indicated by the comparison target part of the distance image, and compares the comparison target part and the distance image kernel. Tracking device that calculates the correlation of In any one of Claims 1-6,
A normal vector database that maps objects to a set of vectors;
A normal vector calculation unit that calculates a set of normal vectors of the surface of the subject indicated by the distance image portion;
A tracking device further comprising: a normal vector identification unit that searches the corresponding vector corresponding to the set of normal vectors from the target with reference to the normal vector database. In any one of Claims 1-7,
A temperature distribution database that associates a target with a temperature distribution;
An infrared image collection unit that collects a subject temperature distribution of the subject indicated by the distance image portion from an infrared camera;
A tracking device further comprising: an infrared image identification unit that searches the target for a corresponding target corresponding to the subject temperature distribution with reference to the temperature distribution database. A distance image sensor for generating a distance image indicating a three-dimensional shape of the field of view;
A tracking device comprising: a turning device that changes a direction of the distance image sensor.

Images