JP2004053278A - Forward vehicle tracking system and forward vehicle tracking method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a front vehicle tracking system and a front vehicle tracking method that can reliably track vehicles and are capable of robust, accurate measurement. <P>SOLUTION: The front vehicle tracking system comprises a laser radar 2 for scanning a front in the advance direction of vehicles; a laser radar data memory 4; a vehicle discovery point 5 for detecting front vehicles according to the reflection point of detection waves of the laser radar 2; an intensity distribution type creation section 6 for creating the intensity distribution type of reflection points; a vehicle tracking section 7 by utilizing intensity; a laser radar measurement evaluation section 8 for evaluating the measurement of laser radar; an electronic camera 3 for imaging the front in an advance direction; an image memory 9; a vehicle template creation section 10 for creating a template for tracking vehicles; a vehicle tracking processing section 11 for tracking vehicles by template matching; an image processing tracking reliability judgment section 12; and an integral processing section 13. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a forward vehicle tracking system and a forward vehicle tracking method mounted on a vehicle.
[0002]
[Prior art]
As a preceding vehicle tracking technology using conventional template matching, there is, for example, JP-A-2002-99906. This method tracks the preceding vehicle using a combination of an infrared camera that can capture the heat source and a laser radar.The template is cut at the position on the image that corresponds to the position detected by the laser radar, and tracking is performed by both. By performing, for example, even if a weak laser radar can not be detected in rainy weather, the vehicle can be tracked by an image capable of capturing heat, and the vehicle is robustly tracked by taking advantage of each other. .
[0003]
[Problems to be solved by the invention]
However, in this method, since an infrared camera is used, there is a problem that the accuracy of template matching using the heat source of the vehicle is low in an environment with a high temperature such as summer. In addition, although the tracking accuracy of the vehicle is taken into consideration, there is no particular effort to improve the accuracy of the measurement in the lateral direction, and since the heat source is captured as a configuration, it is difficult to detect the edge of the vehicle with high accuracy. Since the edge cannot be strictly detected on the above, it is difficult to measure a subtle movement of the vehicle, especially a fine movement in the lateral direction.
[0004]
An object of the present invention is to provide a forward vehicle tracking system and a forward vehicle tracking method that can reliably track a vehicle and perform robust and accurate measurement as a whole system.
[0005]
[Means for Solving the Problems]
In order to solve the above problems, the present invention provides a detection wave output unit that scans forward in a traveling direction, a vehicle detection unit that detects a preceding vehicle by a reflection point of the detection wave output by the detection wave output unit, and a detection wave Considering also the distance measured by the output unit, an intensity distribution type creating unit that creates an intensity distribution type of the reflection point, and a vehicle tracking unit that uses the intensity distribution to measure the position of a preceding vehicle while also using the intensity distribution. A detection wave output means measurement and evaluation means for evaluating the tracking result of the detection wave output means, a camera for imaging the front in the traveling direction, and a tracking template for the vehicle in front in consideration of the distance measured by the detection wave output means. Vehicle template creating means for creating, vehicle tracking processing means for performing vehicle tracking by template matching, and image for evaluating tracking reliability of a preceding vehicle by image processing of a camera And physical tracking reliability determination unit, a tracking result of evaluation by the detection wave output means and integration identifying and tracking evaluation results of image processing, and has a integrated processing means for calculating the position of the forward vehicle.
[0006]
【The invention's effect】
ADVANTAGE OF THE INVENTION According to this invention, a vehicle can be tracked reliably and robust and highly accurate measurement can be performed as a whole system.
[0007]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the drawings described below, those having the same functions are denoted by the same reference numerals, and repeated description thereof will be omitted.
[0008]
Embodiment 1
FIG. 1 is a configuration diagram of a forward vehicle tracking system according to the first embodiment. FIG. 2 is a diagram of a vehicle equipped with a laser radar and a camera, and a diagram illustrating a reference coordinate system for representing the position of a detection target used in the following description. FIG. 2A is a diagram viewed from the side. , (B) is a view seen from above.
[0009]
In the figure, reference numeral 1 denotes a self-vehicle (FIG. 2), 2 denotes a scanning laser radar (hereinafter referred to as a laser radar) which is a detection wave output means, 3 denotes an electronic camera, 4 denotes a laser radar data memory, and 5 denotes a vehicle found. Unit, 6 is a reflection intensity (hereinafter referred to as intensity) distribution type creating unit, 7 is a vehicle tracking unit using intensity, 8 is a laser radar measurement and evaluation unit, 9 is an image memory, 10 is a vehicle template creating unit, and 11 is a vehicle template creating unit. A vehicle tracking processing unit, 12 is an image processing tracking reliability determination unit, and 13 is an integrated processing unit (overall evaluation unit, overall determination unit).
[0010]
As shown in FIG. 1, the forward vehicle tracking system according to the first embodiment includes a laser radar 2 mounted on a host vehicle 1 and a camera 3.
The processing unit using the laser radar 2 includes a laser radar data memory 4 for storing data for each scanning angle input from the laser radar 2, a vehicle finding unit 5 for finding a preceding vehicle from the data, The intensity distribution type creation unit 6 stores the intensity distribution of the vehicle as an intensity distribution type indicating the characteristics of the vehicle being tracked, and the intensity utilization that more reliably measures the position of the vehicle while using the intensity distribution after the vehicle is discovered. The vehicle includes a vehicle tracking unit 7 and a laser radar measurement and evaluation unit 8 that evaluates the reliability of the vehicle tracking result.
[0011]
On the other hand, the processing unit using the camera 3 creates an image memory 9 for storing the image input from the camera 3 and a vehicle tracking template at a position on the image corresponding to the position where the laser radar 2 has found the vehicle. A vehicle template creation unit 10 that performs vehicle tracking by template matching after considering the size change due to a distance change from the created template after the vehicle tracking is started; And an image processing tracking reliability determination unit 12 for evaluating tracking reliability.
[0012]
The integrated processing unit 13 integrates a tracking evaluation result obtained by the laser radar 2 and a tracking evaluation result obtained by image processing captured by the camera 3 to calculate a final vehicle position during tracking.
[0013]
The camera 3 and the laser radar 2 are attached to the front part of the own vehicle 1 as shown in FIG. 2, and the laser radar 2 performs one-dimensional scanning in a direction parallel to a road surface ahead of the own vehicle 1. The distance and direction to the object in front are measured, and the camera 3 captures an image of the front of the vehicle 1 in the same direction. Also, here, a coordinate system for expressing the object position is provided as shown in FIG. That is, the axes of the laser radar 2 are defined as a Z axis, a direction perpendicular to the central axis and parallel to the road surface is defined as an X axis, and a direction perpendicular to the road surface is defined as a y axis. The position in the Z-axis direction is referred to as “distance”, and the distance in the X-axis direction is referred to as “lateral position”.
[0014]
FIG. 3A is a diagram showing a measurement position of the laser radar 2 when a vehicle is measured by the laser radar 2, FIG. 3B is a diagram showing a front image in that case, and FIG. It is a figure showing intensity distribution of vehicles in a short distance.
[0015]
The laser radar 2 detects an object that reflects light existing in front and measures the distance to the object that reflects the light from the time difference between the transmission of the transmission wave transmitted from the laser radar 2 and the reception of the wave. Device. The transmitted wave has the same energy for each scan angle, but the intensity of the received light depends on the reflectance of the light. In a road environment, a signboard made of iron, a reflector (reflex reflector; hereinafter, referred to as a reflector) attached to a roadside called a delineator, and a vehicle traveling in front of the vehicle may be used to reflect light. Among them, the reflectors mounted on the roadside or on the back of the vehicle are reflectors mounted so that the driver can easily recognize the presence of the vehicle ahead or the roadside, and therefore reflect light efficiently. ing. Therefore, when these reflectors are detected by the laser radar 2, it is possible to receive light of high intensity. Normally, the intensity becomes smaller as the distance to the object becomes longer, but since the vehicle is equipped with a reflector having high reflection efficiency, detection and measurement can be performed stably even at a long distance.
[0016]
The laser radar data memory 4 stores data measured by the laser radar 2. The tracking process is performed using the data stored in the laser radar data memory 4.
[0017]
The vehicle detection unit 5 shown in FIG. 1 detects a vehicle from the distribution of detection points of the laser radar 2 as shown in FIGS. 3A and 3C, and measures its azimuth and distance. This processing is performed, for example, as shown in FIG. 3, by detecting a set of left and right points moving in the same direction in chronological order or a continuous point having a width of about the vehicle width, which is detected at intervals of about the width of the vehicle. A conventional method may be applied.
[0018]
As described above, the vehicle is provided with a reflector, which is a reflector that efficiently reflects light. However, when the distance is short, the laser radar 2 receives reflected light from the body surface or the number plate. You can also. When the reflected light from the body is at such a distance that the light can be received, the intensity distribution of the vehicle has a strong intensity distribution at the position of the reflector and a relatively weak intensity distribution on the body surface as shown in FIG. Become.
[0019]
FIG. 4 is a conceptual diagram showing the relationship between the distance and the intensity distribution. The intensity decreases as the distance increases. However, since the position of the reflector is specific to the vehicle, is invariable, and varies depending on the type of vehicle, the shape of the intensity distribution is one of the values indicating the characteristics of the tracked vehicle.
[0020]
The processing by the intensity distribution type creating unit 6 and the processing by the vehicle tracking unit 7 using the intensity are processes utilizing the intensity distribution characteristics of the vehicle obtained from the laser radar 2. First, the intensity distribution type creating unit 6 targets the vehicle detected in the process of the vehicle finding unit 5 and stores the intensity distribution of the vehicle as a vehicle tracking intensity distribution type. When the vehicle 20A is a tracking vehicle (front vehicle) in a scene as shown in FIG. 3A, the intensity distribution in FIG. 3C is an intensity distribution type to be stored. At this time, since the position of the peak of the intensity distribution changes according to the distance, the distance when the intensity distribution type is created is also stored. In the case of FIG. 3, the distance zA (the distance between the host vehicle 1 and the vehicle 20A) may be stored at the same time.
[0021]
3A and 3B, reference numeral 21 denotes a road reflector. Since the stationary object such as the road reflector 21 approaches at the same relative speed as the own vehicle speed, it is determined to be a stationary object. The vehicles 20A and 20B are determined to be vehicles because the two pairs move at the same speed and at the relative speed indicating the moving object.
[0022]
The vehicle tracking unit 7 using the intensity performs position measurement by distribution type matching using the intensity distribution type. FIGS. 5A and 5B are diagrams showing this state, in which FIG. 5A shows an object position measurement by detection point grouping, and FIG. 5B shows a state in which a horizontal position is detected by matching with an intensity distribution.
[0023]
The detection of the preceding vehicle from the laser radar 2 by the vehicle detection unit 5 can be determined from the detection of an object having a size similar to the above-described relative speed (FIG. 3) and the vehicle width and the relative speed thereof. However, the data of the detection point is unstable especially in a weak portion such as body reflection. In particular, as shown in FIG. 5A, since the body surface of the end of the vehicle is oblique to the own vehicle 1 (the laser radar 2 mounted thereon), the detection of the reflection point is more unstable than the back surface. Become. However, when the position of the vehicle is detected by grouping points as in the conventional method, the distance (zA) can be accurately measured, but the measurement of the lateral position (xA) becomes unstable. (However, in this case, as shown in the figure, the center of the object detection result (dashed square in the figure) is assumed to be the horizontal position of the object.) Here, the reflector that is stably detected is used here. As shown in FIG. 5B, distribution type matching using the intensity distribution type created using the data of the tracked vehicle is performed so that the movement of the position can be finely measured. As a result, not only the distance but also the horizontal movement can be measured more accurately.
[0024]
The laser radar measurement / evaluation unit 8 in FIG. 1 evaluates the reliability of the vehicle being tracked by the laser radar 2. Here, for example, the reliability of the distance measurement value is determined by the temporal continuity that an object of the same width can be detected stably without a sudden change in the tracking distance (for example, the number of continuations or the change in distance). The degree of reliability). Further, the lateral position of the vehicle being tracked can be determined from the intensity distribution evaluation value of the laser radar measurement and evaluation unit 8. Specifically, the evaluation value can be calculated by obtaining a normalized correlation using the defined intensity distribution type and the actually observed intensity. For example, in the case of the vehicle 20B at time t + Δt in FIG. 5B, the shape of the intensity distribution is significantly different from that of the distribution type because one of the reflectors cannot be detected. There is a high possibility that the measurement of the lateral position by the laser radar 2 is greatly shifted. Thus, the normalized correlation with the intensity distribution becomes the evaluation value of the horizontal position. This evaluation value is used by the integration processing unit 13 described later.
[0025]
Next, a vehicle tracking process using the camera 3 will be described.
The image memory 9 stores an image captured by the camera 3. The tracking process is performed by image processing using the image stored in the image memory 9.
Tracking by image processing is performed with the vehicle detected by the laser radar 2 as a target. First, the vehicle template creation unit 10 cuts a template at a position on an image where a vehicle that should be present at that position is captured according to the position and distance at which the laser radar 2 has found the vehicle. The definition of this location can be calculated in the following way.
[0026]
FIGS. 6A and 6B are diagrams for explaining the correspondence between the position on the image and the actual three-dimensional position and size, where FIG. 6A shows the position and FIG. 6B shows the size.
In FIG. 6, 31 is a camera lens, 14 is the optical axis of the camera, 32 is the center of the actual vehicle, 33 is the imaging surface, z is the distance to the vehicle, f is the focal length of the camera lens 31, and xc is on the image. The center of the vehicle, θ is the direction in which the object at the position detected at the center xc of the vehicle on the image exists, w is the actual width of the vehicle, and wc is the width of the vehicle on the image.
[0027]
The azimuth θ at which the object at the position detected at the center xc of the vehicle on the image exists can be obtained by Expression 1.
[0028]
θ = tan ^-1 (F / xc) Equation 1
When the actual width w of the vehicle is known, the distance z to the vehicle can be calculated from the width wc of the vehicle on the image by Expression 2.
[0029]
z = fw / wc Equation 2
The detection position can be calculated by applying Expressions 1 and 2 to the detection position on the image.
[0030]
Further, after the vehicle is found, the position of the vehicle in the image input at each image processing interval is detected by a tracking process using template matching with a vehicle template to be referred to (hereinafter referred to as a reference template). I do. This is performed by a vehicle tracking processing unit (vehicle position detecting unit) 11 that matches the reference template. Normally, the distance z of a vehicle imaged in front of the vehicle changes, so that the size on the image changes according to the distance. For this reason, in the template matching of the vehicle tracking processing unit 11, by enlarging or reducing the reference image or the input image, the size of the vehicle in the reference image and the size of the vehicle captured in the input image become the same size. As described above, template matching is performed while scaling either image.
[0031]
FIG. 7 is a diagram illustrating a state of a process when the process of matching the image on the input image side with the reference template is performed.
[0032]
41 is an image (distance zb) at the time of creating the reference template, 42 is an input image (distance z (t)), and 43 is a scanning range. In the image 41 when the reference template is created, the vehicle is S = z (t) / zb times.
[0033]
Here, it is known that the laser radar 2 changes the distance zb when the reference template is created and the distance z (t) when a new image is input. Therefore, the size of the vehicle on the image when the inter-vehicle distance changes by Δz is calculated from the size of the vehicle at the time of creating the reference template according to the principle of Equation 2, so that the size is adjusted and the template is adjusted. Matching may be performed. For the calculation of the correlation (template matching), a normalized correlation using the mutual luminance values may be used, and the position having the highest correlation is the detected position of the vehicle in the new input image.
[0034]
That is, in FIG. 7, when the distance at the time of creating the reference template is zb, the distance z (t) at the time of inputting a new image is measured by the laser radar 2. The ratio between the size of the vehicle in the input image and the size of the vehicle in the template is 1 / zb: 1 / z (t). In other words, the matching with the reference template is performed while scaling S = zb / z (t). In the input image 42, since the magnification is S1, the distance can be calculated as 1 / S1 times the distance zb at the time of creating the reference template. That is, the position (xp, yp) at which the correlation is maximum is the detected position of the vehicle in the input image 42.
[0035]
Further, if the vehicle is lost in the laser radar 2 or if the image has a wider detection angle and is out of the detection angle in the laser radar 2 but is within the detection range in the image, the distance at the laser radar 2 is longer. If the measurement cannot be performed, template matching may be applied in which enlargement or reduction is performed while assuming that the distance has changed by a minute distance change Δz from the previous distance. FIG. 8 is a diagram showing this state. That is, FIG. 8 is an explanatory diagram of the template matching using the reference template while enlarging and reducing when the distance measurement by the laser radar 2 is not performed, and an explanatory diagram of the distance measuring method from the enlargement / reduction ratio.
[0036]
As a procedure in this case, an image obtained by multiplying the input image 42 by a plurality of magnifications (S1, S2,...) Assuming a change in the distance of the vehicle is cut out while shifting one pixel at a time within the scanning range 43, and the correlation is calculated. calculate. The position of the vehicle is obtained from the magnification S when the correlation has the highest value and the position (xp, yp).
[0037]
In other words, the correlation in all the positions and scalings is calculated while shifting the scanning range 43 by one pixel and scaling the distance change by Δz, and the position where the correlation value becomes the maximum and the scaling value are calculated. Ask for.
[0038]
As described above, in the first embodiment, when the distance by the laser radar 2 cannot be measured, the distance within the scanning range 43 of the image is shifted by a predetermined number of pixels while the minute distance change Δz from the previously measured distance is shifted. The template matching that enlarges or reduces the change in distance is used. With such a configuration, even when the distance cannot be measured by the laser radar 2, the vehicle can be reliably tracked, and robust and accurate measurement can be performed as a whole system.
[0039]
Next, the image processing tracking reliability determination unit 12 determines the reliability of the vehicle tracking of the template matching in the image processing. The evaluation value may be the maximum correlation value in the processing of the vehicle tracking processing unit 11.
[0040]
The tracking failure / success is determined by comparing the maximum correlation value with a threshold value, and when the correlation value is equal to or less than a predetermined value, it is determined that tracking by matching has failed, and when the correlation value becomes equal to or more than the predetermined value. Determines that there is reliability.
[0041]
Next, the processing of the integration processing unit 13 will be described.
The integrated processing unit 13 evaluates a tracking evaluation value by image processing captured by the laser radar 2 and the camera 3. For this, the evaluation value of the laser radar 2 obtained by the laser radar measurement evaluation unit 8 described above and the tracking evaluation value by the image processing obtained by the image processing tracking reliability determination unit 12 are used.
[0042]
Here, for example, when the detection point of the laser radar 2 is lost or the tracking performance is low, the distance and the horizontal position are calculated from the result of the image processing. Conversely, when the correlation value of the image processing is equal to or smaller than the threshold, the tracking is performed. Is deemed to have failed, and the distance and the lateral position are calculated from the result of the measurement evaluation of the laser radar 2.
[0043]
Here, when calculating the distance and the lateral position from the result of the measurement evaluation of the laser radar 2, when calculating the distance and the lateral position, the result measured by the laser radar 2 is obtained as a position having a high correlation by the distance and intensity distribution type. The determined horizontal position (xA in FIG. 5) is defined as a horizontal position.
[0044]
On the other hand, when calculating the distance and the horizontal position from the image processing, the following method is used. The distance cannot be measured by the image processing, but the distance change from the previous processing result can be measured. Therefore, the distance may be measured from the template and the reference template that have obtained the highest correlation value, or the size change between the size of the vehicle and the size of the vehicle in the previous process. When the distance is calculated, the horizontal position can be calculated by the geometric calculation described with reference to FIG.
[0045]
In the tracking by the laser radar 2, the vehicle can be tracked, but when the intensity evaluation value is low, the distance is calculated by the laser radar, and the calculation of the lateral position is performed by using the result of the image processing to ensure a more reliable lateral tracking. The position can be calculated.
[0046]
As described above, the configuration of the forward vehicle tracking system according to the first embodiment includes, as shown in FIG. 1, the laser data 2 for scanning ahead of the own vehicle 1 (FIG. 2) in the traveling direction and the laser data 2. A vehicle detection unit 5 that detects a preceding vehicle by a reflection point that reflects the output detection wave, an intensity distribution type creation unit 6 that creates an intensity distribution type of the reflection point in consideration of the distance measured by the laser data 2, A vehicle tracking unit 7 that uses the intensity distribution to measure the position of the preceding vehicle while using the intensity distribution; a laser data measurement and evaluation unit 8 that evaluates the reliability of the tracking result of the preceding vehicle using the laser data 2; A camera template 3 for capturing the front of the vehicle in the traveling direction and a template for tracking the preceding vehicle at a position on the image corresponding to the position detected by the laser data 2 in consideration of the distance measured by the laser data 2. , A vehicle tracking processing unit 11 that performs vehicle tracking by template matching in consideration of a size change due to a distance change from the generated template, and a tracking reliability of a preceding vehicle by image processing of the camera 3. Image processing tracking reliability determining unit 12 for evaluating the performance, integrated determination of the tracking evaluation result based on the laser data 2 and the tracking evaluation result based on the image processing, and calculating the final position of the preceding vehicle with respect to the own vehicle 1. And a processing unit 13.
[0047]
The laser data 2 corresponds to a detection wave output unit in the claims, the vehicle detection unit 5 corresponds to a vehicle detection unit, the intensity distribution type creation unit 6 corresponds to an intensity distribution type creation unit, , The vehicle tracking unit 7 corresponds to a vehicle tracking unit using intensity, the laser data measurement and evaluation unit 8 corresponds to a detection wave output unit measurement and evaluation unit, the vehicle template creation unit 10 corresponds to a vehicle template creation unit, The vehicle tracking processing unit 11 corresponds to a vehicle tracking processing unit, the image processing tracking reliability determination unit 12 corresponds to an image processing tracking reliability determination unit, and the integration processing unit 13 corresponds to an integration processing unit.
[0048]
Further, the forward vehicle tracking method according to the first embodiment detects the forward direction of the host vehicle 1 using the laser data 2, captures the forward direction of the own vehicle 1 with the camera 3, and outputs the laser data 2. A reflection point that reflects the wave is detected, a relative position of the reflection point with respect to the own vehicle 1 is measured, a preceding vehicle is detected from the reflection point, and an intensity distribution type indicating an intensity distribution characteristic of the preceding vehicle is obtained from the intensity distribution of the reflection point. Using the image captured by the camera 3, a template for tracking the preceding vehicle is created at a position on the image corresponding to the position detected by the laser data 2, and the intensity distribution matching of the laser data 2 and the image processing are performed. The vehicle is tracked by two means of template matching, and the position of the preceding vehicle is measured by the measured value with the higher tracking certainty.
[0049]
With this configuration, in the first embodiment, the laser radar 2 mounted on the own vehicle 1 and detecting and measuring a vehicle ahead in the traveling direction while scanning, and an object position tracking such as template matching for the vehicle existing in front. In a configuration including a camera 3 capable of tracking and measuring the position of a vehicle on an image by a technique and an image processing unit, when the laser radar 2 detects a vehicle, the characteristics of the vehicle that can be measured by the laser radar 2 are as follows. Vehicle tracking is performed by two means, that is, tracking of the vehicle's intensity distribution itself and template matching in image processing, and the position of the vehicle is measured by the measurement value with the higher tracking certainty. . As a result, even when one of the camera 3 and the laser radar 2 has lost the vehicle, it is possible to reliably track the vehicle, and it is possible to perform robust and reliable measurement as a whole system. Further, even when measurement can be performed by both sensors, it is possible to obtain a more reliable measurement result by using the measurement result with the higher tracking reliability.
[0050]
Embodiment 2
Next, a second embodiment of the present invention will be described.
In the second embodiment, by taking advantage of the mutual measurement of the laser radar 2 and the image processing, the integrated processing unit 13 selects the one that can measure the distance and the horizontal position with high accuracy, so that the overall system has higher accuracy. An accurate system. This high-precision measurement may be selected by the following method.
[0051]
FIG. 9A is a diagram illustrating a detection angle when the camera 3 is wider than the laser radar 2, and FIG. 9B is a diagram illustrating a detection angle when the camera 3 is wider than the laser radar 2.
[0052]
In FIG. 9, reference numeral 51 denotes a laser radar detection angle, 52 denotes a camera detection angle, 53 denotes an area where the laser radar 2 can perform higher precision measurement, and 54 denotes an area where image processing can perform higher precision measurement.
[0053]
First, as the measurement distance in the horizontal direction, usually, the finer one of the horizontal resolution (an angle corresponding to one pixel in the case of an image and one scan angle in the case of the laser radar 2) may be used. However, since the detectable range is often different, as shown in FIG. 9A, when the image is wider than the laser radar 2 and when only the image is captured, the image processing result is used. In the opposite case, the result of the laser radar 2 may be used.
[0054]
Next, a description will be given of how to improve the accuracy of the distance.
FIG. 10 illustrates the laser radar 2 and the image processing. It is calculated from the characteristics of the measurement principle. 6 is a graph showing a relationship between distance measurement accuracy (minimum resolution of distance measurement) and the length of a measurement distance.
[0055]
In FIG. 10, 55 indicates the measurement distance resolution of the laser radar 2, and 56 indicates the measurement distance resolution of image processing.
[0056]
The measurement principle of the laser radar 2 is a time difference from transmission of a transmission wave to reception of the transmission wave, which is determined by the wavelength of the resolution light. Therefore, the resolution is a constant value regardless of the distance. (However, if the distance is too short, there are problems such as the received light being too strong, and depending on the shape of the preceding vehicle, the reflecting surface does not enter the detection angle, and the measurement accuracy may be reduced.) On the other hand, the distance measurement by the image processing, that is, the distance change measurement is performed by a change in the size on the image as shown in Expression 2, and for example, a displacement of one pixel of the size wc on the image of the detected object is measured. Assuming that the resolution is the minimum, the distance and the resolution have the relationship of Expression 3.
[0057]
Δz = w · f / wc−w · f / (wc ± 1) = ± z / (w · f ± z) Equation 3
Here, w is the actual width of the vehicle, z is the distance to the vehicle, wc is the width on the image, and f is the focal length. This resolution Δz increases as the distance z increases. In other words, it can be said that the measurement by the image processing has higher accuracy as the distance is shorter, and becomes coarser as the distance increases. For example, a vehicle with a long distance is small, but a vehicle with a width of 2 pixels on the image is missing one pixel, and if the width of the vehicle is erroneously one pixel, the measurement distance is measured to be a half distance, Even if an image having a width of 100 pixels becomes 101 pixels on the image, the error is only about 1% (= 100/101), which is apparent from a small error.
[0058]
For this reason, it is possible to increase the accuracy by selecting the measurement result by the image with the laser radar 2 for the distant place and the image by the measurement result with the close distance. do it.
[0059]
That is, when the distance and the horizontal position are combined, if the image is wider, the far distance is measured by the laser radar 2, and the end of the laser radar 2, the diagonally forward direction, and the short distance are calculated by using the image processing result. When the radar 2 has a wider angle, the measurement of the short distance may be performed by image processing, and otherwise, the measurement results of the distance and the lateral position of the laser radar 2 may be used (see FIGS. 9A and 9B).
[0060]
This makes it possible to track the vehicle if only one of them is within the range of the detection angle and the other is detected, and to use a highly accurate distance measurement result to provide a robust and high- Accurate measurement becomes possible.
[0061]
As described above, the forward vehicle tracking system according to the second embodiment uses the data of the laser radar 2 at a distance longer than a predetermined distance, and uses the data of the camera 3 at a short distance smaller than the predetermined distance. If the preceding vehicle is detected at the end of the detection angle of the laser radar 2 or the camera 3 having the smaller detection angle, the data of the sensor having the larger detection angle is used.
[0062]
With such a configuration, in the second embodiment, the determination of the certainty of the first embodiment is not limited to the evaluation of the correlation value of the intensity distribution matching and the template matching, but also the measurement resolution performance of the sensors. Utilizing the measurement accuracy, the measurement can be ensured, and the measurement with the fine resolution can be performed. In the case of detecting at the end of the smaller detection angle where the angle that can be detected by the laser radar 2 is closer in the distance and the image or the image or the angle that can be detected by the laser radar 2 in the closer distance, the data is set to the data of the sensor with the wider detection angle. . This not only increases the robustness, but also makes it possible to use highly accurate measurement results, so that high-accuracy position measurement can be performed for the entire system.
[0063]
Embodiment 3
Next, a third embodiment of the present invention will be described.
In the third embodiment, the application range of the result of the laser radar 2 when the image is wider than the laser radar 2 in the above-described second embodiment is adopted. The detection angle of No. 2 is set to be smaller than the actual width, and in the case of a vehicle detected at the end of the laser radar detection angle, the result of the image processing is adopted.
[0064]
FIGS. 11A and 11B are diagrams illustrating the reason and a method of determining the detection angle end. FIG. 11A is a diagram illustrating the detection points of the laser radar 2 and FIG. 11B is a diagram illustrating the detection points of the laser radar 2 at the detection angle end.
[0065]
As shown in FIG. 11B, when the vehicle is detected at the detection angle end of the laser radar 2, the reflection of the vehicle is actually detected along the detection angle even though the reflection is outside the detection angle. A phenomenon that a point remains and a detection point having a measurement error is observed easily occurs. In this case, when the lateral position and the distance of the vehicle are measured using this point, an error is likely to appear. In the third embodiment, in order to avoid such erroneous measurement, tracking of template matching by image processing is performed to an extent that the correlation value becomes equal to or more than a threshold value only for a short distance with good measurement accuracy by image processing. In this case, the measurement result of the image processing is applied instead of the measurement value of the laser radar 2 even when a vehicle is present in the detection range of the laser radar 2.
[0066]
As a result, erroneous measurement caused by the above-described phenomenon can be prevented.
[0067]
As described above, when the detection angle of the laser radar 2 is smaller than the detection angle of the camera 3, the forward vehicle tracking system of the third embodiment determines whether or not the preceding vehicle is detected at the detection angle end. Is determined from an angle at which the variation in the value of the distance between the detection points on the vehicle in front of the laser radar 2 tends to increase, or from the observation that a phenomenon in which the variation in the value of the distance between the detection points increases is observed. If the variation becomes large, the data of the camera 3 is used.
[0068]
With such a configuration, in the third embodiment, when the detection angle of the laser radar 2 is smaller in the detection angle end of the second embodiment, the detection of the detection angle end is detected by the tracking vehicle. If it is detected at the end of the corner, the determination is made from the angle at which the variation in the measurement tends to increase. Alternatively, the configuration is determined based on observation of the phenomenon. Normally, the end of the detection angle of the laser radar 2 often has poor distance accuracy, and thus there is a problem that the distance and the lateral position are not easily obtained correctly. By taking into account such instability, it is possible to eliminate the use of the measurement result including the error, so that the vehicle position can be measured more reliably and more accurately.
[0069]
Embodiment 4
Finally, a fourth embodiment of the present invention will be described.
FIG. 12 illustrates a state in which a vehicle that is located at a distance such that the entire vehicle is imaged in front of the vehicle gradually approaches, and a portion of the vehicle obliquely forward approaches a position that is outside the range of the image. . (A) shows the case of the outer template, and (b) shows the case of also preparing the inner template. 61 is an outer template, 62 is an inner template.
[0070]
As described above, when the vehicle in front approaches the close distance or moves in the direction moving away in the left-right direction, a part of the vehicle protrudes from the range of the angle of view. In such a situation, as shown in FIG. 12B, matching cannot be performed with the outer template 61 including the shape of the entire vehicle, and correct vehicle tracking cannot be performed.
[0071]
Here, since the movement trajectory of the tracked vehicle from the laser radar 2 and the image processing is known, it is predicted that the tracked vehicle will move to a position where there is a high possibility that the tracked vehicle will go out of the angle of view in advance, and it will be out of the angle of view. The template is cut back into a part or the inside of the vehicle having characteristics other than the part (the inner template 62), and when it is expected that the vehicle comes out of the angle of view, the template using the part or the inner template (the inner template 62) Try to match. For example, if it is found that the vehicle approaches by 1 m for each process, the size of the vehicle, the position on the image detected immediately before the vehicle, and the vehicle in Δt seconds can be calculated according to the principles described in the expressions 1 and 2. Considering the movable distance, it is possible to predict to some extent the time required for the end of the vehicle to move out of the angle of view. With such a prediction, the template can be prepared before it goes out of the angle of view.
[0072]
FIGS. 13A and 13B are diagrams showing how the inner template and the left and right partial templates are defined. FIG. 13A shows when the vehicle approaches, and FIG. 13B shows when the vehicle moves to the left. 63 is a defined template, 64 is a vehicle internal tracking template, and 65 is a vehicle right side tracking template.
[0073]
For example, when the distance is very short, the left and right sides of the vehicle are likely to go outside the angle of view. (A pattern, a shape) is defined in a certain part (FIG. 13A). In addition, when moving to the left or right, moving to the left defines the right side of the vehicle, and moving to the right defines the portion that is likely to be captured in the image on the left side of the vehicle (see FIG. 13 (b)).
[0074]
FIG. 14 shows the relationship between a diagram defining a template inside or on the right side of the vehicle and the corresponding position on the real coordinate system of the position on the vehicle included in the template. (A) shows the case where the template is defined inside the vehicle, and (b) and (c) show the case where the template is defined on the right side of the vehicle.
[0075]
71 is the right side of the inner template (horizontal position xR, distance z), 72 is the left side of the inner template (horizontal position xL, distance z), and xl and xr are pixels.
[0076]
Normally, when the template is re-cut by tracking only the image and switching is performed so that tracking is performed only on the right side, for example, tracking within the image is possible as shown in FIG. Does not know which three-dimensional distance or horizontal position corresponds to which. For example, in FIG. 14C, if it is not known which distance x'-xl on the image corresponds to which distance in the real coordinate system, if the template is cut again, tracking on the image can be performed but the three-dimensional coordinate system can be obtained. The distance and lateral position cannot be tracked and measured. Therefore, when an operation such as switching a template to a partial template is performed by image processing alone, the position on the image can be obtained, but the three-dimensional distance and position of the portion cannot be measured. However, since the distance can be measured by the laser radar 2 in this case, as shown in FIGS. 14A and 14B, even when the template is cut back to the inside of the vehicle or to the left and right, the side of the actual vehicle is The position and distance can be measured continuously before re-cutting.
[0077]
That is, in FIG. 14B, since the absolute distance can be measured by the laser radar 2, the three-dimensional positions defining the inner template and the right template can be calculated. There is no problem in the measurement of distance, lateral position.
[0078]
By combining the laser radar 2 and the image according to such a principle, even when the vehicle is located at a position where a part of the vehicle comes out of the image, the vehicle can be accurately tracked.
[0079]
Further, when the vehicle moves left and right, the detection angle of the laser radar 2 is usually narrower, so that the measurement may be performed only by the image processing alone. By measuring the change in distance from the enlargement / reduction ratio of the template, the distance can be measured. In addition, if either the left or right part of the vehicle is captured in the image, tracking can be performed using only the template on the right side or the left side, and tracking in a wider angle range is also possible.
[0080]
Also, in the tracking by image processing, if there is an environment where both the inner template and the predefined outer template can be used, it is possible to achieve more accurate tracking by performing tracking using two types of templates It becomes.
[0081]
As described above, the forward vehicle tracking system according to the fourth embodiment uses a plurality of templates in vehicle tracking in template matching. The plurality of templates have a size relationship with the image data particularly when approaching the vehicle, and one of the templates is the entire front vehicle and the other is a part of the front vehicle.
[0082]
Further, in the vehicle tracking method according to the fourth embodiment, in the vehicle tracking in the template matching, the image data has a large / small relationship, one of which is the whole of the front vehicle and the other is a part of the front vehicle. Using the template.
[0083]
With such a configuration, in the fourth embodiment, in the vehicle tracking in the template matching in the image processing of the first embodiment, when the tracked vehicle approaches, or a part of the tracked vehicle is located at the end of the angle of view. In this case, the configuration is such that template matching is performed by cutting back to a template using the features inside the vehicle. Normally, when the vehicle approaches or moves toward the end of the angle of view, a part of the vehicle may protrude from the angle of view. Even in such a case, it is possible to switch the internal shape of the vehicle as a tracking target, so that accurate tracking can be performed. Further, even when the vehicle moves in the left and right directions in a direction away from the vehicle, measurement can be performed if a part of the vehicle remains at the angle of view, so that a wider range of vehicle tracking can be performed.
[0084]
Further, here, since the distance is known by the laser radar 2, it is possible to determine in advance that the vehicle has approached the extent that the vehicle protrudes, and then determine the switching timing to the template inside the vehicle. In addition, when performing an operation such as switching a template to a partial template by image processing alone, the position on the image can be obtained, but the actual three-dimensional distance and displacement of the position by re-cutting the position of the template are determined. , It becomes impossible to measure three-dimensional movement. However, in this case, since the distance is always measured by the laser radar 2, it can be determined at which part of the actual vehicle the template is defined, so that the three-dimensional distance and the lateral position are adversely affected. Switching of templates is possible.
[0085]
Although the present invention has been specifically described based on the embodiments, the present invention is not limited to the above-described embodiments, and it is needless to say that various modifications can be made without departing from the gist of the present invention.
[Brief description of the drawings]
FIG. 1 is a system configuration diagram according to a first embodiment of the present invention;
FIG. 2 is an explanatory diagram of a mounting position of a camera on a vehicle and a coordinate system setting.
FIG. 3 is an explanatory diagram of a state of a detection point of a laser radar and an intensity distribution at the time of vehicle detection.
FIG. 4 is an intensity distribution shape for each change in vehicle distance.
FIG. 5 is an explanatory diagram of improving the lateral position accuracy of vehicle position measurement using the intensity distribution of a laser radar.
FIG. 6 is an explanatory diagram of a relationship between a detection position of a laser radar, an imaging position on an image, and a size.
FIG. 7 is an explanatory diagram of template matching for performing enlargement / reduction based on a measurement distance result in a laser radar.
FIG. 8 is an explanatory diagram of template matching using a reference template while enlarging / reducing when distance measurement by a laser radar is not performed, and an explanatory diagram of a distance measuring method from an enlargement / reduction ratio.
FIG. 9 is an explanatory diagram of a detection range of a laser radar and a camera, and a region where each measurement can be performed with high accuracy.
FIG. 10 is a graph showing a relationship between a distance measured by a laser radar and a distance measurement based on a change in size in an image and a distance measurement resolution.
FIG. 11 is an explanatory diagram of a state of detection point variation when a vehicle is positioned at a laser radar detection angle end and a reason why a measurement error occurs.
FIG. 12 is an explanatory diagram of template matching using an inside template in tracking an approaching vehicle.
FIG. 13 is a diagram showing a state of definition of an inner template and a part of right and left templates.
FIG. 14 is an explanatory diagram of the relationship between the position on the image and the three-dimensional position when the template is cut back inside or on the left or right part;
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... own vehicle, 2 ... laser radar, 3 ... camera, 4 ... laser radar data memory, 5 ... vehicle detection part, 6 ... intensity distribution type creation part, 7 ... vehicle tracking part using intensity, 8 ... laser radar measurement Evaluation section 9, 9 image memory, 10 vehicle template creation section, 11 vehicle tracking processing section, 12 image processing tracking reliability determination section, 13 integrated processing section, 14 optical axis of camera, 15 laser radar Scanning surface, 16: Camera optical axis and central axis of laser radar, 20A, 20B: Other vehicles, 21: Road reflector, 31: Camera lens, 32: Center of actual vehicle, 33: Imaging surface, 41 ... Reference template creation image, 42: input image, 43: scanning range, 51: laser radar detection angle, 52: camera detection angle, 53: area where laser radar can measure with higher accuracy, 54: image processing Is a region where high-precision measurement is possible, 55: measurement distance resolution of laser radar, 56: measurement distance resolution of image processing, 61: outer template, 62: inner template, 63: defined template, 64: vehicle internal tracking template , 65... Vehicle right side tracking template, 71... Right side of the inside template, 72.

Claims (8)

Detection wave output means for scanning forward in the traveling direction of the vehicle,
Vehicle detection means for detecting a preceding vehicle by a reflection point reflecting the detection wave output by the detection wave output means,
Considering also the distance measured by the detection wave output means, intensity distribution type creating means for creating an intensity distribution type of the reflection point,
Vehicle tracking means using intensity to measure the position of the preceding vehicle while also utilizing the intensity distribution,
Detection wave output means measurement evaluation means for evaluating the reliability of the tracking result of the preceding vehicle by the detection wave output means,
A camera for imaging the front of the vehicle in the traveling direction,
Considering also the distance measured by the detection wave output means, vehicle template creation means to create a tracking template of the preceding vehicle at a position on the image corresponding to the position detected by the detection wave output means,
Vehicle tracking processing means for performing vehicle tracking by template matching in consideration of a size change due to a distance change from the created template;
Image processing tracking reliability determination means for evaluating the tracking reliability of the preceding vehicle by the image processing of the camera,
Integrated processing means for integrally determining a tracking evaluation result by the detection wave output means and a tracking evaluation result by the image processing and calculating a final position of the preceding vehicle with respect to the own vehicle. Forward vehicle tracking system. At a distant location equal to or greater than a predetermined distance, the data of the detection wave output means is used, and at a short distance smaller than the predetermined distance, the data of the camera is used,
When the preceding vehicle is detected at the detection angle end having a smaller detection angle among the detection wave output means or the sensor of the camera, data of the sensor having a larger detection angle is used. The forward vehicle tracking system according to claim 1. When the detection angle of the detection wave output means is smaller than the detection angle of the camera,
The determination as to whether or not the preceding vehicle is detected at the detection angle end is based on the angle at which the variation in the value of the distance between the detection points on the preceding vehicle by the detection wave output means tends to increase, or Judgment from observing the phenomenon that the variation of the distance value of the detection point becomes large,
3. The front vehicle tracking system according to claim 2, wherein the data of the camera is used when the variation increases. 2. The front vehicle tracking system according to claim 1, wherein a plurality of templates are used in the vehicle tracking in the template matching. 5. The front vehicle tracking system according to claim 4, wherein the plurality of templates have a magnitude relationship with the image data, one being the entirety of the front vehicle and the other being a part of the front vehicle. When the distance cannot be measured by the detection wave output means, template matching is used in which the distance within the scanning range of the image is shifted by a predetermined number of pixels and the distance change by a minute distance change Δz from the previously measured distance is enlarged or reduced. 2. The forward vehicle tracking system according to claim 1, wherein: Scanning the forward direction of the own vehicle by the detection wave output means,
An image of the front of the vehicle in the traveling direction is taken by an electronic camera,
Detecting a reflection point reflecting the detection wave output by the detection wave output means,
Measuring the relative position of the reflection point with respect to the own vehicle, detecting the vehicle ahead from the reflection point,
Create an intensity distribution type indicating the intensity distribution characteristics of the preceding vehicle from the intensity distribution of the reflection point,
Using the image captured by the camera, to create a template for tracking the preceding vehicle at a position on the image corresponding to the position detected by the detection wave output means,
The forward vehicle is tracked by two means of intensity distribution matching of the detection wave output unit and template matching in image processing, and the position of the forward vehicle is measured by a measurement value with a higher tracking certainty. The forward vehicle tracking method. In the vehicle tracking in the template matching, a plurality of templates having a magnitude relationship with image data, one being the whole of the front vehicle, and the other being a part of the front vehicle, are used. The method for tracking a forward vehicle according to claim 7.

Images