JP2019053011A - Object speed calculation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To appropriately calculate a speed of a distant object by reducing the influence of noise included in the distance information of the distant object. SOLUTION: Among recognized objects, a distant object determination unit 51 determines an object having a set ratio distance or more with respect to a recognition limit distance which is a limit recognizable by a stereo camera 2 as a distant object, and determines the time. When the recognized object is determined to be a distant object, the scale changing unit 52 changes the time scale for speed calculation from the normal time scale to the expansion direction. The velocity calculation unit 53 calculates the velocity by the distance change per time scale Δt for an object at a normal distance instead of a distant object, and when it is determined to be a distant object and the time scale is changed, the distant object By calculating the speed of a distant object based on the change in distance on the time scale with respect to the above, it is possible to appropriately detect an object having a large approach speed even in a distant place where the accuracy of distance information is a concern. [Selection diagram] Fig. 1

Description

  The present invention relates to an object speed calculation device that calculates the speed of an object based on distance information obtained from a captured image of a stereo camera.

  2. Description of the Related Art In recent years, in vehicles such as automobiles, a technology has been adopted in which a driving environment is imaged with a camera and the driving environment is recognized by image processing, avoiding collisions with front obstacles, tracking control for preceding vehicles, wandering and lane departure. Various support controls for drivers, such as alarm control and steering control, have been put into practical use. In particular, as disclosed in Patent Document 1, a technology for three-dimensionally recognizing a driving environment using a stereo camera that captures the same object from different viewpoints is adopted as one of the leading technologies in various types of support control. Has been.

Japanese Patent No. 3315054

  However, in the technique of recognizing an object by processing a captured image of a stereo camera, the speed of the object can be calculated from a position change amount (distance change amount) at a predetermined time interval (time scale). Since the distance information of a certain target object includes more noise than the distance information of an object that is relatively close, there is a case where the accurate distance of the target object cannot be obtained.

  In particular, when there is an object that approaches the host vehicle at a high speed from a distance, it is necessary to calculate the speed immediately after recognizing the object in the distance in order to perform brake control for collision avoidance and steering control for avoidance. However, even if the speed is calculated on a time scale similar to that of an object at a normal distance, it is difficult to ensure the accuracy necessary for brake control and steering control.

  The present invention has been made in view of the above circumstances, and provides an object speed calculation device capable of appropriately calculating the speed of a distant object by reducing the influence of noise included in the distance information of the distant object. It is an object.

  An object speed calculation device according to an aspect of the present invention is an object speed calculation device that calculates the speed of an object based on distance information of the object obtained by processing an image captured by a stereo camera, the stereo camera A far object determination unit that determines the object that is equal to or larger than the set distance with respect to the limit distance that can be recognized by the remote object, and calculates the speed of the object when the object is determined to be the far object. A time scale changing unit that changes the time scale when performing from the normal time scale to the enlargement direction, and when the object is not the far object, the speed of the object is calculated on the normal time scale, and the object is In the case of a distant object, a speed calculation unit that calculates the speed of the object on a time scale obtained by enlarging the normal time scale is provided.

  According to the present invention, it is possible to appropriately calculate the speed of a distant object by reducing the influence of noise included in the distance information of the distant object.

Configuration diagram of stereo image processing system Explanatory diagram showing the principle of stereo distance measurement Explanatory diagram showing the relationship between parallax and distance Flow rate calculation flowchart

Embodiments of the present invention will be described below with reference to the drawings.
In FIG. 1, reference numeral 1 denotes a stereo image processing system that performs various recognition processes by processing an image captured by a stereo camera 2. It is installed as a function of the driving support system that performs driving support.

  In this embodiment, the stereo image processing system 1 stores a stereo camera 2 including two cameras 2a and 2b, a pre-processing unit 10 that pre-processes imaging signals of the stereo camera 2, and a pre-processed captured image. The input image memory 15, the image correction unit 20 that performs various correction processes on the captured image stored in the input image memory 15, and the stereo matching process for the output from the image correction unit 20 to image distance information A stereo processing unit 30 for generating the distance image, and an output image memory 40 for storing the distance image generated by the stereo processing unit 30 and the images captured by the cameras 2a and 2b (original images after preprocessing and correction processing); A recognition processing unit 50 that performs various recognition processes using the distance image and the original image obtained by the stereo camera 2 is provided.

  The stereo camera 2 is composed of two cameras 2a and 2b synchronized with each other with variable shutter speed. One camera 2a is a reference camera for capturing a reference image for stereo processing, and the other camera 2b is for stereo processing. As a comparative camera for capturing a comparative image, the optical axes of the cameras are arranged in parallel. The cameras 2a and 2b are arranged, for example, at a predetermined baseline length (optical axis interval) inside the front window at the upper part of the passenger compartment so that the optical axes of the cameras 2a and 2b are parallel to each other.

  In the present embodiment, each camera 2a, 2b has an image sensor such as a CCD or a CMOS image sensor, and an amplifier that amplifies an analog image signal from the image sensor or a digital signal having a predetermined number of bits. It is configured as an imaging unit incorporating various circuits such as an A / D converter that converts the signal.

  The pre-processing unit 10 performs brightness correction and shading correction including electronic shutter control for the cameras 2a and 2b, amplifier gain / offset control and adjustment, and γ correction using a lookup table (LUT). The preprocessing unit 10 performs address control when data is written to the input image memory 15, and a pair of camera images (digital images) are stored at predetermined addresses in the input image memory 15.

  The image correction unit 20 sets image processing coordinates, image size adjustment, parameter settings for setting various parameters in address control, etc., and correction processing for correcting optical positional deviation including lens distortion corresponding to a pair of captured images Affine correction for performing filter correction, filter correction for performing noise removal processing, and the like are performed.

  In the affine correction, the image data read from the input image memory 15 is distorted by the lenses of the cameras 2a and 2b, the displacement of the mounting positions of the cameras 2a and 2b, the variation of the focal length, and the imaging of each camera. A process of correcting an optical position shift caused by a surface shift or the like is executed. Optical positional deviation correction is performed by geometrically correcting rotational deviations, translational deviations, etc. in images captured by the cameras 2a, 2b using an affine transformation correction table, and nonlinear distortions such as lens distortions are nonlinear. The correction is made by a correction table, whereby the optical positions of the cameras 2a and 2b are equivalently precisely adjusted.

  In the filter correction, noise included in the digital image (gradation image) stored in the input image memory is removed using a spatial filter such as a 3 × 3 filter. This noise removal processing is applied to image data that has undergone geometric correction including lens distortion correction by switching the corresponding filter value.

  The pair of image data output from the image correction unit 20 is input to the stereo processing unit 30. The stereo processing unit 30 obtains a shift amount (parallax) of a corresponding position from a stereo image (reference image and comparison image) of the surrounding environment outside the host vehicle imaged by the stereo camera 2, and based on the shift amount. A distance image is generated and stored in the output image memory 40.

  In stereo matching processing, a small area (also referred to as a block or window, hereinafter referred to as a block) is set around a certain point in the reference image, and the same size is set around a certain point in the comparison image. A well-known region search method in which a corresponding block is provided to search for corresponding points can be employed. In the corresponding point search process by this region search method, the correlation calculation with the block of the reference image is performed while shifting the block on the comparison image, and the shift amount of the coordinates of the block having the largest correlation value is calculated. This shift amount is replaced with, for example, a luminance value at a corresponding position in the image coordinate system, and stored as a distance image in an image form.

  The recognition processing unit 50 calculates the distance in the real space using the distance information included in the distance image generated by the stereo processing unit 30, and performs various recognition processes. For example, the distance information from the distance image is subjected to a grouping process for grouping data within a predetermined threshold to extract white line data, side wall data such as guardrails and curbs along the road, and Objects are classified and extracted into other three-dimensional objects such as motorcycles, ordinary vehicles, large vehicles, pedestrians, and utility poles.

  The recognized data of each three-dimensional object is calculated for each position in a coordinate system with the own vehicle as the origin and the longitudinal direction and the width direction of the own vehicle as axes. In particular, in the vehicle data of two-wheeled vehicles, ordinary vehicles, and large vehicles, the length in the front-rear direction is estimated in advance, for example, 3 m, 4.5 m, 10 m, etc., and the width direction indicates the center position of the detection width. Using this, the center position where the vehicle exists is calculated.

  Further, the recognition processing unit 50 also functions as an object speed calculation device, calculates a relative speed with respect to the host vehicle from the change in the axial direction of the distance from the host vehicle with respect to the three-dimensional object data, and automatically calculates the relative speed. By calculating in consideration of the speed of the vehicle, the speed of each three-dimensional object in each axial direction is calculated. Each information obtained in this way, that is, white line data, side walls data such as guardrails and curbs along the road, and three-dimensional object data (type, distance from own vehicle, center position coordinates, speed, etc.) Therefore, the recognition processing unit 50 recognizes a moving object such as a pedestrian or a light vehicle around the own vehicle, and other vehicles running on a road connected to the road on which the own vehicle runs.

  Such white line data from the recognition processing unit 50, guardrails existing along the road, side walls data such as curbs, three-dimensional object data, and the like are controlled for vehicle control (not shown) via the communication bus 100 forming the in-vehicle network. Sent to the controller. The control controller recognizes the driving environment based on the information from the recognition processing unit 50 and the vehicle information such as the vehicle speed and yaw rate of the host vehicle. For example, the control controller maintains the driving position of the host vehicle in the lane. Lane departure prevention control to prevent the vehicle from deviating outside the driving lane, pre-crash brake control to predict collision risk and avoid collision or reduce collision damage, follow-up traveling according to the capture result of the preceding vehicle It performs driving support control for drivers such as Adaptive Cruise Control (ACC), which automatically switches between constant speed driving.

Here, the accuracy of the speed of the object calculated by the recognition processing unit 50 depends on the distance information obtained from the stereo camera 2. As shown in FIG. 2, in the distance measurement by the stereo method, the shift amount (parallax) of the corresponding point between the reference image captured by the stereo camera 2 and the comparison image is x, and the optical axis interval between the reference camera and the comparison camera When (base line length) is r and the focal length of the lens is f, the distance D from the lens center to the object P can be obtained by the following equation (1) based on the principle of triangulation.
D = r · f / x (1)

  As shown in FIG. 3, the distance D of the object according to the equation (1) decreases as the object is farther, and the parallax x becomes smaller, so that the accuracy is lowered due to the influence of noise or the like. In other words, the distance D obtained by the stereo processing, excluding the influence of the base line length r and the lens focal length f, increases the error of stereo matching of the corresponding point on the image plane as the distance to the target point (object) increases. The effect will increase and the distance accuracy will decrease.

  For this reason, the recognition processing unit 50 includes a far object determination unit 51, a time scale change unit 52, and a speed calculation unit 53 as functional units related to the speed calculation, and calculates the speed in consideration of the distance of the object. Thus, it is possible to appropriately detect an object having a large approach speed even in a distant place where accuracy of distance information is a concern.

  As shown in FIG. 3, the far object determination unit 51 recognizes an object that is not less than a distance DL of a set ratio with respect to a recognition limit distance Dmax that is a limit recognizable by the stereo camera 2 among the recognized objects. Judge as. The distance DL is greatly affected by noise, and it is difficult to obtain the necessary accuracy in vehicle control by speed calculation on a normal time scale. For example, the distance DL is set to DL = 0.8 × Dmax. ing.

  In the present embodiment, a time condition that is within a set time (for example, 1.5 sec) after recognition and a vehicle speed condition that the vehicle speed of the host vehicle is equal to or higher than the set vehicle speed (for example, 100 km / h or more) are added. Then, it is determined whether or not the object is a distant object. Then, for each recognized object, identification information as to whether or not the object is a distant object is given and sent to the time scale changing unit 52 and the speed calculating unit 53 together with the distance information.

  The condition that is within the set time after the recognition can be omitted, but in the present embodiment, as described later, the speed of an object at a normal distance is filtered by the speed calculator 53 as described later. It is calculated by processing. For this reason, in the present embodiment, a condition that takes into account the delay time from when the object is recognized until the filter processing is stabilized is added as a far object determination condition.

  Similarly, although the vehicle speed condition of the host vehicle can be omitted, the necessity of grasping the speed of a distant object with a required accuracy is high when the host vehicle has a high speed and approaches the object rapidly. For this reason, in the present embodiment, a condition that the speed of the host vehicle is equal to or higher than the set vehicle speed (for example, 100 km / h or higher) is added to determine whether or not the vehicle is a distant object.

When it is determined that the recognized object is a distant object, the time scale changing unit 52 changes the speed calculation time scale from the normal time scale to the enlargement direction, and sends it to the speed calculation unit 53. Specifically, when calculating the velocity for an object at a normal distance by the distance change per time scale Δt, for a distant object, as shown in the following equation (2), the normal time scale Δt The speed is calculated with a time scale ΔTL obtained by enlarging at a magnification N.
ΔTL = N × Δt (2)

  The normal time scale Δt is preset as, for example, a time based on the frame rate of the stereo camera 2 (for example, the time between one frame), and the speed of the object (relative speed) is calculated from the distance change ΔD between the frames and the time scale Δt. ) Is calculated. Further, the magnification N can be arbitrarily set, but is set to N = 10, for example, when a situation with a high risk is assumed such as when the speed of the host vehicle is high and the vehicle approaches a distant object at a high speed. The

  When it is determined that the object is not a distant object but an ordinary distance, the speed calculator 53 calculates the speed of the object by filtering position data for a predetermined number of frames, for example. The speed of the object at this normal distance can be obtained by using, for example, a bandpass filter or a high-pass filter having a time constant based on the time scale Δt. This can be realized by a predetermined transfer function that outputs a relative speed V with the host vehicle.

When calculating the speed of an object through a filter, it may take time for the filter operation to stabilize, and it may be difficult to accurately grasp an object that approaches rapidly from a distance due to the influence of noise, etc. . For this reason, when the speed calculation unit 53 is determined as a distant object and the time scale of the speed calculation is changed by the time scale changing unit 52, as shown in the following equation (3), the time calculation unit 53 uses the time scale ΔTL for the distant object. A distance VL (relative speed) of a distant object is calculated from the distance change ΔD.
VL = ΔD / ΔTL (3)

  That is, an object having a large approach speed to the host vehicle has a large distance change per time. Therefore, by calculating a speed VL of a distant object using a time scale ΔTL obtained by enlarging a normal time scale Δt, distance data is obtained. The influence of the contained noise can be reduced.

  In the present embodiment, the speed calculator 53 receives the determination result from the far object determination unit 51 that the object is not a distant object, and calculates the speed on a normal time scale. The unit 53 receives the determination result of the far object determination unit 51, and when the far object determination unit 51 determines that the object is not a far object, the time scale change unit 52 sets the magnification N to N = 1, and the speed calculation unit You may make it send to 53.

  Next, processing related to the object speed calculation of the recognition processing unit 50 will be described with reference to the flowchart shown in FIG.

  In the object speed calculation process, first, in the first step S1, the travel information of the own vehicle and the distance information of the object recognized from the captured image of the stereo camera 2 are read. In step S2, whether the vehicle speed of the own vehicle is equal to or higher than the set vehicle speed. Check for no. If the vehicle speed of the host vehicle is equal to or higher than the set vehicle speed, the process proceeds to step S3. If the vehicle speed of the host vehicle is less than the set vehicle speed, the process proceeds to step S5 to calculate the speed of the object using the normal time scale Δt, and this process is performed. Exit.

  In step S <b> 3, it is checked whether the distance of the object is equal to or larger than the distance DL of the set ratio with respect to the recognition limit distance Dmax of the stereo camera 2. When the distance is less than the distance DL, the process proceeds from step S3 to step S5, and the speed of the object is calculated on the normal time scale Δt. Further, in step S4, it is checked whether or not the target object is within a set time after being recognized.

  When the set time has been exceeded since the target object has been recognized, the process proceeds from step S4 to step S5, and the speed of the object is calculated on the normal time scale Δt. If it is within the set time, it is determined that the object is a distant object. To step S6. In step S6, the speed VL of the distant object is calculated using the time scale ΔTL obtained by enlarging the normal time scale Δt with the magnification N, and the process is exited.

  As described above, in the present embodiment, the time scale for calculating the speed of a distant object is changed from the normal time scale to the enlargement direction, whereby a distant object approaching the host vehicle. Can be appropriately calculated by reducing the influence of noise included in the distance information. In this case, even if the speed calculation responsiveness is reduced by expanding the speed calculation time scale, it is not particularly hindered because it is applied only to a distant object.

DESCRIPTION OF SYMBOLS 1 Stereo image processing system 2 Stereo camera 30 Stereo processing part 50 Recognition processing part 51 Distant object determination part 52 Time scale change part 53 Speed calculation part Dmax Recognition limit distance N Magnification ΔTL Time scale with respect to distant object Δt Normal time scale

Claims (4)

An object speed calculation device that calculates the speed of an object based on distance information of the object obtained by processing an image captured by a stereo camera,
A far object determination unit that determines the object that is equal to or greater than a distance of a set ratio with respect to a limit distance that can be recognized by the stereo camera as a distant object;
When the object is determined to be the far object, a time scale changing unit that changes a time scale when calculating the speed of the object from a normal time scale to an expansion direction;
When the object is not the distant object, the speed of the object is calculated on the normal time scale. When the object is the distant object, the speed of the object is calculated on a time scale obtained by expanding the normal time scale. An object speed calculation device comprising: a speed calculation unit.   2. The object speed calculation device according to claim 1, wherein the far object determination unit further determines whether or not the vehicle is the far object by adding a condition that a speed of the host vehicle is equal to or higher than a set speed.   The said far object determination part further determines whether it is the said far object by adding the conditions which are within the setting time since the said object was recognized, The said far object is characterized by the above-mentioned. Object speed calculation device.   The said time scale change part makes the said normal time scale the time scale based on the frame rate of the said stereo camera, The object speed calculation apparatus as described in any one of Claims 1-3 characterized by the above-mentioned.

Images