JP2001021345A - Obstacle detecting device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an obstacle detecting device capable of precisely detecting an approaching obstacle without any erroneous detection. SOLUTION: The concentration change data of read pictures are prepared so that feature extraction can be realized, and an obstacle can be estimated (S40, S50). Correlation processing is operated between the read pictures and an already existing template is operated so that an optical flow can be detected, and the obstacle can be estimated from the flow vector and the already existing template (S30). The already existing template is the template of the obstacle prepared in processing until the precious time (S110, S120, S150). Then, matching processing for collating the estimated obstacle obtained by the correlation processing and the estimated obstacle obtained by the feature extraction is operated (S90), and when they are matched, the estimated obstacle is specified as the obstacle (S100).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an obstacle detecting device for detecting a moving obstacle such as another vehicle approaching a vehicle.

[0002]

2. Description of the Related Art Conventionally, a camera (imaging element) mounted on a vehicle captures an image behind the vehicle and processes the image data to detect a moving object such as another vehicle traveling behind the vehicle. Object detection devices are known. In this type of apparatus, an optical flow method is known as an image processing method for detecting a moving object such as another vehicle approaching from behind.

In the optical flow method, a flow vector (motion vector) is calculated from the correlation between the pixel densities of the current frame and the past frame, and a moving object is determined from the vector to detect an obstacle. In particular,
Divide the image into multiple small blocks, search for the block with the highest correlation with each block from the image after a fixed time, calculate the vector for each block, and identify the obstacle by combining the vector result for each block I do.

[0004]

However, for an obstacle whose size changes, such as a vehicle approaching from behind, the correlation between each block in the past image and the image after a certain time has elapsed. Therefore, good matching could not be obtained, and a highly accurate optical flow could not be obtained. For this reason, there has been a problem that the detection accuracy is low, such as erroneous detection in which an obstacle is detected when there is no obstacle.

In order to prevent erroneous detection when the size of an image changes due to the approach of an obstacle as described above, it is conventional to increase the number of block divisions of an image. There is a problem that the processing time is long and the detection accuracy when the size does not change is reduced.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problem, and an object of the present invention is to provide an obstacle detecting device capable of detecting an approaching obstacle with high accuracy without erroneous detection.

[0007]

In order to achieve the above object, according to the first aspect of the present invention, an image pickup means and a feature extraction of current image data obtained from the image pickup means are performed, and the feature extraction result is obtained. A feature extraction unit for estimating an obstacle based on, and an optical flow detection unit for calculating a motion vector by examining a correlation using previous and current image data obtained from the imaging unit and estimating an obstacle from the motion vector; ,
The gist of the present invention is to include a matching unit that matches the obstacle estimated by the feature extraction unit and the obstacle estimated by the optical flow detection unit.

A second aspect of the present invention is based on the first aspect, wherein the feature extracting means detects a change in density of the image data and performs feature extraction. According to a third aspect of the present invention, in the second aspect of the present invention, the optical flow detecting means includes a template creating means for creating a template in a section area where an obstacle is presumed to exist in the previous image data. The gist is to estimate a obstacle by calculating a motion vector by template matching between a time template and current image data.

According to a fourth aspect of the present invention, in the invention according to any one of the first to third aspects, the template creating means includes a section surrounding the obstacle estimated by the feature extracting means based on a feature extraction result. The point is to create a template in the area.

(Operation) According to the first aspect of the present invention, the feature extracting means performs feature extraction on the current image data acquired from the imaging means, and estimates an obstacle based on the feature extraction result. The optical flow detection means checks the correlation using the previous and current image data, calculates a motion vector, and estimates an obstacle from the motion vector. The collating unit collates the two estimated obstacles respectively estimated by the feature extracting unit and the optical flow detecting unit. As a result of the collation, when it is determined that they match, it is identified as an obstacle. Since the obstacle is specified from the two aspects of the motion and the feature, erroneous determination is reduced and the reliability of the obstacle detection is improved.

According to the second aspect of the present invention, the feature extracting means detects a change in the density of the current image data and performs feature extraction. Simple processing is required because the change in density is detected. Claim 3
According to the invention, the optical flow detecting means creates the template in the section area where the obstacle is presumed to exist in the previous image data by the template creating means. Then, an obstacle is estimated by calculating a motion vector by template matching between the template at the previous time and the current image data. Since a partial area where an obstacle exists is used as a template, the matching process is simplified, and the calculation process of the motion vector can be completed in a short time.

According to the fourth aspect of the present invention, the template creating means creates the template in the area surrounding the obstacle estimated by the feature extracting means based on the feature extraction result.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below with reference to FIGS. As shown in FIG. 2, the obstacle detection device 1 is mounted on an automobile 2 and a camera 3
It has. The camera 3 incorporates an image pickup device (image sensor) 4 as image pickup means. As the imaging device 4, for example, a CCD image sensor or a MOS image sensor is used. The camera 3 is attached to a place where the rear of the automobile 2 can be imaged, for example, a rear bumper or a door mirror as shown in FIG.

The camera 3 is connected to a control device 5 provided in the automobile 2 through electric wiring such as a harness.
The control device 5 includes a microcomputer (hereinafter simply referred to as a microcomputer) 6, an image memory 7, and a memory 8.
The microcomputer 6 has a built-in counter 9. The feature extracting means, the optical flow detecting means, the collating means and the template creating means are constituted by the microcomputer 6.

The image data picked up by the image element 4 is transferred to the image memory 7. The microcomputer 6 is an image memory 7
And performs image processing. Memory 8
Stores program data shown in the flowchart of FIG. 1 for performing image processing for obstacle detection. The microcomputer 6 executes the program data at predetermined time intervals, and detects a moving obstacle such as another vehicle running behind the vehicle. The control device 5 is electrically connected to an alarm device (not shown) to control the alarm device. For example, when detecting an obstacle (other vehicle or the like) suddenly approaching from behind when attempting to change lanes, the alarm device is activated. To inform the operator of the approach of the obstacle.

Next, an obstacle detection method will be outlined with reference to image data shown in FIGS. The program data of FIG. 1 is created based on the following image processing method. Initial recognition FIG. 3 shows a density change image (edge image) obtained by performing density change detection processing (differential processing) on image data (image density data) It read at time t shown in FIG. ) Dt. When an obstacle approaching from behind is a vehicle as shown in FIG. 4, an edge E1 of a contour portion of a vehicle having a sharp contrast such as a boundary between a road surface and a vehicle, and a boundary between a hood and a glass among vehicles is extracted (contour line extraction). Law). The position and the size of the obstacle (vehicle) are estimated from the contour information extracted from the edge image Dt. Specifically, a process of extracting a set of density changes (edges) is performed, and the existence range of the obstacle is estimated from the extraction range. Then, the estimated existence range of the obstacle is partitioned in the image data It of FIG. 4, and the region of the partitioned density image is stored as a template T1 (a thick frame in FIG. 4).

Obstacle Estimation by Optical Flow Next, at time t + 1 after a lapse of a predetermined time, image data (image density data) shown in FIG. 6 is read. An optical flow (motion vector) is calculated from the correlation between the previous (time t) template T1 and the current (time t + 1) image density data It + 1. Then, the initial template T1 (dotted frame in FIG. 6) is moved from the initial position by the flow vector B (arrow in FIG. 6) in the direction of the vector by the moving amount of the vector size in FIG. It is estimated that an obstacle having the size of the template T1 should exist at the position of the thick line frame K. in this case,
Since the obstacles are close to each other and correlated with those having different sizes, the estimated position of the optical flow has low reliability.

Obstacle Estimation by Feature Extraction On the other hand, density change detection processing (differential processing) is performed on the image density data at time t + 1. FIG. 5 shows a density-change image (edge image) Dt + 1 obtained by differentiating the image density data It + 1 at time t + 1 and extracting an edge E2 of a contour portion having a large density change. The position and the size of the obstacle (vehicle) are estimated from the contour information extracted from the edge image Dt. Specifically, a process of extracting a set of density changes (edges) is performed, and the existence range of the obstacle is estimated from the extraction range.

An estimated obstacle based on the optical flow obtained by comparing the two estimated obstacles,
Is compared with the obstacle estimated by the feature extraction obtained in step (1).
That is, it is checked whether there is an edge E2 characterizing the obstacle near the thick line frame in FIG. If an edge characterizing the vehicle is obtained in the vicinity of the thick line frame in FIG. 6, it can be understood that the obstacle estimated from the motion information matches the obstacle estimated from the feature extraction. In other words, if a moving object matches a vehicle-like object, it is identified as a moving vehicle. Obstacles approaching are different in size due to their different sizes, and the correlation is relatively low.The reliability of the optical flow is low and there is a risk of erroneous determination, but the moving object has the characteristics of a vehicle. The erroneous determination is prevented by seeing whether the Then, when the two estimated obstacles are matched by collation, a template T2 at time t + 1 is created based on the contour information of the edge image Dt + 1 as shown in FIG. The determination as to whether an obstacle (vehicle) approaches or separates is determined from the direction of the flow vector B.

In the same manner, the following steps are repeated at predetermined time intervals to detect an approaching obstacle. Here, the reason why the density change image is used for estimating a vehicle-like object is to shorten the processing time by extracting only the characteristic portion.

The program data shown in FIG. The microcomputer 6 executes the program data of FIG. 1 at predetermined time intervals to perform an obstacle detection process. Hereinafter, the obstacle detection processing by the microcomputer 6 will be described with reference to FIG.

First, a step (hereinafter simply referred to as S) 10
Then, the image data is read. In S20, it is determined whether there is an existing template (obstacle template). When there is an existing template, the process proceeds to S30, and when there is no existing template, the process proceeds to S40.

In S30, a correlation process is performed. That is, the existing (for example, time t) template T1 and the current (time t +
1) An optical flow is calculated from the correlation with the image density data It + 1, and an obstacle is estimated from the existing template T1 and the flow vector B. That is, the process of adding the motion of the flow vector B to the existing template T1 shown by the dotted frame in FIG. 6 to move the existing template T1 is performed by calculation, and the thick line in FIG. It is estimated that there will be an obstacle near the range of the frame K.

In S40, the current image density data It + 1
Is subjected to density change detection processing (differential processing) to generate density change data (edge image) Dt + 1. In the differentiation processing, horizontal edges and vertical edges are detected. In the case of the initial processing without the existing template T1, the density change data shown in FIG. 3 in which the density change portion (outline portion) of the vehicle in the image is featured by the differential processing of the image density data It shown in FIG. (Edge image) Dt is obtained.

On the other hand, when the existing template T1 exists, the image density data at time t + 1 shown in FIG. 6 is differentiated, and the density change portion (outline portion) of the vehicle in the image is extracted. The density change data (edge image) Dt + 1 shown in FIG.

In S50, an obstacle (vehicle) is estimated from contour information based on the feature extraction result of the density change data Dt, Dt + 1. In S60, it is determined whether there is an obstacle estimated from the result of the optical flow (correlation processing) in S30. If there is an estimated obstacle, the process proceeds to S70, and if there is no estimated obstacle, the process proceeds to S170.

In S70, it is determined whether there is an estimated obstacle estimated from the density change (feature extraction) result in S50. If there is an estimated obstacle, the process proceeds to S80, and if there is no estimated obstacle, the process proceeds to S130.

In S80, matching processing is performed between the density change estimated obstacle and the correlation processing estimated obstacle. That is, it is checked whether the position and the size of the estimated obstacle estimated to be a vehicle from the feature extraction and the estimated obstacle estimated to be the one moving from the motion based on the flow vector match with each other. . The alignment of the position
In the vicinity of an obstacle (bold frame in FIG. 6) estimated from the motion,
It is checked whether an edge E2 having the characteristics of the vehicle exists. The size matching is based on the obstacle estimated from the motion (FIG. 6).
And the edge E obtained from the feature extraction
This is performed by comparison with the size of the obstacle estimated from (2). The image reading time interval is very short, and the degree of change in the size of the obstacle due to approaching or moving away from the obstacle is limited, so if it is within the allowable error range determined according to the distance from the vehicle to the obstacle, Suppose a match is found.

In S90, it is determined whether or not the result of S80 is consistent. If there is a match, the process proceeds to S100, and if there is no match, the process proceeds to S120. In S100, since the two estimated obstacles match, it is identified as an obstacle.

In S110, a template T2 of the obstacle specified in S100 is created. That is, based on the contour information of the density change data (edge image) Dt + 1 (FIG. 5) of the current time (time t + 1), an obstacle in the image density data It + 1 of FIG. 7 is estimated and partitioned. This section (Fig. 7
(Thick line frame) is created as a template T2.

S120 is a process in a case where the two estimated obstacles do not match, and a template of a new obstacle is created on the assumption that the reason for the mismatch is the appearance of a new obstacle.

The processing of S130 to S160 is a processing in a case where an estimated obstacle is obtained by optical flow (correlation processing) but an estimated obstacle is not obtained by feature extraction (density change). The causes include a case where the edge becomes difficult to catch due to a change in light conditions such as a weather condition or a shade, and a case where the template itself is wrong. If the feature of the edge suddenly becomes unrecognizable in the past several times of data, it is highly likely that such a change is caused by such a change in light conditions. Further, when the edge feature cannot be continuously captured in the past several times of data, there is a high possibility that the template itself is wrong. In addition, since a vehicle in the center of the image cannot suddenly disappear from the screen, an error in the template can be determined from such positional information. Further, since the range in which the vehicle can move in a short time is limited, it is possible to determine a template error from the size of the flow vector. S130 is processing for determining a template error which is a reason why an obstacle cannot be specified from such a viewpoint.

In S130, from the above viewpoint, it is determined whether or not there is a high possibility of a template error based on data such as the past position information of the obstacle, the magnitude of the flow vector, and the past unspecified count value. When it is determined that the possibility of a template error is high, the counter 9
Is incremented and “1” is added to the unspecified number of times.

In S140, it is determined whether or not the unspecified number is equal to or greater than a predetermined value. When the unspecified number is less than the regulation value, the process proceeds to S150, and when the unspecified number is equal to or more than the regulation value, the process proceeds to S160.

In S150, the existing template is held. That is, when the unspecified number of times is less than the regulation value, the existing template is retained as it is because there is a sufficient possibility that the template itself is correct simply because the edge feature cannot be captured due to light conditions or the like.

In S160, the template of the obstacle is deleted. That is, when the unspecified number of times is equal to or larger than the regulation value, the existing template is almost recognized as wrong, and the wrong existing template is deleted.

The processes in S60, S170, and S120 are processes in a case where neither an estimated obstacle by optical flow (correlation processing) nor an estimated obstacle by feature extraction (density change) is obtained. That is, neither the flow vector nor the edge feature can be captured.

In S170, after it is determined in S60 that there is no estimated obstacle by optical flow (correlation processing), the estimated obstacle by feature extraction (density change) (S50)
It is determined whether or not there is. That is, the determination content is the same as S70. If there is an estimated obstacle, the process proceeds to S120 to create a template of a new obstacle, and if there is no estimated obstacle, the current processing cycle ends. S30 constitutes an optical flow detecting means, S40 and S50 constitute a feature extracting means, and S80 to S100 constitute a matching means. Also, S110 and S120 constitute a template creation unit.

By repeatedly executing the flowchart of FIG. 1 at predetermined time intervals, when a vehicle traveling behind appears in the image of the camera 3, estimation based on the flow vector from the movement and feature extraction of the vehicle are performed. It is determined whether or not the vehicle is an obstacle by matching the two estimation results with the estimation based on the vehicle. For this reason, even if the obstacle approaches or separates from the own vehicle and the sizes of the two images at different times are different, erroneous determination hardly occurs.
The detection accuracy is improved. For example, when it is determined that there is a possibility that a sufficient inter-vehicle distance from a vehicle approaching from behind the vehicle may not be ensured when the vehicle changes lanes, a warning is issued to the driver from the warning device by sound or voice. In the present embodiment, at the time of such a warning, a warning error in which a warning is issued even when an approaching obstacle does not exist is reduced.

As described in detail above, according to the present embodiment,
The following effects can be obtained. (1) The estimation result from optical flow (correlation processing) is compared with the estimation result from feature extraction (density change),
Since a method is used in which the two estimation results are consistent and the method is used to identify it as an obstacle, erroneous determination of obstacles due to the difference in size due to the approaching / separating of the obstacles is unlikely to occur. Detection accuracy can be improved.

(2) Since an optical flow method based on template creation of an obstacle and template matching between the template and the image data is employed, the matching process is simplified, the operation time of the optical flow can be reduced, and real-time high-speed processing can be achieved. It can also handle processing.

(3) A contour line for extracting a feature of an outline of an obstacle by a density change detection processing (differential processing) of image data, estimating an existence range of the obstacle from the extracted outline information, and creating a template. Since the extraction method is employed, the time required to create a template can be reduced as compared with the region extraction method, and this also enables high-speed processing.

(4) It has a function of finding and reviewing an error in the template itself by the unspecified process determination (S130), so that optical flow detection is performed by template matching using an obstacle template.
It is possible to prevent erroneous detection of an obstacle due to an error in the template itself.

(5) When the unspecified number of times is equal to or more than the specified value in the unspecified process determination (S130), the template of the observing obstacle is erased, so that the feature of the edge is captured due to a change in light conditions or the like. When there is no template, the template can be kept as it is, and when the mistake of the template is almost certain, the template can be erased.

(6) If there is no match in the matching process (S80), a template for a new obstacle is created, so that existing obstacles and obstacles newly appearing on the screen are distinguished and new obstacles are created. It is also possible to detect the movement of an object while watching it individually.

(7) Since an erroneous detection of an obstacle can be prevented and the obstacle can be detected with high accuracy, an appropriate warning can be given to the driver. The embodiments of the present invention can also be implemented in the following modes.

It is not limited to adopting the optical flow detection method by template matching using an obstacle template. As in the related art, a method of dividing an image into a plurality of small blocks and searching for a block having the highest correlation with each block from the image after a predetermined time can be employed for optical flow detection.
Also in this case, it is possible to prevent an erroneous detection of an approaching obstacle by matching the estimation result with the estimation result from the edge feature based on the density change data.

The method of creating a template is not limited to the method of creating a template based on contour information from the edge characteristics of the density change. For example, although the processing speed becomes slow, a method of creating a template by estimating an obstacle using the density information of the image data as it is can also be adopted.

The template of the obstacle used for the template matching at the time of optical flow detection is not limited to the density data, and density change data may be used. -The obstacle detection device is not limited to detecting an obstacle behind the vehicle. For example, it can also be used to detect obstacles in front of and on the side of the vehicle.

The obstacle detection device may be mounted on a vehicle for purposes other than warning. The present invention can be applied to any device that detects a moving object and performs a certain process based on the detection result. For example, it can be adopted in an automatic driving system.

The technical ideas other than the claims grasped from the above embodiment and other examples are described below together with their effects. (1) In claim 2, the feature extracting means estimates a obstacle by extracting a set of density changes based on density change data. According to this configuration, the feature extraction processing is simple.

(2) In any one of claims 1 to 4,
Unspecified determining means (S130 to S160) for determining whether or not the template is incorrect when an estimated obstacle by the optical flow detecting means is obtained and an estimated obstacle by the feature extracting means is not obtained. ing.

(3) In the technical idea of the above (2),
The unspecified determination means retains the template if the number of unspecified determinations is less than a specified value, and deletes the template if the number of unspecified determinations exceeds the specified value. According to this configuration, the template can be held when the feature of the density change is difficult to catch due to light conditions or the like, and can be deleted when there is an error in the template itself.

(4) Claims 1-4 and (1)-
A vehicle provided with any of the obstacle detection devices according to the technical concept of (3). According to this vehicle, obstacles around the vehicle can be accurately detected.

[0055]

According to the first and second aspects of the present invention, the obstacle estimated from the feature extraction and the obstacle estimated from the motion vector are collated, and when it is determined that they match, the obstacle is determined. Since the obstacle is identified as an obstacle, an erroneous determination is unlikely to occur even for an obstacle whose size changes when approaching, so that the detection accuracy can be improved.

According to the third aspect of the present invention, a template area is defined as a template in a section area where an obstacle is present in image data, and template matching is performed between the template of this partial area and the current image data. Can reduce the calculation time, and can respond to real-time high-speed processing.

According to the fourth aspect of the present invention, the template is created using the characteristic extraction result of the characteristic extracting means.
The template creation processing time can be reduced and real-time high-speed processing can be supported.

[Brief description of the drawings]

FIG. 1 is a flowchart of an obstacle detection process according to an embodiment.

FIG. 2 is a block diagram of an obstacle detection device.

FIG. 3 is a graph showing density change data of an image at time t.

FIG. 4 is an image data diagram at time t.

FIG. 5 is a density change data diagram of an image at time t + 1.

FIG. 6 is an image data diagram at time t + 1.

FIG. 7 is an image data diagram in which a template at time t + 1 is created.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Obstacle detection apparatus, 2 ... Car as a vehicle, 3 ... Camera, 4 ... Image sensor as imaging means, 5 ... Control device,
6: microcomputer as feature extraction means, optical flow detection means, collation means and template creation means, 7: image memory, 8: memory, 9: counter.

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI theme coat ゛ (reference) // G08G 1/16 G01V 9/04 SF term (reference) 2F065 AA02 AA12 AA51 BB15 CC11 DD03 FF04 JJ03 JJ26 QQ13 QQ21 QQ24 QQ32 QQ36 QQ41 QQ51 UU05 2F112 AD03 BA06 CA05 CA12 FA03 FA14 FA21 FA27 FA45 5H180 AA01 CC04 LL01 LL06

Claims (4)

[Claims] 1. An image pickup means, a feature extraction means for extracting a feature of current image data obtained from the image pickup means, and estimating an obstacle based on the result of the feature extraction; The motion vector is calculated by examining the correlation using the image data of the optical flow detecting means for estimating an obstacle from the motion vector, the obstacle estimated by the feature extracting means, and the optical flow detecting means estimating the obstacle. An obstacle detection device comprising: a collation unit configured to collate with an obstacle. 2. The obstacle detection device according to claim 1, wherein the feature extraction unit detects a change in density of the image data and performs feature extraction. 3. The optical flow detecting means includes template creating means for creating a template in a sectioned area where an obstacle is presumed to exist in the previous image data, and includes a template of a previous time template and a current image data. 3. An obstacle is estimated by calculating a motion vector by matching.
An obstacle detection device for a vehicle according to Claim 1. 4. The obstacle according to claim 3, wherein the template creating unit creates a template in a partitioned area surrounding the obstacle estimated by the feature extracting unit based on the feature extraction result. Detection device.