JP2000048183A - Roadway recognition device - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To reduce the false recognition rate in a travel path recognition method. A guide line candidate including noise is extracted from a road surface image captured from a vehicle-mounted camera. Furthermore, a guide line candidate pair whose horizontal separation distance is within a predetermined range from the guide line candidate is extracted. From the obtained guide line candidate pair, the y-coordinate in the vertical direction of the guide line forming point and the separation distance at that point are obtained, and the correlation distribution is obtained. There is a predetermined positive correlation between the y-coordinate of the genuine guide line forming point and the distance between the points. This predetermined correlation is applied to the above-mentioned correlation distribution, and only the formation points satisfying the correlation are selected and set as genuine formation points. A guide line is estimated using the genuine formation points. Since noise formation points have been removed, the estimation is less affected by noise. Therefore, the false recognition rate can be reduced.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a travel path recognition device for making an automobile run independently, which is equipped with an image processing system and detects the travel path from a road surface image picked up so that the vehicle can travel safely. The present invention relates to a travel path recognition device that enables the following. In particular, the present invention relates to a travel path recognition device that obtains a correlation distribution between an obtained travel index and a travel path width, and makes a comprehensive determination based on the result. The present invention can also be applied to a travel path recognition device that issues a warning to a driver when an automobile deviates from the travel path.

[0002]

2. Description of the Related Art Conventionally, there is a traveling path detecting device for an independent traveling vehicle using image processing. For example, JP-A-5-31
The traveling path detection device for a mobile vehicle disclosed in Japanese Patent No. 4396 is one of them. It is a system that performs image processing in real time using a camera mounted on a vehicle and detects a guide line. Its feature is that it captures a guide line drawn on the road surface during traveling, divides the captured image into a plurality of search areas (hereinafter referred to as windows), and recognizes the guide line comprehensively. is there. The captured image is two straight lines (L, R) that intersect at a distance as shown in FIG. 11, and a plurality of windows (r1, l1, r2, l2) are provided at a place where the image is expected to appear. ,...) Are set.
l and r mean right and left, and the setting method of the window is the same for the left and right, so here the right window (r1, r
2,...) Will be described.

[0003] First, a window r1 is set in the lower right at a predetermined size as an initial window. Since the initial window is set to be large, the guide line is reliably captured. In that area, a differential image is obtained by performing image processing such as a smoothing operation, a binarization operation, and a Sobel operation. Since the differential image emphasizes the change point of the intensity, the guide line is obtained as a pair of left and right edges. The coordinates of these edge pairs are obtained by searching for a sharp change in intensity in the horizontal direction on the differential image, and the representative point of the guide line is obtained by taking the center point of the plurality of edge pairs. Then, the center of the next window (for example, r2) is set to be on the extension line, and the same operation is repeated several times. The first feature is that by repeating this, a point sequence consisting of the center point of the edge pair is finally recognized as a guide line.

However, curbs, ruts,
Due to road surface flaws, various edges appear on the image. In such a case, first, the distance between each edge pair is calculated to determine whether or not the line is a genuine guide line. The distance between the edge pairs is compared with a guide line width registered in advance. If the distance is outside a predetermined range, the distance is excluded. If the distance falls within the predetermined range, it is determined as a guide line candidate. Then, the midpoint is selected as a representative point of the guide line. Therefore, this representative point increases or decreases depending on the amount of noise. For example, a guide line may be recognized at three points, or a guide line may be recognized at more than ten points.

A second feature is that the size of the window is changed by utilizing the size of the representative points. That is, in the traveling path detecting device for a mobile vehicle disclosed in Japanese Patent Application Laid-Open No. 5-314396, the reliability of the guide line is evaluated by the following index to determine the size of the next window.

## EQU00001 ## Reliability = number of candidate points / number of scans in each window (1) For example, if there is no noise in a certain window and only a guide line is observed, the number of candidate points and the number of scans match. Its reliability is 1. When there is missing noise, the number of candidate points is smaller than the number of scans, so the reliability is smaller than one. That is, when the noise is large, the reliability decreases, and when the noise is small, the reliability increases. Then, the next window r is calculated according to the index represented by the equation (1).
2 is determined. That is, if the reliability is high, it is assumed that the noise will be small next time, and the size of the next window is set small. Conversely, if the reliability is low, it is possible that noise has been caught, and the size of the window is set to be large in order to reliably capture the guidance line next time.

By repeating such an operation, windows r3, r5,... Of different sizes are obtained thereafter, and the guide line is finally recognized. Therefore, when there is a lot of noise, the guidance line can be reliably detected, and when there is little noise, the guidance line can be detected efficiently, and safe independent traveling is possible.

[0007]

However, on actual roads, signs, road surface marks, and the like may be drawn with the same width as the guide lines. In some cases, the overtaking prohibition line (yellow) and the center line (white) are drawn twice. Further, a rut after rain or the like also generates a similar edge pair. In such a case, the index in the above equation (1) exceeds 1, and although there is noise, the next window width may be reduced and an incorrect line segment may be recognized as a guide line. That is, it works effectively for the loss of the guide wire, but cannot cope with the case where there are many analogous guide wires.

Further, in the above-mentioned conventional method, continuity is assumed for a guide line and discontinuity is assumed for noise, and an algorithm is assembled based on the assumption. However, when a road is photographed, for example, there is a forbidden zone drawn in oblique lines, and noise may appear continuously. In such a case, the above method may continue to increase the window width, and may capture more noise. Even in such a case, the above method does not function as a guide line recognition device. Therefore, reliable independent running has not always been guaranteed.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems. The vertical coordinates of a genuine guide line forming point on an image pickup screen and the horizontal travel path width at that point are predetermined. Focusing on the fact that there is a correlation, estimating genuine guide lines based on the correlation, and ensuring safe driving even on roads where missing guide lines or confusing lines similar to guide lines such as ruts occur. It is to provide a recognition device.

[0010]

In order to achieve this object, a travel path recognition apparatus according to the present invention captures an image of a travel path using an image pickup device mounted on an automobile and uses the obtained road surface image as a travel index. A travel path recognition device for recognizing a guide line, comprising: a guide line candidate extracting unit for extracting a plurality of guide line candidates from a road surface image; and a road surface image from a plurality of guide line candidates extracted by the guide line candidate line extracting unit. A guide line candidate pair extracting means for extracting a guide line candidate pair whose horizontal separation distance is within a predetermined range, and a vertical line image of a formation point forming one of the guide line candidates of the guide line candidate pair. A guide line forming point extracting unit that takes a correlation distribution between the coordinates of the direction and the separation distance of the guide line candidate pair at the forming point, and sets a forming point having a predetermined correlation as a guide line forming point, and a guide line forming point extracting unit Induction extracted by With forming point and a guiding line estimating means for estimating the authenticity of the guide wire. The vertical direction on the image is the projection of a function of the position and height of the vehicle in the real space in the traveling direction, and the horizontal direction on the image is the projection of the horizontal position in the real space. Therefore, the vertical direction and the horizontal direction on the image are used in this sense, and when the horizontal axis of the camera is inclined from the horizontal direction in the real space, the projection on the horizontal image in the real space is projected on the image. And the direction perpendicular to the horizontal direction is defined as the vertical direction.

[0011]

The guide line candidate extracting means, which is a component of the present invention, extracts a plurality of guide line candidates from a road surface image captured by an imaging device mounted on an automobile.
This extraction includes, for example, binarization processing, Sobel operation processing for edge extraction, and Hough for extracting continuous linear components.
Conversion processing and the like are performed. As a result, one guide line is converted into a pair of edge pair lines, and those having a distance between the edge pair lines within a predetermined range are extracted as guide line candidates.
The extracted guide line candidates include not only genuine guide lines but also analogs thereof, and are usually plural.

The traveling path is composed of two guide lines having a predetermined horizontal separation distance on a captured image.
The guide line candidate pair extracting means extracts a guide line candidate pair whose separation distance is within a predetermined range from the plurality of guide line candidates.

In addition, the guide line candidate pairs include, for example, similar objects such as ruts. In order to determine this, the guide line forming point extracting means calculates the vertical coordinate on the image of the forming point forming one of the guide line candidates of the guide line candidate pair and the horizontal separation distance at that point. Take the correlation distribution of This is because the separation distance (travel path width) of the genuine guide line pair projected on the imaging screen increases downward (y direction), which is a vertical direction on the screen. That is, there is a positive correlation between the horizontal separation distance on the image and the y coordinate of one of the guide line forming points. Therefore, when a correlation distribution is obtained, the distribution is substantially linearly distributed, and the center line thereof has a predetermined angle with respect to the y-axis, which will be described later. Utilizing this relationship, the guide line forming point extracting means obtains the above-mentioned correlation distribution even for the image during traveling, and obtains a histogram in the direction of the predetermined angle. By selecting the formation point having the maximum frequency, a guide line formation point is extracted.

The guide line estimating means estimates a genuine guide line using the guide line forming points extracted by the guide line forming point extracting means. For this estimation, for example, a least squares method or an extended Kalman filter is desirable.

As described above, a genuine guide line forming point is obtained by calculating the correlation distribution between the vertical coordinates on the screen of any one forming point of the guide line candidate pair and the horizontal separation distance at that point. Since the guide line is determined, even if noise such as a missing guide line or a rut occurs during image processing, the influence of the guide line can be almost eliminated. Therefore, a genuine guide line can be reliably obtained, and a more reliable travel path recognition device can be obtained.

[0016]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a configuration diagram showing an example of the present embodiment.
100, a ROM 110 in which a recognition program is written, and a RAM 12 as a work area memory when the program is executed.
0, an I / O interface 130 for inputting / outputting to / from a control device (not shown), and an image input device 160 which performs high-speed A / D conversion of a video signal sent from a camera and takes it into a frame memory 165 dedicated to images. It is configured. All of these components are connected by a system bus 140 including an address bus, a data bus, and various signal lines.
Data is transmitted and received and controlled by various programs written in the CPU 100 and the ROM 110. All means constituting the present invention are formed by the traveling road recognition processing program stored in the ROM 110 and the above-described computer system that executes the program.

Next, the operation of the traveling road recognition apparatus according to the present invention will be described with reference to the flowchart of the traveling road recognition program shown in FIG. 2 and the processing results shown in FIG. Here, the guidance line is a white line drawn on both sides of the travel path. The image processing apparatus is turned on by a guide line recognition start switch (not shown), and is executed from step S100.

In step S100, the image input device 16
With the value of 0, the road surface image is taken into the frame memory 165 from the camera 150 as the imaging device. The frame memory 165 includes a plurality of RAMs.
One unit (one sheet) is a total of 512 × 512 RAM corresponding to each pixel of the CCD image sensor. An arbitrary pixel is represented by coordinates (x, y) and intensity I, and the intensity I is 0
It is digitized in ~ 255 steps. The upper left corner of the screen is the origin of the xy coordinate system, where y is set vertically downward and x is set rightward in the horizontal direction.

Next, the process proceeds to step S110. Step S
Reference numeral 110 denotes pre-processing for image processing described later, and a plurality of guide line candidates are extracted from the road surface image. Here, image processing or the like is first performed on the image captured in the frame memory 165 by Sobel operation or the like which is a matrix weighting operation, and a differential image (angle image) representing a density image and a density gradient is calculated, respectively. .

Since the differential image emphasizes the change point of the intensity, the guide line is obtained as a pair of left and right edges. For example, as shown in FIG. 4, the edge pair has a positive peak on the left side and a negative peak on the right side. Coordinates (Xi, Yk), (Xj, Yk) of those edge pairs
Is obtained by searching for positive and negative peaks in the horizontal direction on y = Yk of the differential image. Then, a point sequence or a line segment having an edge pair satisfying the following expression is extracted as a guide line candidate.

Dmin <Xj-Xi <Dmax (2) where Xj-Xi is the distance between the edge pairs, which is the guide line width, Dmin is its minimum value, and Dmax is its maximum value.
This is because the line width of the guide line is small on the far side and large on the near side on the imaging surface. A guide line candidate is selected by the equation (2). Xi is adopted when selecting a branch road to the left, and Xj is selected when traveling continuously. By performing such processing and selecting one of Xi and Xj, the guide line candidate shown in FIG. 3A is extracted on the image.

[0021] The travel path is usually composed of a pair of guide lines. The distance is, for example, about 3.
It is regulated to 5 m. In the following step S120, a guide line candidate pair and its data are extracted from the plurality of guide line candidates using the constraint condition. The method will be described with reference to the detailed flowchart shown in FIG. 5 and the sectional view taken along the candidate line shown in FIG. FIG. 6 shows the scanning line k shown in FIG.
FIG.

In step S121 of FIG. 5, k representing a scanning line index is initialized. The scanning line index k represents a division number for dividing FIG. 3A in the y direction.
For example, when dividing into 20, the last number is k = 19. Subsequently, the process proceeds to step S122, where the scanning line index k is updated, and a new scanning line k is designated.

In step S123, a peak on the scanning line k is searched, and numbers 1 to N are assigned to candidate points of the guide line candidate (FIG. 6). The coordinates are also stored. For example, the α-th candidate point on the scanning line k is Pkα (Xkα, Y
k). Xkα and Yk are an x coordinate and ay coordinate, respectively. Subsequently, the process proceeds to step S124.

In step S124, the number of candidate points on the scanning line k is checked. If the number of candidate points is 0 or 1, no guide line pair has been observed, and thus step S122 is performed.
And the next scan line is examined. 2 in step S124
If more than one candidate point Pk is detected, step S1
It moves to 25.

In step S125, any two sets of candidate points Pk are selected from a plurality of candidate points Pk detected as shown in FIG.
All combinations of α (Xkα, Yk) and Pkβ (Xkβ, Yk) are generated. Then, a separation distance Wkαβ is calculated for the set (however, N ≧ β> α). The index α of the separation distance Wkαβ is the separation distance counting start point in the x-axis direction, and β is the end point. Subsequently, the process proceeds to step S126.

In step S126, a set of candidate point pairs satisfying the following equation is selected from all the separation distances Wkαβ calculated for all sets of candidate points. Here, Wmin is its minimum value and Wmax is its maximum value. The predetermined distance is set as the separation distance Wkαβ, which is the width of the traveling path, since the distance is small and the vicinity is large in the captured image.

Wmin <Wkαβ <Wmax (3) If this equation (3) is satisfied, the data is determined as a guide line candidate pair, and data Dk including the start point indices k and α and the separation distance.
m (Xkα, Yk, Wkαβ) is stored as the guide line candidate pair data. Here, m is an index assigned to a pair of guide line candidate pairs satisfying the expression (3) on the scanning line k.

Subsequently, the flow shifts to step S127, where it is checked whether or not the index k of the scanning line k is the last. If not final, the process proceeds to step S122 and the above process is repeated. Then, Dkαm (Xkam, Ykm, W
kαβm). FIG. 7 shows the data structure. On the other hand, the index k of the scanning line k
If this is the last, this routine ends, and step S13
Go to the yw correlation analysis of 0 (FIG. 2).

In step S130, the data group Dk
m (Xkα, Yk, Wkαβ) to (Yk, Wkαβ)
And a correlation distribution is obtained on the yw plane (FIG. 3).
(B)). If there is only a genuine guide line pair without noise in the travel road image, the separation distance of the projected guide line pair increases as the y-coordinate at the bottom of the screen increases. Thus, a true guide line has a positive correlation. Also, its distribution on the yw plane is located almost on a straight line,
The center line forms a predetermined angle θ with the y-axis (FIG. 3B ′).

The predetermined angle θ is uniquely defined from the geometrical relationship between the guide line pair via the camera and the image sensor. As shown in FIG. 8A, when a pair of guide lines A1 and A2 are photographed through an imaging camera lens fixed at a height H, a two-dimensional C
On the CD image sensor, the image is projected as shown in FIG.
The leading ends of the guide lines A1 and A2, which are sufficiently far from the camera, are located at the center of the screen on the image sensor, and the guide lines A1 and
The guide line A2 is projected as a1 and a2 on the bottom of the screen.
A virtual line WL at a height H away from the optical axis of the camera is projected on the image sensor with a width wl at a distance h from the center of the screen.
Therefore, a relationship of tan θ ≒ wl / h = WL / H is obtained. Thus, the predetermined angle θ is set in advance by WL and H. Although shown in FIG. 8 as a straight road for the sake of explanation, the above relationship is approximately established even when the guide lines A1 and A2 are curved.

According to this model, a similar correlation distribution is obtained for a running image, and a density value sum (histogram) is obtained in the direction of a predetermined angle θ. The maximum frequency component is yw of the guide line forming point. (FIG. 3C). Index α of the yw component (Yk, Wkαβ) of this guide line forming point
Xy components (Xkα, Yk) and (Xkβ,
Yk) and plotted on the xy plane (FIG. 3).
(D)). After that, it moves to step S140.

In step S140, an approximate curve is obtained by the least squares method or the extended Kalman filter method based on the guide line forming points converted into xy coordinates, and the parameters and the like are output to a control device or the like in step S150. . By repeating such a routine, the guide line of the traveling path is always recognized. Noise generated by a rut, a curb, or the like is removed by the yw correlation analysis in step S130, so that erroneous recognition is reduced. Therefore, the traveling path recognition device enables safe independent traveling.

While one embodiment of the present invention has been described above, various other modifications are possible. For example, in the yw correlation analysis in step S130, the analysis was performed using the sum of density values. As shown in FIG. 9, a straight line having a predetermined angle θ with respect to the y-axis was scanned in the direction of the arrow, and the number of candidate points on the straight line was determined. May be counted. The point on the straight line where the most candidate points are detected is the guide line forming point.

In step S130, the xy component is obtained from the yw component, and the process proceeds to step S140, which is a guide line estimation. In some cases, three guide lines may be recognized in the xy coordinates. For example, this is a case where the vehicle travels on a central guide line. In preparation for this case, a step (not shown) for obtaining the sum of density values on the xy plane may be added to the beginning of step S140 as shown in FIG. When there is a guide line at the center, the maximum frequency appears at the center as shown in the figure. As a result, a distinction can be made between normal traveling and a warning or the like may be output from step S150.

In addition, various modifications are conceivable. A guide line candidate pair is extracted from a road surface image taken from a vehicle-mounted camera, and the y coordinate of the formation point of the guide line candidate and the horizontal separation distance at the point are determined. Any method can be used as long as the guide wire is estimated by correlation analysis.

[Brief description of the drawings]

FIG. 1 is a configuration diagram showing a configuration of a traveling road recognition device according to one embodiment of the present invention.

FIG. 2 is a flowchart showing a processing procedure of a CPU of the travel path recognition device.

FIG. 3 is an explanatory diagram showing image data of a candidate line generated in a CPU processing process.

FIG. 4 is an explanatory diagram of an edge pair.

FIG. 5 is a flowchart showing a processing procedure of a CPU for extracting a guide line candidate pair.

FIG. 6 is an explanatory diagram of a guide line candidate point and a guide line candidate point pair on a scanning line k.

FIG. 7 is a structural diagram showing a data structure of guide line candidate pair data.

FIG. 8 is a diagram illustrating a relationship between a guide line on a road surface and a guide line on an image.

FIG. 9 is an explanatory diagram of a method of extracting a guide line forming point in a modified example.

FIG. 10 is an explanatory diagram showing a guide line recognition method when the vehicle deviates from the travel path.

FIG. 11 is an explanatory diagram showing a conventional method for determining a guide line.

[Explanation of symbols]

 Reference Signs List 100 CPU 110 ROM 120 RAM 140 System bus 150 Camera 160 Image input device 165 Frame memory A1, A2 Guide wire W Separation distance

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G06T 7/00 G06F 15/70 330E F-term (Reference) 2F065 AA01 CC40 FF01 FF04 JJ03 JJ26 QQ03 QQ04 QQ13 QQ17 QQ18 QQ24 QQ31 QQ32. FA72

Claims (1)

[Claims] An apparatus for recognizing a guide line serving as a travel index from an obtained road surface image by capturing an image of a travel road with an image pickup device mounted on an automobile, comprising the steps of: A plurality of guidance line candidate extracting means, and a plurality of guidance line candidates extracted from the plurality of guidance line candidates extracted by the guidance line candidate line extracting means, for extracting a guidance line candidate pair having a horizontal separation distance of a road surface image within a predetermined range. A line candidate pair extracting means, and a correlation between a vertical coordinate of a road surface image of a formation point forming one of the guide line candidates of the guide line candidate pair and the separation distance of the guide line candidate pair at the formation point. Take the distribution,
A guide line forming point extracting unit that sets a forming point having a predetermined correlation as a guide line forming point; and a guide line estimation that estimates a genuine guide line using the guide line forming points extracted by the guide line forming point extracting unit. And a means for recognizing a traveling road.