JPH09218937A - Compartment line detector - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately detect a compartment line (white line lane mark) present far from a straight line part on an image. SOLUTION: An alarm main body 20 is broadly divided into an image input part 21, a lane mark detection part 22, a three-dimensional object detection part 23 and a position judgement part 24. Image signals inputted from an image pickup device 10 to the image input part 21 are transmitted to the lane mark detection part 22 and the three-dimensional object detection part 23. In the lane mark detection part 22, in steps S111-S114, edge extraction inside the image, the extraction of lane mark candidates by pairing, Hough transformation and the edge extraction near the lane mark are performed. Also, obtained edge points are transformed from an image coordinate system to a birds-eye coordinate system in the step S115 and the edge points transformed to the birds-eye coordinate system are approximated to a cubic curve by using the method of least squares in the step S116. By outputting the white line lane mark as the function of a cubic polynomial, even the white line lane mark far away is estimated by a curve function.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION The present invention relates to, for example, a CCD.
(Charge Coupled Device) The present invention relates to a marking line detection device that detects a marking line (white line lane mark) on a road by using image information obtained by an imaging device such as a camera.

[0002]

2. Description of the Related Art As a technique for detecting marking lines from an image,
In the conventional detection device, an edge is extracted based on image information obtained from a monocular image, and the extracted edge is subjected to a Hough transform to extract a straight line portion of a lane marking. In addition, based on the straight line portion, edge search on the image was performed to detect the lane markings.

The above processing will be described more specifically below.
FIG. 9 shows an image taken by the image pickup apparatus. The image signal of this video is taken into the image processing device, and for example, edge extraction is performed according to a relative change in brightness. Although there are many methods for edge extraction, as an example, the edge of the dividing line is extracted using the difference between two pixels in the horizontal direction. The result of edge extraction using the difference is shown in FIG. In FIG. 10, white points are edge points (up edges) where the difference is positive, and black points are edge points (down edge) where the difference is negative. Here, paying attention to the edge that crosses the lane marking, it can be seen that the white edge point located on the left side and the black edge point located on the right side are paired in the lane marking part. FIG. 11 shows the result of extracting points that are judged to be division lines by performing pairing (a process of extracting a pair of white edge points and black edge points) using this property. If Hough transformation is applied to this, straight lines such as L1 and L2 in FIG. 12 can be extracted.

[0004]

However, in FIG. 14, the straight line portions of the lane markings obtained by the Hough transformation are as shown by lines a and b in the figure.
It cannot correspond to a right-turned marking line like a line. Further, in a region far from the vehicle where the edge points of the lane markings are hardly extracted, a large error occurs by estimating the lane markings by extending the straight line.

On the other hand, the lane markings on the road are composed of straight lines, arcs, and clothoid curves, so that this can be approximated to a cubic equation at an error level that poses no practical problem. However, in the image input to the image pickup device, the straight line on the road surface is a straight line on the image, but the arc and clothoid curve are completely different. Therefore, when a cubic approximation curve is applied to the demarcation line on the image, as shown by the lines c and d in FIG. 14, the curve is in agreement with the original data to some extent, but there is no data. The predicted line of the area is far from the e and f lines (actual division lines).

Further, as another conventional technique of this type, there is disclosed an "environment recognizing device for a moving vehicle" in Japanese Patent Laid-Open No. 3-28710.
However, in this apparatus, the curve approximation of the lane markings is performed in the image coordinate system. In such a case, the lane marking detection results are as shown by lines c and d in FIG. In other words, the prediction curve could not be obtained in the distant region where there is no edge data.

The present invention has been made in view of the above problems, and an object of the present invention is to provide a marking line detecting device capable of accurately detecting marking lines even in a distant region. is there.

[0008]

According to the present invention, as an image pickup device, for example, a CCD camera whose optical axis is substantially horizontal to a road surface is used, and an image in front of the vehicle is acquired by the CCD camera. Further, in the present invention, an overhead view of the road surface viewed from directly above is created based on the imaging information acquired by the imaging device, and the lane markings (white line lane marks) are detected on the overhead view. Then, on the overhead coordinates, the lane markings on the road are estimated by function approximation. The lane markings on the road consist of a combination of straight lines, circular arcs, and clothoid curves, and the clothoid curves are called relaxation curves, and are used in the sections that transition from straight lines to circular arcs. Further, the clothoid curve can be approximated by a cubic expression, and the straight line and the arc can also be approximated by a cubic expression with an error that causes no practical problem.

As a concrete means thereof, in the invention described in claim 1, the dividing line edge on the image coordinates is extracted from the image information obtained by the image pickup means (for example, CCD camera) (the dividing line edge extracting means). . In addition, the lane marking edges on the extracted image coordinates are converted into overhead coordinates or three-dimensional coordinates, and the transformed lane marking edges are approximated by an quadratic or higher nth-order curve (nth-order approximation means). Further, the marking line is estimated based on the n-th order curve (marking line estimating means).

According to the first aspect of the present invention, by applying the quadratic or higher order nth-order approximation formula to the lane marking information converted into the overhead coordinates, the error of error is greater than the approximation in the image coordinate system. Fewer predictive curves can be detected. That is, a highly accurate approximate curve can be obtained, and thus a distant lane line that does not appear in the image information, that is, a distant lane line without a lane line edge from the image information can be predicted from the approximate curve.

On the other hand, according to the second aspect of the present invention, the lane marking edge on the image coordinates is extracted from the image information obtained by the image pickup means (for example, CCD camera) (lane marking edge extraction means). Further, the lane marking edges on the extracted image coordinates are converted into overhead coordinates or three-dimensional coordinates, and the converted lane marking edges are function-approximated (function approximating means). Further, the approximate function is inversely transformed into the original image coordinates by using the constant at the time of the coordinate transformation, and the lane marking is estimated based on the result of the inverse transformation (lane marking estimating means).

In short, when the image information from the CCD camera is converted to the bird's-eye view coordinate system or the three-dimensional coordinate system, the angle between the optical axis of the CCD camera and the road surface is an important factor. When the vehicle is running, the angle formed between the optical axis and the road surface changes due to the influence of the unevenness of the road surface. In such a case, the effect is generated at the time of conversion into the overhead coordinate system or the three-dimensional coordinate system, and the accuracy of the function curve is deteriorated. On the other hand, according to the configuration of claim 2, after the function approximation, a constant (for example, C
By returning the bird's-eye view coordinates or the three-dimensional coordinates to the original image coordinates by using (a constant related to the angle of the optical axis of a CD camera or the like), it is possible to suppress an error caused by the coordinate conversion caused by the angle change. it can. As a result, it is possible to eliminate the above-mentioned problems caused by the unevenness of the road surface, etc., and detect a prediction curve having a smaller error than performing approximation by image coordinates, and further improve the detection accuracy of lane markings.

[0013]

BEST MODE FOR CARRYING OUT THE INVENTION

(First Embodiment) A first embodiment in which the present invention is embodied in an obstacle detecting device will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the structure of an obstacle detection device according to this embodiment. In FIG. 1, an image pickup device 10 as an image pickup means has a pair of left and right CCD cameras (not shown) for picking up an image of the front of the vehicle.
The image signal obtained in step 1 is input to the alarm device main body 20. FIG. 9 shows an image obtained by the image pickup apparatus 10. In the image, lane markings (referred to as white line lane marks in the present embodiment) 81, 82 and a vehicle 83 traveling in front of the host vehicle are shown. 84, telephone poles 85, etc. are displayed.

Further, the alarm device main body 20 has a speed signal from the vehicle speed sensor 30, a yaw rate / pitch rate detection signal from the yaw rate sensor / pitch rate sensor 40,
A brake signal is input from the brake switch 50. The alarm device main body 20 performs image processing on the basis of the image signal to detect a three-dimensional object and a white line lane mark, and collates with information of a speed signal, a yaw rate / pitch rate signal, and a brake signal to obstruct the own vehicle. Determine the degree of danger to an object. Then, the alarm device main body 20 outputs an alarm from the speaker 60 according to the degree of danger to the obstacle or the like, and outputs a distance to the obstacle or the like from the display device 70. In this embodiment, the alarm device main body 20 constitutes the lane marking edge extraction means, the nth order approximation means, and the lane marking estimation means.

FIG. 2 is a functional block diagram showing the configuration of the alarm device main body 20 for each action, and the obstacle determination processing in the present embodiment will be described below with reference to FIG. In FIG. 2, the alarm device body 20 is roughly divided into an image input unit 21, a lane mark detection unit 22, a three-dimensional object detection unit 23, and a position determination unit 24. The image input unit 21 receives the image signal from the imaging device 10, and the image signal is transmitted to the lane mark detection unit 22 and the three-dimensional object detection unit 23.
In such a case, the lane mark detection unit 22 is transmitted with the image signal of one of the left and right CCD cameras, and the three-dimensional object detection unit 23 is transmitted with the image signals of both the left and right CCD cameras.

The lane mark detecting section 22 detects a white lane mark according to the procedure of steps S111 to S116 below. That is, in step S111, a point at which the relative brightness difference between predetermined pixels in the horizontal direction in the image exceeds a predetermined threshold is extracted as an edge point. By this edge extraction processing, edge points as shown in FIG. 10 are extracted. In FIG. 10, white points are edge points (up edges) where the difference is positive, and black points are edge points (down edge) where the difference is negative. At this time,
Focusing on the edge that crosses the white line lane mark, it can be seen that the lane mark portion is a pair of an up edge located on the left side and a down edge located on the right side.

Next, in step S112, pairing is performed to connect the pair of the up edge and the down edge, and only the edge points that are candidates for the white line lane mark are extracted as shown in FIG. The procedure of such pairing will be outlined below. That is, FIG. 3 shows that the white line lane marks extending in parallel converge to one point (vanishing point) on the image, and in FIG. 3, the white line lane marks on the image at a position separated by Y downward from the vanishing point. The width is W. In this case, the mounting height h of the CCD camera, the horizontal angle of view θh and the number of pixels nh of the camera, the vertical angle of view θv and the number of pixels nv, and the actual width d of the white lane mark on the road are known. Then, the width W of the white line lane mark on the image can be approximately calculated by the following equation (1).

[0019]

[Equation 1] Therefore, in the above formula (1), the width W of the white line lane mark on the image is expressed as a function of the vertical scanning position Y based on the vanishing point. Then, the edge points of the lane mark candidates can be extracted by selecting the up edge and the down edge in which the interval on the image during horizontal scanning is within the width W obtained by the above equation (1). In this case, for example, an intermediate position between the horizontal position of the up edge and the horizontal position of the down edge is set as the edge point candidate (FIG. 11). At this time,
Edge points corresponding to the vehicles 83, 84 and telephone poles 85 in FIG. 9 are deleted from the lane mark candidates.

Next, in step S113, Hough transform processing is performed on the edge points of the lane mark candidates, and a straight line which is a basis for obtaining the edge points on the actual white line lane mark is detected. FIG. 12 shows straight lines L1 and L2 extracted by the Hough transform. In the following step S114, based on the straight lines L1 and L2 extracted by the Hough transform,
The edge points of the white line lane marks are extracted from the lane mark candidates of. The result is shown in FIG.

In step S115, the edge points obtained as described above are taken from the image coordinate system to the bird's-eye view coordinate system (XZ coordinate).
Convert to FIG. 15 is a diagram showing a result of having the edge points of the white line lane marks scattered on the overhead view. The procedure for converting from the image coordinate system to the bird's-eye view coordinate system is shown below.

FIG. 4 is a side view of the imaging device 10 and the road surface, FIG. 5 is a top view of FIG. 4 (overhead view), and FIG. 6 is an image taken by the imaging device 10. . Here, the coordinate system of FIG. 4 is (Y, Z), and the coordinate system of FIG. 5 is (X,
Z). Further, although the road surface is treated as a plane in the overhead coordinate system of FIG. 5, it can be expanded to three-dimensional coordinates (X, Y, Z) if the coordinate system of FIG. 4 viewed from the side is also taken into consideration.

When an arbitrary point p1 is taken on the road surface, its position on the image is (x1, y1). In this case, the relational expression of the disappearance line y0 in FIG. 6 and Zm in FIG. 4 can be expressed by the following equations (2) and (3).

[0024]

[Equation 2]

[0025]

(Equation 3) Here, ym, θv, θv0 in the equations (2) and (3),
If h is a known value, y0 and Zm are calculated as predetermined values (fixed values).

Further, Z1 in FIG. 4 can be expressed by the following equation (4) including y0 and Zm.

[0027]

(Equation 4) Further, θh1 in FIG. 5 is expressed by the following equation (5),

[0028]

(Equation 5) X1 in FIG. 5 is expressed by the following equation (6).

[0029]

(Equation 6) By using the above equations (2) to (6), (X1, Z1) in the overhead coordinate system can be determined.

In step S116, the edge points converted into the bird's-eye view coordinate system are approximated to a cubic curve using the least square method. Then, by outputting the white line lane marks as a function of a cubic polynomial, F1 (Z) and F2 in FIG.
The curve function of (Z) is obtained. At this time, F1 (Z) is a function of the white line lane mark on the right side, and F2 (Z) is a function of the white line lane mark on the left side, and the cubic approximation curve by these is the linear part of the straight line portion obtained by the Hough transformation. It will be predicted as an extension.

On the other hand, in the three-dimensional object detection unit 23, when the image signals from the pair of left and right CCD cameras are input from the image input unit 21, three-dimensional objects such as other traveling vehicles are known using the known stereoscopic method in step S121. The position in the image and the actual distance are output. Here, the distance detection procedure by the stereoscopic method will be briefly described with reference to FIG. 7. In FIG.
The imaging device 10 includes an object M for forming two viewpoints.
Two lenses 11 and 12 are arranged facing each other.
Behind the two lenses 11, 12, their optical axes S1, S2
CCDs 13 and 14 that match each other are arranged. Here, “De” in the figure is the distance between the lenses 11 and 12 and the object M, “f” is the focal length of the lenses 11 and 12, “xa”,
"Xb" is the distance between the optical axis S1, S2 and the point where the object M passes through the lenses 11, 12 to the CCDs 13, 14 and "P" is the pitch between the optical axes S1, S2. In such a case, the distance De is

[0032]

(Equation 7) Can be obtained by That is, since the focal length f and the pitch P are known, the distance De to the object M can be known by obtaining (xa + xb). Here, (xa + xb) corresponds to the parallax between the left and right images, and the luminance values of the objects in the left and right images are compared while being slightly shifted (for example, by one pixel), and can be obtained as the best matching shift amount. . As a general method for obtaining the parallax, correlation calculation is used, and the shift amount i of the point where the correlation value V (i) obtained by the following equation (8) is the minimum is the parallax.

[0033]

(Equation 8) However, in Expression (8), “an” and “bn” are a sequence of luminance (a sequence) when a unit area (for example, 8 pixels × 8 pixels) of the image is a one-dimensional image, and “n” is a horizontal direction of the image. The pixel number in the direction, “W”, is the size of the unit area (8 in this case). It should be noted that the numerical sequence showing the arrangement of the luminance is "am, n", "b
It is also possible to obtain the correlation value V (i) from the two-dimensional image in the unit area, and obtain the parallax from the shift amount i at which the correlation value V (i) becomes the minimum value.

Further, in the following step S122, the position of the three-dimensional object on the image coordinates is converted into the overhead coordinate system. At this time, if there is a three-dimensional object (for example, another vehicle) at an arbitrary point p1 shown in FIGS. 4 to 6, Z1 on the bird's-eye view conversion is directly obtained as the distance De of the equation (8), and this Z1 is calculated as described above. X1 can be obtained by substituting the equations (5) and (6). According to this processing, the vehicles 83 and 84 shown in FIG. 8 are detected at the positions P1 and P2 in the overhead view shown in FIG.

Finally, the position determining unit 24 determines the relationship between the three-dimensional object and the lane mark function as shown in FIG. 16, and determines the positional relationship between the three-dimensional object and the white line lane mark. in this case,
As described above, the white line lane mark is represented by the functions F1 (Z) and F2 (Z) on the overhead coordinate. Further, the three-dimensional object has positions (Xr, Z) and (Xl, Z) at the left and right ends of the object.
Then, the position determination unit 24 executes the routine of FIG. 8 as an example of the position determination processing, and generates alarm information according to the necessity at each time.

That is, in FIG. 8, step S201.
When the routine is started in step S202, it is determined in step S202 whether there is a three-dimensional object inside the white lane mark on the right side, and in the next step S203, whether there is a three-dimensional object inside the white line lane mark on the left side. To determine. More specifically, the determination process is performed according to the relationship of the following expressions (9) and (10). Here, "Xra" indicates the right end X direction position of the three-dimensional object, "Xla" indicates the left end X direction position of the three-dimensional object, and "Za" indicates the Z direction position.

[0037]

[Equation 9]

[0038]

(Equation 10) Further, in step S204, the relative speed is obtained from the speed signal from the vehicle speed sensor 30 and the change in the distance to the three-dimensional object, and it is determined from the relationship between the relative speed, the distance, and the own vehicle speed whether the current traveling state is dangerous. To do. In step S205,
Based on the presence or absence of the brake signal from the brake switch 50, it is determined whether or not the driver is trying to avoid the danger. If all the determinations in steps S202 to S205 are affirmative, a three-dimensional object (obstacle) in step S206.
A sound is generated to warn the driver that there is a possibility of collision with, and then the process ends in step S207.
If any of the steps S202 to S205 is negatively determined, the process is immediately ended in the step S207.

According to this embodiment described in detail above, the following effects can be obtained. (A) First, for a region that is relatively close to the host vehicle, linear approximation is performed by Hough transform. In this case, a large number of edge points can be obtained for the white line lane mark in the short-distance region, so that accurate straight line approximation is possible.

(B) On the other hand, in a relatively distant area, only a small number of edge points can be obtained for the white lane mark, but the image coordinates are converted into the overhead coordinates and the lane markings converted into the overhead coordinates. By applying a cubic approximation formula to the information, it is possible to detect a prediction curve with less error than approximation in the image coordinate system. That is, the white line lane mark on the road is made up of a combination of straight lines, arcs, and clothoid curves, and the white line lane mark on the road including the clothoid curve component can be accurately detected by performing the cubic approximation. In this case, since the cubic curve portion obtained on the overhead coordinates is extended as an extension line of the straight line portion of the white line lane mark obtained by the Hough transform, the white line lane mark can be predicted over a wide area. As a result, it is possible to accurately estimate a distant lane line that does not appear in image information, which has been a problem in the past, that is, a distant lane line that has no lane line edge from the image information.

(C) By converting the lane mark information and the three-dimensional object information into an overhead coordinate system and comparing and determining their relationship on the coordinates, it is possible to enhance the accuracy of obstacle detection and to ensure safety. Driving can be realized.

(Second Embodiment) Next, a second embodiment will be described. However, in the configuration of the present embodiment, the description of the same components as those in the above-described first embodiment will be omitted. Then, the differences from the first embodiment will be mainly described below.

That is, in the first embodiment, the image information obtained by the image pickup device 10 is converted from the image coordinate system to the bird's-eye view coordinate system, and the white line lane marks are cubically approximated by the bird's-eye view coordinates. In the embodiment, the image information is converted from the image coordinate system (two-dimensional coordinate system) to the three-dimensional coordinate system, and the white line lane marks are cubically approximated by the three-dimensional coordinates. The details will be described below.

The lane mark detection procedure in this embodiment is the same as the other processing except that the contents of S115 in FIG. 2 are changed. That is, the following process is performed in S115 of FIG. A predetermined frame α1 is set for the edge points of the white line lane marks extracted as shown in FIG. 17 (a diagram corresponding to FIG. 13), and an enlarged view of the same α1 is shown in FIG.
Shown in In FIG. 18, a window α2 of width w × height h is set for the point p1. The window α2 is set so that the horizontal width of the white line lane mark is within the window, and the width w is reduced as it goes to the top of the screen. When the width w of the window α2 is expressed by an equation, the equation (11)
Becomes

[0045]

[Equation 11] However, in the equation (11), “wmax” is a value that allows the width of the lane mark at the bottom of the screen to be sufficiently large, and “wmin” is the minimum value within the range where the correlation calculation can be performed accurately. In the present embodiment, wmax = 50, wmin = 8 and h = 8.

A distance Z1 is obtained by using a correlation calculation for this window α2. In this case, when performing the correlation calculation of the window α2, the lane mark detection unit 2
2 requires a pair of left and right image information from the imaging device 10. X1 is derived from the above equation (6), and Y1 is derived from the following equations (12) and (13).

[0047]

(Equation 12)

[0048]

(Equation 13) As described above, the image information regarding the white line lane mark is converted into the three-dimensional coordinates. Then, similar to the first embodiment, the white line lane mark is detected by the cubic curve approximation by the least square method.

On the other hand, also in the three-dimensional object detection unit 23 in FIG. 2, conversion processing is performed in the same manner as the three-dimensional coordinate conversion of the lane mark. That is, coordinate conversion is performed using the above equations (12) and (13). Note that equations (12) and (13)
In the above, the angle θv0 formed by the optical axis of the image pickup device 10 (CCD camera) and the road surface is omitted as a minute one, but in this case, both the white line lane mark and the three-dimensional object are three-dimensionally coordinated in the same procedure. Since the conversion processing to the has been performed, it does not affect the accuracy of the obstacle judgment.

As described above, according to the second embodiment, the first
Similar to the embodiment described above, the approximated curve of the white line lane mark can be obtained with high accuracy, and consequently the distant white line lane mark that does not appear in the image information can be predicted.

(Third Embodiment) Next, a third embodiment of the invention as defined in claim 2 will be described. However, in the configuration of the present embodiment, the description of the same components as those in the above-described first embodiment will be omitted. Then, the differences from the first embodiment will be mainly described below. In the present embodiment, the alarm device main body 20 constitutes the lane marking edge extraction means, the function approximation means, and the lane marking estimation means.

FIG. 19 is a functional block diagram showing the structure of the alarm device main body 20 according to this embodiment for each operation.
In FIG. 19, the image signal input to the image input unit 21 is transmitted to the lane mark detection unit 22. In the lane mark detector 22, as in the case of the first embodiment shown in FIG. 2 (steps S111 to S116), step S311 is performed.
In step S316, the edge points of the lane marks on the image coordinates are converted to the overhead coordinates using the image signal, and a cubic curve approximation is performed from the edge points in the overhead coordinate system to detect the white line lane marks. Further, in the present embodiment, step S317 is newly added.
In 317, the lane mark detection result in the overhead coordinate system is reconverted into the image coordinate system.

At this time, it is difficult to handle the lane mark function on the bird's-eye view generated in step S316 by inversely converting it in the form of the function and converting it to the image coordinate system. Therefore, first, the cubic function F1 (Z) of the white line lane mark,
A value obtained by increasing the distance measurement resolution Zan from Z = 0 to the maximum measurement distance Z = Zmax is substituted into F2 (Z) to create a lane mark table in the overhead coordinate system. next,
Image coordinate conversion is performed for each point on this table,
Create a lane mark table in the image coordinate system. Here, the inverse transform formula is a reverse transform of the above formulas (2) to (6) and becomes the following formulas (14) and (15).

[0054]

[Equation 14]

[0055]

(Equation 15) (X1, y1) obtained by the equations (14) and (15)
Indicates the position of the point p1 on the image coordinates in FIG. 21, for example. At this time, the curve function of the white line lane mark is f1
(Y), f2 (y).

On the other hand, in the three-dimensional object detection unit 23, the first
In the same way as in the embodiment described above, the stereoscopic object detection process by the stereoscopic method is performed in step S321, but the process of converting the detection result into the overhead coordinates is not performed (the process corresponding to step S122 in FIG. 2 is omitted. Exist).

The position determining unit 24 obtains the relationship between the three-dimensional object and the white line lane mark in the image coordinate system shown in FIG. 21, and determines the positional relationship between the three-dimensional object and the white line lane mark. That is,
By the image coordinate conversion (inverse conversion) in step S317 of the lane mark detection unit 22, the white line lane marks are a table of a set of points in the image coordinate system, which is a function f1.
(Y) and f2 (y). In addition, by extracting the three-dimensional object by the three-dimensional object detection unit 23, the x-coordinates xl, xr and the y-coordinate yl of the left and right ends ol, or of the lower part of the three-dimensional object in the image coordinate system.
yr and distance Z are calculated.

Therefore, the position determination unit 24 determines the positional relationship between the three-dimensional object and the lane mark by the position determination processing which is a part of the routine of FIG. 8 modified. FIG. 20 corresponds to FIG.
Steps S202 and S203 of Step S40
2. It has been changed to S403, and step S40
At 2, it is judged whether or not there is a three-dimensional object inside the right white line lane mark, that is, whether or not f1 (yr) ≧ xr is satisfied. At step S403, a three-dimensional object inside the left white line lane mark is determined. Or not, that is, f2 (yl) ≤xl
Is determined.

Note that, in FIG. 22, the lane mark function f1 (y), finally on the image coordinates according to FIGS.
It is the figure which arranged f2 (y) and three-dimensional thing (other vehicles) P3 and P4.

In short, at the time of conversion to the overhead coordinate system, a constant (C for the road surface) indicating the relationship between the image pickup device 10 and the road surface is obtained.
The calculation processing is performed based on the angle θv0 of the optical axis shift of the CD camera, θv1, etc.). In this case, when the device is mounted on a moving body such as a vehicle, for example, θv0 and the like fluctuate due to the unevenness of the traveling road surface, which affects the lane mark position Z1 on the overhead coordinates and causes an error in detecting the white line lane mark. Could occur. However, in the present embodiment, since the inverse transformation is performed using the constant θv0 used for the overhead coordinate transformation, it is possible to suppress the occurrence of an error due to the variation of θv0 as described above. As a result, the white line lane mark detection accuracy can be further improved.

Further, when determining the positional relationship between the white line lane mark and the three-dimensional object, each positional deviation in the Z direction is eliminated. That is, according to the stereoscopic method, the distance of the three-dimensional object can be directly detected, and the above-mentioned detection error due to the variation of θv0 does not occur. Further, as described above, the error due to the variation of θv0 is eliminated even when the white line lane mark is detected. As a result, the positional relationship between the white line lane mark and the three-dimensional object can be accurately grasped, and thus it is possible to realize highly accurate determination processing even when determining an obstacle.

The present invention can be embodied in the following modes (1) to (5) in addition to the above-described embodiment. (1) In each of the above embodiments, a straight line portion on the road,
When approximating arcs and clothoid curves 3
Although the quadratic curve approximation is performed, this may be changed. In other words, if you want to further reduce the error, the approximation curve
Higher-order polynomials of degree higher than or equal to may be applied. Although the quadratic curve approximation has a larger error than the cubic curve approximation, it has an advantage that it can be adapted depending on the specifications of the device and the calculation amount can be reduced.

(2) In the above embodiment, the white line lane mark is extracted by pairing the up edge and the down edge on the image, but the extraction method is not limited to this. For example, the edge extraction process may be simplified by simply extracting the position at which the brightness information relatively changes as the edge. Also, Z on the bird's-eye view
To obtain 1, the correlation calculation of the three-dimensional object method using the left and right images may be used.

(3) In the first embodiment (the invention of claim 1), the white line lane mark is required to be approximated by an nth-order curve of quadratic or higher, but the second embodiment (claim) In the invention of item 2, the white lane mark is a linear function (linear equation).
May be approximated, and in such a case, a novel effect which has never been obtained can be obtained. That is, the invention according to claim 2 is CC
The main purpose is to suppress the error due to the angle variation between the optical axis of the D camera and the road surface, and even if the white line lane mark is estimated by the first-order approximation, such purpose can be achieved.

(4) In the third embodiment, as an embodiment of the invention described in claim 2, the conversion processing is performed on the image information of the white line lane mark in the order of image coordinates → overhead view coordinates → image coordinates. However, the conversion process may be performed in the order of image coordinates → three-dimensional coordinates → image coordinates. In this case, the accuracy of the position determination between the white line lane marks and the three-dimensional object is almost the same without performing the inverse conversion from the three-dimensional coordinates to the image coordinates, but if only the white line lane mark detection processing is considered, the three-dimensional coordinates are considered. It can be said that the detection accuracy is improved by performing the inverse conversion from the image coordinates to the image coordinates.

(5) In the above embodiment, the stereoscopic method using a stereo image is used as a method for detecting a three-dimensional object (other vehicle), but this may be changed. For example, a three-dimensional object may be detected using radar light.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing an outline of an obstacle detection device according to an embodiment of the invention.

FIG. 2 is a functional block diagram showing, for each action, the configuration of the alarm device main body according to the first embodiment.

FIG. 3 is a schematic diagram showing a road surface image.

FIG. 4 is a side view of the relationship between the imaging device and the road surface.

5 is a top view of FIG. 4 (overhead view).

FIG. 6 is a diagram showing road surface information on an image.

FIG. 7 is a diagram for explaining a procedure of distance measurement by a stereoscopic method.

FIG. 8 is a flowchart showing a position determination routine.

FIG. 9 is a diagram showing an image captured from an image pickup apparatus.

FIG. 10 is a diagram showing a result of performing edge extraction processing on an image.

FIG. 11 is a diagram showing a result of extracting lane mark candidates by pairing.

FIG. 12 is a diagram showing a result of extracting straight lines by Hough transform.

FIG. 13 is a diagram showing a result of extracting lane mark information from a Hough-transformed straight line.

FIG. 14 is a diagram showing a relationship between lane mark information and an approximate function line.

FIG. 15 is a diagram showing lane mark information on an overhead view.

FIG. 16 is a diagram showing a positional relationship between a white line lane mark and a three-dimensional object on the overhead view.

FIG. 17 is a diagram showing edge points of a white line lane mark.

FIG. 18 is an enlarged view showing α1 of FIG.

FIG. 19 is a functional block diagram showing, for each action, the configuration of the alarm device main body according to the third embodiment.

FIG. 20 is a flowchart showing a part of a position determination routine according to the third embodiment.

FIG. 21 is a diagram showing road surface information on an image.

FIG. 22 is a diagram showing a positional relationship between a white line lane mark and a three-dimensional object on the image.

[Explanation of symbols]

10 ... Image pickup device as image pickup means, 20 ... Marking line edge extraction means, nth order approximation means, marking line estimation means, alarm device main body as function approximation means.

Front page continuation (51) Int.Cl. ⁶ Identification code Office reference number FI Technical display location G08G 1/04 G08G 1/04 D 1/16 1/16 C H04N 7/18 H04N 7/18 J

Claims (3)

[Claims] 1. A lane marking detection device for detecting a lane marking on a road based on image information obtained by using an image pickup means, wherein a lane marking edge on image coordinates is obtained from the image information obtained by the image pickup means. Demarcation line edge extraction means for extracting the demarcation line edge extraction means, and converting the demarcation line edges on the image coordinates extracted by the demarcation line edge extraction means into bird's-eye view coordinates or three-dimensional coordinates, and the converted demarcation line edges The lane marking detection device, comprising: an nth approximation means for approximating the nth curve; and a lane marking estimating means for estimating the lane marking based on the nth curve obtained by the nth approximation means. 2. A lane marking detection device for detecting a lane marking on a road based on image information obtained by using an image pickup means, wherein a lane marking edge on image coordinates is obtained from the image information obtained by the image pickup means. Demarcation line edge extraction means for extracting the demarcation line edge extraction means and the demarcation line edge on the image coordinates extracted by the demarcation line edge extraction means are converted to overhead coordinates or three-dimensional coordinates, and the converted demarcation line edge is approximated by a function. A function approximating means, and a lane marking estimating means for inversely transforming the approximate function by the function approximating means into the original image coordinates by using the constant at the time of the coordinate transformation and estimating the lane markings based on the result of the inverse transformation. A lane marking detection device comprising. 3. The marking line detecting apparatus according to claim 2, wherein the approximating means approximates the marking line edge by an nth-order curve of quadratic or higher degree.