JP2003168123A - Lane boundary judgment device - Google Patents

Abstract

(57) [Summary] [Problem] To stably and continuously output a reliable lane boundary position even when a lane marking on a road surface is a complicated marking (composite line). A complex line determining means is provided for determining a triplet that satisfies the condition shown in the following equation (2) with respect to the marking pattern of FIG. However, di shown below (1 ≦
i ≦ 7) is an appropriate constant. "Formula (2): | w1-w
3 | ≦ d2, w1 ≧ w2, d2 ≦ w2 ≦ d5, d2 ≦ L
≦ d6 ”. For example, the width w2 of the lane boundary line in FIG.
Is 10 cm to 20 cm, d2 = 10
cm, d5 = 20 cm. As a result of these decisions,
If Expression (2) is satisfied, the block marking corresponding to the shaded area in FIG. 1A is removed as noise. In other words, this process deletes or invalidates the data of the corresponding white line marking determined to be a block marking (ie, noise) in the search information table.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a judging device for judging the boundary position of a lane marking drawn on a road surface in real time based on an input image from a vehicle-mounted camera. For example, an automatic lane keeping operation of a vehicle. It can be used as a device useful for a vehicle driving support system for executing the above.

[0002]

2. Description of the Related Art FIG. 21 is a composite line (lane marking) within the field of view of a vehicle-mounted camera of a vehicle traveling on a lane such as an expressway.
It is a top view which illustrates the marking pattern of. In addition, FIG.
FIG. 4 illustrates a marking pattern of a compound line (lane marking) within the field of view of a vehicle-mounted camera of a vehicle traveling on an overtaking lane. The vehicle-mounted camera in the figure is provided in the vicinity of the rearview mirror in the vehicle, and the shaded portion indicates a portion that is out of the field of view of the camera.

Currently, a driving support system for executing lane keeping operation (lane keeping operation) by automatic control using an in-vehicle camera is becoming widespread, and a vehicle equipped with such a driving support system is commercially available to the general public. ing. In these driving support systems, the position of the lane marking (lane boundary position) that defines the lane is detected by analyzing the image of the road ahead input from the vehicle-mounted camera with a computer.

[0004]

However, the lane markings on the road surface may be complex lines such as those shown in FIGS. 21 and 22, for example. Lane marking)), it is difficult or uncertain to determine the boundary position of the lane (lane) in the conventional driving support system capable of performing lane keeping operation (lane keeping operation). There has been a problem that the lane keeping operation must be interrupted, or the calculated boundary position of the lane vibrates laterally, so that a stable output result cannot be obtained.

The present invention has been made to solve the above problems, and its object is to ensure a reliable lane boundary position even when the lane marking on the road surface is, for example, the complex line as described above. Is to realize a lane boundary determination device capable of stably outputting continuously.

[0006]

Means for Solving the Problems, and Functions and Effects of the Invention In order to solve the above problems, the following means are effective. That is, the first means of the present invention is a determination device for determining the boundary position of the lane (lane) defined by the lane markings drawn on the road surface in real time during traveling based on the image input from the camera. In, the vertical direction is the direction of the lane markings or the traveling direction of the host vehicle, and the direction orthogonal to this vertical direction is the search direction for the lane markings, and the lane markings are based on the search information obtained by the search in this search direction. A composite line determination means for determining whether or not a composite line composed of a plurality of columns or a plurality of markings having a vertical context, and a boundary position for estimating a boundary position based on the composite line determination result Estimating means.

FIG. 1 is a plan view exemplifying a marking pattern of a composite line obtained when a lane marking is searched along the above-mentioned search direction. For example, FIG.
This corresponds to the search result when the search is performed in the horizontal direction at the search position indicated by 2 (a). In addition, FIG.
The marking patterns of (c) and (d) also correspond to the same symbols. Also, the shaded portion of each marking pattern in the figure indicates the block marking of the composite line. However, "composition line block marking" here
Is a rectangular marking drawn from the side in the vicinity of the lane (lane) boundary marking line to draw the driver's attention such as deviation from the lane, and is usually a vertical marking pitch. However, it is shorter than the boundary marking line.

For example, when such a marking is detected on the road surface image, it is determined by the above-mentioned composite line judging means whether or not the marking pattern is within the standard range of the general or normal composite line. Can be determined. It is also possible to determine whether or not the composite line is continuous in front of the vehicle by accumulating (statistically operating) such determination results with time or spatially in the image plane. Will also be possible.

Therefore, by using these judgment results, it is possible to regard a part of the composite line as noise and ignore it. For example, the shaded portion of the marking pattern shown in (a) to (d) of FIG. 1 can be determined to be a block marking of a composite line, and thus this marking is unnecessary marking for detecting a lane boundary. It can be estimated that So, for example, by ignoring this block label,
It is possible to detect the correct boundary position of the running lane.
For example, with the above-described operation, even when the lane marking on the road surface is the above-described complex line, the accurate lane boundary position can be detected stably and continuously.

The second means of the present invention is the above first means.
In the composite line determination means of the means, it is determined whether or not the lane marking is a composite line, based on a plurality of pieces of search information at a plurality of search positions on the input image whose search directions are substantially parallel to each other. That is. The judgment result such as whether the marking pattern is within the standard range of general or normal composite line is
The reliability is improved by collecting a large number of the samples. According to the second means, the determination result for one marking pattern can be spatially accumulated (statistical operation) in the image plane. Therefore, it becomes more reliable or easier to estimate whether or not the composite line is continuous in front of the vehicle.

The third means is the above-mentioned first or second.
In the composite line determination means of the means, it is determined whether or not the lane marking is composed of a composite line, based on the relationship between the latest search information and the past search information. For example, the judgment result as to whether or not the marking pattern is within the standard range of the general or normal composite line can be obtained by collecting a large number of the samples with the lapse of time. It is possible to more reliably or easily estimate whether or not the line is continuous in front of the vehicle.

Further, 1 obtained by the above second means
The reliability of recognition of the lane marking can be improved by collating the determination result (boundary line position) before the unit time (1 control cycle) with the determination result (boundary line position) based on the current input image. It is possible. According to such a collating unit, even when a lot of noise is reflected in the road surface image, the boundary position of the currently running lane can be definitely determined.

A fourth means is the above-mentioned first to third.
In the composite line determination means of any one of the above, the lane markings are sandwiched from both sides by at least one of the number of markings, the width of the markings, or the gap width between the markings included in the search information. It is to judge whether or not it is composed of a composite line composed of one marking line. According to this means, the composite line having the marking pattern of FIG. 1 (a), FIG. 21 (a), FIG. 22 (a), etc. is determined as a triple line (block marking + lane boundary line + block marking). Is possible.
As the search information at this time, for example, w1 (width of the marking), w2, w3, L1 (gap width between the markings), L2, L, or the figure (marking pattern) itself shown in FIG. The position etc. can be considered.

The fifth means is the above-mentioned first to fourth.
In the composite line determination means of any one of the means, the lane marking is one of which the block marking approaches one side from at least one of the number of markings, the width of the markings, or the gap width between the markings included in the search information. It is to judge whether or not it is composed of a composite line composed of the marking lines of. According to this means, FIG.
(B), (c), FIG. 21 (b), (c), FIG.
It is possible to determine a composite line having marking patterns such as (b) and (c) as a double line (lane boundary line + block marking / block marking + lane boundary line). As the search information at this time, for example, w1, w2, and L shown in FIG.
1 or the position of the figure (marking pattern) itself can be considered.

The sixth means is the above-mentioned first to fifth means.
The lane markings are arranged in parallel in the vertical direction based on at least one of the number of markings included in the search information, the width of the markings, or the gap width between the markings. Alternatively, it is to judge whether or not the line consists of a composite line composed of two columns of block markings. According to this means, the lane boundary line located at the center of the above-mentioned triple line is periodically interrupted on the composite line having the marking pattern of FIG. 1 (d), FIG. 21 (d), FIG. 22 (d), etc. It is possible to determine that the marking pattern of the part where the mark is present (triple line with a missing dot: block marking + block marking). As the search information at this time, for example, w1, w2, L shown in FIG.
Alternatively, the position of the figure (marking pattern) itself may be considered.

Further, a seventh means is the above first to sixth means.
In any one of the means, the line type mode determination unit for determining the "line type mode" of the lane marking of the lane currently being driven based on the past or present search information is provided. Is. For example, when a state where the above-mentioned triple line is spatially continuous to the front on the left side of the running lane is detected, the left lane marking is defined as "in the left triple line mode".

If such a line type mode determination is always performed, for example, even when the above-mentioned triple line is detected on the left side when "in the left single line mode", those marking patterns are limited to that time. Can be ignored. That is, when a contradiction (mismatch) occurs between the line type mode and the type of the composite line, it is possible to handle those marking patterns as noise. This makes it possible to realize a robust boundary position determination against noise.

The eighth means changes the line type mode until the line type mode determination section of the seventh means described above continuously satisfies the "predetermined mode changing condition" for a certain period of time. A mode change holding means for holding the processing is provided. By introducing a means to determine the continuity of the linetype mode over time, the linetype mode can be changed more reliably even in the situation where the mode determination tends to be unstable due to the influence of noise. Can be carried out.
That is, this makes it possible to realize more robust boundary position determination against noise.

The ninth means is the seventh or eighth aspect described above.
The boundary position estimating means of the means is provided with a noise removing unit that determines and removes unnecessary portions such as block markings included in the lane markings from the search information based on the line type mode determined by the line type mode determining unit. That is. For example, with such means, it is possible or easy to estimate (remove) the search information of the portion that should be removed as noise based on the line type mode.

In a tenth means, the width of the lane marking searched for as a single line in the boundary position estimating means of any one of the seventh to ninth means is a predetermined value D.
If S is exceeded, the boundary position is estimated based on the line type mode determined by the line type mode determination unit. As the above value DS, for example, an upper limit value of a standard appropriate range of the width of a lane boundary line composed of a general single line is suitable.

According to the tenth means, when the angle of the optical axis of the camera is shallow with respect to the position of the lane marking on the road surface,
The composite line can be more accurately determined when the resolution of the captured image is low, when there is a lot of noise on the image, or when the lane marking is not drawn on the road surface in a desired format. The operation of this means will be described in detail in the embodiment (second embodiment) of the present invention described later.

An eleventh means is the boundary position estimating means of any one of the above first to tenth means,
This is to estimate the position of one mark having the shortest width detected as a part of the composite line as the boundary position. For example,
In the figure of FIG. 1B (marking pattern of left double line), the relation of w1 ≦ w2 is required. In such a case, the position of one mark having the shortest width is estimated to be the boundary position. The means to do is effective. For example, it is possible to accurately estimate the lane boundary position by utilizing such a criterion.

A twelfth means is the boundary position estimating means of any one of the first to eleventh means,
Detected as a compound line, 2 arranged in parallel in the vertical direction
This is to estimate the middle point in the search direction of the block marking of one or two columns as the boundary position. For example, in FIG.
When the above-described void marking pattern in FIG. 21 (d), FIG. 22 (d), etc. is detected, the position of the lane boundary line is estimated, as indicated by the broken line in FIG. 1 (d). be able to.
Therefore, according to such means, information that should be ignored as noise (search information for two block markings)
Based on, the boundary position can be estimated. As a result, it is possible to augment the samples of the lane boundary position information, so that it is possible to more accurately estimate the boundary position.

Further, the thirteenth means is any one of the above first to twelfth, when the composite line determination result determines that the lane marking is composed of one or one column of block markings. The boundary position estimating means of one means estimates the boundary position based on the position of the block marking, suspends the estimation of the boundary position, or makes the boundary position indefinite.

For example, FIG. 21 (f) and FIG. 22 (f)
The positions of the dots indicated by ● (black circles) are the positions of one block marking shown in (f) of the figure, that is, 2
It can be estimated from the position of the marking pattern at the portion where the marking outside the heavy line (lane boundary line) is periodically interrupted. Therefore, from such an estimation,
It is possible to estimate the position of the point indicated by (black circle) as the position of the lane boundary line (lane boundary position).

Alternatively, FIG. 21 (f) and FIG. 22 (f)
One block marking shown in is judged as noise,
It can be ignored. In such a case, the position of the lane boundary line (lane boundary position) may be estimated from other information. That is, in the above case, the position of the lane boundary line is estimated based on other reliable information by temporarily suspending the estimation of the boundary position or making the boundary position indefinite. Thus, the boundary position at the search position can be estimated at the same time. By the means of the present invention described above, the above problems can be effectively or rationally solved.

[0027]

BEST MODE FOR CARRYING OUT THE INVENTION The present invention will be described below based on specific embodiments. However, the present invention is not limited to the examples shown below. (First Embodiment) FIG. 2 is a block diagram illustrating the configuration of a lane boundary determining apparatus 100 according to an embodiment of the present invention. Hereinafter, the outline of the configuration of the lane boundary determination apparatus 100 according to the present embodiment will be described first with reference to FIG.

The sign extracting means 1 is a road surface image analytically generated based on an input image from a vehicle-mounted camera (see FIG. 4).
A known white line extraction means is applied to (b)) to extract the marking (white line). However, the marking on the road surface may be relatively bright such as yellow. For example, these markings can be extracted by subjecting the generated road surface image to a differential process regarding luminance (FIG. 7).

The boundary marking selecting section 2 of FIG. 2 comprises a composite line judging means 110 and a boundary position estimating means 120, and the search information extracted by the marking extracting means 1 (FIG. 8).
Therefore, it has a function of removing various noises including unnecessary markings that adversely affect the determination of the boundary position of the lane. This search information is obtained by the search in this search direction (FIG. 7) with the vertical direction being the context direction of the lane markings or the traveling direction of the host vehicle, and the direction orthogonal to this vertical direction being the search direction relating to the lane markings. Is.

Based on the above search information, the composite line determination means 110 determines whether or not the lane marking is composed of a composite line composed of a plurality of columns or a plurality of markings having a vertical context. The composite line determination unit 110 includes a line type classification unit 111.
And a line type mode determination unit 112. Line type sorting section 1
Step 11 determines whether or not the marking pattern formed by a part of the search information is within the standard range of the general or normal composite line. In addition, the line type mode determination unit 112 determines the “line type mode” of the lane marking of the lane that is currently traveling, based on the above-mentioned search information of the past or the present.

The boundary position estimating means 120 estimates the boundary position based on the above composite line determination result. As a criterion for this estimation, for example, the line type of the lane markings classified by the line type classification unit 111 (triple line, double line, single line, hollow line, etc.)
Alternatively, the classification of the line type mode at each time determined by the line type mode determination unit 112 can be used. The boundary position estimating means 120 identifies (estimates) various noises including unnecessary markings that adversely affect the determination of the boundary position of the lane based on these determination criteria, and the search information (FIG. 8) described above. From the list as unnecessary information. In the present embodiment, the boundary position estimating means 120 executes a group of subroutines (noise removal n (1
≦ n ≦ 7): This is realized by FIGS. 10, 11, and 12).

The straight line detecting means 3 detects the lane boundary line extending in the direction of the lane markings according to a well-known straight line detecting method based on the above-mentioned search information from which noise has been removed. The lane selection unit 4 selects the most reliable (likely) lane boundary line set from the set of lane boundary lines detected by the straight line detection unit 3 according to a known lane selection method. Thereby, the left and right boundary positions of the lane can be determined.

FIG. 3 shows a lane boundary determining apparatus 100 (FIG. 2) using a computer system having an in-vehicle camera.
It is a flowchart which illustrates the execution procedure (200) of each means at the time of implement | achieving. Although this flowchart (execution procedure 200) has a loop, the processing for one round of this loop is periodically executed using, for example, a timer interrupt. In this flowchart, first, in step 210, the initialization of the entire lane boundary determination processing is executed. The initialization processing includes, for example, initialization processing of various flags and counters described below, which will be described later.

LF3 ... Left triplet mode flag (initial value is 0) RF3 ... Right triplet mode flag (initial value is 0) LF2 ... Left doublet mode flag (initial value is 0) RF2 ... Right double line mode flag (initial value is 0) LM2 ... Left compound line mode counter (initial value is
0) LM1 ... Left single line mode counter (The initial value is
γ) RM2 ... Right compound line mode counter (initial value is
0) RM1 ... Right single line mode counter (initial value is
γ) where γ is a single line mode return threshold value described later.

Next, at step 220, the latest latest photographed image is input from the vehicle-mounted camera. 1 of the input image
An example is illustrated in FIG. In step 230, a road surface image (plan view of the road surface) in a bird's-eye view from above in the vertical direction with respect to the road surface is analytically generated from the input image (FIG. 4A). 1 of the road surface image generated by this processing
An example is shown in FIG.

5 and 6 show the input image (FIG. 4).
It is explanatory drawing which shows the relationship between the correction image (FIG. 5 (a)) which carried out distortion correction of (a), and the said road surface image (FIG. 4 (b), FIG. 5 (b)). For example, the following numerical values of various physical quantities are included in the image analysis parameters (geometric conversion parameters such as external and internal parameters of the camera). (Example of analysis parameter) h: height of camera [m] f: focal length of camera [m] φ ₀ : depression angle of camera [rad] Note that point C in FIG. The projection range of the road surface image after conversion is shown.

FIG. 6 shows a plane Σ (∋
FIG. 6 is a perspective view illustrating the relationship between P) and the imaging surface (∋Q). FIG. 6 shows that the vehicle-mounted camera has a distance (height) h from the ground (plane Σ) and the optical axis is a vehicle on a horizontal plane. Shows the relationship between the road surface and the imaging surface when the vehicle is attached to the vehicle body with the depression angle of the optical axis being the angle φ. However, it is assumed that the above-described corrected image (FIG. 5A) is arranged on this imaging surface. At this time, the relationship (projection) between the coordinate value (x, y) of an arbitrary point Q on the imaging surface (correction image) and the coordinate value (X, Y) of the corresponding point P on the plane Σ approximating the road surface. Conversion) is given by the following equation (1).

## EQU1 ## y = f / tan (φ ₀ + tan ^-1 (Y / h)), x = fX / (h sinφ ₀ + Y cosφ ₀ ) ... (1)

That is, in step 230, this equation (1)
According to the image value of the point (X, Y) on the road surface image is determined from the image value of the point (x, y) on the corresponding imaging surface (correction image (on FIG. 5A)) , "Road surface image (X, Y)", etc.) are generated.

When there is no image pickup surface image (x, y) corresponding to the road surface image (X, Y), that portion is shown in FIG.
The rectangular image is an invalid image area such as the lower right corner and the lower left corner. By these geometric conversion processes, the image pattern on the projected imaging surface (corrected image (on FIG. 5A)) is
On the road surface (on plane Σ) when the subject is on the road surface
Is projected in the correct position.

Next, at step 240, the markings on the road surface image (X, Y) are extracted. This marking is usually based on the premise of white or the like, but since the operation of the extraction processing described below is based on the differential processing regarding luminance, yellow or the like can also be the target of the following processing. Figure 7
FIG. 4 is an explanatory diagram for explaining the operation of the sign extracting means 1 (FIG. 2), that is, the present step 240. As shown in FIG. 7, the width and position of the marking (white line) can be obtained from the peak coordinates of the differential value in the X direction with respect to the luminance.

FIG. 8 is a table structure diagram of the search information table 10 exemplifying the storage mode (data format) of the lane markings stored as the search information in this embodiment. "Lines" in this table structure diagram are road surface image data (pixels)
5 is a line (Y coordinate) on one screen of FIG.
(B) shows the increasing direction of the line number I (1 ≦ I ≦ m). Also, the "white line pair" on this table structure diagram is
It is the peak pair of FIG. 7 extracted on the road surface image, and the white line pair number J (1 ≦ J ≦ NI) is shown in FIG. 5B.
Shows the increasing direction of. The increasing direction of the white line pair number J matches the direction of the X coordinate on one screen of the road surface image data (pixels). In addition, the number of white line pairs NI (1 ≦
I ≦ m) is stored.

An algorithm for realizing the boundary marking selecting unit 2 (FIG. 2), which is the most characteristic feature of the present invention, will be exemplified below. FIG. 9 is a flowchart illustrating an execution procedure 300 for implementing step 250 (boundary marking selecting unit 2) of FIG. This step Step 250
In (execution procedure 300), noise in a broad sense including unnecessary markings is removed from the search information table 10 so that the remaining white line markings are adopted as markings representing a part of the lane boundary line. That is, adopting a part of the markings as useful boundary markings is equivalent to eliminating (removing) other unnecessary portions. This removal processing is executed by a subroutine group (noise removal n (1 ≦ n ≦ 7) described later.

Hereinafter, the procedure of the removing process for detecting (estimating) and removing noise in a broad sense from the search information table 10 will be described in detail.

In this flowchart (execution procedure 300), first of all, initial processing (step 310, step 3)
15, step 320) is executed. 0 in this initial processing
The following counters are cleared. (1) L3C (left triple line counter) The triple line marking pattern located in the left half of the screen detected from the search information table 10 obtained as a result of searching one screen of road surface image data (see FIG. 1 ( a)) The total number of cases. (2) R3C (Right triple line counter) The triple line marking pattern located in the right half of the screen detected from the search information table 10 obtained as a result of searching one screen of the road surface image data (Fig. 1 ( a)) The total number of cases. (3) I (line number)

Next, in steps 325 and 330, initialization is performed when executing the processing in units of rows. That is, in step 325, the line number I is incremented by 1. In step 330, the white line pair number J is initialized to 0. K is the number of markings to be operated in step 340, and matches the increment of J. However, the initial value of K is 1.

At step 335, the white line pair number J is increased by the increment K. In step 340, processing corresponding to the line type classification unit 111 (FIG. 2) is executed. Figure 10
An execution procedure 400 for realizing the line type classification unit 111 (step 340 in FIG. 9) is illustrated in FIG.

In step 410, the marking pattern shown in FIG. 1A is determined to be a triple line if it satisfies the condition shown in the following expression (2). However, di (0 ≦ i ≦
7) is an appropriate constant, and generally, as the value of the subscript i increases, the value of the constant di tends to increase.

[0048]

[Equation 2]   | W1-w3 | ≦ d1,   w1 ≧ w2   d2 ≦ w2 ≦ d5   d2 ≦ L ≦ d6 (2)

It is desirable that each value of these constants di is set to a suitable or optimum value in view of the actual lane marking format (standard) in the country or region to which this embodiment is applied. Further, by linking with the car navigation system, the country or region to which the present embodiment is applied is dynamically obtained in real time, and each di (0 ≦ i
You may comprise so that the value of <= 7) may be optimized automatically. For example, the width w2 of the lane boundary line in FIG.
In the area of cm to 20 cm, for example, d2 = (1
0-ε) cm and d5 = 20 cm may be set.

However, as a case where it is desirable to set the threshold value (particularly the lower limit value) of the marking width wn known from the standard, experience, etc. to a small value by an appropriate positive number ε as described above, for example. For example, the following can be considered. (Case 1) When the white line looks worn and thin. (Case 2) When the white line looks thin due to dirt such as tire marks. That is, it is more preferable that the above formula (2) be determined appropriately or optimally so as to be robust against various noises, taking these various circumstances into consideration. Incidentally, these circumstances are exactly the same for other discriminants described later.

In step 420, it is determined whether the position of the marking pattern, which is determined to be a triple line, is on the left or right of the road surface image. This determination is made, for example, in FIG.
This can be performed by using a judgment criterion such as which of the left and right sides the falling peak position (FIG. 7, FIG. 8) of the (J + 1) th sign of is located with respect to the center line of the screen.

In step 422, the value of L3C (left triple line counter) is incremented by 1. This counter L3C
Is referred to when the line type mode determination unit 112 in FIG. 2 (that is, step 355 in FIG. 9 or the execution procedure 500 in FIG. 14) determines the left line type mode. In step 424, a subroutine (noise removal 1) described later is called and executed. This subroutine (noise removal 1) is for removing the block marking corresponding to the shaded area in FIG. 1A as noise. That is, this subroutine deletes or invalidates the data of the corresponding white line marking determined to be noise (block marking) in the search information table 10 (FIG. 8).

At step 426, the value of R3C (right triple line counter) is incremented by 1. This counter R3C
Is referred to when the line type mode determination unit 112 in FIG. 2 (that is, step 360 in FIG. 9) determines the right line type mode. In step 428, a subroutine (noise removal 2) described later is called and executed.

In step 430, the marking pattern shown in FIG. 1B is determined to be the left double line if it satisfies the condition shown in the following expression (3). However, when the marking pattern that satisfies the following expression (3) is located on the right side of the road surface image, the marking pattern is not determined as the left double line.

## EQU00003 ## d2.ltoreq.w1.ltoreq.d5, d3.ltoreq.w2.ltoreq.d6, d0.ltoreq.L1.ltoreq.d4 (3) In step 435, a subroutine (noise removal 3) described later is called and executed.

In step 440, the marking pattern shown in FIG. 1C is determined to be the right double line if it satisfies the condition shown in the following equation (4). However, when the marking pattern that satisfies the following expression (4) is located on the left side of the road surface image, the marking pattern is not determined as the right double line.

## EQU00004 ## d3.ltoreq.w1.ltoreq.d6, d2.ltoreq.w2.ltoreq.d5, d0.ltoreq.L1.ltoreq.d4 (4) In step 445, a subroutine (noise removal 4) described later is called and executed.

Next, in step 450 of FIG. 11, in FIG.
Regarding the marking pattern (blank) of (d), the following equation (5)
Those that satisfy the condition shown in are determined to be the above-mentioned "middle-out" marking patterns.

[Equation 5]   | W1-w2 | ≦ d1,   d2 ≦ L ≦ d6 (5)

In step 460, it is determined whether the position of the marking pattern, which is determined to be "middle-out", is on the left or right side of the road surface image. This determination is made, for example, in FIG.
This can be performed by using a determination criterion such as whether the midpoint of the falling peak position of each of the Jth and J + 1th markings in (d) is located on the left or right side of the center line of the screen.

In step 464, a subroutine (noise removal 5) described later is called and executed. Step 46
In 8, the subroutine (noise removal 6) described later is called and executed.

In step 470, it is determined whether or not the width w1 of the J-th marking which is the current determination target satisfies the following expression (6). Those that satisfy the white line width condition are presumed to be a single lane boundary line (single line).

## EQU00006 ## d2.ltoreq.w1.ltoreq.d7 (6) In step 475, the increment K of J is set to 1. In step 480, a subroutine (noise removal 7) described later is called and executed. The markings removed by this subroutine (noise removal 7) are mainly noises that are not directly related to the lane markings. The above-described sub-routine (noise removal n) distribution processing corresponds to the line type classification unit 111 in FIG. The above process (execution procedure 4)
When (00) is completed, the process is transferred to the caller (control is returned).

The processing of the boundary position estimating means 120 (FIG. 2) described above will be described below with reference to FIG. In the present embodiment, the boundary position estimating means 120 is embodied by a group of subroutines (noise removal n (1 ≦ n ≦ 7)) that executes the above noise removal processing.

FIG. 12 shows a list of boundary position estimation standards that define the operations of these noise removing units (subroutine name: noise removing 1 to 7). By the operation of the line type classification unit 111 (FIG. 1) illustrated in FIGS. 10 and 11, each line type shown in the list of FIG. 12 is classified. However, a single line (single lane boundary line) does not require noise removal processing, and therefore is not included in the list of FIG.

The meaning of each symbol in the list (boundary position estimation reference table) of FIG. 12 is as follows. (Symbol) ×: Unconditionally determines (estimates) the marking to be operated as noise, and deletes or invalidates the corresponding white line pair data on the search information table 10 described above. △: Only when the "mark selection condition" in the table is not satisfied, the mark to be operated is judged (estimated) as noise,
The corresponding white line pair data on the search information table 10 is deleted or invalidated. -: Not subject to operation or reference.

(Noise removal 1) Therefore, for example, in the subroutine "noise removal 1", only when the value of the flag LF3 is 1, the (J + 1) th marking (lane boundary line located at the center) in FIG. Is excluded from the deletion target and the corresponding data is left on the table. That is, it is adopted as a valid sign. After that, the value of the increment K of J is unconditionally set to 3, and the control is returned to the calling source.

(Noise Removal 6) Further, for example, in the subroutine "noise removal 6", the Jth and J + 1th markings in FIG. 1D are excluded from the deletion target table only when the value of the flag RF3 is 0. Leave the relevant data above. That is, it is adopted as a valid sign. Incidentally, thereafter, the value of the increment K of J is unconditionally set to 2, and the control is returned to the calling source.

FIG. 13 shows the relationship between each line type mode and the mode flag (1 bit). These four flags are
It is updated in step 355 (subroutine for performing left line type mode determination) and step 360 (subroutine for performing right line type mode determination) in FIG. 9 corresponding to the line type mode determination unit 112 (FIG. 2). That is, the subroutine for determining the left and right line type modes is executed when the noise removal processing for one screen of the road surface image for the search information table 10 is completed. It should be noted that such a processing procedure for each screen is realized by steps 345 and 350 in FIG.

FIG. 14 shows the line type mode determining section 112 of FIG.
Execution procedure 500 for realizing (left line type mode determination in FIG. 9)
Is illustrated. The execution procedure 500 also realizes the "mode change holding means" of the eighth means of the present invention at the same time.

In this flowchart, first, in step 510, the line type classification unit 111 (FIGS. 2 and 10) is used.
Reference is made to the left triplet line counter L3C, which is updated in step 422 of FIG. This counter L3C is the total number of triple line marking patterns (FIG. 1 (a)) located in the left half of the screen, which is detected from the search information table 10 obtained as a result of searching one screen of the road surface image data. Holding

As a result, if the value of the counter L3C is larger than the predetermined threshold value α, the process proceeds to step 520, and if not, the process proceeds to step 530. Step 5 below
20 and step 525, and step 530 and step 535, the following two types of counters (LM1, LM
2) is updated.

(1) LM1 (left single line mode counter) The number of times that the following "left single line state determination condition" is continuously satisfied is counted (step 530). Therefore,
If the "left single line state determination condition" is not satisfied even once, the count value is returned to 0 on the spot (step 520). This guarantees the continuity of the left single line state during the period counted by this counter. << Left Single Line State Judgment Condition >> The number (L3C) of triple line marking patterns (FIG. 1 (a)) detected on the left side of the road surface image is less than or equal to a predetermined threshold value α in one screen. thing.

(2) LM2 (left composite line mode counter) The number of times that the following "left composite line state determination condition" is continuously satisfied is counted (step 525). Therefore,
If the "left composite line state determination condition" is not satisfied even once, the count value is returned to 0 on the spot (step 535). This ensures the continuity of the left composite line state during the period counted by this counter. << Left Composite Line State Judgment Condition >> The number (L3C) of the marking patterns (FIG. 1 (a)) of the triple line marking detected on the left side of the road surface image is larger than the predetermined threshold α in one screen. thing.

<Mode changing condition 1> The value of the left single line mode counter LM1 exceeds a predetermined threshold value γ. This condition is checked in step 560, and when this condition is satisfied, the left line type mode is set to the single line mode according to the definition of FIG. 13A by the operations of step 570 and step 575. It

<Mode changing condition 2> The value of the left composite line mode counter LM2 exceeds a predetermined threshold value β. This condition is checked in step 540, and if this condition is satisfied, the left line type mode is set to the triple line mode according to the definition of FIG. 13A by the operations of step 550 and step 555. It

If the mode changing conditions 1 and 2 are not satisfied, the mode is not changed. As a result, the "mode change holding means" described above is realized. By such means, the line type mode is changed only when the situation is constant, so that it is possible to realize a robust mode determination process against noise.

In the above example, the judgment of the left single line state and the left compound line state is made by the threshold value α using steps 510 to 535. It is also possible to compare the average value of the number of line markings (L3C) for a certain period (period over a plurality of control cycles) with a predetermined threshold value α '. In addition to this average value, the number of triple line markings (L3C) for a certain period (eg, 20)
The minimum value of (m running period) may be compared with a predetermined threshold value α ″ at the same time. By various modifications of these statistical operations, a more robust algorithm against noise can be configured.

For example, in the logic of FIG. 14, the value of β is 9
If the condition of step 510 is not satisfied 10 times or more in succession, the mode does not shift to the left triple line mode (ie,
Do not execute step 550). However, as a modification of the above, for example, the average value of the counter L3C is obtained 10 times, and if the average value for 10 times is equal to or more than a predetermined threshold value α ', the mode is shifted to the left triple line mode (that is, ,
The logic may be configured such that step 550 is executed).

In steps 580 and 585, the left double line mode flag LF2 is set. This flag setting process may be performed in substantially the same manner as the mode setting process from step 510 to step 575, but on a normal highway, if the right side is a triple line, as shown in FIG. Is a double line (or a triple line), and when the left side is a double line, the right side is a triple line, so that the applicable range of the lane boundary determination device 100 of the present embodiment is limited to an expressway. In doing so, such a setting process of the left double line mode flag LF2 may be performed. The frequency of appearance of noise that can be erroneously determined as a triple line in the road image is usually small enough,
For example, when the line type mode determination is performed based on the determination result of the triple line as described above, it is possible to configure a determination unit that is robust against noise. When the above process is completed,
The execution procedure 500 of FIG. 14 is terminated, and the process is returned to the calling source.

According to the following procedure 1 to 3, as an alternative process to the above steps 580 and 585,
The setting process of the left double-line mode flag LF2 can be independently configured in substantially the same manner as the mode setting process of steps 510 to 575 of FIG. (Processing procedure 1) L2C (left double line counter) and R, which are the same counters as the counters L3C and R3C,
2C (right double line counter) is installed and L3C and R
It is initialized (cleared to 0) at the same timing as 3C.

(Processing procedure 2) The line type classification unit 111 uses the counter L2C to determine the total number of double line marking patterns (FIG. 1 (b) or (c)) located in the left half of the screen. To count. However, this total number may be counted for each marking pattern ((b) or (c)). These processes are the same for the counter R2C.

(Processing procedure 3) Based on the value of the counter L2C, the line type mode determination unit 112, for example, FIG.
In substantially the same manner as steps 510 to 575, the switching processing of the left line type mode is executed. Note that step 580 and step 585 in FIG. 14 are deleted. These processes are the same for the line type mode switching process on the right side.

Such a processing form is not necessarily related to the presence / absence of a triple line in the lane marking, but is effective in a country, region or road where a double line is marked independently of the presence / absence of a triple line. is there. These processing modes may be carried out by selecting an optimum one suitable for an actual concrete environment such as a country, an area, and a road.

Also in step 360 of FIG. 9,
The right line type mode determination process may be performed in exactly the same manner as above. When the above processing is completed, the execution procedure 3 in FIG.
00 is terminated and the process is returned to the calling source. Through the above processing, step 250 of the execution procedure 200 of FIG. 3 is executed. As a result, the process of the boundary marking selecting unit 2 in FIG. 2 is completed.

Hereinafter, lane boundary line candidate point data (retained as valid data) adopted by the above procedure,
That is, the following well-known statistical operation ([1] is performed on the data of many valid white line points remaining on the search information table 10.
The lane boundary position can be determined (estimated) by performing straight line detection and [2] lane selection). That is, FIG.
In step 260, the following "straight line detection" is executed. Further, in step 270 of FIG. 3, the following “lane selection” is executed. Then, in step 280 of FIG. 3, the boundary position obtained based on these statistical operations is output to a predetermined output destination (eg, lane departure warning device or the like).

[1] Straight Line Detection (Hough Transform) By the processing of the composite line determination, the noise corresponding to the block white line of the composite line can be removed from the white line point list, but random noise due to dirt on the road surface remains. . Therefore, the straight line on the road surface image is detected by the well-known Hough transform.
The Hough transform is a typical method that can robustly detect straight lines from a noisy image.

Since the lane boundary is defined as the boundary line between the driving lane and the white line, the rising peak position of the white line pair is represented on the right half plane of the road surface image, and the falling peak position of the white line pair is represented on the left half plane of the road surface image as a representative point ( Hough transform is executed as the voting point. The Hough transform sequentially reads the positions of white line points from the list of white line points, and votes to all straight lines that may pass the position of the white line points. Voting is done in a two-parameter space that represents a straight line (the intercept and slope of the straight line). After voting all the white line dot lists, a plurality of peaks of voting values accumulated in the parameter space are detected. This peak indicates the parameter value of a straight line that is a candidate for a lane boundary. This parameter value and the number of votes are stored in memory as a list.

[2] Lane selection As a result of the Hough transform, a plurality of straight lines that are candidates for lane boundaries in the road surface image are detected. In lane selection, all straight lines detected by Hough transformation are combined to form a pair, and the pair with the highest certainty is selected from those corresponding to the left and right lane boundaries of the lane. It satisfies the following conditions.

1) The right and left straight line length is 0.5 m or more. 2) The lane parameters calculated from the straight line pair are within the setting range. The setting range is as follows. (A) Position deviation: -Lane width calculated value / 2 to + Lane width calculated value / 2 (b) Lane width: LW0-Standard deviation to LW0 + Standard deviation (c) Yaw angle: -Yaw angle threshold value to + Yaw angle threshold value (D) Pitch angle: −Pitch angle threshold value to + Pitch angle threshold value Here, LW0 is a reference value of the lane width,
It is usually selected from the following. (General road) ・ ・ ・ 2.75m, 3.00m (Expressway) ・ ・ ・ 3.25m, 3.50m, 3.75
m

FIG. 15 shows the definition of each lane parameter used in these determination processes (lane selecting means 4 in FIG. 2). The formulas for the lane parameters (position deviation, lane width, yaw angle, pitch angle) when two white lines on the left and right as shown in Fig. 15 are detected on the road surface image are shown below. At this time, it is assumed that the road surface is locally flat and cos Δφ≈1. The depth d of the road surface image is a distance at which the lane mark can be regarded as a straight line.

(Variable definition) e: position deviation [m], w: lane width [m], θ: yaw angle [rad], Δφ: pitch angle variation [ra with camera depression angle as a reference value]
d], h: Camera height [m], d: Depth distance [m] on the road surface image, p _{R, L} : Position [m], q _{R, L where the} lower edge of the road surface image (the position directly below the camera) intersects with a straight line : Difference [m] between p _{R, L} and the position where the straight line intersects the upper edge of the road image.

A pair satisfying the above conditions is extracted, and the one having the maximum sum of the left and right straight line lengths is determined as the pair indicating the boundary line of the currently running lane. As a result of the Hough transform, the straight line length is not directly obtained, but for example, the maximum sum of straight line lengths can be determined from the number of votes obtained from the Hough transform.

The pair selected above is taken as the result of the lane boundary. When the environmental condition deteriorates, the false recognition of noise may not be completely removed. Therefore, if necessary, the noise may be filtered by a known low-pass filter or the like and then output to a lane departure warning device or the like. For example, by applying the above-described embodiment, the accurate lane boundary position can be stably and continuously detected even when the lane marking on the road surface is, for example, the complex line as described above.

(Second Embodiment) In the second embodiment, an improvement example of the first embodiment will be illustrated. FIG. 16 is a plan view of the lane marking according to the second embodiment. This Figure 16
Both (a) and (b) exemplify the lane markings appearing on the left side of the vehicle, and the shaded portions indicate the lane markings drawn on the road surface. When the angle of the optical axis of the camera is shallow with respect to the position of the lane markings on the road surface, when the resolution of the captured image is low, when there is a lot of noise on the image, or when the lane markings are drawn on the road surface in the desired format. If not, for example, as illustrated in FIG. 16, the block markings that compose the composite line and the lane boundary line that indicates the lane boundary may appear to be integrated.

In these cases, the lane markings are recorded on the table 10 of FIG. 8 as if they were one line, and the white line width (width of a single line) at that time is as shown in the figure. The white line width wa and wb are recorded as values. Therefore,
In such a case, in the above-described first embodiment, it becomes difficult to accurately grasp the boundary position of the lane, and the boundary position of the lane may vibrate. In the second embodiment, means for dealing with such a problem will be illustrated.

FIG. 17 is a flow chart illustrating a new extension portion for the execution procedure 400 (FIGS. 10 and 11) for realizing the line type classification unit 111 (FIG. 9 step 340) of FIG. 2 according to the second embodiment. Is. That is, the second embodiment solves the above problem by adding (inserting) the process (steps 655 to 669) of FIG. 17 immediately before step 470 of FIG. However, the above-mentioned counters L3C, R3C, L2C, R2C
For the above, (procedure 1), (procedure 2),
It shall be initialized, updated, and referenced in accordance with (Processing Procedure 3). That is, in the second embodiment, for example, these counters are updated (incremented) at the following timings.

(Counter L3C) Immediately before each subroutine of noise removal 1, noise removal 5, and data correction 1 (FIG. 18). (Counter R3C) Immediately before each subroutine of noise removal 2, noise removal 6, and data correction 3 (not shown). (Counter L2C) Immediately before each subroutine of noise removal 3 and data correction 2 (FIG. 18). (Counter R2C) Immediately before each subroutine of noise removal 4 and data correction 4 (not shown).

The following threshold values may be used in the following processing. (Threshold value) D2D: Lower limit value D2 of the width wa of the white line with the gap shown in FIG. 16 (a): Upper limit value W2 of the white line with the gap of FIG. 16 (a) D3D: The gap of FIG. 16 (b) Lower limit value D3 of the width wb of the white line clogged with: The upper limit value of the width wb of the white line clogged with the gap shown in FIG. 16 (b). These values are appropriate values based on actual road conditions and the resolution of the camera. Empirically set and set.

In step 655 of FIG. 17, it is determined whether or not the width of the Jth mark, that is, the value of the white line width w1 currently being analyzed is larger than a predetermined value DS. This value DS is
For example, the upper limit value of the standard appropriate range of the width of the lane boundary line composed of a general single line is suitable, and generally, the value is the threshold value D2D or less. That is, the second embodiment is characterized by the processing when the width w1 of a single line that appears to be one is larger than the predetermined value DS. Step 658,
The variable K in step 669 is the increment K of J described above.

FIG. 18 is a flow chart exemplifying a subroutine (correction of left data) for realizing step 664 of FIG. In this flowchart, step 71
The lane marking pattern of FIG. 16B (left triple line with a closed gap: white line width = w by a series of processes from 0 to 740)
The process corresponding to b) is performed, and steps 750 to 780 are performed.
By the series of processes up to, the process corresponding to the lane marking pattern (left double line with a closed gap: white line width = wa) in FIG. 16A is performed.

For example, in step 710, FIG.
The left triple line mode flag LF3 in (a) is checked.
Further, in step 715, the proper range of the white line width of the triple line with the narrow gap is checked. In step 730 (data correction 1), the representative point (peak position) of the corresponding white line recorded in the table 10 of FIG.
The position is corrected to the position indicated by the thick vertical line 6 (b) (the position of the approximate center point of the width wb). It is more desirable to carry out the process of "data correction 1" by learning from FIG. 19 as follows, for example.

FIG. 19 is a conceptual diagram for explaining more preferable processing contents of the subroutine (data correction 1) for realizing step 730 of FIG. That is, in the case of the left triple line with a narrow gap, the position is shifted to the right by half the standard width D0 of the lane boundary line from the midpoint of the white line width wb (● in the figure).
Mark) is adopted as the representative point above.

Steps 750 to 780 in FIG.
As for the above, the correction process is executed in substantially the same manner as the above steps 710 to 740. FIG. 20 is a conceptual diagram for explaining the processing contents of a subroutine (data correction 2) that realizes step 770 of FIG. That is, in the case of the double line on the left side where the gap is closed, the position is shifted to the right by the standard width D0 of the lane boundary line from the left end of the white line width wa (marked with ● in the figure) in substantially the same manner as above. ) Is adopted as the representative point above.

Even when the data correction processing on the right side (subroutine "correct right data" in step 668 of FIG. 17) is executed,
It may be carried out in the same manner as above, paying attention to the left-right symmetry.
For example, the above-mentioned "data correction 3", which is not shown, can be constructed by learning from the above-mentioned data correction 1, and the above-mentioned "data correction 4" can be learned by following the above-mentioned data correction 2. Can be configured. By the above data correction processing,
Since more appropriate representative points are left on the table 10 in FIG. 8, the above-mentioned problem can be solved.

Further, since noise may be mistakenly recognized as a thick line, a method of determining the line type mode by comprehensively judging the appearance situation of the left composite line and the appearance situation of the right composite line is adopted. Is more desirable. According to such a method, more robust line type mode determination can be performed as illustrated below. For example, when improving the execution procedure 500 of FIG. 14, it is desirable to take the following method.

That is, for example, the following conditional expressions (7)-
When any one of (9) is established, it is determined that there is a double line and a triple line (or a triple line and a triple line).

[Equation 7]   L2C> threshold value α1 and R3C> threshold value α2 (7)

[Equation 8]   L3C> threshold value α2 and R3C> threshold value α2 (8)

[Equation 9]   L3C> threshold value α2 and R2C> threshold value α1 (9)

Similar to the method used in the execution procedure 500 of FIG. 14, the number of scenes (shooting screens) satisfying the above conditions is counted, and if such a scene continues for a set number of times or more,
Switch to the composite line mode. Further, the double line or the triple line may be determined on the side having the most points. For example, a more robust line type mode determination can be performed by such a line type mode determination method.

In the above embodiment, the case of using the camera for photographing the situation in front of the vehicle is shown, but the orientation of the camera may be arbitrary. That is, it is possible to obtain the action and effect of the present invention even when the image is taken directly under, behind, or on the side of the vehicle. In implementing the present invention, there is no restriction on the number of cameras. For example, various application modes such as improving detection accuracy by capturing a wide range with a plurality of cameras may be implemented.

[Brief description of drawings]

FIG. 1 is a plan view illustrating a marking pattern of a compound line (lane marking).

FIG. 2 is a lane boundary determination device 10 according to an embodiment of the present invention.
The block diagram which illustrates the structural form of 0.

FIG. 3 is a flowchart illustrating an execution procedure (200) of each unit when the lane boundary determination apparatus 100 is realized by a computer system having an in-vehicle camera.

FIG. 4 is a photograph illustrating an input image (a) from a vehicle-mounted camera and a road surface image (b) analytically generated based on the input image.

FIG. 5 is an explanatory diagram showing the relationship between a corrected image obtained by correcting the distortion of an input image (FIG. 4A) and a road surface image (FIG. 4B).

6 is a perspective view of the field of view of the camera, which further supplements the description according to FIG.

FIG. 7 is an explanatory view for explaining the operation of the sign extracting means 1 (FIG. 2).

FIG. 8 is a table structure diagram of a search information table 10 illustrating a storage mode (data format) of lane markings stored as search information according to an embodiment of the present invention.

9 is a block diagram of the boundary marking selecting unit 2 of FIG.
10 is a flowchart illustrating an execution procedure (300) for realizing 0).

10 is a diagram illustrating a line type classification unit 111 of FIG. 2 (step 34 of FIG. 9).
0) A flowchart illustrating the execution procedure (400) for realizing (0) (first half).

11 is a diagram illustrating a line type classification unit 111 of FIG. 2 (step 34 of FIG. 9).
0 is a flowchart illustrating the execution procedure (400) for realizing (0) (second half).

FIG. 12 is a list of boundary position estimation standards that define the operation of a noise removal unit (subroutine name: noise removal 1 to 7).

FIG. 13 is a definition table showing a relationship between each line type mode and a mode flag (1 bit).

14 is a flowchart illustrating an execution procedure (500) for realizing the line type mode determination unit 112 (left line type mode determination of FIG. 9) of FIG.

15 is a lane selection means 4 of FIG. 2 (step 2 of FIG. 3).
70) is an explanatory view showing the definition of each lane parameter used in FIG.

FIG. 16 is a plan view of a lane marking according to the second embodiment of the present invention.

FIG. 17 is a flowchart illustrating a new extension portion for the execution procedure 400 (FIGS. 10 and 11) that realizes the line type classification unit 111 (FIG. 9, step 340) according to the second embodiment of the present invention.

FIG. 18 is a flowchart illustrating a subroutine (correction of left data) for realizing step 664 of FIG.

FIG. 19 is a conceptual diagram illustrating the processing contents of a subroutine (data correction 1) that realizes step 730 of FIG.

20 is a conceptual diagram illustrating the processing content of a subroutine (data modification 2) that implements step 770 of FIG.

FIG. 21 is a plan view exemplifying a marking pattern of a compound line (lane marking) within the field of view of a vehicle-mounted camera of a vehicle traveling on a traveling lane.

FIG. 22 is a plan view exemplifying a marking pattern of a compound line (lane marking) within the field of view of an on-vehicle camera of a vehicle traveling on an overtaking lane.

[Explanation of symbols]

1 ... Mark extraction means 2 ... Boundary mark adoption section 3 ... Straight line detection means 4 ... Lane selection means 10 ... Search information table 100 ... Lane boundary determination device 110 ... Composite line determination means 111 ... Line type classification unit 112 ... Line type mode determination Part 120 ... Boundary position estimating means C ... Camera viewpoint Σ ... Road surface image plane I ... Row number of road surface image data (pixels) (1 ≦ I ≦
m) J ... Numbers attached to the markings from the left side of the road surface image (1 ≦ J
≦ NI) L3C ... left triplet line counter R3C ... right triplet line counter LF3 ... left triplet mode flag RF3 ... right triplet mode flag LF2 ... left doublet mode flag RF2 ... right doublet mode flag LM2 ... left compound line mode counter LM1 ... left single line mode counter α ... left compound line state determination threshold β ... left compound line mode entry threshold γ ... single line mode return threshold

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁷ Identification symbol FI theme code (reference) G08G 1/16 G08G 1/16 C (72) Inventor Takahashi Shin 41 Address 1 Toyota Central Research Institute Co., Ltd. (72) Inventor Yoshiki Ninomiya 41, Nagakute-cho, Aichi-gun, Aichi-gun Nagakute-cho, Yokouchi Central Research Institute Inc. (72) Inventor Hisashi Satonaka Toyota, Aichi Prefecture Toyota Town No. 1 Toyota Automobile Co., Ltd. (72) Inventor Makoto Nishida No. 1 Toyota Town, Toyota City, Aichi Prefecture Toyota Auto Co., Ltd. (72) Inventor Munehiro Takayama 2-1-1 Asahi Town, Kariya City, Aichi Prefecture Aisin Seiki Co., Ltd. F term (reference) 5B057 AA16 BA02 CD12 DA07 DB02 DB09 DC03 DC19 DC22 DC30 DC36 5H180 AA01 CC04 CC24 LL07 5L096 BA04 CA04 EA07 FA03 FA38 FA64 FA69 FA73 GA02

Claims (13)

[Claims] 1. A determination device for determining a boundary position of a lane (lane) defined by a lane marking drawn on a road surface in real time based on an image input from a camera, the lane The direction of the marking or the direction of travel of the host vehicle is the vertical direction, and the direction orthogonal to this vertical direction is the search direction for the lane markings, and based on the search information obtained by the search in this search direction, the lane markings are , A composite line determination means for determining whether or not the composite line is composed of a plurality of columns or a plurality of markings having a context in the vertical direction, and the boundary position is estimated based on the composite line determination result. A lane boundary determination device, comprising: 2. The composite line determination means determines that the lane marking is a composite line based on a plurality of pieces of search information at a plurality of search positions on the input image in which the search directions are substantially parallel to each other. The lane boundary determination device according to claim 1, wherein it is determined whether or not the lane boundary is determined. 3. The composite line determination means determines whether or not the lane marking is composed of a composite line based on a relationship between the latest search information and the past search information. The lane boundary determination device according to claim 1 or 2. 4. The lane marking is sandwiched by block markings on both sides from at least one of the number of markings included in the search information, the width of the markings, or the gap width between the markings. The lane boundary determination device according to any one of claims 1 to 3, wherein it is determined whether or not it is composed of a composite line configured by one marking line. 5. The composite line determination means determines, based on at least one of the number of markings, the width of the markings, or the gap width between the markings included in the search information, that the lane markings have block markings approaching one side. The lane boundary determination device according to claim 1, wherein the lane boundary determination device determines whether or not it is composed of a composite line composed of book marking lines. 6. The lane markings are arranged in parallel in the vertical direction from at least one of the number of markings, the width of the markings, or the gap width between the markings included in the search information. The lane boundary determination device according to any one of claims 1 to 5, wherein it is determined whether or not it is composed of a composite line composed of two or two columns of block markings. 7. The composite line determination means includes a line type mode determination unit that determines a line type mode of the lane marking that is currently traveling based on the past or present search information. The lane boundary determination device according to any one of claims 1 to 6. 8. The line type mode determination unit has a mode change holding means for holding the line type mode change processing until the search information continuously satisfies a "predetermined mode change condition" for a certain period of time. The lane boundary determination device according to claim 7, wherein: 9. The boundary position estimating means determines and removes an unnecessary portion such as a block marking included in the lane marking from the search information based on the line type mode determined by the line type mode determining unit. The lane boundary determination device according to claim 7 or 8, further comprising a noise removal unit. 10. The boundary position estimating means is based on the line type mode determined by the line type mode determination unit when the width of the lane markings searched for as a single line exceeds a predetermined value. The lane boundary determination device according to any one of claims 7 to 9, wherein the boundary position is estimated. 11. The boundary position estimating means has the shortest width 1 detected as a part of the composite line.
The lane boundary determination device according to any one of claims 1 to 10, wherein the position of one marking is estimated as the boundary position. 12. The boundary position estimating means estimates, as the boundary position, an intermediate point in the search direction of two or two columns of block markings that are detected as the composite line and are arranged in parallel in the vertical direction. The lane boundary determination device according to any one of claims 1 to 11, wherein: 13. The boundary position estimating means determines the boundary position by the block marking when it is judged in the composite line judgment result that the lane marking is composed of one or one column of block markings. The lane boundary according to any one of claims 1 to 12, wherein the boundary position is estimated based on the position of the lane boundary, the boundary position estimation is suspended, or the boundary position is indefinite. Judgment device.