DE102022126904A1 - Neighboring lane detection based on lane models - Google Patents

Abstract

The invention relates to a method for detecting neighboring lanes (18, 24, 25) with the provision of individual images of the lanes (18, 24, 25), the extraction of lane features (23) from the individual images, wherein the lane features (23) define the course of the lane boundaries (19, 20, 21, 22) of the lanes (18, 24, 25), the provision of a first lane model for modeling the course of the lane boundaries (19, 20, 21, 22) on the basis of the extracted lane features (23), wherein a respective course is described by its coordinates in a common coordinate system. The method further includes calculating the width (w) of a first of the lanes (18) from the coordinates, assigning the width (w) to at least one of the neighboring lanes (24, 25), and estimating the course of the lane boundaries (19, 20, 21, 22) of the at least one neighboring lane (24, 25) from the assigned width (w).

Description

The present invention relates to a method for detecting neighboring lanes, including providing single images or individual images of the lanes, extracting lane features from the single images, wherein the lane features define the course of the lane boundaries of the lanes, and providing a first lane model for modeling the course of the lane boundaries from the extracted lane features, wherein a respective course is described by its coordinates in a common coordinate system. Furthermore, the present invention is directed to a driver assistance system, a device and a computer program product that is capable of carrying out the method for detecting neighboring lanes.

Various driver assistance systems (ADAS) are known from the state of the art. Such driver assistance systems can have various levels of automation, starting from level 1, such as ACC (adaptive cruise control), up to level 5, fully autonomous driving. To implement these driving functions that support the driver, or even fully autonomous driving functions, it is necessary for such systems to be able to detect objects, obstacles or other road users ahead. In addition, in most of the systems it is also important to estimate whether such objects or obstacles are located in the lane or in the driving area that the vehicle is likely to drive through, for example in order to adjust the speed and/or maintain a certain safety distance from a vehicle ahead. If, for example, a vehicle ahead is detected but it is not in the lane of the vehicle, it is therefore not necessary to warn the driver or automatically slow down the vehicle. It is therefore necessary not only to know whether objects ahead, such as vehicles driving ahead, are generally present, but also to know as reliably as possible whether a vehicle driving ahead of the vehicle is in the lane area in which the vehicle is to travel or not.

Lane detection using camera systems was one of the first computer vision ADAS features developed. Such lane detection systems can be based on lane models for multi-camera and fisheye camera systems.

US 2010/0054538 A1 discloses, for example, a method for universal lane boundary detection based on an image of a road ahead of a vehicle. According to the proposed method, a region of interest (ROI) is determined in the image and road markings are detected by detecting lane markings in the region of interest.

Also the WO 2019/224103 A1 discloses a method for tracking a lane for a vehicle, using road markings or lane features of the lane to model the lane according to a first lane model. Furthermore, a second lane model is provided which is based on an outlier algorithm. A respective confidence value is calculated for each of the lane models and on the basis of the confidence values it is decided which model should be used to calculate current information about the lane.

Additional problems can arise when semi-autonomous or fully autonomous vehicles change lanes on a multi-lane road. It can cause stability problems if there is an additional lane that is unknown to the ADAS.

It is an object of the present invention to increase the stability of lane detection in a multi-lane environment. Furthermore, a corresponding device and a corresponding computer program product are to be provided.

According to the present invention, this object is achieved by a method, a device and a computer program product as defined in the independent claims.

1. a) Bereitstellen von Einzelbildern der Fahrspuren,
2. b) Extrahieren von Fahrspurmerkmalen aus den Einzelbildern, wobei die Fahrspurmerkmale den Verlauf von Fahrspurbegrenzungen der Fahrspuren definieren,
3. c) Bereitstellen eines ersten Fahrspurmodells zum Modellieren des Verlaufs der Fahrspurbegrenzungen auf der Basis der extrahierten Fahrspurmerkmale, wobei ein jeweiliger Verlauf durch seine Koordinaten in einem gemeinsamen Koordinatensystem beschrieben wird, sowie
4. d) Berechnen einer Breite einer ersten der Fahrspuren aus den Koordinaten,
5. e) Zuweisen der Breite zu mindestens einer der Nachbarfahrspuren,
6. f) Abschätzen des Verlaufs der Fahrspurbegrenzungen der mindestens einen Nachbarfahrspur aus der zugewiesenen Breite.

Accordingly, the present invention is directed to a method for detecting adjacent lanes in a multi-lane road, comprising the steps

1. a) Providing individual images of the lanes,
2. b) Extracting lane features from the individual images, whereby the lane features define the course of lane boundaries of the lanes,
3. c) providing a first lane model for modelling the course of the lane boundaries on the basis of the extracted lane features, wherein a respective course is described by its coordinates in a common coordinate system, and
4. d) Calculating a width of a first of the lanes from the coordinates,
5. e) Assigning the width to at least one of the adjacent lanes,
6. f) Estimating the course of the lane boundaries of at least one adjacent lane from the assigned width.

For example, a vehicle traveling in a lane or in the so-called own lane may need to detect or track the own lane to provide assistance information to the driver. To do so, the vehicle may include one or more cameras (for example, a multi-camera system or a fisheye camera system) to provide images (also called single images) of the lane. Lane features describing the course of the lane boundaries of the lane are extracted from these images. Lane features may include a heading angle, a lateral position (with respect to the vehicle), a lane width and curvature, or lane markings. A first lane model for modeling or predicting the course of the lane boundaries based on the extracted lane features is provided.

According to the invention, a respective course of a lane boundary is described by its coordinates in a coordinate system that can be centered on the vehicle. In other words, the vehicle can be arranged at the intersection of a virtual x-axis and virtual y-axis, with the x-axis running along the vehicle's longitudinal direction and the y-axis running perpendicular to the x-axis along the vehicle's transverse direction. The own lane can be modeled as a set of two coupled boundaries (left and right) that can be described by third-order polynomials that differ only in their intersection with the described y-axis (see equation 1): $$\begin{array}{l}{y}_{Leet}=c_3{x}^3+c_2{x}^2+c_{1}x+c_{0Leet}\hfill \\ y_{RiGHt}=c_3{x}^3+c_2{x}^2+c_{1}x+c_{0RiGHt}\hfill \end{array}$$

Where the intersection of the y-axis with the left boundary (c _Left ) is positive, whereas the intersection with the right boundary (c _Right ) is negative.

According to the invention, the width of a first of the lanes, for example the width of the own lane, is calculated from the coordinates of the courses of the lane boundaries, for example as the difference between the left and right intersection points, c _Left and c _Right .

The width of the first of the lanes is then assigned to at least one of the neighboring lanes. In other words, it is assumed that the at least one neighboring lane has the same width as the first of the lanes. The course of the lane boundaries of the at least one neighboring lane is then estimated from the assigned width. This can be done, for example, by virtually moving one or both of the coupled boundaries of the own lane according to the amount of the calculated width in a direction away from the position of the vehicle in the common coordinate system.

The method according to the invention is advantageous in that it enables the estimation or detection of neighboring lanes without necessarily collecting real data describing the neighboring lanes, assuming that the neighboring lanes have the same width as the own lane. Consequently, the method according to the invention provides an efficient way of providing information about neighboring lanes to an ADAS, which in turn improves the overall stability of lane detection in a multi-lane environment.

According to a preferred embodiment, the first of the lanes, preferably the own lane, and the at least one of the neighboring lanes share a common lane boundary. In other words, the first of the lanes and the at least one neighboring lane run directly next to each other. In still other words, the left boundary of the first of the lanes (or the own lane) can simultaneously be the right boundary of the at least one neighboring lane (the at least one neighboring lane that runs on the left side of the own lane). According to the described embodiment, the step of estimating the course of the lane boundaries of the at least one neighboring lane from the assigned width (w) includes adding the assigned width to the coordinates of the common lane boundary and/or subtracting the assigned width from the coordinates of the common lane boundary (see equation 2): $$\begin{array}{l}{y}_{Leet}=c_3{x}^3+c_2{x}^2+c_{1}x+c_{0Leet}+w\hfill \\ y_{RiGHt}=c_3{x}^3+c_2{x}^2+c_{1}x+c_{0RiGHt}-w\hfill \end{array}$$

After the course of the boundaries of the neighboring lanes has been estimated, they can be modelled using the same method as for the first of the lanes or the own lane. The neighboring lane, which runs to the left of the own lane, can be modelled as coupled model consisting of the adjacent left boundary and the left self-boundary, while the neighboring lane running to the right of the self-boundary lane can be modeled as a coupled model consisting of the adjacent right boundary and the right self-boundary. Once the lane models for the different lanes are estimated, they can be modeled and tracked independently.

• - Aktualisieren des ersten Fahrspurmodells mit den extrahierten Fahrspurmerkmalen,
• - Bereitstellen eines statischen zweiten Fahrspurmodells auf der Basis eines Ausreißerdetektionsalgorithmus,
• - Berechnen eines verfolgten Konfidenzwerts für das aktualisierte erste Fahrspurmodell und eines statischen Konfidenzwerts für das statische zweite Fahrspurmodell, und
• - Entscheiden auf der Basis beider Konfidenzwerte, ob das aktualisierte erste Fahrspurmodell oder das statische zweite Fahrspurmodell zum Berechnen von aktuellen Informationen über die Fahrspuren verwendet werden soll.

According to a further embodiment, the method comprises the steps

• - Updating the first lane model with the extracted lane features,
• - Providing a static second lane model based on an outlier detection algorithm,
• - calculating a tracked confidence value for the updated first lane model and a static confidence value for the static second lane model, and
• - Based on both confidence values, decide whether the updated first lane model or the static second lane model should be used to calculate current information about the lanes.

In other words, a static second lane model is provided. In contrast to the first lane model, the static second lane model is not updated with current lane features. Rather, the second lane model is based on a fixed outlier detection algorithm. This algorithm distinguishes between outliers and inliers according to predetermined criteria. With respect to the extracted lane features, a confidence value is calculated for each model, for example based on a statistical hypothesis. In particular, a tracked confidence value for the updated first lane model and a static confidence value for the static second lane model are calculated. Using these confidence values, a decision is made to use the updated first lane model or the static second lane model for further calculations related to the lanes. Preferably, the model that provides the higher confidence value is used. This guarantees that the calculated current information about the lane (for example the lane coordinates) is more accurate. Therefore, the estimation of the course of the lane boundaries of the neighboring lanes is also more accurate.

In a preferred embodiment, the updating of the first lane model is performed by predicting a new version of the first lane model using Kalman filtering and adapting the new version of the first lane model to the extracted lane features. Such a procedure allows the model to be easily updated with a plurality of feature elements. These feature elements describe the lane and form a state vector for the Kalman filter.

Preferably, an independent Kalman filter is implemented for each of the lanes. This allows the system to cope with situations where the own and neighboring lane(s) are part of the same road, when initially the neighboring lane(s) appears, but after a while forms a separate road.

Preferably, the first lane model and the second lane model track a lane geometry over time. Thus, the lane models preferably provide current information about lane dimensions, lane position, and/or lane orientation. Such information may be classified with respect to the type of lane.

• - Berechnen einer Trennlinie oder Aufteilungslinie der Fahrspur, vorzugsweise einer Mittellinie, für jede der Fahrspuren,
• - Verwenden der Trennlinie einer jeweiligen Fahrspur zum Aufteilen der extrahierten Fahrspurmerkmale in verschiedene Sätze von Fahrspurmerkmalen, wobei die Fahrspurmerkmale von jedem der Sätze von Fahrspurmerkmalen einer der Fahrspurbegrenzungen der jeweiligen Fahrspur zugewiesen werden,
• - Bestimmen der Nähe von jedem der Fahrspurmerkmale zu seiner zugewiesenen Fahrspurbegrenzung,
• - Bewerten, ob ein jeweiliges Fahrspurmerkmal die abgeschätzte Fahrspurbegrenzung unterstützt, gemäß einem vorbestimmten Bewertungsalgorithmus auf der Basis der Nähe, und
• - Aktualisieren des ersten Fahrspurmodells auf der Basis der unterstützenden Fahrspurmerkmale, die ihre jeweilige Fahrspurbegrenzung unterstützen.

According to a further embodiment, the method comprises the steps

• - Calculating a lane dividing line or division line, preferably a centre line, for each of the lanes,
• - using the dividing line of a respective lane to divide the extracted lane features into different sets of lane features, wherein the lane features of each of the sets of lane features are assigned to one of the lane boundaries of the respective lane,
• - Determining the proximity of each of the lane features to its assigned lane boundary,
• - Evaluating whether a respective lane feature supports the estimated lane boundary according to a predetermined proximity-based evaluation algorithm, and
• - Updating the initial lane model based on the supporting lane features that support their respective lane boundaries.

Preferably, the lane features relate to lane markings, and prior to updating the first lane model with lane markings, the lane markings are divided into two separate sets for a respective left and a respective right boundary of a respective lane by calculating the centerline or dividing line using the first lane model on Preferably, the dividing line is a second order polynomial or a third order polynomial. The sign (positive or negative) of the lateral distance from the dividing line can be used to classify the lane markings into left and right lane markings. Such a separation into a left and right boundary ensures high quality when the corresponding lane model is adapted.

According to a further embodiment, the dividing line or centerline of a respective lane is calculated by adding the assigned width to the coordinates of the dividing line or centerline of its neighboring lane and/or by subtracting the assigned width from the coordinates of the dividing line or centerline of its neighboring lane (see equation 3): $$\begin{array}{l}{m}_{splitline}LL=m_{splitline}+w\hfill \\ m_{splitline}RR=m_{splitline}-w\hfill \end{array}$$

Where m _splitline denotes the dividing line or center line of the own lane, m _splitline LL denotes the dividing line or center line of the left neighboring lane and m _splitline RR denotes the dividing line or center line of the right neighboring lane.

The above-mentioned object can also be achieved by a driver assistance system that is designed to carry out a method according to any method for detecting neighboring lanes as described herein. Any other device for detecting neighboring lanes can be designed to carry out the described methods. Modifications and advantages of the method according to the invention can also apply to the driver assistance system according to the invention or the device according to the invention and vice versa.

Furthermore, the above object can also be achieved by a computer program product with program code means, which are stored in particular in a computer-readable medium, in order to carry out the method for detecting adjacent lanes as described herein when the computer program product is processed on a computer device of an electronic control unit.

Further features of the invention emerge from the claims, the figures and the description of the figures. The features and combinations of features mentioned above in the description as well as the features and combinations of features mentioned below in the description of the figures and/or shown in the figures alone can be used not only in the combination specified in each case, but also in other combinations without departing from the scope of the invention. Thus, embodiments of the invention are also to be regarded as included and disclosed which are not explicitly shown and explained in the figures, but which emerge and can be generated from the explained embodiments through separate combinations of features. Embodiments and combinations of features are also to be regarded as disclosed which therefore do not have all the features of an originally formulated independent claim. In addition, embodiments and combinations of features, in particular through the embodiments set out above, which go beyond or deviate from the combinations of features set out in the references to the claims are to be regarded as disclosed.

• 1 ein Prinzipblockdiagramm eines Fahrspurerfassungssystems;
• 2 ein schematisches Blockdiagramm des Verarbeitungsschemas eines Fahrspurmodellmoduls;
• 3 eine schematische Darstellung eines Eigenfahrspurmodells;
• 4 eine schematische Darstellung einer Eigen- und von zwei Nachbarfahrspuren und ihrer entsprechenden Begrenzungen;
• 5 eine schematische Darstellung der Fahrspurmarkierungsaufteilung pro Begrenzung;
• 6 ein schematisches Ablaufdiagramm einer Verfolgung während einer F ah rspurwechselsituation.

The present invention will now be described in more detail in conjunction with the accompanying figures, which show in:

• 1 a schematic block diagram of a lane detection system;
• 2 a schematic block diagram of the processing scheme of a lane model module;
• 3 a schematic representation of a private lane model;
• 4 a schematic representation of one of the lanes and two adjacent lanes and their corresponding boundaries;
• 5 a schematic representation of the lane marking distribution per boundary;
• 6 a schematic flow diagram of a pursuit during a lane change situation.

The following embodiments represent preferred examples of the present invention.

1 shows a lane detection system. A system input module 1 provides, for example, a front image, a left image, a right image, and a rear image. These images are sent to a lane marking detection module 2, which detects lane markings 23 from the images (also called individual images herein). A transformation module 3 transforms the image information of the lane marking detection module 2, for example, into world coordinates. Thereafter, a feature tracking module 4 tracks the features such as lane markings 23. From this analysis, the feature tracking module 4 can receive vehicle geometry data from the system input module 1.

The vehicle geometry information from the system input module 1 together with feature tracking information from the feature tracking module 4 is input to a lane position logic module 5, which may also be called a lane tracking module. Consequently, the lane position logic module 5 receives information about detected lane markings 23 (i.e., for example, bounding boxes and positions of interior features), preferably in vehicle coordinates. From this information, the lane position logic module 5 attempts to determine a lane 18 (cf. 3 ), which preferably consists of two parallel second-order polynomials as left and right boundaries 19, 20 (cf. 3 ) and the position and orientation of the own vehicle 17 (cf. 3 ) within this lane 18. Furthermore, the lane position logic module 5 may track the road geometry over time. The output of the lane position logic module 5 is passed to a lane boundary classification module 6, which classifies, for example, the type and color of the lane boundary. In a system output module 7, a lane type from the lane boundary classification module 6 and an estimated lane geometry for the current timestamp may be provided as results from the entire lane detection system.

2 shows a processing scheme of the lane position logic module or lane tracking module 5 of 1 . Data segments 8 from the feature tracking module 4 are input to a matching unit 9, which matches a predicted lane from a lane prediction unit 10 to the data segments 8 from the feature tracking module 4. Before inputting the data segments 8, they may be divided into candidates for the different lanes by a division module SM, as described with reference to 5 described below. The lane prediction unit 10 is based on a first lane model. It can receive odometry data from an odometry unit 11. The tuning unit 9 provides information to an updating unit 12 which updates the lane tracking or detection, i.e. it updates the first lane model.

In parallel, a computing unit 13 can calculate a best model from the measurements. In particular, it can calculate a compensation lane hypothesis for each timestamp from the incoming lane marking features 23, i.e. the data segments from the feature tracking module 4. To do this, the computing module 13 preferably uses a combination of RANSAC (random sample consensus) and least squares fitting. The result can be called a "static best model" (also referred to herein as a static second lane model).

A decision logic 14 can use confidence values for a given hypothesis to decide whether the updated or tracked first model from the update unit 12 or the static second lane model from the calculation unit 13 should be used to correct the lane tracking. For this correction, a correction unit 15 can receive the respective information from the decision unit 14. As a result, the lane model for the lane prediction unit 10 can be updated or the lane model remains unchanged with respect to the previous time step. It is preferable to stick to the result of the previous time step if both the static second lane model and the tracked first lane model provide too low confidence values. The current lane model finally provides output information 16, which can include classification information such as the lane type or the lane geometry.

The proposed method enables an extension of a lane detection system, as in 1 and 2 described, from the detection of the own lane 18 to a multi-lane system, whereby either the own and/or neighboring lanes can be detected. The proposed method is based on the assumption that the own lane 18 is modelled by a parameter model and using a Kalman filter, preferably in a system according to 1 and 2 , is being pursued.

3 shows a schematic representation of the described own lane model, wherein the own vehicle 17 is arranged on the modeled own lane 18. The own lane 18 can be modeled as a set of two coupled boundaries (left and right) 19 and 20, respectively, which only differ at their intersection point 19' and 20' respectively (see equation 1). Therefore, the width w of the lane 18 can be calculated as the difference at the intersection point of the left own boundary 19 and the right own boundary 20. According to the 3 In the rear axle coordinate system shown, the intersection point 19' of the left boundary 19 is positive, whereas the intersection point 20' of the right boundary 20 is negative.

The method described here to extend lane modeling and tracking to detect more than one lane is based on the assumption that the width w of the different lanes on the same road is similar. Combining this assumption with the assumption of the coupled lane model enables, as soon as the own lane 18 is detected and modelled, the adjacent left boundary 21 and the adjacent right boundary 22 ( 4 ) using equation 2.

4 shows the neighboring lanes 24, 25 and the own lane 18 and their corresponding boundaries 19, 20, 21, 22. Using the same approach as for the own lane 18, which is modeled by the left own boundary 19 and the right own boundary 20, the left neighboring lane 24 can be modeled as a coupled model consisting of the neighboring left boundary 21 and the left own boundary 19, while the right neighboring lane 25 can be modeled using the right own boundary 20 and the neighboring right boundary 22. Once the lane models (own, neighboring left and neighboring right) are estimated, they can be modeled and tracked independently.

5 shows a preferred embodiment in which the extracted lane markings 23 are divided into candidates for the different boundaries 19, 20, 21, 22. In the same way as in WO 2019/224103 A1 performed using the center of the left and right boundaries to divide the lane markings into left and right candidates, the center line 27 between the adjacent left boundary 21 and the left self-boundary 19 is used to divide the lane markings 23 into adjacent left and left candidates. Note that using the constant width estimation, the line dividing the adjacent left and left candidates can be estimated from the center line 26 of the own-left and own-right (see Equation 3). 5 shows the dividing line 26 to divide between left and right eigenboundaries 19, 20; 27 to divide between the adjacent left boundary 21 and the left eigenboundary 19; 28 to divide between the right eigenboundary 20 and the adjacent right boundary 22. 5 also shows on the right side the lane marking candidates for each boundary, where candidates in the area marked 21 are counted as candidates for the adjacent left boundary 21, candidates in the area marked 19 are counted as candidates for the left self-boundary 19, candidates in the area marked 20 are counted as candidates for the right self-boundary 20, and candidates in the area marked 22 are counted as candidates for the adjacent right boundary 22.

In a preferred embodiment, the lane detection system of 1 As a next step, the 3rd order polynomial model is used (see 2 ). For the WO 2019/224103 A1 In the system shown, two different models are built: the static model and the updated model. The static model is calculated from the instantaneous detections received as input to the lane marking detection module 2. Whereas the updated model is calculated using the knowledge of the lanes predicted by the Kalman filter. The following describes how the calculation of each of the models of WO 2019/224103 A1 should be modified to extend the system to include adjacent lane detection.

The static model for the own lane 18 is computed using RANSAC. From a list of features (sampled points of the incoming lane markings 23) for the left and right boundaries 19, 20, n features (6 by default) are randomly selected for each boundary 19, 20. With these features, a balancing candidate model or best fit candidate model is computed using a polynomial. These steps are repeated until either all features have been classified as inmates or the number of iterations reaches a certain value, which is configurable and 50 by default. If a valid model exists after this process, it is evaluated which lane markings 23 fit the model and the model is recalculated using these lane markings 23.

It must be noted that the features are points sampled from the lane markings 23. Therefore, it could happen that a feature of a lane marking 23 can fit a model, whereas the whole lane marking 23 does not fit. Using all lane markings 23 that fit the model, the final static eigenmodel is calculated.

Once the static eigenmodel is calculated, the adjacent left boundary 21 and the adjacent right boundary 22 can be estimated using Equation 2 using the method proposed here.

In a next step, it can be checked whether lane markings 23 are present that support the estimated neighboring lane boundaries 21, 22. For this purpose, for each of the lane marking candidates, the corresponding neighboring boundaries 21, 22 are checked, which correspond to the estimated neighboring boundaries 21, 22. This can be done by calculating the MSE (mean square error) between the points of the lane markings 23 and the respective boundary 21, 22. For the neighboring boundaries 21, 22, the MSE error is also valid, but the threshold used for the self-boundaries 19, 20 must be adjusted. On the one hand, the lane markings 23 associated with the neighboring boundaries 21, 22 have a higher reprojection error due to their higher distance from the camera. On the other hand, relaxing the MSE threshold allows the lane to have small changes in width. If there are lane markings 23 that match the neighboring left boundary 21, these lane markings 23 and the lane markings 23 that match the left self-boundary 19 are used to calculate the model of the left neighboring lane. If there are lane markings 23 that match the adjacent right boundary 22, these lane markings 23 and the lane markings 23 that match the right self-boundary 20 are also used to calculate the model of the right adjacent lane. Otherwise, the model of the left adjacent lane and/or the model of the right lane may be considered invalid.

The updated model for the own lane 18 is calculated based on the predicted model. For each of the left own boundary 19 or right own boundary 20, the system checks whether there are lane markings 23 that match the boundary 19, 20 using an MSE threshold. If there are lane markings 23 that support the boundaries 19, 20, the model is recalculated using these lane markings 23. The same procedure is proposed for calculating the updated model of the left neighbor lane 24 and the updated model of the right neighbor lane 25. As explained for the static model, the thresholds for the neighbor models must be updated.

• - die Breite der Modelle muss ähnlich sein;
• - die Fahrspuren 18, 24, 25 können nicht einander kreuzen;
• - die rechte Nachbarfahrspur 25 muss sich rechts von der Eigenfahrspur 18 befinden;
• - die linke Nachbarfahrspur 24 muss sich links von der Eigenfahrspur 18 befinden.

Once the static model and the updated model are built for each lane 18, 24, 25, the lane position logic module 5 checks which model is more appropriate for each lane 18, 24, 25. The selection of models can be independent for each of the lanes 18, 24, 25, but the selected models must be consistent among the lanes 18, 24, 25:

• - the width of the models must be similar;
• - lanes 18, 24, 25 cannot cross each other;
• - the right-hand neighbouring lane 25 must be located to the right of the own lane 18;
• - the left neighbouring lane 24 must be to the left of the own lane 18.

For tracking, three independent Kalman filters can be implemented to independently track the three lanes 18, 24, 25. The advantage of using three independent Kalman filters is that the system can cope with situations where the own lane 18 and one or more neighboring lanes 24, 25 are part of the same road, when initially the neighboring lane 24, 25 appears, but after a while the neighboring lane 24, 25 forms a separate road. For simplicity, the tracking model is not described here, details are in WO 2019/224103 A1 to find.

6 shows a schematic flow chart for tracking during a lane change situation. In a step S1, a lane change is detected. If a change to the right is detected in decision block S2, a new left neighboring lane 24' is considered to be equal to the former own lane 18 (S3). The lane that was tracked as the right neighboring lane 25 becomes the new own lane 18' (S4). Finally, the former right neighboring lane 25 is reset (S5).

In the example described above, a lane change to the right is detected. This means that the lane that was followed for a while as the right neighboring lane 25 becomes the new own lane 18' and the temporal consistency of the tracker and the confidence for the detection are much higher than if one would start to follow the neighboring lanes in the lane change case.

If no change to the right is detected in decision block S2, the system proceeds to decision block S6. If a change to the left is detected in decision block S6, the new right neighbor lane 25' is considered to be equal to the former own lane 18 (S7). The lane that was tracked as the left neighbor lane 24 becomes the new own lane 18' (S8). Finally, the former left neighbor lane 24 is reset (S9).

In other words, in case of a lane change, only the role of the respective lane is changed, whereas the system can keep tracking the lanes during the whole process. This is another advantage of using different tracking filters for the different lanes 18, 24, 25.

Overall, the described embodiments show how a method for lane detection can be extended to the detection of neighboring lanes.

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions.

Cited patent literature

• US 2010/0054538 A1 [0004]
• WO 2019/224103 A1 [0005, 0041, 0042, 0049]

Claims (11)

Method for detecting neighboring lanes (18, 24, 25), the method comprising the steps of a) providing individual images of the lanes (18, 24, 25), b) extracting lane features (23) from the individual images, the lane features (23) defining the course of the lane boundaries (19, 20, 21, 22) of the lanes (18, 24, 25), c) providing a first lane model for modeling the course of the lane boundaries (19, 20, 21, 22) on the basis of the extracted lane features (23), a respective course being described by its coordinates in a common coordinate system, characterized by d) calculating the width (w) of a first of the lanes (18) from the coordinates, e) assigning the width (w) to at least one of the neighboring lanes (24, 25), f) estimating the course of the lane boundaries (19, 20, 21, 22) of at least one adjacent lane (24, 25) based on the assigned width (w). Procedure according to Claim 1 , wherein the first of the lanes (18) and the at least one of the adjacent lanes (24, 25) share a common lane boundary, wherein step f) includes adding the assigned width (w) to the coordinates of the common lane boundary and/or subtracting the assigned width (w) from the coordinates of the common lane boundary. Method according to one of the preceding claims with the steps - updating the first lane model with the extracted lane features (23), - providing a static second lane model based on an outlier detection algorithm, - calculating a tracked confidence value for the updated first lane model and a static confidence value for the static second lane model, and - deciding on the basis of both confidence values whether the updated first lane model or the static second lane model should be used to calculate current information about the lanes (18, 24, 25). Procedure according to Claim 3 , wherein the updating of the first lane model is performed by predicting a new version of the first lane model using Kalman filtering and adapting the new version of the first lane model to the extracted lane features (23). Procedure according to Claim 4 , where an independent Kalman filter is implemented for each of the neighboring lanes (18, 24, 25). Method according to one of the Claims 3 until 5 , where the first lane model and the second lane model track a lane geometry over time. A method according to any preceding claim, comprising - calculating a lane (18, 24, 25) dividing line (26, 27, 28) for each of the lanes (18, 24, 25), - using the dividing line (26, 27, 28) of a respective lane (18, 24, 25) to divide the extracted lane features (23) into different sets of lane features (23), wherein the lane features (23) of each of the sets of lane features (23) are assigned to one of the lane boundaries (19, 20, 21, 22) of the respective lane (18, 24, 25), - determining the proximity of each of the lane features (23) to its assigned lane boundary (19, 20, 21, 22), - evaluating whether a respective lane feature (23) exceeds the estimated lane boundary (19, 20, 21, 22) according to a predetermined proximity-based evaluation algorithm, and - updating the first lane model based on the supporting lane features (23) that support their respective lane boundary (19, 20, 21, 22). Procedure according to Claim 7 , wherein the dividing line (26, 27, 28) of a respective lane (18, 24, 25) is calculated by adding the assigned width (w) to the coordinates of the dividing line (26, 27, 28) of its neighboring lane (18, 24, 25) and/or by subtracting the assigned width (w) from the coordinates of the dividing line (26, 27, 28) of its neighboring lane (18, 24, 25). Driver assistance system designed to carry out a method according to one of the preceding claims. Device which is adapted to carry out a method according to one of the Claims 1 until 8th to carry out. Computer program product with program code means, in particular stored in a computer-readable medium, for carrying out the method according to one of the Claims 1 until 8th to be carried out if the computer program pro product is processed on a computer device of an electronic control unit.

Images