DE102023123801A1 - Method and system for adjusting a vehicle's environmental perception - Google Patents

Abstract

The present invention relates to a method for adjusting an environment perception of an ego vehicle, in which the ego vehicle comprises means for V2X communication, in which the ego vehicle can be controlled or is controlled by an ADAS and/or an ADS, in which the ego vehicle has a plurality of sensors observing an environment and a storage medium, in which data (11) measured by the sensors is interpreted by the ADAS/ADS to carry out autonomous driving processes, in which the following steps are carried out: storing at least one driving scenario for which control of the ego vehicle can be carried out or is carried out by the ADAS/ADS; storing data from the sensors measured during a journey in the storage medium; storing V2X data (12) transmitted during the journey in the storage medium; identifying (13) the at least one driving scenario in the measured data during the journey; Comparison (14) of the sensor data, which originate from sensors specified according to the at least one driving scenario, with V2X data transmitted during the identified driving scenario, which are considered to be true data; use of a deviation determined during the comparison for adjustment (16) of the respective sensors and/or adjustment of the interpretation of the measured data by the ADAS/ADS. Furthermore, a system (10) for adjusting a vehicle's perception of the environment is presented.

Description

The present invention relates to a method for adjusting an environment perception for ADAS/ADS of a vehicle. Furthermore, a system for adjusting the environment perception in ADAS/ADS is presented.

In "vehicle-to-X" communication, which is carried out by an ego vehicle with an external communication participant, standardized, wireless interfaces with technical systems in the vehicle's environment are used to exchange information. This applies, for example, to communication between vehicles, referred to as "vehicle-to-vehicle" communication, in which vehicles report their braking decelerations to each other, for example, in order to prevent rear-end collisions, or report a planned lane change to other vehicles so that they can merge. But communication with installed infrastructure is also possible; for example, traffic lights can report the duration until the next green phase to the vehicle, or surveillance cameras installed at intersections can report detected objects to vehicles for trajectory planning.

Irrespective of this, the ego vehicle requires its own sensors, which provide measurement data for assessing a particular driving situation. On the other hand, the "vehicle-to-X" communication in vehicles surrounding the ego vehicle records information that relates to the assessment of a particular driving situation. It would therefore be particularly helpful for an ego vehicle that is at least partially controlled by a driver assistance system (ADAS) and/or an automated driving system (ADS) to process such information.

The publication DE 10 2018 115 198 A1 discloses a virtual crowdsourcing sensor generator, whereby sensor data from several different motor vehicles are processed and sensor data for a target vehicle is generated.

In the printed publication US 11,198,443 B2 A V2X communication for the exchange of sensor information from different motor vehicles is described.

The publication DE 10 2020 205 988 A1 discusses a method for coordinating a maneuver of a motor vehicle using V2X communication.

Against this background, it is an object of the present invention to present a method for comparing data transmitted by communication with a road user surrounding the ego vehicle and data recorded by sensors of the vehicle, in particular when the self-recorded data is used by an ADAS/ADS. Furthermore, a system is to be presented on which the method is carried out.

• • Hinterlegen mindestens eines Fahrszenarios, für welches mit dem ADAS/ADS eine Steuerung des Ego-Fahrzeugs ausführbar ist oder ausgeführt wird;
• • Hinterlegen von während einer Fahrt gemessenen Daten der Sensoren in dem Speichermedium;
• • Hinterlegen von während der Fahrt übermittelten V2X-Daten in dem Speichermedium;
• • Identifizieren des mindestens einen Fahrszenarios in den gemessenen Daten während der Fahrt;
• • Abgleich der Sensordaten, welche aus gemäß dem mindestens einen Fahrszenario vorgegebenen Sensoren stammen, mit während dem identifizierten Fahrszenario übermittelten V2X-Daten, wobei die V2X-Daten als wahre - also fehlerfreie - Daten betrachtet werden,
• • Verwendung einer bei dem Abgleich festgestellten Abweichung zur Adjustierung der jeweiligen Sensoren und/oder Adjustierung der Interpretation der gemessenen Daten durch das ADAS/ADS.

To solve the above-mentioned problem, a method for adjusting an environment perception of an ego vehicle is proposed, in which the ego vehicle comprises means for V2X communication, referred to as V2X with vehicle-to-X, where V is the ego vehicle in the V2X communication and X is a target vehicle or an object of a traffic infrastructure which has means for V2X communication. The ego vehicle can be controlled by an ADAS, referred to as ADAS with a driver assistance system, and/or an ADS, referred to as ADS with an automated driving system, or is controlled by them. The ego vehicle has a large number of sensors that monitor an environment and a storage medium. The ADAS/ADS interprets data measured by the sensors to carry out autonomous driving processes. The following steps are carried out:

• • Storing at least one driving scenario for which control of the ego vehicle can be carried out or is carried out using the ADAS/ADS;
• • Storing data measured by the sensors during a journey in the storage medium;
• • Storing V2X data transmitted during the journey in the storage medium;
• • Identifying at least one driving scenario in the measured data during the journey;
• • Comparison of the sensor data, which originate from sensors specified in accordance with at least one driving scenario, with V2X data transmitted during the identified driving scenario, whereby the V2X data are considered as true - i.e. error-free - data,
• • Use of a deviation identified during the comparison to adjust the respective sensors and/or adjust the interpretation of the measured data by the ADAS/ADS.

The method according to the invention thus advantageously uses both data from the vehicle's own sensors and data from V2X communication to identify driving scenarios relevant for autonomous control by ADAS/ADS. Within these driving scenarios, selected data from the vehicle's own sensors are compared with the data from the V2X communication.

In one embodiment of the method according to the invention, at least one sensor observing the environment is selected from the following list: radar, lidar, camera.

In one embodiment, an automated driving system designed as "Adaptive Cruise Control", abbreviated to ACC, detects a "braking target vehicle" driving scenario in which a vehicle immediately ahead reduces its speed. The speed of the target vehicle is determined by merging radar and camera data. At the same time, the target vehicle reports its real speed via V2V communication to the ego vehicle, whose ADAS/ADS now compares it with the speed of the target vehicle previously determined by itself. For certain data, such as the transmitted speed of the target vehicle, a several times higher accuracy can be assumed, e.g. 1% instead of 10%. A deviation between the two speeds can therefore, for example, lead to an uncertainty in the speed determination based on the vehicle's own sensors, which means that if the uncertainty value is increased, the ego vehicle must brake earlier than was previously the case.

Another example is the identification of a driving scenario for a turning intention and lateral acceleration of the target vehicle, which can be compared via a reported steering angle and a measured lateral acceleration of the target vehicle.

Another example is a sensor-based perception of the respective position of other road users, which is compared with the positions of an infrastructure camera at intersections, for example, reported back via V2X communication. The respective relative positions are determined by offsetting them against the position of the ego vehicle perceived by the infrastructure camera.

It is also conceivable to align a sensor-based perception of speed limits or a traffic light display with the respective information transmitted by V2X information with the respective infrastructure, such as traffic lights or intelligent guide posts.

Yet another example is a sensor-based perception of parking spaces and lane boundaries in a parking garage, which can be compared with information from a perception sensor system in the parking garage.

In a further embodiment of the method according to the invention, a field of view of a sensor monitoring the environment is changed based on the detected deviation and/or the ADAS/ADS is re-parameterized and/or applied and/or a performance report is created for the ADAS/ADS. The performance report or efficiency report of driver assistance systems can also include an overview of the behavior of the vehicle's own sensors in various driving scenarios.

In yet another embodiment of the method according to the invention, the detected deviation is used to expand training data for machine learning methods - which also includes AI-based perception software in the field of artificial intelligence.

In a further embodiment of the method according to the invention, sensor data and V2X data stored for a period of time of the identified driving scenario are transmitted to a server outside the vehicle. Since only data on already identified driving scenarios are transmitted, the amount of data sent can advantageously be kept low in comparison to the total amount of recordable data.

• • Hinterlegen mindestens eines Fahrszenarios, für welche durch das ADAS/ADS eine Steuerung des Ego-Fahrzeugs ausführbar ist oder ausgeführt wird;
• • Hinterlegen von während einer Fahrt gemessenen Daten der Sensoren in dem Speichermedium;
• • Hinterlegen von während der Fahrt übermittelten V2X-Daten in dem Speichermedium;
• • Identifizieren des mindestens einen Fahrszenarios in den gemessenen Daten während der Fahrt;
• • Abgleich der Sensordaten, welche aus gemäß dem mindestens einen Fahrszenario vorgegebenen Sensoren stammen, mit während dem identifizierten Fahrszenario übermittelten V2X-Daten, wobei die V2X-Daten wahre Daten darstellen;
• • Verwendung einer bei dem Abgleich festgestellten Abweichung zur Adjustierung der jeweiligen Sensoren und/oder Adjustierung der Interpretation der gemessenen Daten durch das ADAS/ADS.

Furthermore, a system for adjusting an environment perception of an ego vehicle is claimed, wherein the system comprises an ego vehicle with means for V2X communication, with vehicle-to-X referred to as V2X and V as ego vehicle and X as a target vehicle or an object of a traffic infrastructure, an ADAS, which refers to a driver assistance system, and/or an ADS, which refers to an automated driving system. The ego vehicle can be controlled by the ADAS/ADS. The ego vehicle has a plurality of sensors observing an environment and a storage medium. The ADAS/ADS is configured to interpret data measured by the sensors to carry out autonomous driving processes. The system is designed to carry out the following steps:

• • Storing at least one driving scenario for which control of the ego vehicle can be or is carried out by the ADAS/ADS;
• • Storing data measured by the sensors during a journey in the storage medium;
• • Storing V2X data transmitted during the journey in the storage medium;
• • Identifying at least one driving scenario in the measured data during the journey;
• • comparing the sensor data originating from sensors specified according to the at least one driving scenario with V2X data transmitted during the identified driving scenario, wherein the V2X data represents true data;
• • Use of a deviation identified during the comparison to adjust the respective sensors and/or adjust the interpretation of the measured data by the ADAS/ADS.

In one embodiment of the system according to the invention, at least one sensor observing the environment is selected from the following list: radar, lidar, camera.

In a further embodiment of the system according to the invention, the system is designed to change a field of view of a sensor observing the environment based on the detected deviation and/or to re-parameterize and/or apply the ADAS/ADS based on the detected deviation and/or to create a performance report for the ADAS/ADS.

In yet another embodiment of the system according to the invention, the system is designed to provide extended training data for machine learning methods by means of the detected deviation.

In a further embodiment of the system according to the invention, the system is designed to transmit sensor data and V2X data stored for a period of the identified driving scenario to a server outside the vehicle.

Since validation of vehicle-specific sensors and perception software implemented in the ego vehicle represents a major challenge when releasing the systems due to functional inadequacies of ADAS/ADS, the system according to the invention can support a validation process and make it more efficient.

Further advantages and embodiments of the invention will become apparent from the description and the accompanying drawings.

It is understood that the features mentioned above and those to be explained below can be used not only in the combination specified in each case, but also in other combinations or on their own, without departing from the scope of the present invention.

1 shows schematically an embodiment of the system according to the invention.

In 1 shows a schematic design of the system 10 according to the invention for adjusting the environment perception of an ego vehicle. In measured data 11 from sensors of the ego vehicle that observe the environment, a scenario identification 13 is carried out within the ADAS/ADS and a deviation is determined in a comparison 14 using V2X data 12 received during an identified driving scenario from a vehicle-to-X communication. The deviation determined in the comparison 14 is used, on the one hand, to create a performance report 15 for the ADAS/ADS and to provide an additional training data set for parameterization or application of the ADAS/ADS, and on the other hand, to adjust the sensors of the ego vehicle 16 and/or to adjust their field of view.

list of reference symbols

10

System scheme for adjusting the perception of the environment

11

Data from ego vehicle with sensors for environmental monitoring

12

data from V2X communication

13

scenario identification

14

comparison of the data

15

Performance report, set from new training data

16

sensor adjustment, field of view change

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions.

Cited patent literature

• DE 10 2018 115 198 A1 [0004]
• US 11,198,443 B2 [0005]
• DE 10 2020 205 988 A1 [0006]

Claims (10)

Method for adjusting an environment perception of an ego vehicle, in which the ego vehicle comprises means for V2X communication, referred to as V2X with vehicle-to-X, wherein in the V2X communication V is the ego vehicle and X is a target vehicle or an object of a traffic infrastructure, which has means for V2X communication, in which the ego vehicle can be controlled or is controlled by an ADAS, referred to as ADAS with a driver assistance system, and/or an ADS, referred to as ADS with an automated driving system, in which the ego vehicle has a plurality of sensors observing an environment and a storage medium, in which data (11) measured by the sensors is interpreted by the ADAS/ADS to carry out autonomous driving processes, in which the following steps are carried out: • Storing at least one driving scenario for which control of the ego vehicle can be carried out or is carried out by the ADAS/ADS; • Storing data (11) of the sensors measured during a journey in the storage medium; • Storing V2X data (12) transmitted during the journey in the storage medium; • Identifying (13) the at least one driving scenario in the measured data (11) during the journey; • Comparing (14) the sensor data, which originate from sensors specified according to the at least one driving scenario, with V2X data (12) transmitted during the identified driving scenario, wherein the V2X data (12) are considered to be true data; • Using a deviation determined during the comparison to adjust (16) the respective sensors and/or adjust the interpretation of the measured data by the ADAS/ADS. procedure according to claim 1 , in which at least one sensor observing the environment is selected from the following list: radar, lidar, camera. Method according to one of the preceding claims, in which, due to the detected deviation, a field of view of a sensor observing the environment is changed and/or the ADAS/ADS is newly parameterized and/or applied and/or a performance report (15) is created for the ADAS/ADS. Method according to one of the preceding claims, in which training data for machine learning methods are expanded by means of the determined deviation. Method according to one of the preceding claims, in which data (11) and V2X data (12) stored for a period of the identified driving scenario are transmitted to a server outside the vehicle. System for adjusting an ego vehicle's perception of the environment, the system (10) comprising an ego vehicle with means for V2X communication, with Vehicle-to-X referred to as V2X and V as ego vehicle and X as a target vehicle or an object of a traffic infrastructure, an ADAS, which refers to a driver assistance system, and/or an ADS, which refers to an automated driving system, the ego vehicle being controllable by the ADAS/ADS, the ego vehicle having a plurality of sensors observing an environment and a storage medium, the ADAS/ADS being configured to interpret data (11) measured by the sensors to carry out autonomous driving processes, the system (10) being designed to carry out the following steps: • Storing at least one driving scenario for which control of the ego vehicle can be carried out or is carried out by the ADAS/ADS; • Storing data measured by the sensors during a journey in the storage medium; • Storing V2X data (12) transmitted during the journey in the storage medium; • Identifying (13) the at least one driving scenario in the measured data (11) during the journey; • Comparing (14) the sensor data, which originate from sensors specified according to the at least one driving scenario, with V2X data (12) transmitted during the identified driving scenario, wherein the V2X data (12) represent true data; • Using a deviation determined during the comparison to adjust (16) the respective sensors and/or adjust the interpretation of the measured data by the ADAS/ADS. system according to claim 6 , where at least one sensor monitoring the environment is selected from the following list: radar, lidar, camera. System according to one of the Claims 6 or 7 , wherein the system is designed to change a field of view of a sensor observing the environment based on the detected deviation and/or to re-parameterize and/or apply the ADAS/ADS based on the detected deviation and/or to create a performance report (15) for the ADAS/ADS. System according to one of the Claims 6 until 8 , wherein the system is designed to provide extended training data for machine learning methods by means of the detected deviation. System according to one of the Claims 6 until 9 , wherein the system is designed to transmit stored data (11) and V2X data (12) to a server outside the vehicle for a period of time of the identified driving scenario.

Images