DE102023119911B4 - Method and system for the automated validation of functions of a driver assistance system - Google Patents

Abstract

Method for the automated validation of functions of a driver assistance system (ADAS system); comprising:
- recording (S10) first sensor data (270) by at least one sensor such as a camera, a lidar system and/or a radar system of a first sensor device (250) of a test vehicle (200) when driving along a route for testing at least one function of an ADAS system (230), wherein the camera of the first sensor device (250) is a digital camera with full HD resolution;
- processing (S20) the first sensor data (270) from the ADAS system (230) and generating test data (280) using the processed first sensor data (270);
- transmitting (S30) the test data (280) to an evaluation module (500) and/or to a first database (290);
- recording (S40) second sensor data (270) by at least one sensor such as a camera, a lidar system and/or a radar system of a second sensor device (350) of a ground truth module (300) when driving along a route, wherein the camera of the second sensor device (350) is a digital camera with 4K resolution, wherein the sensors of the second sensor device (350) have a higher performance in terms of resolution, frame rate, range and computing resources than the sensors of the first sensor device (250);
- processing (S50) the second sensor data (370) by a software application (330) and generating ground truth data (380) using the processed second sensor data (370);
- transmitting (S60) the ground truth data (380) to the evaluation module (500) and/or to a second database (390);
- comparing (S70) the test data (280) with the ground truth data (380) by a software application (550) comprising comparison algorithms, wherein the comparison algorithms comprise convolved neural networks (Convolutional Neural Network, CNN) and/or histogram-based algorithms;
- generating (S80) evaluation results (580) for validating at least one function of the ADAS system (230).

Description

The invention relates to a method, a system and a computer program product for the automated validation of functions of a driver assistance system.

Modern Advanced Driver Assistance Systems (ADAS) include a range of technologies to improve vehicle safety and driving comfort. ADAS systems use various sensors, cameras and algorithms to analyze the vehicle environment and assist the driver in real time. To this end, the ADAS system includes a variety of functions that are constantly being developed to further improve vehicle safety and driving comfort.

A typical feature is adaptive cruise control (ACC), which uses radar or laser sensors to maintain a safe distance from the vehicle in front. It automatically adjusts the vehicle's speed to the flow of traffic so that the driver does not have to constantly adjust his speed. Another feature is lane departure warning (LDW), which uses cameras or sensors to detect unintentional lane departure. The driver is warned by visual, acoustic or haptic signals and prompted to correct the lane. Another feature is automatic emergency braking (AEB), which automatically applies the brakes to avoid a collision with a vehicle or pedestrian.

Another important function is traffic sign recognition (TSR), which uses cameras and image recognition algorithms to detect and interpret traffic signs such as speed limits, stop signs and information signs. The relevant information can be displayed in the vehicle cockpit or on the head-up display, reminding the driver of important traffic rules. Traffic sign recognition can also activate other functions of an ADAS system. For example, when an ADAS-equipped vehicle approaches a stop sign, the traffic sign algorithm detects the sign and transmits the relevant information to the vehicle's control system. The control system can then use this information to automatically apply the brakes and bring the vehicle to a stop. Similarly, when a speed limit sign is detected, the ADAS system can adjust the vehicle's speed to ensure that the posted speed limit is observed.

1. a) Vorverarbeitung: Das Bild- oder Videobild wird zunächst vorverarbeitet, um die Bildqualität zu verbessern, Rauschen zu reduzieren sowie Kontrast und Helligkeit des Bildes zu erhöhen.
2. b) Merkmalsextraktion: Der Algorithmus extrahiert relevante Merkmale aus dem Bild, z.B. Farbe, Form und Textur, die zur Unterscheidung verschiedener Verkehrszeichentypen verwendet werden können.
3. c) Klassifikation: Die extrahierten Merkmale werden einem maschinellen Lernalgorithmus zugeführt, z.B. einem neuronalen Netz, das mit einem Datensatz von Verkehrszeichenbildern trainiert wird, um die Verkehrszeichen im Eingabebild zu erkennen und zu klassifizieren.
4. d) Nachbearbeitung: Nach der Erkennung und Klassifizierung der Verkehrszeichen können Nachbearbeitungstechniken eingesetzt werden, um falsch positive Ergebnisse zu entfernen, die Verkehrszeichen über die Zeit zu verfolgen und die Erkennungsgenauigkeit zu verbessern.

The traffic sign algorithm is a machine vision algorithm for detecting and classifying traffic signs in images or video recordings. The algorithm uses a combination of image processing techniques and machine learning algorithms to detect the shape, color and content of traffic signs. The algorithm essentially consists of the following procedural steps:

1. a) Preprocessing: The image or video image is first preprocessed to improve the image quality, reduce noise and increase the contrast and brightness of the image.
2. b) Feature extraction: The algorithm extracts relevant features from the image, e.g. color, shape and texture, which can be used to distinguish different traffic sign types.
3. c) Classification: The extracted features are fed to a machine learning algorithm, e.g. a neural network, which is trained with a dataset of traffic sign images to detect and classify the traffic signs in the input image.
4. d) Post-processing: After traffic signs are detected and classified, post-processing techniques can be used to remove false positives, track traffic signs over time, and improve detection accuracy.

The traffic sign algorithm is therefore an essential component of a driver assistance system, as it is able to recognize a wide range of traffic signs and trigger a response from the driver assistance system, especially warning, regulatory and information signs. The algorithm must be accurate and robust and work under different lighting and weather conditions to ensure the correct functioning of the driver assistance system. A well-functioning traffic sign algorithm is therefore crucial for improving the safety and efficiency of vehicles on the road.

Validation is carried out to check the performance, functionality and reliability of an ADAS system and/or individual functions of an ADAS system. Validation of an ADAS system involves a complex procedure for testing and verifying its performance, functionality and safety. The aim is to ensure that the ADAS system functions as intended, meets the legal requirements and reliably supports the driver. Each individual system component, e.g. adaptive cruise control, lane departure warning or collision warning, is tested both individually and in combination. The functional tests check whether the ADAS system is working correctly, giving the driver precise feedback and reacting appropriately to different scenarios. An important aspect is the calibration of the sensors to check their accuracy and alignment and to ensure that they provide consistent and reliable data.

To validate the various functions of an ADAS system, such as the traffic sign algorithm, test drives are performed regularly. Different test cases are used to ensure that the ADAS system works correctly under different conditions. These test cases can be simulated or performed in real environments.

In the example of the traffic sign algorithm, a test driver evaluates it from the customer's perspective. Based on the test driver's evaluation, a statistical evaluation is carried out that allows statements to be made about the performance and the type of errors. For example, it is checked whether a displayed traffic sign, which was recognized via the vehicle camera or retrieved from map data, is actually valid. This data evaluation is done manually and is therefore time- and labor-intensive. This is due to both the amount of data and the data complexity, as ADAS systems generate a large amount of data during the test drive, including sensor readings, camera images, vehicle parameters and driver inputs, and also record complex and multi-dimensional data, including readings from various imaging sensors such as radar, lidar and cameras, which differ in their resolution and recording quality. In addition, human errors can occur during manual evaluation by overlooking critical details or misinterpreting information, leading to inaccuracies and inconsistencies in the evaluation process.

The US 2022/90995 A1 discloses a system for validating road object data using ground truth data, wherein the ground truth data is determined from information from sensor systems of different vehicles.

The US 2022/048533 A1 discloses a method and system for validating automatic control software for an autonomously driving vehicle, wherein the validation is performed using ground truth data provided in real time by other vehicles participating in traffic.

The DE 10 2020 200 650 A1 discloses a method for operating an automated lane guidance system of a motor vehicle for guiding the motor vehicle in a lane, wherein the automated lane guidance takes into account image data which is determined by means of a first sensor system. An evaluation of an execution of the automated lane guidance takes place by means of a second sensor system, wherein it is assessed on the basis of the determined sensor data of the second sensor system whether the lane guidance is carried out correctly.

The DE 10 2021 124 913 A1 discloses a method for evaluating the performance of subsystems of a system, wherein the system is an autonomous vehicle (AV) software stack. The method includes perturbing a first attribute of a first subsystem and determining a change in a second attribute of a second subsystem of the system, the second attribute responsive to the perturbation of the first attribute and at least an output of the first subsystem being transmitted to the second subsystem, and determining a value for a performance metric of the system based on a correlation between the first and second attributes, the performance metric being an objective function output that measures a quality of drivability of the AV.

The DE 10 2019 115 809 A1 discloses a method for controlling an autonomous vehicle. A preprocessor processes image data acquired by sensors at a current time step to generate preprocessed image data that is concatenated with additional inputs to produce a dynamic scene output. A convolutional neural network (CNN) processes the dynamic scene output to produce a spatial context feature vector that is used for computing control commands (e.g., steering angle, acceleration rate, and/or braking rate control command).

The invention is based on the object of showing possibilities how the validation of an ADAS system can be carried out automatically in order to significantly reduce the time required for manual evaluation and to increase the reliability and accuracy of the validation and thus of the ADAS system.

This object is achieved with respect to a method by the features of patent claim 1, with respect to a system by the features of patent claim 4, with respect to an ADAS system by the features of patent claim 5, and with regard to a computer program product by the features of patent claim 6. The further claims relate to preferred embodiments of the invention.

The present invention enables accurate and reliable validation of a modern driver assistance system (ADAS) by automatically comparing ground truth data with test data of an ADAS system using suitable comparison algorithms. Accurate validation ensures that the ADAS system functions as intended, thereby increasing overall road safety. In addition, the performance of the ADAS system can be evaluated. By comparing the test data with the ground truth data, accuracy, reliability and other performance indicators can be measured. This evaluation helps to identify opportunities for improvement and fine-tune the ADAS system for optimal performance.

• - Aufnehmen von ersten Sensordaten durch mindestens einen Sensor wie eine Kamera, ein Lidarsystem und/oder ein Radarsystem einer ersten Sensoreinrichtung eines Erprobungsfahrzeugs beim Befahren einer Fahrstrecke zum Testen zumindest einer Funktion eines ADAS-Systems, wobei die Kamera der ersten Sensoreinrichtung eine Digitalkamera mit Full-HD-Auflösung ist;
• - Bearbeiten der ersten Sensordaten von dem ADAS-System und Erzeugen von Testdaten mittels der bearbeiteten ersten Sensordaten;
• - Übertragen der Testdaten an ein Auswertemodul und/oder an eine erste Datenbank;
• - Aufnehmen von zweiten Sensordaten durch mindestens einen Sensor wie eine Kamera, ein Lidarsystem und/oder ein Radarsystem einer zweiten Sensoreinrichtung eines Ground-Truth-Moduls beim Befahren einer Fahrstrecke, wobei die Kamera der zweiten Sensoreinrichtung eine Digitalkamera mit 4K-Auflösung ist, wobei die Sensoren der zweiten Sensoreinrichtung über eine höhere Leistungsfähigkeit hinsichtlich Auflösung, Bildfrequenz, Reichweite, Rechenressourcen als die Sensoren der ersten Sensoreinrichtung verfügen;
• - Bearbeiten der zweiten Sensordaten von einer Softwareapplikation und Erzeugen von Ground-Truth-Daten mittels der bearbeiteten zweiten Sensordaten;
• - Übertragen der Ground-Truth-Daten an das Auswertemodul und/oder an eine zweite Datenbank;
• - Vergleichen der Testdaten mit den Ground-Truth-Daten durch eine Softwareapplikation, wobei die Softwareapplikation Vergleichsalgorithmen umfasst, wobei die Vergleichsalgorithmen gefaltete neuronale Netze (Convolutional Neural Network, CNN) und/oder histogrammbasierte Algorithmen verwenden;
• - Erzeugen von Auswerteergebnissen zur Validierung der zumindest einen Funktion des ADAS-Systems.

According to a first aspect, the invention provides a method for the automated validation of functions of a driver assistance system (ADAS system). The method comprises the following method steps:

• - Recording first sensor data by at least one sensor such as a camera, a lidar system and/or a radar system of a first sensor device of a test vehicle when driving along a route for testing at least one function of an ADAS system, wherein the camera of the first sensor device is a digital camera with full HD resolution;
• - Processing the first sensor data from the ADAS system and generating test data using the processed first sensor data;
• - Transferring the test data to an evaluation module and/or to a first database;
• - Recording second sensor data by at least one sensor such as a camera, a lidar system and/or a radar system of a second sensor device of a ground truth module when driving along a route, wherein the camera of the second sensor device is a digital camera with 4K resolution, wherein the sensors of the second sensor device have a higher performance in terms of resolution, frame rate, range, computing resources than the sensors of the first sensor device;
• - processing the second sensor data by a software application and generating ground truth data using the processed second sensor data;
• - Transferring the ground truth data to the evaluation module and/or to a second database;
• - comparing the test data with the ground truth data by a software application, wherein the software application comprises comparison algorithms, wherein the comparison algorithms use convolutional neural networks (CNN) and/or histogram-based algorithms;
• - Generating evaluation results to validate at least one function of the ADAS system.

In a further development, one function of the ADAS system is a traffic sign algorithm.

Advantageously, the first database and/or the second database and/or the evaluation module is/are integrated in a cloud computing infrastructure.

According to a second aspect, the invention provides a system for the automated validation of functions of a driver assistance system (ADAS system).

The system comprises a test vehicle with an ADAS system and a first sensor device with at least one sensor such as a camera, a lidar system and/or a radar system, the camera of the first sensor device being a digital camera with full HD resolution, a ground truth module with a software application and a second sensor device and an evaluation module with a software application. The first sensor device is designed to record first sensor data when the test vehicle travels a route to test at least one function of an ADAS system. The ADAS system is designed to process the first sensor data to generate test data and to transmit the test data to the evaluation module. The second sensor device is designed to record second sensor data by at least one sensor such as a camera, a lidar system and/or a radar system when traveling a route, the camera of the second sensor device being a digital camera with 4K resolution, and the sensors of the second sensor device having a higher performance in terms of resolution, frame rate, range and computing resources than the sensors of the first sensor device. The software application is designed to process the second sensor data to generate ground truth data and to transmit the ground truth data to the evaluation module. The software application includes comparison algorithms and is designed to compare the test data with the ground truth data and to generate evaluation results for validating the at least one function of the ADAS system, and wherein the comparison algorithms use convolutional neural networks (CNN) and/or histogram-based algorithms.

According to a third aspect, the invention provides an ADAS system for a motor vehicle which uses the method according to the first aspect for recognizing traffic signs having a main traffic sign and an additional sign.

According to a fourth aspect, the invention provides a computer program product comprising an executable program code configured to carry out the method according to the first aspect when executed.

The invention is explained in more detail below with reference to embodiments shown in the drawing.

• 1 ein Blockdiagramm zur Erläuterung eines Ausführungsbeispiels eines erfindungsgemäßen Systems;
• 2 ein Flussdiagramm zur Erläuterung der einzelnen Verfahrensschritte eines erfindungsgemäßen Verfahrens;
• 3 ein Blockdiagramm eines Computerprogrammprodukt gemäß einer Ausführungsform des dritten Aspekts der Erfindung.

It shows:

• 1 a block diagram for explaining an embodiment of a system according to the invention;
• 2 a flow chart explaining the individual method steps of a method according to the invention;
• 3 a block diagram of a computer program product according to an embodiment of the third aspect of the invention.

Further features, aspects and advantages of the invention or its embodiments are explained in the following description in conjunction with the patent claims.

1 shows a system 100 according to the invention for the automated validation of a driver assistance system (ADAS system). The system 100 according to the invention comprises a test vehicle 200, a first database 290, a ground truth module 300, a second database 390, an evaluation module 500 and an output module 700. The test vehicle 100 comprises an ADAS system 230 with at least one driving function such as a traffic sign algorithm and a first sensor device 250. The ground truth module 300 comprises a high-resolution second sensor device 350 for creating ground truth data 380.

In the context of the invention, the term "ground truth" refers to the objectively known and correct values or images associated with a specific data set. The ground truth data 380 represents accurate and reliable information with which the results of the ADAS system 230 are compared and evaluated when the test vehicle 100 is driven on a test track.

The ground truth data 380 serves as a reference or benchmark to evaluate the performance and accuracy of an ADAS system under test (SUT). For a traffic sign algorithm that must perform various image classification tasks, the ground truth data 380 provides the correct class or category for each image in a dataset and thus also the prediction of the correct traffic sign. The detection of traffic signs by the ADAS system 230 while driving on a test track can be compared to the ground truth data 380 to measure the accuracy and precision of the traffic sign detection or other performance indicators of the traffic sign algorithm.

In the context of the present invention, the ground truth data 380 are generated in particular by the high-resolution second sensor device 350. However, it is also possible for the ground truth data 380 to be obtained from other relevant sources.

The ADAS system 230, the first sensor device 250, the first database 290, the ground truth module 300, the second sensor device 300, the second database 390, the evaluation module 500 and the output module 700 can each be equipped with a processor and/or a memory unit. However, they can also be integrated in a common computing unit, in particular in the test vehicle 200.

In the context of the invention, a "module" can be understood as, for example, a processor and/or a processor unit and/or a memory unit for storing program instructions. The processor is specially designed to execute the program instructions in order to carry out or implement the method according to the invention or a step of the method according to the invention. In particular, a module can be integrated into a cloud computing infrastructure. However, it can also be provided that a computer unit, such as a processor, comprises several modules.

In the context of the invention, a “processor” can be understood as meaning, for example, a machine or an electronic circuit. A processor can in particular be a central processing unit (CPU), a microprocessor or a microcontroller. , e.g. an application-specific integrated circuit or a digital signal processor, possibly in combination with a memory unit for storing program instructions, etc. A processor can also be a virtualized processor, a virtual machine or a soft CPU.

In the context of the invention, a "memory unit" or a "memory module" can be understood as, for example, a volatile memory in the form of a random access memory (RAM) or a permanent memory such as a hard disk or a data carrier or, for example, an exchangeable memory module. The memory module can also be designed as a cloud storage solution.

In the context of the invention, “data” means both raw data and already processed data from measurement results from sensors such as cameras and from other data sources.

In particular, the first database 290, the second database 390 and the evaluation module 500 can be integrated in a cloud computing infrastructure. Components of the ground truth module 300, such as a computing unit for generating the ground truth data 380, can also be housed in the cloud computing infrastructure. This makes it possible to achieve a fast calculation result, since cloud-based solutions offer the advantage of high and therefore fast computing power.

The term cloud computing infrastructure refers to the set of hardware, software, network, and storage resources used to support cloud computing services. Cloud computing infrastructure provides a platform for delivering a wide range of services, including data storage and management, application development and deployment, and virtual computing resources. It includes services for providing computing resources such as processing power and storage to support cloud-based applications. Security technologies such as firewalls, encryption, and identity and access management tools are used to protect data and applications from unauthorized access. Cloud computing infrastructure can significantly increase computing power and the speed of data processing.

Furthermore, communication connections are provided for the exchange and transmission of data between the individual modules, which are designed in particular as a wireless communication connection, e.g. as a mobile radio connection, and/or as a near-field communication connection, e.g. Bluetooth®, Ethernet, NFC (near field communication) or Wi-Fi®. In particular, 5G or 6G radio modules are provided for the communication connection between the vehicle 200 and a cloud computing infrastructure. This also applies to the communication connection between the ground truth module 300 and a cloud computing infrastructure. In order to protect the connection to the cloud computing infrastructure via a mobile radio connection, cryptographic encryption methods can also be provided.

The test vehicle 200 includes the first sensor device 250 that provides data for the ADAS system 230. The first sensor device 250 includes one or more cameras arranged in the vehicle 200. These cameras typically use different imaging technologies and are arranged at different locations on the vehicle to provide a comprehensive view of the surroundings. The most common type of camera used in driver assistance systems is a forward-facing camera, which is mounted on the front of the vehicle and detects and captures other vehicles, pedestrians, and obstacles in the vehicle's path. It is often used for functions such as lane departure warning, automatic emergency braking, and adaptive cruise control. There is also a rear-facing camera, which is typically mounted on the rear of the vehicle and captures the area behind the vehicle including other vehicles, pedestrians, and obstacles. It is often used for functions such as rearview cameras and cross-traffic alert. Side cameras can also be used. These cameras are typically mounted on the side of the vehicle and capture areas that the driver cannot see through the mirrors. They are often used for functions such as hidden area detection and lane change assistance. In addition, surround view cameras can be used. These cameras are usually mounted on several sides of the vehicle and provide a 360-degree view of the vehicle's surroundings. They are often used for functions such as parking assistance and collision avoidance.

The cameras of the first sensor device 250 are typically designed as RGB cameras. An RGB camera is a digital camera that captures color information in the visible range of the electromagnetic spectrum and divides it into three channels: red, green and blue (RGB). These three channels are then combined to produce the entire color spectrum of the image. The cameras operate in the visible spectral range, which is the world perceptible to the human eye. However, some cameras can also operate in the infrared or ultraviolet spectrum, which can be useful for applications such as night vision or detecting certain types of hazards. IR cameras use infrared imaging technology to capture the environment in low or no light conditions, such as at night or in tunnels. They are often used for functions such as night vision and pedestrian detection.

In addition, the first sensor equipment 250 may include a lidar system and/or a radar system. A lidar (light detection and ranging) system is a remote sensing technology that measures distances and creates high-resolution 3D maps of the environment. A lidar system typically consists of a laser transmitter that emits laser beams, usually in the infrared spectrum. These laser beams are directed at the environment of the vehicle. A scanner, e.g. a rotating mirror, directs the laser beams of the laser transmitter in different directions. This allows the lidar system to cover a large field of view. The laser beams reflected from the objects in the environment are received by a photodetector. The signals received by the photodetector are processed by a special signal processing unit. It analyzes the time-of-flight information, the intensity of the reflected laser beams and other data to create a detailed 3D map of the environment including the positions and shapes of the objects. Lidar systems provide high-resolution and accurate 3D maps and enable the precise detection, localization and tracking of objects. They are effective in detecting and identifying objects such as vehicles, pedestrians and obstacles under various driving conditions.

A radio detection and ranging (radar) system is a sensor technology that uses radio waves to detect and locate objects. A radar system has an antenna to transmit and receive radio waves. It sends the radio waves out from the radar transmitter and receives the waves returned by objects in the surrounding area. A signal processor analyzes the received signals and extracts relevant information such as time delay, frequency shift, and intensity of the reflected waves. By analyzing the frequency shift of the received waves (Doppler effect) and the time delay, the radar system can estimate the relative speed and distance of objects. This information helps determine the presence, location, and movement of objects such as vehicles or pedestrians. Radar systems provide accurate distance measurements and reliable object detection. They are especially useful in adverse weather conditions that can limit visibility, as radio waves are less affected by factors such as rain, fog, or dust compared to other sensor technologies.

Cameras, lidar systems and radar systems all have their strengths and limitations, and the combination of information from multiple sensing modalities is used in ADAS systems to improve overall perception and increase the accuracy and reliability of object detection and tracking. In addition, other types of sensors such as ultrasonic sensors can be used within the scope of the invention.

The second sensor system 350 essentially contains the same types of sensors such as cameras and/or a lidar system and/or a radar system. However, the resolution of the individual sensors is significantly higher and the number of sensors can also be larger.

The resolution of a camera refers to the number of pixels that make up the image captured by the camera. A camera with a higher resolution captures more detailed and sharper images than a camera with a lower resolution. The resolution of a camera is usually expressed by the number of horizontal and vertical pixels in the image, e.g. 1920 x 1080 (also known as Full HD) or 3840 x 2160 (also known as 4K). Full HD cameras are typically used for the first sensor device 250 because the required computing power and storage capacity for image processing of a 4K camera in the vehicle are too large. For the second sensor device 350 of the ground truth module 300, on the other hand, 4K cameras are used because a significantly higher resolution is desired here and the ground truth module 300 has greater computing resources, either directly through corresponding processors or through integration into a cloud computing infrastructure.

The frame rate of a vehicle camera for an ADAS system 230 refers to the rate at which the camera captures and stores images per second. This parameter is also referred to as frame rate or frames per second (FPS) and is an important criterion when selecting a camera for ADAS applications. A higher frame rate means that the camera can capture and store more images per second, enabling a smoother and more detailed representation of the environment. Here too, the cameras of the second sensor device 350 have a higher frame rate than the cameras of the first sensor device 250. Typical frame rates for cameras of the first sensor device 250 for ADAS applications are between 30 and 60 frames per second, while the more powerful cameras of the second sensor device 350 can record up to 120 frames per second. The higher frame rate allows the cameras of the second sensor device 350 to take factors such as vehicle speed into account much better and thus map the complexity of the environment more accurately and with higher resolution.

Higher frame rates increase the requirements for storing and processing the captured image data. As already mentioned, the ground truth module 300 has such computing resources, while a conventional ADAS system 230 is not equipped with such computing resources, as this would significantly increase the overall cost and also the complexity of the ADAS system 230. When selecting a camera for a vehicle-based ADAS system 230, a balance must therefore be found between image recording frequency, image quality and system requirements in order to ensure optimal performance and efficiency. For the validation of the ADAS system 230, however, this means that the design of the camera system installed in the vehicle is not sufficient to be able to make a satisfactory assessment of the reliability of the ADAS system 230 based solely on the data provided by the first sensor device 250.

These considerations apply equally to lidar and radar systems. The main differences between high-resolution lidar and radar systems and low-resolution systems are the level of detail and range. For example, high-resolution lidar systems capture a more detailed image of the environment. They emit a larger number of laser beams and have a higher point density in the resulting point cloud, allowing for more detailed object detection, e.g. distinguishing small objects or detecting complicated object shapes. In contrast, low-resolution lidar systems have fewer laser beams and produce point clouds with fewer data points, resulting in a lower level of detail. High-resolution lidar systems therefore typically offer higher accuracy in ranging and object localization. They can measure smaller distances and detect objects with greater accuracy, resulting in a more accurate representation of the environment. Lower-resolution lidar systems may have lower accuracy due to larger measurement intervals and limited point density. However, the range of high-resolution lidar systems may be lower compared to lower-resolution systems. While they are capable of capturing detailed information within a limited range, their ability to detect objects at greater distances may be compromised. Low-resolution lidar systems, on the other hand, can cover a larger area, but with lower resolution and detail.

Radar systems typically have lower resolution than lidar systems. However, high-resolution radar systems use advanced signal processing techniques and more sophisticated antenna arrays to improve resolution. They can provide more detailed information about objects, such as their size, shape, and movement patterns. Lower-resolution radar systems have fewer antenna elements and cannot capture finer details as well. High-resolution radar systems therefore provide greater accuracy in detecting and tracking objects. They can provide accurate measurements of relative speeds, angles, and positions, allowing a better estimate of the trajectory of objects. Lower-resolution radar systems may have lower accuracy and are subject to certain limitations, such as lower accuracy in measuring speed. In terms of range, high-resolution radar systems can maintain better resolution and accuracy over longer distances, while lower-resolution systems may have poorer detection and tracking performance.

High-resolution lidar and radar systems offer finer detail, higher accuracy, and better localization capabilities compared to low-resolution systems. However, low-resolution systems often offer greater coverage and detection range, making them suitable for certain applications where high levels of detail are not necessarily required or feasible. The choice between high-resolution and low-resolution systems depends on the specific requirements of the ADAS application and the trade-off between resolution, accuracy, range, and cost.

In order to test the functions of the ADAS system 230, the first sensor device 250 records first sensor data 270 from the surroundings of the vehicle 200 when driving on a test track with a large number of traffic scenarios and sends them to the ADAS system 230, which reacts to these first sensor data 270 and generates test data 280 therefrom. As described, the first sensor device 230 comprises at least one camera and/or a lidar system and/or a radar system and/or another sensor for perception and data capture. The first sensor data 270 may include images, point clouds, sensor readings, or other relevant information. The captured sensor data 270 is transmitted or sent to the ADAS system 230. The first sensor data 270 is typically transmitted over a wired or wireless connection so that the ADAS system 230 can access and analyze the information. The algorithms and processing components of the ADAS system 230 analyze the received first sensor data 270 to interpret the environment, identify objects, and make decisions based on the sensor information. Based on the analysis of the first sensor data 270, the ADAS system 230 generates test data 280. This test data 280 may include outputs such as detected objects, calculated distances, vehicle trajectories, lane information, or other relevant information that the ADAS system 230 provides as part of its functionality. The generated test data 280 serves as a result or response of the ADAS system 230 and reflects its behavior and performance in a particular traffic scenario.

The algorithms of the ADAS system 230 include, in particular, extraction and classification algorithms to extract and classify essential features from the first sensor data 270 acquired by the first sensor device 250. The extraction and classification algorithms use, in particular, artificial intelligence algorithms such as neural networks.

A neural network consists of neurons that are arranged in several layers and connected to one another in different ways. A neuron is able to receive information from outside or from another neuron at its input, evaluate it in a certain way and pass it on in a modified form to another neuron at the neuron's output or output it as the final result. The hidden neurons are located between the input neurons and the output neurons. Depending on the type of network, there can be several layers of hidden neurons. They ensure that the information is forwarded and processed. Output neurons finally provide a result and pass it on to the outside world. The different arrangement and interconnection of the neurons creates different types of neural networks, such as a feedforward network (FNN), a recurrent network (RNN) or a convolutional neural network (CNN). The networks can be trained through unsupervised or supervised learning.

The extraction algorithm recognizes, for example, whether the first sensor data 270 recorded contains a traffic sign or other information. Test data 280 is created from this extracted information, which is transmitted according to the invention to the evaluation module 500. The test data 280 can also be stored in the database 290 in order to be available for evaluation at a later time.

The second sensor device 350 also records the surroundings of the vehicle 200. In particular, it is provided that the ground truth module 300 or the second sensor device 350 is also installed in the vehicle 200, so that the surroundings recorded by the first sensor device 250 and the second sensor device 350 are identical in each case. Since the sensors of the second sensor device 350 have a significantly higher resolution than the first sensor device 250, the surroundings are recorded with significantly greater accuracy. The second sensor data 370 recorded by the second sensor device 350 are further processed by a software application 330 to create ground truth data 380. The software application 330 includes a large number of perception algorithms, such as image processing algorithms, extraction algorithms, classification algorithms, which in particular use artificial intelligence algorithms, such as neural networks. Since the recorded second sensor data 370 is significantly more complex than the first sensor data 270, the neural networks used also have a higher complexity in order to be able to process this second sensor data 370. In particular, these neural networks are trained with correspondingly high-resolution data sets.

In addition to the recorded second sensor data 370

0, other data sources such as map data, historical data, weather data, traffic data, etc. can be used. This data is stored in particular in the database 390. In addition, target specifications, key figures, etc. can be taken into account.

In particular, the map data contains various types of information that cannot be collected by vehicle sensors alone. This data is collected from various sources such as GPS satellites, road markings and other sources. The data typically includes information about the road geometry such as curvature, slope and width of the road as well as the location of intersections, traffic lights and pedestrian crossings. pedestrian crossings. In addition, the map data may include information about traffic conditions, such as traffic flow, collected in real time from various sources, such as roadside sensors and GPS-enabled devices used by other vehicles.

The ground truth data 380 are now transmitted to the evaluation module 500. The evaluation module 500 has a software application 550 that contains comparison algorithms that are based in particular on artificial intelligence algorithms. Comparison algorithms play a crucial role in image analysis because they make it possible to quantify the similarity or dissimilarity between images. According to the invention, a large number of comparison algorithms can be used, with the choice of the appropriate algorithm depending on the type of data, the desired properties of the comparison and the specific question relating to the performance of the ADAS system 230.

An example of comparison algorithms are histogram-based algorithms that compare the intensity distributions of pixels in two images. These include histogram intersection, histogram correlation, and histogram intersection. Histogram comparison methods are useful for tasks such as image matching, object detection, and content-based image search.

Deep learning-based approaches, such as convolutional neural networks (CNNs), extract features from images and enable comparison based on learned data representations. The images are inserted into a feature space where similarity can be calculated using metrics such as cosine similarity or Euclidean distance.

The software application 550 is now used to compare the test data 280 with the ground truth data 380. By comparing with the ground truth data 380, it can be determined whether the test data 280 has the required accuracy and reliability with regard to the performance of the ADAS system 230. By comparing the highly accurate ground truth data 380 with the vehicle-based test data 280, the accuracy and reliability of the ADAS system 230 can now be checked. This allows the algorithms of the ADAS system 230 to be automatically validated with regard to their ability to detect objects in the environment of the vehicle 200 and to react to them appropriately and safely.

According to the invention, the precise ground truth data 380 thus serve to evaluate and validate the performance and reliability of the ADAS system 230 and to identify areas in need of improvement. Since the ground truth data 380 represents the reference point for evaluating the performance of the ADAS system 230, comparison algorithms based on artificial intelligence can be used, which enable an automated analysis of the test data 280 and thus also an automated validation of the ADAS system 230.

According to the invention, the evaluation of the test data 280 by comparison with the ground truth data 380 can be carried out both in real time while the test vehicle 200 is driving along a route and at a later point in time with the recorded test data 280 and the stored ground truth data 380. In the case of an evaluation in real time, the use of a cloud computing infrastructure and the use of 5G or 6G radio modules for mobile data transmission are particularly suitable.

Evaluation results 580 are created from the comparison. These evaluation results 580 can be passed on to an output module 700, which uses a software application 750 to create output data 780 from the evaluation results 580, in particular in the form of reports, histograms, diagrams, text messages, software codes, Power Point presentations, etc., which can be used to validate the ADAS system 230.

In 2 The procedural steps for the automated validation of functions of a driver assistance system (ADAS system) are presented.

In a step S10, first sensor data 270 are recorded by at least one sensor such as a camera, a lidar system and/or a radar system of a first sensor device 250 of a test vehicle 200 while driving along a route for testing at least one function of an ADAS system 230.

In a step S20, the first sensor data 270 is processed by the ADAS system 230 and test data 280 is generated using the processed first sensor data 270.

In a step S30, the test data 280 are transmitted to an evaluation module 500 and/or to a first database 290.

In a step S40, second sensor data 270 are collected by at least one sensor such as a camera, a lidar system and/or a radar system a second sensor device 350 of a ground truth module 300 while driving along a route, wherein the sensors of the second sensor device 350 have a higher performance in terms of resolution, frame rate, range, and computing resources than the sensors of the first sensor device 250.

In a step S50, the second sensor data 370 are processed by a software application 330 and ground truth data 380 are generated using the processed second sensor data 370.

In a step S60, the ground truth data 380 are transmitted to the evaluation module 500 and/or to a second database 390.

In a step S70, the test data 280 are compared with the ground truth data 380 by a software application 550, wherein the software application 550 comprises comparison algorithms.

In a step S80, evaluation results 580 are generated to validate at least one function of the ADAS system 230.

3 schematically illustrates a computer program product 900 comprising an executable program code 950 configured to carry out the method according to the first aspect of the present invention.

The present invention enables accurate and reliable validation of a modern driver assistance system (ADAS) by automatically comparing ground truth data with test data of an ADAS system using suitable comparison algorithms. Accurate validation ensures that the ADAS system functions as intended, thereby increasing overall road safety. In addition, the performance of the ADAS system can be evaluated. By comparing the test data with the ground truth data, accuracy, reliability and other performance indicators can be measured. This evaluation helps to identify opportunities for improvement and fine-tune the ADAS system for optimal performance.

reference sign

100

system

200

test vehicle

230

ADAS system

250

first sensor device

270

first sensor data

280

test data

290

first database

300

ground truth module

330

software application

350

second sensor device

370

second sensor data

380

ground truth data

390

second database

500

evaluation module

550

software application

580

evaluation results

700

output module

750

software application

780

output data

900

computer program product

950

program code

Claims (6)

Method for the automated validation of functions of a driver assistance system (ADAS system); comprising: - recording (S10) first sensor data (270) by at least one sensor such as a camera, a lidar system and/or a radar system of a first sensor device (250) of a test vehicle (200) when driving on a route for testing at least one function of an ADAS system (230), wherein the camera of the first sensor device (250) is a digital camera with full HD resolution; - processing (S20) the first sensor data (270) from the ADAS system (230) and generating test data (280) using the processed first sensor data (270); - transmitting (S30) the test data (280) to an evaluation module (500) and/or to a first database (290); - Recording (S40) of second sensor data (270) by at least one sensor such as a camera, a lidar system and/or a radar system of a second sensor device (350) of a ground truth module (300) when driving along a route, wherein the camera of the second sensor device (350) is a digital camera with 4K resolution, wherein the sensors of the second sensor device (350) have a higher performance in terms of resolution, frame rate, range and computing resources than the sensors of the first sensor device (250); - Processing (S50) of the second sensor data (370) by a software application (330) and generating ground truth data (380) using the processed second sensor data (370); - transmitting (S60) the ground truth data (380) to the evaluation module (500) and/or to a second database (390); - comparing (S70) the test data (280) with the ground truth data (380) by a software application (550) that includes comparison algorithms, wherein the comparison algorithms use convolutional neural networks (CNN) and/or histogram-based algorithms; - generating (S80) evaluation results (580) for validating the at least one function of the ADAS system (230). procedure according to claim 1 , wherein a function of the ADAS system (230) is a traffic sign algorithm. procedure according to claim 1 or 2 , wherein the first database (290) and/or the second database (390) and/or the evaluation module (500) is/are integrated in a cloud computing infrastructure. System (100) for the automated validation of functions of a driver assistance system (ADAS system), comprising a test vehicle (200) with an ADAS system (230) and a first sensor device (230) with at least one sensor such as a camera, a lidar system and/or a radar system, wherein the camera of the first sensor device (250) is a digital camera with full HD resolution, a ground truth module (300) with a software application (330) and a second sensor device (350) and an evaluation module (500) with a software application (550); wherein the first sensor device (250) is designed to record first sensor data (270) when the test vehicle (200) travels a route to test at least one function of an ADAS system (230); wherein the ADAS system (230) is designed to process the first sensor data (270) to generate test data (280) and to transmit the test data (280) to the evaluation module (500); wherein the second sensor device (350) is designed to record second sensor data (270) by at least one sensor such as a camera, a lidar system and/or a radar system when driving along a route, wherein the camera of the second sensor device (350) is a digital camera with 4K resolution, wherein the sensors of the second sensor device (350) have a higher performance in terms of resolution, frame rate, range and computing resources than the sensors of the first sensor device (250); wherein the software application is designed to process the second sensor data (370) to generate ground truth data (380) and to transmit the ground truth data (380) to the evaluation module (500); wherein the software application (550) comprises comparison algorithms and is designed to compare the test data (280) with the ground truth data (380) and to generate evaluation results (580) for validating the at least one function of the ADAS system (230), and wherein the comparison algorithms use convolutional neural networks (CNN) and/or histogram-based algorithms. ADAS system (230) for a motor vehicle, characterized in that for validating at least one function of the ADAS system (230) the method according to one of the Claims 1 until 3 is used. Computer program product (900) comprising an executable program code (950) which is configured to carry out the method according to one of the Claims 1 until 3 executes.

Images