DE102021111736A1 - PROCEDURE, COMPUTER PROGRAM PRODUCT, PARKING ASSISTANCE SYSTEM AND VEHICLE - Google Patents

Abstract

A method for operating a parking assistance system (110) for a vehicle (100) is proposed. The parking assistance system (110) is set up at least for semi-autonomous control of the vehicle (100). In the proposed method, optically recorded markers (ID1 - ID4) are checked for plausibility using meta information about the marker (ID1 - ID4) and other recorded information. The markers (ID1 - ID4) are in particular ARUCO markers. The plausibility check can ensure that the detected marker (ID1 - ID4) was correctly detected and is arranged in the correct position and can therefore be used for reliable orientation.

Description

The present invention relates to a method, a computer program product, a parking assistance system and a vehicle.

Vehicles are known which have an automated parking function which is particularly suitable for automatically parking the vehicle in a suitable multi-storey car park or a suitable parking space. Such systems are referred to, for example, as automated valet parking systems. A distinction is made between two types. In a first type, the vehicle is self-driving, with the parking garage having, for example, suitable features for orienting the vehicle, such as ARUCO codes. In a second type, the vehicle can be controlled remotely, with the multi-storey car park having sensors and path planning means, for example, in order to control the vehicle. There can be various intermediate levels between these two types, in which the functions are distributed differently between the vehicle and the parking garage.

A problem with in-vehicle systems is that the vehicle or the parking assistance system must orientate itself for the correct control and planning of the trajectory, i.e. it should be able to determine its own position and attitude in relation to a specific reference point and/or a reference coordinate system, such as one card, appreciate.

It is known to estimate the position and attitude of a vehicle based on odometry data such as wheel speed and steering angle in conjunction with a kinematic vehicle model. This is quite accurate for short distances, but the error due to wheel slip and the like increases significantly the longer the distance traveled, so that orientation based solely on this is not reliable for driving in a parking garage or the like.

It is also known to arrange orientation markings in the area at predetermined positions, by means of which the vehicle can orientate itself. So-called ARUCO codes are an example of this. These codes are captured and recognized by an on-board camera. For example, position information assigned to the respective code can be determined from a database. However, the problem arises that incorrect recognition of the codes or incorrect arrangement of the codes can possibly lead to incorrect determination of the position. This can lead to an incorrect continuation of the journey, a termination of the journey and/or a dangerous situation.

Against this background, an object of the present invention is to improve the operation of a parking assistance system.

1. a) Empfangen eines Datensatzes umfassend eine Zuordnung von wenigstens einer jeweiligen ersten Information zu einem jeweiligen Marker für mehrere eindeutig unterscheidbare Marker,
2. b) Empfangen eines optischen Sensorsignals einer Umgebung des Fahrzeugs von einer Sensoreinheit des Fahrzeugs während einer autonomen Fahrt des Fahrzeugs, wobei in dem optischen Sensorsignal wenigstens ein bestimmter Marker der mehreren Marker enthalten ist,
3. c) Ermitteln des in dem empfangenen optischen Sensorsignal enthaltenen bestimmten Markers und Aufrufen der zugeordneten ersten Information,
4. d) Ermitteln einer zweiten Information in Abhängigkeit des empfangenen Datensatzes und/oder des empfangenen optischen Sensorsignals und/oder eines weiteren von einer weiteren Sensoreinheit des Fahrzeugs empfangenen Sensorsignals,
5. e) Plausibilisieren des bestimmten Markers in Abhängigkeit eines Vergleichs der aufgerufenen ersten Information mit der ermittelten zweiten Information, und
6. f) Verwenden der aufgerufenen ersten Information und/oder einer weiteren, in dem Datensatz zu dem Marker gespeicherten Information zum Fortsetzen der autonomen Fahrt, wenn der bestimmte Marker plausibilisiert ist, oder Verwerfen der aufgerufenen ersten Information und/oder der weiteren, in dem Datensatz zu dem bestimmten Marker gespeicherten Information, wenn der bestimmte Marker nicht plausibilisiert ist.

According to a first aspect, a method for operating a parking assistance system for a vehicle is proposed. The parking assistance system is set up at least for semi-autonomous control of the vehicle. The procedure has the steps:

1. a) Receiving a data record comprising an assignment of at least one respective first piece of information to a respective marker for a plurality of clearly distinguishable markers,
2. b) receiving an optical sensor signal of an environment of the vehicle from a sensor unit of the vehicle while the vehicle is driving autonomously, wherein at least one specific marker of the plurality of markers is contained in the optical sensor signal,
3. c) determining the specific marker contained in the received optical sensor signal and calling up the assigned first information item,
4. d) determining second information as a function of the received data set and/or the received optical sensor signal and/or a further sensor signal received from a further sensor unit of the vehicle,
5. e) checking the plausibility of the specific marker as a function of a comparison of the first information called up with the determined second information, and
6. f) Using the retrieved first information and/or additional information stored in the data record for the marker to continue the autonomous journey if the specific marker has been checked for plausibility, or discarding the retrieved first information and/or additional information in the data record information stored on the specific marker if the specific marker is not checked for plausibility.

This method can advantageously be used to determine whether the first piece of information called up is reliable or not by checking the specific marker for plausibility. If the first information is reliable, the autonomous driving can be continued based on this information. On the other hand, the autonomous driving can be continued without the first information if is not reliable. Consequently the execution of the autonomous driving is more reliable, more stable and safer overall.

The fact that the parking assistance system is set up for autonomous control of the vehicle means in particular that the parking assistance system has an automation level according to the SAE classification system. The SAE classification system was published in 2014 by SAE International, an automotive standards organization, as J3016, "Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle Automated Driving Systems". It is based on six different levels of automation and takes into account the level of system intervention and driver attention required. The SAE levels of automation range from level 0, which corresponds to a fully manual system, through driver assistance systems in levels 1 to 2, to partially autonomous (levels 3 and 4) and fully autonomous (level 5) systems, which no longer require a driver . An autonomous vehicle (also known as a driverless car, self-driving car and robotic car) is a vehicle that is able to sense its surroundings and navigate without human input and corresponds to SAE automation level 5. The parking assistance system has a particular Degree of automation greater than or equal to 4.

As part of the method, the vehicle is controlled autonomously by the parking assistance system, ie the vehicle drives autonomously. The autonomous driving is carried out in particular by the parking assistance system. It is, for example, an autonomous parking process in a multi-storey car park.

The data set comprising the assignment of the respective first information to a respective marker for a plurality of clearly distinguishable markers is provided in particular before the vehicle drives autonomously and is received by the vehicle. For example, the data record can already be provided when a user books a parking space in a parking lot or in a multi-storey car park via a corresponding portal. As an alternative or in addition, the data set can be provided when entering the parking lot or the multi-storey car park. In embodiments, the dataset for a particular area may be provided upon entry of the vehicle into the particular area. The data set is provided in particular by an operator of the parking lot or of the multi-storey car park. The data set can be transmitted using any possible means of data transmission, in particular wirelessly, such as using Bluetooth®, WLAN, a 3G, 4G or 5G mobile network, radio and the like.

The data set can advantageously include a digital map on which the positions of the markers are entered. The parking assistance system can then orient itself based on the digital map and, for example, carry out path planning.

In particular, the markers are designed in such a way that they have optically distinctive features that can be recognized reliably and with as little computing effort as possible using computer-aided image processing. Visually striking features are particularly hard contrasts and clearly drawn geometric shapes. For example, the markers are black and white and include squares, rectangles, triangles, circles, and the like. The markers can be designed in particular as ARUCO markers. In order to avoid confusing markers, the markers within a data set are designed to be as different as possible, so that each marker with a number of characteristics differs from other markers.

The optical sensor signal is in particular a camera image of the surroundings of the vehicle, which was captured by a corresponding camera of the vehicle. The vehicle preferably has at least one front camera and one rear camera. Additional cameras may be present.

The optical sensor signal is fed to suitable image processing or image analysis, which searches for markers contained in the optical sensor signal. The image processing can include appropriate algorithms and/or can include trained artificial intelligence, such as a CNN (convolutional neural network).

The determined marker is clearly identifiable in particular using an assigned identification number or index. The correspondingly assigned first information in the data record can be determined and called up by means of this identification number. It can therefore be said that the identification number is determined. The identification number is encoded in the appearance of the marker in particular on the basis of a specific dictionary or a specific formation rule.

According to one embodiment of the method, the assigned first information comprises position information of the respective marker, orientation information of the respective marker, arrangement information of the respective marker, counting information of the respective marker and/or visibility information of the respective marker. The first piece of information can also be referred to as meta information about the respective marker.

The position information includes, for example, an indication of the position of the marker in a specific coordinate system or in a map that includes, for example, the parking lot or the multi-storey car park. The position information can also include a distance from an entrance and/or a distance from a predetermined lane. The orientation information includes, for example, a specific angle of a surface normal of the marker relative to the predetermined lane, such as perpendicular to the lane or parallel to the lane. The arrangement information includes, for example, information regarding the arrangement of the marker relative to other features of the environment, such as "directly adjacent to a ceiling", "one meter above the floor", "on a pillar", "one meter next to a door" or the like. The counting information includes, for example, an indication that the specific marker should be the fifth marker that should be detected along the trajectory driven. The visibility information includes, for example, qualitative information about the visibility of the specific marker, such as that the marker is poorly lit, and/or includes information that at least one other specific marker can be visible at the same time as the specific marker. A marker is preferably 0.35 to 1.1 m wide and/or 0.35 to 1.1 m high. In particular, the marker is 0.4 m wide and 0.40 m high or 1 m wide and 1 m high. The marker is preferably rectangular, in particular square. Alternatively, the marker can be round, triangular or arrow-shaped. In particular, the markers can be placed above the ground, in particular more than 2 m above the ground, or also on the ground.

The second piece of information is, in particular, information determined independently of the specific marker, on the basis of which the specific marker can be checked for plausibility. The second information can be determined as a function of the received data set and/or the received optical sensor signal and/or a further sensor signal received from a further sensor unit of the vehicle.

According to a further embodiment of the method, the determined second piece of information comprises odometrically determined position information of the vehicle, current orientation information and/or arrangement information of the specific marker determined as a function of the received optical sensor signal, current counting information of the specific marker determined as a function of a course of the autonomous journey and/or current visibility information of the specific marker determined as a function of the received optical sensor signal.

For example, sensor signals from one or more wheel speed sensors and sensor signals from one or more steering angle sensors are received (these are examples of other sensor units of the vehicle) and position information of the vehicle is determined odometrically using a kinematic vehicle model. Determined odometrically means, for example, that a known position is updated using odometer data. A reference to the digital map is preferably established.

The optical sensor signal received can also be used to determine current orientation information for the marker. The current orientation information is in particular indicative of an actual orientation of the specific marker.

On the basis of the received optical sensor signal, current visibility information can be determined, which indicates, for example, which of the multiple markers are contained in the optical sensor signal and have been recognized. At least two of the markers are preferably always visible at the same time, which enables the vehicle's position to be determined unambiguously in a two-dimensional coordinate system, for example by means of triangulation or multilateration.

The retrieved first information and the ascertained second information are compared with one another, and depending on the result of the comparison, the specific marker is checked for plausibility or not checked for plausibility. In the present case, the fact that the marker is checked for plausibility is understood to mean that the determined marker is regarded as being reliably and correctly recognized. Therefore, in this case, the first information called up can be used to continue the autonomous journey. If the marker is not checked for plausibility, the first information called up is discarded and not used to continue the autonomous journey.

• Ermitteln eines Differenzwerts der Positionsinformation des bestimmten Markers und der Positionsinformation des Fahrzeugs, und
• Vergleichen des Differenzwerts mit einem vorbestimmten Schwellwert und/oder einem vorbestimmten Intervall, wobei der bestimmte Marker plausibilisiert wird, wenn der Differenzwert kleiner oder gleich dem vorbestimmten Schwellwert ist und/oder innerhalb des vorbestimmten Intervalls liegt.

According to a further embodiment of the method, the assigned first information includes position information of the specific marker and the determined second information includes odometrically determined position information of the vehicle, step e) comprising:

• determining a difference value of the position information of the specific marker and the position information of the vehicle, and
• Comparing the difference value with a predetermined threshold value and/or a predetermined interval, the specific marker being checked for plausibility if the difference value is less than or equal to the predetermined threshold value is and/or is within the predetermined interval.

The predetermined threshold value or the predetermined interval can be individually predetermined for a respective marker and/or be determined individually and currently in a predetermined manner. For example, the threshold value or the interval can be currently determined as a function of a currently determined accuracy of the determined position information of the vehicle.

• Ermitteln einer Relativposition des Fahrzeugs relativ zu dem in dem empfangenen optischen Sensorsignal enthaltenen bestimmten Marker in Abhängigkeit des empfangenen optischen Sensorsignals,
• Ermitteln einer aktuellen Fahrzeugposition in Abhängigkeit der ermittelten Relativposition und der Positionsinformation des bestimmten Markers, und
• Kalibrieren der odometrisch ermittelten Positionsinformation des Fahrzeugs auf Basis der ermittelten aktuellen Fahrzeugposition.

According to a further embodiment of the method, this also includes:

• Determining a relative position of the vehicle relative to the specific marker contained in the received optical sensor signal as a function of the received optical sensor signal,
• Determining a current vehicle position depending on the determined relative position and the position information of the specific marker, and
• Calibrating the odometrically determined position information of the vehicle on the basis of the determined current vehicle position.

The relative position of the vehicle can be determined, for example, based on the position and the size of the marker in the optical sensor signal. This can be done on the basis of ray-optical principles.

This embodiment has the advantage that the odometrically determined vehicle position is very reliable again after calibration has taken place, since accumulated errors are reset.

It should be noted that the markers are preferably distributed and arranged in such a way that at least two markers are detected by the vehicle at the same time, so that the determined vehicle position can be determined very precisely on the basis of the relative position of the vehicle to two of the markers.

• Ermitteln eines Differenzwerts der Soll-Orientierung des bestimmten Markers und der ermittelten Ist-Orientierung des bestimmten Markers, und
• Vergleichen des Differenzwerts mit einem vorbestimmten Schwellwert, wobei der bestimmte Marker plausibilisiert wird, wenn der Differenzwert kleiner oder gleich dem vorbestimmten Schwellwert ist.

According to a further embodiment of the method, the assigned first information includes orientation information of the specific marker, which includes a target orientation of the specific marker relative to a driving corridor, and the determined second information includes current orientation information, which determined a function of the received optical sensor signal includes actual orientation of the particular marker, wherein step e) includes:

• Determining a differential value of the target orientation of the specific marker and the determined actual orientation of the specific marker, and
• Comparing the difference value with a predetermined threshold value, the specific marker being checked for plausibility if the difference value is less than or equal to the predetermined threshold value.

• Ermitteln eines Differenzwerts der Soll-Anordnung des bestimmten Markers und der ermittelten Ist-Anordnung des bestimmten Markers, und
• Vergleichen des Differenzwerts mit einem vorbestimmten Schwellwert, wobei der bestimmte Marker plausibilisiert wird, wenn der Differenzwert kleiner oder gleich dem vorbestimmten Schwellwert ist.

According to a further embodiment of the method, the associated first information includes an arrangement information of the specific marker, which includes a target arrangement of the specific marker relative to a feature in the vicinity of the specific marker, and the determined second information includes an arrangement information, which is a dependent of the received optical sensor signal comprises the actual arrangement of the specific marker relative to the feature, wherein step e) comprises:

• Determining a differential value of the desired arrangement of the specific marker and the determined actual arrangement of the specific marker, and
• Comparing the difference value with a predetermined threshold value, the specific marker being checked for plausibility if the difference value is less than or equal to the predetermined threshold value.

• Ermitteln einer Soll-Relativposition der in dem optischen Sensorsignal enthaltenen Marker auf Basis der jeweiligen zugeordneten Positionsinformation,
• Ermitteln einer Ist-Relativposition der in dem optischen Sensorsignal enthaltenen Marker auf Basis einer auf Basis des optischen Sensorsignals ermittelten jeweiligen Relativposition des Fahrzeugs zu dem jeweiligen Marker und der Positionsinformation des Fahrzeugs, und
• Vergleichen der Soll-Relativposition mit der Ist-Relativposition, wobei die in dem optischen Sensorsignal enthaltenen Marker plausibilisiert werden, wenn ein Unterschied zwischen der Soll-Relativposition und der Ist-Relativposition kleiner oder gleich einem vorbestimmten Schwellwert ist.

According to a further embodiment of the method, the assigned first information includes position information of the specific marker and the ascertained second information comprises odometrically ascertained position information of the vehicle, wherein the received optical sensor signal contains at least two markers, and wherein step e) comprises:

• Determination of a target relative position of the markers contained in the optical sensor signal on the basis of the respectively assigned position information,
• Determining an actual relative position of the marker contained in the optical sensor signal on the basis of a respective relative position of the vehicle to the respective marker determined on the basis of the optical sensor signal and the position information of the vehicle, and
• Comparing the target relative position with the actual relative position, the markers contained in the optical sensor signal being checked for plausibility if a difference between the target relative position and the actual relative position is less than or equal to a predetermined threshold value.

In this embodiment, the markers are checked for plausibility on the basis of their relative position. This plausibility check is particularly preferred since it is very robust and also enables an incremental plausibility check marker-by-marker in the course of the autonomous journey, provided that at least two markers are visible at the same time.

• Plausibilisieren des bestimmten Markers, wenn die Soll-Anzahl gleich der ermittelten Ist-Anzahl ist.

According to a further embodiment of the method, the associated first information includes counting information of the specific marker, which includes a target number that indicates how many of the multiple markers should already be determined in the course of the autonomous journey, and the determined second information includes current Counting information that includes an actual number that indicates how many markers have already been determined in the course of the autonomous journey, step e) including:

• Checking the specific marker for plausibility if the target number is equal to the determined actual number.

The target number can also be determined on the basis of the first information and a route driven.

The actual number refers in particular to how many different markers were determined in the course of the journey.

• Ermitteln, welche der mehreren Marker in dem optischen Sensorsignal enthalten sind, wobei der Schritt e) umfasst:
  • Plausibilisieren des bestimmten Markers, wenn die ermittelten Marker der Sichtbarkeitsinformation des bestimmten Markers entsprechen.

According to a further embodiment of the method, the assigned first information item comprises visibility information of the specific marker, which indicates which other markers of the plurality can be contained simultaneously in the received optical sensor signal, step d) comprising:

• Determining which of the plurality of markers are contained in the optical sensor signal, step e) comprising:
  • Checking the specific marker for plausibility if the determined markers correspond to the visibility information of the specific marker.

According to a further embodiment of the method, a plausibility index is determined for the specific marker, which is a measure of how reliable the plausibility check of the specific marker is, the plausibility index being determined on the basis of one or more of the plausibility check methods described above, and the parking assistance system of a predetermined driving corridor, which is assigned to the specific marker, deviates as a function of the plausibility index.

The driving corridor is in particular part of the data set and/or the first information. If the plausibility index is low, i.e. the marker was checked for plausibility only with low reliability, the parking assistance system can, for example, bring the vehicle at a greater or lesser distance from the marker and/or from obstacles stored in the digital map, such as pillars and/or walls to avoid collision with an obstacle or the like located close to or opposite to the marker.

In this case, several of the aforementioned plausibility criteria are combined in order to determine a plausibility index that enables an even more reliable plausibility check. For example, both a plausibility check based on the target position and the actual position and a plausibility check based on the target arrangement and the actual arrangement and additionally a plausibility check based on the visibility information are carried out. Each of the plausibility checks supplies a plausibility check result which, for example, uses a number between 0 and 1 to indicate a percentage of how plausible the marker is on the basis of the respective criterion. On the basis of the three individual plausibility check results, a plausibility check index is determined, for example, into which the individual results are included, and these can be weighted differently. The plausibility check index can include, for example, a mean value or the like of the individual plausibility check results.

• Fortsetzen der autonomen Fahrt auf Basis empfangener Sensorsignale, ohne die aufgerufene erste Information zu verwenden, wenn der bestimmte Marker nicht plausibilisiert wurde.

According to a further embodiment of the method, step f) comprises:

• Continuing the autonomous journey based on received sensor signals without using the first information called up if the specific marker has not been checked for plausibility.

In this case, the autonomous journey is continued, for example, on the basis of an odometrically determined vehicle position, and the path planning is in particular based on determined obstacles and/or objects, which are recorded and determined, for example, using ultrasonic sensors and/or image recognition and/or radar and/or lidar are carried out.

According to a further embodiment of the method, a reliability measure for the reliability of determining the specific marker in step c) is determined as a function of the received optical sensor signal, steps d) - f) only being carried out if the determined reliability measure is greater than or equal to a certain threshold.

The reliability measure can be determined based on a contrast range of the optical sensor signal, for example. The contrast range can be reduced due to contamination of the optical sensor or the respective marker be greatly reduced that the marker cannot be reliably determined. Furthermore, the marker can, for example, be so badly illuminated that it can no longer be reliably determined.

According to a further embodiment of the method, the multiple markers are in the form of different ARUCO markers.

According to a second aspect, a computer program product is proposed which comprises instructions which, when the program is executed by a computer, cause the latter to execute the method according to the first aspect.

A computer program product, such as a computer program means, can be made available or supplied by a server in a network, for example, as a storage medium such as a memory card, USB stick, CD-ROM, DVD, or in the form of a downloadable file. This can be done, for example, in a wireless communication network by transferring a corresponding file with the computer program product or the computer program means.

• eine Empfangseinheit zum Empfangen eines Datensatzes umfassend eine Zuordnung von wenigstens einer jeweiligen ersten Information zu einem jeweiligen Marker für mehrere eindeutig unterscheidbare Marker, und zum Empfangen eines optischen Sensorsignals einer Umgebung des Fahrzeugs von einer Sensoreinheit des Fahrzeugs während einer autonomen Fahrt des Fahrzeugs, wobei in dem optischen Sensorsignal wenigstens ein bestimmter der mehreren Marker enthalten ist,
• eine Ermittlungseinheit zum Ermitteln des in dem empfangenen optischen Sensorsignal enthaltenen bestimmten Markers, zum Aufrufen der zugeordneten ersten Information, und zum Ermitteln einer zweiten Information in Abhängigkeit des empfangenen Datensatzes und/oder des empfangenen optischen Sensorsignals und/oder eines weiteren von einer weiteren Sensoreinheit des Fahrzeugs empfangenen Sensorsignals,
• eine Plausibilisierungseinheit zum Plausibilisieren des bestimmten Markers in Abhängigkeit eines Vergleichs der aufgerufenen ersten Information mit der ermittelten zweiten Information, und
• eine Steuereinheit zum Durchführen der autonomen Fahrt unter Verwendung der aufgerufenen ersten Information und/oder weiteren, in dem Datensatz zu dem bestimmten Marker gespeicherten Informationen in Abhängigkeit des Plausibilisierens.

According to a third aspect, a parking assistance system for a vehicle is proposed, which is set up at least for semi-autonomous control of the vehicle. The parking assistance system includes:

• a receiving unit for receiving a data set comprising an assignment of at least one respective first piece of information to a respective marker for a plurality of clearly distinguishable markers, and for receiving an optical sensor signal of an environment of the vehicle from a sensor unit of the vehicle during an autonomous drive of the vehicle, wherein in the optical sensor signal contains at least one specific one of the several markers,
• a determination unit for determining the specific marker contained in the received optical sensor signal, for calling up the associated first information, and for determining second information as a function of the received data set and/or the received optical sensor signal and/or another from another sensor unit of the vehicle received sensor signal,
• a plausibility check unit for checking the specific marker for plausibility as a function of a comparison of the retrieved first information item with the ascertained second information item, and
• a control unit for carrying out the autonomous journey using the retrieved first information and/or additional information stored in the data set for the specific marker depending on the plausibility check.

The embodiments and features described for the proposed method apply accordingly to the proposed parking assistance system. The advantages and definitions described for the proposed apply correspondingly to the proposed parking assistance system.

The respective unit of the parking assistance system can be implemented in terms of hardware and/or software. In the case of a hardware implementation, the respective unit can be embodied, for example, as a computer or as a microprocessor. In the case of a software implementation, the respective unit can be designed as a computer program product, as a function, as a routine, as an algorithm, as part of a program code or as an executable object. Furthermore, each of the units mentioned here can also be designed as part of a higher-level control system of the vehicle, such as a central electronic control device and/or an engine control unit (ECU: Engine Control Unit).]

The fact that the parking assistance system is set up for autonomous control of the vehicle means in particular that the parking assistance system has an automation level according to the SAE classification system. The SAE classification system was published in 2014 by SAE International, an automotive standards organization, as J3016, "Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle Automated Driving Systems". It is based on six different levels of automation and takes into account the level of system intervention and driver attention required. The SAE levels of automation range from level 0, which corresponds to a fully manual system, through driver assistance systems in levels 1 to 2, to partially autonomous (levels 3 and 4) and fully autonomous (level 5) systems, which no longer require a driver . An autonomous vehicle (also known as a driverless car, self-driving car and robotic car) is a vehicle that is able to sense its surroundings and navigate without human input and corresponds to SAE automation level 5. The parking assistance system has a particular Degree of automation greater than or equal to 4.

According to a fourth aspect, a vehicle with an optical sensor unit for detecting sen and outputting an indicative of the area around the vehicle optical sensor signal and proposed with a parking assistance system according to the third aspect.

The vehicle is, for example, a passenger car or a truck. The vehicle preferably includes a number of sensor units that are set up to detect the driving state of the vehicle and to detect an environment of the vehicle. Examples of such sensor units of the vehicle are image recording devices such as a camera, radar (radio detection and ranging) or also a lidar (engl. light detection and ranging), ultrasonic sensors, location sensors, wheel angle sensors and/or wheel speed sensors. The sensor units are each set up to output a sensor signal, for example to the parking assistance system, which carries out the partially autonomous or fully autonomous driving as a function of the detected sensor signals.

Further possible implementations of the invention also include combinations of features or embodiments described above or below with regard to the exemplary embodiments that are not explicitly mentioned. The person skilled in the art will also add individual aspects as improvements or additions to the respective basic form of the invention.

• 1 zeigt eine schematische Ansicht eines Fahrzeugs aus einer Vogelperspektive;
• 2 zeigt ein Beispiel für einen Datensatz;
• 3 zeigt eine schematische Ansicht einer beispielhaften Betriebssituation eines Parkassistenzsystems;
• 4 zeigt eine schematische Ansicht einer beispielhaften Soll-Anordnung eines Markers;
• 5 zeigt eine schematische Ansicht einer beispielhaften Ist-Anordnung eines Markers;
• 6 zeigt ein schematisches Blockschaltbild eines Ausführungsbeispiels für ein Parkassistenzsystem; und
• 7 zeigt ein schematisches Blockschaltbild eines Ausführungsbeispiels für ein Verfahren zum Betreiben eines Parkassistenzsystems.

Further advantageous refinements and aspects of the invention are the subject matter of the dependent claims and of the exemplary embodiments of the invention described below. The invention is explained in more detail below on the basis of preferred embodiments with reference to the enclosed figures.

• 1 Fig. 12 shows a schematic view of a vehicle from a bird's eye view;
• 2 shows an example of a record;
• 3 shows a schematic view of an exemplary operating situation of a parking assistance system;
• 4 shows a schematic view of an exemplary target arrangement of a marker;
• 5 shows a schematic view of an exemplary actual arrangement of a marker;
• 6 shows a schematic block diagram of an exemplary embodiment of a parking assistance system; and
• 7 shows a schematic block diagram of an exemplary embodiment of a method for operating a parking assistance system.

Elements that are the same or have the same function have been provided with the same reference symbols in the figures, unless otherwise stated.

1 FIG. 12 shows a schematic view of a vehicle 100 from a bird's eye view. The vehicle 100 is, for example, a car that is arranged in an environment 200 . Car 100 has a parking assistance system 110, which is embodied as a control unit, for example. In addition, a plurality of surroundings sensor devices 120 are arranged on the car 100 , these being optical sensors 120 , for example. The optical sensors 120 include, for example, visual cameras, a radar and/or a lidar. The optical sensors 120 can each capture an image of a respective area from the environment 200 of the car 100 and output it as an optical sensor signal.

The car 100 can further in which 1 have sensors that are not shown, such as an ultrasonic sensor, a wheel speed sensor, a steering angle sensor, a microphone, an acceleration sensor, an antenna with a coupled receiver for receiving electromagnetically transmittable data signals, and the like.

The parking assistance system 110 is, for example, as shown in FIG 6 explained trained and is to perform the basis of 7 procedure explained set up.

In particular, parking assistance system 110 is set up to control vehicle 100 autonomously. This is based in particular on the sensor signals received from the optical sensors 120 and various other sensor units. The parking assistance system 110 advantageously has at least a level 4 automation level according to the SAE-J3016 classification system or higher. It should be noted that vehicle 100 has four optical sensor units 120 here, merely as an example, but a vehicle with only one optical sensor unit 120, in particular a front camera, can also be set up to carry out the methods explained below.

2 shows an example of a data record DATA. The data record DATA includes an assignment of five pieces of first information POS, OR, AR, CNT, VIS to a respective marker ID1-ID4 for a number of clearly distinguishable markers ID1-ID4. In this example, the DATA record is presented in the form of a table, but this is not meant to be limiting.

In the first column, four different ARUCO markers are shown as examples. Everyone ARUCO marker encodes a unique identification number ID according to a predetermined scheme. The parking assistance system 110 (see 1 or 6 ) is set up to determine the identification number ID according to the predetermined scheme based on the optical sensor signal.

Position information POS comprising two coordinates x, y is assigned to each of the markers ID1-ID4 in the second column.

In the third column, orientation information OR is assigned to each of the markers ID1 - ID4, which in this case indicates an orientation of a surface normal of the respective marker relative to a lane 300 (see 3 ) of the vehicle 100 (see 1 or 3 ) indicates.

In the fourth column, an item of arrangement information AR is assigned to each of the markers ID1-ID4, which information specifies, for example, how the respective marker is arranged relative to other features in the environment 200. This is based on the 4 and 5 explained in more detail.

In the fifth column, counting information CNT is assigned to each marker ID1-ID4, which indicates, for example, how many other markers should have already been detected by vehicle 100 along a predetermined trajectory. This is based on the below 3 explained in more detail.

In the sixth column, visibility information VIS is assigned to each of the markers ID1-ID4, which indicates, for example, which of the markers ID1-ID4 can be visible at the same time or should be visible at the same time.

3 shows a schematic view of an exemplary operating situation of a parking assistance system 110 that controls a vehicle 100 autonomously. The vehicle is, for example, based on the 1 explained trained. In the 3 a schematic section of a multi-storey car park is shown. The multi-storey car park is equipped with optically detectable markers ID1-ID4, which are arranged at specific positions in the multi-storey car park, so that vehicles 100 moving in the multi-storey car park can detect them and use them for orientation. For this purpose, a data record DATA (see 2 ) which includes the information required for orientation, for example position information, information about the travel corridor 300 and the like. The data record DATA can also include a digital map of the parking garage, so that the parking assistance system 110 can plan a route from a starting point, for example the entrance to the parking garage, to a destination, for example a specific parking space. However, the path planning can also take place incrementally, with the parking assistance system 110 only planning the path from a respective marker ID1-ID4 to the next marker ID1-ID4.

The data set DATA also includes at least one assigned first piece of information POS, OR, AR, CNT, VIS for each marker ID1 - ID4 (see 2 ). This serves to check the markers ID1-ID4 detected and recognized by the parking assistance system 110, the plausibility check being carried out using at least one second piece of information which is in particular independent of the first piece of information POS, OR, AR, CNT, VIS. The second information is ascertained, for example, on the basis of the received data set DATA and/or the received optical sensor signal and/or a further sensor signal received from a further sensor unit of vehicle 100 . The second information relates, for example, to odometrically determined position information of the vehicle 100, current orientation information and/or arrangement information of the respective marker ID1 - ID4 determined as a function of the received optical sensor signal, current counting information relating to the markers ID1 - ID4 and/or current visibility information of the markers ID1-ID4 determined as a function of the received optical sensor signal.

Vehicle 100 is controlled by parking assistance system 110 within driving corridor 300, preferably on the basis of data set DATA and detected markers ID1-ID4. in the in 3 illustrated situation, the vehicle 100 is just next to the marker ID1. A front camera 120 (see 1 ) of the vehicle 100 detects, for example, the marker ID2 and the marker ID3 simultaneously in this position, which means that these markers are contained in an optical sensor signal. In order to use the markers ID2, ID3 for orientation, they are first checked for plausibility by the parking assistance system 110. It can thus be avoided that a misorientation occurs, for example on the basis of a marker arranged at an incorrect position or a marker that has been incorrectly identified. Such a misorientation could result, for example, in the driving corridor 300 being determined incorrectly, which could result in a collision with an object in the area.

The plausibility check is carried out, for example, on the basis of all five items of first information POS, OR, AR, CNT, VIS present in the data record DATA for the two markers ID2, ID3. First it is checked whether the target position x1,y1; x2, y2 of the respective marker ID2, ID3 matches a determined actual position or at least within one predetermined tolerance range. The actual position is determined on the basis of the odometrically determined vehicle position and a position of the marker ID2, ID3 relative to the vehicle 100 determined on the basis of the received optical sensor signal. The result of this check can be output, for example, in the form of a numerical value that is a measure of the reliability of the plausibility check. It is also checked whether the actual orientation of the visible markers ID2, ID3 corresponds to the target orientation OR, which is the case in this case because a surface normal of marker ID2 is perpendicular to the driving corridor 300 (90°) and the surface normal of marker ID3 stands parallel to driving corridor 300 (0°). Furthermore, the arrangement AR of the two markers ID2, ID3 is checked, which is based on the 4 and 5 is explained.

4 shows a schematic view of an exemplary target arrangement AR of the marker ID3. In this case, the target arrangement AR relates to features in the vicinity of the marker ID3, these features being an edge between the wall and a ceiling, an edge between the wall and a floor, and a door 220 in this example. The target arrangement AR includes, for example, a respective distance D1-D3 from these features. It should be noted that the target arrangement AR can in particular also include a rotation angle of the marker ID3, which indicates a rotation of the marker ID3 by a specific angle, for example.

5 shows an example of an actual arrangement of the marker ID3. This is determined, for example, on the basis of the received optical sensor signal. The marker ID3, for example, has broken out of its holder on the wall and is now lying on the ground, which is why at least the distances D1 and D2 (in the 5 not shown) not those of the target arrangement AR, as in the 4 shown correspond. In this case, the marker ID3 would not be checked for plausibility, for example.

Back to 3 count information CNT is also checked. Here, the parking assistance system 110 determines, for example, how many markers ID1-ID4 it has passed along the trajectory driven. The marker ID1 is the first marker that was passed, which is why the counting information CNT = 1 is correct. The same also applies to the markers ID2 and ID3. Finally, the visibility information VIS is also checked. In the situation shown, markers ID2 and ID3 are visible. This simultaneous visibility is stored as a possible variant in the visibility information VIS as plausible by the information (ID2, ID3) in brackets. The visibility information VIS can include different groups of markers, which is shown here by the brackets. For example, marker ID3 is included in three groups. Thus, the marker ID3 can be visible at the same time as the markers ID1 and ID2, or be visible at the same time as the marker ID2 (depicted situation) or is only visible alone (when the vehicle 100 has passed the marker ID2).

In the present case, for example, both markers ID2 and ID3 are checked for plausibility and parking assistance system 110 carries out the path planning based on the information contained in data record DATA about these markers ID2, ID3 and its relative position to these markers ID2, ID3.

6 shows a schematic block diagram of an embodiment of a parking assistance system 110 for a vehicle 100 (see 1 or 3 ). The parking assistance system 110 is set up at least for the semi-autonomous control of the vehicle 100 . Parking assistance system 110 includes a receiving unit 112 for receiving a data set DATA (see 2 ) comprising an assignment of at least one respective first piece of information POS, OR, AR, CNT, VIS (see 2 ) to a respective marker ID1 - ID4 (see 2 - 4 ) for several clearly distinguishable markers ID1 - ID4, and for receiving an optical sensor signal of an environment 200 (see 1 ) of the vehicle 100 from a sensor unit 120 (see 1 ) of vehicle 100 during an autonomous drive of vehicle 100, with at least one specific marker of the plurality of markers ID1 - ID4 being contained in the optical sensor signal, a determination unit 114 for determining the specific marker ID1 - ID4 contained in the received optical sensor signal, for calling up the associated first information POS, OR, AR, CNT, VIS, and for determining second information as a function of the received data set DATA and/or the received optical sensor signal and/or a further sensor signal received from a further sensor unit of vehicle 100, a plausibility check unit 116 for checking the plausibility of the specific marker ID1 - ID4 depending on a comparison of the retrieved first information POS, OR, AR, CNT, VIS with the ascertained second information, and a control unit 118 for carrying out the autonomous driving using the retrieved first information and/or others , in the dataset DATA information stored about the plausibility-checked marker ID1-ID4, depending on the plausibility check.

7 FIG. 12 shows a schematic block diagram of an exemplary embodiment of a method for operating a parking assistance system 110 for example the parking assistance system 110 of 1 , 3 or 6 . The parking assistance system 110 is set up at least for the semi-autonomous control of a vehicle 100, for example the vehicle 100 of FIG 1 or 3 . In a first step S1, a data set DATA (see 2 ) comprising an assignment of at least one respective first piece of information POS, OR, AR, CNT, VIS (see 2 ) to a respective marker ID1 - ID4 (see 2 - 4 ) received for several clearly distinguishable markers ID1 - ID4. In a second step S2, an optical sensor signal of an environment 200 (see 1 ) of the vehicle 100 from a sensor unit 120 (see 1 ) of the vehicle 100 during an autonomous drive of the vehicle 100, wherein the optical sensor signal contains at least one specific marker ID1-ID4 of the plurality of markers ID1-ID4. In a third step S3, the specific marker ID1-ID4 contained in the received optical sensor signal is determined and the associated first information item POS, OR, AR, CNT, VIS is called up. In a fourth step S4, second information is ascertained as a function of the received data set DATA and/or the received optical sensor signal and/or a further sensor signal received from a further sensor unit of vehicle 100. In a fifth step S5, the specific marker ID1-ID4 is checked for plausibility as a function of a comparison of the retrieved first information item POS, OR, AR, CNT, VIS with the ascertained second information item. In a sixth step S6, the autonomous journey is continued using the retrieved first information POS, OR, AR, CNT, VIS and/or other information stored in the data set DATA for the specific marker ID1 - ID4 if the specific marker ID1 - ID4 is checked for plausibility, or the retrieved first information POS, OR, AR, CNT, VIS and/or the other information stored in the data record DATA for the specific marker ID1-ID4 is discarded if the specific marker ID1-ID4 is not checked for plausibility .

Although the present invention has been described using exemplary embodiments, it can be modified in many ways.

Reference List

100

vehicle

110

parking assistance system

112

receiving unit

114

investigation unit

116

plausibility unit

118

control unit

120

sensor unit

200

vicinity

210

pillar

220

door

300

lane

AR

arrangement information

AR1 - AR4

arrangement

CNT

counting information

D1

Distance

D2

Distance

D3

Distance

DATA

record

ID

Marker ID

ID1-ID4

marker

OR

orientation information

POS

position information

S1 - S6

process steps

VIS

visibility information

x1 - x4

x position

y1 - y4

y position

Claims (15)

Method for operating a parking assistance system (110) for a vehicle (100), wherein the parking assistance system (110) is set up at least for partially autonomous control of the vehicle (100), the method having the steps: a) receiving (S1) a data set (DATA) comprising an assignment of at least one piece of first information (POS, OR, AR, CNT, VIS) to a respective marker (ID1 - ID4) for a plurality of clearly distinguishable markers (ID1 - ID4), b) receiving (S2) an optical sensor signal Environment (200) of the vehicle (100) from a sensor unit (120) of the vehicle (100) during autonomous driving of the vehicle (100), wherein in the optical sensor signal at least one specific marker (ID1 - ID4) of the plurality of markers (ID1 - ID4) is included, c) determining (S3) contained in the received optical sensor signal specific marker (ID1 - ID4) and calling up the associated first information (POS, OR, AR, CNT, VIS), d) determining (S4) a second Information depending on the received data set (DATA) and/or the received optical sensor signal and/or a further sensor signal received from a further sensor unit of the vehicle (100), e) plausibility checking (S5) of the specific marker (ID1 - ID4) depending on a comparison of on called first information (POS, OR, AR, CNT, VIS) with the determined second information, and f) using (S6) the called first information (POS, OR, AR, CNT, VIS) and / or another, in which Data set (DATA) for the specific marker (ID1 - ID4) stored information to continue the autonomous journey if the specific marker (ID1 - ID4) is plausible, or discard the first information called (POS, OR, AR, CNT, VIS) and/or the additional information stored in the data record (DATA) for the specific marker (ID1-ID4) if the specific marker (ID1-ID4) has not been checked for plausibility. procedure after claim 1 , characterized in that the assigned first information is a position information (POS) of the respective marker (ID1 - ID4), an orientation information (OR) of the respective marker (ID1 - ID4), an arrangement information (AR) of the respective marker (ID1 - ID4) , counting information (CNT) of the respective marker (ID1-ID4) and/or visibility information (VIS) of the respective marker (ID1-ID4). procedure after claim 1 or 2 , characterized in that the determined second information is an odometrically determined position information of the vehicle (100), a determined depending on the optical sensor signal received current orientation information and / or arrangement information of the specific marker (ID1 - ID4), depending on a course of the autonomous journey determined current counting information of the specific marker (ID1-ID4) and/or current visibility information of the specific marker (ID1-ID4) determined as a function of the received optical sensor signal. Method according to one of the preceding claims, characterized in that the assigned first information includes position information (POS) of the specific marker (ID1 - ID4) and that the determined second information includes odometrically determined position information of the vehicle (100), wherein step e ) comprises: determining a difference value of the position information (POS) of the specific marker (ID1 - ID4) and the position information of the vehicle (100), and comparing the difference value with a predetermined threshold value and/or a predetermined interval, the specific marker (ID1 - ID4) is checked for plausibility if the difference value is less than or equal to the predetermined threshold value and/or is within the predetermined interval. procedure after claim 4 , characterized by : determining a relative position of the vehicle (100) relative to the specific marker (ID1 - ID4) contained in the received optical sensor signal as a function of the received optical sensor signal, determining a current vehicle position as a function of the determined relative position and the position information (POS) of the determined marker (ID1 - ID4), and calibrating the odometrically determined position information of the vehicle (100) on the basis of the determined current vehicle position. Method according to one of the preceding claims, characterized in that the assigned first information comprises orientation information (OR) of the specific marker (ID! - ID4) which defines a target orientation of the specific marker (ID1 - ID4) relative to a travel corridor (300 ) includes, and wherein the determined second information includes current orientation information, which includes an actual orientation of the specific marker (ID1 - ID4) determined as a function of the received optical sensor signal, wherein step e) includes: determining a difference value of the target orientation (OR) of the specific marker (ID1 - ID4) and the determined actual orientation of the specific marker (ID1 - ID4), and comparing the difference value with a predetermined threshold value, the specific marker (ID1 - ID4) being checked for plausibility if the difference value is less than or equal to the predetermined threshold. Method according to one of the preceding claims, characterized in that the assigned first information includes an arrangement information (AR) of the specific marker (ID1 - ID4) which a target arrangement of the specific marker (ID1 - ID4) relative to a feature (220) in the vicinity of the specific marker (ID1 - ID4), and wherein the ascertained second information comprises current arrangement information which shows an actual arrangement of the specific marker (ID1 - ID4) relative to the feature (220 ), wherein step e) comprises: determining a difference value of the target arrangement (AR) of the specific marker (ID1 - ID4) and the determined actual arrangement of the specific marker (ID1 - ID4), and comparing the difference value with a predetermined one Threshold value, the specific marker (ID1 - ID4) being checked for plausibility if the difference value is less than or equal to the predetermined threshold value. Method according to one of the preceding claims, characterized in that the associated first information includes position information (POS) of the specific marker (ID1 - ID4) and that the determined second information includes odometrically determined position information of the vehicle (100), wherein in the received optical sensor signal contains at least two markers (ID1 - ID4), step e) comprising: determining a target relative position of the markers (ID1 - ID4) contained in the optical sensor signal on the basis of the respective assigned position information (POS), determining an actual - Relative position of the markers (ID1 - ID4) contained in the optical sensor signal based on a respective relative position of the vehicle (100) to the respective marker and the position information of the vehicle (100) determined on the basis of the optical sensor signal, and comparing the target relative position with the actual relative position, which in the optical sensor signal contained markers (ID1 - ID4) are checked for plausibility if a difference between the target relative position and the actual relative position is less than or equal to a predetermined threshold value. Method according to one of the preceding claims, characterized in that the assigned first information comprises visibility information (VIS) of the specific marker, which indicates which other of the plurality of markers (ID1 - ID4) can be contained simultaneously in the received optical sensor signal, the Step d) includes: Determining which of the plurality of markers (ID1 - ID4) are contained in the optical sensor signal, wherein step e) includes: Plausibility checking of the specific marker (ID1 - ID4) if the determined markers (ID1 - ID4) of the corresponding to visibility information (VIS) of the particular marker (ID1 - ID4). procedure after a Claims 4 until 9 , characterized in that a plausibility index is determined for the specific marker (ID1 - ID4), which is a measure of how reliable the plausibility check of the specific marker (ID1 - ID4) is, the plausibility index being based on one or more of the claims 4 until 9 claimed plausibility methods is determined, and wherein the parking assistance system (110) deviates from a predetermined driving corridor (300), which is assigned to the specific marker (ID1 - ID4), depending on the plausibility index. Method according to one of the preceding claims, characterized in that step f) comprises: Continuing the autonomous driving on the basis of received sensor signals, without the called first information (POS, OR, AR, CNT, VIS) and / or the other, in which Data set (DATA) to use the specific marker (ID1 - ID4) stored information if the specific marker (ID1 - ID4) has not been checked for plausibility. Method according to one of the preceding claims, characterized in that a reliability measure for the reliability of determining the specific marker (ID1 - ID4) is determined in step c) as a function of the received optical sensor signal, steps d) - f) only then being carried out if the determined degree of reliability is greater than or equal to a specific threshold value. Computer program product, comprising instructions which, when the program is executed by a computer, cause the latter to carry out the method according to one of Claims 1 - 12 to execute. Parking assistance system (110) for a vehicle (100), wherein the parking assistance system (110) is set up at least for partially autonomous control of the vehicle (100), with: a receiving unit (112) for receiving a data record (DATA) comprising an assignment of at least one piece of first information (POS, OR, AR, CNT, VIS) to a respective marker (ID1 - ID4) for a plurality of clearly distinguishable markers (ID1 - ID4 ), and for receiving an optical sensor signal of an environment (200) of the vehicle (100) from a sensor unit (120) of the vehicle (100) during an autonomous drive of the vehicle (100), wherein in the optical sensor signal at least one specific one of the plurality of markers (ID1 - ID4) is included, a determination unit (114) for determining the specific marker (ID1 - ID4) contained in the received optical sensor signal, for calling up the associated first information (POS, OR, AR, CNT, VIS), and for determining second information as a function of the received information data set (DATA) and/or the received optical sensor signal and/or another sensor signal received from another sensor unit of the vehicle, a plausibility check unit (116) for checking the specific marker (ID1 - ID4) for plausibility as a function of a comparison of the retrieved first information item (POS, OR, AR, CNT, VIS) with the ascertained second information item, and a control unit (118) for carrying out the autonomous journey using the retrieved first information (POS, OR, AR, CNT, VIS) and/or additional information stored in the data set (DATA) for the specific marker (ID1 - ID4). depending on the plausibility check. Vehicle (100) with an optical sensor unit (120) for detecting and outputting an optical sensor signal indicative of the area (200) of the vehicle (100) and with a parking assistance system (110). Claim 14 .

Images