DE102024207833A1 - Method for controlling at least a partially autonomous vehicle using voice input - Google Patents

Abstract

The invention relates to a method for controlling an at least partially autonomous vehicle by a driver (100) using voice input. The method comprises the following steps:
Capture, by means of a microphone (303), the driver's voice input (100);
Determine, using a speech recognition unit (140), a potential driving instruction from the driver's speech input (100);
Comparing the potential driving instruction with a list of possible driving instructions, wherein the list of possible driving instructions is a function of possible driving instructions of an at least semi-autonomous vehicle control unit (190);
If the potential driving instruction is included in the list of possible driving instructions:
Output, by means of a loudspeaker (103), an instruction to the driver (100), wherein the instruction is a function of the driving instruction;
Verification unit (150) checks the driver's (100) response to the output of the instruction; and
If, by means of the verification unit (150), the driver's response (100) is judged to be correct, the potential driving instruction is used as an actuator instruction to control the actuators (200) of the vehicle by means of the at least semi-autonomous vehicle control unit (190).

Description

Field of invention

The invention relates to a method for controlling an at least partially autonomous vehicle by a driver, in particular by means of voice input. The invention further relates to a device, a vehicle, a use, and a non-volatile, computer-readable storage medium.

background

Current technology includes voice input systems in vehicles that allow for the control of a variety of non-safety-critical vehicle functions. However, it would be a significant relief for the driver if a partially autonomous vehicle could also be controlled via voice input for safety-critical functions. This requires measures to verify the voice input.

Summary

The object of the invention is to provide a method by which a driver's voice input can be verified. This object is achieved by the subject matter of the independent claims. Further developments of the invention are described in the dependent claims and the following description.

• ausgeben, mittels eines Lautsprechers, einer Handlungsanweisung an den Fahrer, wobei die Handlungsanweisung eine Funktion der Fahranweisung ist;
• prüfen, mittels der Verifizierungseinheit, einer Reaktion des Fahrers auf das Ausgeben der Handlungsanweisung;
• wenn, mittels der Verifizierungseinheit, die Reaktion des Fahrers als korrekt beurteilt wird, verwenden der potentiellen Fahranweisung als Aktoranweisung zum Steuern der Aktoren des Fahrzeugs mittels der zumindest teilautonomen Fahrzeugsteuerungseinheit.

One aspect concerns a method for controlling an at least partially autonomous vehicle by a driver using speech input, comprising the steps of: capturing, by means of a microphone, the driver's speech input; determining, by means of a speech recognition unit, a potential driving instruction from the driver's speech input; comparing the potential driving instruction with a list of possible driving instructions, wherein the list of possible driving instructions is a function of possible driving instructions of an at least partially autonomous vehicle control unit; if the potential driving instruction is included in the list of possible driving instructions:

• issue, by means of a loudspeaker, an instruction to the driver, the instruction being a function of the driving instructions;
• to check, using the verification unit, a reaction of the driver to the output of the instruction;
• If, by means of the verification unit, the driver's reaction is judged to be correct, the potential driving instruction is used as an actuator instruction to control the vehicle's actuators by means of the at least semi-autonomous vehicle control unit.

A vehicle can be a road-bound vehicle, such as a passenger car, a truck, or a special-purpose vehicle. The vehicle can also be, for example, a boat. The vehicle is at least a partially autonomous vehicle, i.e., a vehicle of at least Level 1 according to SAE standard J3016. The speech input is natural language input, meaning that in at least some embodiments, it may be permissible for not just one term to be understood as an instruction to the vehicle's actuators—or to the vehicle control unit, which controls the actuators at least partially autonomously—but rather for a multitude of terms to be permissible. For example, instead of just "slow down," terms such as "slower," "not so fast," etc., can also be considered actuator instructions, i.e., potential driving instructions. Even complex driving maneuvers, such as overtaking a vehicle in front or stopping the vehicle in the next parking space, can be considered a possible actuator instruction to the at least partially autonomous vehicle control unit in this context.

The driver's voice input is captured using a microphone. The microphone can be, for example, a dynamic microphone, condenser microphone, electret condenser microphone, piezoelectric microphone, and/or another type of microphone. The microphone can be configured as a multiple microphone array, for example, as a microphone field or microphone array. The microphone can be used to improve the This includes speech quality, particularly when utilizing multiple microphones. A microphone is considered particularly suitable for this procedure if it is able to clearly capture the driver's voice and also sound from a loudspeaker, as described below.

The speech recognition unit can use the signal from the microphone to determine whether the voice input is a potential driving instruction directed at the vehicle's actuators. To do this, the speech recognition unit may not only be able to recognize words from the voice input, but it may also include a list or database containing permissible actuator instructions and their respective meanings. Using the example above, the speech recognition unit recognizes the words "slow down," "slower," and "not so fast" (a) as permissible, i.e., as potential driving instructions, and (b) assigns the meaning "slow down" to all of these words. Other examples of actuator instructions might be: "left," "right," "overtake," "accelerate," etc. If the words are not recognized as actuator instructions, the voice input can, for example, be ignored. In any case, words that are not recognized as actuator instructions or potential driving instructions are not forwarded to the vehicle's actuators (or to the vehicle control unit). The vehicle's actuators can include, for example, the drivetrain, brakes, and steering.

If the voice input is recognized as a potential driving instruction, the potential driving instruction is compared with a list of possible driving instructions. This list is a function of the possible driving instructions of an at least semi-autonomous vehicle control unit. In other words, the list of possible driving instructions contains those driving instructions that can be implemented by the vehicle control unit. Comparing the potential driving instruction with the list of possible driving instructions can be particularly easy because the speech recognition unit can output a standardized "meaning" (in the example above, "slow down"). This can be easily compared with similarly standardized terms from the list of possible driving instructions. The list of possible driving instructions may depend on the autonomy level of the vehicle control unit. For example, a potential driving instruction "slow down" can be implemented by most vehicle control units and is therefore included in their list of possible driving instructions. As another example, a potential driving instruction "overtake" can only be implemented by vehicle control units with a higher level of autonomy and therefore only be included in their list of possible driving instructions, but not in the lists of other vehicle control units. If a potential driving instruction is not included in the list of possible driving instructions, this driving instruction, or the voice input, is ignored. An error message may be displayed.

In some implementations, the list of possible driving instructions can be attributed. These attributes can indicate that the list of possible driving instructions depends not only on the autonomy level of the vehicle control unit but also on the current vehicle situation. An example of a dependency on the current vehicle situation is that the potential driving instruction "overtake" can only be implemented (or may only be implemented, according to legal regulations) on a highway by vehicle control units of a certain autonomy level, but not on other roads. If a potential driving instruction is not included in the attributed list of possible driving instructions and/or cannot be implemented in the current vehicle situation, this driving instruction, or the corresponding voice input, is ignored. An error message may be displayed.

If voice input is included in the list of possible driving instructions, an instruction can be issued to the driver. The instruction could be, for example, "Confirm the deceleration by pressing button A" or "Confirm the deceleration by nodding your head." The output can be via a loudspeaker. The loudspeaker can be, for example, a dynamic, electrostatic, piezoelectric loudspeaker, and/or another type of transducer. The loudspeaker can consist of one or more speakers, including separate speakers, each reproducing a specific frequency range. For example, a piezoelectric loudspeaker can be configured to reproduce a frequency range above 22 kHz, and a dynamic loudspeaker can be configured to reproduce a frequency range below 22 kHz. The loudspeaker is preferably located near the driver's head or ears, for example, in a headrest.

The driver can react to the initial instruction. A reaction to the examples above could be, for instance, pressing button A or nodding the head. The verification unit can then assess this reaction as correct. To do this, the verification unit can query one or more sensors. For example, pressing button A can be detected using a haptic sensor – such as a push button. Alternatively, a head nod can be detected using an interior camera configured to capture (at least) this predefined movement of the driver, along with an image recognition unit designed to analyze the movement captured by the camera, so that the verification unit recognizes the driver's head nod as such. If a different reaction from the driver, or no reaction at all, is detected, it will not be assessed as correct. A time window can be defined within which the driver's reaction is considered correct. This time window could, for example, encompass a few seconds after the initial instruction is issued. If the driver's correct reaction is not observed within this time window, then that reaction will also not be considered correct. To ensure that a particular reaction does not become "habitual," the initial instruction can be varied from case to case.

If the driver's reaction is judged to be correct by the verification unit, the actuator instruction can be used to control the vehicle's actuators or forwarded to the at least semi-autonomous vehicle control unit.

This provides a method for verifying driver voice input. This allows verification—and in at least some cases, confirmation—that the driver's voice input was indeed intended as an actuator command. Furthermore, the capabilities of the at least partially autonomous vehicle control unit can be considered during the voice input verification process. This can contribute to a particularly consistent user experience for the vehicle's human-machine interface. Additionally, this can reduce the load on the vehicle control unit, as it is relieved of requests or driving instructions that it cannot execute anyway.

By checking the driver's voice input, potential misunderstandings—such as instructions misinterpreted by the speech recognition unit—can be significantly reduced or even eliminated. Furthermore, errors caused by incorrect language settings (e.g., Japanese instead of German) can be minimized. Additionally, the risk of the driver misunderstanding instructions, for example, due to hearing impairment (e.g., after a loud concert) and/or excessive ambient noise (e.g., in a construction zone), can be reduced.

In some embodiments, the method includes the following additional step: After issuing the instruction, the driver is authenticated using a driver authentication unit. The driver authentication unit can, for example, include iris recognition. This can be implemented, for instance, using an interior camera – possibly in conjunction with a database of authorized drivers. Alternatively or additionally, driver authentication can be achieved using voice recognition. In one embodiment, if authentication is not successful, a further instruction can be issued to the driver, which can be confirmed, for example, by haptic input.

In some embodiments, the method includes the following additional step: After issuing the driving instruction, the driver's fitness to drive is checked using a fitness-to-drive detection unit. Fitness to drive can be assessed, for example, using microphone signals and/or image data from the interior camera. If driver fatigue is detected, the issuance and execution of the driving instruction can be prevented. Instead of the driving instruction, the driver can then be offered, for example, a route to a safe stopping point (e.g., the next rest area).

In some embodiments, the method includes the following additional step: After issuing the instruction, the driver's attention is checked using a proximity sensor. The proximity sensor can be implemented, for example, using a capacitive sensor and/or an IR sensor (IR: infrared). The proximity sensor can be located, for example, in the headrest and/or in an area of the headliner where the driver's head is typically positioned. If the driver's head moves further than a predefined distance from the headrest, it can be assumed that the driver is looking down or to the side and is therefore not paying attention. Optionally, confirmation, for example, via haptic input, can be requested in this situation.

• Ausgeben, mittels des Lautsprechers, eines Kontrollsignals;
• Erfassen, mittels des Mikrofons, des Kontrollsignals;
• Wenn, mittels der Verifizierungseinheit, das Kontrollsignal als korrekt beurteilt wird, verwenden der Aktoranweisung zum Steuern der Aktoren des Fahrzeugs.

In some embodiments, the method includes the following additional steps:

• Output, via the loudspeaker, a control signal;
• Capture the control signal using the microphone;
• If the control signal is judged to be correct by the verification unit, the actuator instruction is used to control the vehicle's actuators.

The control signal can be, for example, an inaudible audio signal with a characteristic modulation. This audio signal could be, for instance, modulated near-ultrasound, e.g., in the 25-30 kHz range. The modulation could be, for example, a predefined signal sequence. If this audio signal with the characteristic modulation is detected, the verification unit can assess the control signal as correct. If this audio signal is not detected, the actuator instruction can be ignored. The audio signal might not be detected, for example, if the ratio of the audio signal's sound pressure level to the ambient noise falls below a predefined value. In this case, the ambient noise might be so loud that the speech recognition unit would be too unreliable for controlling the vehicle via voice input.

In some embodiments, haptic input from the driver is the driver's reaction to the output of the instruction. Haptic input can be determined by a haptic sensor. The haptic sensor can be, for example, a button, switch, rotary knob, pressure sensor, proximity sensor, and/or a capacitive touch sensor on a screen or input field. The haptic sensor can be located, for example, on the steering wheel or in the area of a center console. This can be a particularly intuitive input method because many vehicles use haptic sensors on the steering wheel or in the area of the center console for numerous types of input.

In some embodiments, a predefined movement of the driver is the driver's reaction to the issuance of the instruction. The predefined movement can, for example, include a gesture such as waving or nodding.

In some embodiments, a movement of the steering wheel by the driver is the driver's reaction to the output of the instruction. The movement of the steering wheel can be interpreted as an objection by the driver to the actuator instruction, for example, if the driver wants to steer manually and does not want to transfer the steering to the at least partially autonomous vehicle control unit. The driver's reaction is therefore considered correct if the driver does not move the steering wheel.

In some embodiments, the driver's movement of a pedal is their response to the output of the instruction. The pedal could be, for example, a brake or accelerator pedal, or possibly a clutch pedal. The movement of the pedal can also be interpreted as the driver's objection to the actuator instruction, for instance, if the driver intends to brake or accelerate themselves. The driver's response is therefore considered correct if the driver does not move the pedal.

• Eine Schnittstelle zu einem Mikrofon, das zum Erfassen der sprachlichen Eingabe des Fahrers eingerichtet ist;
• Eine Spracherkennungseinheit, die zum Erkennen, ob die sprachliche Eingabe eine potentielle Fahranweisung ist, eingerichtet ist;
• Eine Schnittstelle zu einer Liste von möglichen Fahranweisungen, wobei die Liste der möglichen Fahranweisungen eine Funktion von möglichen Fahranweisungen einer zumindest teilautonomen Fahrzeugsteuerungseinheit ist;
• Eine Schnittstelle zu einem Lautsprecher, der zum Ausgeben einer Handlungsanweisung an den Fahrer eingerichtet ist, wenn die sprachliche Eingabe in der Liste der möglichen Fahranweisungen enthalten ist;
• Eine Verifizierungseinheit, eingerichtet zum Prüfen einer Reaktion des Fahrers auf das Ausgeben der Handlungsanweisung mittels eines Sensors; und Eine Schnittstelle zu einer zumindest teilautonomen Fahrzeugsteuerungseinheit, eingerichtet zum Steuern der Aktoren des Fahrzeugs auf Basis der der potentiellen Fahranweisung als Aktoranweisung, wenn die Verifizierungseinheit die Reaktion des Fahrers als korrekt beurteilt.

One aspect concerns a device for controlling a vehicle by a driver using voice input. The device features:

• An interface to a microphone designed to capture the driver's voice input;
• A speech recognition unit designed to detect whether the spoken input is a potential driving instruction;
• An interface to a list of possible driving instructions, wherein the list of possible driving instructions is a function of possible driving instructions of an at least partially autonomous vehicle control unit;
• An interface to a loudspeaker that is set up to issue an instruction to the driver when the voice input is included in the list of possible driving instructions;
• A verification unit, equipped to check the driver's response to the output of the instruction by means of a sensor; and an interface to an at least semi-autonomous vehicle control unit, equipped to control the vehicle's actuators based on the potential driving instruction as an actuator instruction, if the verification unit judges the driver's response to be correct.

The device can therefore advantageously build upon existing interfaces already present in many vehicles. This can significantly simplify and/or reduce the cost of controlling a vehicle via voice input.

In some embodiments, the loudspeaker is still configured to output a control signal, the microphone is still configured to receive the control signal from the loudspeaker, and the vehicle control unit is still configured to control the vehicle's actuators based on the actuator command. However, this requires that the verification unit evaluates the control signal from the microphone as correct. In cases where the control signal uses low-frequency ultrasound, the loudspeaker can, for example, be implemented as an additional piezoelectric transducer.

• als ein Haptiksensor, der zum Erfassen einer haptischen Eingabe des Fahrers eingerichtet ist, wobei der Haptiksensor als Taster, Schalter, Drehsteller, Drucksensor, Näherungssensor, und/oder als kapazitiver Berührungssensor eines Bildschirms oder eines Eingabefeldes ausgeführt ist; und/oder
• als eine Innenraumkamera, die zum Erfassen einer vordefinierten Bewegung des Fahrers eingerichtet ist, und
• eine Bilderkennungseinheit, die zum Auswerten der von der Innenraumkamera erfassten Bewegung des Fahrers eingerichtet ist; und/oder
• als ein Lenkradbewegungssensor, der zum Erfassen einer Bewegung eines Lenkrads durch den Fahrer eingerichtet ist; und/oder
• als ein Pedalbewegungssensor, der zum Erfassen einer Bewegung eines Pedals durch den Fahrer eingerichtet ist.

In some embodiments, the sensor is designed as at least one of the following devices:

• as a haptic sensor designed to detect haptic input from the driver, wherein the haptic sensor is designed as a button, switch, rotary knob, pressure sensor, proximity sensor, and/or as a capacitive touch sensor of a screen or input field; and/or
• as an interior camera designed to capture a predefined movement of the driver, and
• an image recognition unit designed to evaluate the driver's movements captured by the interior camera; and/or
• as a steering wheel motion sensor designed to detect movement of a steering wheel by the driver; and/or
• as a pedal motion sensor designed to detect movement of a pedal by the driver.

• eine Schnittstelle zu einer Fahrerauthentifizierungseinheit;
• eine Schnittstelle zu einer Fahrtüchtigkeitserkennungseinheit; und/oder
• eine Schnittstelle zu einem Näherungssensor.

In some embodiments, the device further comprises at least one of the following:

• an interface to a driver authentication unit;
• an interface to a fitness-to-drive assessment unit; and/or
• an interface to a proximity sensor.

One aspect concerns a vehicle with a device as described above and/or below.

One aspect concerns the use of a device as described above and/or below for the safe control of a vehicle by a driver using voice input.

One aspect concerns a non-volatile, computer-readable storage medium containing a program which, when executed on a processor of a device as described above and/or below, instructs the device to perform the steps as described above and/or below.

For further clarification, the invention is described with reference to embodiments illustrated in the figures. These embodiments are to be understood as examples only, and not as limitations.

Brief description of the characters

• 1 schematisch eine Vorrichtung zum Steuern eines Fahrzeugs gemäß einer Ausführungsform;
• 2 schematisch ein System zum Steuern eines Fahrzeugs gemäß einer weiteren Ausführungsform;
• 3 ein Flussdiagramm mit einem Verfahren gemäß einer Ausführungsform.

This shows:

• 1 schematically a device for controlling a vehicle according to one embodiment;
• 2 schematically a system for controlling a vehicle according to a further embodiment;
• 3 a flowchart with a method according to one embodiment.

Detailed description of embodiments

1 Figure 1 schematically shows a device for controlling a vehicle according to one embodiment. A driver 100 is shown, who uses a microphone 303 to control the at least partially autonomous vehicle by means of voice input. The microphone 303 can be configured as a single microphone; more typically, the microphone 303 is configured as a plurality of microphones, e.g., as a microphone field or microphone array. The microphone 303 can include measures for improving speech quality, particularly by utilizing the plurality of microphones, e.g., implemented by means of a microcontroller. The microphone 303 can be a microphone that is "already present" in the vehicle (e.g., for implementing a hands-free system), so that only an interface to the microphone 303 needs to be provided, which supplies the microphone signal Ma, possibly optimized. The voice input (i.e., in the form of the microphone signal Ma) is directed to a speech recognition unit 140, which can recognize whether the voice input is an actuator instruction. The speech recognition unit 140 can not only recognize words from the spoken input, but it can also include a list or database (not shown) in which permissible actuator instructions and their respective meanings are stored. The actuator instruction Ft is then sent to a verification unit 150.

The verification unit 150 has an interface Lf to the at least partially autonomous vehicle control unit 190. The vehicle control unit 190 includes a list of possible driving instructions that can be implemented by the vehicle control unit and/or by means of the actuators 200. The list of possible driving instructions can be compatible with the terms output by the speech recognition unit 140.

The verification unit 150 can optionally have additional interfaces. For example, in 1 An interface to a driver authentication unit 170 is shown. The driver authentication unit 170 can, for example, include iris recognition. This can be implemented, for instance, using the interior camera 120. The driver authentication unit 170 can also include a database of authorized drivers (not shown), for example, to compare the result of the iris recognition with a driver profile. 1 It also shows an interface to a fitness-to-drive assessment unit 160. Fitness to drive can be assessed, for example, based on signals from the microphone 303 and/or on image data from the interior camera 120. 1 Figure 165 further shows an interface to a proximity sensor 165. The proximity sensor 165 can be implemented, for example, using a capacitive sensor and/or an IR sensor (IR: infrared). The proximity sensor can be located, for example, in the headrest and/or in an area of the headliner where the driver's head 100 is usually located.

The verification unit 150 also has an interface Sa to a loudspeaker 103, which can be used to issue an initial instruction to the driver 100. This initial instruction is only issued if the voice input is recognized as an actuator instruction. The initial instruction could be, for example, "Confirm the actuator instruction by pressing button A" or "Confirm the actuator instruction by nodding your head." Pressing button A can be detected, for example, by means of a haptic input device, such as a button located on a steering wheel 110. A head nod can be detected by means of an interior camera 120, which is configured to capture the driver's movements, together with an image recognition unit (not shown) that is configured to evaluate the driver's movements captured by the interior camera and provides a signal K. In this way, the driver's head nod can be recognized by the verification unit 150 as a "head nod." The verification unit 150 can further check whether signals P are present from the pedals 112 or whether signals L are present from the steering wheel 110. The signals P can be determined by a pedal sensor 113. The signals L from the steering wheel 110 can be determined by a steering wheel sensor 114. The movement of the steering wheel 110 and/or one of the pedals 112 can be interpreted as an objection by the driver to the actuator instruction, for example, if the driver wants to steer, brake, and/or accelerate themselves and does not want to transfer these actions to at least a partially autonomous vehicle control unit.

The verification unit 150 checks the driver's 100 response to the output of the instruction using one of the aforementioned sensors. If the verification unit 150 assesses the driver's 100 response as correct, the actuator instruction is transmitted via interface Af to the at least semi-autonomous vehicle control unit 190, which controls the vehicle's actuators 200 via interface Ak.

2 Figure 1 schematically shows a system for controlling a vehicle according to a further embodiment. Identical reference numerals denote identical or similar elements as in Figure 2. 1 In 2 A person 100 is depicted – hereinafter referred to as the “driver” 100 – sitting in a driver’s seat 101. The steering wheel 110 has easily accessible haptic input instruments 114 for the driver 100, which offer various input options, e.g., input buttons or input fields. The signals B from the haptic input instruments 114 and signals L from the steering wheel 110 are transmitted to a user control unit 180. The system includes an interior camera 120. This can be used to detect driver reactions to action prompts Fa, which may also relate to driving instructions assigned by the verification unit 150. The action prompts Fa are transmitted via a voice output 130 to one or more loudspeakers 103, which are advantageously located near the driver’s head, e.g., in a headrest. The system has a microphone array comprising several microphones 303, 304, which are positioned in the vehicle in such a way as to optimally capture speech input from the driver 100 and audio signals from the loudspeaker 103.

A sound evaluation unit 302 can be provided for processing the signals from microphones 303 and 304. The sound evaluation unit 302 can separate the driver's voice input 100 from the audio signals of the loudspeaker 103 and can combine the microphone signals from 303 and 304, e.g., by filtering and/or delaying them, so that they amplify the voice input and the loudspeaker signals while attenuating other sound sources inside and outside the vehicle. The combined microphone signal Ma is then output to an audio output unit 130, a speech recognition unit 140, and optionally to a driver fitness detection unit 160 and a driver authentication unit 170. The audio output unit 130 generates the audio signal Sa. It can also receive action prompts Fa from the verification unit 150 and, if necessary, forward them to the user control unit 180. 2 Both signal paths are only described with the designation "Fa". Furthermore, it can generate an audio signal such that, for example, based on a characteristic modulation from the microphone signal Ma that is imperceptible to humans, it can be determined that the output acoustic signal Sa is the speech reproduction of a digital output signal Fa from the secure verification unit 150, e.g., a call to action. Additionally, the audio output unit 130 can calculate the ratio SN of a disturbing ambient noise to the desired signal Sa by comparing the microphone signal Ma with the input signal Fa. If the signal-to-noise ratio SN is too low or it cannot be confirmed that the emitted acoustic signal Sa is the speech reproduction of a digital output signal Fa of the secure verification unit 150, the forwarding of an action request Fa to the user control 180 by the audio output unit 130 can be suppressed or the output of Fa to the user control 180 can be replaced by an error message, because the action request Fa and the possibly contained driver instruction are not transmitted due to a technical malfunction (e.g. of the loudspeaker 301 or the audio output 130) or cannot be understood by the driver 100 due to excessive ambient noise.

Furthermore, the system features a speech recognition unit 140, which recognizes the driver's instructions Ft in the combined microphone signal Ma. These instructions are then forwarded to the verification unit 150. Based on the recognized voice instruction from the driver Ft, and if a clear match can be found in a list Lf of possible driving instructions provided by the autonomous vehicle control unit 190, the verification unit 150 deduces a specific action request Fa corresponding to a particular driving instruction from the list Lf. This action request Fa is then sent to the audio output unit 130. If this action request for the driving instruction is confirmed by the audio output unit through the evaluation of the combined microphone signals Ma, it is forwarded to the user control unit 180. This unit then sends an actuator instruction Af, corresponding to the driving instruction Fa, to the at least partially autonomous vehicle control unit 190. The vehicle control unit can then generate the necessary control signals Ak to execute the actuator instruction Af. The at least semi-autonomous vehicle control unit 190 uses vehicle sensors 192 (e.g. ultrasonic sensors, radar sensors, LiDAR and external camera sensors) to detect the vehicle environment to control the actuators 200, e.g. drives ("power train"), brakes and steering of the vehicle via control signals Ak.

2 The diagram also shows an optional driver authentication unit 170, which can be implemented, for example, based on iris recognition with an interior camera, and/or voice recognition, and a list of drivers or driver profiles. If authentication fails, for example, the driver's voice is not recognized via the "Na" signal, a haptic operating alternative can be suggested. 2 Furthermore, an optional fitness-to-drive detection unit 160 is shown, which, for example, assesses fitness to drive based on microphone signals Ma and interior camera image data Kd, and, for example, if fatigue is detected, prevents the issuance and execution of a driving instruction from the driver and, for example, instead of the driver instruction, suggests a route to a safe suggests a stopping point, e.g. to the next rest area, e.g. via the Sh signal. 2 This also shows an optional function of user control 180. If the driving instruction data (Fa) is correctly recognized in the audio signal, this function can check whether the driver's head is near the headrest. If not, it can be assumed that the driver is distracted, e.g., looking at the ground. In this case, the voice input can be ignored and an error message can optionally be displayed.

3 Figure 400 shows a flowchart illustrating a method according to one embodiment. In step 402, a microphone 303 (see, for example, Figure 402) is used to measure the temperature of a device. 1 In step 404, a speech input from driver 100 is captured. A potential driving instruction is determined from this input by a speech recognition unit 140. In step 406, the potential driving instruction is compared with a list of possible driving instructions. This list is a function of possible driving instructions from an at least partially autonomous vehicle control unit 190. If the potential driving instruction is included in the list, an action instruction is issued to driver 100 via a loudspeaker 103 in step 410. This action instruction is a function of the driving instruction. In step 412, the verification unit 150 checks for a response from driver 100 to the output of the action instruction.

In an optional step 414, driver 100 is authenticated using a driver authentication unit 170. If the authentication is unsuccessful, the voice input is ignored in step 416, and the procedure for that voice input is terminated. In an optional step 418, driver 100's fitness to drive is checked using a driving ability detection unit 160. If the fitness-to-drive assessment is incorrect, the voice input is ignored in step 420, and the procedure for that voice input is terminated. In an optional step 422, driver 100's attention is checked using a proximity sensor 165. If the attention check is incorrect, the voice input is ignored in step 424, and the procedure for that voice input is terminated.

In step 426, the driver's reaction 100 is determined. If the driver's reaction 100 is not judged to be correct, the speech input is ignored in step 428, and the procedure for this speech input is terminated. If the driver's reaction 100 is judged to be correct, in step 430, the potential driving instruction is used as an actuator instruction to control the vehicle's actuators 200 by means of the at least partially autonomous vehicle control unit.

Claims (15)

Method for controlling at least a semi-autonomous vehicle by a driver (100) using speech input, comprising the steps of: detecting, by means of a microphone (303), the speech input of the driver (100); determining, by means of a speech recognition unit (140), a potential driving instruction from the speech input of the driver (100); comparing the potential driving instruction with a list of possible driving instructions, wherein the list of possible driving instructions is a function of possible driving instructions of an at least semi-autonomous vehicle control unit (190); if the potential driving instruction is included in the list of possible driving instructions: outputting, by means of a loudspeaker (103), an instruction to the driver (100), wherein the instruction is a function of the driving instruction; verifying, by means of the verification unit (150), a response of the driver (100) to the output of the instruction; and if, by means of the verification unit (150), the driver's response (100) is judged to be correct, use the potential driving instruction as an actuator instruction to control the actuators (200) of the vehicle by means of the at least semi-autonomous vehicle control unit (190). Procedure according to Claim 1 , where the list of possible driving instructions is attributed, where the attribute contains a restriction of the possible driving instructions. Method according to one of the preceding claims, comprising the further step: after issuing the instruction, authenticating the driver (100) by means of a driver authentication unit (170). Method according to one of the preceding claims, comprising the further step: after issuing the instruction, checking the driver's fitness to drive (100) by means of a fitness-to-drive detection unit (160). Method according to one of the preceding claims, comprising the further step: after issuing the instruction, checking the driver's attention (100) by means of a proximity sensor (165). A method according to any of the preceding claims, comprising the further steps of: outputting a control signal by means of the loudspeaker (103); detecting the control signal by means of the microphone (303); and if, by means of the verification unit (150), the control signal is judged to be correct, using the potential driving instruction as an actuator instruction to control the actuators (200) of the vehicle by means of the at least semi-autonomous vehicle control unit (190). Method according to one of the preceding claims, whereby a haptic input of the driver (100) is the driver's (100) reaction to the output of the instruction. Method according to one of the preceding claims, wherein a predefined movement of the driver (100) is the driver's (100) reaction to the issuance of the instruction. Method according to one of the preceding claims, wherein a movement of a steering wheel (110) and/or a movement of a pedal (112) by the driver (100) is the driver's (100) reaction to the issuance of the instruction. Device for controlling an at least partially autonomous vehicle by a driver (100) by means of speech input, the device comprising: an interface to a microphone configured to capture the speech input of the driver (100); a speech recognition unit (140) configured to recognize whether the speech input is a potential driving instruction; an interface to a list of possible driving instructions, wherein the list of possible driving instructions is a function of possible driving instructions of an at least partially autonomous vehicle control unit (190); an interface to a loudspeaker (103) configured to output an instruction to the driver (100) when the speech input is included in the list of possible driving instructions; a verification unit (150) configured to check a response by the driver (100) to the output of the instruction by means of a sensor; and an interface to at least a semi-autonomous vehicle control unit (190), set up to control the actuators (200) of the vehicle on the basis of the potential driving instruction as an actuator instruction, if the verification unit (150) judges the driver's (100) response to be correct. Device according to Claim 10 , wherein the sensor is configured as at least one of the following devices: as a haptic sensor (114) configured to detect haptic input from the driver (100), wherein the haptic sensor (114) is configured as a button, switch, rotary control, pressure sensor, proximity sensor, and/or as a capacitive touch sensor of a screen or input field; and/or as an interior camera function configured to detect a predefined movement of the driver (100), and an image recognition unit configured to evaluate the movement of the driver (100) detected by the interior camera; and/or as a steering wheel motion sensor (111) configured to detect movement of a steering wheel (110) by the driver (100); and/or as a pedal motion sensor (113) configured to detect movement of a pedal (112) by the driver (100). Device according to Claim 10 or 11 , furthermore comprising at least one of: an interface to a driver authentication unit (170); an interface to a fitness-to-drive detection unit (160); and/or an interface to a proximity sensor (165). Vehicle with a device according to Claim 10 or 11 . Use of a device according to Claim 10 or 11 for the safe control of a vehicle by a driver (100) using voice input. Non-volatile, computer-readable storage medium containing a program that, when placed on a processor of a device according to one of the Claims 10 - 12 is executed, instructs the device to perform the steps according to one of the Claims 1 - 9 to execute.