DE102024126727B3 - Method for automatically populating a database with user information using a voice dialogue system, as well as a voice dialogue system and a motor vehicle, comprising the voice dialogue system - Google Patents

Abstract

The invention relates to a method for automatically populating a database (8) with user information using a speech dialogue system (10). For this purpose, in step a, a database (8) is provided which has one or more data fields. Then, in step b, a query is generated according to a recognized category of a conversation phase, which is at least partially related to the recognized category, in the form of natural language information for conducting a conversation with the user (2) and for collecting user information. Subsequently, in step c, feedback from the user (2) is received, thereby realizing the conversation with the user (2), and user information is extracted from the feedback. Finally, in step d, the user information extracted by the speech dialogue system (10) is stored in the at least one data field.

Description

The invention relates to a method for automatically filling a database with user information using a speech dialogue system.

Mobile services often require user information or data about one or more occupants of a vehicle, their relationships to one another, and their preferences for using the services they employ, such as navigation and/or in-car infotainment systems. For example, when searching for a restaurant, the composition of the occupants and/or their eating habits and/or any food intolerances are crucial. Similarly, for selecting entertainment such as music and/or audiobooks, knowledge of the preferences of at least one occupant in the vehicle is important. Although the technical implementation of such services is generally relatively simple, adapting them to individual users and/or the specific circumstances—such as the composition of the occupants—in order to provide appropriate recommendations is complex. The necessary knowledge is multifaceted and cannot be easily acquired through predefined scripts or standardized questionnaires. Furthermore, the occupant expects natural language interaction, allowing them to speak freely and/or engage in open dialogues. With the advent of Large Language Models (LLMs) or machine language models, and especially services like Chat-GPT®, a technology is now available that can meet these requirements. This technology can process information about the vehicle user in real time and offer personalized recommendations and/or services based on this information.

The US 2023 / 0 290 342 A1 describes a dialogue system comprising: a database, a speech recognition module configured to convert a user's utterance in a vehicle into text, an intention determination module configured to identify the user's intention based on the text, an emotion determination module configured to identify the user's emotional state based on the identified user intention, and a controller configured to compare data to compare the user's intention and emotional state with rules stored in the database and to determine whether to output a response to the user's utterance based on the result of the comparison.

The DE 10 2019 217 751 A1 Disclosing a method for operating a speech dialogue system, the method involves capturing speech input, generating a first response output based on the speech input using non-goal-directed dialogue analysis, and generating a second response output based on the speech input using goal-directed dialogue analysis. A first relevance probability is determined for the first response output and a second relevance probability is determined for the second response output, and a speech output is generated based on the response output with the highest relevance probability. The speech dialogue system comprises a capture unit configured to capture speech input, first and second dialogue analysis units configured to generate first and second response outputs based on the speech input, a control unit configured to determine a first relevance probability for the first response output and a second relevance probability for the second response output, and an output unit configured to generate a speech output based on the response output with the highest relevance probability.

The EP 2 140 341 A1 reveals an emotive consulting system and procedure.

The DE 10 2017 213 235 A1 discloses a method for determining user feedback when a user uses a device, and a control device for carrying out the method.

The US 2003 / 0 217 061 A1 reveals methods and systems for providing additional context-related content.

The known methods require a lengthy and uncontrolled collection of user data. They do not request information that corresponds to a predefined category, but only information derived from user behavior. In particular, this approach lacks information about relationships with other people or users, as well as shared preferences, interests, and activities. The collected information is not always accurate or relevant because users, especially within a group, deviate from their usual habits and do things that are irrelevant for later system decisions or controls.

The invention is based on the objective of providing a method for the automated collection of user information.

The problem is solved by the subject matter of the independent patent claims. Advantageous embodiments of the invention are described by the dependent patent claims, the following description, and the figures.

1. a. Bereitstellen einer Datenbank, insbesondere in einem Klartextformat wie z. B. JSON, JavaScript Object Notation und/oder XML, Extensible Markup Language, aufweisend ein oder mehrere Datenfelder, die z. B. einer oder mehreren vorgegebenen Kategorien zugeordnet sind,
2. b. Gemäß einer erkannten Kategorie einer Gesprächsphase: Erzeugen und Stellen einer Abfrage in Form einer natürlich-sprachlichen Information durch das Sprachdialogsystem an den Nutzer zum Sammeln oder Speichern von Nutzerinformationen, wobei die Anfrage zumindest teilweise (z. B. zumindest 30 bis 60 Prozent in Prozentpunkten, z. B. realisiert durch natürlich sprachliche Mustererkennung) mit der erkannten Kategorie zusammenhängt,
3. c. Empfangen einer Rückmeldung des Nutzers und Extrahieren von Nutzerinformationen aus der Rückmeldung,
4. d. Ablegen der durch das Sprachdialogsystem extrahierten Nutzerinformationen in das zumindest eine Datenfeld.

The invention relates to a method for automatically populating a database with user information using a voice dialogue system. In other words, the method relates to the automated acquisition of user information. The following steps are performed for this purpose:

1. a. Providing a database, in particular in a plain text format such as JSON, JavaScript Object Notation and/or XML, Extensible Markup Language, having one or more data fields which are assigned to one or more predefined categories, e.g.
2. b. According to a recognized category of a conversation phase: Generating and posing a query in the form of natural language information by the speech dialogue system to the user for collecting or storing user information, whereby the query is at least partially related to the recognized category (e.g. at least 30 to 60 percent in percentage points, e.g. realized by natural language pattern recognition).
3. c. Receiving user feedback and extracting user information from the feedback,
4. d. Storing the user information extracted by the speech dialogue system in at least one data field.

In other words, step b initiates and/or conducts a conversation with the user in natural language. The query can initiate this conversation. The conversation can then be completed by receiving the response in step c.

In other words, the query can be a natural-language conversation with the user. The query is therefore not just a question, but is embedded in a conversation with the user.

"Automated" means that the process of filling the database with user information by the voice dialogue system is automatic and/or at least partially without manual intervention or human intervention.

In other words, the process can involve a dynamic description of the initially empty data fields. These data fields can then be populated with content, particularly with the preferences of the driver or user and/or other users.

The term "conversation phase" can refer to a (predefined) segment within an ongoing conversation between at least two users and/or between at least one user and the speech dialogue system. For example, if two users are discussing music, the speech dialogue system can categorically classify this conversation phase as "music preference" and ask and/or generate corresponding queries to extract user information.

Overall, it may be possible for the vehicle interior and/or the driver to be (continuously) monitored by the voice dialogue system. As soon as voice activity is detected, e.g., via hotword recognition, the voice dialogue system can populate one or at least one data field with user information by identifying and/or assigning recognized driver preferences through appropriate queries.

A data field, which can also be called a "database field", refers to a single component within a record in a database.

"Natural language" means that the query and the entire dialogue between the user and the speech dialogue system is written or expressed in a language used by people in everyday life, without specific adaptations or formalisms for technical systems.

Step b. can stipulate that the query is only generated and presented to the driver when or as soon as an information criterion is recognized by the voice dialogue system. This information criterion, in the context of a conversation phase, includes the verbal recognition of specific words that correspond at least partially, literally and/or semantically (e.g., implemented using word embeddings), to the categories assigned to the data fields. Additionally or alternatively, the information criterion can include the detection of more than one person in the vehicle and/or the selection of a specific route. Examples of such words are "restaurant," "food," "activity," "sports," and/or "music." This list of words is not exhaustive and can be expanded, for example, manually.

• Sprachdialogsystem: „Ich liebe Filme, du auch?“
• Nutzer: „Ja, absolut.“
• Sprachdialogsystem: „Was ist dein Lieblingsfilm? Meiner ist The Big Lebowski.
• Nutzer: „Ja, cool. Aber ich bin eher so der Rocky Fan."
• Zu extrahierende Informationen: Lieblingsfilm „Rocky“.

In step b., it can also be provided that the speech dialog system, after the query has been posed, stores the user's feedback in, for example, a ring buffer for a specified time, e.g., 10 seconds to 3 minutes, and uses pattern recognition to extract corresponding user information. The system performs actions. The voice dialogue system can inform the user about the success of the extraction or, if necessary, request additional information and then ask the user the same or a similar question. A query might look like this, for example:

• Voice dialogue system: "I love movies, do you too?"
• User: "Yes, absolutely."
• Voice dialogue system: "What is your favorite movie? Mine is The Big Lebowski."
• User: "Yeah, cool. But I'm more of a Rocky fan."
• Information to be extracted: Favorite film "Rocky".

If an initial query yields insufficient user information, where "insufficient" means a quantity of tokens or words below a certain threshold (e.g., less than one to ten tokens), a similar query can be posed to the driver. This similar query may involve generating and posing a query that is at least partially syntactically and/or semantically similar.

The "similar query" approach can be achieved by generating a probability distribution for different queries within a speech dialog query. The speech dialog system can then output the query with the highest probability first. This can be implemented using a specially configured Naive Bayes model.

By using similar queries, the speech dialogue system can refine the dialogue and/or ensure that the information provided is relevant and useful.

Extraction can be achieved using at least one of the aforementioned natural language pattern recognition (NLR) techniques. The intended approach is to generate the feedback as text using automatic speech recognition (ASR). Extraction is then performed using a machine learning model (LLM). This ensures that complex dialogues between the speech dialogue system and the user are captured. Alternatively, the feedback can undergo preprocessing, specifically tokenization, lemmatization, and part-of-speech tagging, to retain only relevant tokens. Named Entity Recognition (NER) has proven particularly advantageous in this regard. Such an NER model analyzes the preprocessed text and identifies entities that correspond to specific categories, such as people, places, organizations, and/or times. This is typically accomplished using machine learning algorithms like Conditional Random Fields (CRF), Hidden Markov Models (HMM), or approaches such as transformer-based models. The identified entities can be classified (using the LLM) according to their category. For example, personal names are assigned to the category "Person" and/or place names to the category "Place." This classification allows the extracted information to be structured in a categorized format. The extracted entities and their categories can then be stored in the database, which contains corresponding data fields and/or tables and/or records to hold this information. Each data field and/or record can therefore be assigned to a category or entity.

The term "filling" refers to filling or replenishing the database with user information.

The term "voice dialogue system" can refer to, for example, a voice assistant and/or an in-car voice control system.

"User information" can refer in particular to preferences and/or personal data and/or behavioral data and/or interests and/or location data and/or voice data and/or relationship data and/or device and usage data and/or user status data. "Relationship data" refers to a relationship or social connection, e.g., friendly and/or professional and/or familial, between at least two users, which can be derived, for example, from the telephone directory.

Providing a database in a plain text format such as JSON or XML offers several advantages. These formats enable a flexible data structure that can be easily adapted to new requirements without changing the database schema. Their readability and interoperability facilitate development and integration with other systems.

After the user provides feedback, the voice dialogue system automatically extracts the user information from the response. This automated extraction can improve the quality of the user information already collected. Finally, the extracted user information can be stored in at least one categorically appropriate data field. This enables structured and organized data management and/or improves data integrity. Additionally, the invention offers the advantage of scalability and/or the unrestricted expandability with regard to data collection.

The invention also includes further developments that result in additional advantages.

According to a further development, it is provided that a control or action is assigned to the user information and that the control is triggered upon recognition of a completeness criterion. First, the speech dialogue system can actively extract user information using the inventive method, i.e., by querying the system. For example, the speech dialogue system records the culinary preferences of each user in the vehicle, and as soon as user information regarding the culinary preference has been recorded for all relevant users, the control can be activated. The control can then include filtering restaurants based on the group's shared preferences. Restaurants that do not match the specified criteria can be excluded, while suitable options are displayed. Further specifications can include dietary restrictions (e.g., gluten-free) and/or budget limits and/or preferred location. For this purpose, it can be provided that the driver or the group is first asked whether and which action should be taken, so that the action is only triggered upon a positive response from the user.

Overall, the training can include the possibility that, for example, a web crawler, based on user information extracted from at least one vehicle occupant, automatically filters relevant activities and/or locations and/or suggestss them as route stops. If there is more than one vehicle occupant, the system can also consider an overlap of their preferences.

This simplifies decision-making processes in groups, as the voice dialogue system automatically offers a selection of options that correspond to the shared preferences of all group members. This can enable a conflict-free journey.

A further development provides that a data field is assigned to a predefined category or dynamically assigned to a newly created category, with the query being formulated based on at least one category. The category can, in particular, comprise one or more preferences of the driver or another vehicle occupant, such as "favorite food" and/or "hobbies" and/or "taste in music." Such categories can either be predefined or dynamically created, for example, by initiating a conversation with the user (using the query) in the vehicle and categorizing the conversation (as a conversation phase). The database according to the invention can thus be considered a dynamically managed list whose contents in the data fields can be overwritten and/or supplemented. This can be implemented using natural language pattern recognition from the prior art.

One advanced feature envisions the voice dialogue system receiving a request from the driver or a vehicle occupant and generating a query based on that request, which is then presented to the user. In other words, it could be implemented, for example, that the driver initiates the dialogue with the voice dialogue system, and the system then generates a query based on their request. This would allow the driver to independently add and/or overwrite content, such as information related to their preferences.

A request can refer to the statement or question that a user addresses to the speech dialogue system via spoken language (direct request) and/or to at least one other user (indirect or implicit request).

When a user submits a request via an interface of the speech dialog system, this request can be classified using at least one specially trained language model (LLM) and assigned to at least one data field. This is particularly useful when the request has not yet been assigned to a data field and/or is not yet defined. This offers user-friendliness and fast processing, as the user can formulate their request in a natural way.

At least one data field or record can therefore be pre-assigned to a class or category, or dynamically assigned to one. For example, if it is detected that the user is talking about music (during a conversation) and the category "Music" is not yet assigned to a data field, or if a data field does not yet have such an assignment, a new data field with this assignment can be created. User information assigned to the category "Music" can then be added as content to the "Music" data field.

The speech dialogue system can therefore respond to the user's request by asking for additional details to better understand the request and/or provide further information.

A training program stipulates that data storage is initiated or triggered when a predefined set of user information and/or a set of topic-specific (or categorically recognized and assigned) user information is detected and/or received. "Predefined set" could mean, for example, that a certain minimum number of data points or tokens, or a complete set of information, must be present before the information is stored in one or more data fields. The set might be considered complete, for example, when at least 2 to 80 data points have been detected and/or received, where "data points" refers to individual extracted entities and/or tokens. This approach has the advantage that user information is only collected and/or stored once a predefined set of user information has been reached. This avoids the need to create and store a data field, record, and/or database for only a few pieces of user information that may not be relevant anyway.

Further training stipulates that the speech dialogue system must have at least a Large Language Model (LLM) that includes at least a prompt manager and a dialogue manager, whereby the prompt manager uses pattern recognition to identify a category of the (currently necessary or appropriate) conversation phase and, depending on the identified category, produces corresponding context for the dialogue manager, whereby the dialogue manager includes a communication unit visible to the user and creates a query based on the identified category.

The LLM is the essential component of a speech dialogue system capable of understanding and generating natural language. Examples of LLMs include GPT-4 (Generative Pre-Trained Transformer) from OpenAL and/or BERT (Bidirectional Encoder Representations from Transformers) from Google.

Depending on the detected conversation phase, the prompt manager can generate appropriate context. For example, if the user asks for a restaurant in their vicinity, the prompt manager can determine the user's location, and this information is then used by the dialogue manager to communicate or provide the requested information to the user.

The dialog manager can act as an interface to the user and controls interactions. It can ensure that the contexts generated by the prompt manager are presented in a user-friendly format. To this end, the database can store user information about the user's recent queries, and the dialog manager can then use this information to ask follow-up questions (further queries).

"Context" can refer to the collection and use of all relevant information that the speech dialog system needs to output a suitable query via the dialog manager.

According to a training course, the voice dialogue system is designed to create a new database, new data fields, and/or at least one new data record when a new user is detected. This can be achieved by monitoring the interactions and input of at least one user. Such a monitoring system can be derived from existing technology. For example, a biometric identification method and/or an authentication system can be implemented. As soon as a new user is detected, the voice dialogue system can automatically initiate the process of creating new data records, new data fields, and/or at least one new database for that user.

These new data fields can be completely empty or already contain information, such as the user's biometric characteristics, based on which the user can be, or has already been, identified. An example of the technical implementation could be a script file that is executed when a new user is detected. This script file can establish a connection to the database and create new tables or data fields and/or at least one new record for the new user.

This enables flexible and scalable management of user data and helps to ensure a personalized and user-centric experience for every user.

A beneficial further development approach involves the speech dialogue system incorporating logic that identifies existing and at least partially similar user information by comparing input data with the user information stored in the data field, thus avoiding data field duplication. "Partially similar" means that user information is similar to a certain percentage. Word embeddings can be used additionally or alternatively to check for semantic similarity.

Duplicates can be identified by comparing the input data with the user data stored in the database. This can be achieved using the LLM (Language Lifecycle Management). Alternatively or additionally, a similarity analysis (e.g., cosine similarity and/or Jaccard similarity) can be applied to textual data, while numerical or categorical data can be compared directly. Once duplicates are identified, the speech dialog system can take appropriate measures to prevent data field duplication. This might mean that no new records are created, but rather existing records or data fields are updated or extended to ensure consistent data management. The speech dialog system can provide the user with feedback about the identification of duplicates and, if necessary, request confirmation before taking further action. Integrating this logic into the speech dialog system ensures that data consistency is maintained and/or duplications are avoided.

Additionally or alternatively, it may be provided that one or more data fields are overwritten if it is detected that the driver or another vehicle occupant or user has changed their preference regarding a previously defined category. For example, if the data field with the category "Favorite Food" contains the word or content "Pizza" and the driver or user states that their favorite food is now "Burger," "Burger" could be prioritized and/or the content "Pizza" could be deleted or overwritten.

Further training stipulates that the voice dialogue system must be connected to at least one other system, so that queries from that other system are initiated via the voice dialogue system. In other words, the voice dialogue system can exchange data with one or more systems. This allows, for example, the testing of in-vehicle improvements by asking the driver questions about a function or feature in the vehicle. Specific examples include: "How do you like the steering wheel?" or "Is the seat comfortable?" In particular, a query about a function or feature can be asked if the driver used it shortly beforehand, for example, 10 seconds to 10 minutes prior.

By connecting to at least one other system, the voice dialogue system can dynamically capture the user's preferences (regarding vehicle functions).

According to a training course, the speech dialogue system is designed to include one or at least one language learning module (LLM) trained on at least one text dataset to capture speech patterns across domains. After training, the LLM is fine-tuned to suit one or more specific domains. The text dataset can originate from various sources, such as books, articles, websites, and chat histories. This training allows the LLM to become familiar with a broad spectrum of speech data.

“Cross-domain” can refer to the LLM’s ability to capture and/or assign language patterns from different areas or subject fields.

• • einen Dialog zwischen dem Sprachdialogsystem und dem Nutzer zu überwachen,
• • die Datenbank mit den Nutzerinformationen kategorisch zu befüllen,
• • zumindest eine weitere Datenbank und/oder zumindest einen Datensatz und/oder zumindest ein weiteres Datenfeld zu erzeugen und/oder den Inhalt des Datenfelds und/oder die Information, welches Datenfeld erzeugt wurde, anzuzeigen,
• • befüllte Datenfelder auszulesen,
• • zumindest eine Abfrage in Abhängigkeit einer erkannten Kategorie einer Gesprächsphase an den Nutzer (2) zu stellen, und
• • eine Anfrage zu empfangen.

The LLM is set up for this purpose:

• • to monitor a dialogue between the speech dialogue system and the user,
• • to categorically populate the database with user information,
• • to create at least one additional database and/or at least one additional data record and/or at least one additional data field and/or to display the content of the data field and/or the information about which data field was created,
• • to read filled data fields,
• • to pose at least one query to the user (2) depending on a recognized category of a conversation phase, and
• • to receive a request.

The listed functions of the LLM can lead to an improved user experience and/or enable personalized and/or context-sensitive interaction with the user.

One advanced training program envisions the speech dialogue system performing state monitoring, quantifying a user's readiness to communicate using a predefined scale and adjusting the interaction accordingly. Depending on the user's perceived readiness, for example, fewer queries can be asked—only one to three per hour—to make the interaction less intrusive. Conversely, with a high readiness, more questions are asked—four to thirty per hour—to gather more user information. Readiness to communicate can be quantified, for example, using state-of-the-art emotion recognition. Additionally or alternatively, speech analysis and/or evaluation of interaction patterns can be used. State-of-the-art techniques can be used to quantify or determine a user's readiness to communicate. Speech analysis, for example, can evaluate changes in tone of voice or speaking speed, which can provide clues about their willingness to communicate. "Interaction patterns" can refer to, for example, the time a user takes to respond to a query and/or how many queries the user makes or how many queries the user answers, which can be an indicator of their willingness to communicate. This can increase comfort and/or the user experience, thereby enabling a conflict-free journey.

Further training stipulates that condition monitoring includes: evaluating: a number of users within a specified radius, e.g., up to 3 or 5 meters, around the voice dialogue system, e.g., using an infrared sensor and/or facial recognition, and/or evaluating health data of at least one user, and/or detecting fatigue in at least one user.

The voice dialogue system can be connected to wearables such as smartwatches or fitness trackers that measure health data like heart rate and blood pressure to assess the user's stress level. For example, an elevated heart rate (e.g., above 100 bpm) and/or elevated blood pressure (e.g., above 120/80 mmHg) can indicate increased stress, potentially reducing the user's willingness to communicate. The driver's smartwatch, for instance, might report an elevated heart rate, allowing the voice dialogue system to register this as a possible stress indicator and adopt a calmer and/or less demanding communication style.

Health data can be transmitted to the voice dialogue system via a wireless connection (e.g. Bluetooth and/or WLAN, Wireless Local Area Network).

For example, cameras and infrared sensors installed in a vehicle can monitor the eye movements and/or blinking of occupants to detect signs of fatigue. Upon detecting signs of fatigue, the voice dialogue system can adjust its communication by asking less complex and/or shorter questions (e.g., consisting of a maximum of 5 to 20 words) or even suggesting a break.

If facial recognition detects, for example, that there are three other passengers in the vehicle besides the user, the voice dialogue system can issue queries or information louder than usual (e.g. 5 to 10 decibels louder) to address all users in the vehicle.

The voice dialogue system can therefore adapt to the current state of at least one user.

The condition monitoring system can include a monitoring LLM that communicates with and/or exchanges data with the LLM.

The speech dialogue system can use a feedback system to learn whether the user exhibits a low, medium, or high willingness to communicate when stress levels are elevated and/or fatigue is detected.

This enables the speech dialogue system to respond to the user's willingness to communicate in a variety of ways and to adapt the interaction accordingly.

The voice dialogue system can be installed in a motor vehicle. The motor vehicle can be a car, in particular a passenger car or truck, or a bus or motorcycle.

For use cases or application situations that may arise during the procedure and are not explicitly described here, it may be provided that, according to the procedure, an error message and/or a request for user feedback is issued and/or a default setting and/or a predetermined initial state is set.

The invention also includes the control device for the speech dialogue system. The control device can comprise a data processing device or a processor circuit configured to execute an embodiment of the method according to the invention. For this purpose, the processor circuit can comprise at least one microprocessor and/or at least one microcontroller and/or at least one FPGA (Field Programmable Gate Array) and/or at least one DSP (Digital Signal Processor). In particular, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an NPU (Neural Processing Unit) can be used as the microprocessor. Furthermore, the processor circuit can comprise program code configured to execute the embodiment of the method according to the invention when executed by the processor circuit. The program code can be stored in a data memory of the processor circuit. The processor setup can be based, for example, on at least one circuit board and/or on at least one SoC (System on Chip).

As a further solution, the invention also includes a computer-readable storage medium comprising program code which, when executed by a computer or a computer network, causes it to execute an embodiment of the method according to the invention. The storage medium can be provided at least partially as a non-volatile data storage medium (e.g., as flash memory and/or as an SSD - solid state drive) and/or at least partially as a volatile data storage medium (e.g., as RAM - random access memory). The storage medium can be located within the computer or computer network. However, the storage medium can also be operated, for example, as an app store server and/or cloud server on the internet. The computer or computer network can provide a processor circuit with, for example, at least one microprocessor. The program code can be provided as binary code and/or assembly code and/or source code of a programming language (e.g., C) and/or as a program script (e.g., Python). The computer-readable storage medium can alternatively be implemented by a signal containing computer-readable data, e.g., a time-varying voltage signal and/or a radio signal.

The invention also includes combinations of the features of the described embodiments. The invention therefore also includes realizations that each exhibit a combination of the features of several of the described embodiments, provided that the embodiments have not been described as mutually exclusive.

• 1 eine schematische Darstellung einer Ausführungsform des erfindungsgemäßen Verfahrens und
• 2 ein Flussdiagramm einer Ausführungsform des erfindungsgemäßen Verfahrens.

The following are exemplary embodiments of the invention described. This is illustrated by:

• 1 a schematic representation of an embodiment of the method according to the invention and
• 2 a flowchart of an embodiment of the method according to the invention.

The exemplary embodiments described below are preferred embodiments of the invention. In these exemplary embodiments, the described components each represent individual features of the invention, which can be considered independently and each further develops the invention independently. Therefore, the disclosure is intended to include combinations of features of the embodiments other than those shown. Furthermore, the described embodiments can also be supplemented by further features of the invention already described.

In the figures, identical reference symbols denote functionally equivalent elements.

In 1 The following are shown: a speech dialog system 10, a database 8, an interface 6, an LLM 11, a status monitor 12, a plaintext 13, a prompt manager 14, a dialog manager 15, a monitoring LLM 16, a monitoring system 17, a user 2, and one or more systems 4. The respective components are represented symbolically and show only one possible implementation of the idea.

According to 1 and 2 A method for automatically populating a database 8 with user information using a speech dialogue system 10 can be carried out as follows. In step a., a database 8 containing one or more data fields can be provided. Then, in step b., a query in the form of natural language information can be generated and presented to the corresponding user 2 by the speech dialogue system 10 to collect user information. This can be implemented according to predefined categories or dynamically, by recognizing a conversation phase or topic, e.g., in the interior of a vehicle. Subsequently, in step c., feedback from user 2 can be received, and user information can be extracted from the feedback, particularly using the LLM (Language Lifecycle Management). Finally, in step d., the user information extracted by the speech dialogue system 10 can be stored in at least one data field.

Steps a. and b. can be represented by step S10, while step c. can be represented by step S20 and step d. by step S30.

According to a particular embodiment, a (dynamic) database 8 containing user information can be automatically populated using an (intelligent) speech dialog system 10. Categories for data fields and/or content for data fields can be freely formulated and entered into the database 8 by a user 2, with the system or speech dialog system 10 automatically filling in all data fields at runtime.

• - Klartext 13 und Datenbank 8 für strukturierte Inhalte: Eine Datenbank 8 in einem Klartextformat 13 wie beispielsweise JSON. Die Datenbank 8 kann Kategorien für Datenfelder beinhalten oder erzeugen, die dem System bereits bekannt sind, sowie weitere Kategorien für Datenfelder, die von Dritten oder weiteren Systemen 4 angefragt wurden.
• - Schnittstelle 6 um von außen neue (freie) Datenfelder oder Kategorien für Datenfelder zu definieren: Um Datenfelddopplungen zu vermeiden und/oder eine einheitliche Struktur zu schaffen, kann eine Logik mit einer Large Language Modelling Komponente oder einem LLM 11 vor die Datenbank 8 gesetzt werden, die frei formulierte Anfragen von außen entgegennimmt und automatisiert in die Datenbank 8 einpflegt. Die Logik kann auch Anfragen entgegennehmen und die richtigen Datenfelder an den Anfragenden zurückgeben.

• - Sprachdialogsystem 10 (LLM -basiert), das mithilfe der textuellen Beschreibung, also z. B. bereits vorhandenen Nutzerinformationen in der Datenbank 8, weitere Nutzerinformationen abfragt. Das Sprachdialogsystem 10 kann aus mindestens einem Large Language Model 11, mindestens einem Prompt-Manager 14 und mindestens einem Dialog-Manager 15 oder dialogführendem Modell bestehen oder diese umfassen. Der Prompt-Manager 14 kann dabei die aktuell nötige Gesprächsphase und das Thema erkennen und entsprechend Kontext (Prompts) für das dialogführende Modell erstellen. Es können hier weitere LLMs oder regelbasierte Kontrollsysteme implementiert sein, die das Dialogmanagement zusätzlich überwachen und steuern können.
• - Das Sprachdialogsystem 10 kann proaktiv neue Nutzerinformationen in die dynamische Datenbank 8 ablegen. Dies kann der Fall sein, wenn der Nutzer 2 z. B. zufällig Informationen über sich preisgibt, oder im Vorfeld eine Grundmenge an Informationen definiert wird, die z. B. in einer initialen Kennenlernphase zwischen System und Nutzer 2 grundsätzlich gesammelt werden soll.
• - Das Sprachdialogsystem 10 kann eingerichtet sein, ein Verständnis darüber zu verfügen, welche Daten bereits vorhanden sind und fragt daher keine Datenfelder doppelt ab.

The system can consist of or include the following components:

• - Plain text 13 and database 8 for structured content: A database 8 in a plain text format 13 such as JSON. The data bank 8 can include or create categories for data fields that are already known to the system, as well as additional categories for data fields that have been requested by third parties or other systems 4.
• - Interface 6 for defining new (free) data fields or categories for data fields externally: To avoid data field duplication and/or create a uniform structure, logic using a Large Language Modeling component or an LLM 11 can be placed in front of database 8. This logic receives freely formulated queries from the outside and automatically inserts them into database 8. The logic can also receive queries and return the correct data fields to the requester.

• - A speech dialogue system 10 (LLM-based) that uses textual descriptions, such as existing user information in database 8, to query further user information. The speech dialogue system 10 can consist of or comprise at least one Large Language Model 11, at least one Prompt Manager 14, and at least one Dialogue Manager 15 or dialog-leading model. The Prompt Manager 14 can recognize the currently required conversation phase and topic and create corresponding context (prompts) for the dialog-leading model. Additional LLMs or rule-based control systems can be implemented here to further monitor and control the dialogue management.
• - The speech dialogue system 10 can proactively store new user information in the dynamic database 8. This can occur, for example, if user 2 accidentally reveals information about themselves, or if a basic set of information is defined in advance, which is to be collected, for example, during an initial familiarization phase between the system and user 2.
• - The speech dialog system 10 can be configured to understand which data is already available and therefore does not query data fields twice.

The idea therefore includes the intelligent combination of database 8 for user information and speech dialogue system 10, which makes it possible to obtain any user information about the customer or user 2 via a simple interface 6.

• - Dritte (Apps, Services, ...) können über eine Schnittstelle 6 auf die Datenbank 8 für Nutzerinformationen zugreifen.
• - Der Kunde kann mit dem Sprachdialogsystem 10 kommunizieren.
• - Der Kunde kann von einer Zustandsüberwachung 12 überwacht werden, also einer Überwachung 12 unterzogen werden.

Another embodiment provides the following:

• - Third parties (apps, services, ...) can access the database 8 for user information via an interface 6.
• - The customer can communicate using the voice dialogue system 10.
• - The customer can be monitored by a condition monitoring system 12, i.e., subjected to monitoring 12.

• - Die Datenbank 8 für Nutzerinformationen kann nach außen für Dritte nur über eine Schnittstelle 6 erreichbar sein. Die Daten über die Kunden können in einer Klartextdatenbank 8 (z.B. JSON) abgelegt werden.
• - Ein (speziell) trainiertes künstliches neuronales Netzwerk (Large Language Model 11) kann dazu eingerichtet sein, die Datenbank 8 zu befüllen und auszulesen, in dem es Datenfelder erzeugt oder vorhandene Datenfelder als Prompt (Befehl) hineinbekommt.
• - Das neuronale Netz oder LLM 11 kann zudem Informationen erzeugen die für das Sprachdialogsystem 10 relevant sind und überwacht außerdem den Dialog zwischen System und Kunde, um dadurch neue Datenfelder zu generieren.
• - Das Sprachdialogsystem 10 kann im Kern aus zwei speziell trainierten Large Language Models 11 bestehen: Einem Prompt-Manager 14 und einem Dialog-Manager 15.
• - Der Dialog Manager 14 kann mit der Datenbank 8 oder Nutzerinformationsdatenbank 8 kommunizieren und erstellt dementsprechend spezielle Prompts.
• - Diese Prompts können vom Model zur Dialoggenerierung (Dialog-Manager 15) aufgenommen werden. Das Modell zur Dialoggenerierung kann die nach außen sichtbare Kommunikationseinheit für den Kunden sein.
• - Die Zustandsüberwachung 12 kann eine „Willingness to Communicate“ oder Kommunikationsbereitschaft berechnen oder quantifizieren und überwacht den Innenraumzustand, wie zum Beispiel Anzahl der Personen, Gesundheitsdaten und Müdigkeit.
• - Ein weiteres trainiertes Large Language Model kann mit der Überwachungseinheit oder Überwachungs-LLM 16 kommunizieren und generiert ebenfalls Prompts für das LLM 11 im Sprachdialogsystem 10 mit dem der Kunde interagiert, sodass auch die Zustandsüberwachung 12 Einfluss auf das Dialogmanagement hat.

In detail, the components can then function as follows:

• - Database 8 for user information can only be accessed externally by third parties via an interface 6. Customer data can be stored in a plain text database 8 (e.g., JSON).
• - A (specially) trained artificial neural network (Large Language Model 11) can be set up to populate and read database 8 by creating data fields or receiving existing data fields as a prompt (command).
• - The neural network or LLM 11 can also generate information relevant to the speech dialogue system 10 and also monitors the dialogue between system and customer in order to generate new data fields.
• - The speech dialog system 10 can consist of two specially trained Large Language Models 11: a Prompt Manager 14 and a Dialog Manager 15.
• - The Dialog Manager 14 can communicate with Database 8 or User Information Database 8 and creates special prompts accordingly.
• These prompts can be captured by the dialogue generation model (Dialogue Manager 15). The dialogue generation model can be the externally visible communication unit for the customer.
• - The condition monitoring 12 can calculate or quantify a “Willingness to Communicate” or communication readiness and monitors the indoor condition, such as the number of people, health data and fatigue.
• - Another trained Large Language Model can communicate with the monitoring unit or monitoring LLM 16 and also generates prompts for the LLM 11 in the speech dialog system 10 with which the customer interacts, so that the status monitoring 12 also has an influence on the dialog management.

Overall, the examples show how an intelligent and generic module for obtaining user information can be provided.

Reference symbol list

2

users

4

Systems

6

interface

8

database

10

Voice dialogue system

11

LLM

12

Condition monitoring

13

plain text

14

Prompt Manager

15

Dialogue Manager

16

Surveillance LLM

17

surveillance

Claims (15)

A method for automatically populating a database (8) with category-related user information using a speech dialogue system (10), comprising the following steps: a. Providing a database (8) comprising one or more data fields, b. Based on a recognized category of a conversation phase: generating and posing a query, in the form of natural language information to the user (2) by the speech dialogue system (10) to collect user information, wherein the query is at least partially related to the recognized category, c. Receiving feedback from the user (2) and extracting user information from the feedback, d. Storing the user information extracted by the speech dialogue system (10) in the at least one data field. Procedure according to Claim 1 , wherein a control is assigned to the user information and the control is triggered upon detection of a completeness criterion of the user information, wherein the control is configured according to the user information. Method according to one of the preceding claims, wherein the query is formulated based on the at least one category. Method according to one of the preceding claims, wherein the voice dialog system receives a request from the user (2) or another vehicle occupant and, depending on the request, generates the query and sends it to the user (2). Method according to one of the preceding claims, wherein the storage is triggered when a predetermined basic set of user information and/or a predetermined basic set of topic-specific user information, each of which has been defined in advance, is recognized and/or received. Method according to one of the preceding claims, wherein the speech dialog system (10) comprises at least one Large Language Model, LLM, (11) comprising at least one prompt manager (14) and one dialog manager (15), wherein the prompt manager (14) recognizes a category of the conversation phase by means of pattern recognition and produces context for the dialog manager (15) depending on the recognized category, wherein the dialog manager (15) comprises a communication unit visible to the user (2) and creates a query based on the recognized category. Method according to one of the preceding claims, wherein the speech dialog system (10) upon recognizing a new user (2) generates a corresponding new database (8) to be filled and/or new data fields and/or one or at least one new data record. Method according to one of the preceding claims, wherein the speech dialog system (10) comprises logic which, by comparing input data of the feedback with the user information stored in the data field, identifies user information that is already present and at least partially similar in a predefined manner, thereby avoiding data field duplications. Method according to one of the preceding claims, wherein the speech dialog system (10) is connected to at least one further system for the exchange of user information, wherein the query is initiated by the further system via the speech dialog system. Method according to one of the preceding claims, wherein the speech dialogue system comprises one or more LLMs (11) trained on at least one text data set to capture cross-domain speech patterns, wherein the LLM (11) is adapted to one or more domains by means of fine-tuning after training. Method according to one of the preceding claims, wherein the speech dialogue system (10) a status monitoring (12) is performed which quantifies the communication readiness of a user (2) on the basis of a predefined scale and adjusts the interaction with the corresponding user (2) depending on the quantified communication readiness. Procedure according to Claim 11 , wherein the condition monitoring (12) includes: evaluating the number of users (2) in a specified radius around the speech dialogue system (10) and/or evaluating health data of at least one user (2) and/or detecting fatigue of at least one user (2). Processor circuit with an LLM, wherein the processor circuit is configured to perform a method according to one of the preceding method claims. Speech dialog system (10) comprising a processor unit comprising program instructions which, when executed by the processor unit, cause it to perform a method according to one of the preceding method claims. motor vehicle having a voice dialogue system (10) according to Claim 14 .