DE102024004449A1 - database system - Google Patents

Abstract

A database system is described, with at least one database, at least one user interface, an authorization system and an explanation system for explaining the metadata fields, wherein the explanation system has an agent that supports the at least one user in entering data records, wherein the agent has a large language model that is connected to a policy database and a metadata directory, wherein the agent is configured to make suggestions for the metadata fields on the basis of the policy database and the metadata directory.

Description

Database systems of the type mentioned above are known in the state of the art.

Large corporate database systems require comprehensive mechanisms to ensure the consistency of data sets, especially with regard to metadata. This includes clear standards and instructions, effective authorization systems and regular user training. At the same time, these systems must remain flexible enough to adapt to changing requirements without compromising data quality.

The complexity of large database systems in large companies arises, among other things, from the large number of actors involved, the different data types and the high demands on consistency and reliability of the stored information. A central aspect of this complexity is the entry of data sets, especially with regard to metadata, which is essential for the organization and findability of the data. Metadata serves as structured additional information to the actual data and helps to classify, describe and link it.

In large organizations, the quality and consistency of metadata entry often varies greatly because records are entered by different users or systems. Different departments often have different definitions of terms or standards for using metadata fields. This can lead to fields being filled out inconsistently or incompletely, making it much more difficult to search and analyze data. In addition, the use of optional metadata fields or a lack of standardized entry processes can lead to redundant or incorrect entries.

Another complexity factor lies in the often dynamic and rapidly changing requirements of large companies. New business processes, regulatory requirements or technical changes regularly require adjustments to the database structure and the definition of metadata fields. These changes must be reflected in the existing workflows and user instructions in order to ensure long-term consistency. At the same time, there is the challenge of ensuring that historical data can still be interpreted correctly.

Finally, authorizations and access rights also play an important role. Different user groups often have different authorizations to enter and edit data records. This can further jeopardize consistency if, for example, some users cannot fill in mandatory fields or have limited visibility of certain fields. To counteract this, complex authorization systems and user instructions are often necessary, which, however, only increase complexity.

From the US 2019/0057147 A1 is known as a software platform that ingests, processes, and makes data available for search from a variety of data sources. It also describes a data portal that allows users to quickly find information and enhance metadata to leverage the organization's data assets. Various aspects of disclosed software platforms include technologies for retrieving metadata without touching the underlying data assets, technologies for using table or database provenance as a factor in determining document relevance, processes for receiving data transmitted from data sources, and processes for disclosing what a user must do to access specific data assets.

Inconsistent or poor-quality (meta)data can significantly impair the functionality of a database system. Inconsistent or missing metadata make it difficult to find and correctly interpret data sets, which leads to incorrect analyses and decision-making. In addition, redundant data sets are often created due to different spellings or usage of terms, which increases storage requirements and makes data consolidation more difficult. Processes based on automated algorithms can produce incorrect results or fail due to inconsistencies, which significantly reduces the efficiency and reliability of data processing.

In addition, such inconsistencies hinder collaboration between departments, as teams often have to refer to conflicting or incorrect data. This leads to time-consuming manual corrections and reconciliations. In regulated industries, incomplete or incorrect metadata can also lead to regulatory violations, which can result in financial penalties or legal consequences. In the long term, such problems harm the acceptance of the system: users lose confidence in the reliability of the data and resort to inefficient, manual processes, thereby undermining the original benefits of the database system.

This raises the task of further developing database systems of the type mentioned above to ensure higher metadata quality.

The object is achieved by a database system according to claim 1. Further embodiments and developments are the subject of the dependent claims.

• - wenigstens einer Datenbank, in der Datensätze und den Datensätzen zugeordnete Metadaten gespeichert sind, wobei pro Datensatz eine Mehrzahl an Metadatenfeldern vorgesehen ist,
• - mindestens einer Benutzeroberfläche zur Eingabe von Datensätzen einschließlich der Metadaten durch wenigstens einen Benutzer,
• - ein Berechtigungssystem, durch das Benutzern oder Benutzergruppen unterschiedliche Rollen sowie Zugriffsrechte und/oder Bearbeitungsrechte für die Datensätze und/oder Metadatenfelder zugewiesen sind, wobei Änderungen und Abfragen der wenigstens einen Datenbank nur auf dieser Basis erlaubt sind,
• - ein Erläuterungssystem zur Erläuterung der Metadatenfelder,

wobei
das Erläuterungssystem einen Agenten aufweist, der den wenigstens einen Benutzer bei der Eingabe von Datensätzen unterstützt, wobei der Agent ein großes Sprachmodell aufweist, das mit einer Richtliniendatenbank und einem Metadatenverzeichnis verbunden ist, wobei der Agent zur Unterbreitung von Vorschlägen für die Metadatenfelder auf der Basis der Richtliniendatenbank und des Metadatenverzeichnisses eingerichtet ist.A database system is described with

• - at least one database in which data records and metadata associated with the data records are stored, with a plurality of metadata fields being provided for each data record,
• - at least one user interface for the input of data records including metadata by at least one user,
• - an authorization system through which users or user groups are assigned different roles as well as access rights and/or editing rights for the data records and/or metadata fields, whereby changes and queries to the at least one database are only permitted on this basis,
• - an explanation system to explain the metadata fields,

where
the explanation system comprises an agent that assists the at least one user in entering data records, the agent comprising a large language model connected to a policy database and a metadata dictionary, the agent being configured to make suggestions for the metadata fields based on the policy database and the metadata dictionary.

The large language model serves as the basis for a networked automatic suggestion of metadata entries based on semantic or lexematic information, which the user can either accept or reject.

Such a large language model is often referred to as a Large Language Model (LLM). An LLM is an artificial neural network that has been trained to interpret and generate human language and to solve tasks based on it. It is based on the architecture of a so-called transformer model, which works particularly efficiently by processing large amounts of data in parallel. The basis of the functioning of an LLM is the analysis and processing of text in the form of mathematical representations, so-called vectors.

The model is first pre-trained with very large amounts of text data from various sources such as books, websites or articles. In the process, it learns to recognize patterns and relationships in language, e.g. which words often appear together or how sentence structures are constructed. This learning process is based on the technique of the so-called "self-attention" mechanism, with which the model can identify and prioritize relevant parts of a sentence or text. This enables it to better understand the context of words and deliver more precise results.

When using an LLM, an input, also called a prompt, is typically fed into the model. The model analyzes this input and produces an output based on its training. It uses probabilities to predict which word or string best fits the input. This enables tasks such as answering questions, writing texts, or translating into other languages.

A key feature of an LLM is its ability to generalize. It can use the knowledge it has learned to interpret inputs that it did not see during training.

The database system is designed to be populated by users assigned specific roles, each responsible for entering specific types of information.

The authorization system ensures that each user enters information according to their assigned role and relevant policies.

When logging into the database system, it can be checked whether the user has previously attended training on how to use the database system. The training is intended to explain the database system and related processes to the user and to familiarize them with the various functions, functionalities, processes and responsibilities. If the check is negative, training can be recommended or started automatically before the user can use the system. If training has been completed, the user is given access and editing rights appropriate to their role. This check can be based on an email address as an identifier together with a password before the actual user interface can be displayed. In this way, it can be checked whether a user has been registered and it can be registered whether a user has attended training before this login process. to redirect him, if necessary automatically, to training or to grant access to the interface.

In a first further embodiment, it is provided that the authorization system is designed to check a system training status of the at least one user and to enforce a system training for the at least one user.

The integrated agent assists the user in adding metadata through the user interface. Within a specific field, which can be displayed for example next to the metadata fields, the user can have the opportunity to enter their questions about the requested metadata, which promotes their understanding and the entry of correct metadata.

By connecting the agent to the policy database, which stores specific rules associated with the different types of metadata (e.g. data sets, responsible users, associated IT systems), specific explanations can be derived as to why certain metadata needs to be added, edited, etc.

In a further refinement, the agent is planned to be a chatbot.

The chatbot is trained with guidelines and assists users by providing guidance and recommendations during the data entry process. In addition, the chatbot is regularly updated with metadata-related guidelines so that it can provide up-to-date advice. It is also able to detect conflicting or incorrect information and automatically suggest corrected data, improving the accuracy and reliability of the database.

In a further refinement, it is planned that a metadata analysis system is connected to the metadata directory to check whether the metadata conforms to the guidelines.

In a further refinement, it is planned that the large language model is focused on the data of the policy database and the metadata directory by means of fine-tuning, prompt engineering and/or retrieval-augmented generation.

In a further refinement, a reporting module is provided for the creation of reports on the data sets, metadata and/or analyses.

In this case, the relationship between different metadata entities (e.g. IT systems, data domains, data, business objects or others) can, for example, be automatically presented in different reports that depict the semantic relationship in the form of a semantic network between all related objects and, in certain embodiments, also allow a further representation (drilldown) from one entity (e.g. domain level) to all related entities (e.g. data sets, use cases, etc.).

As new business requirements arise, reports can be edited to add additional automatically generated reports to the system and remove those that are no longer needed.

In a further further embodiment, it is provided that the user interface is designed to display different user interfaces depending on the role, the access rights and/or editing rights, wherein at least a first user interface is provided for data-entering users and at least a second user interface is provided for administrators.

In a further refinement, the large language model is designed to provide metadata suggestions to administrators.

In a further further embodiment, it is provided that a workflow module is provided which is configured to assign tasks for entering data records and/or metadata to a plurality of users.

In a further refinement, it is provided that the workflow module is connected to the metadata directory, wherein the workflow module is configured to send metadata to the metadata directory.

Further advantages, features and details emerge from the following description, in which - if necessary with reference to the drawing - at least one embodiment is described in detail. Identical, similar and/or functionally identical parts are provided with the same reference symbols.

• 1 ein Datenbanksystem mit einer Datenbank,
• 2 die Struktur eines Datensatzes, sowie
• 3 ein Ablaufdiagramm eines Verfahrens.

They show schematically:

• 1 a database system with a database,
• 2 the structure of a data set, as well as
• 3 a flow chart of a procedure.

1 shows a database system 2 with a database 4.

In the database 4, a variety of data can be stored together with associated metadata (see 2 ) can be stored.

The database system 2 has user interfaces 6 and 8, with the help of which users B and administrators A can access the database 4.

If user B wants to log in to the database system 2 via the user interface 6, it is first checked whether appropriate access already exists and whether user B has fulfilled all necessary training requirements. These can be repeated cyclically to ensure that user B stays up to date on a regular basis. If this is not the case, appropriate training 10 is carried out, which can depend on the relevant role of user B. If this check was successful, user B can access the database 4 according to the corresponding assigned user profile and role and enter or query data and metadata.

Corresponding user inputs that user B makes in the database 4 are forwarded to a chatbot 12, which has the task of interpreting and checking the user input to determine whether user B's inputs are correct and compliant. The chatbot 12 can then make appropriate suggestions that help user B to enter the metadata correctly. The chatbot 12 can be based on a large language model.

The chatbot 12 obtains the corresponding suggestions from another large language model 14, which is connected to the database 4 and has access to a metadata directory 16 in the database 4. The LLM 14 is supplied with metadata by a user group BG1, whereby the user group BG1 is responsible for defining the relevant metadata that may be stored in the database 4. This defined metadata is then stored in the metadata directory 16.

The chatbot 12 is further connected to a policy database 18 in which policies valid in the company are stored to ensure that the suggestions of the chatbot 12 to the user B are policy-compliant.

If user B cannot find a selection of metadata that meets his needs, metadata suggestions from user B can be sent via the LLM 14 via a distribution list 20 to a second user group BG2, which has the task of reviewing the metadata suggestions from user B and, if necessary, approving them. To this end, automated information and confirmation processes can ensure that added responsible metadata providers can edit, confirm or reject all information added in connection with them if this information is changed or originally saved. A change by a user B can automatically trigger a notification to him. The second user group BG2 can partially or completely overlap with the first user group BG1.

Via the user interface 8, an administrator A can access the database 4 and can make changes to the database 4 and to the policies that are necessary to ensure the appropriate data quality, in particular with regard to the corresponding metadata. A metadata analysis system 22 is used for this purpose, so that the administrator A can more quickly obtain information about the metadata used and possible policy violations.

A reporting module 24 is used to generate appropriate reports from the data and metadata of the database 4, whereby the reports can be customized to reflect new business requirements.

The database 4 can be linked to other databases 26 in order to be able to map and link, for example, a company-wide network of different databases. These databases 26 are connected to the metadata directory 16 in order to enable the automatic synchronization of metadata between the systems. For example, one system can contain information about the data domain and the business owner of a data set, another the ontological position and the associated IT systems for a business object with its ID as a unique identifier.

2 shows the structure of a data set DS.

The data set DS basically contains data D and metadata MD. The metadata MD contains a number of metadata fields MDF1 to MDF4, of which each metadata field MDF1 to MDF4 has certain functions and can be filled with certain data types, e.g. words, characters or status states. The metadata fields MDF1 to MDF4 can only be filled with a defined number and type of metadata, which are stored in the metadata directory 16 and which are defined via the policy database 18.

The user interface 6 differs from the user interface 8. The user interface 6 includes a selection of metadata fields that can be searched for the entities under consideration, e.g. various data objects and data sets. As soon as a searched entity has been selected in this metadata field, all associated information at the level under consideration, e.g. associated data sets, data domains, IT systems and others, is displayed in an automatically generated report and additionally in a semantic network in which all relationships are visible.

The user interface 8 is similar to the user interface 6 in this respect, but allows additional rights for administrators A, who are identified as such by the database system 2, which enable them to edit various fields in the data records DS. When entries are changed, a message is sent to the relevant responsible persons in the user group BG2, who must confirm or reject the corresponding change. A rejection can trigger a change process in which the administrator must make or suggest alternative entries within a certain time frame. If these deadlines are not met by the user group BG2, corresponding warning messages can be sent to the user group BG2.

The administrator A can also receive suggestions from the Large Language Model 14 and the chatbot 12. Based on the analysis of related metadata MD and the performance of semantic and lexical analyses of all metadata in the metadata directory 16, suitable metadata information or responsible users can be suggested to be added to the metadata field under consideration MDF1 to MDF4.

The general user B also benefits from this feature. User B is automatically suggested relationships between the information he is looking for and potentially useful related entities in the semantic network. These suggestions result from a static analysis of correctness based on the degrees of separation and correlation of the metadata of this entity, as well as a linguistic (semantic and lexematic) analysis of the entity itself using an LLM.

3 shows a possible flow chart of a procedure for using the database system 2.

This assumes that there are different users who have editing rights on database system 2 for their respective domains but only read rights for the entire system. Administrator A has additional rights to adjust references between data entities that are not shown in the flowchart.

In a first step VS1, a first user who is responsible for providing metadata MD is assigned the task of entering metadata into database 4. However, the training process must be completed beforehand.

In a second step VS2, the first user completes the corresponding training and is authorized by the database system 2 to use the database system 2. The first user opens the database system 2 via the user interface 6 and checks metadata that is stored about him in the system. If the metadata is correct, the first user can request that the metadata stored about him be adjusted. If the first user has any questions, he can ask them via the corresponding chatbot 12.

In steps VS4 to VS7, steps VS1 to VS3 are repeated for a second user and a third user.

In step VS8, the third user fills out a questionnaire that is specified using threshold values and checks whether it is necessary to enter further metadata. The corresponding threshold values are stored in the metadata analysis system 22.

In a further step VS9, the third user checks whether the metadata suggestions of the chatbot 12 are correct, with the chatbot 12 assisting the third user in selecting the correct metadata. If the preliminary check is negative, the corresponding metadata no longer needs to be entered according to step VS10.

If the test is positive, other users receive a message in a step VS11, which is triggered by the database system 2, with the request to complete the corresponding training, whereby these other users are then activated after the training has been completed. The other users then add the metadata in the database system 2 that still needs to be filled in. In the case of further questions, these can be answered using the chatbot 12.

In a subsequent step VS12, as soon as all users have completed their task, at least one administrator or workflow representative is informed that the relevant metadata has been collected and that the process can be completed.

In a subsequent step VS13, the list of recorded metadata is updated in the metadata directory 16 and is available for reporting via the reporting module 24. The corresponding reports can be created automatically.

In a subsequent step VS14, it can be checked whether the corresponding metadata MD in the system has been updated in the metadata directory 16.

According to the last step VS14, a data asset assessment can be initiated at the next iteration of the listed process after the process steps VS1 to VS11. Changes in the metadata data, reports, hierarchical structure and the like can be adopted automatically.

Although the invention has been illustrated and explained in detail by preferred embodiments, the invention is not limited by the disclosed examples and other variations can be derived therefrom by the person skilled in the art without departing from the scope of the invention. It is therefore clear that a multitude of possible variations exist. It is also clear that embodiments mentioned by way of example really only represent examples that are not to be understood in any way as a limitation of the scope of protection, the possible applications or the configuration of the invention. Rather, the preceding description and the description of the figures enable the person skilled in the art to implement the exemplary embodiments in concrete terms, whereby the person skilled in the art, with knowledge of the disclosed inventive concept, can make a variety of changes, for example with regard to the function or the arrangement of individual elements mentioned in an exemplary embodiment, without departing from the scope of protection defined by the claims and their legal equivalents, such as a further explanation in the description.

list of reference symbols

2

database system

4

database

6, 8

user interface

10

training

12

chatbot

14

LLM

16

metadata directory

18

policy database

20

distributor

22

metadata analysis system

24

reporting module

26

other databases

A

Administrator

B

user

BG1, BG2

user group

D

Data

DS

data set

MD

metadata

MDF1 - MDF4

metadata field

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions.

Cited patent literature

• US 2019/0057147 A1 [0007]

Claims (10)

Database system (2), with - at least one database (4) in which data records (DS) and metadata (MD) associated with the data records (DS) are stored, wherein a plurality of metadata fields (MDF1 - MDF4) are provided for each data record (DS), - at least one user interface (6, 8) for entering data records (DS) including the metadata (MD) by at least one user (B, A), - an authorization system (10) by which users (B, A) or user groups (BG1, BG2) are assigned different roles as well as access rights and/or editing rights for the data records (DS) and/or metadata fields (MDF1 - MDF4), wherein changes and queries of the at least one database (4) are only permitted on this basis, - an explanation system (12, 14) for explaining the metadata fields (MDF1 - MDF4), characterized in that the explanation system (12, 14) has an agent (12) which informs the at least one user (B, A) assists in the input of data records (DS), wherein the agent (12) has a large language model (14) which is connected to a policy database (4) and a metadata directory (16), wherein the agent (12) is arranged to make suggestions for the metadata fields (MDF1 - MDF4) on the basis of the policy database (4) and the metadata directory (16). database system (2) according to claim 1 , characterized in that the authorization system (10) is designed to check a system training status of the at least one user (B, A) and to enforce a system training for the at least one user (B, A). database system (2) according to claim 1 or 2 , characterized in that the agent is a chatbot (12). Database system (2) according to one of the preceding claims, characterized in that a metadata analysis system (22) for checking guideline conformity of the metadata (MD) is connected to the metadata directory (16). Database system (2) according to one of the preceding claims, characterized in that the large language model (14) is focused on the data of the policy database (4) and the metadata directory (16) by means of fine-tuning, prompt engineering and/or retrieval-augmented generation. Database system (2) according to one of the preceding claims, characterized in that a reporting module (24) is provided for creating reports on the data sets (DS), metadata (MD) and/or analyses therefor. Database system (2) according to one of the preceding claims, characterized in that the user interface (6, 8) is designed to display different user interfaces depending on the role, the access rights and/or editing rights, wherein at least a first user interface (6) is provided for data-entering users (B, A) and at least a second user interface (8) is provided for administrators (8). Database system (2) according to one of the preceding claims, characterized in that the large language model (14) is configured to provide metadata suggestions to administrators (A). Database system (2) according to one of the preceding claims, characterized in that a workflow module is provided which is designed to assign tasks for entering data records (DS) and/or metadata (MD) to a plurality of users (B). database system (2) according to claim 9 , characterized in that the workflow module is connected to the metadata directory (16), wherein the workflow module is configured to send metadata (MD) to the metadata directory (16).

Images