CN118819941A - Fault diagnosis method, device, equipment, storage medium and program product - Google Patents

Abstract

The application discloses a fault diagnosis method, a device, equipment, a storage medium and a program product, which relate to the technical field of artificial intelligence and comprise the following steps: receiving input fault description information, and determining an operation and maintenance tool list related to the fault description information; performing a multi-step diagnosis, each step diagnosis comprising: filling the fault description information, the operation and maintenance tool list and the history information into a first prompt word template to obtain a first prompt word; the first prompt word indicates rules and formats for fault diagnosis; the rule includes at least: invoking an abnormal index detection tool to determine an abnormal index, obtaining knowledge based on the abnormal index and the knowledge recall tool to diagnose the cause of the abnormality, and using an operation and maintenance tool to assist diagnosis; inputting the first prompt word into a large language model to generate a decision result; if the decision result indicates a calling tool, the calling tool obtains the calling result and then enters the next diagnosis; and if the decision result indicates to output the diagnosis result, outputting the diagnosis result. The application improves the accuracy of fault diagnosis.

Description

Fault diagnosis method, device, equipment, storage medium and program product

Technical Field

The present application relates to the field of artificial intelligence technology, and in particular to a fault diagnosis method, apparatus, device, storage medium and program product.

Background Art

With the advancement of technology, the software architecture of business systems is becoming more and more complex. In order to ensure the stable and reliable operation of business systems, the mean time between failures (MTBF) of business systems needs to be as long as possible and the mean time to repair (MTTR) as short as possible. This requires rapid fault diagnosis and resolution after a business system failure occurs.

The current fault diagnosis method is mainly manual diagnosis. Generally, after the operation and maintenance department receives customer complaints or equipment alarms, they will feedback to the SRE (Site Reliability Engineering) engineers through office personnel. The SRE engineers will make specific analysis and operations based on their experience. This method is not only inefficient but also costly.

In order to improve the efficiency of fault diagnosis, a solution has been proposed to perform auxiliary diagnosis based on a large language model (LLM). This method improves the efficiency of fault diagnosis, but there is an obvious hallucination problem in the fault diagnosis process, and its accuracy still needs to be further improved.

Summary of the invention

In view of the above problems, the present application provides a fault diagnosis method, apparatus, device, storage medium and program product to improve the accuracy of fault diagnosis. The specific scheme is as follows:

A first aspect of the present application provides a fault diagnosis method, the method comprising:

Receive input fault description information;

Determine a list of operation and maintenance tools related to the fault description information;

A multi-step diagnosis is performed using a large language model based on the fault description information and the operation and maintenance tool list, wherein each diagnosis step includes:

The fault description information, the operation and maintenance tool list and the historical information are added to the first prompt word template to obtain a first prompt word; the first prompt word indicates the rules and format of the large language model when performing fault diagnosis based on the fault description information and the historical information; wherein the rules at least include: calling an abnormal indicator detection tool to determine the abnormal indicator during the diagnosis process, diagnosing the cause of the abnormality based on the abnormal indicator and the knowledge recall tool to obtain knowledge, and using the operation and maintenance tools in the operation and maintenance tool list to assist in diagnosis;

Inputting the first prompt word into the large language model to obtain a decision result generated by the large language model;

If the decision result indicates to call the abnormal indicator detection tool or the operation and maintenance tool or the knowledge recall tool, after calling the abnormal indicator detection tool or the operation and maintenance tool or the knowledge recall tool and obtaining the calling result, proceed to the next diagnosis step; wherein, in the first diagnosis step, the historical information is empty, and in the non-first diagnosis step, the historical information includes the decision result and the calling result obtained in the historical diagnosis step;

If the decision result indicates outputting a diagnosis result, a diagnosis result is output based on the decision result.

In a possible implementation, determining an operation and maintenance tool related to the fault description information includes:

Obtaining a vector representation of the fault description information;

Based on the vector representation of the fault description information and the vector representation of the description information of each operation and maintenance tool pre-stored in the database, calculating the similarity between the description information of each operation and maintenance tool in the database and the fault description information;

Determine a list of operation and maintenance tools related to the fault description information; the similarity between the description information of the operation and maintenance tools in the operation and maintenance tool list and the fault description information is greater than the similarity between the description information of the operation and maintenance tools not in the operation and maintenance tool list and the fault description information.

In a possible implementation, the process of calling the knowledge recall tool to obtain a calling result includes:

The abnormality indicator is input into the knowledge recall tool so that the knowledge recall tool matches the abnormality indicator with each knowledge item in the knowledge base and outputs the fault analysis steps in the knowledge item matching the abnormality indicator as the call result.

In a possible implementation, each knowledge item in the knowledge base includes the following fields: fault name, fault description, fault-related indicators, and fault analysis steps;

The knowledge items are extracted in the following way:

The knowledge document is constructed into a document fragment tree according to the chapter structure; the root node of the document fragment tree corresponds to the knowledge document; each non-root node corresponds to a chapter in the knowledge document; the root node includes the title of the knowledge document, if the non-root node is a leaf node, the non-root node includes all the contents of the corresponding chapter, if the non-root node is not a leaf node, the non-root node includes the chapter title of the corresponding chapter and an overview of the corresponding chapter;

The large language model is called to traverse the nodes in the document fragment tree, and each time a node is traversed, knowledge items are extracted from the traversed node; each knowledge item is extracted from a node.

In a possible implementation, the root node also includes a summary of the title of the knowledge document;

If the non-root node is a leaf node, the non-root node also includes a summary of all contents of the corresponding chapter;

If the non-root node is not a leaf node, the non-root node further includes a chapter title of the corresponding chapter and a summary of the overview of the corresponding chapter.

In a possible implementation, calling the large language model to traverse the nodes in the document fragment tree, and extracting knowledge items from each traversed node, including:

The document fragment tree is added to the second prompt word template to obtain a second prompt word; the second prompt word instructs the large language model to traverse the nodes in the document fragment tree one by one, and for each traversed node, write at least one detailed diagnostic information that conforms to a preset format according to the content of the node, and the detailed diagnostic information extracted from different nodes is different;

Each piece of detailed diagnostic information is a knowledge item.

A second aspect of the present application provides a fault diagnosis device, comprising:

A receiving module, used for receiving input fault description information;

A tool determination module, used to determine a list of operation and maintenance tools related to the fault description information;

A diagnosis module, used to use a large language model to perform multi-step diagnosis based on the fault description information and the operation and maintenance tool list, wherein each diagnosis step includes: adding the fault description information, the operation and maintenance tool list and the historical information to a first prompt word template to obtain a first prompt word; the first prompt word indicates the rules and format of the large language model when performing fault diagnosis based on the fault description information and the historical information; wherein the rules at least include: calling an abnormal indicator detection tool to determine the abnormal indicator during the diagnosis process, obtaining knowledge based on the abnormal indicator and the knowledge recall tool to diagnose the cause of the abnormality, and using the operation and maintenance tools in the operation and maintenance tool list to assist in diagnosis; inputting the first prompt word into the large language model to obtain a decision result generated by the large language model; if the decision result indicates calling the abnormal indicator detection tool or the operation and maintenance tool or the knowledge recall tool, after calling the abnormal indicator detection tool or the operation and maintenance tool or the knowledge recall tool and obtaining the calling result, entering the next diagnosis step; wherein in the first diagnosis step, the historical information is empty, and in the non-first diagnosis step, the historical information includes the decision result and the calling result obtained by the historical diagnosis step;

An output module is used to output a diagnosis result based on the decision result if the decision result indicates to output a diagnosis result.

A third aspect of the present application provides a computer program product, comprising computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the fault diagnosis method of the first aspect or any implementation of the first aspect.

A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

The memory is used to store computer programs;

The processor is used to execute the computer program so that the electronic device can implement the fault diagnosis method of the first aspect or any implementation manner of the first aspect.

A fifth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can use the fault diagnosis method of the above-mentioned first aspect or any implementation method of the first aspect.

By means of the above technical scheme, the fault diagnosis method, apparatus, device, storage medium and program product provided by the present application, after receiving the input fault description information, determine the operation and maintenance tool list related to the fault description information, and use the large language model to perform multi-step diagnosis based on the fault description information and the operation and maintenance tool list, wherein each diagnosis step includes: adding the fault description information, the operation and maintenance tool list and the historical information to the first prompt word template to obtain the first prompt word; the first prompt word indicates the rules and format of the large language model when performing fault diagnosis based on the fault description information and the historical information; wherein the above rules at least include: calling the abnormal indicator detection tool to determine the abnormal indicator during the diagnosis process, based on Abnormal indicators and knowledge recall tools obtain knowledge to diagnose the cause of the abnormality, and tools in the operation and maintenance tool list can be used to assist in diagnosis; the first prompt word is input into the large language model to obtain the decision result generated by the large language model; if the decision result indicates to call the abnormal indicator detection tool or the operation and maintenance tool or the knowledge recall tool, after calling the abnormal indicator detection tool or the operation and maintenance tool or the knowledge recall tool and obtaining the call result, enter the next diagnosis step; wherein, in the first diagnosis step, the historical information is empty, and in non-first diagnosis steps, the historical information includes the decision results and call results obtained in the historical diagnosis steps; if the decision result indicates to output the diagnosis result, the diagnosis result is output based on the decision result. In the process of fault diagnosis based on the large language model, the present application performs multi-step automatic fault diagnosis by calling tools and related knowledge, overcomes the hallucination problem of the large language model in the fault diagnosis process, and improves the accuracy of fault diagnosis.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

FIG1 is a flowchart of an implementation of the fault diagnosis method provided by the present application;

FIG2 is a schematic diagram of a structure of a document fragment tree provided by the present application;

FIG3 is another schematic diagram of the structure of the document fragment tree provided by the present application;

FIG4 is a schematic diagram of a principle example of fault diagnosis based on a large language model provided by the present application;

FIG5 is a schematic diagram of a structure of a fault diagnosis device provided by the present application;

FIG. 6 is a schematic diagram of the structure of an electronic device provided in the present application.

DETAILED DESCRIPTION

The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

As shown in FIG1 , a flowchart of an implementation of a fault diagnosis method provided in an embodiment of the present application may include:

Step S101: receiving input fault description information.

The fault description information may be edited by SRE engineers based on device alarms or customer complaints received by the operation and maintenance department. As an example, the fault description information may include, but is not limited to: occurrence time, abnormal description information, severity level (e.g., warning or severe), and additional features (e.g., abnormal status, etc.). If the fault has ended, the fault description information may also include the end time.

Step S102: Determine a list of operation and maintenance tools related to the fault description information.

When diagnosing a problem, SRE not only uses experience and knowledge, but also frequently uses some operation and maintenance tools (such as indicator monitoring systems, log monitoring systems, operating system command lines, SQL optimization tools, etc.). In order for the Large Language Model (LLM) to diagnose faults, this application also needs to let the LLM know which operation and maintenance tools can be called.

Operation and maintenance tools can also be called optimization tools. In order to facilitate the large language model to distinguish different operation and maintenance tools, this application establishes a structured hierarchy to classify and organize operation and maintenance tools based on the characteristics of operation and maintenance. Among them, the first-level classification is divided into three categories: monitoring tools, anomaly detection tools, and optimization configuration tools. Corresponding to monitoring tools and optimization configuration tools, the second-level classification is divided according to different operation and maintenance objects, and corresponding to anomaly detection tools, the second-level classification is divided according to the detection method. As shown in Table 1, an example of classifying operation and maintenance tools in a database scenario (i.e., a scenario for fault diagnosis of a database) provided in an embodiment of the present application. In other scenarios (for example, operating system scenarios), operation and maintenance tools and the way of classifying operation and maintenance are usually different.

Table 1

The operation and maintenance tools in the operation and maintenance tool list related to the fault description information may be determined based on the similarity between the fault description information and the description information of the operation and maintenance tools.

Step S103: Use the large language model to perform multi-step diagnosis based on the fault description information and the operation and maintenance tool list, where each diagnosis step includes:

The fault description information, the operation and maintenance tool list and the historical information are added to the first prompt word template to obtain the first prompt word; the first prompt word indicates the rules and format of the large language model when performing fault diagnosis based on the fault description information and the historical information; wherein the rules at least include: calling the abnormal indicator detection tool to determine the abnormal indicator during the diagnosis process, diagnosing the cause of the abnormality based on the abnormal indicator and the knowledge recall tool to obtain knowledge, and the operation and maintenance tools in the operation and maintenance tool list can be used to assist in the diagnosis.

In addition to the above rules and formats, the first prompt word may also include the following information:

1) Description of the current fault diagnosis expert, used to describe the role and function of the current diagnosis expert. This information is the content of the first prompt word template.

2) Alarm details, including but not limited to: occurrence time, abnormal description information, severity level (such as warning or severe, etc.), additional features (such as abnormal status, etc.). This information is added to the first prompt word template, that is, the fault description information entered by the user. If the alarm has been eliminated, the alarm details can also include the end time.

3) List of tools needed to diagnose the current fault. In the fault diagnosis process of this application, some tools must be used, such as knowledge recall tools and abnormal indicator detection tools. Therefore, these tools that must be used can be pre-written into the information in the first prompt word template (that is, the information that characterizes the tools that must be used is the information in the first prompt word template). Some tools are only used in certain situations. These tool lists are recalled from the database, that is, the operation and maintenance tool list determined in step S102.

In order to allow the large language model to use the fault knowledge in the knowledge base when diagnosing faults, this application adds a knowledge recall tool "match_diagnose_knowledge" to the first prompt word template. As an example, the definition of the knowledge recall tool is as follows:

"{

"name": "match_diagnose_knowledge",

"description": "Search for relevant fault diagnosis knowledge in the diagnostic knowledge base.",

"parameters": {

"type": "object",

"properties": {

"metric_name": {

"type": "string",

"description": "Indicator name, used to search for indicator-related knowledge"

}

},

"required": [

"metric_name"

]

}

}”

4) Historical messages, including decision results generated by the large language model, tool call results, etc. This information is updated after each step of diagnosis.

As shown below, an example of the first prompt word provided in the embodiment of the present application is:

“

You are an IT system failure expert, and your company's IT system has encountered an exception. The start time of the exception is ${start_time}, and the end time is ${end_time}.

The abnormal alarm information is as follows:

${alert_info}

# Response rules and format

- During the diagnostic process, you can use the following tools:

${tools}

===============

- Return the following format to call the tool:

Thinking: (your thinking)

Action: (An action name, can be one of: [metric_abnormal_detect, match_diagnose_knowledge, optimize_index_selection, Speak], pay attention to upper and lower case)

Action input: (parameters of the action)

You can first call the tool to determine the abnormal indicators, the format is as follows:

Thinking: Now that I have obtained the start and end time of the anomaly, check whether the CPU usage during this time period is abnormal.

Action: metric_abnormal_detect

Action input: {"start_time": ${start_time}, "end_time": ${end_time}, "metric_name": "cpu_usage"}

Next, you must use the tool to diagnose the root cause. You must use the following format (no other options are allowed):

Think: Use the knowledge gained from metrics and match_diagnose_knowledge to diagnose the cause of the anomaly.

Action: match_diagnose_knowledge

Action input: {"metric_name": "cpu_usage"}

In addition, if you find that an index is missing, you need to call the optimize_index_selection API to get the recommended index using the following format:

Thinking: Since the index is missing, I need to call the optimize_index_selection API to get the recommended index.

Action: optimize_index_selection

Action input: {"start_time": ${start_time},"end_time": ${end_time}}

Once you have the observations from match_diagnose_knowledge, analyze the root cause and announce it to other experts, using the following format:

Thinking: I now know the root cause of the anomaly.

Action: Speak

Action input: ({"diagnose": the root cause you found, "solution": the optimal solution for the root cause, separated by '\n', "knowledge": the diagnostic knowledge you used})

===============

The conversation history is as follows

${chat_history}

The tool execution results are as follows

${tool_observation}

Remember to pay attention to the response format instructions and strictly follow the rules specified above!

Based on the above history, what will you do next as ${agent_name}?

”

In the example of the first prompt word, "alert_info" is the fault description information entered by the user;

"tools" includes a list of operation and maintenance tools related to the fault description information, as well as the operation and maintenance tools required for fault diagnosis;

The tools in “[metric_abnormal_detect, match_diagnose_knowledge, optimize_index_selection, Speak]” are the tools in “tools”. They are written here to emphasize the operation and maintenance tools for large models.

Among them, "optimize_index_selection" is an operation and maintenance tool related to the fault description information, and the other tools are tools that come with the first prompt word template. Among them, metric_abnormal_detect is a tool for abnormal metric detection, match_diagnose_knowledge is a knowledge recall tool, and Speak is an output tool; of course, the list of operation and maintenance tools related to the fault description information may also include tools that come with the first prompt word template. In this case, "tools" needs to be deduplicated.

"{chat_history}" is the historical decision results output by the large language model;

"{tool_observation}" is the tool call result to be updated;

"agent_name" indicates the name of the fault expert. This information is included in the first prompt word template.

The first prompt word is input into the large language model to obtain a decision result generated by the large language model.

Based on the first prompt word example above, the format of the decision result generated by the large language model is:

“

Thinking: xxx

Action: match_diagnose_knowledge

Action input: {“metric_name”:“cpu”}

”

The example of the above decision result indicates calling the knowledge recall tool "match_diagnose_knowledge". The parameter required when calling the knowledge recall tool is the indicator name, that is, cpu.

If the decision result indicates to call a tool (an abnormal indicator detection tool or an operation and maintenance tool in the operation and maintenance tool list or a knowledge recall tool), after calling the tool and obtaining the calling result, proceed to the next diagnosis step; wherein, in the first diagnosis step, the historical information is empty, and in the non-first diagnosis step, the historical information includes the decision results and calling results obtained in the historical diagnosis step.

When the decision result indicates to call the abnormal indicator detection tool, operation and maintenance tool, or knowledge recall tool, the abnormal indicator detection tool, operation and maintenance tool, or knowledge recall tool is called according to the instruction of the decision result. After calling the abnormal indicator detection tool, operation and maintenance tool, or knowledge recall tool and obtaining the calling result, the next step of diagnosis can be carried out.

In the present application, each step of diagnosis requires the use of a large language model. When calling the large language model, a prompt word needs to be input into the large language model. Each time the prompt word is input into the large language model, the fault description information, the operation and maintenance tool list and the historical information need to be added to the first prompt word template. That is to say, the difference in prompt words when the large language model is called for different fault diagnoses lies in the difference in historical information (i.e., {chat_history} and {tool_observation}). Therefore, the call result can be added to the "{tool_observation}" in the first prompt word, and the decision result can be added to the "{chat_history}" in the first prompt word to obtain an updated first prompt word, and the larger language model is called based on the updated first prompt word to perform the next diagnostic step.

Step S104: If the decision result indicates to output a diagnosis result, output a diagnosis result based on the decision result.

In the present application, if the decision result indicates calling the tool "Speak", it means that the decision result indicates outputting a diagnosis result, and then the diagnosis result is output based on the decision result.

Optionally, the decision result indicates output of information in the following format:

"diagnose": the root cause you discovered;

"solution": optimized solution to the root cause;

"knowledge": The diagnostic knowledge you use.

The fault diagnosis method provided in the embodiment of the present application, in the process of fault diagnosis based on the large language model, not only refers to relevant knowledge, but also calls relevant tools, so that the large language model can perceive the current system environment, and perform step-by-step fault diagnosis based on system environment information and relevant knowledge, thereby overcoming the hallucination problem of the large language model in the fault diagnosis process, thereby improving the accuracy of fault diagnosis.

In an optional embodiment, an implementation manner of determining the operation and maintenance tool related to the fault description information may be:

Obtain a vector representation of the fault description information. The fault description information is text information. You can first obtain the vector representation of each word in the fault description information, and then fuse the vector representations of each word to obtain the vector representation of the fault description information. Alternatively, you can use the BERT model to process the fault description information to obtain the vector representation of the fault description information.

Based on the vector representation of the fault description information and the vector representation of the description information of each operation and maintenance tool pre-stored in the database, the similarity between the description information of each operation and maintenance tool in the database and the fault description information is calculated.

The description information (i.e. definition) of the operation and maintenance tool can be text information in the following format:

"{

"name": "Tool name",

"description": "Detailed description of the tool",

"parameters": "Parameters when calling the tool"

}”

For example, the definition of a single-metric anomaly detection tool is:

"{

"name": "metric_abnormal_detect",

"description": "Abnormal indicator inspection."

"parameters": {'type': 'object', 'properties': {}}

}

”

For the indicator anomaly detection tool, this application pre-vectorizes the definition of the indicator anomaly detection tool into text and stores it in a vector database, so that the vector representation can be directly read when the operation and maintenance tool is recalled later to improve the processing speed.

Optionally, the following formula (1) can be used to calculate the similarity between the fault description information and the jth operation and maintenance tool in the database:

(1)

Where s represents the fault description information; t _j represents the description information of the j-th operation and maintenance tool in the database; sim(s, t _j ) represents the similarity between the fault description information and the j-th operation and maintenance tool in the database; emb() represents vectorized processing.

Determine a list of operation and maintenance tools related to the fault description information; wherein the similarity between the description information of the operation and maintenance tools in the operation and maintenance tool list and the fault description information is greater than the similarity between the description information of the operation and maintenance tools not in the operation and maintenance tool list and the fault description information.

An operation and maintenance tool in the database whose similarity with the fault description information is greater than a threshold may be determined as an operation and maintenance tool related to the fault description information.

Alternatively, the operation and maintenance tools in the database may be sorted in descending order according to the similarity with the fault description information, and the top N operation and maintenance tools may be determined as the operation and maintenance tools related to the fault description information.

In an optional embodiment, one implementation method of invoking the knowledge recall tool to obtain the invocation result may be:

The above abnormal indicators are input into the knowledge recall tool so that the knowledge recall tool matches the abnormal indicators with each knowledge item in the knowledge base and outputs the fault analysis steps in the knowledge item matching the abnormal indicators as the call result.

As an example, the BM25 algorithm can be used to calculate the correlation score between the abnormality index and each knowledge item in the knowledge base. Specifically, the following formulas (2)-(3) can be used to calculate the correlation score between the abnormality index and the fault-related index in each knowledge item in the knowledge base:

(2)

(3)

in:

It is the correlation score between the anomaly indicator set Q and the knowledge item D.

is the i-th index in the set Q.

Is an indicator The frequency of occurrence in knowledge item D.

is the length of knowledge item D.

It is the average length of all knowledge items in the knowledge base.

k and b are adjustable parameters. Typically, k is between 1.2 and 2, and b is usually set to 0.75.

Is an indicator The inverse document frequency of .

N is the total number of knowledge items in the knowledge base.

The knowledge base contains indicators The number of knowledge items.

In an optional embodiment, each knowledge item in the knowledge base includes the contents of the following fields: fault name, fault description, fault-related indicators, and fault analysis steps.

As shown in Table 2, the description information of each field of a knowledge item provided in the embodiment of the present application is as follows:

Table 2

Optionally, the knowledge items of the above structure can be extracted by:

The knowledge document is constructed into a document fragment tree according to the chapter structure; the root node of the document fragment tree corresponds to the knowledge document; each non-root node corresponds to a chapter in the knowledge document; the root node includes the title of the knowledge document, if the non-root node is a leaf node, the non-root node includes all the contents of the corresponding chapter, if the non-root node is not a leaf node, the non-root node includes the chapter title of the corresponding chapter and an overview of the corresponding chapter.

In this application, knowledge documents may include but are not limited to the following: historical troubleshooting documents, operation and maintenance documents, instruction manuals, etc.

The inventors of the present application have discovered that in order to adapt to the requirements of large language models for the amount of input data, knowledge documents need to be divided into multiple segments. However, the prior art divides knowledge documents into multiple segments according to the number of words or paragraphs. This segmentation method has the problem of inaccurate knowledge recall and low accuracy of fault diagnosis.

In order to enable the large language model to apply knowledge more accurately and improve the accuracy of fault diagnosis, the present application divides the knowledge document according to the chapter structure of the knowledge document when segmenting the knowledge document. If a segment exceeds the maximum data block size that the large language model can handle (for example, 8k words), the segment is further subdivided recursively. Then, a tree structure is constructed based on the relationship between the chapters. As an example, in the tree structure, the root node is the knowledge document title, and the remaining nodes represent the segmented document blocks. For the i-th node (not the root node), it represents the i-th chapter. If there are child nodes, its child nodes represent the child chapters of the i-th chapter. When the i-th node is a leaf node, the i-th node is the content of the i-th chapter. When the i-th node is not a leaf node, the i-th node is the title of the i-th chapter, or the title and overview.

As shown in Figure 2, it is a schematic diagram of the structure of the document fragment tree provided by the embodiment of the present application. In this example, each node contains only the content in the knowledge document, for example, the root node (the node labeled 1) is the document title; the child nodes of the root node (the nodes labeled 1.1, 1.2 and 1.3, representing the first chapter, the second chapter and the third chapter of the knowledge document respectively) are the highest-level titles in the document, or they can be overviews of titles and chapters; the leaf node labeled 1.1.1 is the content of a sub-chapter of the first chapter, and so on.

As shown in Figure 3, another structural diagram of the document fragment tree provided in the embodiment of the present application is shown. In this example, each node contains not only the content in the knowledge document, but also a summary of the content. For example, the root node is the document title and the summary of the document title; the child nodes of the root node are the highest-level title in the document and its summary, or it can be an overview of the title and chapter, and a summary of the overview of the title and chapter; the leaf nodes are all the content and content summaries of the corresponding chapters.

The large language model is called to traverse the nodes in the document fragment tree. Each time a node is traversed, knowledge items are extracted from the traversed node; each knowledge item is extracted from a node.

Optionally, the document fragment tree can be added to the second prompt word template to obtain a second prompt word; the second prompt word instructs the large language model to traverse the nodes in the document fragment tree one by one, and for each traversed node, write at least one detailed diagnostic information that conforms to a preset format based on the content of the node, and the detailed diagnostic information extracted from different nodes is different.

Each piece of detailed diagnostic information is a knowledge item.

An example of the second prompt word provided in the embodiment of the present application is as follows:

“

Your task is to write a detailed diagnostic knowledge based on the content of a given document paragraph in the document fragment tree. (e.g.: about high IO usage, about slow query).

Document fragment tree: xxx

Try to extract knowledge blocks one by one in the order of the chapters currently being read. Each knowledge block must strictly follow the JSON format.

The following is the JSON format of the knowledge fragment:

{

"name": "high_io_usage",

"desc": "High IO usage may cause performance degradation. It is important to identify the table or query that causes high IO usage and optimize it.",

"steps": "Step 1: Check IO statistics of user tables by querying 'pg_stat_user_tables' and 'pg_stat_user_indexes'.\nStep 2: Check IO statistics of user tables and indexes by querying 'pg_statio_user_tables' and 'pg_statio_user_indexes'.\nStep 3: Identify tables or indexes with high IO and analyze the queries that cause high IO.\nStep 4: Optimize queries or consider adding indexes to improve IO performance.",

"metrics": ["pg_stat_user_tables", "pg_stat_user_indexes", "pg_statio_user_tables", "pg_statio_user_indexes"]"

}

Note: "metrics" in the knowledge fragment data should be specific indicator names such as "user table status", "user index status", "user table I/O status", "user index I/O status", etc., instead of "metric1" and "metric2"; the "desc" attribute should be a detailed description; "steps" should describe the detailed steps to diagnose the current problem.

Do not repeatedly extract the following knowledge fragments:

{Existing operation and maintenance knowledge items}

Do not search for the following subsections repeatedly:

{Already used chapters}

{Summary of Chapter I}

”

In the example of the second prompt word above, "existing operation and maintenance knowledge items" refers to knowledge items that have been generated, which can be specifically represented by the fault names in the generated knowledge items. That is to say, if a knowledge item with the same fault name has been extracted before, it will not be extracted later. "Chapters that have been used" refers to text fragments that have been traversed before, that is, nodes that have been traversed, that is, each node is traversed only once. "Summary of Chapter i" refers to the summary of the node currently traversed, that is, the summary of the current node is traversed only once.

As shown in FIG4 , a schematic diagram of a principle example of fault diagnosis based on a large language model provided in an embodiment of the present application is shown. In the schematic diagram:

1. The fault description information entered by the user is given to the prompt word generator to generate a prompt word.

2. The prompt word is input into the large language model to obtain the decision result.

3. If the decision result indicates an output result, the intelligent executive returns the diagnosis result to the user.

4. If the decision result indicates to call a tool, the intelligent executor calls the tool or recalls knowledge from the knowledge base according to the decision result, and returns the execution result (tool call result or recalled knowledge) to the prompt word generator to generate a new prompt word.

5. The new prompt word is input into the large language model to obtain a new decision result, and then return to execute steps 3-4.

Corresponding to the method embodiment, the present application further provides a fault diagnosis device. A structural schematic diagram of the fault diagnosis device provided in the embodiment of the present application is shown in FIG5 , and may include:

Receiving module 501, tool determination module 502, diagnosis module 503 and output module 504;

The receiving module 501 is used to receive input fault description information;

The tool determination module 502 is used to determine a list of operation and maintenance tools related to the fault description information;

The diagnosis module 503 is used to use a large language model to perform multi-step diagnosis based on the fault description information and the operation and maintenance tool list, wherein each diagnosis step includes: adding the fault description information, the operation and maintenance tool list and the historical information to a first prompt word template to obtain a first prompt word; the first prompt word indicates the rules and format of the large language model when performing fault diagnosis based on the fault description information and the historical information; wherein the rules at least include: calling an abnormal indicator detection tool to determine the abnormal indicator during the diagnosis process, obtaining knowledge based on the abnormal indicator and the knowledge recall tool to diagnose the cause of the abnormality, and using the operation and maintenance tools in the operation and maintenance tool list to assist in diagnosis; inputting the first prompt word into the large language model to obtain a decision result generated by the large language model; if the decision result indicates calling an operation and maintenance tool or a knowledge recall tool, after calling the operation and maintenance tool or the knowledge recall tool and obtaining the calling result, entering the next diagnosis step; wherein, in the first diagnosis step, the historical information is empty, and in a non-first diagnosis step, the historical information includes the decision result and the calling result obtained by the historical diagnosis step;

The output module 504 is configured to output a diagnosis result based on the decision result if the decision result indicates to output a diagnosis result.

The fault diagnosis device provided in the embodiment of the present application, in the process of fault diagnosis based on the large language model, not only refers to relevant knowledge, but also calls relevant tools, so that the large language model can perceive the current system environment, and perform step-by-step fault diagnosis according to the system environment information and relevant knowledge, thereby overcoming the hallucination problem of the large language model in the fault diagnosis process, thereby improving the accuracy of fault diagnosis.

In an optional embodiment, when the tool determination module 502 determines the operation and maintenance tool related to the fault description information, it is used to:

Obtaining a vector representation of the fault description information;

Based on the vector representation of the fault description information and the vector representation of the description information of each operation and maintenance tool pre-stored in the database, calculating the similarity between the description information of each operation and maintenance tool in the database and the fault description information;

Determine a list of operation and maintenance tools related to the fault description information; the similarity between the description information of the operation and maintenance tools in the operation and maintenance tool list and the fault description information is greater than the similarity between the description information of the operation and maintenance tools not in the operation and maintenance tool list and the fault description information.

In an optional embodiment, the process of the diagnosis module 503 calling the knowledge recall tool to obtain the calling result includes:

The abnormality indicator is input into the knowledge recall tool so that the knowledge recall tool matches the abnormality indicator with each knowledge item in the knowledge base and outputs the fault analysis steps in the knowledge item matching the abnormality indicator as the call result.

In an optional embodiment, each knowledge item in the knowledge base includes the contents of the following fields: fault name, fault description, fault-related indicators, and fault analysis steps;

The device further comprises: a knowledge extraction module, configured to extract knowledge items in the following manner:

The knowledge document is constructed into a document fragment tree according to the chapter structure; the root node of the document fragment tree corresponds to the knowledge document; each non-root node corresponds to a chapter in the knowledge document; the root node includes the title of the knowledge document, if the non-root node is a leaf node, the non-root node includes all the contents of the corresponding chapter, if the non-root node is not a leaf node, the non-root node includes the chapter title of the corresponding chapter and an overview of the corresponding chapter;

The large language model is called to traverse the nodes in the document fragment tree, and each time a node is traversed, knowledge items are extracted from the traversed node; each knowledge item is extracted from a node.

In an optional embodiment, the root node also includes a summary of the title of the knowledge document;

If the non-root node is a leaf node, the non-root node also includes a summary of all contents of the corresponding chapter;

If the non-root node is not a leaf node, the non-root node further includes a chapter title of the corresponding chapter and a summary of the overview of the corresponding chapter.

In an optional embodiment, the knowledge extraction module calls the large language model to traverse the nodes in the document fragment tree, and extracts knowledge items from each traversed node, including:

The document fragment tree is added to the second prompt word template to obtain a second prompt word; the second prompt word instructs the large language model to traverse the nodes in the document fragment tree one by one, and for each traversed node, write at least one detailed diagnostic information that conforms to a preset format according to the content of the node, and the detailed diagnostic information extracted from different nodes is different;

Each piece of detailed diagnostic information is a knowledge item.

An electronic device is also provided in an embodiment of the present application. Referring to FIG6, a structural diagram of an electronic device suitable for implementing an embodiment of the present application is shown. The electronic device in the embodiment of the present application may be a terminal device, such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, etc.; of course, the terminal device may independently complete the above-mentioned fault diagnosis method, or may interact with the server device to complete the above-mentioned fault diagnosis method. The electronic device shown in FIG6 is only an example and should not bring any limitation to the functions and scope of use of the embodiment of the present application.

As shown in FIG6 , the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to the bus 604.

Typically, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 608 including, for example, a memory card, a hard disk, etc.; and communication devices 609. The communication device 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although FIG. 6 shows an electronic device with various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.

An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any fault diagnosis method provided in the embodiment of the present application.

A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any fault diagnosis method provided in the embodiment of the present application.

It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product. Professionals and technicians may use different methods to implement the described functions for each specific solution, but such implementation should not be considered beyond the scope of this application.

The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. A fault diagnosis method, the method comprising:

receiving input fault description information;

determining an operation and maintenance tool list related to the fault description information;

performing multi-step diagnosis based on the fault description information and the operation and maintenance tool list by using a large language model, wherein each diagnosis step comprises:

Filling the fault description information, the operation and maintenance tool list and the history information into a first prompt word template to obtain a first prompt word; the first prompt word indicates rules and formats of the large language model when fault diagnosis is performed based on the fault description information and the historical information; wherein the rule at least comprises: invoking an abnormal index detection tool to determine an abnormal index in a diagnosis process, diagnosing an abnormal cause based on the abnormal index and knowledge acquired by a knowledge recall tool, and using an operation and maintenance tool in the operation and maintenance tool list to assist diagnosis;

Inputting the first prompt word into the large language model to obtain a decision result generated by the large language model;

If the decision result indicates that the abnormal index detection tool or the operation and maintenance tool or the knowledge recall tool is called, after the abnormal index detection tool or the operation and maintenance tool or the knowledge recall tool is called, a calling result is obtained, the next diagnosis step is entered; in the first diagnosis step, the history information is empty, and in the non-first diagnosis step, the history information comprises a decision result and a calling result obtained in the history diagnosis step;

and outputting a diagnosis result based on the decision result if the decision result indicates to output the diagnosis result.

2. The method of claim 1, wherein the determining the operation and maintenance tool associated with the fault description information comprises:

obtaining a vector representation of the fault description information;

Calculating the similarity between the description information of each operation and maintenance tool in the database and the fault description information based on the vector representation of the fault description information and the vector representation of the description information of each operation and maintenance tool stored in the database in advance;

determining an operation and maintenance tool list related to the fault description information; and the similarity between the description information of the operation and maintenance tools in the operation and maintenance tool list and the fault description information is larger than that between the description information of the operation and maintenance tools not in the operation and maintenance tool list and the fault description information.

3. The method of claim 1, wherein invoking the knowledge recall tool to obtain a call result comprises:

And inputting the abnormal index into the knowledge recall tool so that the knowledge recall tool can match the abnormal index with each knowledge item in a knowledge base, and outputting a fault analysis step in the knowledge item matched with the abnormal index as a calling result.

4. A method according to claim 3, wherein each knowledge item in the knowledge base comprises the contents of the following fields: fault name, fault description, fault related index and fault analysis;

The knowledge item is extracted by the following steps:

constructing a document fragment tree of the knowledge document according to the chapter structure; the root node of the document fragment tree corresponds to the knowledge document; each non-root node corresponds to a section in the knowledge document; the root node comprises the title of the knowledge document, if the non-root node is a leaf node, the non-root node comprises all contents of the corresponding chapter, and if the non-root node is not a leaf node, the non-root node comprises the chapter title of the corresponding chapter and the summary of the corresponding chapter;

Invoking the large language model to traverse the nodes in the document fragment tree, traversing to one node each time, and extracting knowledge items from the traversed node; each knowledge item is extracted from a node.

5. The method of claim 4, wherein the root node further comprises a summary of a title of the knowledge document;

if the non-root node is a leaf node, the non-root node further comprises a summary of all content of the corresponding chapter;

if the non-root node is not a leaf node, the non-root node also includes a section title of the corresponding section and a summary of the corresponding section.

6. The method of claim 4, wherein the invoking the large language model traverses nodes in the document snippet tree, each traversing to a node, extracting knowledge items from the traversed nodes, comprising:

Filling the document fragment tree into a second prompting word template to obtain a second prompting word; the second prompt word indicates the large language model to traverse the nodes in the document fragment tree one by one, and for each traversed node, at least one piece of detailed diagnosis information which accords with a preset format is written according to the content of the node, and the detailed diagnosis information extracted from different nodes is different;

each piece of detailed diagnostic information is a knowledge item.

7. A fault diagnosis apparatus characterized by comprising:

the receiving module is used for receiving the input fault description information;

A tool determining module for determining an operation and maintenance tool list related to the fault description information;

A diagnostic module for performing multi-step diagnosis based on the fault description information and the operation and maintenance tool list by using a large language model, wherein each diagnosis step comprises: filling the fault description information, the operation and maintenance tool list and the history information into a first prompt word template to obtain a first prompt word; the first prompt word indicates rules and formats of the large language model when fault diagnosis is performed based on the fault description information and the historical information; wherein the rule at least comprises: invoking an abnormal index detection tool to determine an abnormal index in a diagnosis process, diagnosing an abnormal cause based on the abnormal index and knowledge acquired by a knowledge recall tool, and using an operation and maintenance tool in the operation and maintenance tool list to assist diagnosis; inputting the first prompt word into the large language model to obtain a decision result generated by the large language model; if the decision result indicates that the abnormal index detection tool or the operation and maintenance tool or the knowledge recall tool is called, after the abnormal index detection tool or the operation and maintenance tool or the knowledge recall tool is called, a calling result is obtained, the next diagnosis step is entered; in the first diagnosis step, the history information is empty, and in the non-first diagnosis step, the history information comprises a decision result and a calling result obtained in the history diagnosis step;

And the output module is used for outputting a diagnosis result based on the decision result if the decision result indicates to output the diagnosis result.

8. A computer program product comprising computer readable instructions which, when run on an electronic device, cause the electronic device to implement the fault diagnosis method of any one of claims 1 to 6.

9. An electronic device comprising at least one processor and a memory coupled to the processor, wherein:

the memory is used for storing a computer program;

The processor is configured to execute the computer program to enable the electronic device to implement the fault diagnosis method as claimed in any one of claims 1 to 6.

10. A computer storage medium carrying one or more computer programs which, when executed by an electronic device, enable the electronic device to implement the fault diagnosis method of any one of claims 1 to 6.