CN117272207A - Data center anomaly analysis method and system - Google Patents

Abstract

The embodiment of the invention provides a data center abnormality analysis method and system, and provides a data center operation abnormality analysis and prediction method. First, operation data of a target data center is acquired. And then, loading the operation data of the target data center into an anomaly analysis prediction network generated through priori learning, wherein the anomaly analysis prediction network performs knowledge learning generation based on the associated data center operation data and static scheduling data. The anomaly analysis prediction network can generate a corresponding operation anomaly class. Finally, according to the determined operation abnormality category, abnormality diagnosis data is determined. The method can effectively analyze and predict the possible abnormal situation in the running process of the data center, and provides powerful support for the management and maintenance of the data center.

Description

Data center anomaly analysis method and system

Technical field

The present invention relates to the technical field of data centers, and specifically to a data center anomaly analysis method and system.

Background technique

With the development of information technology, data centers serve as core facilities for storing and processing large amounts of data, and their operating status is crucial to the stability of the entire IT system. However, due to the complexity of the data center and the variability of the operating environment, various types of abnormal situations may occur during operation, such as hardware failures, software errors, network interruptions, etc.

Traditional anomaly detection methods usually rely on manual monitoring or preset threshold alarms, and these methods have certain limitations. For example, manual monitoring is inefficient and error-prone; the preset threshold method cannot adapt to the dynamic changes in the data center operating environment and may produce a large number of false positives or false negatives.

In addition, when an abnormality occurs in the data center, fault diagnosis needs to be performed quickly and accurately to minimize the service interruption time caused by the failure. However, current fault diagnosis methods often rely on the experience of professional maintenance personnel, and may not be able to make effective diagnoses when facing complex or unknown types of anomalies.

Therefore, there is an urgent need for a new method that can effectively analyze and predict abnormal situations during data center operation, and perform fast and accurate fault diagnosis based on abnormal categories.

Contents of the invention

In view of this, the purpose of embodiments of the present invention is to provide a data center abnormality analysis method and system. The present invention provides a data center abnormal operation analysis and prediction method. First, obtain the operating data of the target data center. Then, the operation data of the target data center is loaded into an anomaly analysis and prediction network generated through prior learning. This anomaly analysis and prediction network is generated based on knowledge learning based on the associated data center operation data and static scheduling data. Through this abnormality analysis and prediction network, corresponding operating abnormality categories can be generated. Finally, abnormal diagnosis data is determined based on the determined operating abnormality category. This method can effectively analyze and predict abnormal situations that may occur during the operation of the data center, providing strong support for the management and maintenance of the data center.

According to one aspect of the embodiment of the present invention, a data center anomaly analysis method and system are provided. The method includes:

Obtain target data center operation data;

The target data center operation data is loaded into the a priori learned anomaly analysis and prediction network to generate an operation anomaly category determined by the anomaly analysis and prediction network. The anomaly analysis and prediction network is based on the associated data center operation data and static scheduling data. Generated through knowledge learning;

Abnormality diagnosis data is determined based on the operation abnormality category.

In an alternative implementation, the anomaly analysis and prediction network includes a self-attention unit and a fully connected output unit, and the target data center operation data is loaded into the a priori learned anomaly analysis and prediction network to generate the The abnormal operation categories determined by the above-mentioned anomaly analysis and prediction network include:

Load the target data center operation data into the self-attention unit, and generate the target self-attention features determined by the self-attention unit;

The target self-attention feature is loaded into the fully connected output unit to generate an abnormal operating category determined by the fully connected output unit.

In an alternative implementation, the training step of the anomaly analysis and prediction network includes:

Generate positive learning features and negative learning features based on the linked operation scheduling event data in the data center log data, and generate template learning data based on the positive learning features and the negative learning features;

Based on the template learning data, the parameters of the initialized anomaly analysis and prediction network are updated to generate an anomaly analysis and prediction network with updated parameters.

In an alternative implementation, the positive learning features and negative learning features are generated based on the linked operation scheduling event data in the data center log data, including:

Obtain the linked running scheduling event data in the data center log data as basic training template data;

Perform regular conversion on the basic training template data to generate the active learning features;

The data center operation data and noise features in the data center log data are randomly scrambled to generate the passive learning features.

In an alternative implementation, the self-attention unit includes a first self-attention unit and a second self-attention unit, and the parameters of the initialized anomaly analysis and prediction network are updated based on the template learning data, Generate anomaly analysis and prediction network with updated parameters, including:

For the running scheduling event data pair in the template learning data, load the static scheduling event of the running scheduling event data pair into the first self-attention unit, and generate the target determined by the first self-attention unit Static self-attention features: load the dynamic scheduling events of the running scheduling event data pair into the second self-attention unit, and generate the target dynamic self-attention features determined by the second self-attention unit;

Determine target training error parameters based on the target static self-attention features and the target dynamic self-attention features, with the goal of minimizing the target training error parameters, perform parameter updates on the self-attention units, and generate parameter updates The subsequent self-attention unit;

The parameters of the fully connected output unit are updated according to the self-attention unit after the parameter update, and a fully connected output unit after the parameter update is generated.

In an alternative implementation, the running scheduling event data pair includes multiple dynamic scheduling events, and the dynamic scheduling events of the running scheduling event data pair are loaded into the second self-attention unit, Generating target dynamic self-attention features determined by the second self-attention unit includes:

Integrate each of the dynamic scheduling events to generate integrated data center operation data;

The integrated data center operation data is loaded into the second self-attention unit, and the target dynamic self-attention feature determined by the second self-attention unit is generated.

In an alternative implementation, the running scheduling event data pair includes multiple dynamic scheduling events, and the dynamic scheduling events of the running scheduling event data pair are loaded into the second self-attention unit, Generating dynamic self-attention features determined by the second self-attention unit includes:

Load each of the dynamic scheduling events into the second self-attention unit respectively, and generate dynamic self-attention features of each of the dynamic scheduling events determined by the second self-attention unit;

The dynamic self-attention features of each of the dynamic scheduling events are summed to generate the target dynamic self-attention feature.

In an alternative implementation, the first self-attention unit and the second self-attention unit are respectively connected to the fully connected output unit, and the self-attention unit updated according to the parameters is The fully connected output unit is updated with parameters and a fully connected output unit with updated parameters is generated, including:

For the static scheduling data in the template learning data, determine the static self-attention characteristics of the static scheduling data according to the first self-attention unit;

Constructing target anomaly learning template data based on the static self-attention feature and the anomaly category of the static scheduling data;

According to the target anomaly learning template data, the parameters of the initialized fully connected output unit are updated to generate a fully connected output unit with updated parameters.

According to another aspect of the embodiment of the present invention, a data center anomaly analysis method and system are provided. The system includes:

The acquisition module is used to obtain the operation data of the target data center;

A generation module configured to load the target data center operation data into the a priori learned anomaly analysis and prediction network, and generate an operation anomaly category determined by the anomaly analysis and prediction network, and the anomaly analysis and prediction network operates according to the associated data center Data and static scheduling data are generated through knowledge learning;

A determining module, configured to determine abnormal diagnosis data according to the abnormal operation category.

According to another aspect of the embodiment of the present invention, a server is provided, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus; The memory is used to store computer programs; the processor is used to implement any of the above data center anomaly analysis method steps when executing the computer program.

According to another aspect of an embodiment of the present invention, a readable storage medium is provided. A computer program is stored on the readable storage medium. When the computer program is run by a processor, the computer program can perform the steps of the above-mentioned data center anomaly analysis method.

In order to make the above objects, features and advantages of the embodiments of the present invention more obvious and easy to understand, a detailed description will be given below with reference to the embodiments and the accompanying drawings.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present invention more clearly, the drawings required to be used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and therefore do not It should be regarded as a limitation of the scope. For those of ordinary skill in the art, other relevant drawings can also be obtained based on these drawings without exerting creative efforts.

Figure 1 shows a schematic component diagram of a server provided by an embodiment of the present invention;

Figure 2 shows a schematic flow chart of the data center anomaly analysis method provided by an embodiment of the present invention;

Figure 3 shows a functional module block diagram of a data center anomaly analysis system provided by an embodiment of the present invention.

Detailed ways

In order to enable students in the technical field to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only These are some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

The terms "first", "second", "third", etc. (if present) in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific sequence. Or sequence. It is to be understood that the figures so used are interchangeable under appropriate circumstances so that the embodiments of the invention described herein, for example, can be practiced in sequences other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus that encompasses a series of steps or units and need not be limited to those explicitly listed. Those steps or elements may instead include other steps or elements not expressly listed or inherent to the process, method, product or apparatus.

FIG. 1 shows an exemplary component diagram of server 100. Server 100 may include one or more processors 104, such as one or more central processing units (CPUs), each of which may implement one or more hardware threads. Server 100 may also include any storage media 106 for storing any kind of information such as code, settings, data, and the like. For example, without limitation, the storage medium 106 may include any one or more combinations of the following: any type of RAM, any type of ROM, flash memory device, hard disk, optical disk, etc. More generally, any storage medium can use any technology to store information. Further, any storage medium can provide volatile or non-volatile retention of information. Further, any storage media may represent fixed or removable components of server 100. In one instance, when processor 104 executes the associated instructions stored in any storage medium or combination of storage media, server 100 may perform any operation of the associated instructions. The server 100 also includes one or more drive units 108 for interacting with any storage media, such as a hard disk drive unit, an optical disk drive unit, and the like.

Server 100 also includes input/output 110 (I/O) for receiving various inputs (via input unit 112) and for providing various outputs (via output unit 114). One particular output mechanism may include a presentation device 116 and an associated graphical user interface (GUI) 118 . Server 100 may also include one or more network interfaces 120 for exchanging data with other devices via one or more communication units 122 . One or more communication buses 124 couple together the components described above.

The communication unit 122 may be implemented in any manner, for example, through a local area network, a wide area network (eg, the Internet), a point-to-point connection, etc., or any combination thereof. Communications unit 122 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers 100, etc. governed by any protocol or combination of protocols.

Figure 2 shows a schematic flow chart of the data center anomaly analysis method and system provided by an embodiment of the present invention. The data center anomaly analysis method and system can be executed by the server 100 shown in Figure 1. The detailed steps of the data center anomaly analysis method The introduction is as follows.

Step S110, obtain target data center operation data;

Step S120: Load the target data center operation data into the a priori learned anomaly analysis and prediction network, and generate an operation anomaly category determined by the anomaly analysis and prediction network. The anomaly analysis and prediction network is based on the associated data center operation data and Static scheduling data is generated through knowledge learning;

Step S130: Determine abnormal diagnosis data according to the abnormal operation category.

Based on the above steps, this embodiment provides a method for analyzing and predicting abnormal operation of a data center. First, obtain the operating data of the target data center. Then, the operation data of the target data center is loaded into an anomaly analysis and prediction network generated through prior learning. This anomaly analysis and prediction network is generated based on knowledge learning based on the associated data center operation data and static scheduling data. Through this abnormality analysis and prediction network, corresponding operating abnormality categories can be generated. Finally, abnormal diagnosis data is determined based on the determined operating abnormality category. This method can effectively analyze and predict abnormal situations that may occur during the operation of the data center, providing strong support for the management and maintenance of the data center.

In an alternative implementation, the anomaly analysis and prediction network includes a self-attention unit and a fully connected output unit, and the target data center operation data is loaded into the a priori learned anomaly analysis and prediction network to generate the The abnormal operation categories determined by the above-mentioned anomaly analysis and prediction network include:

Load the target data center operation data into the self-attention unit, and generate the target self-attention features determined by the self-attention unit;

The target self-attention feature is loaded into the fully connected output unit to generate an abnormal operating category determined by the fully connected output unit.

In an alternative implementation, the training step of the anomaly analysis and prediction network includes:

Generate positive learning features and negative learning features based on the linked operation scheduling event data in the data center log data, and generate template learning data based on the positive learning features and the negative learning features;

Based on the template learning data, the parameters of the initialized anomaly analysis and prediction network are updated to generate an anomaly analysis and prediction network with updated parameters.

In an alternative implementation, the positive learning features and negative learning features are generated based on the linked operation scheduling event data in the data center log data, including:

Obtain the linked running scheduling event data in the data center log data as basic training template data;

Perform regular conversion on the basic training template data to generate the active learning features;

The data center operation data and noise features in the data center log data are randomly scrambled to generate the passive learning features.

In an alternative implementation, the self-attention unit includes a first self-attention unit and a second self-attention unit, and the parameters of the initialized anomaly analysis and prediction network are updated based on the template learning data, Generate anomaly analysis and prediction network with updated parameters, including:

For the running scheduling event data pair in the template learning data, load the static scheduling event of the running scheduling event data pair into the first self-attention unit, and generate the target determined by the first self-attention unit Static self-attention features: load the dynamic scheduling events of the running scheduling event data pair into the second self-attention unit, and generate the target dynamic self-attention features determined by the second self-attention unit;

Determine target training error parameters based on the target static self-attention features and the target dynamic self-attention features, with the goal of minimizing the target training error parameters, perform parameter updates on the self-attention units, and generate parameter updates The subsequent self-attention unit;

The parameters of the fully connected output unit are updated according to the self-attention unit after the parameter update, and a fully connected output unit after the parameter update is generated.

In an alternative implementation, the running scheduling event data pair includes multiple dynamic scheduling events, and the dynamic scheduling events of the running scheduling event data pair are loaded into the second self-attention unit, Generating target dynamic self-attention features determined by the second self-attention unit includes:

Integrate each of the dynamic scheduling events to generate integrated data center operation data;

The integrated data center operation data is loaded into the second self-attention unit, and the target dynamic self-attention feature determined by the second self-attention unit is generated.

In an alternative implementation, the running scheduling event data pair includes multiple dynamic scheduling events, and the dynamic scheduling events of the running scheduling event data pair are loaded into the second self-attention unit, Generating dynamic self-attention features determined by the second self-attention unit includes:

Load each of the dynamic scheduling events into the second self-attention unit respectively, and generate dynamic self-attention features of each of the dynamic scheduling events determined by the second self-attention unit;

The dynamic self-attention features of each of the dynamic scheduling events are summed to generate the target dynamic self-attention feature.

In an alternative implementation, the first self-attention unit and the second self-attention unit are respectively connected to the fully connected output unit, and the self-attention unit updated according to the parameters is The fully connected output unit is updated with parameters and a fully connected output unit with updated parameters is generated, including:

For the static scheduling data in the template learning data, determine the static self-attention characteristics of the static scheduling data according to the first self-attention unit;

Constructing target anomaly learning template data based on the static self-attention feature and the anomaly category of the static scheduling data;

According to the target anomaly learning template data, the parameters of the initialized fully connected output unit are updated to generate a fully connected output unit with updated parameters.

FIG. 3 shows a functional module diagram of the data center anomaly analysis system 200 provided by an embodiment of the present invention. The functions implemented by the data center anomaly analysis system 200 may correspond to the steps performed by the above method. The data center anomaly analysis system 200 can be understood as the above-mentioned server 100, or the processor of the server 100, or can be understood as a component independent of the above-mentioned server 100 or processor that implements the functions of the present invention under the control of the server 100, such as As shown in FIG. 3 , the functions of each functional module of the data center anomaly analysis system 200 will be described in detail below.

The acquisition module 210 is used to acquire target data center operation data;

The generation module 220 is used to load the operation data of the target data center into the a priori learned anomaly analysis and prediction network, and generate the operation anomaly category determined by the anomaly analysis and prediction network. The anomaly analysis and prediction network is based on the associated data center. Operation data and static dispatch data are generated through knowledge learning;

The determination module 230 is used to determine abnormal diagnosis data according to the abnormal operation category.

Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described systems, devices and units can be referred to the corresponding processes in the foregoing method embodiments, and will not be described again here.

It is obvious to those skilled in the art that the present invention is not limited to the details of the above-described exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be regarded as illustrative and non-restrictive from any point of view, and the scope of the present invention is defined by the appended claims rather than the above description, and it is therefore intended that all claims falling within the claims All changes within the meaning and scope of equivalent elements are included in the present invention.

Claims (10)

1. A data center anomaly analysis method, characterized in that the method includes: Obtain target data center operation data; The target data center operation data is loaded into the a priori learned anomaly analysis and prediction network to generate an operation anomaly category determined by the anomaly analysis and prediction network. The anomaly analysis and prediction network is based on the associated data center operation data and static scheduling data. Generated through knowledge learning; Abnormality diagnosis data is determined based on the operation abnormality category. 2. The data center anomaly analysis method according to claim 1, wherein the anomaly analysis and prediction network includes a self-attention unit and a fully connected output unit, and the target data center operation data is loaded into a priori In the learned abnormality analysis and prediction network, the operating abnormality categories determined by the abnormality analysis and prediction network are generated, including: Load the target data center operation data into the self-attention unit, and generate the target self-attention features determined by the self-attention unit; The target self-attention feature is loaded into the fully connected output unit to generate an abnormal operating category determined by the fully connected output unit. 3. The data center anomaly analysis method according to claim 2, characterized in that the training step of the anomaly analysis and prediction network includes: Generate positive learning features and negative learning features based on the linked operation scheduling event data in the data center log data, and generate template learning data based on the positive learning features and the negative learning features; Based on the template learning data, the parameters of the initialized anomaly analysis and prediction network are updated to generate an anomaly analysis and prediction network with updated parameters. 4. The data center anomaly analysis method according to claim 3, characterized in that the positive learning features and negative learning features are generated based on the linked operation scheduling event data in the data center log data, including: Obtain the linked running scheduling event data in the data center log data as basic training template data; Perform regular conversion on the basic training template data to generate the active learning features; The data center operation data and noise features in the data center log data are randomly scrambled to generate the passive learning features. 5. The data center anomaly analysis method according to claim 3, wherein the self-attention unit includes a first self-attention unit and a second self-attention unit, and the learning data based on the template is The initialized anomaly analysis and prediction network is updated with parameters, and an anomaly analysis and prediction network with updated parameters is generated, including: For the running scheduling event data pair in the template learning data, load the static scheduling event of the running scheduling event data pair into the first self-attention unit, and generate the target determined by the first self-attention unit Static self-attention features: load the dynamic scheduling events of the running scheduling event data pair into the second self-attention unit, and generate the target dynamic self-attention features determined by the second self-attention unit; Determine target training error parameters based on the target static self-attention features and the target dynamic self-attention features, with the goal of minimizing the target training error parameters, perform parameter updates on the self-attention units, and generate parameter updates The subsequent self-attention unit; The parameters of the fully connected output unit are updated according to the self-attention unit after the parameter update, and a fully connected output unit after the parameter update is generated. 6. The data center anomaly analysis method according to claim 5, wherein the operation scheduling event data pair contains a plurality of dynamic scheduling events, and the dynamic scheduling events of the operation scheduling event data pair are loaded into In the second self-attention unit, generating the target dynamic self-attention features determined by the second self-attention unit includes: Integrate each of the dynamic scheduling events to generate integrated data center operation data; The integrated data center operation data is loaded into the second self-attention unit, and the target dynamic self-attention feature determined by the second self-attention unit is generated. 7. The data center anomaly analysis method according to claim 5, wherein the operation scheduling event data pair contains a plurality of dynamic scheduling events, and the dynamic scheduling events of the operation scheduling event data pair are loaded into In the second self-attention unit, generating dynamic self-attention features determined by the second self-attention unit includes: Load each of the dynamic scheduling events into the second self-attention unit respectively, and generate dynamic self-attention features of each of the dynamic scheduling events determined by the second self-attention unit; The dynamic self-attention features of each of the dynamic scheduling events are summed to generate the target dynamic self-attention feature. 8. The data center anomaly analysis method according to claim 5, wherein the first self-attention unit and the second self-attention unit are respectively connected to the fully connected output unit, and the basis is The self-attention unit with updated parameters updates the parameters of the fully connected output unit to generate a fully connected output unit with updated parameters, including: For the static scheduling data in the template learning data, determine the static self-attention characteristics of the static scheduling data according to the first self-attention unit; Constructing target anomaly learning template data based on the static self-attention feature and the anomaly category of the static scheduling data; According to the target anomaly learning template data, the parameters of the initialized fully connected output unit are updated to generate a fully connected output unit with updated parameters. 9. A data center anomaly analysis system, characterized by including: The acquisition module is used to obtain the operation data of the target data center; A generation module configured to load the target data center operation data into the a priori learned anomaly analysis and prediction network, and generate an operation anomaly category determined by the anomaly analysis and prediction network, and the anomaly analysis and prediction network operates according to the associated data center Data and static scheduling data are generated through knowledge learning; A determining module, configured to determine abnormal diagnosis data according to the abnormal operation category. 10. A server, characterized in that it includes: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus; the memory is used to store Computer program; the processor is configured to implement the steps of the data center anomaly analysis method described in any one of claims 1-8 when executing the computer program.