TWI725543B - A method for predicting system anomalies caused by rare events - Google Patents

Abstract

The present invention is a method for predicting system anomalies caused by rare events, and used for predicting whether a system anomalies is about to occur in the future by integrating machine learning with various data collected from the system. A better representation for the original monitoring data can get by using data pre-processing and data feature selection and processing proposed in the present invention. Without changing the existing system, the data processing technology can further improve the accuracy of a prediction model and reduce a training model time. In addition, the integration of machine learning technology and the complete training mechanism for model can greatly reduce the human participation and the whole mechanism can be automated, and can to find out the cause that is most likely to cause the system anomalies, so as to quickly and effectively achieve the goal of preventing the abnormality in advance.

Description

Methods of predicting rare events that cause system anomalies

The present invention relates to a technology for predicting system abnormalities, in particular, it relates to a method for predicting rare events that cause system abnormalities.

For general operating systems, the probability of system abnormality is very low, but once a system abnormality occurs, it usually takes a long time to eliminate it. Simply put, by observing the long-term collected system monitoring data and analyzing the abnormal characteristics of the system, you will find this Such anomalies will last for a long time but almost never happen. In other words, the difficulty of such prediction lies in the highly-imbalanced data, that is, the amount of system anomalies is extremely rare. Compared with the situation without anomalies, the system management From a very small amount of data, it is difficult to predict system abnormalities. In addition to data collection and integration, data feature extraction and data prediction, it will become an important issue for system administrators.

In summary, if a technology can be found to quickly and effectively predict system abnormalities, especially for extremely rare system abnormalities, the entire prediction mechanism can be fully automated without changing the existing system and reducing human involvement. Become the goal that people in the technical field are eager to pursue solutions.

The purpose of the present invention is to propose a method for predicting system abnormalities. The main technology includes three aspects, namely data pre-processing, data feature selection and processing, and machine learning, which are generated by integrating, digitizing and regularizing system monitoring data. A model that can be used to predict system abnormalities, so as to successfully predict system abnormalities.

In order to achieve the above and other purposes, the present invention proposes a method for predicting rare events that cause system anomalies, which includes: collecting original monitoring data from each server, and then integrating the original monitoring data with time as the key to integrate The original monitoring data is integrated into a data; the missing parts in the original monitoring data are filled; the text part of the original monitoring data is digitized so that the original monitoring data becomes numerical data; all of the original monitoring data The value is normalized; based on the continuous data in the original monitoring data within a period of time, the original monitoring data is grouped and discretized to count the total number of continuous data in the period of time, as a representation of the original monitoring And then train the parameters representing the original monitoring data to establish an anomaly classification prediction model; and use the model characteristics of the anomaly classification prediction model to obtain at least one parameter that caused the anomaly, so that the at least one parameter and the correlation High-performance parameters predict the possible causes of system abnormalities.

In the aforementioned method, the step of filling the missing part in the original monitoring data includes filling the missing part with a specific value, and filling in an average value or a specific value for continuous values, and the text in the original monitoring data Part of the digitization step is to replace the missing part with the specific value with a serial number.

In the aforementioned method, the step of digitizing the text part of the original monitoring data is to sequentially arrange the content of different types of text data in a numerical sequence starting from 0.

In the foregoing method, the step of grouping the original monitoring data is to use a k-means grouping algorithm to group the original monitoring data.

In the foregoing method, the step of training the parameters representing the original monitoring data includes using a random forest algorithm to set the hyperparameters of the random forest algorithm to a range, and then using a grid search method to set the parameters within the range Search for parameters at certain intervals.

In the aforementioned method, the training represents the parameters of the original monitoring data, and the step of establishing the anomaly classification prediction model includes using an under-sampling method to reduce the amount of the original monitoring data.

In the aforementioned method, the step of training the parameters representing the original monitoring data includes using a random forest algorithm to determine only the first time point as abnormal for the continuous abnormal data, and the remaining time points are regarded as normal.

In the foregoing method, the step of using the model characteristics of the anomaly classification prediction model to obtain at least one parameter that caused the anomaly refers to using the Information Gain algorithm to perform feature selection, so as to select the one that has a greater impact on the abnormality of the system. The parameter serves as the at least one parameter.

In the foregoing method, the step of using the model characteristics of the abnormal classification prediction model to obtain at least one parameter that caused the abnormality includes using a linear gate valve unit to control the degree of influence of the previous prediction data on the result, and using a residual network The result is fed back to the current result to be predicted.

In summary, the present invention combines various data collected in the system with machine learning to predict whether the system will be abnormal in the future. Through the data pre-processing and data feature selection and processing described in the present invention, the original data can be better Representation method, and does not have the system environment Special requirements and no need to modify the existing system. The data processing technology can further improve the accuracy of the anomaly classification prediction model and reduce the training model time. By fusing machine learning technology and a complete model training mechanism, it can greatly reduce human effort. The entire mechanism can be fully automated to find out the most likely cause of system abnormalities to achieve the goal of preventing abnormalities in advance. Therefore, the method of predicting rare events that cause system abnormalities in the present invention can improve the existing mechanism , Especially in the case of extremely rare system abnormalities, it can still successfully predict most of the errors quickly and effectively.

S11~S17‧‧‧Step

S30~S39‧‧‧Process

Figure 1 is a step diagram of the method for predicting rare events that cause system anomalies according to the present invention.

Figure 2 illustrates a schematic diagram of the classification problem considered in the data pre-processing of the present invention.

Figure 3 is a flow chart of the present invention for predicting rare events that cause system abnormalities.

Figure 4 illustrates a schematic diagram of the present invention for judging whether the model needs to be updated.

Figure 5 illustrates a schematic diagram of data selection and merging of the present invention.

Figure 6 illustrates a schematic diagram of the lost data processing of the present invention.

Figure 7 illustrates a schematic diagram of the CPU usage rate of the system in normal and about to be abnormal in an embodiment of the present invention.

Figure 8 illustrates a schematic diagram of the discretization of continuous data according to the present invention.

Figures 9A and 9B illustrate a schematic diagram of the forecast target item data processing of the present invention.

Figure 10 illustrates the use of linear gate valve units to build a residual network of the present invention intention.

The following describes the technical content of the present invention with specific specific embodiments. Those familiar with the art can easily understand the advantages and effects of the present invention from the content disclosed in this specification. However, the present invention can also be implemented or applied by other different specific embodiments.

Figure 1 is a step diagram of the method for predicting rare events that cause system anomalies according to the present invention. In step S11, collect original monitoring data from each server, and then integrate the original monitoring data with time as the main key, so as to integrate the original monitoring data into a piece of data. In this step, you can set a data aging so that the prediction model only uses the data within the time effect for model training. For example, use the state table to record the prediction results within one hour and set the threshold for retraining the model. Quickly determine whether the current prediction model can still predict correctly, so the timestamp chapters in all the data are aligned, and then the original monitoring data collected from each monitoring unit from each server is combined with the timestamp chapter as the reference key value , To integrate the original monitoring data into the same data.

In step S12, the missing part in the original monitoring data is filled. In detail, the original monitoring data sometimes has missing data due to unexpected reasons, so this step is to fill in the missing parts of the original monitoring data.

In one embodiment, filling the missing part in the original monitoring data includes filling the missing part with a specific value (such as -1), and filling in an average value or a specific value (such as -1) for continuous values. In short, the missing part can be filled with a specific value (such as -1), or for continuous values, an average value or a specific value (such as -1) can be filled in, instead of filling in the original value of 0. Because it is mainly to be separated from the actual value of 0.

In step S13, the text part of the original monitoring data is digitized, so that all the original monitoring data become numerical data. This step is to explain that the establishment of the prediction model must be numerical data. Assuming that the original monitoring data is originally a value, it does not need to be processed and can be used directly. If the original monitoring data is text data, it must be expressed in numerical values, so it is necessary to The text part of the original monitoring data is digitized.

In one embodiment, the text part of the original monitoring data is digitized by sequentially arranging the content of different types of text data in a numerical sequence starting from 0. In other words, the original monitoring data is based on different types of text. The data content is digitized, that is, the text data can be arranged in a numerical sequence of numbers 0, 1, 2,... to become discrete numerical data.

In addition, if in the foregoing step S12, a specific value (such as -1) is used to fill the missing part, in this step, the original number (such as -1) to fill the missing part is replaced by a serial number.

In step S14, all values in the original monitoring data are normalized. Specifically, since each monitoring item has different numerical value characteristics, if these data are directly used for training of anomaly classification prediction model without processing, the influence of items with larger numerical characteristics on the anomaly classification prediction model It will be larger, but the importance of each item in the expectation should be the same, which will cause the abnormal classification prediction model to fail to achieve the expected effect, so this step is to normalize all the values.

Regarding numerical normalization, for example, each data can be subtracted from its average and divided by the standard deviation, so that the value is adjusted to the interval of 0~1. In addition, for discrete data recorded with serial numbers, you can choose to use one-hot encoding (one -hot encoding), for example, the original data has three values 0, 1, and 2, It will use three-dimensional vectors, namely (1,0,0), (0,1,0), (0,0,1) to record.

In step S15, based on the continuous data in the original monitoring data for a period of time, the original monitoring data is grouped and discretized, so as to count the total number of continuous data in the period of time, so as to represent the original monitoring data. The parameters of the data. In this step, the original monitoring data is grouped into groups using the k-means grouping algorithm, which includes counting the total number of various data after classification, and using this total as the representation method of the various data.

In step S16, the parameters representing the original monitoring data are trained to establish an abnormal classification prediction model. In this step, the training data should be balanced when the anomaly classification prediction model is established. For example, an under-sampling method is used to reduce the amount of data for a larger number of events, and then an anomaly classification prediction model is established.

In one embodiment, the random forest algorithm can be used to train the parameters of the original monitoring data, that is, when the anomaly classification prediction model is established, a range of hyperparameters of the random forest algorithm is set and the grid is performed. Search, perform a parameter search within the range at a certain interval, and then use the verification data to filter out a better model, and the final prediction will use the aforementioned better models to predict together, and add through the majority decision method Pass the threshold to get the final result.

In step S17, the model characteristics of the abnormal classification prediction model are used to obtain at least one important parameter that causes the abnormality, and the possible cause of the system abnormality is predicted from the important at least one parameter and the parameter with high correlation. Specifically, the use of the model characteristics of the abnormal classification prediction model to obtain at least one important parameter that causes the abnormality refers to the use of Information Gain (InfoGain) algorithm to perform feature selection, so as to determine the one that has a greater impact on the abnormality of the system. The parameter as the important at least one parameter, that is to use the InfoGain calculus The method performs feature interpretation on the trained anomaly classification prediction model to find out which parameters in the anomaly classification prediction model have a greater impact on the cause of system anomalies.

In one embodiment, the aforementioned random forest algorithm is used to train the parameters representing the original monitoring data, including that for continuous abnormal data, only the first time point is judged to be abnormal, and the remaining time points are regarded as normal, that is, In other words, because the system abnormality is not meaningful to predict the abnormality after it occurs, and because the system abnormality usually lasts for a period of time after the occurrence of the system abnormality, when testing, the design of the present invention is to predict the time point before the occurrence of the continuous abnormality. For continuous abnormalities, only the first time point is judged to be abnormal, and the remaining time points are regarded as normal.

In addition, in step S17, it includes using the linear gate valve unit to control the degree of influence of the previous prediction data on the result, and using the residual network to feed back the result to the current result to be predicted. In short, due to the time series data Before and after the data are related, considering the results before the current time point is helpful for the current prediction, so the present invention proposes to use the linear gate valve unit to control the degree of influence of the previous prediction data on the result, and use the residual network to make the result Feedback to the current result to be predicted.

In addition, the method of predicting rare events that cause system abnormalities in the present invention includes retraining the abnormal classification prediction model. Specifically, after the monitoring data is continuously collected for a period of time, there may be many programs under the environment and the system. Therefore, the old abnormal classification prediction model may no longer be able to predict the system abnormality of the new environment. Therefore, it is possible to return to step S11 to retrain the new prediction model.

Specifically, the technology used in the present invention can be divided into three technical aspects, including data pre-processing, data feature selection and processing, and machine learning. First of all, the difficulty in predicting abnormalities in the extremely rare system of the present invention lies in the highly-imbalanced data, that is, the system The amount of system abnormalities is extremely rare. Therefore, the three technical aspects of the present invention include data pre-processing, data feature selection and processing, and machine learning. The following first describes the three technical aspects to consider the technical issues and the selected processing methods.

The first aspect is data pre-processing, which corresponds to steps S11-S14 in Figure 1 of this case. The first aspect must consider the following points: (1) The problem to be predicted is a classification problem, which means that many systems can be Anomalies are regarded as the same kind, and the ultimate goal is to predict system anomalies, so they can be regarded as a binary classification problem or a multi-class classification problem. Experiments show that treating a multi-class classification problem can more effectively catch various errors; (2) similar to the above Concept, the anomaly detection problem can be regarded as a classification problem of supervised learning or unsupervised learning. The experimental results show that the method of supervised learning will have better results than unsupervised learning. Therefore, this problem can be regarded as a supervised multivariate classification. For example, as shown in Figure 2; (3) Because the system abnormality often lasts for a period of time, when it is to be predicted, if an existing system abnormality has occurred, the prediction is of little significance. Therefore, the present invention only predicts The beginning of the continuous system anomaly; (4) There are often some missing parts in the data. Generally, the missing data will be filled in with the data at the previous time point or the data at the previous and after time points will be used for linear internal difference, but the above two processing methods After experimentation, they are all inappropriate. The reason is that the loss of these data is caused by the downtime of some other servers, so the original name will fill these vacancies in a new state.

It can be seen from the above that the main goal of data pre-processing technology is to process the original monitoring data, so that the original monitoring data and the predicted target can have a better representation method. In short, different from the data presented on the general report, the machine learning model ( For example, the abnormal classification prediction model of the present invention has different requirements for the original data. The most common one is to standardize the value. In this way, the data from different monitoring items will not cause the machine learning model to be affected by the original value distribution of the data. The impact of varying degrees violates the assumption that every project is as important as the predicted goal.

The second aspect is the selection and processing of data characteristics, which corresponds to step S15 in Figure 1 of this case. The second aspect must consider the following points: (1) The characteristics of the data are often very general data, such as the maximum number of connections allowed And the connection in use, these data characteristics can be combined into the utilization rate of the connection (that is, the utilization rate = the connection in use / the maximum number of connections allowed); (2) For the value of a data feature, These values can be grouped into groups to generate a new data feature. For example, the level of CPU usage can be divided into three levels: low, medium, and high. This procedure is discretization; (3) In addition, further discretization is required. Data is used for statistics. For example, the CPU usage level in 10 time periods is low, medium, and high. If the high rate exceeds a certain threshold, it means that errors are likely to occur; (4) In addition, the characteristics of important data Selection is also an indispensable part. First, similar data characteristics must be combined to find out the true dimensions of the data. For example, through the statistical analysis of principal component analysis (PCA) and the method of simplifying the data set, it is necessary to further Make the selection of data characteristics.

It can be seen from the above that the main goal of data feature selection and processing technology is to understand the distribution and characteristics of the original monitoring data through observation of the original monitoring data, so as to design other more simplified description methods, compared to directly using the original data to train the machine Learning the prediction model. The present invention can reduce the items to be learned by the prediction model through a simplified description method, because the more important part has been emphasized by a simplified description method, so that the prediction model can learn more reasons that may affect the prediction label, and improve Finally, predict the accuracy of the prediction model. In addition, the processing of predictive tags is also very important. In the embodiment, sometimes the system may have a short-term abnormality. The reason may come from different reasons such as the reset of the system rules, the launch of a new version of the program, or the restart of the machine. However, these conditions should not be classified as system abnormalities. Therefore, the prediction label can choose to adopt the situation of continuous abnormal events, so that the system conditions that should be normal will not be predicted, which affects the prediction model. The accuracy of the model.

The third aspect is machine learning, which corresponds to steps S16-S17 in Figure 1 of this case. To catch errors in highly-imbalanced data, it is usually difficult to achieve by only relying on a machine learning model. The present invention selects random forest (random forest) algorithm, and searches through grid search (grid search). ) Method to brush the parameters, find the best parameters, and use time-series cross validation to verify the quality of the model in consideration of chronological order. In addition, because the previous prediction is highly correlated with the current prediction, we have introduced residual network technology, and through a simple linear gate valve unit, the previous prediction results are incorporated into the current prediction analysis to make predictions, and then, Then set a threshold for the results of the model to filter out inappropriate predictions so that most errors can be detected under an acceptable false alarm rate. Finally, use the results of the model and the Information Gain algorithm And Gini Impurity (Gini Impurity) and other algorithms to get the most likely cause of abnormality (data characteristics), for reference to prevent the occurrence of system abnormalities.

It can be seen from the above that the main goal of machine learning technology is to select suitable algorithms and suitable hyperparameters. The present invention selects the random forest algorithm, and for the hyperparameter selection of the random forest algorithm, the present invention uses a grid search method to continuously verify through verification data, and finally selects a better hyperparameter combination, and then, Use the combination prediction method to predict the prediction model of the better hyperparameter combination at the same time, and use the majority decision method plus the pass threshold to get the final result. We also added a residual network to the outer layer of the above model, so that the model can apply the previous prediction results to this analysis. Furthermore, in addition to system abnormality prediction, the present invention also uses the InfoGain algorithm to perform feature interpretation on the trained prediction model, and find out which parameters in the prediction model have a greater impact on the cause of system abnormality.

In addition, because the system environment or its subordinate programs are subject to change, after a long period of time, the old prediction model may no longer be able to predict the abnormal state of the system in the new environment. Therefore, in other embodiments, For a period of time, such as every six months, use the newly collected monitoring data to retrain a new prediction model.

In summary, the technical aspects of data pre-processing include data selection and consolidation, lost data processing, data digitization processing, and data normalization processing. The technical aspects of data feature selection and processing include continuous data discretization and prediction goals. Project data processing and other procedures, and the technical aspects of machine learning include procedures such as the use of algorithms to build models, the determination of abnormal items, and the retraining of prediction models.

Integrating the foregoing procedures, you can get the flowchart described in Figure 3, where the data pre-processing is oriented to include processes S30-S34, the data feature selection and processing is oriented to include processes S35-S36, and the machine learning is oriented to include processes S37-S39. To be more specific, the entire flowchart includes process S30 for data selection and merging, process S31 for lost data processing, process S32 for data digitization processing, process S33 for data normalization processing, and process S34 for continuous data discretization. The process S35 is the data processing of the forecast target item, the process S36 is the method of using the random forest algorithm to establish a model, the process S37 is the use of a linear gate valve unit to establish a residual network, the process S38 is the determination of an abnormal item, and the process S39 is the renewal of the prediction model. training. The follow-up will be further explained with specific implementation examples.

The present invention collects data from the system to train the machine learning model, but data that is too long ago is not suitable for training, because system problems will be slowly repaired, and errors that are too long ago are likely to no longer occur, so the data is time-effective , And the trained model is also time-sensitive, so it is necessary to keep retraining with new data. Generally speaking, without major system changes In the case of, the time limit for the collected data is about half a year. Assuming that there is a significant system change, it is likely that the type of system abnormality will be completely different, so it may need to collect data and train again to maintain accuracy.

The process S30 is data selection and integration. The present invention designs a simple judgment model to judge whether the model needs to be updated, as shown in Figure 4 and refer to the following table 1. A small table can be used to record the forecast within one hour, and each item represents the time divided by 5. For example, the predicted result of 18 minutes belongs to Project 3. Since the model is to predict the result after 10 points, after 10 points, you can interpret whether the item is correct. Fill in T if it is correct, or fill in F. When filling in the form for the first time, all the fields are empty values ψ.

In this example, the prediction model is started at 0~4 minutes, so the value is filled from field 0. The logic of filling the form is described through the simple state transition diagram in Figure 4. The state of each field is independent. But the S and V values are shared. Each time the table is filled in, the sum (S) and non-empty value (V) in the table will also be updated. When V=12, it means that there are no empty values in the table. At this time, we can calculate the accuracy of the model S /V, when the accuracy is lower than the set threshold (for example, 50% in this embodiment), it is necessary to consider whether to retrain the model. Since this judgment will always be called, this judgment method minimizes the extra system burden.

Next, set a data time limit. In this example, it is half a year. The prediction model only uses the data in the time effect for model training. Then, the time stamps in all the data are aligned Qi, the alignment unit is 5 points, that is, downward alignment. For example, 0 minutes 0 seconds to 4 minutes 59 seconds will be classified into the time stamps of 0 minutes 0 seconds. Finally, the monitoring collected from each monitoring unit The data is integrated into the same data with the time stamp chapter as the combined reference key value. As shown in Figure 5, the data includes the web server, the database server, and the data at the bottom of the system (among which, Ok means no problem, Failure means failure), integrate all data with time as the main key.

The process S31 is the processing of lost data. There are often many missing data in the data. There are several ways to deal with missing data. You can directly complement the value 0. If it is a continuous value (such as the memory usage rate), you can also add the average value, or It is possible to add a value that has never appeared before, such as the value -1. Considering that the system sometimes loses data because the server is down, you should add a value that has never appeared before to indicate this abnormal state, as in Section 6. As shown in the figure.

Specifically, sometimes it may be that the system is too busy, making the system unable to respond to the detection of the probe program. At this time, it is more suitable to use the method of interpolation and average value if the data is lost. Therefore, for the supplementary average value or the value that has not appeared before, the present invention uses the sampling method to supplement the value, that is, analyzes the data within 1 month, and calculates the supplementary value -1 or the average value as N and A, respectively , Calculated that the ratio of the value -1 should be complemented is (N/N+A), and the ratio of the average value should be complemented is (A/N+A), assuming that the missing data collected by the system in the future will also meet this 1 The distribution of missing data in the month, so in the future, when the data encounters missing data, a random process will be determined, using the previous ratio as the probability to determine the complement value -1 or the average value.

The process S32 is data digitization processing. There are two main types of features in the data, one is a continuous value, such as CPU usage, memory usage, etc., and the other is a discrete value, such as server state. As explained above in this case, the model must be trained Type needs to be all numerical values, so the status is converted to numbers, such as OK to 0, Failure to 1; In addition, the discrete value in this embodiment does not take a negative number, so the value -1 that was originally a missing value will be the last The number is replaced, which is 2 in this example, as shown in Figure 7.

The process S33 is data normalization processing. Since the values in the original data are often of different sizes, for support vector machines (SVM) or neural networks (Neural Network, NN) and other algorithms that use weights (weight), if the values are normalized , Let the value be between 0 and 1 (max-min normalization), or subtract the average and divide by the standard deviation (zero-mean normalization), which can make the model easier to train. The above-mentioned standard normalization process data processing method makes the importance of each feature item the same, and the model determines the importance of the feature item.

For these algorithms that use weighting types, because the values are related to each other, some discrete values are actually not related to each other. For example, the state of the server that just converted text into numbers, in order to make each state fair. Yes, you can do one-hot encoding, so the original three values 0, 1, 2 will become three-dimensional vectors [1,0,0], [0,1,0], [0,0, 1] This is equivalent to giving the three states an independent weight for training to achieve fair training. In addition, if it is an algorithm like decision tree (decision tree, random forest), you can use equality operators, while simulated annealing (adaboost, gradient boosting) type algorithms have greater flexibility in training, so you don’t have to do these Treatment can be used selectively.

The process S34 is the discretization of continuous data. This process discretizes some continuous features, such as CPU usage. First, the continuous CPU usage values are grouped by k-means algorithm, and then discretized. As shown in Figure 8, set The k-means algorithm should divide the data into three categories, so the CPU usage data can be grouped into three categories, namely 0%~30%, 30%~70%, and 70%~100%. The above classification of data into these three categories means that if three data distribution centroids are to be found for the distribution of CPU data, then the cutting method will be Divided into these three types of data, this also means that using these three types of data can describe the characteristics of the original data, instead of using the original complex data, which can simplify the training burden of the machine learning model. Before counting for a long period of time (for example, from one hour ago to now), these three types occurred several times and turned into three features. As shown in the figure, the three values of the data that the system keeps normal are respectively [ 7,1,2], the data that the system is about to be abnormal is [3,2,5].

The above recording method is like making the system indicators into fingerprints. Different types of fingerprints tend to appear in different system states. For the same or similar system states, the system fingerprints will be similar or even identical to each other. Use this The method has the following advantages: (1) It can consider the state of a longer time ago; (2) The number of features (feature) will not increase too much; (3) The frequency of the state occurrence is considered, and the time point of the state occurrence is not considered . Therefore, these special features can help the model further obtain various information from a long time ago to increase the accuracy of prediction.

The process S35 is the data processing of the forecast target item. In this process, the future (for example, 10 minutes) status of the server that you want to predict is taken out and used as the predicted label (predict_status), as shown in Figure 9A. Based on the fact that system abnormalities usually occur for a continuous period of time, when the abnormality has occurred, it is not meaningful for us to predict the future abnormality. Normal should be predicted before the continuous abnormality occurs. In other words, we only need to consider We predict the accuracy of the first system anomaly, so we keep the continuous anomaly data only the first data, as shown in Figure 9B. If there are other normal data between continuous abnormalities, calculate the ratio of the number of normal data to the continuous abnormal data before and after. If the normal data only accounts for the sum of the three types of data Below 10%, then the whole whole can be regarded as a single continuous data event.

The data processing of process S35 will be executed in two stages. In the first stage, all the data will be scanned to confirm which part of the abnormalities should be recognized as continuous abnormalities, and then the data will be deleted in the second stage. It should be noted that the first stage may encounter the possibility of two or more continuous abnormal intervals overlapping, such as continuous-normal-continuous-normal-continuous. In this situation, the first three sets of data may be regarded as continuous abnormalities. The last three sets of data are also regarded as continuous anomalies. Because the middle continuous anomalies repeatedly appear in the before and after anomaly intervals, these five sets of data will be regarded as the same set of continuous anomalies and will be deleted together in the second stage.

The process S36 is a method for establishing a model using the random forest algorithm. This process uses data to train the random forest model. In order to deal with the extremely rare imbalance problem, the class_weight parameter must be set to balanced, and under-sampling is performed during training, which is about to be too much The normal data is reduced, allowing the model to focus more on training abnormal data. Use grid search to scan through all important parameters in random forests and set specific parameter ranges. For example, n_estimators ranges from 1 to 20 with an interval of 1, so there are 20 combinations of this parameter. You can search through all good parameter combinations, about tens of thousands of combinations. After searching, all good models are usually selected not to exceed 200, and then these models are used together to make predictions, and the results of the predictions are combined (ensemble), the combined method is threshold voting, similar to the majority voting method, but as long as there are more than the threshold number of models, it is abnormal, so it is predicted to be abnormal.

The process S37 is to establish a residual network using the linear gate valve unit. Due to the relevance of the time series data before and after the data, considering the results before the current time point is helpful for the current prediction, the present invention proposes to use a linear gate valve unit to establish a residual network to feed back the past prediction results to the current prediction The result is as shown in Figure 10. Specifically, the residual network will first concatenate the performance parameter (X) used in the main prediction model with the previous prediction result (Y') to form a vector of dimensions (X+Y'), and then use a Simple linear conversion, through (X+Y')*2 parameters, convert this vector into a 2-dimensional vector (a, b), and finally, calculate the residual R=a*σ(b), where σ(x) It is a sigmoid function (1/(1+exp(-x))), which can transform the input into the range of (0,1), so σ(b) can control how much a can become the final residual. In summary, the current prediction final result Y _final will be obtained by adding the prediction Y of the main prediction model to the residual R, and the adjustment of the (X+Y')*2 parameters in the residual network will also be through the reverse To the propagation algorithm, use the loss value calculated by the loss function to adjust.

The flow S38 is the determination of abnormal items. In this process, the trained model uses the InfoGain algorithm for feature selection to select more important features, that is, factors that are likely to cause system abnormalities, so that the model predicts that the system is likely to be about to When an abnormality occurs, the factors that cause the system abnormality are checked first, so as to achieve the purpose of preventing the system abnormality before it occurs. In addition, this method of analyzing and predicting models can automatically find out the monitoring items that have a large impact on the system, and then automatically establish rules.

The process S39 is the retraining of the predictive model. As mentioned at the same beginning, when the time passes or the system changes significantly, the process S30 to the process S37 need to be repeated, the old data is removed and new data is added, and the model is retrained.

In summary, the method for predicting rare events that cause system abnormalities proposed in the present invention can detect system abnormalities when system abnormalities are extremely rare and with acceptable false alarm rates, and can be made without modifying the existing system. Prediction, after predicting that errors may occur, use the results of machine learning algorithms to generate the most likely causes of errors to prevent errors occur.

The above-mentioned embodiments only exemplarily illustrate the principles and effects of the present invention, and are not used to limit the present invention. Anyone familiar with this technique can modify and change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the rights of the present invention should be listed in the scope of patent application described later.

S11~S17‧‧‧Step

Claims (9)

A method for predicting rare events that cause system anomalies includes: collecting original monitoring data from each server, and then integrating the original monitoring data with time as the key to integrate the original monitoring data into a data; filling the original monitoring data The missing part of the monitoring data; the text part of the original monitoring data is digitized, and the content of the different types of text data is sequentially arranged in a numerical sequence starting from 0, so that the original monitoring data are all numerical data; All values in the original monitoring data are normalized; based on the continuous data in a period of time in the original monitoring data, the original monitoring data is grouped and discretized, so as to count the total number of continuous data in the period of time. As a parameter representing the original monitoring data, the parameters representing the original monitoring data are trained to establish an anomaly classification prediction model; and the model characteristics of the anomaly classification prediction model are used to obtain at least one parameter that caused an abnormality to occur. The at least one parameter and the parameter with high correlation predict the possible cause of the system abnormality. For example, the method for predicting rare events that cause system abnormalities as described in item 1 of the scope of patent application, wherein the step of filling the missing part in the original monitoring data includes filling the missing part with a specific value, and filling in the continuous value Upper average or specific value. For example, the method for predicting rare events that cause system anomalies as described in item 2 of the scope of patent application, wherein the step of digitizing the text part of the original monitoring data is to use a number to fill the missing part with a specific value replace. For example, the method for predicting rare events that cause system abnormalities as described in item 1 of the scope of patent application, wherein the step of grouping the original monitoring data is to use the k-means grouping algorithm to group the original monitoring data. For example, the method for predicting rare events that cause system abnormalities as described in item 1 of the scope of patent application, wherein the step of training the parameters representing the original monitoring data includes using the random forest algorithm to supersede the random forest algorithm. A range of parameters is set, and then a grid search method is used to search for parameters within the range at a certain interval. For example, the method for predicting rare events that cause system anomalies as described in item 1 of the scope of patent application, wherein the step of training the parameters representing the original monitoring data to establish an anomaly classification prediction model includes the use of under-sampling (under- sampling) method to reduce the amount of the original monitoring data. A method for predicting rare events that cause system anomalies includes: collecting original monitoring data from each server, and then integrating the original monitoring data with time as the key to integrate the original monitoring data into a data; filling the original monitoring data The missing part in the monitoring data; the text part of the original monitoring data is digitized so that the original monitoring data becomes numerical data; all the values in the original monitoring data are normalized; the original monitoring data is Based on the continuous data in a period of time, the original monitoring data is grouped and discretized to count the total number of continuous data in the period of time, which is used as a parameter to represent the original monitoring data. Parameter training to build anomaly classification prediction model; and Use the model characteristics of the abnormal classification prediction model to obtain at least one parameter that caused the abnormality, and predict the possible cause of the system abnormality from the at least one parameter and the parameter with high correlation, wherein the pair represents the parameter of the original monitoring data The steps of training include the use of random forest algorithm to determine that only the first time point is abnormal for continuous abnormal data, and the remaining time points are regarded as normal. For example, the method for predicting rare events that cause system abnormalities as described in item 1 or 7 of the scope of patent application, wherein the step of using the model characteristics of the abnormal classification prediction model to obtain at least one parameter that caused the abnormality refers to the use of information gain ( The Information Gain algorithm performs feature selection, and takes the parameter that has a greater influence on the system abnormality as the at least one parameter. For example, the method for predicting rare events that cause system abnormalities as described in item 1 or 7 of the scope of patent application, wherein the step of using the model characteristics of the abnormal classification prediction model to obtain at least one parameter that caused the abnormality includes using a linear gate valve unit Control the degree of influence of the previous prediction data on the result, and use the residual network to feed back the result to the current result to be predicted.

Images