[go: up one dir, main page]

CN119847857A - Database anomaly detection and prediction method and system based on deep learning - Google Patents

Database anomaly detection and prediction method and system based on deep learning Download PDF

Info

Publication number
CN119847857A
CN119847857A CN202411775938.0A CN202411775938A CN119847857A CN 119847857 A CN119847857 A CN 119847857A CN 202411775938 A CN202411775938 A CN 202411775938A CN 119847857 A CN119847857 A CN 119847857A
Authority
CN
China
Prior art keywords
database
deep learning
data
model
features
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202411775938.0A
Other languages
Chinese (zh)
Inventor
赵春蕾
侯永东
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Inspur Cloud Information Technology Co Ltd
Original Assignee
Inspur Cloud Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Inspur Cloud Information Technology Co Ltd filed Critical Inspur Cloud Information Technology Co Ltd
Priority to CN202411775938.0A priority Critical patent/CN119847857A/en
Publication of CN119847857A publication Critical patent/CN119847857A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/3034—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a storage system, e.g. DASD based or network based
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
    • G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3089—Monitoring arrangements determined by the means or processing involved in sensing the monitored data, e.g. interfaces, connectors, sensors, probes, agents
    • G06F11/3093—Configuration details thereof, e.g. installation, enabling, spatial arrangement of the probes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3452—Performance evaluation by statistical analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Quality & Reliability (AREA)
  • Computing Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Computer Hardware Design (AREA)
  • Testing And Monitoring For Control Systems (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

本发明公开了一种基于深度学习的数据库异常检测与预测方法及系统，属于数据库管理技术领域，该方法的实现包括以下步骤：1)数据预处理：从数据库日志和性能监控工具中收集原始数据，并进行清洗和格式化；2)特征提取：从预处理后的数据中提取有用的特征；3)深度学习模型训练：构建和训练深度学习模型，以识别数据库操作中的正常和异常模式；4)异常预测：使用训练好的深度学习模型对实时数据库操作进行监控，并预测潜在的异常行为。本发明能够自动学习数据库操作的正常模式，并有效地识别和预测异常行为，从而提高数据库系统的安全性和稳定性。

The present invention discloses a database anomaly detection and prediction method and system based on deep learning, which belongs to the field of database management technology. The implementation of the method includes the following steps: 1) data preprocessing: collecting raw data from database logs and performance monitoring tools, and cleaning and formatting; 2) feature extraction: extracting useful features from the preprocessed data; 3) deep learning model training: building and training deep learning models to identify normal and abnormal patterns in database operations; 4) abnormality prediction: using the trained deep learning model to monitor real-time database operations and predict potential abnormal behaviors. The present invention can automatically learn the normal mode of database operation, and effectively identify and predict abnormal behaviors, thereby improving the security and stability of the database system.

Description

Database anomaly detection and prediction method and system based on deep learning
Technical Field
The invention relates to the technical field of database management, in particular to a database abnormality detection and prediction method and system based on deep learning.
Background
With the rapid development of information technology, database systems have become an integral part of enterprises and organizations for storing and managing large amounts of data. However, the complexity and increasing amount of data in database systems makes the systems more susceptible to abnormal behavior and failure. Traditional anomaly detection methods, such as rule-based systems and statistical methods, tend to be difficult to adapt to rapidly changing environments and complex data patterns. Thus, there is a need for a more advanced technique to improve the accuracy and efficiency of database anomaly detection.
Disclosure of Invention
Aiming at the defects, the technical task of the invention is to provide a database abnormality detection and prediction method and a database abnormality detection and prediction system based on deep learning, which can automatically learn the normal mode of database operation and effectively identify and predict abnormal behaviors, thereby improving the safety and stability of a database system.
The technical scheme adopted for solving the technical problems is as follows:
a database anomaly detection and prediction method based on deep learning, the implementation of the method comprises the following steps:
1) Data preprocessing, namely collecting original data from database logs and performance monitoring tools, and cleaning and formatting the original data;
2) Extracting useful features from the preprocessed data;
3) Deep learning model training, namely constructing and training a deep learning model to identify normal and abnormal modes in database operation;
4) And (3) abnormality prediction, namely monitoring the operation of the real-time database by using a trained deep learning model and predicting potential abnormal behaviors.
And monitoring database operation in real time through the deep learning model, identifying potential abnormal behaviors and predicting possible system faults. By the method, the safety and stability of the database system can be obviously improved, and the loss caused by abnormal behaviors is reduced.
Further, the data preprocessing specifically includes:
data cleaning, namely removing invalid or wrong data records, including format errors, missing values and the like;
data formatting, namely unifying data from different sources into a format, so that subsequent processing is facilitated;
and (3) data normalization, namely performing normalization processing on the data to eliminate dimension influence among different features.
Further, the feature extraction extracts useful features from the preprocessed data, which features can represent behavior patterns of database operations.
Further, the feature extraction specifically includes:
the statistical feature extraction, namely calculating the statistical features of database operation, including average value, variance, maximum value, minimum value and the like;
Extracting time sequence characteristics of database operation, including autocorrelation, periodicity and the like;
pattern recognition feature extraction, namely recognizing patterns in database operation by using a machine learning method and extracting relevant features.
Further, the deep learning model training specifically includes:
Selecting a proper deep learning model, wherein the deep learning model comprises a Convolutional Neural Network (CNN), a cyclic neural network (RNN) or a long-short-term memory network (LSTM);
model training, namely training a deep learning model by using the marked normal and abnormal database operation data;
And (3) model verification, namely evaluating the performance of the model through methods such as cross verification and the like, and performing tuning.
Further, the deep learning model specifically includes:
the data representation, database operation data, may be represented as a sequence, wherein each operation is an event at a point in time, such as a query, update, insert or delete, etc.;
The LSTM network consists of a plurality of LSTM units, each unit comprises an input gate, a forgetting gate and an output gate, and the gates control the flow of information so as to avoid the gradient disappearance problem of the traditional RNN;
Feature input, wherein the features of each time point comprise statistical features, time sequence features and pattern recognition features, and the features are input into an LSTM network to capture the dynamic behavior of database operation;
a loss function, training a model using the cross entropy loss function to distinguish normal and abnormal behavior, the model being aimed at minimizing the difference between the predicted tag and the real tag;
the model weight is updated by using an Adam optimization algorithm, and the model weight can adapt to different learning rates because the model weight combines the advantages of a gradient descent method and a momentum method;
model evaluation, namely evaluating the performance of the model through indexes including accuracy, recall rate, F1 score and the like, wherein the indexes can comprehensively reflect the detection capability of the model;
And (3) model deployment, namely deploying the trained model into a production environment, monitoring database operation in real time, and predicting abnormal behaviors.
Further, the anomaly prediction specifically includes:
Real-time data flow processing, namely collecting database operation data in real time, and preprocessing and extracting features;
abnormality detection, namely classifying real-time data by using a deep learning model and identifying abnormal behaviors;
and (3) predicting abnormal behaviors possibly occurring in the future according to the historical data and the current behavior mode.
The invention also claims a database anomaly detection and prediction system based on deep learning, which comprises:
the data preprocessing module is used for collecting original data from the database log and the performance monitoring tool, and cleaning and formatting the original data;
A feature extraction module for extracting useful features from the preprocessed data;
The deep learning model training module is used for constructing and training a deep learning model to identify normal and abnormal modes in database operation;
the anomaly prediction module is used for monitoring the operation of the real-time database by using the trained deep learning model and predicting potential anomaly behaviors;
The system specifically realizes the detection and prediction of the database abnormality by the database abnormality detection and prediction method based on deep learning.
The invention also claims a database abnormality detection and prediction device based on deep learning, which comprises at least one memory and at least one processor;
The at least one memory for storing a machine readable program;
The at least one processor is configured to invoke the machine-readable program to implement the method described above.
The invention also claims a computer readable medium having stored thereon computer instructions which, when executed by a processor, cause the processor to perform the above-described method.
Compared with the prior art, the database anomaly detection and prediction method and system based on deep learning have the following beneficial effects:
The method monitors database operation in real time through the deep learning model, identifies potential abnormal behaviors and predicts possible system faults. By the method or the system, the safety and the stability of the database system can be obviously improved, and the loss caused by abnormal behaviors is reduced.
The method and the system have better effects in the aspects of improving the detection accuracy, reducing false alarms, monitoring in real time, predicting future anomalies, reducing maintenance cost and the like in the aspect of database anomaly detection.
Drawings
Fig. 1 is a schematic diagram of a database anomaly detection and prediction method based on deep learning according to an embodiment of the present invention.
Detailed Description
The invention will be further described with reference to the drawings and the specific examples.
The embodiment of the invention provides a database anomaly detection and prediction method based on deep learning, which comprises the following steps:
1. Data preprocessing, namely collecting original data from database logs and performance monitoring tools, and cleaning and formatting the original data;
2. Extracting useful features from the preprocessed data, wherein the features can represent behavior patterns of database operations;
3. Deep learning model training, namely constructing and training a deep learning model to identify normal and abnormal modes in database operation;
4. And (3) abnormality prediction, namely monitoring the operation of the real-time database by using a trained deep learning model and predicting potential abnormal behaviors.
The data preprocessing specifically comprises the following steps:
data cleaning, namely removing invalid or wrong data records, including format errors, missing values and the like;
data formatting, namely unifying data from different sources into a format, so that subsequent processing is facilitated;
and (3) data normalization, namely performing normalization processing on the data to eliminate dimension influence among different features.
The feature extraction specifically comprises the following steps:
Statistical feature extraction, namely calculating the statistical features of database operation, such as average value, variance, maximum value, minimum value and the like;
Extracting time sequence characteristics of database operation, including autocorrelation, periodicity and the like;
pattern recognition feature extraction, namely recognizing patterns in database operation by using a machine learning method and extracting relevant features.
The deep learning model training specifically comprises the following steps:
Selecting a proper deep learning model, wherein the deep learning model comprises a Convolutional Neural Network (CNN), a cyclic neural network (RNN) or a long-short-term memory network (LSTM);
model training, namely training a deep learning model by using the marked normal and abnormal database operation data;
And (3) model verification, namely evaluating the performance of the model through methods such as cross verification and the like, and performing tuning.
The anomaly prediction specifically comprises the following steps:
Real-time data flow processing, namely collecting database operation data in real time, and preprocessing and extracting features;
abnormality detection, namely classifying real-time data by using a deep learning model and identifying abnormal behaviors;
and (3) predicting abnormal behaviors possibly occurring in the future according to the historical data and the current behavior mode.
Where the deep learning model is the core component that automatically learns and extracts features from a large amount of data to identify and predict abnormal behavior. The deep learning model specifically comprises:
the data representation, database operation data, may be represented as a sequence, wherein each operation is an event at a point in time, such as a query, update, insert or delete, etc.;
The LSTM network consists of a plurality of LSTM units, each unit comprises an input gate, a forgetting gate and an output gate, and the gates control the flow of information so as to avoid the gradient disappearance problem of the traditional RNN;
Feature input, wherein the features of each time point comprise statistical features, time sequence features and pattern recognition features, and the features are input into an LSTM network to capture the dynamic behavior of database operation;
a loss function, training a model using the cross entropy loss function to distinguish normal and abnormal behavior, the model being aimed at minimizing the difference between the predicted tag and the real tag;
the model weight is updated by using an Adam optimization algorithm, and the model weight can adapt to different learning rates because the model weight combines the advantages of a gradient descent method and a momentum method;
Model evaluation, namely evaluating the performance of the model through indexes such as accuracy, recall rate, F1 score and the like, wherein the indexes can comprehensively reflect the detection capability of the model;
And (3) model deployment, namely deploying the trained model into a production environment, monitoring database operation in real time, and predicting abnormal behaviors.
Through the technical scheme, the database abnormal behavior can be efficiently detected and predicted, and the safety and stability of a database system are improved.
The embodiment of the invention also provides a database abnormality detection and prediction system based on deep learning, which realizes the detection and prediction of the database abnormality by the database abnormality detection and prediction method based on deep learning.
The system comprises:
1. And a data preprocessing module.
The data preprocessing module is responsible for collecting raw data from database logs and performance monitoring tools, and cleaning and formatting the raw data for subsequent processing. The module comprises the following steps:
data cleaning, namely removing invalid or wrong data records, such as format errors, missing values and the like;
data formatting, namely unifying data from different sources into a format, so that subsequent processing is facilitated;
and (3) data normalization, namely performing normalization processing on the data to eliminate dimension influence among different features.
2. And the characteristic extraction module.
The feature extraction module is responsible for extracting useful features from the preprocessed data, which can represent the behavior patterns of database operations. The module comprises the following steps:
Statistical feature extraction, namely calculating the statistical features of database operation, such as average value, variance, maximum value, minimum value and the like;
Extracting time sequence characteristics of database operation, such as autocorrelation, periodicity and the like;
pattern recognition feature extraction, namely recognizing patterns in database operation by using a machine learning method and extracting relevant features.
3. And a deep learning model training module.
The deep learning model training module is responsible for constructing and training deep learning models to identify normal and abnormal patterns in database operations. The module comprises the following steps:
model selection, namely selecting a proper deep learning model, such as a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN) or a long-short-term memory network (LSTM);
model training, namely training a deep learning model by using the marked normal and abnormal database operation data;
And (3) model verification, namely evaluating the performance of the model through methods such as cross verification and the like, and performing tuning.
4. And an anomaly prediction module.
The anomaly prediction module is responsible for monitoring the operation of the real-time database by using a trained deep learning model and predicting potential anomaly behaviors. The module comprises the following steps:
Real-time data flow processing, namely collecting database operation data in real time, and preprocessing and extracting features;
abnormality detection, namely classifying real-time data by using a deep learning model and identifying abnormal behaviors;
and (3) predicting abnormal behaviors possibly occurring in the future according to the historical data and the current behavior mode.
Among other things, deep learning models are core components that automatically learn and extract features from large amounts of data to identify and predict abnormal behavior. The following is a detailed description of the deep learning model, including:
the data representation database operation data may be represented as a sequence, where each operation is an event at a point in time. These events may be queries, updates, inserts or deletions, etc.
Model architecture LSTM is chosen as the base model because it is able to efficiently process time series data and capture long-term dependencies. The LSTM network is made up of a plurality of LSTM cells, each cell containing an input gate, a forget gate, and an output gate that control the flow of information to avoid the gradient vanishing problem of a conventional RNN.
The characteristic input comprises a statistical characteristic, a time sequence characteristic and a mode identification characteristic at each time point. These features are input into the LSTM network to capture the dynamic behavior of database operations.
Loss function the model is trained using cross entropy loss functions to distinguish normal and abnormal behavior. The goal of the model is to minimize the difference between the predicted tag and the real tag.
Optimization algorithm Adam optimization algorithm is used to update model weights because it combines the advantages of gradient descent and momentum methods to adapt to different learning rates.
Model evaluation, namely evaluating the performance of the model through indexes such as accuracy, recall rate, F1 score and the like. These indicators can fully reflect the detectability of the model.
And (3) model deployment, namely deploying the trained model into a production environment, monitoring database operation in real time, and predicting abnormal behaviors.
And monitoring database operation in real time through the deep learning model, identifying potential abnormal behaviors and predicting possible system faults. The system can obviously improve the safety and stability of the database system and reduce the loss caused by abnormal behaviors.
The embodiment of the invention also provides a database abnormality detection and prediction device based on deep learning, which comprises at least one memory and at least one processor;
The at least one memory for storing a machine readable program;
The at least one processor is configured to invoke the machine-readable program to implement the database anomaly detection and prediction method based on deep learning described in the foregoing embodiments.
The embodiment of the invention also provides a computer readable medium, on which computer instructions are stored, which when executed by a processor, cause the processor to execute the database anomaly detection and prediction method based on deep learning described in the above embodiment. Specifically, a system or apparatus provided with a storage medium on which a software program code realizing the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or apparatus may be caused to read out and execute the program code stored in the storage medium.
In this case, the program code itself read from the storage medium may realize the functions of any of the above-described embodiments, and thus the program code and the storage medium storing the program code form part of the present invention.
Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (e.g., CD-ROMs, CD-R, CD-RWs, DVD-ROMs, DVD-RAMs, DVD-RWs, DVD+RWs), magnetic tapes, nonvolatile memory cards, and ROMs. Alternatively, the program code may be downloaded from a server computer by a communication network.
Further, it should be apparent that the functions of any of the above-described embodiments may be implemented not only by executing the program code read out by the computer, but also by causing an operating system or the like operating on the computer to perform part or all of the actual operations based on the instructions of the program code.
Further, it is understood that the program code read out by the storage medium is written into a memory provided in an expansion board inserted into a computer or into a memory provided in an expansion unit connected to the computer, and then a CPU or the like mounted on the expansion board or the expansion unit is caused to perform part and all of actual operations based on instructions of the program code, thereby realizing the functions of any of the above embodiments.
While the invention has been illustrated and described in detail in the drawings and in the preferred embodiments, the invention is not limited to the disclosed embodiments, and it will be appreciated by those skilled in the art that the code audits of the various embodiments described above may be combined to produce further embodiments of the invention, which are also within the scope of the invention.

Claims (10)

1.一种基于深度学习的数据库异常检测与预测方法，其特征在于，该方法的实现包括以下步骤：1. A database anomaly detection and prediction method based on deep learning, characterized in that the implementation of the method includes the following steps: 1)数据预处理：从数据库日志和性能监控工具中收集原始数据，并进行清洗和格式化；1) Data preprocessing: Collect raw data from database logs and performance monitoring tools, and clean and format them; 2)特征提取：从预处理后的数据中提取有用的特征；2) Feature extraction: extract useful features from preprocessed data; 3)深度学习模型训练：构建和训练深度学习模型，以识别数据库操作中的正常和异常模式；3) Deep learning model training: Build and train deep learning models to identify normal and abnormal patterns in database operations; 4)异常预测：使用训练好的深度学习模型对实时数据库操作进行监控，并预测潜在的异常行为。4) Anomaly prediction: Use the trained deep learning model to monitor real-time database operations and predict potential abnormal behaviors. 2.根据权利要求1所述的一种基于深度学习的数据库异常检测与预测方法，其特征在于，所述数据预处理，具体包括：2. According to a method for detecting and predicting database anomalies based on deep learning according to claim 1, it is characterized in that the data preprocessing specifically includes: 数据清洗：去除无效或错误的数据记录，包括格式错误、缺失值；Data cleaning: remove invalid or erroneous data records, including format errors and missing values; 数据格式化：将不同来源的数据统一格式；Data formatting: unify the format of data from different sources; 数据归一化：对数据进行归一化处理，以消除不同特征之间的量纲影响。Data normalization: Normalize the data to eliminate the dimensional effects between different features. 3.根据权利要求1所述的一种基于深度学习的数据库异常检测与预测方法，其特征在于，所述特征提取，从预处理后的数据中提取有用的特征，所述特征能够代表数据库操作的行为模式。3. According to a deep learning-based database anomaly detection and prediction method according to claim 1, it is characterized in that the feature extraction extracts useful features from the preprocessed data, and the features can represent the behavioral patterns of database operations. 4.根据权利要求1或3所述的一种基于深度学习的数据库异常检测与预测方法，其特征在于，所述特征提取，具体包括：4. A database anomaly detection and prediction method based on deep learning according to claim 1 or 3, characterized in that the feature extraction specifically includes: 统计特征提取：计算数据库操作的统计特征，包括平均值、方差、最大值和最小值；Statistical feature extraction: Calculate the statistical features of database operations, including mean, variance, maximum and minimum values; 时序特征提取：提取数据库操作的时间序列特征，包括自相关性、周期性；Time series feature extraction: extract the time series features of database operations, including autocorrelation and periodicity; 模式识别特征提取：使用机器学习方法识别数据库操作中的模式，并提取相关特征。Pattern recognition and feature extraction: Use machine learning methods to identify patterns in database operations and extract relevant features. 5.根据权利要求1所述的一种基于深度学习的数据库异常检测与预测方法，其特征在于，所述深度学习模型训练，具体包括：5. According to a method for detecting and predicting database anomalies based on deep learning according to claim 1, it is characterized in that the deep learning model training specifically includes: 模型选择：选择合适的深度学习模型；深度学习模型包括：卷积神经网络、循环神经网络或长短期记忆网络；Model selection: Choose a suitable deep learning model; deep learning models include: convolutional neural network, recurrent neural network or long short-term memory network; 模型训练：使用标注的正常和异常数据库操作数据训练深度学习模型；Model training: Use labeled normal and abnormal database operation data to train deep learning models; 模型验证：评估模型的性能，并进行调优。Model validation: Evaluate the performance of the model and perform tuning. 6.根据权利要求1或5所述的一种基于深度学习的数据库异常检测与预测方法，其特征在于，所述深度学习模型具体实现包括：6. A database anomaly detection and prediction method based on deep learning according to claim 1 or 5, characterized in that the specific implementation of the deep learning model includes: 数据表示：数据库操作数据表示为序列，其中每个操作是一个时间点上的事件；这些事件可以是查询、更新、插入或删除；Data representation: Database operation data is represented as a sequence, where each operation is an event at a point in time; these events can be queries, updates, inserts, or deletes; 模型架构：选择LSTM作为基础模型，LSTM网络由多个LSTM单元组成，每个单元包含输入门、遗忘门和输出门，这些门控制信息的流动；Model architecture: LSTM is selected as the basic model. The LSTM network consists of multiple LSTM units. Each unit contains an input gate, a forget gate, and an output gate. These gates control the flow of information. 特征输入：每个时间点的特征包括统计特征、时序特征和模式识别特征；这些特征被输入到LSTM网络中，以捕捉数据库操作的动态行为；Feature input: The features at each time point include statistical features, time series features, and pattern recognition features; these features are input into the LSTM network to capture the dynamic behavior of database operations; 损失函数：使用交叉熵损失函数来训练模型，以区分正常和异常行为；模型的目标是最小化预测标签和真实标签之间的差异；Loss function: The model is trained using the cross entropy loss function to distinguish between normal and abnormal behavior; the goal of the model is to minimize the difference between the predicted label and the true label; 优化算法：使用Adam优化算法来更新模型权重；Optimization algorithm: Use the Adam optimization algorithm to update the model weights; 模型评估：通过能够全面反映模型的检测能力的指标评估模型的性能；指标包括精确度、召回率和F1分数；Model evaluation: Evaluate the performance of the model through indicators that can fully reflect the detection ability of the model; indicators include precision, recall rate and F1 score; 模型部署：将训练好的模型部署到生产环境中，实时监控数据库操作，并预测异常行为。Model deployment: Deploy the trained model to the production environment, monitor database operations in real time, and predict abnormal behaviors. 7.根据权利要求1所述的一种基于深度学习的数据库异常检测与预测方法，其特征在于，所述异常预测，具体包括：7. The method for detecting and predicting database anomalies based on deep learning according to claim 1, wherein the anomaly prediction specifically comprises: 实时数据流处理：实时收集数据库操作数据，并进行预处理和特征提取；Real-time data stream processing: collect database operation data in real time, and perform preprocessing and feature extraction; 异常检测：使用深度学习模型对实时数据进行分类，识别异常行为；Anomaly detection: Use deep learning models to classify real-time data and identify abnormal behavior; 异常预测：根据历史数据和当前行为模式，预测未来可能发生的异常行为。Anomaly prediction: Based on historical data and current behavior patterns, predict abnormal behaviors that may occur in the future. 8.一种基于深度学习的数据库异常检测与预测系统，其特征在于，该系统包括：8. A database anomaly detection and prediction system based on deep learning, characterized in that the system comprises: 数据预处理模块，用于从数据库日志和性能监控工具中收集原始数据，并进行清洗和格式化；Data preprocessing module, used to collect raw data from database logs and performance monitoring tools, and clean and format them; 特征提取模块，用于从预处理后的数据中提取有用的特征；Feature extraction module, used to extract useful features from preprocessed data; 深度学习模型训练模块，用于构建和训练深度学习模型，以识别数据库操作中的正常和异常模式；A deep learning model training module for building and training deep learning models to identify normal and abnormal patterns in database operations; 异常预测模块，用于使用训练好的深度学习模型对实时数据库操作进行监控，并预测潜在的异常行为；The anomaly prediction module is used to monitor real-time database operations using a trained deep learning model and predict potential abnormal behaviors; 该系统具体通过权利要求1至7任一所述的基于深度学习的数据库异常检测与预测方法实现数据库异常检测与预测。The system specifically implements database anomaly detection and prediction through the deep learning-based database anomaly detection and prediction method described in any one of claims 1 to 7. 9.一种基于深度学习的数据库异常检测与预测装置，其特征在于，包括：至少一个存储器和至少一个处理器；9. A database anomaly detection and prediction device based on deep learning, characterized by comprising: at least one memory and at least one processor; 所述至少一个存储器，用于存储机器可读程序；The at least one memory is used to store a machine-readable program; 所述至少一个处理器，用于调用所述机器可读程序，实现权利要求1至7任一所述的方法。The at least one processor is used to call the machine-readable program to implement the method described in any one of claims 1 to 7. 10.一种计算机可读介质，其特征在于，所述计算机可读介质上存储有计算机指令，所述计算机指令在被处理器执行时，使所述处理器执行权利要求1至7任一所述的方法。10. A computer-readable medium, characterized in that computer instructions are stored on the computer-readable medium, and when the computer instructions are executed by a processor, the processor executes any one of the methods of claims 1 to 7.
CN202411775938.0A 2024-12-05 2024-12-05 Database anomaly detection and prediction method and system based on deep learning Pending CN119847857A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202411775938.0A CN119847857A (en) 2024-12-05 2024-12-05 Database anomaly detection and prediction method and system based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202411775938.0A CN119847857A (en) 2024-12-05 2024-12-05 Database anomaly detection and prediction method and system based on deep learning

Publications (1)

Publication Number Publication Date
CN119847857A true CN119847857A (en) 2025-04-18

Family

ID=95355459

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202411775938.0A Pending CN119847857A (en) 2024-12-05 2024-12-05 Database anomaly detection and prediction method and system based on deep learning

Country Status (1)

Country Link
CN (1) CN119847857A (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200160211A1 (en) * 2018-11-21 2020-05-21 Sap Se Machine learning based database anomaly prediction
CN116541743A (en) * 2023-04-07 2023-08-04 中国邮政储蓄银行股份有限公司 Time sequence data abnormality detection method and device for database
CN116644418A (en) * 2023-05-24 2023-08-25 山东浪潮科学研究院有限公司 Database anomaly detection method based on genetic algorithm and convolutional neural network
CN117540372A (en) * 2023-11-22 2024-02-09 西藏朗杰信息科技有限公司 Database intrusion detection and response system for intelligent learning
CN117786405A (en) * 2023-12-20 2024-03-29 天翼安全科技有限公司 DQN-based database abnormal traffic detection method, device and equipment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200160211A1 (en) * 2018-11-21 2020-05-21 Sap Se Machine learning based database anomaly prediction
CN116541743A (en) * 2023-04-07 2023-08-04 中国邮政储蓄银行股份有限公司 Time sequence data abnormality detection method and device for database
CN116644418A (en) * 2023-05-24 2023-08-25 山东浪潮科学研究院有限公司 Database anomaly detection method based on genetic algorithm and convolutional neural network
CN117540372A (en) * 2023-11-22 2024-02-09 西藏朗杰信息科技有限公司 Database intrusion detection and response system for intelligent learning
CN117786405A (en) * 2023-12-20 2024-03-29 天翼安全科技有限公司 DQN-based database abnormal traffic detection method, device and equipment

Similar Documents

Publication Publication Date Title
CN108038049B (en) Real-time log control system and control method, cloud computing system and server
CN111027615B (en) Middleware fault early warning method and system based on machine learning
CN111475804A (en) Alarm prediction method and system
CN117743909B (en) A method and device for analyzing heating system failure based on artificial intelligence
US20180046529A1 (en) Log analysis system, log analysis method and program recording medium
CN117234785B (en) Centralized control platform error analysis system based on artificial intelligence self-query
CN119066541A (en) New energy station monitoring data quality evaluation method and system based on multi-source data
CN112988437B (en) Fault prediction method and device, electronic equipment and storage medium
CN111552843B (en) Fault prediction method based on weighted causal dependency graph
CN119691576B (en) Self-adaptive operation and maintenance root cause positioning method and system based on deep learning
CN115269314A (en) Transaction abnormity detection method based on log
CN117972596A (en) A risk prediction method based on operation log
CN118499316B (en) A hydraulic station status monitoring method and system based on cloud data
CN118555188A (en) An intelligent fault diagnosis method for railway dispatching centralized system
CN118133962A (en) Correlation analysis method, device and system of fault event and storage medium
CN108763048B (en) A method for early warning and reliability evaluation of hard disk failure based on particle filter
CN119179704A (en) Machine room information data management method and system
CN115293379B (en) An abnormal detection method for on-orbit spacecraft equipment based on knowledge graph
CN120196543A (en) An automated software testing method and system based on artificial intelligence
CN119313150A (en) A 360° self-inspection and self-correction system for corporate environmental compliance based on big data
CN118628254A (en) Risk behavior identification method and device
CN107085544A (en) A system error location method and device
CN119847857A (en) Database anomaly detection and prediction method and system based on deep learning
CN117909970A (en) Data processing method, device, equipment and medium
CN115017019B (en) Log-based system anomaly detection method, device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination