[go: up one dir, main page]

CN114168409A - Service system running state monitoring and early warning method and system - Google Patents

Service system running state monitoring and early warning method and system Download PDF

Info

Publication number
CN114168409A
CN114168409A CN202111424577.1A CN202111424577A CN114168409A CN 114168409 A CN114168409 A CN 114168409A CN 202111424577 A CN202111424577 A CN 202111424577A CN 114168409 A CN114168409 A CN 114168409A
Authority
CN
China
Prior art keywords
data
operating state
data set
state data
early warning
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111424577.1A
Other languages
Chinese (zh)
Other versions
CN114168409B (en
Inventor
魏章
李斌山
韩丹
赵磊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Mengpa Xinchuang Technology Co ltd
Shanghai Mengpa Information Technology Co ltd
Original Assignee
Beijing Mengpa Xinchuang Technology Co ltd
Shanghai Mengpa Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Mengpa Xinchuang Technology Co ltd, Shanghai Mengpa Information Technology Co ltd filed Critical Beijing Mengpa Xinchuang Technology Co ltd
Priority to CN202111424577.1A priority Critical patent/CN114168409B/en
Publication of CN114168409A publication Critical patent/CN114168409A/en
Application granted granted Critical
Publication of CN114168409B publication Critical patent/CN114168409B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3003Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/302Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/34Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3452Performance evaluation by statistical analysis

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Quality & Reliability (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Computer Hardware Design (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本发明公开了一种业务系统运行状态监控预警方法与系统,其中方法包括:采集待检测的业务系统的运行状态的工况信息,并存储为第一运行状态数据集;对第一运行状态数据集进行质量探索和特征分析,得到第二运行状态数据集;对第二运行状态数据集进行预处理,得到第三运行状态数据集;对第三运行状态数据集进行相似性度量,获取运行状态数据的相似性距离;对第三运行状态数据集进行模式挖掘处理,得到数据关联规则;基于相似性距离以及数据关联规则,生成第一组合模型;基于第一组合模型预测数据变化趋势,进行实时监控预警。本发明可以实时直观反映数据中心的各业务运行指标,保证系统可用性,预防突发事件发生,提高系统运行效率安全运行。

Figure 202111424577

The invention discloses a method and system for monitoring and early warning of the operating state of a business system, wherein the method comprises: collecting operating condition information of the operating state of a business system to be detected, and storing it as a first operating state data set; Perform quality exploration and feature analysis on the data set to obtain a second operating state data set; preprocess the second operating state data set to obtain a third operating state data set; perform similarity measurement on the third operating state data set to obtain the operating state similarity distance of data; perform pattern mining processing on the third operating state data set to obtain data association rules; generate a first combined model based on the similarity distance and data association rules; Monitor alerts. The invention can directly reflect various business operation indicators of the data center in real time, ensure the system availability, prevent the occurrence of emergencies, and improve the system operation efficiency and safe operation.

Figure 202111424577

Description

Service system running state monitoring and early warning method and system
Technical Field
The invention relates to the field of artificial intelligence, in particular to a service system running state monitoring and early warning method and system.
Background
The data center is used as a central place for storing and processing business data inside, and the safe operation in aspects of power, environment, security protection, network, equipment and the like of the data center is related to the operation of a global system. Therefore, the data center safety monitoring is used as an important component of operation, and has important effects on ensuring the safe operation of company business, improving the operation efficiency of daily work and the like.
Traditional data center maintenance work mainly depends on that the computer lab staff attends to on duty, however manual monitoring has very big limitation. Firstly, the requirement of the maintenance of a data center of a machine room on the professional degree of managers is high, and the problem cannot be found at the first time when equipment fails due to blind spots in manual operation; secondly, the data center equipment works all day long, so that a large amount of labor cost is required to be invested for watching, and all-day manual monitoring is difficult to realize; finally, the traditional manual on-duty monitoring has poor real-time performance of alarm and low response efficiency, and is not beneficial to the high-efficiency operation and maintenance of the data center. Once a system or equipment fails, the economic loss caused by the system or equipment is immeasurable, so that real-time intelligent monitoring management of the data center is very important.
With the development of technologies such as big data and artificial intelligence, the creation of an 'unattended' data center with high intelligence is a new development of a data center environment control system.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides the service system operation state monitoring and early warning method and the service system operation state monitoring and early warning system, which can intuitively reflect each service operation index of the data center in real time, ensure the availability of the system, prevent the occurrence of an emergency and improve the operation efficiency and the safe operation of the system.
In a first aspect, the present invention provides a method for monitoring and warning an operation state of a service system, including the following steps:
s101, collecting working condition information of the running state of a service system to be detected, and storing the working condition information as a first running state data set;
step S103, performing quality exploration and characteristic analysis on the first operation state data set to obtain a second operation state data set;
step S105, preprocessing the second running state data set to obtain a third running state data set;
step S107, carrying out similarity measurement on the third running state data set to obtain a similarity distance of the running state data;
step S109, performing pattern mining processing on the third running state data set to obtain a data association rule;
step S111, generating a first combination model based on the similarity distance and a data association rule;
and S113, carrying out real-time monitoring and early warning based on the first combined model prediction data change trend.
Wherein, the quality exploration and feature analysis in the step S103 at least include data missing, abnormal value analysis, overall distribution of data, statistic analysis and correlation analysis.
Wherein the preprocessing in the step S105 includes: data cleansing, data standardization, data reduction and data discretization.
Wherein the data normalization comprises:
Figure BDA0003378506590000031
wherein, XjFor the values of the data objects in the original data set,
Figure BDA0003378506590000032
value, X, normalized for the data objectmaxIs the maximum value, X, in the data objectminIs the minimum value in the data object.
Wherein the data reduction specifically comprises:
Figure BDA0003378506590000033
wherein,
Figure BDA0003378506590000034
the data object is subjected to data reduction to obtain a value, and the original data set with the original length of N is finally converted into the data set with the length of N in a segmented and aggregated manner
Figure BDA0003378506590000035
Wherein the data discretization comprises:
Figure BDA0003378506590000036
wherein,
Figure BDA0003378506590000037
the data object is subjected to data discretization, and finally, a time-series data set is converted into a character string set { u, l, d } according to the data change trend, wherein t is a fluctuation threshold of the data.
Wherein, the step S107 specifically includes:
and according to the third running state data set and aiming at different index data of the working condition information, acquiring a template subsequence in the period of hours, days, weeks and months respectively, calculating the similar distance between the current time subsequence of the same index data and the template subsequence, and predicting the data change in the future time.
Wherein, the step S109 specifically includes:
step S1091, mining a frequent mode of the business system for operating each index data according to the third operation state data set;
step S1093, mining the frequent mode of the business system running different index data according to the frequent mode of each index data;
and S1095, generating an association rule of the third operation state data set according to the frequent mode among the different index data.
Wherein, the step S111 specifically includes:
and generating the first combined model in a weighted mode based on the similarity distance and the prediction result of the association rule.
In a second aspect, the present invention further provides a service system operation state monitoring and early warning system, including:
the data acquisition module is used for acquiring the working condition information of the running state of the service system to be detected and storing the working condition information as a first running state data set;
the data exploration module is used for carrying out quality exploration and characteristic analysis on the first running state data set to obtain a second running state data set;
the data preprocessing module is used for preprocessing the second running state data set to obtain a third running state data set;
the sequence matching module is used for carrying out similarity measurement on the third running state data set to obtain the similarity distance of the running state data;
the pattern mining module is used for carrying out pattern mining processing on the third running state data set to obtain a data association rule;
the model optimization module generates a first combination model based on the similarity distance and the data association rule;
and the monitoring and early warning module is used for carrying out real-time monitoring and early warning based on the first combined model prediction data change trend.
The invention provides a service system running state monitoring and early warning method and system based on artificial intelligence, which can realize uninterrupted identification and early warning of abnormal states within 7-24 hours, reduce the working intensity of data center operators, greatly improve the accuracy and timeliness of fault finding while reducing the labor cost, and improve the stability and real-time performance of data center operation and maintenance work, thereby further improving the running efficiency of internal systems of companies.
Drawings
The above and other objects, features and advantages of exemplary embodiments of the present disclosure will become readily apparent from the following detailed description read in conjunction with the accompanying drawings. Several embodiments of the present disclosure are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar or corresponding parts and in which:
fig. 1 is a flowchart illustrating a method for monitoring and warning an operation state of a service system according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating pretreatment according to an embodiment of the present invention;
FIG. 3 is a flow diagram illustrating a pattern mining process according to an embodiment of the present invention;
fig. 4 is a schematic diagram illustrating a service system operation state monitoring and early warning system according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be described in further detail with reference to the accompanying drawings, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terminology used in the embodiments of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the examples of the present invention and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise, and "a plurality" typically includes at least two.
It is also noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that an article or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such article or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in the article or device in which the element is included.
Alternative embodiments of the present invention are described in detail below with reference to the accompanying drawings.
The first embodiment,
As shown in fig. 1, the present invention discloses a service system operation state monitoring and early warning method, which comprises the following steps:
s101, collecting working condition information of the running state of a service system to be detected, and storing the working condition information as a first running state data set;
step S103, performing quality exploration and characteristic analysis on the first operation state data set to obtain a second operation state data set;
step S105, preprocessing the second running state data set to obtain a third running state data set;
s107, carrying out similarity measurement on the third running state data set to obtain the similarity distance of the running state data;
step S109, performing pattern mining processing on the third running state data set to obtain a data association rule;
step S111, generating a first combination model based on the similarity distance and the data association rule;
and S113, carrying out real-time monitoring and early warning based on the data change trend of the first combined model prediction.
Example II,
As shown in fig. 1, the present invention provides a service system operation state monitoring and early warning method, which includes the following steps:
s101, collecting working condition information of the running state of a service system to be detected, and storing the working condition information as a first running state data set; the service system comprises a plurality of machines, wherein the service system comprises a plurality of pieces of service information, and the service information comprises monitoring index data information of running states of the machines in the service system, and specifically comprises a CPU utilization rate, a memory utilization rate, a disk utilization rate, a CPU idle rate, a buffer area utilization rate, a total file system utilization rate, a disk IO (input/output) waiting time, a response time, a network average total flow per second flowing out/flowing in and the like;
step S103, performing quality exploration and characteristic analysis on the first operation state data set to obtain a second operation state data set; the step is to search data aiming at the collected operation information data set of the service system, and know the gross quality, the size, the characteristics, the sample number, the data type, the probability distribution of the data and the like of the data set so as to better know the data characteristics of the data set;
step S105, preprocessing the second running state data set to obtain a third running state data set;
s107, carrying out similarity measurement on the third running state data set to obtain the similarity distance of the running state data; the method comprises the steps that sequence matching is carried out on related information in a single monitoring index of a monitored system, on a third running state data set, aiming at different index data, a subsequence of a template is obtained in a time angle by taking an hour period, a day period, a week period and a month period respectively, and the similar distance between the subsequence at the current moment and the subsequence of the template is calculated;
step S109, performing pattern mining processing on the third running state data set to obtain a data association rule; the step of pattern mining is a process of searching information hidden in a large amount of equipment operation data through an algorithm;
step S111, generating a first combination model based on the similarity distance and the data association rule;
and S113, carrying out real-time monitoring and early warning based on the data change trend of the first combined model prediction.
The embodiment of the present invention is further described in order to facilitate understanding of the quality exploration and the feature analysis in step S103. The quality exploration in this embodiment aims to check the correctness and validity of data, and its main task is to check whether dirty data exists in the original data, where dirty data generally refers to data that does not meet the requirements and cannot be directly analyzed accordingly.
In a practical application scenario, the quality exploration and feature analysis mentioned in step S103 of the present embodiment at least include data missing, abnormal value analysis, overall distribution of data, statistic analysis and correlation analysis.
To further understand data loss, outlier analysis, overall distribution of data, statistics analysis, and correlation analysis, they are further described below.
Data loss generally refers to both observed loss and loss of variable values in observations.
Outliers are those data that are incorrectly entered and contain unreasonable data, and are typically individual values in a sample whose values deviate significantly from the rest of the observations. In most cases, the variables are not allowed to have negative values, i.e., the negative values are abnormal values.
For abnormal value analysis, certain business background knowledge can be combined to find abnormality from the value taking condition of the variable, so as to judge whether data processing errors exist. The 3-sigma principle (if the data obeys a normal distribution, under the 3-sigma principle, an abnormal value is defined as a value that deviates from the mean by more than three times the standard deviation in a set of measured values), IQR (an abnormal value is generally defined as a value smaller than Q)Ll.5IQR or greater than QuValue of +1.5IQR, QLCalled the lower quartile, QuCalled upper quartile, IQR called quartile spacing, is QuUpper quartile sum QLThe difference between the lower quartile, which includes half of the total observed value. ) And analyzing whether the data are abnormal or not by using a statistical method.
The overall distribution of the data can be observed by a drawing mode, such as a histogram and a frequency chart, the shape and the distribution trend of the data distribution.
The statistic analysis is descriptive statistics on data, and can analyze data conditions such as variable value range, deviation degree and the like.
The most common statistics are the maximum and minimum, which are used to determine whether the value of this variable is outside a reasonable range. Statistics such as mean, mode, quantile, standard deviation, variance, etc. may also be calculated.
The correlation analysis is used for analyzing the strength of linear correlation degree between two continuous variables and can be analyzed by calculating a Pearson correlation coefficient. The formula is as follows:
Figure BDA0003378506590000101
wherein E () is desired, μXDesired E (X), μ for XYRepresenting the desired E (Y) of Y, the pearson correlation coefficient ρ of two consecutive variables (X, Y)X,YEqual to the product of the covariance cov (X, Y) between them divided by their respective standard deviations (σ)XY)。
Example III,
On the basis of the above embodiment, the present embodiment may further include the following:
as shown in fig. 2, the preprocessing of the second operation state data set in step S105 may include: data cleansing, data standardization, data reduction and data discretization.
The pretreatment process in this embodiment is further described below to facilitate better understanding of the pretreatment process by those skilled in the art.
The data normalization of the present embodiment may include:
Figure BDA0003378506590000102
wherein XjFor the values of the data objects in the original data set,
Figure BDA0003378506590000103
value, X, normalized for the data objectmaxIs the maximum value, X, in the data objectminIs the minimum value.
The data reduction processing is applied to the data after the data normalization, which may specifically include:
Figure BDA0003378506590000104
wherein
Figure BDA0003378506590000105
The data object is subjected to data reduction to obtain a value, and the original data set with the original length of N is finally converted into the data set with the length of N in a segmented and aggregated manner
Figure BDA0003378506590000111
The data discretization of the reduced data may specifically include:
Figure BDA0003378506590000112
wherein,
Figure BDA0003378506590000113
the data object is subjected to data discretization, and finally, a time-series data set is converted into a character string set { u, l, d } according to the data change trend, wherein t is a fluctuation threshold of the data.
The data cleaning of the present embodiment adopts methods of filling missing values, replacing invalid data, removing noise in data, and the like, and in order to make clear the difference between the data cleaning and the quality exploration and feature analysis performed by the present embodiment, the difference is further described. The quality exploration and analysis of data of the present embodiment mainly focuses on the discovery of dirty data, and particularly, the main task is to check whether dirty data exists in original data, and the dirty data generally refers to data which is not qualified and cannot be directly analyzed correspondingly. While data cleansing is the correction or discarding of such dirty data.
Example four,
On the basis of the above embodiment, the present embodiment may further include the following:
in this embodiment, the performing the similarity measurement on the preprocessed third operation state data set may specifically include, in step S107:
and according to the third running state data set and aiming at different index data of the working condition information, acquiring a template subsequence in the period of hours, days, weeks and months respectively, calculating the similar distance between the current time subsequence of the same index data and the template subsequence, and predicting the data change in the future time.
For the convenience of understanding the content of step S107 in the present embodiment, some of the content will be described in detail. The index data of the operation state of the service system in this embodiment includes a CPU usage rate, a memory usage rate, a disk usage rate, an access amount, and the like of the operation of the service system.
In this embodiment, when acquiring a template subsequence, random selection is performed based on the template sequence, a subsequence with a period of hours, days, weeks, and months is selected, and the randomly selected sequence is used as the template subsequence.
Example V,
On the basis of the above embodiment, the present embodiment may further include the following:
in this embodiment, in addition to performing the similarity measurement on the third operation state data set, the third operation state data set needs to be subjected to the pattern mining process, specifically, as shown in fig. 3, step S109 of this embodiment may include:
step S1091, mining a frequent mode of the business system for operating each index data according to the third operation state data set;
step S1093, mining the frequent mode of the business system running different index data according to the frequent mode of each index data;
and S1095, generating an association rule of a third operation state data set according to the frequent mode among different index data.
The pattern mining algorithm mainly includes Apriori, FP-tres, and the like, and in the present embodiment, it is preferable to perform pattern mining using Apriori algorithm. For step S1091, the specific method may include:
firstly, the maximum length of the mode is set to be k, and the minimum support threshold of the frequent mode is set to be min.
And mining a frequent pattern of the preprocessed third running state data set by using an association analysis Apriori algorithm, firstly, obtaining a frequent pattern with the length of 2 by connecting every two of the frequent patterns based on a sequence frequent pattern with the size of 1, namely, the frequent pattern is composed of symbols u, l and d in a character string set { u, l and d }, obtaining the frequency of the frequent pattern by completely scanning the discretized data set once, namely, obtaining the frequency of the frequent pattern, namely, the number of times of the support count, and recording a position list of the pattern in the data set, wherein the position list comprises an initial position and an end position of the pattern, filtering the frequent pattern, and reserving the frequent pattern which is greater than a minimum support threshold value min to obtain the frequent pattern set with the length of 2. Repeating the above process to obtain a length-3 frequent pattern set, a position list of the pattern occurrence, a length-3 frequent pattern set, …, a length-k pattern set, a position list of the pattern occurrence, and a length-k frequent pattern set.
For step S1093, the embodiment performs pattern mining on the service system running between different index data by using Apriori association analysis algorithm. The specific method comprises the following steps:
first, the maximum length of the mode is set to k2, the minimum support threshold of the frequent mode is set to min2, and the maximum support threshold of the frequent mode is set to max 2.
Randomly selecting a certain index data operated by a service system, starting a frequent mode 1 in the sequence, connecting the frequent modes in the frequent mode in the sequence with other sequences in pairs to generate a candidate frequent mode with the length of 2 between the sequences; generating a position list of the candidate frequent patterns with the length of 2 among the sequences through the position list of the candidate frequent patterns in the sequences, judging whether the length of the position list is between the minimum support degree min2 and the maximum support degree max2, and if so, adding the candidate frequent patterns with the length of 2 among the sequences into a frequent pattern set among the sequences; otherwise, deleting the inter-sequence candidate frequent pattern with the length of 2; by analogy, two frequent patterns with the length of k2-1 are used for generating the candidate frequent patterns with the length of k2 among the sequences; generating a new position list through the position lists of the two candidate frequent patterns in the sequence, judging whether the length of the new position list is between the minimum support metric min2 and the maximum support metric max2, and if so, adding the candidate frequent patterns with the length of k2 between the sequences into the inter-sequence frequent pattern set; otherwise, deleting the inter-sequence candidate frequent patterns with the length of k2 until the lengths of the frequent patterns in the inter-sequence frequent pattern set reach the set maximum length of k 2.
For step S1095, the method may include:
firstly, setting a minimum confidence threshold of a rule to be min3, generating an association rule of a service system running state data set according to a frequency mode between the excavated different index data, and screening the generated association rule according to the set minimum confidence threshold min3 to obtain a final association rule.
Example six,
On the basis of the above embodiment, the present embodiment may further include the following:
in this embodiment, the generating the first combined model based on the similarity distance and the data association rule in step S111 may specifically include:
generating a first combination model in a weighting mode based on the similarity distance and the prediction result of the association rule;
and according to the experimental result, adjusting parameters and optimizing the first combined model, adjusting the similarity distance and the weight of the output result of the data association rule, and optimizing the first combined model.
In this embodiment, after obtaining the optimized first combination model, the step S113 is adopted to perform real-time monitoring and early warning, which may specifically include:
calculating by the similar distance between the template subsequence and the current time sequence, predicting the change trend of the future set time data by taking the real sequence data as experimental data, and judging that the change trend is abnormal if the change trend exceeds a certain threshold value.
According to the association rule among the mined data, the change rule of another data can be obtained according to a certain data, and when the real data trend does not accord with the predicted data change rule, the abnormal data is judged.
And combining results between the single sequence and the data sequence, and comprehensively judging the abnormal possibility of the single sequence and the data sequence so as to carry out monitoring and early warning.
In the embodiment, after data analysis and processing, various index data of the operation state data of the service system can be displayed on one graph to perform comprehensive dynamic monitoring, so that managers can conveniently master the operation state of the data center equipment.
Example seven,
As shown in fig. 4, an embodiment of the present invention further provides a service system operation state monitoring and early warning system, which includes:
the data acquisition module is used for acquiring the working condition information of the running state of the service system to be detected and storing the working condition information as a first running state data set;
the data exploration module is used for carrying out quality exploration and characteristic analysis on the first running state data set to obtain a second running state data set;
the data preprocessing module is used for preprocessing the second running state data set to obtain a third running state data set;
the sequence matching module is used for carrying out similarity measurement on the third running state data set to obtain the similarity distance of the running state data;
the pattern mining module is used for carrying out pattern mining processing on the third running state data set to obtain a data association rule;
the model optimization module generates a first combination model based on the similarity distance and the data association rule;
and the monitoring and early warning module is used for carrying out real-time monitoring and early warning based on the first combined model prediction data change trend.
Example eight,
The disclosed embodiments provide a non-volatile computer storage medium having stored thereon computer-executable instructions that may perform the method steps of the above embodiments.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local Area Network (AN) or a Wide Area Network (WAN), or the connection may be made to AN external computer (for example, through the internet using AN internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of an element does not in some cases constitute a limitation on the element itself.
The foregoing describes preferred embodiments of the present invention, and is intended to provide a clear and concise description of the spirit and scope of the invention, and not to limit the same, but to include all modifications, substitutions, and alterations falling within the spirit and scope of the invention as defined by the appended claims.

Claims (10)

1.一种业务系统运行状态的监控预警方法,其特征在于,包括以下步骤:1. a monitoring and early warning method for the operating state of a business system, characterized in that, comprising the following steps: 步骤S101、采集待检测的业务系统的运行状态的工况信息,并存储为第一运行状态数据集;Step S101, collecting the operating condition information of the operating state of the business system to be detected, and storing it as a first operating state data set; 步骤S103、对所述第一运行状态数据集进行质量探索和特征分析,得到第二运行状态数据集;Step S103, performing quality exploration and feature analysis on the first operating state data set to obtain a second operating state data set; 步骤S105、对所述第二运行状态数据集进行预处理,得到第三运行状态数据集;Step S105, preprocessing the second operating state data set to obtain a third operating state data set; 步骤S107、对所述第三运行状态数据集进行相似性度量,获取所述运行状态数据的相似性距离;Step S107, performing similarity measurement on the third operating state data set to obtain the similarity distance of the operating state data; 步骤S109、对所述第三运行状态数据集进行模式挖掘处理,得到数据关联规则;Step S109, performing pattern mining processing on the third operating state data set to obtain data association rules; 步骤S111、基于所述相似性距离以及数据关联规则,生成第一组合模型;Step S111, generating a first combined model based on the similarity distance and the data association rule; 步骤S113、基于所述第一组合模型预测数据变化趋势,进行实时监控预警。Step S113: Predict the data change trend based on the first combined model, and perform real-time monitoring and early warning. 2.如权利要求1所述方法,其特征在于,所述步骤S103中所述质量探索和特征分析至少包括数据缺失、异常值分析、数据的整体分布情况、统计量分析和相关性分析。2 . The method according to claim 1 , wherein the quality exploration and feature analysis in the step S103 at least include data missing, outlier analysis, overall data distribution, statistical analysis, and correlation analysis. 3 . 3.如权利要求1所述方法,其特征在于,所述步骤S105中所述预处理包括:数据清洗、数据标准化、数据归约和数据离散化。3. The method according to claim 1, wherein the preprocessing in step S105 comprises: data cleaning, data standardization, data reduction and data discretization. 4.如权利要求3所述方法,其特征在于,所述数据标准化包括:4. The method of claim 3, wherein the data normalization comprises:
Figure FDA0003378506580000011
Figure FDA0003378506580000011
其中,Xj为原始数据集中的数据对象的值,
Figure FDA0003378506580000012
为该数据对象标准化后的值,Xmax为该数据对象中的最大值,Xmin为该数据对象中的最小值。
where X j is the value of the data object in the original dataset,
Figure FDA0003378506580000012
is the normalized value of the data object, X max is the maximum value in the data object, and X min is the minimum value in the data object.
5.如权利要求4所述方法,其特征在于,所述数据归约具体包括:5. The method of claim 4, wherein the data reduction specifically comprises:
Figure FDA0003378506580000021
Figure FDA0003378506580000021
其中,
Figure FDA0003378506580000022
是数据对象经过数据归约之后的值,最终将原长度为n的原始数据集分段聚合转化为长度为N的数据集
Figure FDA0003378506580000023
in,
Figure FDA0003378506580000022
It is the value of the data object after data reduction, and finally the original data set of length n is segmented and aggregated into a data set of length N
Figure FDA0003378506580000023
6.如权利要求5所述方法,其特征在于,所述数据离散化包括:6. The method of claim 5, wherein the data discretization comprises:
Figure FDA0003378506580000024
Figure FDA0003378506580000024
其中,
Figure FDA0003378506580000025
是该数据对象经过数据离散化之后的值,最终根据其数据变化趋势,将时间序列的数据集转化为字符串集合{u,l,d},其中t为数据的波动阈值。
in,
Figure FDA0003378506580000025
is the value of the data object after data discretization, and finally converts the time series data set into a string set {u,l,d} according to its data change trend, where t is the data fluctuation threshold.
7.如权利要求6所述方法,其特征在于,所述步骤S107具体包括:7. The method according to claim 6, wherein the step S107 specifically comprises: 根据所述第三运行状态数据集,并针对工况信息的不同指标数据,分别以小时、天、周、月为周期进行获取一个模板子序列,计算同一指标数据的当前时刻子序列和所述模板子序列之间的相似距离,预测未来时间内的数据变化。According to the third operating state data set, and for different indicator data of the working condition information, obtain a template subsequence in a cycle of hours, days, weeks, and months, respectively, and calculate the current moment subsequence of the same indicator data and the said subsequence. Similarity distances between template subsequences to predict data changes over time in the future. 8.如权利要求7所述方法,其特征在于,所述步骤S109具体包括:8. The method of claim 7, wherein the step S109 specifically comprises: 步骤S1091、根据所述第三运行状态数据集,挖掘业务系统运行每条指标数据的频繁模式;Step S1091, according to the third operating state data set, mining the frequent patterns of each indicator data in the operation of the business system; 步骤S1093、根据每条指标数据的频繁模式,挖掘业务系统运行不同指标数据之间的频繁模式;Step S1093, according to the frequent pattern of each index data, mining the frequent pattern between different index data in the operation of the business system; 步骤S1095、根据所述不同指标数据之间的频繁模式生成所述第三运行状态数据集的关联规则。Step S1095 , generating an association rule of the third operating state data set according to the frequent patterns between the different index data. 9.如权利要求8所述方法,其特征在于,所述步骤S111具体包括:9. The method of claim 8, wherein the step S111 specifically comprises: 基于相似性距离和关联规则的预测结果采取加权的方式生成所述第一组合模型。The first combined model is generated in a weighted manner based on the similarity distance and the prediction result of the association rule. 10.一种业务系统运行状态监控预警系统,其特征在于,包括:10. A business system operation state monitoring and early warning system, characterized in that, comprising: 数据采集模块,采集待检测的业务系统的运行状态的工况信息,并存储为第一运行状态数据集;a data acquisition module, which collects the operating condition information of the operating state of the business system to be detected, and stores it as a first operating state data set; 数据探索模块,对第一运行状态数据集进行质量探索和特征分析,得到第二运行状态数据集;The data exploration module performs quality exploration and feature analysis on the first operating state data set to obtain the second operating state data set; 数据预处理模块,对第二运行状态数据集进行预处理,得到第三运行状态数据集;a data preprocessing module, which preprocesses the second operating state data set to obtain a third operating state data set; 序列匹配模块,对第三运行状态数据集进行相似性度量,获取运行状态数据的相似性距离;The sequence matching module performs similarity measurement on the third operating state data set, and obtains the similarity distance of the operating state data; 模式挖掘模块,对第三运行状态数据集进行模式挖掘处理,得到数据关联规则;a pattern mining module, which performs pattern mining processing on the third running state data set to obtain data association rules; 模型优化模块,基于相似性距离以及数据关联规则,生成第一组合模型;The model optimization module generates the first combined model based on the similarity distance and data association rules; 监控预警模块,基于第一组合模型预测数据变化趋势,进行实时监控预警。The monitoring and early warning module predicts the trend of data changes based on the first combination model, and performs real-time monitoring and early warning.
CN202111424577.1A 2021-11-26 2021-11-26 A business system operation status monitoring and early warning method and system Active CN114168409B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111424577.1A CN114168409B (en) 2021-11-26 2021-11-26 A business system operation status monitoring and early warning method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111424577.1A CN114168409B (en) 2021-11-26 2021-11-26 A business system operation status monitoring and early warning method and system

Publications (2)

Publication Number Publication Date
CN114168409A true CN114168409A (en) 2022-03-11
CN114168409B CN114168409B (en) 2025-10-28

Family

ID=80481395

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111424577.1A Active CN114168409B (en) 2021-11-26 2021-11-26 A business system operation status monitoring and early warning method and system

Country Status (1)

Country Link
CN (1) CN114168409B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114827537A (en) * 2022-04-08 2022-07-29 浙江卡易智慧医疗科技有限公司 Image cloud data monitoring method and system based on time sequence
CN115098575A (en) * 2022-06-29 2022-09-23 戴子威 Visual expression of big data change early warning ball array and system and method for change early warning analysis
CN115129700A (en) * 2022-06-29 2022-09-30 戴子威 Big data change early warning visual expression series map and change early warning analysis system
CN117131110A (en) * 2023-10-27 2023-11-28 南京中鑫智电科技有限公司 A capacitive equipment dielectric loss monitoring method and system based on correlation analysis
CN119293831A (en) * 2024-12-13 2025-01-10 青岛他坦科技服务有限公司 Industrial Internet data processing method and system based on big data

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030009467A1 (en) * 2000-09-20 2003-01-09 Perrizo William K. System and method for organizing, compressing and structuring data for data mining readiness
US20040086180A1 (en) * 2002-11-01 2004-05-06 Ajay Divakaran Pattern discovery in video content using association rules on multiple sets of labels
CN106445788A (en) * 2016-09-30 2017-02-22 国家电网公司 Method and device for predicting operating state of information system
CN109830303A (en) * 2019-02-01 2019-05-31 上海众恒信息产业股份有限公司 Clinical data mining analysis and aid decision-making method based on internet integration medical platform
CN110008253A (en) * 2019-03-28 2019-07-12 浙江大学 The industrial data association rule mining and unusual service condition prediction technique of strategy are generated based on two stages frequent item set
US20210208995A1 (en) * 2020-01-06 2021-07-08 EMC IP Holding Company LLC Facilitating detection of anomalies in data center telemetry

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030009467A1 (en) * 2000-09-20 2003-01-09 Perrizo William K. System and method for organizing, compressing and structuring data for data mining readiness
US20040086180A1 (en) * 2002-11-01 2004-05-06 Ajay Divakaran Pattern discovery in video content using association rules on multiple sets of labels
CN106445788A (en) * 2016-09-30 2017-02-22 国家电网公司 Method and device for predicting operating state of information system
CN109830303A (en) * 2019-02-01 2019-05-31 上海众恒信息产业股份有限公司 Clinical data mining analysis and aid decision-making method based on internet integration medical platform
CN110008253A (en) * 2019-03-28 2019-07-12 浙江大学 The industrial data association rule mining and unusual service condition prediction technique of strategy are generated based on two stages frequent item set
US20210208995A1 (en) * 2020-01-06 2021-07-08 EMC IP Holding Company LLC Facilitating detection of anomalies in data center telemetry

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
赵明明;司红星;刘潮: "基于数据挖掘与关联分析的工控设备异常运行状态自动化检测方法分析", 信息安全与通信保密, no. 004, 31 December 2022 (2022-12-31), pages 2 - 10 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114827537A (en) * 2022-04-08 2022-07-29 浙江卡易智慧医疗科技有限公司 Image cloud data monitoring method and system based on time sequence
CN115098575A (en) * 2022-06-29 2022-09-23 戴子威 Visual expression of big data change early warning ball array and system and method for change early warning analysis
CN115129700A (en) * 2022-06-29 2022-09-30 戴子威 Big data change early warning visual expression series map and change early warning analysis system
CN117131110A (en) * 2023-10-27 2023-11-28 南京中鑫智电科技有限公司 A capacitive equipment dielectric loss monitoring method and system based on correlation analysis
CN117131110B (en) * 2023-10-27 2024-01-23 南京中鑫智电科技有限公司 A capacitive equipment dielectric loss monitoring method and system based on correlation analysis
CN119293831A (en) * 2024-12-13 2025-01-10 青岛他坦科技服务有限公司 Industrial Internet data processing method and system based on big data

Also Published As

Publication number Publication date
CN114168409B (en) 2025-10-28

Similar Documents

Publication Publication Date Title
CN114168409B (en) A business system operation status monitoring and early warning method and system
CN118761745B (en) OA collaborative workflow optimization method applied to enterprise
CN119151297B (en) A project execution risk assessment system and method based on data mining
KR101960755B1 (en) Method and apparatus of generating unacquired power data
CN118780719A (en) A material management method and system for power grid engineering projects based on the Internet of Things
CN115766513A (en) Anomaly detection method and device
CN120763808A (en) A method and system for predicting inverter faults based on machine learning
CN117952318A (en) Industrial park carbon emission data management system and method based on big data
CN118446671A (en) Integrated intelligent operation and maintenance control method and system
CN120197957A (en) An abnormal behavior analysis system for data centers based on artificial intelligence
CN117150032B (en) Intelligent maintenance system and method for generator sets of hydropower station
CN115907716A (en) An automated operation and maintenance management platform
CN118708395B (en) Super equipment fault detection method and system based on multi-dimensional data analysis
CN120804651A (en) Real-time monitoring method and system for digital factory
CN116956174B (en) Classification model for cold head state classification detection and life prediction and generation method of prediction model
US11915180B2 (en) Systems and methods for identifying an officer at risk of an adverse event
Senevirathne et al. Early Disease Outbreak Detection in Spatio-Temporal Data Using Predictive Modeling and Extreme Value Theory
CN120430661A (en) A power outage event analysis, decision-making and command system
CN119397204A (en) A fire alarm prediction method and device based on high-order Markov chain
CN119473798A (en) A method and device for risk mining in batch program operation
CN119782086A (en) An intelligent operation and maintenance monitoring system and method based on machine learning optimization
CN119442088A (en) Anomaly detection method, device, computer equipment and storage medium
CN121256198A (en) Methods, devices, equipment, and media for mine safety monitoring and early warning based on big data.
CN119496286A (en) A virtual duty system for electric power and new energy AI model and voice recognition
CN120234531A (en) Park management service method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant