[go: up one dir, main page]

CN111162949A - An Interface Monitoring Method Based on Java Bytecode Embedding Technology - Google Patents

An Interface Monitoring Method Based on Java Bytecode Embedding Technology Download PDF

Info

Publication number
CN111162949A
CN111162949A CN201911414793.0A CN201911414793A CN111162949A CN 111162949 A CN111162949 A CN 111162949A CN 201911414793 A CN201911414793 A CN 201911414793A CN 111162949 A CN111162949 A CN 111162949A
Authority
CN
China
Prior art keywords
data
interface
information
method based
key
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911414793.0A
Other languages
Chinese (zh)
Inventor
万雪枫
安毅
禹宁
段敬
谷良
刘海涛
段婕
姚思蓓
顾玮
刘兵兵
王艳花
张栋
杨芬
张淑娟
李�杰
王迪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Information and Telecommunication Branch of State Grid Shanxi Electric Power Co Ltd
Original Assignee
Information and Telecommunication Branch of State Grid Shanxi Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Information and Telecommunication Branch of State Grid Shanxi Electric Power Co Ltd filed Critical Information and Telecommunication Branch of State Grid Shanxi Electric Power Co Ltd
Priority to CN201911414793.0A priority Critical patent/CN111162949A/en
Publication of CN111162949A publication Critical patent/CN111162949A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06—Management of faults, events, alarms or notifications
    • H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/004—Artificial life, i.e. computing arrangements simulating life
    • G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06—Management of faults, events, alarms or notifications
    • H04L41/0677—Localisation of faults
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Databases & Information Systems (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

本发明属于字节码嵌入技术的接口监测技术领域，特别涉及一种基于Java字节码嵌入技术的接口监测方法。本发明通过采集分析运行日志和获取信息系统的访问关系；镜像网络关键节点，分析统计数据包中的所有通讯；统计和分析数据，依据数据类型进行业务建模；智能监测系统间接口状态；分析业务模型的原始数据,获取各类业务的智能基线并分类异常告警；监控数据传输链路通道；采集链接层数据和诊断故障，做到代码级别的监测效果，提高了监控质量和便于运营者实时调整和修复系统。

Figure 201911414793

The invention belongs to the technical field of interface monitoring of bytecode embedding technology, and particularly relates to an interface monitoring method based on Java bytecode embedding technology. The invention collects and analyzes the operation log and obtains the access relationship of the information system; mirrors the key nodes of the network, analyzes all communications in the statistical data package; counts and analyzes the data, and conducts business modeling according to the data type; intelligently monitors the interface state between the systems; The raw data of the business model, obtain intelligent baselines of various businesses and classify abnormal alarms; monitor data transmission link channels; collect link layer data and diagnose faults, achieve code-level monitoring effects, improve monitoring quality and facilitate operators in real-time Tweaks and fixes the system.

Figure 201911414793

Description

Interface monitoring method based on Java byte code embedding technology
Technical Field
The invention belongs to the technical field of interface monitoring of byte code embedding technology, and particularly relates to an interface monitoring method based on Java byte code embedding technology.
Background
With the vigorous development of internet technology, large-scale enterprises are difficult to independently and independently select and cooperate with each other, and are mutually beneficial and mutually beneficial in the technical level. In the cooperation process, the construction of the platform and the system is not free from the overlapping of resources and the intersection of information, so that a plurality of interfaces which are not reported or have no service interaction exist, the interfaces occupy a large amount of resources, the operation cost is increased sharply, and the risk probability of platform breakdown is improved. These interfaces tend to be outside the monitoring range and belong to an unsupervised zone. The operation and maintenance personnel are difficult to monitor the running state of the integrated interface, and when the system has errors, the problems that the reason of the errors cannot be found out at any time and the like can be caused. The interface which is in the edge range for a long time is often forgotten by operation and maintenance personnel. For example, what the role of each interface is, what the interface calling rule is, what the interface triggering condition is, how frequently the interface transfers data, what the interface operation success rate is, what the interface execution time is, and the like. Under the condition that operation and maintenance personnel do not know the information, the integrated services are difficult to be effectively and regularly managed and deployed. The current common information system interface monitoring mode is network packet capturing analysis, and application network data is collected and analyzed in a bypass packet capturing mode; or acquiring and analyzing the operation log, and acquiring the access relation of the information system through the acquisition and analysis of the operation log. The two modes can cause the acquisition of excessive information, more filtering is needed to find out really useful information, data real-time interaction is difficult to carry out, and the monitoring timeliness is poor. Aiming at the technical background, the embedded technology based on the Bytecode is provided, Java agents and Java Bytecode technologies are used, the Java agents are deployed in an information system server, Jar packets called by interfaces between systems are introduced into a JVM running a monitoring instance, when a Class file is loaded into the JVM, the Java Bytecode control technology is used for dynamically monitoring method execution, SQL/NoSQL access and application calling (API calling) in real time, the byte information of each interface Jar packet can be dynamically monitored, and accurate problem positioning at a code level is realized. The interface state between the systems can be intelligently monitored, and early warning can be given to abnormal conditions.
Disclosure of Invention
In order to solve the problems of resource occupation and waste and intelligent monitoring management of interfaces which are not reported and have no service interaction existing in mutual cooperation of large enterprises, an interface monitoring method based on Java byte code embedding technology is provided.
In order to achieve the purpose, the invention adopts the following technical scheme:
an interface monitoring method based on Java byte code embedding technology comprises the following steps: comprises the following steps:
step 1, collecting and analyzing an operation log and acquiring an access relation of an information system;
step 2, mirroring the network key nodes, analyzing and counting all communications in the data packet, and determining the data forwarding relation and the flow load of the key routing nodes;
step 3, constructing a machine learning data source, training a neural network model, determining parameters of the machine learning model, and performing business modeling according to data types;
step 4, intelligently monitoring the interface state between systems, and constructing a global topological structure of the data stream;
step 5, monitoring a data transmission link channel;
step 6, analyzing the original data of the service model, acquiring early warning threshold values of various services and classifying abnormal alarms;
and 7, collecting link layer data and diagnosing faults.
Further, in step 1, acquiring and analyzing the operation log and obtaining the access relationship of the information system, the specific method is as follows: collecting the log files of each interface by using a collection program, and classifying and sorting the log files according to the request types to obtain key information; according to the key information in the log file, the relationship between each piece of key information is induced and analyzed, and then the Neo4j database is used for drawing and storing; so as to facilitate visualization and construction of multivariate information graphs.
The key information in the log file comprises an information source, a message request object, request time and request parameters.
Further, the step 2 of mirroring the network key nodes and analyzing and counting all communications in the data packet includes the specific steps of: calculating routing nodes for information transmission in the whole network by using a routing technology, analyzing data flow and forwarding load of the nodes, and judging the key degree of the nodes; and (4) carrying out statistical analysis on all communication pairs in the data packet by using a network packet capturing means to obtain a least square fitting flow chart of each node. And finally obtaining stable forwarding relation and flow load degree between the key routing nodes and each node through continuous repeated statistical analysis.
Further, the step 3 of counting and analyzing data and performing service modeling according to data types includes the specific steps of: using the data obtained in the step 2 and the step 1 as data sources, dividing the data sources into a data source A of 7/10, a data source B of 2/10 and a data source C of 1/10 according to the proportion of 7:2:1, putting the data source A as training data into a machine learning framework for learning and training, and calculating a HuberLoss loss function L in the training processδ(y,f(x))，
Figure BDA0002350904940000031
Where δ is the hyper-parameter of HuberLoss, y is the true value, f (x) is the predicted value of the model; after training is finished, parameters of a processing frame and a machine learning network frame which basically accord with the data source can be obtained; the data source B is used for testing, and the data source C is used for verifying, so that the correctness of the parameters learned by the machine is ensured; and carrying out deepening processing on the data by using an Apriori algorithm in big data analysis, searching the internal relation among the data, and carrying out service modeling according to the requested service type so as to realize automatic discovery of interfaces among information systems and automatic generation of data transmission topology among systems.
Further, the method for intelligently monitoring the interface state between the systems in step 4 and constructing the global topology of the data stream includes: deploying Java agents on a server by using a bytecode embedding technology according to the information acquired in the steps 1 to 3, introducing jar packets called by interfaces between systems into a monitoring instance in a JVM (Java virtual machine), dynamically monitoring method execution, SQL/NoSQL access and API (application program interface) calling between applications in real time by using a bytecode manipulation technology when class files are loaded into the JVM, constructing a global topology of data flow direction, and realizing accurate problem positioning of a code level based on the topology; hash storage and real-time update are carried out on the positioning information, and macroscopic analysis is carried out on the information in all interfaces so as to obtain detailed parameters called by the interfaces between the systems;
the information in all interfaces comprises URL paths, request modes, appearance time, application versions, network connection states, interface request Body bodies, Http headers, IP addresses, error return bodies, equipment and systems and application versions.
Further, the monitoring of the data transmission link channel in step 5 includes the following specific steps: monitoring the whole data transmission link channel by adopting a method of a full data packet seven-layer protocol, decoding the seven-layer protocol, and counting and displaying the decoding content; and (4) combining the topology and interface information obtained in the steps 1-4 to realize the visual display of the data full link relation.
The decoding content comprises the whole number of OGGs, the number of OGGs corresponding to each service system, the task type, the running state and the time consumption of an OGG channel corresponding to each service system, and the number of topics, the running state, the time consumption and sessions corresponding to an information channel process.
Further, step 6 is to analyze the original data of the service model, obtain the early warning threshold values of various services and classify abnormal alarms, and the specific method is as follows: processing the monitoring information by using a mathematical statistics and probability analysis method, calculating the occurrence probability of each type of event and analyzing the occurrence distribution rule of the events; searching early warning threshold values monitored by each interface according to the distribution type and the distribution characteristics, wherein the threshold values are generated based on the specific timeliness and integrity requirements of data transmission of each interface, different requests of different information systems are different in threshold value setting, and different software is configured according to the environment to be iterated and continuously changed;
the early warning threshold value is an average value of historical data generated by an application system at the same time and on different dates, and is used as a measurement standard for measuring whether a request generated by the system in real time is abnormal or not, whether a certain index belongs to the range of the historical average value at the current time can be reflected, and if the fluctuation is large, an alarm is given to remind a user that the request under the current application is in the abnormal category of the historical time; classifying various abnormal alarms according to an early warning threshold;
the method comprises the following steps that the triggering of the alarm is divided into two parts, the first part is a statistical result from an early warning threshold value and comprises a corresponding time baseline, an access amount baseline and an error rate baseline, the second part is an isolated forest algorithm, the abnormal point is intelligently detected, the alarm is classified at the same time, and a root alarm result is output by adopting a short message gateway or mailbox butt joint mode; meanwhile, the fault is diagnosed through data link, and the granularity can reach a communication pair level and a code level.
Further, the step 7 of collecting link layer data and diagnosing faults includes the following specific steps: monitoring the data full link, outputting an alarm result according to the service threshold value in the step 6, and packaging and sending the data by adopting a byte coding technology; key nodes of information in the alarm result are mainly explored; collecting data flow information of a data link layer in a network by using the flow bandwidth and the flow rate of data, classifying and inducing, calculating a correlation coefficient, solving the correlation degree, and arranging variables of a correlation sequence and a column correlation matrix; and then, the fault position is positioned by combining the decision tree model technology and the Hash algorithm, so that the purpose of fault diagnosis is achieved.
Compared with the prior art, the invention has the following advantages:
the existing intelligent supervision technology can only monitor a service layer of a system, and the accurate positioning of an interface layer is rarely involved. The technology can achieve the monitoring effect of the code level, improve the monitoring quality and facilitate the real-time adjustment and repair of the system by operators. Meanwhile, the intelligent scoring of the application and the interface is supported, a scoring mechanism can be defined by a user, a key index is selected to set a score and a weight, and the key index necessarily comprises eight indexes such as response time, interface slow rate, interface error rate, Apdex value, interface failure number, interface program abnormal constant, interface program method execution time, interface SQL slow calling number and the like. Through the combing of the interface health state, the operation and maintenance managers at each level can quickly sense the operation health state of each interface. The interface health report content can self-define index service request quantity, response time, error rate, slow rate, minute-level service data and a key transaction data list to form an automatic template, and the automatic template is automatically sent through monthly reports, weekly reports and daily reports. The project is based on technologies such as machine learning, artificial intelligence and big data, and through dynamic data sensing, intelligent monitoring and intelligent health analysis, the quality and the effect of data operation and maintenance can be improved, the quality of shared data is improved, and operation and maintenance mode transformation is promoted. Theoretical and technical support is provided for automatic data interface combing monitoring, automatic topology finding, intelligent baseline alarming and transmission quality monitoring, panoramic visual display capacity of information system data flow is formed, manual workload of data link combing is reduced, and timely and complete data flow is ensured; the method and the device realize accurate positioning of data transmission faults, eliminate potential safety hazards timely and efficiently, and enhance the robustness of an information system. Technically, a Java bytecode class file (. class) is a "target file" generated by a Java compiler compiling a Java source file (. Java). The class file is a binary stream file with 8-bit bytes, each data item is closely arranged from front to back in sequence, and no gap exists between adjacent items, so that the class file is very compact and light in size, can be quickly loaded to a memory by a JVM (JVM) and occupies less memory space (is convenient for network transmission). After the Java source file is compiled by a Java compiler, each class (or interface) independently occupies a class file, and all information in the class has corresponding description in the class file.
Drawings
FIG. 1 is an interface logbook diagram;
FIG. 2 is an information system relational representation;
FIG. 3 is a display diagram of alarm results;
FIG. 4 is a diagnostic trouble diagram;
FIG. 5 is a dataflow diagram;
FIG. 6 is an exception handling dataflow diagram.
Detailed Description
Example 1
The scheme is applied to a full-service data center system for intelligent monitoring of the interface:
step 1, collecting log files of each interface of 22 sets of systems integrated by a service center by using a collection Agent installed in an application system server, and classifying, arranging and regularizing according to request types. And extracting key information in the log file, such as a message source, a message request object, request time and request parameters. According to the key information in the log file, the relations among all information are induced and analyzed, and the relations are drawn and stored by using a Neo4j database, so that a multivariate information graph is conveniently visualized and constructed. The interface operation log diagram of fig. 1 and the information system relationship representation diagram of fig. 2 are shown.
And 2, acquiring mirror image flow of an outlet route of the data center, calculating a route node for information transmission in the whole network, analyzing data flow and forwarding load of the node, and judging the key degree of the node. And (4) carrying out statistical analysis on all communication pairs in the data packet by using a network packet capturing means to obtain a least square fitting flow chart of each node. And finally obtaining stable key nodes and the communication relation among the nodes through continuous repeated analysis.
And 3, taking the data obtained in the step 2 and the step 1 as a data source, and enabling the data source to be in a range of 7:2:1, dividing the data into 7/10 for a data source A, 2/10 for a data source B and 1/10 for a data source C, putting the data source A as training data into a machine learning frame for learning and training, and calculating a loss function in the training process; after training is finished, parameters of a processing frame and a machine learning network frame which basically accord with the data source can be obtained; the data source B is used for testing, and the data source C is used for verifying, so that the correctness of the parameters learned by the machine is ensured; and carrying out deep processing on the data by utilizing a big data algorithm, searching the internal relation among the data, and carrying out business modeling according to the requested business type so as to analyze the mutual relation among the businesses.
And 4, deploying Java agents on the server by using a bytecode embedding technology according to the related information acquired in the steps 1 to 3, introducing a jar packet called by the intersystem interface into a monitoring instance in the JVM, dynamically monitoring method execution, SQL/NoSQL access and call among applications in real time by using a bytecode manipulation technology when the class file is loaded into the JVM, constructing a global topology of a data flow direction, realizing accurate data transmission monitoring of a code level based on the topology, and performing Hash storage and real-time updating on the monitoring information. And performing macroscopic analysis on information in all interfaces, including URL paths, request modes, occurrence time, application versions, network connection states, interface request Body, Http headers, IP addresses, error return bodies, equipment, systems and application versions. The monitoring system 22 and the interface 47 are arranged in the same example.
And 5, monitoring the whole data transmission link channel by adopting a method of a full data packet seven-layer protocol, and decoding the seven-layer protocol, wherein the seven-layer protocol comprises the whole quantity of OGGs and the quantity of OGGs corresponding to each service system. And counting and displaying the task type, the running state and the time consumption of the OGG channel corresponding to each service system, and the theme, the running state, the time consumption and the number of sessions corresponding to the information channel flow. This example monitors 6988 data sheets, where 6800 data stream transmission rates are real time and 188 data stream transmission frequencies are weeks or months.
FIG. 5 is a marketing domain system association diagram;
and 6, processing the monitoring information by using mathematical statistics and probability analysis methods, calculating the occurrence probability of each type of event and analyzing the occurrence distribution rule of the events. And searching early warning threshold values monitored by each interface according to the distribution type and the distribution characteristics, wherein the threshold values are generated based on the specific timeliness and integrity requirements of data transmission of each interface, different requests of different information systems are different in threshold value setting, and meanwhile, different software is configured according to the environment and is continuously changed in an iterative manner. The early warning threshold value is an average value of historical data generated by an application system at the same time and on different dates, and is used as a measurement standard for measuring whether a request generated by the system in real time is abnormal or not, whether a certain index belongs to the range of the historical average value at the current time can be reflected, and if the fluctuation is large, an alarm can be given to remind a user that the request under the current application is in the abnormal category of the historical time. And classifying various abnormal alarms according to the early warning threshold value. The method comprises the following steps of triggering an alarm, wherein the first part is a statistical result from an early warning threshold value and comprises a corresponding time baseline, an access amount baseline and an error rate baseline, the second part is an isolated forest algorithm, intelligently detecting an abnormal point, classifying the alarm, and outputting a root alarm result by adopting a short message gateway or mailbox butt joint mode. Meanwhile, the fault is diagnosed through data link, and the granularity can reach a communication pair level and a code level. Fig. 3 is a diagram showing an alarm result display.
And 7, monitoring the data full link, outputting an alarm result according to the service threshold value in the step 6, and packaging and sending the data by adopting a byte coding technology. Key nodes in the alarm information are intensively explored. The method comprises the steps of collecting data flow information of a data link layer in a network by using the flow bandwidth and the flow rate of data, classifying and summarizing the information, calculating a correlation coefficient, solving the correlation degree, arranging correlation sequences, arranging a correlation matrix and other variables. And then, the fault position is positioned by combining the decision tree model technology and the Hash algorithm, so that the purpose of fault diagnosis is achieved. As shown in fig. 4, a failure diagnosis diagram.
FIG. 6 is an exception handling dataflow diagram.
TABLE 1 monitoring data links
Figure BDA0002350904940000091
TABLE 2 monitoring failure types
Figure BDA0002350904940000092
Figure BDA0002350904940000101
From the above results, it can be seen that the present invention has excellent utility. No matter how the complexity and the cross complexity of the system are, the monitoring efficiency is over 99 percent, the data transmission fault between the systems is found in time, and the economic loss is avoided.
The method shortens the calculation time, thereby improving the efficiency of the algorithm and increasing the monitoring efficiency. On the problem that a plurality of systems are crossed, the method can obtain better monitoring results.

Claims (8)

1.一种基于Java字节码嵌入技术的接口监测方法，其特征在于：包含以下步骤：1. an interface monitoring method based on Java bytecode embedding technology, is characterized in that: comprise the following steps: 步骤1，采集分析运行日志和获取信息系统的访问关系；Step 1, collect and analyze the operation log and obtain the access relationship of the information system; 步骤2，镜像网络关键节点，分析统计数据包中的所有通讯，确定关键路由节点的数据转发关系和流量负荷；Step 2, mirroring the key nodes of the network, analyzing all communications in the statistical data packets, and determining the data forwarding relationship and traffic load of the key routing nodes; 步骤3，构造机器学习数据源，训练神经网络模型，确定机器学习模型参数，并依据数据类型进行业务建模；Step 3, constructing a machine learning data source, training a neural network model, determining the parameters of the machine learning model, and performing business modeling according to the data type; 步骤4，智能监测系统间接口状态，构建数据流的全局拓扑结构；Step 4, intelligently monitor the interface state between systems, and construct a global topology structure of the data flow; 步骤5，监控数据传输链路通道；Step 5, monitor the data transmission link channel; 步骤6，分析业务模型的原始数据,获取各类业务的智能基线并分类异常告警；Step 6, analyze the original data of the business model, obtain intelligent baselines of various businesses, and classify abnormal alarms; 步骤7，采集链接层数据和诊断故障。Step 7, collect link layer data and diagnose faults. 2.根据权利要求1所述的一种基于Java字节码嵌入技术的接口监测方法，其特征在于：所述步骤1中采集分析运行日志和获取信息系统的访问关系，具体方法为：利用采集程序对每个接口的日志文件进行收集，并按照请求类别进行分类整理，得出关键信息；根据日志文件中的关键信息，归纳和分析每个关键信息间的关系，然后利用Neo4j数据库进行绘制和存储；2. a kind of interface monitoring method based on Java bytecode embedding technology according to claim 1, is characterized in that: in described step 1, collect and analyze the access relation of operation log and acquisition information system, and concrete method is: utilize collection The program collects the log files of each interface, sorts them according to the request category, and obtains the key information; according to the key information in the log file, summarizes and analyzes the relationship between each key information, and then uses the Neo4j database to draw and analyze the relationship between them. storage; 所述日志文件中的关键信息包括信息源、消息请求对象、请求时间及请求参数。The key information in the log file includes information source, message request object, request time and request parameters. 3.根据权利要求1所述的一种基于Java字节码嵌入技术的接口监测方法，其特征在于：所述步骤2中镜像网络关键节点，分析统计数据包中的所有通讯，确定关键路由节点的数据转发关系和流量负荷，具体方法为：利用路由技术，计算全网中信息传递的路由节点，分析节点的数据流量和转发负荷，判定节点的关键程度；利用网络抓包手段，对数据包中所有通讯对进行分析统计，获取各个节点的最小二乘拟合流量图；经过不断重复分析，最终得到稳定的关键节点和各节点间的通讯关系和流量负荷程度。3. a kind of interface monitoring method based on Java bytecode embedding technology according to claim 1, is characterized in that: in described step 2, mirror network key node, analyze all communication in the statistical data packet, determine key routing node The specific method is: using routing technology to calculate the routing nodes for information transmission in the entire network, analyzing the data traffic and forwarding load of the nodes, and determining the criticality of the nodes; All communication pairs in the system are analyzed and counted to obtain the least squares fitting traffic graph of each node; after repeated analysis, the stable key nodes and the communication relationship and traffic load level between each node are finally obtained. 4.根据权利要求1所述的一种基于Java字节码嵌入技术的接口监测方法，其特征在于：所述步骤3中构造机器学习数据源，训练神经网络模型，确定机器学习模型参数，并依据数据类型进行业务建模，具体方法为：利用步骤2和步骤1中获得的数据作为数据源，将数据源按照7:2:1的比例进行划分，分为数据源A占7/10、数据源B占2/10、数据源C占1/10，将数据源A作为训练数据投入机器学习框架进行学习和训练，计算训练过程中HuberLoss损失函数Lδ(y,f(x))，4. a kind of interface monitoring method based on Java bytecode embedding technology according to claim 1, is characterized in that: in described step 3, construct machine learning data source, train neural network model, determine machine learning model parameter, and Carry out business modeling according to the data type. The specific method is as follows: using the data obtained in step 2 and step 1 as the data source, dividing the data source according to the ratio of 7:2:1, and dividing it into data source A accounting for 7/10, Data source B accounts for 2/10, and data source C accounts for 1/10. Data source A is used as training data to be put into the machine learning framework for learning and training, and the HuberLoss loss function Lδ(y, f(x)) during the training process is calculated,
Figure FDA0002350904930000021
Figure FDA0002350904930000021
 其中，Lδ表示损失函数，δ是HuberLoss的超参数，y是真实值，f(x)是模型的预测值；训练完成后可以得到基本符合本数据源的处理框架和机器学习网络框架的参数；利用数据源B作测试，数据源C做验证，从而确保机器学习到的参数的正确性；利用大数据分析中的Apriori算法，将这些数据进行深化处理，寻找数据间的内在联系，并根据请求的业务类型，进行业务建模，以便于实现信息系统间接口的自动发现、系统间数据传输拓扑的自动生成。Among them, L δ represents the loss function, δ is the hyperparameter of HuberLoss, y is the real value, and f(x) is the predicted value of the model; after the training is completed, the processing framework and machine learning network framework parameters that basically conform to this data source can be obtained ; Use data source B for testing and data source C for verification to ensure the correctness of the parameters learned by the machine; use the Apriori algorithm in big data analysis to deepen the processing of these data, find the intrinsic relationship between the data, and according to the The requested business type, and business modeling is carried out, so as to realize the automatic discovery of the interface between the information systems and the automatic generation of the data transmission topology between the systems. 5.根据权利要求1所述的一种基于Java字节码嵌入技术的接口监测方法，其特征在于：所述步骤4中智能监测系统间接口状态，构建数据流的全局拓扑结构，具体方法是：依据步骤1至步骤3获取的信息，利用字节码嵌入技术，在服务器部署Java Agent，将系统间接口调用的jar包引入JVM内监控实例，在class文件被加载进入JVM时，通过字节码操控技术动态对方法执行、SQL/NoSQL访问、应用间调用API进行实时监控，构建出数据流向的全局拓扑，并且基于拓扑实现代码级的精准问题定位；将定位信息进行Hash存储并实时更新，对所有接口中的信息进行宏观分析；5. a kind of interface monitoring method based on Java bytecode embedding technology according to claim 1, is characterized in that: in described step 4, the interface state between intelligent monitoring systems, builds the global topology structure of data flow, and concrete method is : According to the information obtained in steps 1 to 3, using bytecode embedding technology, deploy Java Agent on the server, and introduce the jar package called by the interface between systems into the JVM to monitor the instance, and when the class file is loaded into the JVM, the bytecode The code manipulation technology dynamically monitors method execution, SQL/NoSQL access, and API calls between applications in real time, builds a global topology of data flow, and implements code-level accurate problem location based on the topology; the location information is stored in Hash and updated in real time, Macro-analysis of information in all interfaces; 所述所有接口中的信息包括URL路径、请求方式、出现时间、应用版本、网络连接状态、接口请求Body体、Http头、IP地址、错误返回体、设备和系统、应用版本。The information in all the interfaces includes URL path, request mode, appearance time, application version, network connection status, interface request body, Http header, IP address, error return body, device and system, and application version. 6.根据权利要求1所述的一种基于Java字节码嵌入技术的接口监测方法，其特征在于：所述步骤5中监控数据传输链路通道，具体方法是：采用全量数据包七层协议的方法，对数据传输链路通道整体进行监控，对七层协议进行解码，将解码内容进行统计与展现；6. a kind of interface monitoring method based on Java bytecode embedding technology according to claim 1, is characterized in that: in described step 5, monitor data transmission link channel, concrete method is: adopt full data packet seven-layer protocol The method is to monitor the overall data transmission link channel, decode the seven-layer protocol, and count and display the decoded content; 所述解码内容包括OGG整体数量、每个业务系统对应的OGG数量，每个业务系统对应的OGG通道任务类型、运行状态、耗时，Informatica通道流程对应的主题、运行状态、耗时、Session数量。The decoded content includes the overall number of OGGs, the number of OGGs corresponding to each business system, the task type, running status, and time-consuming of the OGG channel corresponding to each business system, and the theme, running status, time-consuming, and number of sessions corresponding to the Informatica channel process. . 7.根据权利要求1所述的一种基于Java字节码嵌入技术的接口监测方法，其特征在于：所述步骤6中分析业务模型的原始数据,获取各类业务的预警阈值并分类异常告警，具体方法为：用数理统计和概率分析方法，处理监测信息，计算每类事件发生的概率和分析它们发生的分布规律；依据分布类型和分布特点，寻找各接口监测的预警阈值，阈值的生成是基于每个接口传输数据的具体及时性、完整性要求的，不同信息系统的不同请求在阀值的设定都是不同，同时又是根据环境配置不同软件迭代不断变化的；7. a kind of interface monitoring method based on Java bytecode embedding technology according to claim 1, is characterized in that: in described step 6, analyze the original data of business model, obtain the early warning threshold value of various business and classify abnormal alarm The specific methods are: use mathematical statistics and probability analysis methods to process monitoring information, calculate the probability of occurrence of each type of events and analyze the distribution law of their occurrence; according to the distribution type and distribution characteristics, find the early warning threshold for monitoring of each interface, and the generation of the threshold It is based on the specific timeliness and integrity requirements of the data transmitted by each interface. Different requests of different information systems have different threshold settings, and at the same time, they are constantly changing according to the environment configuration of different software iterations; 预警阈值是根据应用系统产生的历史数据，生成同一时刻在不同日期的历史数据的平均值，作为衡量一个系统实时发生的请求是否存在异常的度量标准，能反应出当前时刻下某一指标是否属于历史平均值范围内，如果波动较大会给出告警提醒用户当前应用下请求处于历史该时刻异常范畴；按照预警阈值，将各类异常告警进行分类；The early warning threshold is based on the historical data generated by the application system to generate the average value of the historical data at the same time and on different dates, which is used as a metric to measure whether there is an abnormality in the real-time request of a system, and can reflect whether a certain indicator at the current moment belongs to the Within the range of the historical average value, if the fluctuation is large, an alarm will be given to remind the user that the request under the current application is in the abnormal category at this time in history; according to the early warning threshold, various abnormal alarms are classified; 告警的触发分为两个部分，第一部分是来源于预警阈值的统计结果，包含相应时间基线、访问量基线及错误率基线，第二部分是来自于孤立森林算法，对异常点智能检测，同时对告警进行归类，采用短信网关或邮箱对接的方式，将根告警结果进行输出；同时通过数据链接，对故障进行诊断，颗粒度可达通信对级和代码级。The trigger of the alarm is divided into two parts. The first part is the statistical results derived from the early warning threshold, including the corresponding time baseline, traffic baseline and error rate baseline. The second part is derived from the isolated forest algorithm, which intelligently detects abnormal points. The alarms are classified, and the root alarm results are output by means of SMS gateway or mailbox connection; at the same time, faults are diagnosed through data links, and the granularity can reach the communication pair level and the code level. 8.根据权利要求1所述的一种基于Java字节码嵌入技术的接口监测方法，其特征在于：所述步骤7采集链接层数据和诊断故障，具体方法是：对数据全链路进行监测，依据步骤6中的业务阈值输出告警结果，采用字节编码技术，将数据进行打包和发送；重点试探告警结果中信息的关键节点；利用数据的流动带宽和流动速率，采集网络中数据链路层的数据流动信息，进行分类和归纳，并计算关联系数，求关联度，排关联序，列关联矩阵的变量；然后结合决策树模型技术和Hash算法定位故障位置，从而达到故障诊断目的。8. a kind of interface monitoring method based on Java bytecode embedding technology according to claim 1, is characterized in that: described step 7 collects link layer data and diagnoses fault, and concrete method is: monitor data full link , output the alarm result according to the service threshold in step 6, use byte coding technology to package and send the data; focus on testing the key nodes of the information in the alarm result; use the data flow bandwidth and flow rate to collect data links in the network The data flow information of the layer is classified and summarized, and the correlation coefficient is calculated, the correlation degree is calculated, the correlation sequence is sorted, and the variables of the correlation matrix are listed; then the decision tree model technology and Hash algorithm are combined to locate the fault location, so as to achieve the purpose of fault diagnosis.
CN201911414793.0A 2019-12-31 2019-12-31 An Interface Monitoring Method Based on Java Bytecode Embedding Technology Pending CN111162949A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911414793.0A CN111162949A (en) 2019-12-31 2019-12-31 An Interface Monitoring Method Based on Java Bytecode Embedding Technology

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911414793.0A CN111162949A (en) 2019-12-31 2019-12-31 An Interface Monitoring Method Based on Java Bytecode Embedding Technology

Publications (1)

Publication Number Publication Date
CN111162949A true CN111162949A (en) 2020-05-15

Family

ID=70560139

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911414793.0A Pending CN111162949A (en) 2019-12-31 2019-12-31 An Interface Monitoring Method Based on Java Bytecode Embedding Technology

Country Status (1)

Country Link
CN (1) CN111162949A (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111813405A (en) * 2020-06-29 2020-10-23 四川长虹电器股份有限公司 Method for rapidly updating and publishing target data interface
CN112131090A (en) * 2020-09-30 2020-12-25 北京北信源软件股份有限公司 Business system performance monitoring method and device, equipment and medium
CN112181784A (en) * 2020-10-21 2021-01-05 中国工商银行股份有限公司 Code fault analysis method and system based on bytecode injection
CN112291088A (en) * 2020-12-22 2021-01-29 全知科技(杭州)有限责任公司 Method for automatically combing and classifying Web interfaces
CN112346393A (en) * 2021-01-08 2021-02-09 睿至科技集团有限公司 Method and system for abnormal monitoring and processing of data full-link based on intelligent operation and maintenance
CN112491601A (en) * 2020-11-16 2021-03-12 北京字节跳动网络技术有限公司 Traffic topology generation method and device, storage medium and electronic equipment
CN112817834A (en) * 2021-01-22 2021-05-18 上海哔哩哔哩科技有限公司 Data table evaluation method and device
CN112925489A (en) * 2021-03-15 2021-06-08 中国平安财产保险股份有限公司 Method and device for printing log and computer equipment
CN113762688A (en) * 2021-01-06 2021-12-07 北京沃东天骏信息技术有限公司 Business analysis system, method and storage medium
CN114385387A (en) * 2022-03-23 2022-04-22 恒生电子股份有限公司 Access information generation method and device and computer equipment
CN114693109A (en) * 2022-03-29 2022-07-01 中国工商银行股份有限公司 Application transformation method and related device
CN114928550A (en) * 2022-04-29 2022-08-19 杭州默安科技有限公司 Database interaction account security detection method and system
CN116346590A (en) * 2023-05-30 2023-06-27 国网汇通金财（北京）信息科技有限公司 A Locating System for Full Link Faults

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102122263A (en) * 2011-03-23 2011-07-13 成都勤智数码科技有限公司 A JAVA application system runtime monitoring method and device
CN102622289A (en) * 2011-01-26 2012-08-01 阿里巴巴集团控股有限公司 Program setting method, program monitoring method, and devices and systems for program setting and monitoring
CN107561997A (en) * 2017-08-22 2018-01-09 电子科技大学 A kind of power equipment state monitoring method based on big data decision tree
CN107634848A (en) * 2017-08-07 2018-01-26 上海天旦网络科技发展有限公司 A kind of system and method for collection analysis network equipment information
CN107730087A (en) * 2017-09-20 2018-02-23 平安科技（深圳）有限公司 Forecast model training method, data monitoring method, device, equipment and medium
CN110083436A (en) * 2019-05-14 2019-08-02 上海理想信息产业（集团）有限公司 A kind of business datum real-time monitoring system and method based on Java bytecode enhancing technology
CN110213198A (en) * 2018-02-28 2019-09-06 中标软件有限公司 The monitoring method and system of network flow
CN110505179A (en) * 2018-05-17 2019-11-26 中国科学院声学研究所 Method and system for detecting abnormal network traffic

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102622289A (en) * 2011-01-26 2012-08-01 阿里巴巴集团控股有限公司 Program setting method, program monitoring method, and devices and systems for program setting and monitoring
CN102122263A (en) * 2011-03-23 2011-07-13 成都勤智数码科技有限公司 A JAVA application system runtime monitoring method and device
CN107634848A (en) * 2017-08-07 2018-01-26 上海天旦网络科技发展有限公司 A kind of system and method for collection analysis network equipment information
CN107561997A (en) * 2017-08-22 2018-01-09 电子科技大学 A kind of power equipment state monitoring method based on big data decision tree
CN107730087A (en) * 2017-09-20 2018-02-23 平安科技（深圳）有限公司 Forecast model training method, data monitoring method, device, equipment and medium
CN110213198A (en) * 2018-02-28 2019-09-06 中标软件有限公司 The monitoring method and system of network flow
CN110505179A (en) * 2018-05-17 2019-11-26 中国科学院声学研究所 Method and system for detecting abnormal network traffic
CN110083436A (en) * 2019-05-14 2019-08-02 上海理想信息产业（集团）有限公司 A kind of business datum real-time monitoring system and method based on Java bytecode enhancing technology

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
A. CHANDER等: "Mobile code security by Java bytecode instrumentation", 《 PROCEEDINGS DARPA INFORMATION SURVIVABILITY CONFERENCE AND EXPOSITION II. DISCEX"01》 *
李振东: "基于字节码的软件监控及可信演化框架设计与实现", 《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》 *
陈瑞等: "基于端到端业务监控的运维能力提升研究", 《信息通信》 *

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111813405A (en) * 2020-06-29 2020-10-23 四川长虹电器股份有限公司 Method for rapidly updating and publishing target data interface
CN112131090A (en) * 2020-09-30 2020-12-25 北京北信源软件股份有限公司 Business system performance monitoring method and device, equipment and medium
CN112131090B (en) * 2020-09-30 2024-02-23 北京北信源软件股份有限公司 Service system performance monitoring method, device, equipment and medium
CN112181784A (en) * 2020-10-21 2021-01-05 中国工商银行股份有限公司 Code fault analysis method and system based on bytecode injection
CN112181784B (en) * 2020-10-21 2024-03-26 中国工商银行股份有限公司 Code fault analysis method and system based on byte code injection
CN112491601B (en) * 2020-11-16 2022-08-30 北京字节跳动网络技术有限公司 Traffic topology generation method and device, storage medium and electronic equipment
CN112491601A (en) * 2020-11-16 2021-03-12 北京字节跳动网络技术有限公司 Traffic topology generation method and device, storage medium and electronic equipment
CN112291088A (en) * 2020-12-22 2021-01-29 全知科技(杭州)有限责任公司 Method for automatically combing and classifying Web interfaces
CN113762688A (en) * 2021-01-06 2021-12-07 北京沃东天骏信息技术有限公司 Business analysis system, method and storage medium
CN112346393A (en) * 2021-01-08 2021-02-09 睿至科技集团有限公司 Method and system for abnormal monitoring and processing of data full-link based on intelligent operation and maintenance
CN112346393B (en) * 2021-01-08 2021-04-13 睿至科技集团有限公司 Method and system for abnormal monitoring and processing of data full-link based on intelligent operation and maintenance
CN112817834A (en) * 2021-01-22 2021-05-18 上海哔哩哔哩科技有限公司 Data table evaluation method and device
CN112925489B (en) * 2021-03-15 2023-02-03 中国平安财产保险股份有限公司 Method and device for printing log and computer equipment
CN112925489A (en) * 2021-03-15 2021-06-08 中国平安财产保险股份有限公司 Method and device for printing log and computer equipment
CN114385387B (en) * 2022-03-23 2022-07-19 恒生电子股份有限公司 Access information generation method and device and computer equipment
CN114385387A (en) * 2022-03-23 2022-04-22 恒生电子股份有限公司 Access information generation method and device and computer equipment
CN114693109A (en) * 2022-03-29 2022-07-01 中国工商银行股份有限公司 Application transformation method and related device
CN114928550A (en) * 2022-04-29 2022-08-19 杭州默安科技有限公司 Database interaction account security detection method and system
CN114928550B (en) * 2022-04-29 2024-05-03 杭州默安科技有限公司 A database interactive account security detection method and system
CN116346590A (en) * 2023-05-30 2023-06-27 国网汇通金财（北京）信息科技有限公司 A Locating System for Full Link Faults

Similar Documents

Publication Publication Date Title
CN111162949A (en) An Interface Monitoring Method Based on Java Bytecode Embedding Technology
CN105721187B (en) A kind of traffic failure diagnostic method and device
CN108197261A (en) A kind of wisdom traffic operating system
CN109783322A (en) A kind of monitoring analysis system and its method of enterprise information system operating status
CN109800127A (en) A kind of system fault diagnosis intelligence O&M method and system based on machine learning
CN111585840B (en) Service resource monitoring method, device and equipment
CN106844171A (en) A method for realizing mass operation and maintenance
CN113157521B (en) Monitoring method and monitoring system for block chain full life cycle
CN115378711B (en) Intrusion detection method and system for industrial control network
CN119325104A (en) Link reliability tracking and optimizing method combining real-time monitoring
CN102333007A (en) On-line Web service quality monitoring system and method
CN104574219A (en) System and method for monitoring and early warning of operation conditions of power grid service information system
CN114531338A (en) Monitoring alarm and tracing method and system based on call chain data
CN116166505A (en) Monitoring platform, method, storage medium and equipment for dual-state IT architecture in financial industry
CN119011279A (en) Network security log management method and system
CN119646541A (en) A cloud computer end-to-end fault root cause analysis method and system
CN119149367A (en) Observable analysis method and system based on intelligent analysis
CN120378281A (en) Abnormality detection method and device, nonvolatile storage medium, and electronic device
CN120358147A (en) Monitoring index dependency analysis and topology establishment method and device and computer equipment
CN109995558A (en) Fault information processing method, device, equipment and storage medium
CN112181759A (en) Method for monitoring micro-service performance and diagnosing abnormity
CN120018089A (en) Vehicle Data Wireless Interaction System
CN119537136A (en) An abnormal collection and processing system
CN112583842A (en) Network security situation awareness system platform based on data stream processing
CN108449212B (en) MAS message transmission method based on event association

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20200515