CN117176802B - Full-link monitoring method and device for service request, electronic equipment and medium - Google Patents
Full-link monitoring method and device for service request, electronic equipment and medium Download PDFInfo
- Publication number
- CN117176802B CN117176802B CN202311444556.5A CN202311444556A CN117176802B CN 117176802 B CN117176802 B CN 117176802B CN 202311444556 A CN202311444556 A CN 202311444556A CN 117176802 B CN117176802 B CN 117176802B
- Authority
- CN
- China
- Prior art keywords
- application
- service request
- network
- data
- index data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 120
- 238000000034 method Methods 0.000 title claims abstract description 63
- 230000002776 aggregation Effects 0.000 claims abstract description 58
- 238000004220 aggregation Methods 0.000 claims abstract description 58
- 230000004931 aggregating effect Effects 0.000 claims abstract description 16
- 239000000523 sample Substances 0.000 claims description 24
- 238000012545 processing Methods 0.000 claims description 23
- 238000004590 computer program Methods 0.000 claims description 16
- 238000003860 storage Methods 0.000 claims description 13
- 230000004044 response Effects 0.000 claims description 4
- 230000008569 process Effects 0.000 abstract description 15
- 238000004891 communication Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 7
- 238000012806 monitoring device Methods 0.000 description 7
- 238000012216 screening Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 238000003780 insertion Methods 0.000 description 4
- 230000037431 insertion Effects 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 239000000243 solution Substances 0.000 description 3
- 239000003795 chemical substances by application Substances 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000002347 injection Methods 0.000 description 1
- 239000007924 injection Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a method, a device, electronic equipment and a medium for monitoring a full link of a service request. The method comprises the steps of obtaining a tracking identifier, an application node identifier and application index data associated with the service request by each application node; acquiring the tracking identifier, the data tag and network index data associated with the service request by each network node; and according to the tracking identifier, the application node identifier and the data tag, aggregating the application index data and the network index data corresponding to the service request, and determining the full-link monitoring topology according to an aggregation result. The embodiment of the invention realizes the logic series connection of the whole link from the data center entry application to the back-end application and then returned to the user by the service request, obtains the full-link monitoring topology from the network to the application access, and solves the problem of lack of full-link monitoring from the network to the application end in the service transaction request process.
Description
Technical Field
The present invention relates to the field of communications technologies, and in particular, to a method, an apparatus, an electronic device, and a medium for monitoring a full link of a service request.
Background
With the development of internet technology, users have higher and higher experience requirements in the use process of the information system, and the interaction logic and the internal architecture of the information system are complex.
In order to monitor the service request of the information system, the link tracking scheme in the related technology only monitors each isolated link, and lacks the full link monitoring of the service request process. Therefore, how to monitor the full link accessed by the business transaction in real time, and ensure the healthy and stable external service of the information system becomes the current problem to be solved urgently.
Disclosure of Invention
The invention provides a method, a device, electronic equipment and a medium for monitoring a full link of a service request, which can monitor the full link of service transaction access in real time.
According to an aspect of the present invention, there is provided a full link monitoring method for a service request, including:
acquiring a tracking identifier, an application node identifier and application index data associated with the service request by each application node, wherein the tracking identifier is generated by an entry application of the service request and is added into the service request, and the application node identifier is used for representing sequence information of each application node for processing the service request;
Acquiring the tracking identifier, a data tag and network index data associated with the service request by each network node, wherein the data tag is used for representing the position sequence of the network node corresponding to the service request;
and according to the tracking identifier, the application node identifier and the data tag, aggregating the application index data and the network index data corresponding to the service request, and determining the full-link monitoring topology according to an aggregation result.
According to another aspect of the present invention, there is provided an all-link monitoring apparatus for a service request, the apparatus comprising:
the system comprises an application index acquisition module, a service request acquisition module and a service request processing module, wherein the application index acquisition module is used for acquiring a tracking identifier, an application node identifier and application index data related to the service request by each application node, wherein the tracking identifier is generated by an entry application of the service request and is added into the service request, and the application node identifier is used for representing sequence information of each application node for processing the service request;
the network index acquisition module is used for acquiring the tracking identifier, the data tag and network index data associated with the service request by each network node, wherein the data tag is used for representing the position sequence of the network node corresponding to the service request;
And the data aggregation module is used for aggregating the application index data and the network index data corresponding to the service request according to the tracking identifier, the application node identifier and the data tag, and determining the full-link monitoring topology according to an aggregation result.
According to another aspect of the present invention, there is provided an electronic apparatus including:
at least one processor; and a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the full link monitoring method of service requests according to any of the embodiments of the present invention.
According to another aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to implement the full link monitoring method for service requests according to any of the embodiments of the present invention when executed.
The embodiment of the invention provides a full link monitoring method for service requests, which aggregates application index data corresponding to each application node and network index data corresponding to each network node according to tracking identification, application node identification representing sequence information of processing the service requests by each application node and data labels representing position sequences of the network nodes corresponding to the service requests, thereby realizing logic series connection of the service requests from data center entry application to back end application and returning the service requests to the whole links of users, obtaining full link monitoring topology accessed from network to application, and solving the problem of lack of full link monitoring from the network to the application end in the service transaction request process.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a method for monitoring a full link of a service request according to an embodiment of the present invention;
FIG. 2 discloses a schematic diagram of the code insertion principle;
fig. 3 is a flowchart of another method for monitoring a full link of a service request according to an embodiment of the present invention;
fig. 4 is a system architecture diagram of a full-link monitoring system for service request according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a full-link monitoring device for service request according to an embodiment of the present invention;
fig. 6 shows a schematic diagram of an electronic device that may be used to implement an embodiment of the invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Fig. 1 is a flowchart of a method for monitoring a full link of a service request according to an embodiment of the present invention, where the method may be performed by a full link monitoring device of the service request, and the full link monitoring device of the service request may be implemented in hardware and/or software, and the full link monitoring device of the service request may be configured in an electronic device. As shown in fig. 1, the method includes:
s110, acquiring tracking identification, application node identification and application index data associated with the service request by each application node.
Wherein the tracking identifier is generated by an ingress application of the service request and is added to the service request. Specifically, the tracking identifier may be used to uniquely identify a service request, where different service requests correspond to different tracking identifiers. Since the service request is communicated in the entire link from the data center portal application to the backend application and back to the user, the tracking identifier is also communicated in the entire link.
For example, the tracking identifier may be a trace id generated by the portal application of each service request prior to execution of the service request. At the application interaction level, the tracking identification is transferred by the way that the upper application node calls the lower application node. At the network level, tracking identities are communicated by way of traffic flows through network nodes. The transfer mode is related to a specific request mode, such as HTTP call, and when the upper application node calls the lower application node, the tracking identification is transferred to the lower application node through an HTTP header mode. And the data packet corresponding to the HTTP request is transmitted to the network node B by the network node A, namely the tracking identification is transmitted to the network node B by the network node A.
In the embodiment of the invention, the application node identifier is used for indicating the sequence information of each application node for processing the service request. For example, in the call link of the service request, the application a has a call relationship with the applications B, C, D and E, and specifically, the hierarchical relationship of each application node in the call link is that the application a calls the application B, the application B calls the application C, and the application C calls the application D. Each call is marked as an application node, an application node identifier is generated according to natural number growth, the application node is marked by the application node identifier, and a call link of a service request is formed according to the application node. In the calling link, the hierarchical relationship among the nodes needs to be distinguished, and in the embodiment of the invention, the hierarchical relationship of the application nodes is marked based on the application node identification.
The application index data may be index data generated by the application node processing the service request.
In this embodiment, application monitoring probes may be used to collect application system instances, obtain application performance metrics by injecting jvm techniques, and insert request headers and response headers that track the identity into the HTTP.
The jvm probe is a set of monitoring class loader provided by the virtual machine and an agent interface conforming to the specification of the virtual machine, and can realize a non-invasive monitoring function by combining a byte code instruction. Such as monitoring function calls in a production environment, etc.
According to the embodiment of the invention, the related performance index generated by processing the service request by the application node is obtained by automatically replacing or modifying the specific java class. Specifically, during the starting process of the java container, the java file is determined through specific java agent parameters to start the probe program. The application index data collected by the application monitoring probe includes traffic performance data, backend performance data, container performance data, host performance data, exception data, event data, slow requests, error requests, and the like.
The application index data can be collected by inserting codes into specific classes and expanding class functions. The insertion code can be understood as inserting byte codes into a specific class file, and acquiring application performance data through byte code instructions. Because the insertion process is automatically completed in the memory at the container starting stage, the class file on the disk cannot be actually modified, and all application performance data are uploaded to the data collector at regular time through the HTTP protocol. The data collector is used for collecting application index data and network index data through the data collector, wherein the data collector is a component of the data aggregation platform.
Fig. 2 discloses a schematic diagram of the code insertion principle. As shown in fig. 2, taking java as an example, a java proxy function in the premain () method is used as an interceptor and a translator. The premain () method is logic that a program executes before running main (), and byte code modification and the like can be performed in this logic. When the JVM loads the class file, a transform method of the converter is triggered, and the ASM is utilized to dynamically modify the loaded class file, so that the modification injection of the byte codes is realized. Wherein ASM is a java bytecode manipulation framework that can be used to dynamically generate classes or enhance functionality of existing classes. After the ASM reads information from the class file, the class behavior can be changed, the class information can be analyzed, and even a new class can be generated according to the user requirement, namely, the ASM can directly modify the class file by modifying the byte code.
S120, acquiring the tracking identification, the data tag and network index data associated with the service request by each network node.
The tracking identifier is generated by the portal application of the service request and added to the identifier in the service request, and the tracking identifier can be obtained by analyzing the request information of the data packet corresponding to the service request under different network communication protocols.
The data tag is used for indicating the position sequence of the network node corresponding to the service request. In the process of the flow data of the service request flowing among the network devices, each network device processes the flow, thereby generating network index data related to the network node and the service request. If the network device is marked as a network node, the network index data can be obtained according to the sequence of the flow data flowing through the network node, and a data tag is added to the network index data. For example, traffic data is transmitted from the ingress firewall to the F5 device, from the F5 device to the SSL device, from the SSL device to the LTM device, from the LTM device to the application node, and from the application node to the LTM device, and after the network index data of each layer is acquired, a data tag is added to each network index data. Since the tracking identifiers of the same service request are the same, the sequence of the traffic data flowing through the network node cannot be analyzed through the tracking identifiers, and therefore, the sequence of the locations of the traffic data flowing through the network node needs to be marked through the data tag.
The network performance index may be index data generated by the network node processing the service request.
In the embodiment of the invention, the network monitoring probe is used for mirroring the flow through the firewall equipment of the physical switch to obtain the bypass flow. The acquisition of the data packets of the bypass traffic, in particular, the allocation of a certain number of threads, driven by the DPDK (data plane development kit ), is responsible for receiving the bypass traffic. The monitoring range information is defined in the network monitoring analysis server in advance, the flow data of each logic node (load or application) of the data center is collected according to the monitoring range information, the data packet included in the flow data is analyzed through the network monitoring analysis engine, the tracking identification and the network index data are obtained, and the network index data are sent to the message middleware. Specifically, the network monitoring analysis engine analyzes the data packet according to each data packet field to obtain trace id and network index data associated with each network node, wherein the data packet can be an HTTP data packet or a TCP data packet.
Since the data packet carries a lot of data, the screening model can be predefined, so that the network index data can be obtained from the analyzed data through the screening model. For example, the screening model is stored in the form of dimension-index data, so that network index data to be monitored is screened from the parsed data according to the model.
It should be noted that, the traffic data in the network node behind the certificate encryption device is encrypted data, the SSL certificate needs to be installed through the network convergence distribution device for decryption, and then the HTTP information and the network index data in the decrypted data packet are obtained through the network monitoring probe, so that the tracking identifier is obtained according to the HTTP information.
And S130, according to the tracking identifier, the application node identifier and the data tag, aggregating the application index data and the network index data corresponding to the service request, and determining the full-link monitoring topology according to an aggregation result.
The aggregation can be understood as integrating the application index data and the network index data, and applying a user HTTP request from the entrance to the back-end application and returning the whole link of the user to carry out logic series connection, so as to realize the end-to-end full link monitoring in the service request process.
Illustratively, the application index data collected by the application monitoring probe and the network index data collected by the network monitoring probe are summarized and filtered by the data collector and then sent to the data aggregation tool. Because npm has more index data, each index may not need to be monitored in an actual monitoring scene, and the target index data can be screened from the index data (including application index data and network index data) through the data collector and sent to the data aggregation tool.
Determining application index data and network index data corresponding to the same service request according to the tracking identifier through a data aggregation tool; determining aggregation position information according to the application node identifier and the data tag of the service request; and aggregating the application index data and the network index data corresponding to the service request according to the aggregation position information, namely applying a user HTTP request from an entrance to a back-end application and returning to the whole link of the user to carry out logic series connection, and opening the problem analysis process of the network index and the application index.
It should be noted that, if the full-link monitoring function is deployed separately and provides services to the outside, the server deploying the full-link monitoring method needs to establish a mutually trusted configuration with the target server corresponding to the service request. The servers for deploying the full-link monitoring method can be transversely expanded, a plurality of servers can be deployed according to actual needs, and the full-link monitoring topology of the application combined network can be automatically drawn according to data integration logic through a load balancing control data center platform visualization model.
According to the embodiment of the invention, the application index data corresponding to each application node and the network index data corresponding to each network node are aggregated according to the tracking identification, the application node identification representing the sequence information of each application node for processing the service request and the data label representing the position sequence of the network node corresponding to the service request, so that the logic series connection of the whole link of the service request from the data center entry application to the back end application and then back to the user is realized, the full-link monitoring topology accessed from the network to the application is obtained, and the problem of lack of full-link monitoring from the network to the application end in the service transaction request process is solved.
Fig. 3 is a flowchart of another method for monitoring a full link of a service request according to an embodiment of the present invention, and this embodiment is a detailed description of the foregoing embodiment. As shown in fig. 3, the method includes:
s301, acquiring tracking identification, application node identification and application index data associated with the service request by each application node.
Wherein the tracking identifier is generated by an entry application of the service request and is added to the service request, and the application node identifier is used for representing sequence information of processing the service request by each application node.
Illustratively, acquiring the tracking identifier, the application node identifier, and the application index data associated with the service request by each application node includes: generating the tracking identification through the entry application of the service request, and adding the tracking identification to a request header or a response header of the service request. And determining the application node identification according to the hierarchical relationship of each application node in the application call link of the service request. And collecting application index data generated by executing the service request by each application program through an application monitoring probe.
S302, generating an application monitoring message according to the tracking identifier, the application node identifier and the application index data, sending the application monitoring message to a data collector, and transmitting the application monitoring message to a data aggregation tool through the data collector.
Illustratively, the HTTP message is generated by the application monitoring pointer according to the tracking identifier, the application node identifier and the application index data, and the HTTP message is sent to the data collector by the HTTP protocol. And screening the application index data according to the actual monitoring requirement by a data collector, updating the HTTP message according to the screening result, the tracking identifier and the application node identifier, and transmitting the updated HTTP message to a data aggregation tool.
S303, judging whether the network node is the network device behind the SSL decryption device, if so, executing S304, otherwise, executing S305.
S304, acquiring the tracking identifier, the data tag and the network index data associated with the service request by each network node, and turning to execution S308.
The data tag is used for indicating the position sequence of the network node corresponding to the service request.
Illustratively, acquiring the tracking identifier, the data tag, and the network index data associated with the service request by each network node includes: under the condition that the mirror image flow corresponding to the service request reaches the network node, decrypting the mirror image flow corresponding to the network node, and obtaining the tracking identification in the decrypted mirror image flow; collecting network index data associated with the service request by each network node through a network monitoring probe; and adding a data tag to the network index data corresponding to each network node according to the position sequence of the mirror image flow flowing through each network node.
And generating a network monitoring message according to the tracking identifier, the data tag and the network index data, and sending the network monitoring message to the message middleware. Wherein the message middleware may be kafka et al. And acquiring target network index data meeting set conditions from the message middleware through a data acquisition probe, and acquiring target network monitoring messages in the network monitoring messages according to the target network index data. And sending the target network monitoring message to a data collector, and transmitting the target network monitoring message to a data aggregation tool through the data collector.
S305, the IP address, the port information and the network index data in the npm data packet corresponding to the network node are acquired.
Because the traffic data at the network device before the SSL decryption device is encrypted traffic, HTTP information cannot be obtained by parsing the data packet, and further, according to the HTTP information, the tracking identifier is obtained, then the data packet needs to be obtained NPM from the NPM tool, and the IP address, the port information and the network index data are obtained by parsing the NPM data packet.
The NPM packets may be obtained by NPM (Network Performance Monitor, network performance monitoring) tools.
And S306, matching apm instance information according to the IP address and the port information to obtain corresponding application index data.
Wherein the APM instance information includes IP address, port information, and application index data, the APM instance information can be obtained by an APM (Application Performance Monitor, application performance monitoring) tool.
Because the IP address, the port information and the network index data are obtained from the npm data packet, the IP address, the port information and the application index data are analyzed based on apm instance information, the IP address and the port information are analyzed according to the matching of the IP address and the port information of the npm data packet to apm instance information, and the application index data corresponding to the network index data are determined.
S307, aggregating the analyzed network index data and the matched application index data to obtain a first aggregation result, and turning to S311.
S308, determining the application index data and the network index data corresponding to the same service request according to the tracking identification.
S309, determining the aggregation position information according to the application node identification and the data label of the service request.
The aggregation position information represents the logic front-back relation of the application index number in the application link and also represents the logic front-back relation of the network index data in the network link, so that the application index data and the network index data with the same logic front-back relation can be determined according to the aggregation position.
And S310, aggregating the application index data and the network index data corresponding to the service request according to the aggregation position information to obtain a second aggregation result.
S311, determining the full link monitoring topology according to the first aggregation result and the second aggregation result.
In the embodiment of the invention, under the condition that the network node is the network equipment before SSL decryption equipment, the IP address, port information and network index data in a npm data packet corresponding to the network node are acquired; according to the IP address and the port information, apm instance information is matched, and corresponding application index data is obtained; aggregating the analyzed network index data and the matched application index data to obtain a first aggregation result; the first aggregation result is combined with the second aggregation result to obtain a full-link monitoring topology, aggregation of application index data and network index data corresponding to network nodes in front of SSL decryption equipment is achieved, and the problem that a monitoring link cannot cover the full link from a network to an application end in a service transaction request process due to the fact that encrypted traffic cannot resolve tracking identifiers is solved.
Fig. 4 is a system architecture diagram of a service request full-link monitoring system according to an embodiment of the present invention, where the service request full-link monitoring system is used to implement the service request full-link monitoring method provided in each embodiment. As shown in fig. 4, the system includes: network monitoring probe 410, application monitoring probe 420, message middleware 430, data acquisition probe 440, network data collector 450, application data collector 460, data aggregation tool 470, and Web presentation page 480.
When a service request passes through the approval application, the portal application generates a specific unique identification for identifying the service request. The specific unique identification may be a trace identification (trace id) or the like. And determining the application node identification according to the hierarchical relationship of each application node in the application call link of the service request. Application index data generated by the execution of service requests by various applications of the handset through the application monitoring probe 420. An application monitoring message is generated according to the tracking identifier, the application node identifier and the application index data, the application monitoring message is sent to the application data collector 460, and the application monitoring message is transmitted to the data aggregation tool 470 through the application data collector 460.
Under the condition that the mirror image flow corresponding to the service request reaches the network node, decrypting the mirror image flow corresponding to the network node, and obtaining the tracking identification in the decrypted mirror image flow. Network index data associated with the service requests for each network node is collected by the network monitoring probe 410. And adding a data tag to the network index data corresponding to each network node according to the position sequence of the mirror image flow flowing through each network node. A network monitoring message is generated according to the tracking identifier, the data tag and the network index data, and the network monitoring message is sent to the message middleware 430. Target network index data meeting the set conditions is obtained from the message middleware through the data acquisition probe 440, and a target network monitoring message in the network monitoring message is obtained according to the target network index data. The target network monitor message is sent to the network data collector 450 and the target network monitor message is transmitted by the network data collector 450 to the data aggregation tool 470.
Application index data and network index data corresponding to the same service request are determined by the data aggregation tool 470 according to the tracking identifier. And determining the aggregation position information according to the application node identification and the data label of the service request. And aggregating the application index data and the network index data corresponding to the service request according to the aggregation position information to obtain an application-network full-link monitoring topology, and displaying the application-network full-link monitoring topology diagram through a Web display page 480.
It should be noted that the network data collector 450 and the application data collector 460 may be the same physical machine, and the network data collector 450 for collecting network index data and the application data collector 460 for collecting application index data are logically defined.
Alternatively, the network data collector 450 and the application data collector 460 may be different physical machines for collecting network index data and application index data, respectively.
Fig. 5 is a schematic structural diagram of a full-link monitoring device for service request according to an embodiment of the present invention. The device can execute the full-link monitoring method of the service request provided by any embodiment of the invention, the device can be realized in a form of hardware and/or software, and the full-link monitoring device of the service request can be configured in the electronic equipment.
As shown in fig. 5, the apparatus includes: an application index acquisition module 510, a network index acquisition module 520, and a data aggregation module 530.
An application index obtaining module 510, configured to obtain a tracking identifier, an application node identifier, and application index data associated with the service request by each application node, where the tracking identifier is generated by an entry application of the service request and is added to the service request, and the application node identifier is used to represent sequence information of processing the service request by each application node;
a network indicator obtaining module 520, configured to obtain the tracking identifier, a data tag, and network indicator data associated with the service request by each network node, where the data tag is used to represent a position sequence of the network node corresponding to the service request;
and the data aggregation module 530 is configured to aggregate the application index data and the network index data corresponding to the service request according to the tracking identifier, the application node identifier and the data tag, and determine a full link monitoring topology according to an aggregation result.
Optionally, the application index obtaining module 510 is specifically configured to:
generating the tracking identifier through the entry application of the service request, and adding the tracking identifier to a request header or a response header of the service request;
Determining an application node identification according to the hierarchical relationship of each application node in the application call link of the service request;
and collecting application index data generated by executing the service request by each application program through an application monitoring probe.
Optionally, the apparatus further comprises:
and the first sending module is used for generating an application monitoring message according to the tracking identifier, the application node identifier and the application index data after the application index data generated by executing the service request by each application program is collected through the application monitoring probe, sending the application monitoring message to a data collector, and transmitting the application monitoring message to a data aggregation tool through the data collector.
Optionally, the network indicator obtaining module 520 is specifically configured to:
under the condition that the mirror image flow corresponding to the service request reaches the network node, decrypting the mirror image flow corresponding to the network node, and obtaining the tracking identification in the decrypted mirror image flow;
collecting network index data associated with the service request by each network node through a network monitoring probe;
and adding a data tag to the network index data corresponding to each network node according to the position sequence of the mirror image flow flowing through each network node.
Optionally, the apparatus further comprises:
the second sending module is used for generating a network monitoring message according to the tracking identifier, the data tag and the network index data after adding the data tag to the network index data corresponding to each network node according to the position sequence of the mirror flow flowing through each network node, and sending the network monitoring message to the message middleware;
acquiring target network index data meeting set conditions from the message middleware through a data acquisition probe, and acquiring target network monitoring messages in the network monitoring messages according to the target network index data;
and sending the target network monitoring message to a data collector, and transmitting the target network monitoring message to a data aggregation tool through the data collector.
Optionally, the apparatus further comprises:
the IP data acquisition module is used for acquiring the IP address, the port information and the network index data in the npm data packet corresponding to the network node under the condition that the network node is the network device before the SSL decryption device;
the matching module is used for matching apm instance information according to the IP address and the port information to obtain corresponding application index data, wherein the apm instance information comprises the IP address, the port information and the application index data;
And the first aggregation module is used for aggregating the analyzed network index data and the matched application index data to obtain a first aggregation result.
Optionally, the data aggregation module 530 is specifically configured to:
determining application index data and network index data corresponding to the same service request according to the tracking identification;
determining aggregation position information according to the application node identifier and the data tag of the service request;
aggregating the application index data and the network index data corresponding to the service request according to the aggregation position information to obtain a second aggregation result;
and determining the full link monitoring topology according to the first aggregation result and the second aggregation result.
The full-link monitoring device for the service request provided by the embodiment of the invention can execute the full-link monitoring method for the service request provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Fig. 6 shows a schematic diagram of an electronic device that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 6, the electronic device 10 includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, in which the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 may also be stored. The processor 11, the ROM 12 and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
Various components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as the full link monitoring method of service requests.
In some embodiments, the full link monitoring method of traffic requests may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the above-described full link monitoring method of service requests may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the full link monitoring method of the service request in any other suitable way (e.g., by means of firmware).
Various implementations of the systems and techniques described here above can be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (10)
1. A method for full link monitoring of a service request, comprising:
acquiring a tracking identifier, an application node identifier and application index data of each application node associated with the service request, wherein the tracking identifier is generated by an entry application of the service request and is added into the service request, the application node identifier is used for representing sequence information of each application node for processing the service request, the application index data represents index data generated by the application node for processing the service request, and the application node represents an application program;
Acquiring the tracking identifier, a data tag and network index data associated with the service request by each network node, wherein the data tag is used for representing the position sequence of the network node corresponding to the service request;
and according to the tracking identifier, the application node identifier and the data tag, aggregating the application index data and the network index data corresponding to the service request, and determining the full-link monitoring topology according to an aggregation result.
2. The method of claim 1, wherein the obtaining the tracking identifier, the application node identifier, and the application index data associated with the service request by each application node comprises:
generating the tracking identifier through the entry application of the service request, and adding the tracking identifier to a request header or a response header of the service request;
determining an application node identification according to the hierarchical relationship of each application node in the application call link of the service request;
and collecting application index data generated by executing the service request by each application program through an application monitoring probe.
3. The method of claim 2, further comprising, after collecting application index data generated by each application executing the service request by an application monitoring probe:
And generating an application monitoring message according to the tracking identifier, the application node identifier and the application index data, sending the application monitoring message to a data collector, and transmitting the application monitoring message to a data aggregation tool through the data collector.
4. The method of claim 1, wherein the obtaining the tracking identifier, the data tag, and the network metric data associated with the service request by each network node comprises:
under the condition that the mirror image flow corresponding to the service request reaches the network node, decrypting the mirror image flow corresponding to the network node, and obtaining the tracking identification in the decrypted mirror image flow;
collecting network index data associated with the service request by each network node through a network monitoring probe;
and adding a data tag to the network index data corresponding to each network node according to the position sequence of the mirror image flow flowing through each network node.
5. The method of claim 4, further comprising, after adding a data tag to the network index data corresponding to each network node according to a sequence of locations of the mirror traffic flowing through each network node:
Generating a network monitoring message according to the tracking identifier, the data tag and the network index data, and sending the network monitoring message to a message middleware;
acquiring target network index data meeting set conditions from the message middleware through a data acquisition probe, and acquiring target network monitoring messages in the network monitoring messages according to the target network index data;
and sending the target network monitoring message to a data collector, and transmitting the target network monitoring message to a data aggregation tool through the data collector.
6. The method as recited in claim 1, further comprising:
under the condition that the network node is network equipment before SSL decryption equipment, acquiring an IP address, port information and network index data in a npm data packet corresponding to the network node;
according to the IP address and the port information, apm instance information is matched, corresponding application index data is obtained, wherein the apm instance information comprises the IP address, the port information and the application index data;
and aggregating the analyzed network index data and the matched application index data to obtain a first aggregation result.
7. The method of claim 6, wherein aggregating the application index data and the network index data corresponding to the service request according to the tracking identifier, the application node identifier and the data tag, and determining the full link monitoring topology according to the aggregation result comprises:
Determining application index data and network index data corresponding to the same service request according to the tracking identification;
determining aggregation position information according to the application node identifier and the data tag of the service request;
aggregating the application index data and the network index data corresponding to the service request according to the aggregation position information to obtain a second aggregation result;
and determining the full link monitoring topology according to the first aggregation result and the second aggregation result.
8. A full link monitoring apparatus for service requests, comprising:
the system comprises an application index acquisition module, a service request processing module and a service request processing module, wherein the application index acquisition module is used for acquiring a tracking identifier, an application node identifier and application index data of each application node associated with the service request, wherein the tracking identifier is generated by an entry application of the service request and is added into the service request, the application node identifier is used for representing sequence information of each application node for processing the service request, the application index data represents index data generated by the application node for processing the service request, and the application node represents an application program;
the network index acquisition module is used for acquiring the tracking identifier, the data tag and network index data associated with the service request by each network node, wherein the data tag is used for representing the position sequence of the network node corresponding to the service request;
And the data aggregation module is used for aggregating the application index data and the network index data corresponding to the service request according to the tracking identifier, the application node identifier and the data tag, and determining the full-link monitoring topology according to an aggregation result.
9. An electronic device, the electronic device comprising:
at least one processor; and a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the full link monitoring method of the service request of any one of claims 1-7.
10. A computer readable storage medium storing computer instructions for causing a processor to perform the full link monitoring method of a service request according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311444556.5A CN117176802B (en) | 2023-11-02 | 2023-11-02 | Full-link monitoring method and device for service request, electronic equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311444556.5A CN117176802B (en) | 2023-11-02 | 2023-11-02 | Full-link monitoring method and device for service request, electronic equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117176802A CN117176802A (en) | 2023-12-05 |
| CN117176802B true CN117176802B (en) | 2024-01-16 |
Family
ID=88930071
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311444556.5A Active CN117176802B (en) | 2023-11-02 | 2023-11-02 | Full-link monitoring method and device for service request, electronic equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117176802B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117714327A (en) * | 2024-02-05 | 2024-03-15 | 神州灵云（北京）科技有限公司 | Full-link business request performance indicator tracking methods, systems, equipment and media |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB201209078D0 (en) * | 2011-06-23 | 2012-07-04 | Ibm | Managing events generated from business objects |
| US9462014B1 (en) * | 2015-04-23 | 2016-10-04 | Datiphy Inc. | System and method for tracking and auditing data access in a network environment |
| CN110968479A (en) * | 2019-11-20 | 2020-04-07 | 北京宝兰德软件股份有限公司 | Business-level full-link monitoring method for application program and server |
| WO2020233015A1 (en) * | 2019-05-20 | 2020-11-26 | 平安普惠企业管理有限公司 | Link tracking method and apparatus |
| CN112383436A (en) * | 2020-11-17 | 2021-02-19 | 珠海大横琴科技发展有限公司 | Network monitoring method and device |
| CN113505393A (en) * | 2020-12-30 | 2021-10-15 | 曹凤仙 | Block chain payment data processing method applied to big data and cloud server |
| CN113987074A (en) * | 2021-10-27 | 2022-01-28 | 中国工商银行股份有限公司 | Distributed service full-link monitoring method and device, electronic equipment and storage medium |
| CN114143203A (en) * | 2021-11-05 | 2022-03-04 | 华东师范大学 | Kubernetes container network data packet index acquisition method and system based on dynamic service topological mapping |
| CN114257472A (en) * | 2021-12-07 | 2022-03-29 | 中信银行股份有限公司 | Network topology monitoring method, device, equipment and readable storage medium |
| US11347622B1 (en) * | 2020-10-06 | 2022-05-31 | Splunk Inc. | Generating metrics values for teams of microservices of a microservices-based architecture |
| CN115357761A (en) * | 2022-08-29 | 2022-11-18 | 建信金融科技有限责任公司 | Link tracking method and device, electronic equipment and storage medium |
| CN115529227A (en) * | 2022-08-29 | 2022-12-27 | 金螳螂家数字科技(苏州)有限公司 | A Link Tracking and Abnormal Diagnosis Method Based on Web Request |
| CN116827762A (en) * | 2022-03-22 | 2023-09-29 | 中移雄安信息通信科技有限公司 | Link fault positioning method, device, equipment and computer readable storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120275311A1 (en) * | 2011-04-29 | 2012-11-01 | Tektronix, Inc. | Automatic Network Topology Detection and Modeling |
-
2023
- 2023-11-02 CN CN202311444556.5A patent/CN117176802B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB201209078D0 (en) * | 2011-06-23 | 2012-07-04 | Ibm | Managing events generated from business objects |
| US9462014B1 (en) * | 2015-04-23 | 2016-10-04 | Datiphy Inc. | System and method for tracking and auditing data access in a network environment |
| WO2020233015A1 (en) * | 2019-05-20 | 2020-11-26 | 平安普惠企业管理有限公司 | Link tracking method and apparatus |
| CN110968479A (en) * | 2019-11-20 | 2020-04-07 | 北京宝兰德软件股份有限公司 | Business-level full-link monitoring method for application program and server |
| US11347622B1 (en) * | 2020-10-06 | 2022-05-31 | Splunk Inc. | Generating metrics values for teams of microservices of a microservices-based architecture |
| CN112383436A (en) * | 2020-11-17 | 2021-02-19 | 珠海大横琴科技发展有限公司 | Network monitoring method and device |
| CN113505393A (en) * | 2020-12-30 | 2021-10-15 | 曹凤仙 | Block chain payment data processing method applied to big data and cloud server |
| CN113987074A (en) * | 2021-10-27 | 2022-01-28 | 中国工商银行股份有限公司 | Distributed service full-link monitoring method and device, electronic equipment and storage medium |
| CN114143203A (en) * | 2021-11-05 | 2022-03-04 | 华东师范大学 | Kubernetes container network data packet index acquisition method and system based on dynamic service topological mapping |
| CN114257472A (en) * | 2021-12-07 | 2022-03-29 | 中信银行股份有限公司 | Network topology monitoring method, device, equipment and readable storage medium |
| CN116827762A (en) * | 2022-03-22 | 2023-09-29 | 中移雄安信息通信科技有限公司 | Link fault positioning method, device, equipment and computer readable storage medium |
| CN115357761A (en) * | 2022-08-29 | 2022-11-18 | 建信金融科技有限责任公司 | Link tracking method and device, electronic equipment and storage medium |
| CN115529227A (en) * | 2022-08-29 | 2022-12-27 | 金螳螂家数字科技(苏州)有限公司 | A Link Tracking and Abnormal Diagnosis Method Based on Web Request |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117176802A (en) | 2023-12-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108989136B (en) | Service end-to-end performance monitoring method and device | |
| US11336545B2 (en) | Network device measurements employing white boxes | |
| US9935853B2 (en) | Application centric network experience monitoring | |
| CN113688149A (en) | Monitoring method and device | |
| CN115102730A (en) | Integrated monitoring method for multiple devices | |
| CN117176802B (en) | Full-link monitoring method and device for service request, electronic equipment and medium | |
| CN114125024B (en) | Audio transmission method, electronic device and readable storage medium | |
| CN106648722A (en) | Flume receiving side data processing method and device based on big data | |
| CN115048279A (en) | Server processing method and device, electronic equipment and computer readable medium | |
| CN112769923A (en) | Method, device and storage medium for monitoring network equipment performance index in big data scene | |
| CN118827393B (en) | Application Observation Link Topology Construction Method and Related Equipment Based on eBPF | |
| CN113485282A (en) | Message tracking display method, system, equipment and storage medium for distributed control system | |
| CN112152879A (en) | Network quality determination method and device, electronic equipment and readable storage medium | |
| CN110391950A (en) | A kind of application service test method and device | |
| CN111756836B (en) | Information sending method and device based on event management model | |
| CN115567431A (en) | Network stability testing method and device, tested equipment and storage medium | |
| CN109271310B (en) | Development testing method and device for network function of mobile application program | |
| US12132603B2 (en) | Data processing method, apparatus, and system, and storage medium | |
| CN117251769B (en) | Abnormal data identification method, device, equipment and medium based on monitoring component | |
| CN114615168B (en) | Application level monitoring method and device, electronic equipment, storage medium and product | |
| CN116483471A (en) | Data acquisition plug-in, method, equipment and storage medium | |
| CN119052101A (en) | Storage area network detection and analysis method, detection and analysis equipment and medium | |
| WO2025017654A1 (en) | Method and system for configuring one or more attributes of nodes in a network | |
| CN118349512A (en) | Method, device, equipment and medium for determining topology information between cloud services | |
| CN119299287A (en) | Fault root cause location method and device, storage medium and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |