CN104598348B - A kind of method and system of the long-range external system interface performance of analysis in real time - Google Patents

Abstract

The invention discloses a method and system for analyzing the interface performance of an external system remotely and in real time, which can record the interface information each time the external interface is accessed as a memory log, start log analysis after the application program is executed, and log in the browser Output and display the performance analysis report of each interface in the debugging page window, realizing the remote real-time analysis of the performance of the external interface. Since the logs are stored in the memory and do not use the disk storage space on the server, the hard disk resources of the server are fully saved, and the waste of network resources and inefficiency caused by dragging and fetching log information from a remote server for manual viewing are avoided. In addition, the interface analysis report can be displayed according to the output content format of the application program, which improves user experience.

Description

A method and system for remote real-time analysis of external system interface performance

technical field

The invention belongs to the technical field of data processing and analysis, in particular to a method for remote real-time analysis of the performance of an external system data layer interface.

Background technique

With the rapid development of science and technology, the speed of information globalization and sharing is also accelerating, which is inseparable from the large-scale popularization and application of network technology, and network performance and stability play a key role in whether information can be exchanged smoothly. In network applications, it is often necessary to access the interface of the external system to obtain data through http requests. As the interface of the external system is used more and more frequently, the performance and stability of the system will be more and more affected by the external system, which requires the interface of the external system Perform analysis to obtain external system interface status.

But at present, existing technical solutions can only achieve real-time analysis or remote analysis, and cannot achieve both. The existing real-time analysis is done by outputting intermediate values in the code during the development stage, which is only applicable to the local development environment, not the remote production environment; while the existing remote analysis is to output intermediate values in the remote interface code In remote files, downloading remote files after a certain period of time for manual viewing is not real-time, and viewing is inconvenient. The above-mentioned files that record the interface status of the external system will take up a lot of disk space, and will also take up a lot of network resources during remote transmission. In addition, there are too many recorded information, which may contain sensitive information, and there are also security risks in technology.

Contents of the invention

In order to solve the above problems, the present invention discloses a method for analyzing data layer interfaces, which can perform real-time analysis of the performance of external system interfaces in a remote environment.

In order to achieve the above object, the present invention provides the following technical solutions:

A method for remotely analyzing the performance of an external system interface in real time, comprising the following steps:

Step A, open the webpage to be debugged in the remote environment through a browser;

Step B, verify whether the current user has access rights, and only perform subsequent steps for authorized users;

Step C: Execute the current application function logic. When the external system interface needs to be requested during the execution process, store log data in the memory before and after executing the interface request. The log data includes the following information of the external system interface: unique identifier, requested Start time, end time of request, external interface description information, external interface status information;

Step D, read all logs, merge various logs according to different unique identifiers, obtain log records of different external interfaces, analyze the performance of each external interface according to the pre-established analysis rules, and obtain the performance analysis report of each external interface ;

Step E, output the content originally output by the application on the page, and output the performance report obtained in step D in a format consistent with the content.

Further, the process of requesting the external interface in step C includes the following steps:

Step C-1, convert the interface name, interface input parameters, random number and/or current time to obtain a string, and generate a quasi-unique or unique identifier of limited length for the string;

Step C-2, convert the interface name and interface input parameters to obtain a character string, and then store the log together with the unique identifier in the format of "unique identifier::description identifier::converted description information";

Step C-3, when starting to access the external interface, store in the log according to the format of "unique identifier::start identifier::current time";

Step C-4, execute the interface request;

Step C-5, at the end of the external interface access, store the log in the format of "unique identifier::end identifier::current time";

Step C-6: Convert the state information when executing the interface request to obtain a character string, and store the log in the format of "unique identifier::state identifier::converted state information".

Further, the unique identifier is generated through the following steps: convert the interface name, interface input parameters, random numbers and/or current time to obtain a character string, and then generate a quasi-unique identifier of limited length for the character string.

Further, the conversion method is json encoding.

Further, the method for generating a limited-length quasi-unique identifier for a character string is to calculate an md5 value for the character string.

Further, the performance report output in the step E is in the same format as the original output content.

Further, the performance report includes interface name, interface input parameters, interface time consumption, and interface status.

Further, the method is applied through the PHP language, and further includes the following steps between step A and step C: performing buffering to obtain all contents output by PHP.

A system for remote real-time analysis of external system interface performance, including an authentication module, a data layer public interface module, a log storage module, a log analysis module, and an output display module. The authentication module is used to authenticate users and control Only authorized users can access; the data layer public interface module provides the encapsulation of the request external system interface, and records the start time, description information, end time and status information of the request to the log storage module before and after executing the interface request; The log storage module is used to store the log generated when the data layer public interface module accesses the external interface; the log analysis module is used to read the log data in the log storage module, analyze and merge the log data, and finally obtain Each interface analysis report; the output display module is used to output the original function content in the browser and output each interface performance analysis report according to the content format.

Further, the log storage module is a globally accessible PHP array, and the PHP array only exists in the memory when the PHP program is executed, and disappears after the webpage is generated.

Beneficial effect:

The present invention can record the interface information of each external interface access as a memory log, start log analysis after the application program is executed, and output and display the performance analysis report of each interface in the browser debugging page window, realizing the external interface Remote real-time analysis of performance. Since the logs are stored in the memory and do not use the disk storage space on the server, the hard disk resources of the server are fully saved, and the waste of network resources and inefficiency caused by dragging and fetching log information from a remote server for manual viewing are avoided. In addition, the interface analysis report can be displayed according to the output content format of the application program, which improves user experience.

Description of drawings

Fig. 1 is the schematic flow chart of the method for remote real-time analysis of external system interface performance provided by the present invention;

Fig. 2 is a schematic flow chart of recording logs before and after requesting an external system interface;

FIG. 3 is a schematic flow chart of parsing and merging logs;

Figure 4 is an example diagram of log analysis results;

Figure 5 is a screenshot of the log analysis results page;

FIG. 6 is a schematic structural diagram of a system for remote real-time analysis of external system interface performance provided by the present invention.

Detailed ways

The technical solutions provided by the present invention will be described in detail below in conjunction with specific examples. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention.

Because PHP is one of the most popular WEB development languages, it is widely used. The present invention uses the PHP language as the environment basis to realize a method for remote real-time analysis of the interface performance of the PHP external system, as shown in Figure 1, specifically including the following step:

Step1: Open the webpage to be debugged through a browser, wherein the webpage to be debugged can call various remote external interfaces. The remote environment means that the machine that provides the PHP service and the machine where the browser is located are not the same machine, and the PHP code cannot be changed by editing, and every time the PHP code is changed, it needs to be released once.

Step2: To run the application function, first perform authentication, users without authorization will not be able to access, and only authorized users are allowed to access.

Step3: content buffering, get all the content output by PHP: when the PHP application is a web page, the content buffered is the entire html of the web page; when the PHP application is an ajax interface, the content buffered is the content of the interface response. When developing in other languages such as java, this step does not need to be included.

Step4: Execute the functional logic and record logs before and after requesting the external system interface: the functional logic is the code that needs to be executed by the application itself. During the execution process, several access requests may be sent to multiple interfaces of the external system, and the request will be sent before and after the request interface The start time, end time, description information, and status information of the system are written into the log, and the log is recorded in the memory without occupying physical hard disk space. The present invention does not need to store log files, but only needs to see the log of the current web page when accessing, and the log stored in the current web page is page-level and request-level, thus ensuring that the log is small enough and avoiding storing sensitive information.

Each log data should store the unique identifier of the interface as a distinction between the interface logs. Specifically, the specific process of recording logs before and after requesting the external system interface (see Figure 2) is as follows:

4-1. First generate the unique identifier of the external system interface. Specifically, the unique identifier generation algorithm is: interface name, interface input parameters, random number, current time (accurate to microseconds, when generating the unique identifier, At least one of the random number and the current time) should be put into a temporary PHP array in turn, and the array is json-encoded to obtain a string, and then the md5 value of the string is the unique identifier, and the md5 value is only As an example, any quasi-unique identifier that can generate a character string with a limited length can be applied to this step.

4-2. Record the description information of the external system interface, put the interface name and interface input parameters into a temporary PHP array, perform json encoding, and then, together with the unique identifier, follow the "unique identifier::d::encoded Description" format, added to the end of the log.

4-3. Record the start time of the external system interface, that is, add it to the end of the log in the format of "unique identifier::b::current time accurate to milliseconds" when accessing the external interface is started.

4-4, Execute the interface request.

4-5. Record the end time of the external system interface, that is, at the end of the external interface access, add it to the end of the log according to the format of "unique identifier::e::current time accurate to milliseconds".

4-6. Record the execution status of the interface: first query the status information when executing the interface request, when the request is successful, record the actual executed request and the character indicating "request successful"; when the request fails, record the specific steps of the failure and the reason for the failure; in addition, the address of the interface that was actually executed should be recorded. Then encode the above information into json, and add it to the last line of the log according to the format of "unique identifier::s::encoded status information".

Each log data has a unique identifier of the currently requested external system interface, and uses intermediate bits to identify different types of log storage data, such as "d" indicates interface description information, "b" indicates interface start time, and "e" indicates The end time of the interface, "s" indicates the execution status of the interface, these characters are easy to remember and distinguish, of course, other characters or numbers can also be used as needed. When there is complex data in the log data (such as description information and status information), a method similar to json encoding should be used to serialize the complex data into strings. Of course, other methods that can achieve the same function can also be used; but when only storage When time data is used, no character conversion process is required.

The following is the log record generated when requesting an external interface:

1402546486e137ee98216701bba3ae26::d::{"header":[],"serviceName":"TBS.city.CityContoller.queryBookingCity","params":{"name":null,"letter":null,"code":null ,"hasBranch":null,"delFlag":0,"start":null,"limit":null,"sortname":"","sortorder":null}}

1402546486e137ee98216701bba3ae26::b::1422004699.2984

1402546486e137ee98216701bba3ae26::e::1422004699.4763

1402546486e137ee98216701bba3ae26::s::{"curlTime":null,"exception":null,"time":"2015-01-2317:18:19","info":{"url":"Http:\/\ /10.10.10.232:12602\/tbs\/city\/booking-city\/query？eyJuYW1lIjpudWxsLCJsZXR0ZXIiOm51bGwsImNvZGUiOm51bGwsImhhc0JyYW5jaCI6bnVsbCwiZGVsRmxhZyI6MCwic3RhcnQiOm51bGwsImxpbWl0IjpudWxsLCJzb3J0bmFtZSI6IiIsInNvcnRvcmRlciI6bnVsbH0＝","content_type":"application\/json；charset＝UTF-8","http_code":200,"header_size ":189,"request_size":326,"filetime":-1,"ssl_verify_result":0,"redirect_count":0,"total_time":0.16215,"namelookup_time":1.9e-5,"connect_time":0.000511, "pretransfer_time":0.000555,"size_upload":0,"size_download":107392,"speed_download":662300,"speed_upload":0,"download_content_length":-1,"upload_content_length":0,"starttransfer_time":0.153267," redirect_time":0,"redirect_url":"","primary_ip":"10.10.10.232","certinfo":[],"primary_port":12602,"local_ip":"172.30.111.47","local_port":54031 ,"error_code":0,"error_string":""}}

Step5: Read each log recorded in Step4, and analyze and merge them one by one. The analysis is to deserialize the serialized content in each stored log data into a PHP array, because the serialized content in Step4 is an interface Describe information and execution status, so in this step, these contents need to be deserialized first, and merging is to merge log contents with the same identifier together according to the unique identifier of the interface. Then, according to the merged log data, the performance of each interface is analyzed in turn, and finally the description of the interface and the performance analysis report are obtained.

Specifically, the process of parsing and merging is shown in Figure 3, specifically:

read log;

Create an empty array to record the interface information. In this example, the array is recorded as apilnfo;

Parse the log item by item:

First obtain the first 32 md5 strings of the log;

In apilnfo, an empty unit is initialized with the md5 string as the key of the array;

If the log starts with the md5 string ::d:: obtained above, it means that the log records the interface description information, and the description information is stored in the corresponding unit of apilnfo after json reverse parsing;

If the log starts with the md5 string ::b:: obtained above, it means that the log records the interface start time, and the start time is stored in the corresponding unit of apilnfo;

If the log starts with the md5 string ::e:: obtained above, it means that the log records the end time of the interface, and the end time is stored in the corresponding unit of apilnfo;

If the log starts with the md5 string ::s:: obtained above, it means that the log records the interface status information, and the status information is stored in the corresponding unit of apilnfo after json deparsing;

All log records are parsed and merged according to the above rules, so as to obtain the description and status data of each external interface in each different request.

Then, according to the pre-set analysis rules and data items, perform performance analysis on each interface in turn. In this example, the performance analysis report based on the logging instance generated in Step4 is shown in Figure 4, where:

Time is the interface execution time (obtained by subtracting the start time and end time in the log)

ServiceName&HttpUrl is the interface name (extracted from the description information of the log) & the actually executed interface (extracted from the status information of the log)

Params&Header is the input parameter of the interface (extracted from the description information of the log)

Exception is the status of the interface, null means there is no error and the response is correct; otherwise, the error message of the response is displayed

In the above interface analysis, the interface name, interface input parameters, interface time consumption, and interface status are given, so that the viewer can reproduce the interface problem.

According to needs, other rules can be used to analyze other performances of the external system interfaces, such as analyzing the status of the same interface at various time periods, or analyzing the number of user visits to each interface in the time dimension, and so on.

Step6: Output and display the performance analysis report generated by Step5 on the page to be debugged: First, temporarily store the output content of the original function through the PHP buffer (in a non-PHP environment—such as java, there is no need to buffer), and then according to the original The format of the content shows the performance report, and the performance report is output together with the content in the buffer. The performance report output by the present invention is displayed according to the original content format, so it is necessary to identify the format of the original content first, if the output format of the original content is html, then add a section of html table code at the end of the original html to display the interface analysis report; if the original content is json_encode, base64 encoding, after decoding, put the interface analysis report at the end of the original content, and then encode according to the original content format. When the page to be debugged is opened through the browser, in addition to outputting the original application content, a performance analysis report in the same format as the application content is also output as shown in FIG. 5 .

For the above method, the present invention also provides a system for remote real-time analysis of external system interface performance, as shown in Figure 6, including an authentication module, a data layer public interface module, a log storage module, a log analysis module, and an output display module.

Among them, the authentication module is used to authenticate the user and control the access of the user who has the authority. The specific function can refer to Step 2 in the method; the data layer public interface module provides the encapsulation of the request external system interface, and executes the Before the interface request, record the start time and description information of the request, and record the end time and status information of the request to the log storage module after the request is executed. For the specific functions, please refer to the Step4 step in the method. The application module passes the public interface module of the data layer Access the external system interface and obtain the returned data. The application function module is a well-written PHP application function logic module; the log storage module is used to store the logs generated when the data layer public interface module accesses the external interface. In the PHP environment, the log data adopts a global Accessible PHP array implementation, the PHP array only exists in the memory when the PHP program is executed, after the web page is generated, the life cycle of the PHP process ends, and the PHP array disappears. When adding a log, directly push a unit at the end of the array; the log analysis module uses Read out the log data in the log storage module, analyze and combine them, get the log data of each request and each interface, and analyze the log data, and finally get the analysis report of each interface. For the specific functions, please refer to Step5 in the method Step: The output display module is used to output the original function content and the performance analysis report of each interface in the browser. The output format of the interface performance analysis report is consistent with the original function content format. The specific realized function can refer to Step 6 in the method.

The PHP array mentioned in the present invention is only a preferred example when quoted in the PHP environment, and should not be used as a limitation of the present invention. When applied in other web language environments, other arrays or storage formats should be adaptively used for data storage. temporary storage.

The technical means disclosed in the solutions of the present invention are not limited to the technical means disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features. It should be pointed out that for those skilled in the art, some improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also regarded as the protection scope of the present invention.

Claims (10)

1. A method for remote real-time analysis of external system interface performance, characterized in that, comprising the steps: Step A, open the webpage to be debugged in the remote environment through a browser; Step B, verify whether the current user has access rights, and only perform subsequent steps for authorized users; Step C: Execute the current application function logic. When the external system interface needs to be requested during the execution process, store log data in the memory before and after executing the interface request. The log data includes the following information of the external system interface: unique identifier, requested Start time, end time of request, external interface description information, external interface status information; Step D, read all the logs, merge the log contents with the same identifier according to the unique identifier of the interface to obtain the log records of different external interfaces, analyze the performance of each external interface according to the pre-established analysis rules, and obtain each external interface performance analysis report; Step E, output the content originally output by the application on the page, and output the performance report obtained in step D. 2. the method for remote real-time analysis external system interface performance according to claim 1, is characterized in that, in the described step C, requesting external interface process comprises the following steps: Step C-1, convert the interface name, interface input parameters, random number and/or current time to obtain a string, and generate a quasi-unique or unique identifier of limited length for the string; Step C-2, convert the interface name and interface input parameters to obtain a character string, and then store the log together with the unique identifier in the format of "unique identifier::description identifier::converted description information"; Step C-3, when starting to access the external interface, store in the log according to the format of "unique identifier::start identifier::current time"; Step C-4, execute the interface request; Step C-5, at the end of the external interface access, store the log in the format of "unique identifier::end identifier::current time"; Step C-6: Convert the state information when executing the interface request to obtain a character string, and store the log in the format of "unique identifier::state identifier::converted state information". 3. The method for remote real-time analysis of external system interface performance according to claim 2, wherein the unique identifier is generated through the following steps: after converting the interface name, interface input parameter, random number and/or current time Get a string, and then generate a finite-length quasi-unique identifier for the string. 4. The method for remote real-time analysis of external system interface performance according to claim 2 or 3, characterized in that: the conversion method is json encoding. 5. according to the method for remote real-time analysis external system interface performance described in claim 2 or 3, it is characterized in that: the method for generating the quasi-unique identification of limited length to character string is to ask md5 value to character string. 6. The method for remote real-time analysis of external system interface performance according to claim 1, characterized in that: the performance report output in the step E is in the same format as the original output content. 7. The method for remote real-time analysis of external system interface performance according to claim 1, wherein the performance report includes interface name, interface input parameters, interface time consumption, and interface status. 8. The method for remote real-time analysis of external system interface performance according to claim 1, characterized in that: said method is applied by PHP language, and also includes the following steps between step A and step C: buffering, obtaining PHP Everything that is output. 9. A system for remote real-time analysis of external system interface performance, characterized in that it includes an authentication module, a data layer public interface module, a log storage module, a log analysis module, and an output display module, and the authentication module is used for user Authentication, and only authorized users can access; the data layer public interface module provides the encapsulation of the request external system interface, and records the start time, description information, end time and status information of the request before and after the interface request is executed To the log storage module; the log storage module is used to store the log generated when the data layer public interface module accesses the external interface; the log analysis module is used to read the log data in the log storage module, and analyze the log after merging The data is analyzed to finally obtain the analysis report of each interface; the output display module is used to output the original function content in the browser and output the performance analysis report of each interface. 10. The system for remote real-time analysis of external system interface performance according to claim 9, characterized in that: the log storage module is a globally accessible PHP array, and the PHP array only exists in the memory when the PHP program is executed. Disappears after spawning.