WO2020098311A1 - Method and apparatus for resource leak detection - Google Patents

Abstract

Disclosed in the present invention are a method and an apparatus for resource leak detection. The method comprises: resource data is acquired, the resource data being sent by resource acquisition code located at a preset code position of an application program, and the resource data comprising a data operation type and a unique identifier; on the basis of the data operation type and the unique identifier of the resource data and a preset resource matching rule, matching is performed on the resource data, the resource matching rule being preset correspondence relationship between a resource request operation and a resource release operation; and if matching fails, then an abnormality is determined to be present in the resource data. Acquired resource data is matched by means of a preset correspondence relationship between a resource request operation and a resource release operation, and when matching fails, it may be determined that an abnormality is present in the resource data. Because it is not necessary to change the source code of an application program, resource leak detection is completed without the awareness of the application program, reducing the impact on corresponding application program code and improving the accuracy of resource leak detection.

Description

Method and device for detecting resource leakage

Cross-reference of related applications

This application requires the priority of the Chinese patent application filed on November 12, 2018 in the Chinese Patent Office with the application number 201811339180.0 and the application name as "a method and device for resource leak detection", the entire contents of which are incorporated by reference in this document Applying.

Technical field

Embodiments of the present invention relate to the technical field of code detection, and in particular, to a method and device for resource leak detection.

Background technique

Resource leaking refers to the application of a certain resource, such as memory, file handle, database connection, etc., during the running process, but it is not released after the use of the resource, so that the resource cannot be recycled and reused, resulting in the program occupying resources over time The growing phenomenon. Resource leaks continue to consume system resources, which can cause programs to slow down continuously. Once the leaked resources reach the upper limit of available resources, it will cause serious consequences such as application crashes, system downtime, and server paralysis.

Because resource usage is not intuitively visible to development, testing, and users, and resource leaks have no direct impact on application functions early on, before the available resources reach the upper limit, the operation of the leaked program may be completely normal, so resource leaks have been Problems are difficult to identify and locate, and once an outbreak basically has catastrophic consequences. The problem of resource leakage has become a pain point in the industry.

Summary of the invention

Embodiments of the present invention provide a method and a device for detecting resource leaks, which are used to realize that there is no need to change the source code of application programs, reduce the impact on application program codes, and improve the accuracy of resource leak detection.

A method for resource leak detection provided by an embodiment of the present invention includes:

Acquire resource data, the resource data is sent by a resource acquisition code located at a preset code position of the application; the resource data includes a data operation type and a unique identifier;

Matching the resource data according to the data operation type and unique identifier of the resource data and preset resource matching rules; the resource matching rules are the correspondence between the preset resource application operations and resource release operations;

If the matching fails, it is determined that the resource data is abnormal.

Match the acquired resource data through the preset correspondence between the resource application operation and the resource release operation. When the matching fails, it can be determined that the resource data is abnormal. Since there is no need to change the source code of the application, the application is not aware of it. Under the circumstances, the completion of resource leak detection reduces the impact on application code and improves the accuracy of resource leak detection.

Optionally, matching the resource data according to the data operation type of the resource data and preset resource matching rules includes:

If the data operation type of the resource data is a resource application operation, add the resource data to the active resource list;

If the data operation type of the resource data is a resource release operation, the active resource list is scanned to find a resource application operation that matches the unique identifier of the resource data corresponding to the resource release operation; The match is successful, otherwise it is determined that the match failed.

Optionally, the method further includes:

The active resource list is periodically scanned to check whether there is resource data in the active resource list whose time exceeds a preset threshold, and if there is, it is determined that the resource data has a resource leak.

Optionally, after determining that the resource data is abnormal, it further includes:

Delete the resource data and issue an early warning message.

Optionally, before the obtaining resource data, the method further includes:

Injecting resource acquisition code at the preset code position of the application program.

Correspondingly, an embodiment of the present invention also provides an apparatus for resource leak detection, including:

An acquiring unit, configured to acquire resource data, which is sent by a resource acquiring code located at a preset code position of an application; the resource data includes a data operation type and a unique identifier;

The processing unit is configured to match the resource data according to the data operation type and unique identifier of the resource data and a preset resource matching rule; the resource matching rule is a preset resource application operation and resource release operation Correspondence; if the match fails, it is determined that the resource data is abnormal.

Optionally, the processing unit is specifically used to:

If the data operation type of the resource data is a resource application operation, add the resource data to the active resource list;

If the data operation type of the resource data is a resource release operation, the active resource list is scanned to find a resource application operation that matches the unique identifier of the resource data corresponding to the resource release operation; if the search is successful, it is determined The match is successful, otherwise it is determined that the match failed.

Optionally, the processing unit is also used to:

The active resource list is periodically scanned to check whether there is resource data in the active resource list whose time exceeds a preset threshold, and if there is, it is determined that the resource data has a resource leak.

Optionally, the processing unit is also used to:

After determining that the resource data is abnormal, delete the resource data and issue an early warning message.

Optionally, the processing unit is also used to:

Before the resource data is acquired, the resource acquisition code is injected into the preset code position of the application program.

Correspondingly, an embodiment of the present invention provides a computer device, including:

Memory, used to store program instructions;

The processor is configured to call the program instructions stored in the memory and execute the method for detecting the resource leak according to the obtained program.

Correspondingly, an embodiment of the present invention provides a computer-readable non-volatile storage medium, including computer-readable instructions. When the computer reads and executes the computer-readable instructions, the computer is allowed to perform the above method for detecting resource leakage .

An embodiment of the present invention provides a computer program product. The computer program product includes a calculation program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, The method for causing the computer to perform the above resource leak detection.

BRIEF DESCRIPTION

In order to more clearly explain the technical solutions in the embodiments of the present invention, the drawings required in the description of the embodiments will be briefly described below. Obviously, the drawings in the following description are only some embodiments of the present invention. Those of ordinary skill in the art can obtain other drawings based on these drawings without creative work.

1 is a schematic structural diagram of a system architecture provided by an embodiment of the present invention;

2 is a schematic flowchart of a resource leak detection method according to an embodiment of the present invention;

FIG. 3 is a schematic flowchart of a resource leak detection method according to an embodiment of the present invention;

4 is a schematic flowchart of a data collection module provided by an embodiment of the present invention;

5 is a schematic flowchart of a rule matching module provided by an embodiment of the present invention;

6 is a schematic diagram of an example of code output provided by an embodiment of the present invention;

7 is a schematic structural diagram of an apparatus for detecting resource leakage according to an embodiment of the present invention.

detailed description

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present invention.

FIG. 1 exemplarily shows the structure of a system architecture to which an embodiment of the present invention is applicable. The system architecture may include a code implanting module 110, a data collection module 120, a rule matching module 130, and an early warning module 104.

The code embedding module 110 is used to embed a piece of code in the preset code position of the application program, so that the resource data can be obtained when the code reaches this point. When the application starts, it can trigger the implant function of the code implant module. The implant agent implants a piece of code at the preset code position of the application (application resource location and release resource location) before the application code is loaded. Bury the predefined anchor point according to certain requirements, so that after the application starts, each time it runs to the anchor point position, it outputs the relevant data (resource identifier and call chain) related to the current operating resource.

The data collection module 120 is used to collect the resource data output by the code embedding module 110, filter the collected resource data, and send it to the rule matching module. When an application requests resources and releases resources, the application code runs to the anchor position, the pre-injected code is executed, the current resource data is output to a certain location (such as output to a log file), and the output resource data It is discovered and collected by the data collection module and pushed to the rule matching module 130.

The rule matching module 130 may match the application-release operation according to the pre-defined rules, and regularly check whether any application operation hits the resource leakage rule. Once the leak rules are matched, the rule matching module 130 immediately pushes the suspected resource leak operation data to the early warning module 140.

After receiving the suspected resource leakage data, the early warning module 140 initiates an early warning according to a predetermined plan (such as sending a notification email, etc.).

Based on the above description, FIG. 2 exemplarily shows a flow of a method for detecting a resource leak provided by an embodiment of the present invention, and the flow may be executed by an apparatus for detecting a resource leak.

As shown in Figure 2, the process specifically includes:

Step 201: Obtain resource data.

In the embodiment of the present invention, the resource data is sent by the resource acquisition code located at the preset code position of the application. Therefore, before acquiring the resource data, the resource acquisition code needs to be injected at the preset code position of the application . The preset code location may be the application resource location and the release resource location in the application code. The resource data may include a data operation type and a unique identifier, and the data operation type may include a resource application operation and a resource release operation.

Step 202: Match the resource data according to the data operation type and unique identifier of the resource data and preset resource matching rules.

The resource matching rule is the correspondence relationship between the preset resource application operation and the resource release operation. Specifically, after the resource data is obtained, if the data operation type of the resource data is a resource application operation, the resource data is added to the active resource list. If the data operation type of the resource data is a resource release operation, scan the active resource list to find an application for a resource operation that matches the unique identifier of the resource data corresponding to the resource release operation; if the search is successful, it is determined that the match is successful, indicating the application- If the release match is successful and the corresponding resource application operation has no leakage problem, it can be removed from the active resource list. When the search is unsuccessful, it is determined that the match fails.

Step 203: If the matching of the resource data fails, it is determined that the resource data is abnormal.

In the above step 203, after it is determined that the matching of the resource data fails, it can be determined that the resource data has an abnormality, which is equivalent to the resource release operation having no corresponding resource application operation, and the abnormal release information needs to be analyzed.

It should be noted that the above active resource list may also be scanned periodically to check whether there is resource data in the active resource list whose time exceeds a preset threshold, and if so, it is determined that there is a resource leak in the resource data. The preset threshold can be set according to experience. The periodicity can be set according to experience, for example, 10 minutes, 1 hour, 2 hours, etc.

When it is determined that there is a resource leak in the resource data, the resource data can be deleted and an early warning message is issued.

In order to better explain the embodiments of the present invention, the following will be described through specific implementation scenarios:

As shown in the process shown in FIG. 3, when the application starts, the implant function of the code implant module is triggered, and the implant agent is implanted at the preset application code location (resource application location and release resource location) before the application code is loaded. A piece of code to bury a predefined anchor point according to certain requirements, so that after the application starts, each time it runs to the anchor point, it outputs data related to the current operating resource (resource identifier and call chain).

When the application requests resources and releases resources, the application code runs to the anchor point, the pre-injected code is executed, the current resource data is output to a location (such as output to a log file), and the output resource data is collected by data The module discovers and collects and pushes it to the rule matching module.

The rule matching module matches the application-release operation according to the pre-defined rules, and regularly checks whether any application operation hits a resource leakage rule. Once the leak rule is matched, the rule matching module immediately pushes the suspected resource leak operation data to the early warning module.

After receiving the suspected resource leakage data, the early warning module initiates an early warning according to a predetermined plan (such as sending a notification email, etc.).

Among them, the code implantation module mainly solves two problems: code implantation location and implantation content.

The implantation location must contain two points, the application resource location and the release resource location, and the actual location of the resource application and release needs to be located according to the difference between the actual application and the detected resource type. The code embedding module completes code embedding according to this position.

The content of the implanted code is mainly to obtain and output the current resource information, and the resource information includes the unique identifier of the resource and the call chain. For example, when monitoring the database connection pool, the content of the implanted code may be: output database connection instance information + current call stack information to the log.

The data collection module is responsible for identifying the valid data related to resource leak detection from the numerous data output by the application and pushing it to the rule matching module. The specific steps performed by the data collection module may be as shown in FIG. 4 and specifically include:

Step 401, new data is found.

Regularly listen to the output data of the application to detect whether there is new data. If new data is found, read it line by line.

Step 402: Determine whether it is resource application or resource release data. If yes, go to step 404. If no, go to step 403.

Check whether the data is related to resource application / release, otherwise discard it.

Step 403, discard the data.

Step 404, data verification.

Verify data, check format, integrity, etc.

In step 405, it is judged whether it is valid data. If yes, go to step 407; if no, go to step 406.

Step 406, discard the data.

Step 407: Push to the rule matching module.

The valid data that passes the verification is pushed to the rule matching module, and the invalid data is discarded.

The rule matching module in FIG. 3 is responsible for rule matching on resource operation data, promptly discovering suspicious operations with risk of leakage, and pushing them to the early warning module.

The process shown in Figure 5 includes:

Step 501, resource data is reached.

Step 502, it is determined whether the data operation type of the resource data is a resource application operation, if yes, then go to step 503, if not, go to step 504.

Step 503: Add resource data to the active resource list.

Step 504: Determine whether the data operation type of the resource data is a resource release operation. If yes, go to step 506. If no, go to step 505.

Step 505, discard the data.

Step 506: Scan the active resource list.

In step 507, it is determined whether the resource data corresponding to the resource release operation matches with an active resource. If yes, then go to step 509, if not, go to step 508.

Step 508, the resource data corresponding to the resource release operation is stored in the database.

Step 509, remove the active resource from the active resource list.

In step 510, the scan task is scheduled.

Scan the active resource list regularly.

Step 511: Determine whether the active resource times out. If yes, go to step 513. If no, go to step 512.

Step 512, reserve active resources.

In step 513, the timeout resource information is stored in the database.

Step 514: Remove the active resource from the active resource list.

In step 515, early warning is initiated.

The rule matching module maintains an active resource information list;

When the resource operation data arrives, it is determined whether the data is a resource application operation or a resource release operation.

If it is a resource application operation, the data of the applied resource is added to the active resource list.

If it is a resource release operation, scan the active resource list to find the application resource operation that is consistent with the unique identifier of the released resource. If the search is successful, it means that the application-release match is successful, and the corresponding application resource operation has no leakage problem, so Remove it from the list of active resources; if the search fails, record the resource release operation into the library for analysis of abnormal information.

The predefined scheduled scan task scans the active resource list every t time to check whether there are any active resources that exist longer than the maximum active time threshold T. If there are, it is considered that there is a resource leak problem in the operation of applying for resources. The operation record is stored in the database, the data is removed from the active resource list, and the early warning module is called to initiate the early warning.

The early warning module receives the suspected leaked data pushed by the rule matching module, sends early warning information to stakeholders through email, web page push, etc. according to a predetermined plan, and pushes the leaked resource information and the leaked program call chain. Staff can check the source code for leaks based on resource information and call chains and fix them.

In another implementation of the embodiment of the present invention, in the database connection pool leak detection, the code implantation module implements a method for applying for a database connection (getConnection ()) and a method for releasing a database connection (returnConnection ()) implemented at the bottom of the connection pool ) Into the code, so that each time these two methods are called, the unique identifier of the database connection (Connection) currently performing the apply / release (get / return) operation and the call stack information of the current operation code are output in the log. The actual output effect is shown in Figure 6.

Among them, "[get]" indicates that this is an application for database connection operation, followed by the call stack information of the current operation, and "detectionInsertConInfo" is followed by the current database information, which can uniquely locate a database connection. The output of the release database connection operation is similar to this, the only difference is that "[get]" is replaced with "[return]".

The data matching module scans the log in real time, and determines whether it is a database monitoring related log according to whether there is a "detection insert" code (as shown in Figure 6) in the log, and after verification, the effective monitoring log is pushed to the rule matching module.

The rule matching module receives the database monitoring log data, and if it is a get operation, it is added to the active connection list; if it is a return operation, it scans the active connection list based on the unique identifier of the Connection, detectionInsert ConInfo, to find the get that matches the return operation Operation, completing the matching get operation is regarded as the operation without risk of database connection leakage, and it is removed from the active connection list, and other get operations are kept in the active connection list. Through experiments, it is found that most database operations are in milliseconds, that is, after a get operation occurs, there will be a return operation matching it within a few milliseconds, so the timeout period is set to 1 hour in the scheduled scan task. If a get operation has not been matched by a return operation for more than 1 hour, we believe that a database connection leak is suspected for this get operation. Therefore, the early warning module sends the monitoring output data shown in FIG. 6 as early warning information to the stakeholders by email, and the stakeholders locate the leaked code location through the call stack for troubleshooting and repair.

At present, after the database connection pool leak detection method is fully deployed in the test environment, multiple database connection leak problems have been accurately found and accurately located in multiple business systems such as UnionPay cardless routing management system, merchant management system, risk system, etc. The code snippet and zero false alarms throughout the process have helped various business systems discover and eliminate the risk of multiple connection leaks, achieving very good practical results.

The embodiment of the present invention uses code dynamic implantation technology to bury anchor points at the locations where the resources are actually applied and released, so that when the application runs to these locations, the information of the requested or released resources and the current code are output in real time according to the runtime context The call chain matches the application-release data to determine whether a resource is properly released after use, to achieve the purpose of checking for resource leaks, and accurately locates the location of the resource leak based on the code call chain for the operation of the requested resource.

Based on the same technical concept, FIG. 7 exemplarily shows an apparatus for resource leak detection provided by an embodiment of the present invention. The apparatus can execute a process of resource leak detection.

As shown in Figure 7, the device includes:

The obtaining unit 701 is configured to obtain resource data sent by a resource obtaining code located at a preset code position of an application program; the resource data includes a data operation type and a unique identifier;

The processing unit 702 is configured to match the resource data according to the data operation type and unique identifier of the resource data and a preset resource matching rule; the resource matching rule is a preset resource application operation and a resource release operation Corresponding relationship; if the matching fails, it is determined that the resource data is abnormal.

Optionally, the processing unit 702 is specifically configured to:

If the data operation type of the resource data is a resource application operation, add the resource data to the active resource list;

If the data operation type of the resource data is a resource release operation, the active resource list is scanned to find a resource application operation that matches the unique identifier of the resource data corresponding to the resource release operation; if the search is successful, it is determined The match is successful, otherwise it is determined that the match failed.

Optionally, the processing unit 702 is also used to:

The active resource list is periodically scanned to check whether there is resource data in the active resource list whose time exceeds a preset threshold, and if there is, it is determined that the resource data has a resource leak.

Optionally, the processing unit 702 is also used to:

After determining that the resource data is abnormal, delete the resource data and issue an early warning message.

Optionally, the processing unit 702 is also used to:

Before the resource data is acquired, the resource acquisition code is injected into the preset code position of the application program.

Based on the same technical concept, embodiments of the present invention provide a computer device, including:

Memory, used to store program instructions;

The processor is configured to call the program instructions stored in the memory and execute the method for detecting the resource leak according to the obtained program.

Based on the same technical concept, an embodiment of the present invention provides a computer-readable non-volatile storage medium, including computer-readable instructions, which causes the computer to perform the above-mentioned resource leak when the computer reads and executes the computer-readable instructions Method of detection.

Based on the same inventive concept, embodiments of the present application also provide a computer program product, the computer program product includes a calculation program stored on a non-transitory computer-readable storage medium, the computer program includes program instructions, when the When the program instructions are executed by a computer, the computer is caused to perform the above-mentioned method for detecting resource leaks.

The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram and a combination of the flow and / or block in the flowchart and / or block diagram may be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, special-purpose computer, embedded processing machine, or other programmable data processing device to produce a machine that enables the generation of instructions executed by the processor of the computer or other programmable data processing device An apparatus for realizing the functions specified in one block or multiple blocks of one flow or multiple flows of a flowchart and / or one block or multiple blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, the instructions The device implements the functions specified in one block or multiple blocks of the flowchart one flow or multiple flows and / or block diagrams.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operating steps are performed on the computer or other programmable device to produce computer-implemented processing, which is executed on the computer or other programmable device The instructions provide steps for implementing the functions specified in one block or multiple blocks of the flowchart one flow or multiple flows and / or block diagrams.

Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. In this way, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and equivalent technologies thereof, the present invention is also intended to include these modifications and variations.

Claims (13)

A method for resource leak detection, which includes: Acquire resource data, the resource data is sent by a resource acquisition code located at a preset code position of the application; the resource data includes a data operation type and a unique identifier; Matching the resource data according to the data operation type and unique identifier of the resource data and preset resource matching rules; the resource matching rules are the correspondence between the preset resource application operations and resource release operations; If the matching fails, it is determined that the resource data is abnormal. The method according to claim 1, wherein the matching the resource data according to the data operation type of the resource data and a preset resource matching rule includes: If the data operation type of the resource data is a resource application operation, add the resource data to the active resource list; If the data operation type of the resource data is a resource release operation, the active resource list is scanned to find a resource application operation that matches the unique identifier of the resource data corresponding to the resource release operation; if the search is successful, it is determined The match is successful, otherwise it is determined that the match failed. The method of claim 2, wherein the method further comprises: The active resource list is periodically scanned to check whether there is resource data in the active resource list whose time exceeds a preset threshold, and if there is, it is determined that the resource data has a resource leak. The method according to any one of claims 1 to 3, wherein after determining that the resource data has an abnormality, further comprising: Delete the resource data and issue an early warning message. The method according to any one of claims 1 to 3, wherein before the acquiring resource data, the method further comprises: Injecting resource acquisition code at the preset code position of the application program. An apparatus for detecting resource leaks is characterized by comprising: An acquiring unit, configured to acquire resource data, which is sent by a resource acquiring code located at a preset code position of an application; the resource data includes a data operation type and a unique identifier; The processing unit is configured to match the resource data according to the data operation type and unique identifier of the resource data and a preset resource matching rule; the resource matching rule is a preset resource application operation and resource release operation Correspondence; if the match fails, it is determined that the resource data is abnormal. The device according to claim 6, wherein the processing unit is specifically configured to: If the data operation type of the resource data is a resource application operation, add the resource data to the active resource list; If the data operation type of the resource data is a resource release operation, the active resource list is scanned to find a resource application operation that matches the unique identifier of the resource data corresponding to the resource release operation; if the search is successful, it is determined The match is successful, otherwise it is determined that the match failed. The apparatus according to claim 7, wherein the processing unit is further configured to: The active resource list is periodically scanned to check whether there is resource data in the active resource list whose time exceeds a preset threshold, and if there is, it is determined that the resource data has a resource leak. The device according to any one of claims 6 to 8, wherein the processing unit is further used to: After determining that the resource data is abnormal, delete the resource data and issue an early warning message. The device according to any one of claims 6 to 8, wherein the processing unit is further used to: Before the resource data is acquired, the resource acquisition code is injected into the preset code position of the application program. A computer device, characterized in that it includes: Memory, used to store program instructions; The processor is configured to call the program instructions stored in the memory and execute the method according to any one of claims 1 to 5 according to the obtained program. A computer-readable non-volatile storage medium, characterized in that it includes computer-readable instructions, and when the computer reads and executes the computer-readable instructions, it causes the computer to execute any one of claims 1 to 5. Methods. A computer program product, characterized in that the computer program product includes a calculation program stored on a non-transitory computer-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by a computer, the The computer executes the method according to any one of claims 1 to 5.

Images