CN106294136A - The online test method of concurrent program run duration performance change and system - Google Patents

Abstract

A method and system for online detection of performance changes during parallel program running. The online detection method includes: obtaining the program structure diagram; based on the program structure diagram, inserting the runtime performance change detection code at the node of the program, the performance change detection code includes the in-process performance change detection code, and the in-process performance change detection code can be used in the program During execution, for multiple iterations of a loop node in the program, indicate which iteration or iterations have changed in performance relative to the performance of other iterations in history. The online detection method and system of the embodiment of the present invention can detect the performance failure of the operating environment when the program is running; automatically detect the performance change, and report it to the developer in time after detection; and can use the nodes in the program structure diagram as The basic unit of analysis is to automatically or custom-select the code to be analyzed, without analyzing all parts of the entire program, thereby reducing the runtime performance overhead.

Description

Online detection method and system for performance change during parallel program running

technical field

The present invention relates generally to software development, and more particularly to an online detection technique for performance changes during parallel program execution.

Background technique

Large-scale parallel computing based on high-performance computers is widely used in social production and life such as weather forecasting, oil exploration, and drug research and development.

Modern high-performance computers contain many processors (CPUs), and each CPU has many cores (cores). In order to make full use of computing power, parallel programs allow these processors/computing cores to work together in a certain way. In the field of high-performance computing, the Message Passing Interface (MPI) is the de facto standard model for parallel programs; C/C++ and Fortran are commonly used programming languages. In such a parallel program, the parallel program has several processes, and each process performs its own computing tasks internally, and the processes interact through network communication.

With the increasing size of high-performance computers and parallel programs, performance variation (that is, programs running fast and slow) has become a more and more serious problem.

Traditionally, the following technology (hereinafter referred to as existing scheme 1) has existed: use pre-constructed standard test programs to perform performance evaluation on each node of the high-performance computer, find out the computing nodes with performance failures, Avoid running parallel programs on this node. For this, refer to: Problem Diagnosis in Large-Scale Computing Environments, Mirgorodskiy et al., SC2006.

A disadvantage of this traditional technical solution is that it can only be pre-screened before the program is running, and cannot deal with performance failures that occur during program running. At the same time, it only focuses on system performance failures and ignores the analysis of program behavior.

Traditionally, there is also the following technology (hereinafter referred to as the existing solution 2): taking functions as the granularity, recording the time consumed by each function call into a file, and using a visualization tool to analyze the performance after the program ends. Variety. For this, please refer to the literature: Vampir, ScoreP, SCALASCA, literature: "Tools for High Performance Computing", Wolf et al., 2008.

The disadvantage of this type of method is that the recording of function time will generate a large number of output files, which brings several problems. First, a large number of output files will cause great pressure on the system and affect the performance of the program; second, the programmer The analysis needs to be carried out after the program ends, and the program may take a long time to run; the third is that it takes time and effort to find useful information from a large number of output files.

How to detect the performance changes of the program while the program is running (ie "online") without introducing too much performance overhead (ie "lightweight") is an urgent and challenging problem to be solved.

Contents of the invention

The performance change of parallel programs may come from many aspects. For example, when multiple parallel programs run on the same high-performance computer, there will be competition for shared resources; another example is the interference caused by the operation of the operating system itself; or some hardware or software The impact of the fault on the program.

The performance variation of parallel programs brings a series of problems. The first is that the performance of the program is compromised, lower than expected. What's more serious is that due to the uncertainty of program performance changes, the same program will have different performances when it is run multiple times on the same computer system, which makes it difficult for developers to understand the real behavior of the program. For example, on a typical high-performance computer, running the same program multiple times may have a 10% performance change, and the optimization of the program is expected to bring about a 5% performance improvement. In this case, the developer cannot judge the performance of the program. Does the improvement come from program optimization, or simply due to performance changes.

It is hoped that the performance change of the program can be detected during the running of the program (ie "online") without introducing too much performance overhead (ie "lightweight"), and it can give a hint of the source of the performance change.

In view of the circumstances, the present invention has been made.

According to one aspect of the present invention, there is provided an online detection method for performance changes during the running of a parallel program, which may include: obtaining a program structure diagram; based on the program structure diagram, automatically or customizedly inserting runtime performance at the nodes of the program Change detection codes, the performance change detection codes include in-process performance change detection codes, the in-process performance change detection codes can point out relative to the historical The performance of other iterations of , which iteration or iterations the performance of which changes.

Further, in the online detection method, which iteration(s) have changed performance may include which iteration(s) have worse performance than other iterations in history.

Further, in the online detection method, the in-process performance change detection code can be configured to compare the performance of the current iteration with the performance of the previous iteration in the program running process during the running of the program, record the performance change information in the process, and report to the user Gives an in-process performance change report.

Further, in the online detection method, the performance change detection code may also include an inter-process performance change detection code, including collecting the in-process data of each parallel process running the same task by one of the parallel processes running the same task in the parallel program Performance change information, compare, and give the user a comparative report on the performance change between processes.

Furthermore, in the online detection method, the program structure diagram can be composed of several trees, and each tree has three nodes in the structure diagram for a function definition body in the code: loop LOOP, branch BR and call CALL, each node A point is defined by at least three attributes: a globally unique ID; the type of the node, whether it is a loop, a branch, or a call; the position of the node in the source code, the performance change detection code in the process to The user reports iterations indicating the location of loop nodes in the source code and performance changes meeting predetermined criteria.

Furthermore, in the online detection method, it may also include, before the runtime performance change detection code is inserted, clipping the program structure diagram.

Further, in the online detection method, pruning may include automatically removing nodes whose time-consuming proportion is less than a predetermined threshold from the program structure graph, so that no runtime performance change detection code will be inserted into such nodes.

Further, in the online detection method, the in-process performance change information may include: the sum of the execution time of the node in history; the total number of executions of the node in history; the longest execution time of the node in history. The iteration number and time of .

Further, in the online detection method, the inter-process performance change may include: judging whether the average running time of one or some processes is greater than the average running time of other processes by a predetermined degree.

Further, in the online detection method, the inter-process performance change may include: judging whether the performance change of one or some iterations of the loop node occurs in each process.

According to another aspect of the present invention, there is provided an online detection system for performance changes during the running of a parallel program, which may include: a device for obtaining a program structure diagram, configured to obtain a program structure diagram; an insertion device configured to, based on the program structure diagram, Automatically or custom-instrumented runtime performance change detection codes at the nodes of the program, the performance change detection codes include in-process performance change detection codes, and the in-process performance change detection codes can be used for the program during the running process of the program Multiple iterations of the loop node in , indicating which iteration(s) have changed performance relative to the performance of other iterations in history.

Further, in the online detection system, which iteration(s) have changed performance may include which iteration(s) have worse performance than other iterations in history.

Furthermore, in the online detection system, the in-process performance change detection code can be configured to compare the performance of the current iteration with the performance of previous iterations during the program running, record the in-process performance change information, and report to the user Gives an in-process performance change report.

Further, in the online detection system, the performance change detection code may also include inter-process performance change detection code, and the inter-process performance change detection code can be collected by one of the parallel processes running the same task in the parallel program. Intra-process performance change information of parallel processes is compared, and a comparison report on inter-process performance changes is given to the user.

Furthermore, in the online detection system, the program structure diagram can be composed of several trees, and each tree has three kinds of nodes in the structure diagram for a function definition body in the code: loop LOOP, branch BR and call CALL, each node A point is defined by at least three attributes: a globally unique ID; the type of the node, whether it is a loop, a branch, or a call; the position of the node in the source code, the performance change detection code in the process to The user reports iterations indicating the location of loop nodes in the source code and performance changes meeting predetermined criteria.

Furthermore, the online detection system may also include, before the performance change detection code is inserted during runtime, clipping the program structure diagram.

Furthermore, in the online detection system, pruning may include automatically removing nodes whose time-consuming ratio is less than a predetermined threshold from the program structure graph, so that no runtime performance change detection code will be inserted into such nodes.

Further, in the online detection system, the in-process performance change information may include: the sum of the execution time of the node in history; the total number of execution times of the node in history; the longest execution time of the node in history. The iteration number and time of .

Further, in the online detection system, the inter-process performance change may include: judging whether the average running time of one or some processes is greater than the average running time of other processes by a predetermined degree.

Further, in the online detection system, the inter-process performance change may include: judging whether the performance change of one or some iterations of the loop node occurs in each process.

According to another aspect of the present invention, there is provided a computing device, which may include a central processing unit and a memory storing computer-executable code which, when executed by the central processing unit, performs the following method: obtaining a program structure Figure; and based on the program structure diagram, automatically or customizedly insert runtime performance change detection codes at the nodes of the program, the performance change detection codes include in-process performance change detection codes, and the in-process performance change detection codes can During the running of the program, for multiple iterations of the loop node in the program, indicate which iteration or iterations have changed in performance relative to the performance of other iterations in history.

Compared with the existing solutions, the parallel program performance change online detection solution in the embodiment of the present invention has three main advantages:

(1) The performance failure of the operating environment can be detected when the program is running, unlike the existing scheme 1, which can only be detected before the program runs;

(2) It can automatically detect performance changes and report to developers in a timely manner after detection, without waiting for the program to complete like the existing solution 2, which is necessary for long-running parallel programs (in reality programs may run for hours or days at a time);

(3) Based on the program structure diagram, with the nodes in the program structure diagram as the basic unit of analysis, the code to be analyzed can be automatically or customized, and it is not necessary to analyze all parts of the entire program, thus reducing the running time performance overhead.

Description of drawings

These and/or other aspects and advantages of the present invention will become clearer and easier to understand from the following detailed description of the embodiments of the present invention in conjunction with the accompanying drawings, wherein:

FIG. 1 shows a general flowchart of an online detection method 100 for performance changes during parallel program running according to an embodiment of the present invention.

Fig. 2 schematically shows an exemplary source program code.

FIG. 3 shows a program structure diagram constructed by taking the sample code in FIG. 2 as an example through the automatic analysis of the compiler.

FIG. 4 shows a flowchart of a method 110 for obtaining a program structure diagram from a source program according to an embodiment of the present invention.

FIG. 5 shows the program after instrumenting the function call compute in the sample source program shown in FIG. 2 and outside the overall loop.

Fig. 6(a) exemplarily shows a schematic diagram of a history-based internal process performance change detection method for node 1: CALL(5) in the program of Fig. 5 .

Fig. 6(b) exemplarily shows a schematic diagram of a method for analyzing performance changes among various processes.

7( a )-( c ) are schematic diagrams showing the analysis of in-process performance changes during parallel program running according to an embodiment of the present invention.

(a) and (b) in FIG. 8 show a schematic diagram of a performance change analysis method among processes.

FIG. 9 shows a logic block diagram of a system 200 for online detection of performance changes during parallel program execution according to another embodiment of the present invention.

detailed description

In order to enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

First, some terms involved in this article are explained.

"Changed in performance", "changed in performance", "worse than": means that the performance "changes", "changes" or "worse than" reaches a predetermined level or is the top predetermined ranking on the determined index.

Program structure: In the program source code, in addition to the sequentially executed calculation code, there are usually control structures such as loops, branches, and function calls. The "program structure" in this article refers to the position and hierarchical relationship of the code control structure in the source code.

Compiler-based source code analysis: The source code of a program written in a high-level language must be translated into machine language by a compiler before it can be executed on a computer. The program structure information of the source code is obtained in the browser.

Automatic source code insertion: Use the program to add some extra code to the source code according to certain rules, so that the modified program can meet some specific goals (such as outputting certain information during operation).

Process: A process can be understood as a "running program instance". For example, now there is an executable file a.exe, every time this program is run, a corresponding process will be generated; if many computers or CPUs run a.exe, there will be many a.exe processes. Parallel programming means that many processes work together at the same time. Regarding task division, it is generally the way of single program and multiple data streams. For example, the three processes have to complete 10 iterations, but these processes do different things, just like three people have to do 10 test papers, one person does 10 English test papers, and one person does 10 math test papers, The rest do 10 Chinese test papers. In other words, the code executed by each process is the same, but the data processed is different.

Performance variance: multiple iterations of a loop node within a process, the performance of a certain iteration is different from the historical iteration; or, for multiple processes, the performance of one or some processes is different from other processes .

Before describing in conjunction with the embodiments, in order to make it easier for those skilled in the art to grasp the present invention, the inventive concept of the present invention is firstly outlined. The present invention adopts a combination of dynamic and static methods. During source code compilation, a compiler is used to analyze the code structure of the modified program, construct a program structure graph (Program Structure Graph, PSG), and perform automatic source code insertion based on the structure graph; During the running of the program, the instrumented source code is executed to collect and analyze the performance data of the program, so as to detect the performance change of the parallel program.

FIG. 1 shows a general flowchart of an online detection method 100 for performance changes during parallel program running according to an embodiment of the present invention.

In step S110, a program structure diagram is obtained.

As an example, Fig. 2 schematically shows an exemplary source program code. FIG. 3 shows a program structure diagram constructed by taking the sample code in FIG. 2 as an example through the automatic analysis of the compiler.

A program structure graph consists of several trees, and each tree corresponds to a function definition body in the code. There are three kinds of nodes in the structure diagram: loop LOOP, branch BR and call CALL. Arrows indicate the containment relationship of each node. Each node can have three basic properties:

1. A globally unique ID is obtained by traversing the tree in the program structure diagram, and has no direct relationship with the source code location mentioned in the third point below;

2. Record the type of the node, whether it is LOOP (loop), BR (branch), or CALL (function call);

3. Record the position of the node in the source code. In the example shown in FIG. 3 , it shows the starting line number. In one example, richer location information such as the program's file name, start and end line and column numbers, etc. can also be recorded.

For example, in the example shown in Figure 3, node 0: LOOP (3), means that the global node ID is 0, the node type is LOOP (loop), and the starting line number is the third line in the code; node 1:CALL(5), indicating that the global node ID is 1, the node type is CALL (function call), and the starting line number is the fifth line in the code; 2:BR(6), indicating the global node ID It is 2, the node type is BR (branch), and the starting line number is line 6 in the code.

FIG. 4 shows a flowchart of a method 110 for obtaining a program structure diagram from a source program according to an embodiment of the present invention.

In step S111, the source code of the program is automatically analyzed by a compiler to obtain an original program structure graph PSG.

Different from "Existing Solution 2", the technical solution according to an embodiment of the present invention does not blindly record the time information of each call of all functions, but cuts the program structure diagram according to a certain rule, only the key The node record information.

The pruning rules can be generated automatically, provided by users, or a combination of both.

In the case that the user provides the pruning rules, the user may list the IDs of the nodes to be kept or removed in a configuration file. Therefore, the original program structure graph PSG is clipped according to such a node ID list.

In the case of automatic cropping, the program can be run once first, and then according to the running time, nodes with a small proportion of time consumption (such as nodes whose time consumption is less than 1% of the entire program time) are removed. Here, run the program once first, which can be a small-scale operation. For example, the actual problem to be solved needs to start 10,000 processes to calculate for 24 hours, but in order to cut PSG, first use 100 processes to calculate for 10 minutes, and you can get The performance information of most nodes in the PSG is clipped.

After clipping the program structure diagram, the next step is to reconstruct the program structure diagram. The main process is the merging of each structure tree in the structure diagram and the adjustment of the relationship between nodes to meet the needs of later runtime analysis.

Cutting and reconstructing the program structure diagram according to certain rules can reduce the impact on the performance of the program when it is finally run, and improve the readability of the results.

Returning to Fig. 1, after step S110 is completed, proceed to step S120.

In step S120, based on the program structure diagram, the runtime performance change detection code is automatically or customized at the node of the program, the performance change detection code includes an in-process performance change detection code, and the in-process performance change detection code During the running of the program, for multiple iterations of the loop node in the program, it can be pointed out which iteration or iterations have changed in performance compared to the performance of other iterations in history.

After obtaining the program structure diagram, the program source code is instrumented according to the structure diagram. The purpose of instrumentation is to provide performance information of the nodes in the structure graph at runtime without changing the original algorithm of the program.

In an example, the performance change detection code further includes an inter-process performance change detection code, comprising collecting in-process performance change information of each parallel process running the same task by one of the parallel processes running the same task in the parallel program, A comparison is made and a comparison report is given to the user about performance changes between processes.

FIG. 5 shows the program after instrumenting the function call compute in the sample source program shown in FIG. 2 and outside the overall loop.

vtx_in(1,CALL): Pre-insert point, record the information when the compute node starts to be executed. The parameters of Vtx_in include node ID and node type.

vtx_out(1,CALL): After the insertion point, record the information when the compute is finished. The in-process history-based analysis is done in the vtx_out function.

GLOBAL_ANALYSIS(): The post-loop insertion point, which performs inter-process analysis after all iterations of the loop are completed, including: 1. A certain process collects data from other processes; 2. The process analyzes global performance changes.

As shown in Figure 5, the instrumentation functions vtx_in and vtx_out are added before and after the compute function, which are responsible for recording the start of the compute function call (that is, the (1:CALL(5)) node in the structure diagram shown in Figure 3) end message. The history-based in-process analysis mentioned above is done in vtx_out

As shown in Figure 5, the instrumentation code GLOBAL_ANALYSIS() is also added outside the loop (line 10) for inter-process analysis. Inter-process analysis needs to collect the locally collected data of each process. In an example, this is done by the process with the smallest process number, and no additional process needs to be added, which can ensure that the instrumented program can run in the original environment.

The instrumented code can be compiled and run by the user using an existing compiler, and the performance change of the program can be detected during runtime, and this performance change analysis can be provided to the user in an appropriate way, for example, it can be displayed in on the display screen, or give the user a voice prompt.

The change in performance of the iteration(s) may include the iteration(s) performing worse than other iterations historically.

The in-process performance change detection code can be configured to compare the performance of the current iteration with the performance of previous iterations during the program running, record the in-process performance change information, and give the user a report of the in-process performance change.

The in-process performance change information may include: the sum of the execution time of the node in history; the total number of executions of the node in history; the corresponding iteration number and time of the predetermined number of executions with the longest execution time of the node in history.

In the following, the intra-process performance change detection and the inter-process performance change detection based on the program structure diagram will be exemplarily and schematically described with reference to FIGS. 6( a ) and 6 ( b ).

Fig. 6(a) exemplarily shows a schematic diagram of a history-based internal process performance change detection method for node 1: CALL(5) in the program of Fig. 5 .

If a PSG node is in a loop, the information it performs in each iteration should be similar. The online detection solution according to an embodiment of the present invention records the information of each iteration of the loop internal structure graph node, and compares it with the historical information after each execution, so as to find some abnormal performance changes. After a performance change is discovered, the tool can immediately report the performance change instance, instead of waiting until the entire program runs to the end like the "existing solution 2".

Specifically, for the example shown in Fig. 6(a), for the node 1: CALL(5) shown in Fig. 5, which is in the loop node 0: LOOP(3), when the inserted code is executed Monitor node 1: The information of each iteration of CALL(5) is compared with the historical information after each execution, so that abnormal performance changes can be found, for example, the number of iterations that run significantly slower is found and Record it, and you can also record the total number of iterations, the average running time of each iteration, and so on. As shown in Figure 6(a), shaded rectangles indicate iterations with abnormal performance changes. If each iteration number is 0, 1, 2,..., as shown in the figure, the performance of the iteration numbered 2 is significantly worse. Its number and running time are recorded and reported to the user during the running of the program, so that the user can know in time when the running program is abnormal, so as to analyze the cause and find out the crux.

Fig. 6(b) exemplarily shows a schematic diagram of a method for analyzing performance changes among various processes.

The in-process performance change detection described above only focuses on the performance change of a certain process at a certain moment, but does not know the global information such as how many processes have changed in performance, how large the change is, and so on. Global performance change analysis can be achieved by comparing the performance changes detected within each process. Such a global performance change analysis can give some hints about the source of the performance change. For example, if only a small number of concentrated processes are experiencing performance changes, while others are normal, then the environment in which these processes are running may have changed, and there may be some performance failure, which detects "existing scenario 1" An undetectable system performance failure that occurs at runtime. And if the performance of all global processes changes at a certain moment, there may be some global performance interference, such as sudden network congestion.

As shown in Figure 6(b), for process 0, process 1, ... process i ..., process n, record the representative performance change parameters in each process, and then compare the performance changes detected in each process to obtain Analyze and estimate the global distribution of performance changes. For example, the iteration number for which the performance of process 0 deteriorates is recorded as 2, the running time of iteration number 2, the overall number of iterations for process 0, and the average running time of iterations; and similarly, the iteration number for which the performance of process i is deteriorated is recorded as 1. The running time of iteration No. 1, the overall number of iterations of process i, and the average running time of iterations; ...; record the iteration number 3, the running time of iteration No. 3, and the overall number of iterations of process n0. , Iteration average running time. A more specific exemplary description of how to analyze performance changes between processes will be given later.

According to the online detection method of performance change during parallel program running in the embodiment of the present invention, during parallel program running, the performance change in the process is detected in real time, and the user is prompted about the performance change in the process, so that the user can be informed in time key clues and information, avoiding traditional users to spend a lot of time and effort to find clues in the vast records and reports; and based on the program structure diagram, with the nodes in the program structure diagram as the basic unit of analysis, it can automatically or automatically The code to be analyzed is selected in a defined manner, and all parts of the entire program do not need to be analyzed, thereby reducing the performance overhead at runtime.

An example of an analysis of in-process performance changes during the running of a parallel program according to an embodiment of the present invention will be described below with reference to FIGS. 7(a)-(c).

As mentioned earlier, runtime analysis is based on a diagram of the program structure. In one example, during the source code instrumentation process, instrumentation codes are inserted before and after each node in the program structure graph, so that the performance information of each PSG node when the program is running can be obtained, for example, including: execution time , the number of executions, and so on.

In-process performance change analysis is based on historical information, and the number of loops in the actual program may be very large. Therefore, it is preferable not to directly save all the iteration information, but to select some representative data in the history for recording. Preferably, the recorded information may include:

(1) The sum of the execution time of the node in history;

(2), the total number of times the node has been executed in history;

(3) The number of iterations and time corresponding to several executions of the node with the longest execution time (or the slowest execution) in history.

Specifically, Fig. 7 (a) shows exemplary source code, will monitor the performance change of the bar function in this cycle; Fig. 7 (c) left side shows the corresponding program structure chart of this source code, The in-process performance change of node 1: CALL (3) will be monitored. As mentioned earlier, the "1" in "1: CALL (3)" indicates that the global node ID is 1, and the node type is CALL (function call ), the starting line number is line 3 in the code. Figure 7(b) shows the in-process execution history of node 1: CALL(3), which shows that the bar function has been executed 10 times, and the numbers of each iteration are 0, 1, 2, 3, 4, 5, 6, 7 . The corresponding iteration times are 3, 5, and 3, but not all 10 execution times are recorded, but the total number of times (10), total execution time (19) and the slowest three executions are recorded, among which the slowest three Execution is recorded in the format of (iteration number: time), for example (4:5) means that the execution time of iteration 4 is 5, as shown on the right side in Figure 7(c).

An example of a method for analyzing performance changes between processes is described below in conjunction with (a) and (b) in FIG. 8 .

On the left side of Figure 8(a) (or on the upper side of Figure 8(b)), each iteration within each process is shown, where the ordinate indicates the label of the process, and the abscissa indicates the number of the iteration, for example, the bottom line, Indicates that the running times of iterations 0, 1, 2, 3, 4, and 5 of process 0 are 1, 2, 1, 2, 1, and 1, respectively, where the two slowest iterations are shown in shaded squares, That is, the two slowest iterations of process 0 are iteration 1 and iteration 3; similarly, the top row indicates that the running times of iterations 0, 1, 2, 3, 4, and 5 of process 4 are 1, 4, 1, 3, 5, and 2, where the two slowest iterations are shown in shaded squares, that is, the slowest two iterations of process 4 are iteration 1 and iteration 4; the meanings of the rest of the lines are similar and will not be repeated here. In the example shown, after the loop ends, each process selects its slowest several iterations as performance change cases to report, and uses these "slowest iterations" for comparative analysis between different processes, and infers the global Performance changes.

For example, the inter-process performance change can be divided into the spatial distribution and the temporal distribution of the inter-process performance change.

Figure 8(a) shows an example of the spatial distribution of detection performance variation. The degree of performance change found by each process may be different, some processes have large changes, and some processes have small changes. In Figure 8(a), the five processes all recorded the slowest two executions of a certain PSG node in a certain cycle as the performance change cases within the process, analyzed (analysis 1), and obtained the performance of each process Performance average (vtx_avg), that is, the total time of each iteration in a process divided by the total number of iterations, for example, the performance average of process 4 is (1+4+1+3+5+2)/6=2.67 , where the average performance of No. 4 process is 2.67, the average performance of No. 3 process is 1.50, the average performance of No. 2 process is 1.17, the average performance of No. 1 process is 1.33, and the average performance of No. 0 process is 1.33, it can be seen that the average performance of No. 4 process is much higher than that of other processes. Therefore, from a global perspective, performance changes have spatial locality, and there may be performance failures in the environment where process No. 4 runs. In addition, FIG. 8( a ) also shows the performance average proc_avg of all processes, which can be obtained by dividing the sum of the performance averages of each process by the total number of processes, which is 1.60 in this example.

Fig. 8(b) shows a schematic diagram of the time distribution of performance changes among detection processes. In the example of Fig. 8(b), it is counted how many processes have performance changes in each numbered iteration. In the example in Figure 8, all processes have performance changes during iteration 1 (iteration numbered 1), and only individual processes have performance changes during other iterations, for example, iteration number 2 appears in process 1 Performance changes (in this example, one of the slowest 2 iterations), iteration number 3 shows performance changes in process 0, iteration number 4 shows performance changes in processes 3 and 4, iteration number 5 There was no performance change in either process. Therefore, in this example, the global performance change is localized in time. During the iteration number 1, there may be interference affecting the performance of the entire program, but this interference disappears again during the iteration number 2.

It should be noted that the previous performance change analysis method between performances is only an example, and the items or methods of analysis can be changed, deleted or added according to needs, for example, the variance of performance changes and the running time average of iterations with performance changes can also be analyzed etc.

According to another embodiment of the present invention, an online detection system for performance changes during parallel program running is also provided.

FIG. 9 shows a logic block diagram of a system 200 for online detection of performance changes during parallel program execution according to another embodiment of the present invention. The online detection system 200 may include a program structure diagram obtaining device 210 and an inserting device 220 .

The program structure graph obtaining means 210 is configured to obtain a program structure graph of the parallel program.

The instrumentation device 220 is configured to automatically or customizedly insert runtime performance change detection codes at the nodes of the program based on the program structure diagram, the performance change detection codes include in-process performance change detection codes, and the in-process performance change detection codes are The instrumentation code can point out which iteration or iterations have changed performance relative to the performance of other iterations in the history, for multiple iterations of loop nodes in the program during the running of the program.

The program inserted with the runtime performance change detection code will report the performance change in time when it is compiled and run.

For the functions and working process of the program structure diagram obtaining device 210 and the plug-in device 220, reference may be made to the previous description in conjunction with Figs. 1-8(b), and details will not be repeated here.

According to another aspect of the present invention, a computing device is provided, including a central processing unit and a memory, wherein computer executable code is stored in the memory, and the code executes the following method when executed by the central processing unit: obtaining a program structure diagram; And based on the program structure diagram, automatically or customizedly insert runtime performance change detection codes at the nodes of the program, the performance change detection codes include in-process performance change detection codes, and the in-process performance change detection codes can be used in the program During execution, for multiple iterations of a loop node in the program, indicate which iteration or iterations have changed in performance relative to the performance of other iterations in history.

Compared with the existing solutions, the online detection method and system for parallel program performance change in the embodiment of the present invention have three main advantages:

(1) The performance failure of the operating environment can be detected when the program is running, unlike the existing scheme 1, which can only be detected before the program runs;

(2) It can automatically detect performance changes and report to developers in a timely manner after detection, without waiting for the program to complete like the existing solution 2, which is necessary for long-running parallel programs (in reality programs may run for hours or days at a time);

(3) Based on the program structure diagram, with the nodes in the program structure diagram as the basic unit of analysis, the code to be analyzed can be automatically or customized, and it is not necessary to analyze all parts of the entire program, thus reducing the running time performance overhead.

Having described various embodiments of the present invention, the foregoing description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and alterations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. Therefore, the protection scope of the present invention should be determined by the protection scope of the claims.

Claims (10)

1. An online detection method of performance variation during parallel program operation, comprising: Obtain the program structure diagram; Based on the program structure diagram, the runtime performance change detection code is automatically or customized at the node of the program, the performance change detection code includes the in-process performance change detection code, and the in-process performance change detection code can be run in the program In a procedure, for multiple iterations of a loop node in a program, indicate which iteration or iterations' performance has changed relative to the performance of other iterations in history. 2. The online detection method according to claim 1, said which iteration or iterations have changed performance includes which iteration or iterations have worse performance than other iterations in history. 3. The online detection method according to claim 1, wherein the in-process performance change detection code is configured to compare the performance of the current iteration with the performance of previous iterations in the program running process during the running of the program, and record the performance change information in the process , and report the in-process performance changes to the user. 4. according to the online detection method of claim 1, The performance change detection code also includes an inter-process performance change detection code, including collecting and comparing the in-process performance change information of each parallel process running the same task by one of the parallel processes running the same task in the parallel program, And give the user a comparative report on performance changes between processes. 5. according to the online detection method of claim 1, the program structure diagram is made up of several trees, and each tree has three kinds of nodes in the structure diagram for a function definition body in the code: loop LOOP, branch BR and calling CALL, each A node is limited by at least three attributes: a globally unique ID; the type of the node, whether it is a loop, a branch, or a call; the position of the node in the source code, and the performance change detection in the process The code reports to the user the iterations indicating the location of the loop nodes in the source code and the performance change meeting predetermined criteria. 6. The online detection method according to claim 1, further comprising, before inserting the runtime performance change detection code, cutting out the program structure diagram. 7. The online detection method according to claim 6, wherein the clipping includes automatically removing nodes whose time-consuming ratio is less than a predetermined threshold from the program structure graph, so that such nodes will not be inserted with runtime performance change detection codes. 8. The online detection method according to claim 4, said in-process performance change information comprising: The sum of the execution time of this node in history; The total number of times the node has been executed in history; The corresponding iteration number and time of the predetermined number of executions with the longest execution time of this node in history. 9. An online detection system for performance changes during parallel program operation, comprising: A device for obtaining a program structure diagram, configured to obtain a program structure diagram; The instrumentation device is configured to automatically or customizedly insert runtime performance change detection codes at the nodes of the program based on the program structure diagram, the performance change detection codes include in-process performance change detection codes, and the in-process performance change The detection code can point out which iteration(s) have changed performance relative to the performance of other iterations in history for multiple iterations of a loop node in the program during the running of the program. 10. A computing device comprising a central processing unit and a memory storing computer executable code which, when executed by the central processing unit, performs the following method: Obtain a program structure diagram; and Based on the program structure diagram, the runtime performance change detection code is automatically or customized at the node of the program, the performance change detection code includes the in-process performance change detection code, and the in-process performance change detection code can be run in the program In a procedure, for multiple iterations of a loop node in a program, indicate which iteration or iterations' performance has changed relative to the performance of other iterations in history.