Background
Data centers have gone from concept to maturity. In a data center, a large number of programs are executed on as few servers as possible in order to improve resource utilization. In a server node, there are multiple programs executing on a single node. The advantage of multiple programs executing together is that the utilization of the server can be increased, and the problem is that the performance of the program is reduced. The degree of performance degradation of a program depends on program characteristics, and for some programs, performance degradation is not significant when the program is executed with other programs, and for some programs, performance degradation is significant when the program is executed with other programs.
At the same time, a large number of delay sensitive programs are running in the data center. Customers execute delay sensitive programs in the data center and have certain quality of service requirements on the programs, such as program performance that cannot be lower than 90% of that when executed alone. When a delay sensitive program is executed together with other programs, it is easy to cause severe performance degradation due to performance interference, and thus, the quality of service requirements of customers cannot be satisfied. This is a problem that must be solved. Therefore, a method is needed to ensure the service quality of the delay-sensitive program on the basis of improving the utilization rate of system resources as much as possible.
Disclosure of Invention
The invention aims to provide a method for ensuring the service quality of a delay sensitive program in a data center environment, which can ensure the performance of the delay sensitive program and batch programs when the delay sensitive program and the batch programs are executed together, and simultaneously improve the resource utilization rate of a system as much as possible. According to the method, a resource isolation tool Cgroups is used for dividing resources for delay sensitive programs, and the performance of the delay sensitive programs is guaranteed. Meanwhile, other resources are distributed to batch programs for use. And the utilization rate and the throughput of system resources are improved by using a batch program.
In order to achieve the purpose, the invention adopts the following technical scheme:
a method for ensuring the service quality of a delay sensitive program in a data center environment comprises the following steps:
step (1): respectively setting a plurality of resource limit points for a processor and a memory, calculating the resource sensitivity of each delay sensitive program to be executed at the resource limit points, and drawing a resource sensitivity plane of the delay sensitive program;
step (2): supposing that one delay sensitive program and a plurality of batch programs are executed on each server node, and calculating resources occupied by the delay sensitive program under the condition of meeting the service quality requirement;
and (3): setting the occupied resource of the delay sensitive program according to the calculation result of the step (2);
and (4): removing the resources occupied by the delay sensitive program set in the step (3), and distributing the rest resources to batch programs for use as required; setting the occupied resources of the batch programs according to the distribution result;
and (5): and (4) executing the delay sensitive program according to the result set in the step (3), and executing the batch program according to the result set in the step (4).
The step of respectively setting a plurality of resource limit points for the processor and the memory comprises the following steps:
setting M resource limit points in an occupancy rate range M of processor resources when a delay sensitive program runs, setting N resource limit points in an occupancy rate range N of memory resources when the delay sensitive program runs, and recording the execution time of the delay sensitive program when the delay sensitive program is executed on each resource limit point (i, j), wherein the value range of i is 1 to M, and the value range of j is 1 to N.
The step of calculating the resource sensitivity of each delay-sensitive program to be executed at the resource limit point comprises the following steps:
and calculating the ratio of the execution time of the delay-sensitive program under the condition that the resources are not limited to the execution time of the delay-sensitive program at each resource limit point, wherein the ratio is the program resource sensitivity of the delay-sensitive program at each resource limit point.
The step of drawing the sensitivity plane of the delay sensitive program comprises the following steps:
the resource occupancy rate of a processor when a delay sensitive program runs is taken as an X axis, the resource occupancy rate of a memory when the delay sensitive program runs is taken as a Y axis, the resource sensitivity of the program is taken as a Z axis, a space rectangular coordinate system is established, all program resource sensitivity point values are fitted into a curved surface, and the curved surface is a sensitivity plane of the delay sensitive program.
The steps of setting a plurality of resource limit points for the processor and the memory, calculating the resource sensitivity of each delay sensitive program to be executed at the resource limit points, and drawing the sensitivity plane of the delay sensitive program further comprise:
acquiring resource occupancy rates M of the delay sensitive program to a processor and N of the delay sensitive program to a memory respectively under the condition that resources are not limited by a performance analysis tool; acquiring the resource occupation amount of each batch program to the processor and the resource occupation amount of each batch program to the memory under the condition that the resources are not limited by a performance analysis tool;
the step (2) calculates the resources occupied by the delay sensitive program under the condition of meeting the service quality requirement:
assuming that the service quality requirement of the delay-sensitive program is not lower than 90% of the performance of the delay-sensitive program when the delay-sensitive program is executed alone, for the sensitivity plane of the delay-sensitive program, a new plane parallel to both the X axis and the Y axis is drawn with reference to 90% of the performance of the delay-sensitive program when the delay-sensitive program is executed alone; the resource combination of the processor and the memory occupied by the delay-sensitive program is the node above the new plane and closest to the new plane in the positive Z-axis direction, and it is assumed that the processor selected by the delay-sensitive program is ClatThe selected memory node is Mlat。
For example: taking z as an interface, wherein the value of z is less than 0.9 and is below a new plane, and the value of z is more than 0.9 and is above the new plane;
and (3) setting occupied resources of the delay sensitive program by using a Linux resource isolation tool Cgroups.
In the step (3), C is addedlatAnd MlatWrite control group set Cgroups; the delay sensitive program is placed into the control group for execution.
The step (4) comprises the following steps:
step (41): the processor that the batch program can occupy is Call-ClatThe memory resource that can be occupied by the batch program is Mall-Mlat;CallAs a total amount of processor resources, ClatSetting processor resources occupied by delay sensitive programs; mallFor total amount of memory resources, MlatSetting memory resources occupied by delay sensitive programs;
step (42): the requirements for the number of executable batch programs are as follows: all batch programs occupy no more processor resources than (C)all-Clat) 160%, and all the batch programs occupy no more memory resources than (M)all-Mlat)*160%;
Step (43): and (4) executing the batch program according to the calculation result of the step (42).
The invention has the beneficial effects that:
the invention can improve the resource utilization rate and the throughput of the system as much as possible while ensuring the service quality of the program.
The invention calculates the resource occupation when the service quality of the program needs to be ensured through the resource sensitivity, and the resource occupation value calculated by the resource sensitivity has the advantages that: the value is as small as possible and the quality of service of the program can be guaranteed.
Detailed Description
The invention is further described with reference to the following figures and examples.
Fig. 1 depicts a resource partitioning method.
Firstly, the method comprises the following steps: working off-line
For each program needing to be executed, the resource occupation information of the program on a processor and a memory is obtained by utilizing a performance analysis tool. And simultaneously acquiring resource sensitivity information of the program. The performance analysis tools, such as collectll;
II, secondly: working on-line
For delay sensitive programs:
and calculating the resources which should be occupied by the network. The calculation method is as follows: assuming that the service quality requirement of the program is not less than 90% of the performance when performed alone, for the sensitivity plane of the program, a plane parallel to the X-axis and the Y-axis is drawn with reference to 90% of the performance of the program.
The processor and memory resource combination occupied by the delay sensitive program may be a node above the plane. And dividing resources for the delay sensitive program by using a Linux resource isolation tool Cgroups, and placing the delay sensitive program into a control group for execution.
For batch procedures:
and calculating the resource occupation of the batch programs. The resource occupancy value is (total resource value of server-resource occupancy value of delay sensitive program) × 160%.
The batch program is placed outside the control group for execution.
Although the embodiments of the present invention have been described with reference to the accompanying drawings, it is not intended to limit the scope of the present invention, and it should be understood by those skilled in the art that various modifications and variations can be made without inventive efforts by those skilled in the art based on the technical solution of the present invention.