CN111475251A - Cluster container scheduling method, system, terminal and storage medium - Google Patents

Abstract

The invention provides a method, a system, a terminal and a storage medium for dispatching a cluster container, wherein the method comprises the following steps: collecting the average utilization rate of a CPU and the average utilization rate of a memory of each node of a cluster; collecting CPU resource consumption and memory consumption of each node of the cluster; calculating the CPU-memory utilization similarity of the nodes according to the average CPU utilization, the average memory utilization, the CPU resource consumption and the memory consumption; and evaluating the nodes of the cluster according to the average CPU utilization rate, the average memory utilization rate and the CPU-memory utilization rate similarity, and selecting the optimal node as a container deployment node. According to the invention, the use balance condition of the node resources can be obviously improved by scheduling according to the use condition of the real-time resources of the nodes and the use influence of the deployment Pod on the node use rate.

Description

A cluster container scheduling method, system, terminal and storage medium

technical field

The present invention relates to the technical field of container deployment, in particular to a cluster container scheduling method, system, terminal and storage medium.

Background technique

Kubernetes is Google's open source container cluster management system, which provides functions such as application deployment, maintenance, and expansion. Using Kubernetes can easily and effectively manage containerized applications running across clusters. Pod is the basic unit of resource scheduling in Kubernetes. The component responsible for Pod scheduling is the Kubernetes scheduler, which is responsible for receiving new Pods created by the controller manager, and selecting the node with the highest priority for deployment according to the scheduling algorithm.

The existing scheduling algorithm mainly uses the BalancedResources Allocation strategy. This algorithm calculates the total amount of CPU and memory requests for running Pods on the candidate node Node, as well as the CPU and memory requests for the Pods to be scheduled, respectively. Add and divide by Calculate the total number of nodes, calculate CpuFra and MemFra, and use the following formula to score, and select the node with the highest score for Pod deployment

Score=10-|CpuFra-MemFra|*10

Although this optimal algorithm also considers the CPU and memory resource utilization of the candidate node Node, the algorithm calculates the scheduling priority based on the amount of CPU and memory requests, and cannot accurately reflect the real-time resource usage of the node. Happening.

SUMMARY OF THE INVENTION

In view of the above shortcomings of the prior art, the present invention provides a cluster container scheduling method, system, terminal and storage medium to solve the above technical problems.

In a first aspect, the present invention provides a cluster container scheduling method, including:

Collect the average CPU utilization and average memory utilization of each node in the cluster;

Collect the CPU resource consumption and memory consumption of each node in the cluster;

Calculate the CPU-memory usage similarity of nodes according to the average CPU utilization, average memory utilization, CPU resource consumption and memory consumption;

The nodes of the cluster are evaluated according to the average CPU utilization, average memory utilization, and CPU-memory utilization similarity, and the optimal node is selected as the container deployment node.

Further, the average CPU utilization and average memory utilization of each node in the collection cluster include:

Set the data collection period;

Collect the CPU utilization and memory utilization of each node in the cluster during the collection period;

Calculate the average CPU utilization and average memory utilization of each node during the collection period.

Further, the collection of CPU resource consumption and memory consumption of each node of the cluster includes:

Collect the CPU application amount of the existing containers on the node, and divide the CPU application amount by the total amount of node CPU resources to obtain the node CPU resource consumption degree;

The memory application amount of the existing containers on the node is collected, and the memory consumption degree of the node is obtained by dividing the memory application amount by the total memory amount of the node.

Further, the CPU-memory usage similarity of the node is calculated according to the average CPU utilization, average memory utilization, CPU resource consumption and memory consumption, including:

Setting the first coefficient and the second coefficient, so that the sum of the first coefficient and the second coefficient is 1, and the ratio of the first coefficient to the second coefficient is equal to the ratio of CPU resource consumption to memory consumption;

set a fixed constant;

Take the product of the first coefficient and the average CPU utilization and the difference between the second coefficient and the average memory utilization as a constant coefficient;

The difference between the fixed constant minus the product of the fixed constant and the constant coefficient is used as the CPU-memory usage similarity.

Further, evaluating the nodes of the cluster according to the average CPU utilization, average memory utilization and CPU-memory utilization similarity and selecting the optimal node as the container deployment node, including:

Set CPU weight and memory weight;

Calculate the weighted sum of the average CPU utilization and the average memory utilization of each node;

Screen out the node with the smallest weighted sum in the cluster as the candidate node;

If there are multiple candidate nodes, the candidate node with high CPU-memory usage similarity is selected as the container deployment node.

In a second aspect, the present invention provides a cluster container scheduling system, including:

The average collection unit is configured to collect the average CPU utilization and average memory utilization of each node in the cluster;

The consumption collection unit is configured to collect the CPU resource consumption and memory consumption of each node of the cluster;

The proximity computing unit is configured to calculate the CPU-memory usage similarity of the node according to the average CPU utilization, average memory utilization, CPU resource consumption, and memory consumption;

The node selection unit is configured to evaluate the nodes of the cluster according to the average CPU utilization, average memory utilization, and CPU-memory utilization similarity, and select the optimal node as the container deployment node.

Further, the consumption collection unit includes:

The CPU collection module is configured to collect the CPU application amount of the existing containers on the node, and divide the CPU application amount by the total amount of the node CPU resources to obtain the node CPU resource consumption degree;

The memory collection module is configured to collect the memory application amount of the existing containers on the node, and divide the memory application amount by the total memory amount of the node to obtain the memory consumption degree of the node.

Further, the node selection unit includes:

Weight setting module, configured to set CPU weight and memory weight;

The weighted sum module is configured to calculate the weighted sum of the average CPU utilization and the average memory utilization of each node;

A primary screening module, configured to screen out the node with the smallest weighted sum in the cluster as a candidate node;

The secondary screening module is configured to select the candidate node with high CPU-memory usage similarity as the container deployment node if there are multiple candidate nodes.

In a third aspect, a terminal is provided, including:

processor, memory, which,

The memory is used to store computer programs,

The processor is used to call and run the computer program from the memory, so that the terminal executes the above-mentioned method of the terminal.

In a fourth aspect, a computer storage medium is provided, and instructions are stored in the computer-readable storage medium, which, when executed on a computer, cause the computer to perform the methods described in the above aspects.

The beneficial effect of the present invention is that,

The cluster container scheduling method, system, terminal and storage medium provided by the present invention take into account the resource utilization and resource balance by using the average resource utilization and resource consumption to evaluate the nodes of the cluster. The present invention performs scheduling according to the real-time resource usage of the node and in combination with the influence of deploying Pods on the usage of the node, which can significantly improve the balance of resource usage of the node.

In addition, the present invention has reliable design principle and simple structure, and has a very wide application prospect.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. In other words, other drawings can also be obtained based on these drawings without creative labor.

FIG. 1 is a schematic flowchart of a method according to an embodiment of the present invention.

FIG. 2 is a schematic block diagram of a system according to an embodiment of the present invention.

FIG. 3 is a schematic structural diagram of a terminal according to an embodiment of the present invention.

Detailed ways

In order to make those skilled in the art better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described The embodiments are only some of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Key terms appearing in the present invention are explained below.

Containers, pods, are task processing nodes similar to virtual machines.

FIG. 1 is a schematic flowchart of a method according to an embodiment of the present invention. Wherein, the execution body of FIG. 1 may be a cluster container scheduling system.

As shown in Figure 1, the method 100 includes:

Step 110: Collect the average CPU utilization and average memory utilization of each node in the cluster;

Step 120, collecting the CPU resource consumption and memory consumption of each node of the cluster;

Step 130: Calculate the CPU-memory usage similarity of the node according to the average CPU utilization, the average memory utilization, the CPU resource consumption and the memory consumption;

Step 140: Evaluate the nodes of the cluster according to the average CPU utilization, average memory utilization, and CPU-memory utilization similarity, and select the optimal node as the container deployment node.

In order to facilitate the understanding of the present invention, the cluster container scheduling method provided by the present invention is further described below based on the principles of the cluster container scheduling method of the present invention and the process of scheduling the cluster containers in the embodiment.

Specifically, the cluster container scheduling method includes:

S1. Collect the average CPU utilization and average memory utilization of each node in the cluster.

Assuming that the collection period is set to 1h, Kubernetes collects the CPU utilization and memory utilization of each node in the cluster within 1h. Taking the average CPU utilization as an example, assuming that there are 60 collection points in the 1h collection period, for the CPU utilization of the 60 collection points, the average CPU utilization of the node can be obtained by dividing the sum of all CPU utilizations by 60. . Average memory utilization is calculated the same way.

S2. Collect the CPU resource consumption and memory consumption of each node in the cluster.

The Pod's consumption level (Rc) of CPU resources on the node is divided by the CPU application amount of the Pod by the total amount of CPU resources on the node.

The consumption level (Rm) of the Pod's memory on the node is divided by the Pod's memory application amount by the total amount of memory resources on the node.

S3. Calculate the CPU-memory usage similarity of the node according to the average CPU utilization, the average memory utilization, the CPU resource consumption and the memory consumption.

The first parameter α and the second parameter β are introduced, and the calculation formula of CPU-memory usage similarity (Score) is as follows:

Score=10-|αCpuSr-βMemSr|*10

in,

α+β=1;

α/β=Rc/Rm;

CpuSr is the average CPU utilization, and MemSr is the average memory utilization.

S4. Evaluate the nodes of the cluster according to the average CPU utilization, average memory utilization, and CPU-memory utilization similarity, and select the optimal node as the container deployment node.

Set the CPU weight and the memory weight, calculate the weighted sum of the average CPU utilization and the average memory utilization of each node, and select the node with the smallest weighted sum in the cluster as a candidate node. If multiple candidate nodes are filtered out, the candidate node with high CPU-memory usage similarity is selected as the container deployment node.

As shown in Figure 2, the system 200 includes:

The average collection unit 210 is configured to collect the average CPU utilization and the average memory utilization of each node in the cluster;

The consumption collection unit 220 is configured to collect the CPU resource consumption and memory consumption of each node of the cluster;

The proximity computing unit 230 is configured to calculate the CPU-memory usage similarity of the node according to the average CPU utilization, the average memory utilization, the CPU resource consumption and the memory consumption;

The node selection unit 240 is configured to evaluate the nodes of the cluster according to the average CPU utilization, the average memory utilization and the similarity of CPU-memory utilization, and select the optimal node as the container deployment node.

Optionally, as an embodiment of the present invention, the consumption collection unit includes:

The CPU collection module is configured to collect the CPU application amount of the existing containers on the node, and divide the CPU application amount by the total amount of the node CPU resources to obtain the node CPU resource consumption degree;

The memory collection module is configured to collect the memory application amount of the existing containers on the node, and divide the memory application amount by the total memory amount of the node to obtain the memory consumption degree of the node.

Optionally, as an embodiment of the present invention, the node selection unit includes:

Weight setting module, configured to set CPU weight and memory weight;

The weighted sum module is configured to calculate the weighted sum of the average CPU utilization and the average memory utilization of each node;

A primary screening module, configured to screen out the node with the smallest weighted sum in the cluster as a candidate node;

The secondary screening module is configured to select the candidate node with high CPU-memory usage similarity as the container deployment node if there are multiple candidate nodes.

FIG. 3 is a schematic structural diagram of a terminal system 300 according to an embodiment of the present invention. The terminal system 300 may be used to execute the cluster container scheduling method provided by the embodiment of the present invention.

The terminal system 300 may include: a processor 310 , a memory 320 and a communication unit 330 . These components communicate through one or more buses. Those skilled in the art can understand that the structure of the server shown in the figure does not constitute a limitation of the present invention. It can be either a bus structure, a star structure, or a More or fewer components than shown may be included, or some components may be combined, or a different arrangement of components.

Wherein, the memory 320 can be used to store the execution instructions of the processor 310, and the memory 320 can be implemented by any type of volatile or non-volatile storage terminal or their combination, such as static random access memory (SRAM), electrical Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic Disk or Optical Disk . When the execution instructions in the memory 320 are executed by the processor 310, the terminal 300 is enabled to execute some or all of the steps in the following method embodiments.

The processor 310 is the control center of the storage terminal, using various interfaces and lines to connect various parts of the entire electronic terminal, by running or executing the software programs and/or modules stored in the memory 320, and calling the data stored in the memory, To perform various functions of the electronic terminal and/or process data. The processor may be composed of an integrated circuit (Integrated Circuit, IC for short), for example, may be composed of a single packaged IC, or may be composed of a plurality of packaged ICs connected with the same function or different functions. For example, the processor 310 may only include a central processing unit (Central Processing Unit, CPU for short). In the embodiment of the present invention, the CPU may be a single computing core, or may include multiple computing cores.

The communication unit 330 is used for establishing a communication channel, so that the storage terminal can communicate with other terminals. Receive user data sent by other terminals or send user data to other terminals.

The present invention also provides a computer storage medium, wherein the computer storage medium can store a program, and when the program is executed, it can include some or all of the steps in the embodiments provided by the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (English: read-only memory, ROM for short) or a random access memory (English: random access memory, RAM for short).

Therefore, the present invention evaluates the nodes of the cluster by using the average resource utilization rate and the resource consumption degree, taking into account the resource utilization rate and the resource balance degree. The present invention performs scheduling according to the real-time resource usage of nodes and in combination with the impact of deploying Pods on the usage of nodes, which can significantly improve the balance of resource usage of nodes. The technical effects that can be achieved in this embodiment can refer to the description above. Repeat.

Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such understanding, the technical solutions in the embodiments of the present invention can be embodied in the form of software products in essence or in the parts that make contributions to the prior art. The computer software products are stored in a storage medium such as a USB flash drive, a mobile Hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes, including several instructions to make a computer terminal (It may be a personal computer, a server, or a second terminal, a network terminal, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

It is sufficient to refer to each other for the same and similar parts among the various embodiments in this specification. In particular, for the terminal embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for related parts, please refer to the description in the method embodiment.

In the several embodiments provided by the present invention, it should be understood that the disclosed system and method may be implemented in other manners. For example, the system embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection between systems or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

Although the present invention has been described in detail in conjunction with the preferred embodiments with reference to the accompanying drawings, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, those of ordinary skill in the art can make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions should all fall within the scope of the present invention/any Those skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should all be included within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (10)

1. A method for scheduling a cluster container, comprising:

collecting the average utilization rate of a CPU and the average utilization rate of a memory of each node of a cluster;

collecting CPU resource consumption and memory consumption of each node of the cluster;

calculating the CPU-memory utilization similarity of the nodes according to the average CPU utilization, the average memory utilization, the CPU resource consumption and the memory consumption;

and evaluating the nodes of the cluster according to the average CPU utilization rate, the average memory utilization rate and the CPU-memory utilization rate similarity, and selecting the optimal node as a container deployment node.

2. The method of claim 1, wherein the collecting the average utilization rate of the CPU and the average utilization rate of the memory of each node of the cluster comprises:

setting a data acquisition period;

collecting the CPU utilization rate and the memory utilization rate of each node of the cluster in a collection period;

and calculating the average CPU utilization rate and the average memory utilization rate of each node in the acquisition period.

3. The method of claim 1, wherein the collecting CPU resource consumption and memory consumption of each node of the cluster comprises:

acquiring CPU application amount of an existing container on a node, and dividing the CPU application amount by the total amount of node CPU resources to obtain node CPU resource consumption;

and acquiring the memory application amount of the existing container on the node, and dividing the memory application amount by the total amount of the node memory to obtain the memory consumption of the node.

4. The method of claim 1, wherein calculating the CPU-memory utilization proximity of the node based on the average CPU utilization, the average memory utilization, the CPU resource consumption, and the memory consumption comprises:

setting a first coefficient and a second coefficient, wherein the sum of the first coefficient and the second coefficient is 1, and the ratio of the first coefficient to the second coefficient is equal to the ratio of the CPU resource consumption to the memory consumption;

setting a fixed constant;

taking the difference between the first coefficient and the CPU average utilization rate and the difference between the second coefficient and the memory average utilization rate as constant coefficients;

and the difference value obtained by subtracting the product of the fixed constant and the constant coefficient from the fixed constant is used as the CPU-memory utilization approximation.

5. The method of claim 1, wherein the evaluating nodes of the cluster according to the average utilization rate of the CPU, the average utilization rate of the memory, and the CPU-memory utilization similarity and selecting an optimal node as a container deployment node comprises:

setting a CPU weight and a memory weight;

calculating the weighted sum of the average CPU utilization rate and the average memory utilization rate of each node;

screening out the node with the minimum weighted sum in the cluster as a node to be selected;

and if a plurality of nodes to be selected exist, selecting the nodes to be selected with high CPU-memory utilization rate similarity as container deployment nodes.

6. A cluster container scheduling system, comprising:

the average acquisition unit is configured for acquiring the average utilization rate of the CPU and the average utilization rate of the memory of each node of the cluster;

the consumption acquisition unit is configured for acquiring the CPU resource consumption and the memory consumption of each node of the cluster;

the close calculation unit is configured for calculating the CPU-memory utilization close degree of the node according to the average CPU utilization rate, the average memory utilization rate, the CPU resource consumption degree and the memory consumption degree;

and the node selection unit is configured to evaluate the nodes of the cluster according to the average CPU utilization rate, the average memory utilization rate and the CPU-memory utilization rate similarity and select the optimal node as a container deployment node.

7. The system of claim 6, wherein the consumption collection unit comprises:

the CPU acquisition module is used for acquiring the CPU application amount of the existing container on the node and dividing the CPU application amount by the total amount of the node CPU resources to obtain the consumption degree of the node CPU resources;

and the memory acquisition module is configured for acquiring the memory application amount of the existing container on the node and dividing the memory application amount by the total amount of the node memory to obtain the memory consumption of the node.

8. The system of claim 6, wherein the node selection unit comprises:

the weight setting module is configured for setting a CPU weight and a memory weight;

the weighted summation module is configured to calculate a weighted sum of the CPU average utilization rate and the memory average utilization rate of each node;

the primary screening module is configured to screen out the node with the minimum weighted sum in the cluster as a node to be selected;

and the secondary screening module is configured to select a node to be selected with high CPU-memory utilization rate similarity as a container deployment node if a plurality of nodes to be selected exist.

9. A terminal, comprising:

a processor;

a memory for storing instructions for execution by the processor;

wherein the processor is configured to perform the method of any one of claims 1-5.

10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-5.

Images