CN104199747B - High-availability system obtaining method and system based on health management - Google Patents

Abstract

The invention discloses a method for implementing a high-availability system based on health management: S1. Set at least two control nodes and at least two computing nodes; set one of the control nodes as the main control node, and set the other control nodes as backup Control the nodes and configure their priority order; set the priority of the main control node higher than any backup control node; S2, each control node repeatedly collects and stores the health status data of all computing nodes; S3, judges whether the current control node is The main control node, if yes, go to step S5; otherwise, go to step S4; S4, the standby control node with the highest priority detects whether the main control node is faulty through heartbeat in real time; the standby control node with low priority detects in real time Whether the standby control node with a higher priority level is faulty; if so, modify the priority order; S5. The main control node analyzes the stored health status data, and manages the computing nodes according to the analysis results.

Description

Method and system for implementing high-availability system based on health management

technical field

The invention relates to the technical field of high availability of computers, in particular to a method and system for realizing a high availability system based on health management.

Background technique

The reliability of a computer system is measured by the mean time between failures, that is, how long the computer system can run normally before a failure occurs. Availability of a computer system is defined as the percentage of time the system remains up. High Availability (High Availability; HA) is used to describe a system that has been specially designed to reduce downtime while maintaining high availability of its services. In order to improve the high availability of server equipment or software, currently commonly used methods mainly include:

1. Two-machine cold backup technology: the main machine is in the running state, and the backup machine is in the waiting state. Once the main machine fails, the backup machine immediately starts the service of the main machine.

2. Dual-machine hot backup technology: This technology is the so-called active/standby working mode. The active and standby devices have the same hardware configuration and software system. Once the active device fails, the standby device immediately activates the application service to ensure that the application service can be fully restored to normal use in a short time. High availability software (HA) includes software such as RoseHa.

3. Virtual machine high-availability fault-tolerant technology: After virtualizing the server, perform fault-tolerant backup configuration on the virtual server, and then use heartbeat detection and virtual machine real-time migration technology to realize transparent fault-tolerant computing functions for all applications of the computer system.

The above methods for improving the high availability of server equipment or software have the following defects:

1. Most of the dual-machine cold backup technology and dual-machine hot backup technology are designed for a certain service process, that is, specially designed to improve the high availability of a certain service, and do not support fault-tolerant calculations of other applications. Therefore, users The user business of self-developed application software must modify the application program and call the dedicated API of high-availability software to achieve fault-tolerant processing of user business, making it difficult for user-developed application software to achieve high availability through commercial high-availability software .

2. The system resource overhead of high-availability and fault-tolerant technology for virtual machines is relatively large.

Contents of the invention

In view of this, it is necessary to provide a high-availability system implementation method and system based on health management that has low system resource overhead and supports fault-tolerant computing of various applications.

The technical solution adopted by the present invention to solve the technical problem is: to construct a method for implementing a high-availability system based on health management, and the method for implementing a high-availability system based on health management includes the following steps:

S1. Set at least two control nodes and at least two computing nodes; set one of the control nodes as the master control node, set the other control nodes as standby control nodes and configure the priority order of the standby control nodes; set the master control node priority is higher than that of any standby control node;

S2. Each control node repeatedly collects and stores the health status data of all computing nodes;

S3. Judging whether the current control node is the main control node, if the current control node is the main control node, then jump to step S5; if the current control node is the standby control node, then jump to step S4;

S4. The standby control node with the highest priority detects whether the main control node fails through heartbeat in real time; the standby control node with low priority detects through heartbeat in real time whether the standby control node with a higher If there is a fault, modify the priority order;

S5. The master control node analyzes the health state data stored in the memory of the master control node, and manages the computing nodes according to the analysis result.

The present invention also provides a system for implementing a high-availability system based on health management, wherein the system for implementing a high-availability system based on health management includes the following modules:

The node configuration module is used to set at least two control nodes and at least two computing nodes; set one of the control nodes as the main control node, set the other control nodes as standby control nodes and configure the priority order of the standby control nodes; Set the priority of the master control node to be higher than that of any standby control node;

The data collection module is used to repeatedly collect and store the health status data of all computing nodes through each control node;

The node judgment module is used to judge whether the current control node is the main control node, if the current control node is the main control node, then jump to the data analysis module; if the current control node is the backup control node, then jump to the fault judgment module;

The fault judgment module is used to detect whether the main control node fails in real time through the heartbeat of the standby control node with the highest priority; through the heartbeat of the standby control node with low priority to detect whether the standby control node with a higher priority exists in real time Fault; when the control node fails, modify the priority order;

The data analysis module is configured to analyze the health state data stored in the memory of the main control node through the main control node, and manage the computing nodes according to the analysis results.

The method and system for realizing a high-availability system based on health management provided by the present invention can be configured very flexibly by flexibly configuring the number and priority of control nodes. In addition, the main control node analyzes the health status data stored in the memory of the main control node, manages the computing nodes according to the analysis results, and realizes the prediction of the trending loss failure of the computing nodes in advance. The health status data is obtained through the bypass of the computing node, which does not occupy the CPU, memory, network and other resources of the computing node. Compared with the dual-machine cold backup technology, dual-machine hot backup technology, and virtual machine high-availability fault-tolerant technology, it saves system resource overhead .

Description of drawings

Fig. 1 is a flow chart of a method for implementing a highly available system based on health management in a preferred embodiment of the present invention;

Fig. 2 is the subflow chart of step S2 among Fig. 1;

Fig. 3 is the subflow chart of step S4 among Fig. 1;

Fig. 4 is the subflow chart of step S5 among Fig. 1;

Fig. 5 is the subflow chart of step S58 among Fig. 4;

Fig. 6 is a logic diagram of the high availability system in the embodiment of the present invention;

Fig. 7 is a working flow chart of the main control node and the standby control node in the embodiment of the present invention;

FIG. 8 is a schematic diagram of a control node takeover sequence in an embodiment of the present invention;

Fig. 9 is a work flow chart of health state data collection, analysis and processing in an embodiment of the present invention;

Fig. 10 is a schematic diagram of a historical value matrix of health state data in an embodiment of the present invention.

Fig. 11 is a structural block diagram of a highly available system implementation system based on health management in a preferred embodiment of the present invention;

Fig. 12 is a block diagram of the substructure of the data acquisition module in Fig. 11;

Fig. 13 is a block diagram of the substructure of the failure judgment module in Fig. 11;

Fig. 14 is a substructure block diagram of the data analysis module in Fig. 11;

FIG. 15 is a substructure block diagram of the computing node management unit in FIG. 14 .

detailed description

As shown in Figure 1, an embodiment of the present invention provides a method for implementing a high-availability system based on health management, and the method for implementing a high-availability system based on health management includes the following steps:

S1. Set at least two control nodes and at least two computing nodes. Set one of the control nodes as the master control node, set the other control nodes as standby control nodes and configure the priority order of the standby control nodes. Set the priority of the master control node to be higher than that of any standby control node.

In the implementation of the present invention, the number of control nodes can be added or deleted according to the task importance and high availability requirements of the high-availability system. Preferably, the number of control nodes does not exceed the number of computing nodes, and two or three control nodes are sufficient. The number of computing nodes can be added or deleted according to the task processing scale of the high-availability system. It is recommended to reserve at least two computing nodes.

All computing nodes other than any computing node can be configured as standby computing nodes of the computing node, thus greatly improving the redundancy of computing nodes and improving the high availability of computing nodes.

S2. Each control node repeatedly collects and stores health status data of all computing nodes.

Health status data may include health status data types such as temperature, fan, voltage, and current of computing nodes.

The control node and computing node in the embodiment of the present invention may be a server or a computing device in a network system.

The health status data in the embodiment of the present invention can be obtained through the bypass of the front-end module (BMC module) of the computing node, and does not occupy resources such as the CPU, memory, and network of the computing node. Therefore, compared with dual-machine hot backup and dual-machine cold backup Technology, virtual machine fault tolerance technology, etc. to achieve high availability. The embodiment of the present invention has less impact on the performance of the computing node itself.

Optionally, as shown in Figure 2, the step S2 includes the following sub-steps:

S21. Each control node collects the health state data of all computing nodes once, and saves the collected health state data in the memory of each control node;

S22. Determine whether the number of collections is less than the predetermined number p, if the number of collections is less than the predetermined number p, then wait for the first preset time T1 and then jump to step S21; if the number of collections is greater than the predetermined number p, delete each control node For redundant historical health status data on the memory, keep the latest p health status data collected, then jump to step S23; if the number of times collected is equal to the predetermined number p, then jump to step S23;

S23. Set the health status data of n computing nodes collected by each control node p times into a matrix of historical health status data with p rows and n columns.

Refer to FIG. 10 for the historical value matrix of the health state data in the step S23.

S3. Judging whether the current control node is the master control node, if the current control node is the master control node, jump to step S5. If the current control node is the standby control node, go to step S4.

S4. The standby control node with the highest priority detects whether the main control node fails in real time through the heartbeat. The standby control node with a lower priority detects in real time whether the standby control node with a higher priority is faulty through heartbeat. If the control node fails, the priority order is modified.

Optionally, as shown in Figure 3, the step S4 includes the following sub-steps:

S41. The standby control node with the highest priority detects in real time whether the main control node fails through heartbeat; the standby control node with low priority detects in real time whether the standby control node with a higher priority than the standby control node fails through heartbeat.

S42. If it is detected that the main control node fails, the standby control node with the highest priority is configured as the main control node, and the priorities of the other standby control nodes except the standby control node with the highest priority are respectively increased by one level.

S43. If it is detected that the standby control node fails, the priorities of all the standby control nodes whose priorities are lower than the failed standby control node are respectively increased by one level.

This embodiment can flexibly change the priority of the control node, and when the main control node fails, the standby control node with the highest priority can be set as the main control node in time. A smooth transition is realized when the control node fails.

S5. The master control node analyzes the health state data stored in the memory of the master control node, and manages the computing nodes according to the analysis result.

Optionally, as shown in Figure 4, the step S5 includes the following sub-steps:

S51. The main control node analyzes the historical value matrix of the health status data column by column, and each column of health status data corresponds to a computing node.

S52. Determine whether all computing nodes have been analyzed once. If all computing nodes have been analyzed once, skip to step S57. If there are computing nodes that have not been analyzed, go to step S53.

S53, respectively counting the number of acquisitions of the current computing node exceeding the health state reference range and below the health state reference range;

S54. If the number of acquisitions exceeding the health state reference range is less than k1, and the number of acquisitions below the health state reference range is less than k2, determine that the computing node is in a healthy state, and jump to step S55. If the number of acquisitions exceeding the reference range of the health state is greater than or equal to k1, or the number of acquisitions below the reference range of the health state is greater than or equal to k2, it is determined that the computing node is in an unhealthy state and jumps to step S56.

k1/p represents the tolerance for exceeding the reference range of the health state (the larger is more conservative, the smaller is more aggressive), and K2/p represents the tolerance for being below the reference range of the health state (the larger is more conservative, the smaller is more aggressive). The tolerance can also be configured according to the requirements of the high-availability system, and it can be configured flexibly.

S55. Set the unhealthy flag value of the computing node to 0, and jump to step S51.

S56. Obtain the maximum value of the number of acquisitions exceeding the health state reference range and the number of acquisitions below the health state reference range, set the maximum value as the unhealthy state value of the computing node, and jump to step S51.

S57. Analyze the non-health flag values of all computing nodes, and determine whether there are computing nodes with non-zero values; if there are no non-zero computing nodes, determine that all computing nodes are healthy, and if there are non-zero computing nodes node, it is determined that there is an unhealthy computing node.

S58. Manage the computing nodes according to the set policy, wait for the first preset time T1, and jump to step S21.

Optionally, as shown in Figure 5, the step S58 includes the following sub-steps:

S581. Set the first strategy, the second strategy, and the third strategy. The first strategy includes three options: prompt alarm, automatic migration task out, and alarm simultaneous automatic migration task out; the second strategy includes prompt alarm, automatic shutdown, and alarm simultaneous automatic Shutdown and other three options; the third strategy includes three options: prompt alarm, automatic migration task return, alarm and automatic migration task return.

S582. Determine whether an unhealthy computing node has a processing task, and if the unhealthy computing node has a processing task, query the first strategy and perform processing according to the first strategy. If there is no processing task in the unhealthy computing node, query the second policy and process according to the second policy.

S583. Determine whether a healthy computing node has a processing task. If the healthy computing node has a processing task, no operation is performed on the healthy computing node; if the healthy computing node does not have a processing task, determine that the healthy computing node is Restore from an unhealthy state to a healthy state, and record the current time stamp. If the difference between the current time stamp and the last time stamp is less than the second preset time T2, or if there is no last time stamp, no action will be taken. If the difference is greater than or equal to the second preset time T2, the processing is performed according to the third strategy.

S584. Jump to step S21 after waiting for the first preset time T1.

The embodiment of the present invention triggers task migration of computing nodes by analyzing health status data. Compared with automatically triggering task migration when downtime or manually performing task migration, health status data detection is more scientific and flexible. And before the computing node has a trending loss failure, there is an early response (sub-health) on the health status data, thus improving the system's ability to detect faults in advance. Compared with other technologies, the embodiment of the present invention saves resource overhead of the high availability system itself.

The principle of the embodiment of the present invention will be further described below with reference to FIG. 6 to FIG. 10 .

Fig. 6 is a logical relationship diagram of the high availability system in the embodiment of the present invention. It can be seen that each control node corresponds to all computing nodes. The control node with low priority detects whether the control node with a higher priority is faulty through heartbeat.

This embodiment of the present invention will be described in detail with reference to accompanying drawings 7 and 9 .

The working flow of the master control node and the standby control node of the present invention will be described with reference to FIG. 7 , and the detailed flow is as follows.

Step 1.1: The control node collects the health status data of all computing nodes, and then enters step 1.2.

Step 1.2: Read the configuration file to determine whether the current control node is the master control node. If it is not the main control node (that is, the standby control node), go to step 1.3; if it is the main control node, go to step 1.5.

Step 1.3: Detect whether the main control node and the standby control node with higher priority are faulty through the heartbeat detection network. If there is a control node fault, go to step 1.4. If there is no control node fault, wait for time T1 and go to step 1.1.

Step 1.4: modify the priority order of the control nodes, please refer to FIG. 8 , which is a schematic diagram of the control node takeover sequence in the embodiment of the present invention. If the master control node fails, the first standby control node with the highest priority becomes the master control node; if the standby control node fails, the standby control node with a lower priority than the standby control node automatically raises its priority by one level. After modifying the priority order of the control nodes, go to step 1.2.

Step 1.5: Analyze and process the health status data, please refer to Figure 9.

Fig. 9 is a work flow chart of health state data collection, analysis and processing in an embodiment of the present invention;

The working flow of the health status data analysis and processing of the present invention will be described in conjunction with FIG. 9 , and the detailed flow is as follows. This health status data analysis and processing workflow is applicable to health status data types such as temperature, fan, voltage, and current.

Step 2.1: Collect the health status data of all current computing nodes, and then go to step 2.2.

Step 2.2: If the number of collection times is less than p, it means that the collected historical data is not enough, and enter step 2.1 after waiting time T. If the number of time points collected is greater than or equal to p times, proceed to step 2.3.

Step 2.3: Delete redundant historical data, and only keep the health status data collected for the last p times. At this time, the data of n computing nodes at p time points form a historical value matrix of health status data with p rows and n columns, which can be referred to Figure 10. The row of the matrix corresponds to each time point, and the column of the matrix corresponds to each computing node, go to step 2.4.

Step 2.4: Analyze the historical value matrix of the health status data column by column, each column of data corresponds to a computing node, and proceed to step 2.5.

Step 2.5: If there are still computing nodes that have not been analyzed, go to step 2.6; if all computing nodes have been analyzed, the health status data analysis of the current collection has been completed, go to step 2.10.

Step 2.6: Count the collection times of the current column exceeding the reference range of the health state, count the collection times of the current column below the reference range of the health state, and then proceed to step 2.7.

Step 2.7: If the number of time points exceeding the health state reference range is less than k1, and the number of time points below the health state reference range is less than k2, then it is still determined that the computing node is in a healthy state, and enter step 2.8; if it exceeds the health state reference range The number of time points is greater than or equal to k1, or the number of time points below the health status reference range is greater than or equal to k2, it means that the computing node has exceeded or lower than the health status reference range for many times, and tends to be stable, and it is determined that the computing node is not Health status, go to step 2.9.

Step 2.8: Set the unhealthy flag value of the current computing node to 0, and then go to step 2.4.

Step 2.9: Take the maximum value of the number of collections exceeding the reference range of the health state and the number of time points below the reference range of the health state as the unhealthy flag value of the computing node, and then proceed to step 2.4.

Step 2.10: Analyze the unhealthy flag values of all computing nodes. If there are no computing nodes with non-zero values, it means that all computing nodes are healthy. If there are computing nodes with non-zero values, it means that there are unhealthy computing nodes. Go to step 2.11.

Step 2.11: The first policy setting includes three options: prompt alarm, automatic migration task out, alarm and automatic migration task out; the second policy setting includes three options: prompt alarm, automatic shutdown, and alarm simultaneous automatic shutdown; the third policy There are three options including alerting, automatic migration task return, and automatic migration task return at the same time as the alarm. Perform corresponding operations on computing nodes according to the requirements of setting policies:

Check whether there are processing tasks in unhealthy computing nodes. If there are processing tasks, query the first policy and process them according to the operation of the first policy; if there are no processing tasks, query the second policy and process them according to the operations of the second policy ;

Check whether there are processing tasks in the healthy computing node. If there are processing tasks, no operation is performed; if there are no processing tasks, it means that the computing node has recovered from an unhealthy state to a healthy state, and record the current timestamp. If the current time If the difference between the stamp and the last time stamp is less than the second preset time T2, or if there is no last time stamp, no operation will be done, and if the difference is greater than or equal to the second preset time T2, then the third strategy will be processed

Waiting time T1 returns to step 2.1.

The parameters involved in the analysis and processing of health status data are defined as follows:

n: indicates the number of participating computing nodes;

p: indicates the number of times to collect health status data;

k1: Indicates the number of time points that can tolerate exceeding the reference range of the health state, the value is not greater than p, k1/p represents the tolerance for exceeding the reference range of the health state (the larger the more conservative, the smaller the more aggressive);

k2: Indicates the number of time points that can be tolerated below the reference range of the health state, the value is not greater than p, and k2/p represents the tolerance for below the reference range of the health state (the larger the more conservative, the smaller the more aggressive);

T1: Indicates the time period for each health status data analysis and processing. This value should not be less than the time period for health status data collection. Otherwise, duplicate data will be collected.

As shown in Figure 11, the embodiment of the present invention also provides a system for implementing a high-availability system based on health management, and the system for implementing a high-availability system based on health management includes the following modules:

The node configuration module 10 is configured to configure at least two control nodes and at least two computing nodes. It is also used to set one of the control nodes as the master control node, set the other control nodes as standby control nodes and configure the priority order of the standby control nodes. And it is used to set the priority of the master control node higher than any standby control node.

The data collection module 20 is configured to repeatedly collect and store health status data of all computing nodes through each control node.

Preferably, as shown in Figure 12, the data acquisition module 20 includes the following units:

The data collection unit 21 is configured to collect the health status data of all the computing nodes at intervals of a fixed time T1 by each control node, and save the collected health status data to the memory of each control node.

The number of collection times judging unit 22 is used to judge whether the number of times of collection is less than the predetermined number p, and when the number of collections is less than the predetermined number p, the function of the data collection unit 21 is activated. It is also used to delete redundant historical health state data on the memory of each control node when the number of times of collection is greater than the predetermined number p, and keep the health state data collected for the latest p times. And it is used to directly retain the health status data collected for the latest p times when the number of times of collection is equal to the predetermined number of times p;

The matrix configuration unit 23 is configured to set the health status data of n computing nodes collected by each control node p times into a matrix of historical health status data with p rows and n columns.

The node judging module 30 is used to judge whether the current control node is the main control node, and if the current control node is the main control node, jump to the data analysis module 50 . If the current control node is the standby control node, then jump to the fault judging module 40 .

The fault judging module 40 is configured to use the standby control node with the highest priority to detect whether the master control node fails in real time through heartbeat. It is also used to use the standby control node with a lower priority to detect whether the standby control node with a higher priority than the standby control node fails in real time through heartbeat. It is also used to modify the priority order when the control node fails.

Preferably, as shown in Figure 13, the fault judgment module 40 includes the following units:

The fault judging unit 41 is used to detect whether the main control node fails in real time through the heartbeat of the standby control node with the highest priority; Whether the node has failed.

The first adjustment unit 42 is configured to configure the standby control node with the highest priority as the primary control node when a failure of the primary control node is detected, and set the priorities of other standby control nodes other than the standby control node with the highest priority Each raises a level.

The second adjustment unit 43 is configured to increase the priorities of all the standby control nodes whose priorities are lower than the failed standby control node by one level when it is detected that the standby control node fails.

The data analysis module 50 is configured to analyze the health status data stored in the memory of the main control node through the main control node, and manage the computing nodes according to the analysis results.

Preferably, as shown in Figure 14, the data analysis module 50 includes the following units:

The analysis unit 51 is configured to analyze the historical value matrix of the health status data column by column through the main control node, and each column of health status data corresponds to a computing node.

The analyzing and judging unit 52 is configured to judge whether all computing nodes have been analyzed once, and when all computing nodes have been analyzed once, the function of the computing node judging unit is activated. It is also used to start the function of the statistical unit 53 when there are computing nodes that are not analyzed.

The statistical unit 53 is configured to respectively count the number of acquisitions of the current computing node exceeding the health state reference range and below the health state reference range.

A state judging unit 54, configured to determine that the computing node is in a healthy state when the number of collections exceeding the health state reference range is less than k1, and the number of collections below the health state reference range is less than k2, and start setting the first flag value Function of unit 55. And it is used to determine that the computing node is in an unhealthy state when the number of collections exceeding the reference range of the health state is greater than or equal to k1, or the number of collections below the reference range of the health state is greater than or equal to k2, and the second flag value is activated Sets the function of unit 56.

The first flag value setting unit 55 is configured to set the unhealthy flag value of the computing node to 0, and start the function of the analysis unit 51 .

The second flag value setting unit 56 is configured to obtain the maximum value of the number of acquisitions exceeding the health state reference range and the number of acquisitions below the health state reference range, and set the maximum value as the unhealthy state value of the computing node, And start the function of the analyzing unit 51 .

The computing node judging unit 57 is configured to analyze the unhealthy flag values of all computing nodes, and judge whether there are computing nodes with non-zero values. It is also used to determine that all computing nodes are healthy when there are no computing nodes with non-zero values, and to determine that there are unhealthy computing nodes when there are computing nodes with non-zero values.

The computing node management unit 58 is configured to manage the computing nodes according to the set policy, and start the function of the data collection unit 21 after waiting for a first preset time T1.

Preferably, as shown in FIG. 15, the computing node management unit 58 includes the following subunits:

The strategy setting subunit 581 is used to set the first strategy, the second strategy, and the third strategy. The first strategy includes three options: prompt alarm, automatic migration task out, and alarm and automatic migration task out; the second strategy includes prompt alarm, There are three options: automatic shutdown, alarm and automatic shutdown; the third strategy includes three options: prompt alarm, automatic migration task return, alarm and automatic migration task return.

The first task judging subunit 582 is configured to judge whether there are processing tasks in the unhealthy computing nodes, and when there are processing tasks in the unhealthy computing nodes, query the first strategy and perform processing according to the first strategy. And it is used for querying the second strategy and performing processing according to the second strategy when there is no processing task in the unhealthy computing node.

The second task judging subunit 583 is configured to judge whether a healthy computing node has a processing task, and when the healthy computing node has a processing task, no operation is performed on the healthy computing node. When there is no processing task in a healthy computing node, it is determined that the healthy computing node has recovered from an unhealthy state to a healthy state, and the healthy computing node is timed, and no operation is performed when the timing is less than the second preset time T2 , when the timing reaches T2, it will be processed according to the third strategy.

The skip subunit 584 is configured to start the function of the data collection unit 21 after waiting for the first preset time T1.

The high-availability system based on health management provided by the present invention implements the system, through flexible configuration of the number and priority of control nodes, the configuration is very flexible, and the fault-tolerant calculation of other application programs is supported. In addition, the main control node analyzes the health status data stored in the memory of the main control node, manages the computing nodes according to the analysis results, and realizes the prediction of the trending loss failure of the computing nodes in advance. The health status data is obtained through the bypass of the computing node, which does not occupy the CPU, memory, network and other resources of the computing node. Compared with the dual-machine cold backup technology, dual-machine hot backup technology, and virtual machine high-availability fault-tolerant technology, it saves system resource overhead .

The above device embodiments correspond to the method embodiments one by one, and for the briefness of the device embodiments, refer to the method embodiments.

Each embodiment in this specification is described in a progressive manner, each embodiment focuses on the difference from other embodiments, and the same and similar parts of each embodiment can be referred to each other.

Professionals can further realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software or a combination of the two. In order to clearly illustrate the possible For interchangeability, in the above description, the composition and steps of each example have been generally described in terms of functionality. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not exceed the scope of the present invention.

The steps of the methods or algorithms described in connection with the embodiments disclosed herein may be directly implemented by hardware, software modules executed by a processor, or a combination of both. Software modules can be placed in random access memory, internal memory, read-only memory, electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form known in the technical field in the storage medium.

Embodiments of the present invention have been described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementations, and the above-mentioned specific implementations are only illustrative, rather than restrictive. Those of ordinary skill in the art will Under the enlightenment of the present invention, many forms can also be made without departing from the gist of the present invention and the protection scope of the claims, and these all belong to the protection of the present invention.

Claims (8)

1. A method for realizing a high-availability system based on health management, characterized in that, the method for realizing a high-availability system based on health management comprises the steps: S1. Set at least two control nodes and at least two computing nodes; set one of the control nodes as the master control node, set the other control nodes as standby control nodes and configure the priority order of the standby control nodes; set the master control node priority is higher than that of any standby control node; S2. Each control node repeatedly collects and stores the health status data of all computing nodes; S3. Judging whether the current control node is the main control node, if the current control node is the main control node, then jump to step S5; if the current control node is the standby control node, then jump to step S4; S4. The standby control node with the highest priority detects whether the main control node fails through heartbeat in real time; the standby control node with low priority detects through heartbeat in real time whether the standby control node with a higher If there is a fault, modify the priority order; S5. The master control node analyzes the health status data stored in the memory of the master control node, and manages the computing nodes according to the analysis results; Wherein, step S2 specifically includes the following sub-steps: S21. Each control node collects the health state data of all computing nodes once, and saves the collected health state data in the memory of each control node; S22. Determine whether the number of collections is less than the predetermined number p, if the number of collections is less than the predetermined number p, then wait for the first preset time T1 and then jump to step S21; if the number of collections is greater than the predetermined number p, delete each control node For redundant historical health status data on the memory, keep the latest p health status data collected, then jump to step S23; if the number of times collected is equal to the predetermined number p, then jump to step S23; S23. Set the health status data of n computing nodes collected by each control node p times into a matrix of historical health status data with p rows and n columns. 2. The method for realizing a high-availability system based on health management according to claim 1, wherein said step S4 comprises the following sub-steps: S41. The standby control node with the highest priority detects whether the main control node fails in real time through heartbeat; the standby control node with low priority detects in real time whether the standby control node with a higher priority than the standby control node fails through heartbeat; S42. If it is detected that the master control node fails, the standby control node with the highest priority is configured as the master control node, and the priorities of the standby control nodes other than the standby control node with the highest priority are respectively increased by one level; S43. If it is detected that the standby control node fails, the priorities of all the standby control nodes whose priorities are lower than the failed standby control node are respectively increased by one level. 3. The method for realizing a high-availability system based on health management according to claim 2, wherein said step S5 comprises the following sub-steps: S51. The main control node analyzes the historical value matrix of the health status data column by column, and each column of health status data corresponds to a computing node; S52. Determine whether all computing nodes have been analyzed once, if all computing nodes have been analyzed once, then jump to step S57; if there are computing nodes that have not been analyzed, then jump to step S53; S53, respectively counting the number of acquisitions of the current computing node exceeding the health state reference range and below the health state reference range; S54. If the number of acquisitions exceeding the health status reference range is less than k1, and the number of acquisitions lower than the health status reference range is less than k2, determine that the computing node is in a healthy state, and jump to step S55; if it exceeds the health status reference If the number of acquisitions in the range is greater than or equal to k1, or the number of acquisitions lower than the reference range of the healthy state is greater than or equal to k2, then it is determined that the computing node is in an unhealthy state and jumps to step S56; S55. Set the unhealthy flag value of the computing node to 0, and jump to step S51; S56. Obtain the maximum value of the number of acquisitions exceeding the health state reference range and the number of acquisitions below the health state reference range, set the maximum value as the unhealthy state value of the computing node, and jump to step S51; S57. Analyze the non-health flag values of all computing nodes, and determine whether there are computing nodes with non-zero values; if there are no non-zero computing nodes, determine that all computing nodes are healthy, and if there are non-zero computing nodes node, it is determined that there is an unhealthy computing node; S58. Manage the computing nodes according to the set policy, wait for the first preset time T1, and then jump to step S21. 4. The method for realizing a high-availability system based on health management according to claim 3, wherein the step S58 includes the following sub-steps: S581. Set the first strategy, the second strategy, and the third strategy. The first strategy includes three options: prompt alarm, automatic migration task out, and alarm simultaneous automatic migration task out; the second strategy includes prompt alarm, automatic shutdown, and alarm simultaneous automatic Shutdown and other three options; the third strategy includes three options: prompt alarm, automatic migration task return, alarm and automatic migration task return; S582. Determine whether there is a processing task in the unhealthy computing node, if there is a processing task in the unhealthy computing node, query the first strategy and perform processing according to the first strategy; if there is no processing task in the unhealthy computing node, query the first strategy The second strategy and handle it according to the second strategy; S583. Determine whether a healthy computing node has a processing task. If the healthy computing node has a processing task, no operation is performed on the healthy computing node; if the healthy computing node does not have a processing task, determine that the healthy computing node is Restore from an unhealthy state to a healthy state, and record the current time stamp. If the difference between the current time stamp and the last time stamp is less than the second preset time T2, or if there is no last time stamp, no action will be taken. If the difference is greater than or equal to the second preset time T2, then proceed according to the third strategy; S584. Jump to step S21 after waiting for the first preset time T1. 5. A high-availability system implementation system based on health management, characterized in that, the high-availability system implementation system based on health management includes the following modules: The node configuration module is used to set at least two control nodes and at least two computing nodes; set one of the control nodes as the main control node, set the other control nodes as standby control nodes and configure the priority order of the standby control nodes; Set the priority of the master control node to be higher than that of any standby control node; The data collection module is used to repeatedly collect and store the health status data of all computing nodes through each control node; The node judgment module is used to judge whether the current control node is the main control node, if the current control node is the main control node, then jump to the data analysis module; if the current control node is the backup control node, then jump to the fault judgment module; The fault judgment module is used to detect whether the main control node fails in real time through the heartbeat of the standby control node with the highest priority; through the heartbeat of the standby control node with low priority to detect whether the standby control node with a higher priority exists in real time Fault; when the control node fails, modify the priority order; The data analysis module is used to analyze the health status data stored in the memory of the main control node through the main control node, and manage the computing nodes according to the analysis results; Wherein said data acquisition module includes the following units: The data collection unit is used to collect the health status data of all computing nodes once every fixed time T1 through each control node, and save the collected health status data to the respective memory of each control node; The number of collection judgment unit is used to judge whether the number of times collected is less than the predetermined number p, and when the number of times collected is less than the predetermined number p, the function of the data acquisition unit is started; when the number of collections is greater than the predetermined number p, each control node memory is deleted For redundant historical health status data, keep the health status data collected for the latest p times; when the number of times collected is equal to the predetermined number p, directly keep the health status data collected for the latest p times; The matrix configuration unit is configured to set the health state data of n computing nodes collected by each control node p times into a matrix of historical value of health state data with p rows and n columns. 6. The system for implementing a high-availability system based on health management according to claim 5, wherein the fault judgment module comprises the following units: The fault judgment unit is used to detect whether the main control node is faulty through the heartbeat of the highest priority standby control node in real time; through the heartbeat of the low priority standby control node, it is used to detect whether the standby control node with a higher priority than the standby control node appears in real time Fault; The first adjustment unit is configured to configure the standby control node with the highest priority as the primary control node when a failure of the primary control node is detected, and configure the priorities of the standby control nodes other than the standby control node with the highest priority move up a level; The second adjustment unit is configured to increase the priorities of all standby control nodes whose priorities are lower than the failed standby control node by one level when it is detected that the standby control node fails. 7. The system for realizing a high-availability system based on health management according to claim 6, wherein the data analysis module comprises the following units: The analysis unit is used to analyze the historical value matrix of the health status data column by column through the main control node, and each column of health status data corresponds to a computing node; The analysis and judgment unit is used to judge whether all the computing nodes have been analyzed once. When all the computing nodes have been analyzed once, the function of the computing node judging unit is started; when there are computing nodes that have not been analyzed, the function of the statistical unit is started; The statistics unit is used to count the number of acquisitions of the current computing node exceeding the reference range of the health state and below the reference range of the health state; A state judging unit, configured to determine that the computing node is in a healthy state when the number of collections exceeding the health state reference range is less than k1, and the number of collections below the health state reference range is less than k2, and start the first flag value setting unit function; when the number of acquisitions exceeding the reference range of the health state is greater than or equal to k1, or the number of acquisitions below the reference range of the health state is greater than or equal to k2, it is determined that the computing node is in an unhealthy state, and the second flag is activated the function of the value setting unit; The first flag value setting unit is used to set the unhealthy flag value of the computing node to 0, and start the function of the analysis unit; The second flag value setting unit is used to obtain the maximum value of the number of acquisitions exceeding the health state reference range and the number of acquisitions below the health state reference range, and set the maximum value as the unhealthy state value of the computing node, and function to activate the analysis unit; The computing node judging unit is used to analyze the unhealthy flag values of all computing nodes, and judge whether there are computing nodes with non-zero values; when there are no non-zero computing nodes, it is determined that all computing nodes are healthy. Computing nodes with non-zero value, it is determined that there are unhealthy computing nodes; The computing node management unit is configured to manage the computing nodes according to the set policy, and start the function of the data collection unit after waiting for a first preset time T1. 8. The system for implementing a high-availability system based on health management according to claim 7, wherein the computing node management unit comprises the following subunits: The strategy setting subunit is used to set the first strategy, the second strategy, and the third strategy. The first strategy includes three options: prompt alarm, automatic migration task out, and alarm and automatic migration task out; the second strategy includes prompt alarm, automatic There are three options: shutdown, alarm and automatic shutdown; the third strategy includes three options: prompt alarm, automatic migration task return, alarm and automatic migration task return; The first task judging subunit is used to judge whether there is a processing task in the unhealthy computing node. When there is a processing task in the unhealthy computing node, query the first strategy and perform processing according to the first strategy; when the unhealthy computing node does not When there is a processing task, query the second strategy and process it according to the second strategy; The second task judging subunit is used to judge whether there is a processing task in a healthy computing node, and when there is a processing task in a healthy computing node, no operation is performed on the healthy computing node; when there is no processing task in a healthy computing node, Determine that the healthy computing node has recovered from an unhealthy state to a healthy state, and record the current timestamp, if the difference between the current timestamp and the last timestamp is less than the second preset time T2, or there is no last time stamp, then do not do any operation, if the difference is greater than or equal to the second preset time T2, it will be processed according to the third strategy; The jumping subunit is used to start the function of the data acquisition unit after waiting for the first preset time T1.