WO2015045589A1 - Fault-tolerant system and fault-tolerant system control method - Google Patents

Abstract

Provided is a fault-tolerant system with a plurality of computers which form a majority voting redundancy configuration, each of which is provided with: a task communication unit (140) for acquiring process content from a program which operates in that computer (110B, 110C); and a message processing unit (142) which receives, from any of the other computers (110A) which form the majority voting redundancy configuration, a prescribed message which indicates the process content of a program which operates on the other computer (110A) and returns a replication permissibility message to the other computer (110A) according to the result of a match determination between the process content indicated by the prescribed message and the process content obtained by the task communication unit (140). The message processing unit (142) of the computer which is the other computer (110A) further executes a process for changing the number of computers (110B, 110C), aside from the other computer, for which the other computer is to await receipt of replication permissibility messages, depending on the process content obtained by the task communication unit (140) of the other computer (110A).

Description

Fault tolerant system and fault tolerant system control method

The present invention relates to a fault-tolerant system and a fault-tolerant system control method, and more specifically, in a fault-tolerant system composed of a plurality of computer nodes having different architectures and infrastructure software, etc., the performance difference between the computer nodes. The present invention relates to a technology that can improve the availability of a system by reducing the influence of the overall performance due to the fluctuations and preventing excessive failure determination due to such influence.

In the mission critical computer system, the concept of fault tolerance has been introduced to ensure high availability by giving a predetermined redundancy. As a technology corresponding to such a concept, a system is proposed in which the corresponding system is composed of a plurality of computers, the output signals of the computers operating in synchronization are compared and monitored by majority vote, and the defective computer is separated from the system. Has been. For example, the user may specify “interacting with the interface, monitored variables, ie variables to be voted on, breakpoints on which the majority will be taken, and information such as majority and recovery parameters” and “specified breakpoints” There is a conventional technique (see Patent Document 1) in which a fault is detected in one of the copies by comparing one or more values of user-specified variables generated by different copies.

Japanese Patent No. 3624119

By the way, even in a fault tolerant system (hereinafter referred to as the same kind of FT system) configured by a plurality of computer nodes using the same hardware and the same hardware, the execution time when the same task is executed in each computer node is Variations occur between computer nodes. This variation stems from performance differences between computer nodes in a fault-tolerant system (hereinafter referred to as a heterogeneous FT system) in which computer nodes differing in one or more of the architecture, performance, and basic software of the central processing unit. There is a tendency to become more prominent.

Since the time required to execute a certain program varies depending on various external factors as well as the hardware performance of the computer node, it is generally difficult to predict. For this reason, in both cases of the homogeneous FT system and the heterogeneous FT system, the difference in execution time between the computer nodes is allowed, and the determination of the failed node based on the event of no response or a large processing delay and its elimination Must be done without sacrificing availability.

On the other hand, in the prior art, when making a majority decision between outputs, when comparing variable contents at a certain point in time, it is necessary to collect and compare variable contents from each copy (computer node) synchronously. Nodes need to be synchronized at the appropriate time. This is necessary to absorb the variation in the execution time of each copy at the corresponding time, which may lead to a decrease in the overall processing efficiency. Also, it is difficult to set a failure determination time that allows such variations due to difficulty in predicting the execution time.

Accordingly, an object of the present invention is to reduce the influence of the overall performance due to the difference in performance and variation between the computer nodes in a fault tolerant system composed of a plurality of computer nodes having different architectures and basic software, etc. It is an object of the present invention to provide a technology capable of improving availability in a system by preventing excessive failure determination.

The fault-tolerant system of the present invention that solves the above-described problem includes, in each of a plurality of computers forming a majority redundant configuration, a task communication unit that acquires the processing contents from a program operating inside the computer, and the majority redundant configuration. Receiving a predetermined message indicating the processing content of the program operating on the other computer from any other computer to be formed, and determining whether the processing content indicated by the predetermined message and the processing content obtained by the task communication unit are consistent A message processing unit that returns a message indicating whether or not follow-up is possible to the other computer according to the result, and the message processing unit of the computer that is the other computer is a process obtained by the task communication unit of the other computer. Depending on the content, change the number of computers other than the other computers that are subject to reception of the followability message. It is to a process for further execution, and wherein the.

Further, the fault tolerant control method of the present invention is a method in which, in each of a plurality of computers forming a majority voting redundant configuration, a process for obtaining the processing contents from a program operating inside the computer, and forming the majority voting redundant configuration From another computer, a predetermined message indicating the processing content of the program operating on the other computer is received, and according to a result of the matching determination between the processing content indicated by the predetermined message and the processing content obtained by the processing, Processing for returning a followability message to another computer, and, in the other computer, in accordance with the processing content obtained from the program operating on the other computer, the target of waiting for reception of the followability message Further, a process of changing the number of computers other than the other computer is further executed.

According to the present invention, in a fault-tolerant system composed of a plurality of computer nodes having different architectures, infrastructure software, and the like, the influence of the overall performance due to the performance difference and variation between the computer nodes is reduced, and this influence is also caused. By preventing excessive fault determination, availability in the system is improved.

It is a figure which shows the structural example of the fault tolerant system which concerns on 1st Embodiment. It is a figure which shows the example of a relationship between the some computer nodes which comprise the fault tolerant system which concerns on 1st Embodiment. It is a figure which shows the structural example of the approval message list which the computer node which comprises the fault tolerant system which concerns on 1st Embodiment hold | maintains. It is a figure which shows the structural example of the notification message list which the computer node which comprises the fault tolerant system which concerns on 1st Embodiment hold | maintains. It is a figure which shows the structural example of the message type table which the computer node which comprises the fault tolerant system which concerns on 1st Embodiment hold | maintains. It is a figure which shows the structural example of the data transferred between the some computer nodes which comprise the fault tolerant system which concerns on 1st Embodiment. It is a figure which shows the example of an operation | movement flow of each process part of the computer node in the fault tolerant system which concerns on 1st Embodiment. It is a figure which shows the example 1 of an operation | movement flow in the message processing part of the leader node in the fault tolerant system which concerns on 1st Embodiment. It is a figure which shows the example 2 of an operation | movement flow in the message processing part of the leader node in the fault tolerant system which concerns on 1st Embodiment. It is a figure which shows the example of an operation | movement flow in the message processing part of a follower node in the fault tolerant system which concerns on 1st Embodiment. It is a figure which shows the example 1 of an operation | movement flow in the message processing part of the follower node of the fault tolerant system in 2nd Embodiment. It is a figure which shows the example 2 of an operation | movement flow in the message processing part of the follower node of the fault tolerant system in 2nd Embodiment. It is a figure which shows the example of an operation | movement flow in the message processing part of the leader node of the fault tolerant system in 3rd Embodiment. It is a figure which shows the example of an operation | movement flow in the message processing part of the leader node of the fault tolerant system in 4th Embodiment.

This application claims priority based on Japanese Patent Application No. 2013-199614 filed on Sep. 26, 2013, the contents of which are incorporated herein by reference.

--- First embodiment ---
Hereinafter, a first embodiment of the present invention will be described in detail with reference to the drawings. In the fault-tolerant system of the first embodiment, the same task is executed by the computer that is the leader and the computer that is the follower in the majority redundant configuration. The leader sequentially sends a notification message including the task processing state to the follower, and the follower accepts or rejects it as an approval / denial message (follow-up / possibility message). Respond to the leader. In this case, the leader makes a determination on the continuation of the operation corresponding to the majority redundant configuration (for example, message waiting) for the corresponding task according to the response from the follower, that is, the content of the message, the reception timing, and the message type.

FIG. 1 is a diagram illustrating a configuration example of a fault tolerant system according to the first embodiment. The fault-tolerant system 1 shown in FIG. 1 is a fault-tolerant system composed of a plurality of computer nodes having different architectures and basic software, etc., and reduces the influence of overall performance due to performance differences and variations between computer nodes. It is a computer system that improves availability in the system by preventing excessive failure determination due to influence.

The fault tolerant system 1 includes a plurality of computer nodes 110. In the fault tolerant system 1 of FIG. 1, an example is shown in which the system configuration includes three computer nodes 110A, 110B, and 110C as the computer node 110. Of course, the number of computer nodes constituting the fault-tolerant system 1 is not limited to “3”, and may be, for example, “2” or “4” or more.

In the fault tolerant system 1 corresponding to the majority redundant configuration, one of the computer nodes 110 is a leader node and the remaining computer nodes 110 are follower nodes. The roles of the leader node and the follower node need not be fixed in the computer node 110. For example, for each task, that is, for each of the above-described notification messages related to the task, the role of each computer may be changed using an algorithm such as the computer node 110 that first transmitted the notification message as a leader node. Good. However, at any time during the operation of the fault tolerant system 1, there is at most one leader node. When the leader node does not exist, or when a failure occurs in the leader node and the process stops and the leader node does not exist, the fault tolerant system 1 interrupts the process, and one computer is selected from the follower nodes. Node 110 will be selected as the leader node. As a processing technique for selecting a leader node from among computer nodes having such a majority redundant configuration, an existing one may be adopted. In the first embodiment, among the three computer nodes 110, the computer node 110A is a leader, and the computer node 110B and the computer node 110C are followers.

The hardware configuration of each computer node 110 is as follows. The computer node 110 includes a main storage device 121, one or more central processing units 120, an external communication interface 122, and an inter-node communication interface 123. Among these, the external communication interface 122 is connected to one or more terminals 102 via the external network 111. The inter-node communication interface 123 is connected to another computer node 110 via an inter-node network 112. These networks 111 and 112 are networks configured in conformity with, for example, the Ethernet (registered trademark) standard or the Infiniband standard.

The main storage device 121 in the computer node 110 is a volatile storage device that can read and write data, and is accessed from the central processing unit 120. The main storage device 121 is composed of, for example, a DRAM (Dynamic Random Access Memory).

The central processing unit 120 in the computer node 110 is a general-purpose processor, and reads main software 130, fault-tolerant middleware (hereinafter referred to as FT middleware) 131, an application 132, and the like from the auxiliary storage device 124 and the main memory. The program is expanded on the device 121 and the program is executed. The architecture adopted by the central processing unit 120 and the processing speed thereof may not be the same among a plurality of computer nodes 110. Each computer node 110 may include two or more central processing units 120.

The auxiliary storage device 124 in the computer node 110 is a permanent non-volatile storage device, and stores data read / written from these programs in addition to programs such as the basic software 130, the FT middleware 131, and the application 132. The The auxiliary storage device 124 is configured by, for example, a hard disk, a flash memory, or the like.

Further, the external communication interface 122 in the computer node 110 is an interface for the computer node 110 to communicate with the terminal 102. On the other hand, the inter-node communication interface 123 is an interface for performing communication between the computer nodes 110. Note that the external communication interface 122 and the inter-node communication interface 123 do not have to be separate devices, and one communication interface may also serve as the external communication interface 122 and the inter-node communication interface 123. The external communication interface 122 and the inter-node communication interface 123 are configured by communication interfaces corresponding to the types of the external network 111 and the inter-node network 112, respectively.

Further, the application 132 held in the main storage device 121 is a program for realizing the function provided by the FT system 1. The application 132 is composed of one or more tasks 133. In the FT system 1, the same application 132 is executed by a plurality of computer nodes 110. At this time, the application 132 may have a difference according to a difference in the architecture, infrastructure software, or the like of the computer node 110 to be executed. That is, the instruction group for the central processing unit 120 constituting the application 132 and its task 133 need not be the same among the computer nodes 110.

Further, the basic software 130 held in the main storage device 121 is an operating system capable of multitask processing, and can execute a plurality of tasks 133 and FT middleware 131 in parallel. The basic software 130 accesses the FT middleware 131 to the auxiliary storage device 124, functions to transmit and receive data to the external communication interface 122 and the inter-node communication interface 123, and secures and releases a partial occupied area of the main storage device 121. Provides the functionality of

Also, the FT middleware 131 held in the main storage device 121 is a program for realizing a fault tolerant function in the FT system 1. The FT middleware 131 provides the above-described application 132 with functions such as access to the auxiliary storage device 124, data transmission / reception with respect to the external communication interface 122, and exclusive processing with respect to a part of the main storage device 121. About these functions, the FT middleware 131 is implement | achieved using the function which the basic software 130 grade | etc., Provides. The FT middleware 131 may be an independent program, may be integrated into the application 132, or may be integrated into the basic software 130. Further, the application 132, the FT middleware 131, and the basic software 130 may serve as a single program.

The FT middleware 131 includes a task communication unit 140, an inter-node communication unit 141, and a message processing unit 142. The task communication unit 140, the inter-node communication unit 141, and the message processing unit 142 can be said to be functions that are implemented when the central processing unit 120 of each computer node 110 executes the FT middleware 131.

Among these, the task communication unit 140 receives a processing request from the task 133 and transmits the request to the message processing unit 142. On the other hand, the message processing unit 142 executes the requested processing after synchronizing with other computer nodes 110 and transmits the result to the task communication unit 140. The task communication unit 140 that has obtained the above result from the message processing unit 142 informs the task 133 of the corresponding result.

When the message processing unit 142 communicates with another computer node 110, the inter-node communication unit 141 is requested to transmit data via the inter-node communication interface 123. On the other hand, the inter-node communication unit 141 transmits the data received from the inter-node communication interface 123 to the message processing unit 142. Arbitrary means can be adopted as means for communication between the task 133 and the task communication unit 140, the task communication unit 140 and the message processing unit 142, and the message processing unit 142 and the inter-node communication unit 141. For example, as the above-described transmission means, function calls and their return values, communication means provided by the basic software 130 such as sockets and pipes, and the like can be used.

Further, when the inter-node communication unit 141 receives data from the inter-node communication interface 123, the inter-node communication unit 141 transmits the contents to the message processing unit 142. In addition, when a request for data transmission to another computer node 110 is received from the message processing unit 142, transmission is performed using the inter-node communication interface 123 to the computer node 110 whose contents are designated.

The FT middleware 131 includes an approval message list 150, a notification message list 151, and a message type table 152 in addition to the processing units 140, 141, 142 described above. These retained data and their roles will be described later. The message type table 152 is not necessarily required for processing. This will also be described later.

Subsequently, the relationship between the computer nodes 110 constituting the FT system 1 will be described. FIG. 2 is a diagram illustrating a relationship example between a plurality of computer nodes constituting the FT system 1 according to the first embodiment, and a relationship example of each part necessary for the FT system 1 to realize a fault tolerant function. It is the figure shown about. The fault tolerant function realized by the FT system 1 is a function that can continuously provide the function of the task 133 even if a hardware or software failure of less than half of the computer nodes 110 occurs.

Therefore, as described above, the same task 133 is operated in a plurality of computer nodes 110. When the task 133 performs input from the outside or output to the outside, or when determining the operation order of a plurality of threads, the processing in which operations must be matched between the computer nodes 110 (hereinafter referred to as “operation matching”). In this case, messages are exchanged between the computer nodes 110 so that the task 133 shows the same behavior in all the computer nodes 110.

For this purpose, each task 133 calls the FT middleware 131 at each operation matching point and requests execution of the process. On the other hand, the FT middleware 131 compares the content with the request of the task 133 of the other computer node 110, and if it is the same for the majority of the computer nodes 110, actually executes it. As a result, even if less than half of the computer nodes 110 perform no response, response delay, or abnormal response due to hardware failure or software failure, the operation of the task 133 is continued according to the result of the other computer node 110. be able to.

In the present embodiment, as described above, the computer node 110 is divided into two types, the leader node 110A and the follower nodes 110B and 110C, according to roles, in order to increase the efficiency of consensus formation between the computer nodes 110 at the operation matching point. Shall. At the operation matching point, the leader node 110A determines the operation, and notifies the follower nodes 110B and 110C of the operation content corresponding to the processing request by the notification message 201. The follower nodes 110B and 110C collate with the processing request from their own task 133, and can follow this operation content, that is, if it is possible to realize the same behavior as the leader node 110A, the approval message 202 is impossible. If the message is rejected, a reply message 203 is returned to the leader node 110A. The leader node 110A treats the approval message 202 as an approval vote in the majority vote and the denial message 203 as a negative vote, and when the majority receives the approval message 202, it actually executes this. The basic behavior of the leader node 110A and the follower nodes 110B and 110C accompanying such a majority vote redundant configuration is the same as that of the prior art.

Subsequently, operations of the task communication unit 140, the message processing unit 142, and the inter-node communication unit 141, which are structures in the FT middleware 131, and an approval message list 150, a notification message list 151, which are data structures used at that time, The message type table 152 will be described.

3, 4, and 5 are diagrams illustrating examples of internal structures of the approval message list 150, the notification message list 151, and the message type table 152, respectively. Among these, the approval message list 150 and the notification message list 151 are used by the message processing unit 142 as follows. When the message processing unit 142 receives a processing request from the task communication unit 140, if the computer node 110 is a leader node, the message processing unit 142 transmits this request to another computer node, that is, a follower node. The message processing unit 142 records the reception history of the approval message and the denial message in the approval message list 150 in order to manage whether or not the approval message or the denial message is received from the follower node. On the other hand, the follower node uses a notification message list 151 that is a list of notification messages 201 in order to manage whether or not a notification message is received from the leader node.

The approval message list 150 is a table having values such as a notification reference 300, an approval node set 301, and a denial node set 302 as rows. The notification reference 300 is used for reference to the notification message 201 managed by the row (reference may be any method such as by identifier, by memory address, or by reference destination copy, and so on). This information becomes the identification information of the notification message 201. An approval node set 301 represents a set of computer nodes 110 that are follower nodes that have returned the approval message 202 in response to the notification message 201. Similarly, the denial node set 302 represents a set of computer nodes 110 that are follower nodes that have returned the denial message 203 to the notification message 201.

Further, the notification message list 151 has zero or more notification messages 201 sent from the leader node and not responding with the approval message 202 or the denial message 203 as a list.

Next, the processing structure to the task communication unit 140 and the message processing unit 142 and the data structure used for transmitting the result will be described. Hereinafter, “processing request” refers to a data structure for transmitting a processing request from a task to the message processing unit 142 from the task communication unit 140, and “processing result” refers to a task communication unit from the message processing unit 142. Reference numeral 140 denotes a data structure for transmitting a processing result (processing content) in response to a processing request. As an example of the internal structure of the processing request, there are a request type and a request parameter corresponding to the request type. On the other hand, as an example of the internal structure of a processing result, there are a processing request reference to a corresponding processing request and a processing result value that is a value representing the processing result. If the process to be referred to can be uniquely specified by the transmission method between the task communication unit 140 and the message processing unit 142, the process request reference can be omitted.

In the processing request, the request type indicates the type of request to the FT middleware 131 of the task 133, for example, reading from the auxiliary storage device 124, writing to the auxiliary storage device 124, receiving data from the external communication interface 122, or data Transmission, exclusive processing for a partial area of the main storage device 121, and the like. On the other hand, the request parameters are values having different meanings depending on the request type, and the number thereof also differs depending on the request type. Taking the case where the request type is “read from auxiliary storage device 124” as an example, the file name on the auxiliary storage device 124, the position on the file, the position of the main storage device 121 that stores the read value, and the read The number of bytes is included in the request parameter.

Further, the processing result is used to notify the task communication unit 140 from the message processing unit 142 of the processing result of the processing request notified so far from the task communication unit 140 to the message processing unit 142. The processing result value is a value having a different meaning depending on the request type of the processing request referred to by the processing request reference, and the number thereof also differs depending on the request type. Note that this processing result value does not need to express all the results for the request. For example, if the processing request has a side effect such as receiving data from the external communication interface 122 and copying to the designated main storage device 121, the message processing unit 142 performs processing for copying the designated data content to the main storage device 121. After that, the processing result is notified to the task communication unit 140. In such a case, it can be said that only the size of the received data is returned as the processing result value, and the information of the received data itself is not explicitly included in the processing result value.

FIG. 6 shows an example of the data structure of the notification message 201 used when the processing request is transmitted between the leader node and the follower node, and the approval message 202 and the denial message 203 as responses thereto. Here, the notification message 201 is a reference value to the request to be transmitted, and has a processing request 400 that defines the type of processing request. In addition to or instead of the reference value for the processing request, any content that is sufficient for content comparison in the follower node described later, that is, information that is sufficient to determine that the processing request has the same behavior The accompanying data 406 may be included.

Also, the approval message 202 and the denial message 203 have a notification reference 300 that is a value for referring to the notification message 201 sent from the corresponding leader node. If the notification message 201 can be uniquely identified, it can be substituted by referring to the processing request.

Next, the message type table 152 shown in FIG. 5 will be described. The message processing unit 142 determines the message synchronization type for each processing request as will be described later. The message synchronization type is, for example, “synchronous” or “semi-synchronous”, and specifies the process when the message processing unit 142 proceeds to the next process in response to the response or arrival of the message in response to the processing request. Is. For example, the message processing unit 142 in the leader node waits for responses from all the computer nodes 110 currently operating normally in the FT system 1 in response to a processing request whose message synchronization type is “synchronous”. The process proceeds to the next process, and for a message that is “semi-synchronous”, it waits for half of the approval messages 202 of the computer nodes 110 that are currently operating normally in the FT system 1 and then proceeds to the next process. Perform the action. As an example of this determination method, there is a method using the message type table 152. This message type table 152 is for determining the message synchronization type 304 from the request type 303 for the processing request described above. The message type table 152 is a table having values such as the request type 303 and the message synchronization type 304 as rows. Each line means that the type of message waiting behavior of the processing request having the request type 303 and the notification message 201 having the processing type as a processing request reference is the message synchronization type 304. Note that the determination of the message synchronization type is not limited to this method. For example, even when the message type table 152 does not exist, the computer node 110 assigns a serial number that uniquely identifies the message to the message, sets a message of a certain multiple number as “synchronized”, and sets the other as “semi-standard”. A method such as “synchronization” can also be used.

Subsequently, the operation and cooperation of the computer nodes 110A, 110B, and 110C constituting the FT system 1 will be described. FIG. 7 is a diagram illustrating an example of an operation flow of each processing unit of the computer node 110 in the FT system 1 according to the first embodiment. In this figure, the operation of the computer node 110A as the leader node and the computer node 110B as the follower node are shown. Since the other follower node 110C performs the same operation as the computer node 110B, description thereof is omitted.

First, the operation of the computer node 110A as a leader node will be described. The task communication unit 140 of the computer node 110A receives a request from the application 132 (510), and transmits this request to the message processing unit 142 in the form of a processing request 400 (511). The message processing unit 142 performs processing as needed according to the processing request 400, and sends a notification message 201 including the processing request 400 to the follower node by requesting the inter-node communication unit 141. (512).

Next, the message processing unit 142 of the computer node 110A determines the message synchronization type for the processing request 400 received from the task communication unit 140 (513). The message processing unit 142 waits for a response of the approval message 202 or the denial message 203 from the computer nodes 110B and 110C, which are follower nodes, according to the message synchronization type determined in step 513 (514). At this time, the message processing unit 142 of the computer node 110A is waiting to receive responses from all follower nodes (message synchronization type = “synchronization”), and is waiting to receive approval messages 202 from half of the follower nodes (message synchronization type = Wait according to the message synchronization type, such as “semi-synchronous”). After waiting for the response, the message processing unit 142 performs majority processing on the processing result of the corresponding task obtained from each follower node, and transmits the finally matched result to the task communication unit 140 as the processing result ( 515). The task communication unit 140 notifies the result to the application 132 (516).

On the other hand, the operation of the computer node 110B as a follower node will be described. Note that the operation of the task communication unit 140 of the computer node 110B that is the follower node is the same as that of the computer node 110A that is the leader node, and thus description thereof is omitted. The message processing unit 142 of the computer node 110B as the follower node waits for the notification message 201 from the computer node 110A as the leader node (517). When the notification message 201 is transmitted from the leader node, the message processing unit 142 of the computer node 110B receives the notification message 201 received from the leader node and the content of the processing request received from the task 133 by its own task communication unit 140. To determine whether it is possible to follow the behavior of the leader node (that is, processing can be executed synchronously for the same task). The data is transmitted to the leader node via the inter-node communication unit 141 (519).

Below, the details of the flow enclosed by the dotted line in FIG. 7 will be described. The flow (501) of the message processing unit 142 of the leader node will be described with reference to FIGS. 8 and 9, and the flow (502) of the message processing unit 142 of the follower node will be described with reference to FIG.

8 and 9 are diagrams showing operation flow examples 1 and 2 in the message processing unit of the leader node in the fault tolerant system according to the first embodiment. Specifically, the message processing unit 142 of the leader node has 5 is a flowchart showing an example of processing for receiving a processing request from the task communication unit 140, performing processing corresponding to the processing request, and returning the result to the task communication unit 140.

In this case, the message processing unit 142 of the leader node first receives the processing request from the task communication unit 140 (600), and when there is an external influence, that is, except for the case of transmission from the external communication interface 122, Processing corresponding to this processing request is performed (601). At this time, the message processing unit 142 generates request accompanying data (corresponding to the accompanying data 406 in FIG. 6) if necessary. Based on this, the message processing unit 142 generates a notification message 201 that refers to the received processing request, adds any data associated with the request, and adds it to all follower nodes (computer node 110B and computer node 110C). And transmitted via the inter-node communication unit 141 (602).

Next, the message processing unit 142 adds a line corresponding to the corresponding notification message 201 to the approval message list 150 in the FT middleware 131, that is, including the notification message 201 as the notification reference 300 (603). Next, the message processing unit 142 determines the message synchronization type for the processing request received in step 600 (604).

In step 604, the message processing unit 142 collates the above-described processing request information with the message type table 152 in the FT middleware 131, for example, so that the value of the request type 400 in each record of the message type table 152 is set. The line that matches the value of the processing request received in step 600 (eg, sync, Network_Send, Network_Recv, Lock_Acquire) is identified, and the message synchronization type 304 of the corresponding processing request is determined. However, as described above, the method for determining the message synchronization type is not limited to this method using the message type table 152, and any method can be adopted as long as it can determine the message synchronization type as one. .

As a result of step 604 described above, it is assumed that the message synchronization type of the processing request received in step 600 is “synchronization” (604: synchronization). At this time, the message processing unit 142 of the leader node requires responses from all the follower nodes operating at that time in order to continue the operation related to the processing request. That is, the message processing unit 142 of the leader node waits for an approval message 202 or a denial message 203 from all follower nodes (computer node 110B and computer node 110C) (605).

Thereafter, when the message processing unit 142 is notified of the reception of the approval message 202 from the inter-node communication unit 141 (hereinafter simply referred to as “received”) (606: Ack), the message processing unit 142 in the approval message list 150 ( (The row added in step 603 regarding the notification message 201 related to the corresponding processing request), the identification information of the computer node 110 of the transmission source (that is, the follower node that returned the approval message 202 to the notification message 201) in the approval node set 301. Is added (607).

On the other hand, when the message processing unit 142 receives the denial message 203 from the follower node (606: Nack), similarly, in the corresponding row of the approval message list 150, the message processing unit 142 adds the denial node set 302 to the transmission source computer node 110 (that is, The identification information of the follower node that responded with the denial message 202 to the notification message 201 is added (608).

The message processing unit 142 of the leader node repeats this until all follower nodes enter either the approval node set 301 or the denial node set 302 in the approval message list 150 (609). As a result, the message processing unit 142 of the leader node can confirm that the tasks 133 of all the follower nodes have arrived until the time when the processing request corresponding to the message 201 is executed.

If it is determined in step 609 that all follower nodes have entered either the approval node set 301 or the denial node set 302 in the approval message list 150 (609: Yes), the message processing unit 142 of the leader node performs the same task. It is determined whether or not the processing is to be continued according to the principle of majority vote for the execution result in each computer node (610). When the number of computer nodes that transmitted the approval message 202 is greater than or equal to the number of rejection messages (610: Yes), the message processing unit 142 of the leader node stops the node that transmitted the rejection message 203. This is because the leader node itself naturally agrees with the contents of the notification message 201. If the number of follower nodes that transmitted the approval message 202 and the number of follower nodes that transmitted the denial message 203 are the same, the majority of the nodes agree. Will be. If there are a large number of nodes that have sent the approval message 202 and the processing corresponding to the processing request has not yet been executed, the agreement of the majority of the computer nodes has been obtained at this point, so the leader node This is executed (611). Finally, the message processing unit 142 of the leader node generates the processing result of step 611 (612), transmits it to the task communication unit 140 (613), and waits for the next processing request from the task communication unit 140. Return to processing (600).

On the other hand, when the number of nodes that transmitted the denial message 203 is larger (610: No), it can be considered that a failure has occurred in the leader node itself, so the message processing unit 142 of the leader node performs the operation of the leader node itself. Processing at the time of failure such as stopping is executed (614).

On the other hand, it is assumed that the message synchronization type is “quasi-synchronization” as a result of the above-described step 604 (604: quasi-synchronization). At this time, the leader node requires a response from a majority of the follower nodes operating at that time in order to continue the operation related to the task. The operation is the same as that in the previous stage with respect to the reception of the approval message 202 and the denial message 203 (616, 617, 618, 620), but the conditions are different and whether more than half of the follower nodes are added to the approval node set 301. This is repeated until all follower nodes enter either the approval node set 301 or the denial node set 302 (619). The leader nodes themselves naturally agree with this decision, so that more than half of the follower nodes have sent the approval message 202 means that a majority of the nodes have responded. Subsequent majority decision result determination processing (610) and subsequent processing is the same as in the previous stage. Even after the leader node advances the processing, the remaining follower nodes may transmit the approval message 202 or the denial message 203. For this reason, when receiving these, the message processing unit 142 of the leader node performs an operation of adding to the corresponding approval node set 301 or denial node set 302 as needed. When the denial message 203 is received, it is known that this is a minority, and the node is immediately stopped.

The above-mentioned content is a case where all the follower nodes respond correctly, but as described in the background, the follower node is caused by a failure in the computer node 110, the inter-node network 112, the inter-node communication interface 123, or the like. May become unresponsive or the response may be significantly delayed. Under this situation, the process of waiting for reception (605) performed by the message processing unit 142 of the leader node when the message synchronization type is “synchronization” (604: synchronization) in the above-described flow is affected.

When a predetermined time (hereinafter referred to as timeout time) has elapsed from the transmission time of the notification message 201 in step 602 (606: timeout), the message processing unit 142 of the leader node has received the approval message 202 or the denial message by that time. It is assumed that a failure has occurred in the follower node that did not send 203. Then, the message processing unit 142 of the leader node proceeds to the determination processing (610) of the majority result. At this time, the follower node that is considered to have failed may be considered to be in the denial node set 302 (621), or may be determined by removing it from the total number of computer nodes.

∙ As the above timeout period, a constant value can be assumed in the simplest example. Alternatively, for example, the computer node 110 may determine the timeout time by an arbitrary algorithm. For example, a predetermined algorithm is used to predict a time for executing a predetermined process in accordance with the notification message 201 described above, and an expected arrival time of either an approval / denial message from the follower node is specified, and this is calculated as a timeout time. You may do it. Even if a constant is used, failure determination using the timeout time is performed only for all messages whose message synchronization type is "synchronous", so the majority of computer nodes are operating without significant delay. In some cases, even if there is a temporary delay in less than half of the computer nodes, that is, a minority computer node, it can be absorbed.

Also, an execution constraint time can be added to a request parameter of a processing request in which the message synchronization type 304 is “synchronous”, and a timeout time can be determined based on this. This is advantageous in that it is possible to perform failure determination according to application characteristics such as a periodic time of a task that is periodically executed by explicitly controlling the timeout time from the application.

The processing request in which the message synchronization type 304 is “synchronous” is, for example, an area that is repeatedly executed in a task (“periodic task”) that repeatedly executes the same program at a constant time interval under a known time constraint. For example, a request for receiving a message from the outside that arrives at an unknown timing, that is, a request for receiving data from the external communication interface 122. For two consecutive messages whose message synchronization type 304 is "synchronous", the time from the earlier arrival time to the execution location of the task 133 that issues a processing request corresponding to them, the time from the earlier arrival time When the constraint (hereinafter referred to as “time constraint”) is clear, the time constraint is added to the request parameter of any processing request.

Timeout processing in step 605 using this time constraint is as follows. The message processing unit 142 of the leader node changes the time constraint included in the request parameter or the request parameter of the processing request currently being processed from the reception time of the processing request whose previous message synchronization type 304 is “synchronous”. A timeout is considered when only the included time constraints (which depends on the implementation) are passed. For other message synchronization types 304 or follower nodes, for example, from the reception time of the processing request whose previous message synchronization type 304 is “synchronous”, the time constraint included in the request parameter, or A time obtained by apportioning the time constraint at an appropriate ratio can be used as the timeout time.

As will be described later, when waiting for reception when the message type is “asynchronous”, a failure determination may be performed using a timeout time determined by another method, or a timeout may not be used. . In the latter case, when a majority of the nodes do not respond, the leader node waits indefinitely for an approval / denial message from the follower node. However, if a majority of the nodes are not responding, the FT system 1 is used. This is a state in which the operation cannot be continued and should be stopped, and this is not greatly impaired in terms of availability.

Subsequently, processing in the message processing unit 142 of the follower node will be described. FIG. 10 is a diagram illustrating an example of an operation flow in the message processing unit 142 of the follower node in the FT system 1 according to the first embodiment. Specifically, the message processing unit 142 of the follower node includes the task communication unit 140. 7 is a flowchart illustrating an example of processing for receiving a processing request from the server and performing corresponding processing and returning the result to the task communication unit 140.

In this case, the message processing unit 142 of the follower node first waits for a processing request from the task communication unit 140 or a message from the inter-node communication unit 141 in the follower node (700). Here, when the message processing unit 142 of the follower node receives a processing request from the task communication unit 140 (701: Yes), the notification message list 151 is confirmed and a notification corresponding to the processing request received from the task communication unit 140 is received. It is confirmed whether the message 201 has arrived from the reader (702).

When the corresponding notification message 201 has arrived from the leader node (702: Yes), the message processing unit 142 of the follower node compares the processing request referred to by the notification message 201 with the processing request received from the task communication unit 140. (703). The comparison processing here is the result of processing in accordance with two processing requests, such as the determination of whether the processing request obtained from the task communication unit 140 and the processing request included in the notification message 201 match each other's identification information or accompanying data. As long as they are sufficient to determine that they are the same.

If it is determined in step 703 that the processing request referred to by the notification message 201 is the same as the processing request received from the task communication unit 140 (703: YES), the message processing unit 142 of the follower node The approval message 202 is transmitted to the node through the inter-node communication unit 141 (704).

Further, the message processing unit 142 of the follower node executes a corresponding predetermined process if necessary based on the corresponding notification message 201 (705), generates a processing result (706), and outputs it as a task communication unit. It transmits to 140 (707).

On the other hand, if it is determined in step 703 that the processing request referred to by the notification message 201 is not the same as the processing request received from the task communication unit 140 (703: No), the message processing unit 142 of the follower node Then, the denial message 203 is transmitted to the leader node through the inter-node communication unit 141 (708). At this time, since the behavior of the leader node can be regarded as abnormal from the standpoint of the follower, the message processing unit 142 of the follower node makes a proposal such as re-election of the leader node to each of the other computer nodes. Processing is performed (711).

When the processing request from the leader node has not arrived when the processing request from the task communication unit 140 is received in step 701 described above (702: No), the message processing unit 142 of the follower node receives the request from the leader node. (709). When the processing request from the leader node is received (710: No), the message processing unit 142 of the follower node is the same as when the processing request from the leader node already exists in the notification message list 151 (702: Yes). Proceed to processing.

On the other hand, as in the case of the leader node, in the follower node, the notification message 201 from the leader node may be significantly delayed or cannot be received due to a failure of the leader node or the inter-node communication interface 123 or the like. Therefore, the message processing unit 142 of the follower node receives the corresponding notification message 201 from the leader node even when a certain time (hereinafter referred to as a timeout time) has elapsed from the reception time of the processing request while waiting for step 709. If not, it is considered that a failure has occurred in the leader node or the inter-node communication interface 123 (710: Yes). Then, in the case where it is determined in step 703 that the processing request referred to by the notification message 201 and the processing request received from the task communication unit 140 are not the same, at the time of failure such as a leader node re-election proposal Processing is performed (711).

In step 701, when the message processing unit 142 of the follower node receives the notification message 201 from the inter-node communication unit 141 (701: No), the processing request corresponding to the notification message 201 is sent from the task communication unit 140. Since it has not been received yet, the information of the corresponding notification message 201 is stored in the notification message list 151 for later processing (712). Thereafter, the message processing unit 142 of the follower node returns to the process of waiting for reception of the processing request or notification message 201 (700).

By adopting the above first embodiment, the operation of the leader node and the follower node in the majority redundant configuration is kept the same, and the timeout determination at the leader node (reception waiting time of the approval / denial message from the follower node) Is reduced to only those related to synchronization messages, it is possible to allow a temporary processing and transmission delay of a small number of follower nodes.

Further, by determining the message synchronization type using the message type table 152, for example, the task communication unit 140 provides an API for issuing a synchronization request for performing synchronization, whereby the timing at which the application 132 is synchronized is determined. Control of timeout determination that matches the characteristics of the application 132 can be realized.

In the above description, the case where there is a single task 133 has been described. However, the processing of the task communication unit 140 and the message processing unit 142 is performed for one task 133 by performing parallel processing such as threading for each task or by adding multiplexing processing waiting for input / output. It is necessary to prevent other tasks 133 from being delayed by processing the request.

In addition, the reception of the processing request from the task communication unit 140 and the reception of the notification message 201 from the inter-node communication unit 141 are described as a single flow, but by using a plurality of threads or the like for efficiency, May be executed in parallel.
--- Second Embodiment ---
It can be assumed that the message processing unit 142 of the computer node performs processing using the request list. This request list is a table for selecting, when the message processing unit 142 receives the notification message 201 from another computer node, whether to return the approval message 202 or the denial message 203 in response thereto. Therefore, the request list has sufficient data for determining whether the message to be returned is an approval / denial message, for example, each value of a processing request or request-accompanying data.

Such a form will be described as a second embodiment. The FT system 1 in the second embodiment has the same configuration as that in FIG. The behavior of the follower node in the second embodiment follows the flow shown in FIG. Since other operations and configurations are the same as those of the first embodiment, description of overlapping portions is omitted.

FIG. 11 and FIG. 12 are diagrams showing an operation flow example 1 and an example 2 in the message processing unit of the follower node of the fault tolerant system in the second embodiment. The major difference from the first embodiment is that the processing for the processing request obtained by the task communication unit 140 proceeds without waiting for the notification message 201 from the leader node to arrive.

The message processing unit 142 of the follower node in the second embodiment waits for a processing request from the task communication unit 140 or a notification message from the inter-node communication unit 141 (800). For example, when a processing request is received from the task communication unit 140 (801: Yes), the message processing unit 142 of the follower node checks the notification message list 151 and determines whether the corresponding notification message 201 has arrived from the leader node. Confirm (802).

As a result of step 802, when it is determined that the notification message 201 corresponding to the processing request obtained from the task communication unit 140 has already arrived (802: Yes), the message processing unit 142 of the follower node is the first embodiment. The same processing is executed (803 to 810). Since the corresponding process is the same as that of the first embodiment, the description thereof is omitted.

On the other hand, when it is determined in step 802 that the notification message 201 corresponding to the processing request obtained from the task communication unit 140 has not arrived (802: No), the message processing unit 142 of the follower node responds to the processing request. Corresponding processing is performed (806). However, in the case of processing that affects the outside, such as transmission from the external communication interface 122, only the expected result is calculated without actually performing the processing. Further, the message processing unit 142 of the follower node adds request-accompanying data including information related to the processing request and the result of the processing to the above-described request list (807). The request list in which the records are generated in this way is a collection of records including data in the same format as the processing request 400 and the accompanying data 406 of the notification message 201 shown in FIG.

The message processing unit 142 of the follower node generates a processing result from the result of step 806 described above (809), and transmits the processing result to the task communication unit 140 (810). Thereafter, the message processing unit 142 of the follower node returns to waiting for transmission from the task communication unit 140 and the inter-node communication unit 141 (800).

On the other hand, when the notification message 201 is received from the inter-node communication unit 141 in Step 800 (801: No), the message processing unit 142 of the follower node confirms the request list and corresponds to the processing request reference included in the notification message 201. It is confirmed whether there is a processing request to be performed (811). As a result of this confirmation processing, when it is determined that a processing request corresponding to the notification message 201 received from the inter-node communication unit 141 exists in the request list (811: Yes), the message processing unit 142 of the follower node performs task communication. The notification message 201 obtained from the unit 140 is compared with the corresponding processing request included in the request list (812). This comparison process is similar to that described above with respect to step 703 of FIG.

As a result of such comparison processing, when the processing request referred to by the notification message 201 and the processing request received from the task communication unit 140 are considered to be the same (812: Yes), the message processing unit 142 of the follower node An approval message 202 is transmitted to the leader node (813). On the other hand, when the processing request referred to by the notification message 201 and the processing request received from the task communication unit 140 are not the same (812: NO), the message processing unit 142 of the follower node displays the denial message 203. The data is transmitted to the leader node (814), and a failure process related to the leader node is performed (815). If there is no corresponding processing request in the request list in step 811 described above, the message processing unit 142 of the follower node adds the information of the notification message 201 to the notification message list 151 (816).

With the above processing, the follower node can proceed with each process related to a task without waiting for a notification message from the leader node, and therefore the processing time delay can be reduced. However, since the follower node in this case proceeds with the process without using the result at the leader node, there is a high probability that a difference in processing contents will occur. Therefore, the follower node can also partially apply the present embodiment in which a part of the processing waits for a notification message from the leader node and the other processing proceeds without waiting. The message synchronization type can also be used for the determination in the follower node when performing such partial application. For example, if the message synchronization type is “synchronous” or “semi-synchronous”, it waits for a notification message from the leader node, and if the message synchronization type described in the next embodiment is “asynchronous”, notification from the leader node It is an algorithm that does not wait for a message.
--- Third embodiment ---
In the first and second embodiments described above, an example in which the process is selectively performed when the message synchronization type is “synchronous” or “semi-synchronous” has been described. Here, in addition to the same machine type, an example in which the message synchronization type is “asynchronous” is assumed. FIG. 13 is a diagram showing an example of an operation flow in the message processing unit 142 of the leader node of the FT system 1 in the third embodiment, and is a flowchart showing only a difference from the flow of FIG. Therefore, the description of the same configuration and processing as in the first embodiment is omitted.

Here, the message processing unit 142 of the leader node waits for transmission of a processing request from the task communication unit 140 or reception of the notification message 201 from the inter-node communication unit 141 (1000). If a processing request is received from the task communication unit 140 during standby (1001: Yes), the message processing unit 142 of the leader node determines the message synchronization type after executing steps 601 to 603 in the flow of FIG. (1002). The processing when the message synchronization type is “synchronous” and “semi-synchronous” is the same as the case shown in the first embodiment.

On the other hand, when the message synchronization type is “asynchronous” (1002: asynchronous), the message processing unit 142 of the leader node proceeds to step 611 in the flow of FIG. 8 without waiting for an approval / denial message from the follower node. . When a sufficient number of approval / denial messages have already been received from the follower node, the message processing unit 142 of the leader node at this point determines the majority process (step 610 in FIG. 8) as in the case of “semi-synchronization”. ) May be performed. On the other hand, when majority processing is not performed, it is necessary to perform majority processing after returning a result to the task communication unit 140. In this case, when the notification message 201 is received from the inter-node communication unit 141 (1001: No) ) To run.

When the message processing unit 142 of the leader node receives the notification message 201 from the inter-node communication unit 141 (1001: No), if it is the approval message 202 (1003: Ack), the message processing unit 142 of the approval node in the corresponding approval message list 150 Information of the follower node of the transmission source is added to the approval node set 301 of the row (1004). On the other hand, if the message is the denial message 203 (1003: Nack), the message processing unit 142 of the leader node adds the information of the source follower node to the denial node set 302 (1005). When there is a follower node not yet included in the approval node set 301 or the denial node set 302 regarding this row (1006: No), the message processing unit 142 of the leader node returns to reception waiting (1000). When the approval / denial message is received from all the follower nodes (1006: Yes), the message processing unit 142 of the leader node goes to the majority process. At this time, when there are a large number of rejection messages 203 (1007: No), the message processing unit 142 of the leader node performs a failure process related to the leader node.

On the other hand, when there are many approval messages 202 (1007: Yes), the message processing unit 142 of the leader node deletes the corresponding line in the approval message list 150 (1009), and returns to waiting for reception (1000). At this time, the message processing unit 142 of the leader node may perform processing for stopping the follower node that has sent the denial message 203 as necessary.

If a sufficient number of approval / denial messages have already been received from the follower node, the message processing unit 142 of the leader node may move the processing from step 1006 to majority processing (1007).

According to the above embodiment, the number of times that the leader node waits for the approval / denial message from the follower node can be reduced as compared with the case of the first embodiment. It becomes possible to tolerate appropriately.
--- Fourth embodiment ---
Here, an example is assumed in which there are two types of message synchronization types, “synchronous” and “asynchronous”. FIG. 14 is a diagram showing an example of an operation flow in the message processing unit 142 of the leader node of the FT system 1 in the fourth embodiment, and is a flowchart showing only a difference from the flow of FIG. Therefore, the description of the same configuration and processing as in the first embodiment is omitted. The description of the same configuration and processing as those in the third embodiment is also omitted.

Here, the message processing unit 142 of the leader node waits for transmission of a processing request from the task communication unit 140 or reception of the notification message 201 from the inter-node communication unit 141 (1100). When a processing request is received from the task communication unit 140 during standby (1101: Yes), the message processing unit 142 of the leader node determines the message synchronization type after executing steps 601 to 603 in the flow of FIG. (1102). The processing when the message synchronization type is “synchronization” is the same as that shown in the first embodiment (to step 605 in the flow of FIG. 8).

On the other hand, when the message synchronization type is “asynchronous” (1102: asynchronous), the message processing unit 142 of the leader node proceeds to step 611 in the flow of FIG. 8 without waiting for an approval / denial message from the follower node. . When a sufficient number of approval / denial messages have already been received from the follower node, the message processing unit 142 of the leader node at this point determines the majority process (step 610 in FIG. 8) as in the case of “semi-synchronization”. ) May be performed. On the other hand, when majority processing is not performed, it is necessary to perform majority processing after returning the result to the task communication unit 140. In this case, when the notification message 201 is received from the inter-node communication unit 141 (1101: No) ) To run. In step 1101, when the notification message 201 is received from the inter-node communication unit 141 (1001: No), the subsequent processing (1103 to 1109) is the same as in the third embodiment.

According to the above embodiment, the number of times that the leader node waits for the approval / denial message from the follower node can be reduced as compared with the case of the first embodiment. It becomes possible to tolerate appropriately.

The best mode for carrying out the present invention has been specifically described above. However, the present invention is not limited to this, and various modifications can be made without departing from the scope of the present invention.

According to the present embodiment, in the fault-tolerant system, messages for synchronization between computer nodes are classified, and depending on the type, only a part of the messages waits for the result of majority voting. The behavior change of majority voting such as executing asynchronously is executed to reduce the number of processing waits between computer nodes, thereby relatively suppressing the influence of performance variation between computer nodes. Such an effect also leads to reduction of processing delay and efficiency reduction in the entire system. Also, when performing failure determination, use the time that is clear from the application constraints as the failure determination criterion, and match the reference point and message type to allow for performance differences while allowing excessive failure determination. It is also possible to prevent a reduction in availability.

Therefore, in a fault-tolerant system composed of multiple computer nodes with different architectures and basic software, etc., the influence of overall performance due to performance differences and variations between computer nodes is reduced, and excessive fault determination due to such influences is reduced. By preventing this, availability in the system can be improved.

記載 At least the following will be made clear by the description in this specification. That is, in the fault tolerant system, the message processing unit of the computer that has become the other computer collates the processing content obtained by the task communication unit of the other computer with a predetermined standard, and the processing content is either synchronous or semi-synchronous. If the processing content corresponds to synchronization, for all the computers other than the other computer, wait for reception of the followability message, When the processing content corresponds to quasi-synchronization, it is possible to wait for reception of the follow-up propriety message for at least half of the computers other than the other computers.

In this way, depending on the type of the same task that each computer should execute in parallel, that is, whether synchronization or semi-synchronization is necessary, the computer that is the leader in the fault-tolerant system should wait for receipt of a message indicating whether the task can be tracked Control the number of follower calculators. As a result, the time to wait for the above message is selectively increased / decreased according to necessity, and the above arrival time difference of the above message due to the performance difference or variation between computers is appropriately absorbed, thereby preventing excessive failure determination or The availability in the system can be improved.

In the fault tolerant system, the message processing unit of the computer that has become the other computer collates the processing content obtained by the task communication unit of the other computer with a predetermined standard, and the corresponding processing content is synchronous, semi-synchronous, and Judgment is made whether it is asynchronous or not, and if the processing contents correspond to synchronization, all the computers other than the other computer are subjected to waiting for reception of the followability message. If the processing content corresponds to quasi-synchronization, wait for reception of the followability message for at least half of the computers other than the other computer, and the processing content is If it is asynchronous, it does not wait to receive the followability message from a computer other than the other computer. It may be.

In this way, depending on the type of the same task that each computer should execute in parallel, that is, whether synchronization, quasi-synchronization, or asynchronous is necessary, the computer that is the leader in the fault-tolerant system receives a message indicating whether the task can be followed or not. Control the number of follower computers to wait for. As a result, the time to wait for the above message is selectively increased / decreased according to necessity, and the above arrival time difference of the above message due to the performance difference or variation between computers is appropriately absorbed, thereby preventing excessive failure determination or The availability in the system can be improved. In particular, when the corresponding task is processed asynchronously between the computers, the computer that is the leader does not need to wait to receive the above-mentioned message from the follower computer, and the use resources of each computer required for fault determination are made efficient. As a result, the availability of the system can be further improved.

In the fault tolerant system, the message processing unit of the computer that has become the other computer collates the processing content obtained by the task communication unit of the other computer with a predetermined standard, and the corresponding processing content is either synchronous or asynchronous. If the processing content corresponds to synchronization, the processing waits for reception of the follow-up propriety message for all computers other than the other computer, and the processing When the content corresponds asynchronously, it may be configured not to wait for reception of the follow-up enable / disable message from a computer other than the other computer.

In this way, the followers to wait for the follow-up message about the corresponding task in the fault-tolerant system, depending on the type of the same task that each computer should execute in parallel, that is, whether synchronization or asynchronous is necessary Control the number of calculators. As a result, the time to wait for the above message is selectively increased / decreased according to necessity, and the above arrival time difference of the above message due to the performance difference or variation between computers is appropriately absorbed, thereby preventing excessive failure determination or The availability in the system can be improved. In particular, when the corresponding task is processed asynchronously between the computers, the computer that is the leader does not need to wait to receive the above-mentioned message from the follower computer, and the use resources of each computer required for fault determination are made efficient. As a result, the availability of the system can be further improved.

In the fault tolerant control method, the message processing unit of the computer that has become the other computer collates the processing content obtained by the task communication unit of the other computer with a predetermined standard, and the processing content is synchronous or semi-synchronous. It is determined whether it corresponds to any one, and when the processing content corresponds to synchronization, for all the computers other than the other computer, wait for reception of the followability message, When the processing content corresponds to quasi-synchronization, waiting for reception of the followability message may be executed for at least half of the computers other than the other computers.

In this way, depending on the type of the same task that each computer should execute in parallel, that is, whether synchronization or semi-synchronization is necessary, the computer that is the leader in the fault-tolerant system should wait for receipt of a message indicating whether the task can be tracked Control the number of follower calculators. As a result, the time to wait for the above message is selectively increased / decreased according to necessity, and the above arrival time difference of the above message due to the performance difference or variation between computers is appropriately absorbed, thereby preventing excessive failure determination or The availability in the system can be improved.

In the fault tolerant control method, the message processing unit of the computer that has become the other computer collates the processing content obtained by the task communication unit of the other computer with a predetermined standard, and the corresponding processing content is synchronous, semi-synchronous, And if the processing content corresponds to synchronization, and if all of the computers other than the other computer are the target, wait for reception of the follow-up enable / disable message. If the processing content corresponds to quasi-synchronization, wait for reception of the followability message for at least half of the computers other than the other computer, and execute the processing content Is not asynchronous, it does not wait to receive the follow-up message from other computers than the other computer. Good.

In this way, depending on the type of the same task that each computer should execute in parallel, that is, whether synchronization, quasi-synchronization, or asynchronous is necessary, the computer that is the leader in the fault-tolerant system receives a message indicating whether the task can be followed or not. Control the number of follower computers to wait for. As a result, the time to wait for the above message is selectively increased / decreased according to necessity, and the above arrival time difference of the above message due to the performance difference or variation between computers is appropriately absorbed, thereby preventing excessive failure determination or The availability in the system can be improved. In particular, when the corresponding task is processed asynchronously between the computers, the computer that is the leader does not need to wait to receive the above-mentioned message from the follower computer, and the use resources of each computer required for fault determination are made efficient. As a result, the availability of the system can be further improved.

In the fault tolerant control method, the message processing unit of the computer that has become the other computer collates the processing content obtained by the task communication unit of the other computer with a predetermined standard, and the processing content is either synchronous or asynchronous. If the processing content corresponds to synchronization, for all the computers other than the other computer, wait for reception of the followability message, When the processing contents are asynchronous, it may be determined not to wait for reception of the followability message from a computer other than the other computer.

In this way, the followers to wait for the follow-up message about the corresponding task in the fault-tolerant system, depending on the type of the same task that each computer should execute in parallel, that is, whether synchronization or asynchronous is necessary Control the number of calculators. As a result, the time to wait for the above message is selectively increased / decreased according to necessity, and the above arrival time difference of the above message due to the performance difference or variation between computers is appropriately absorbed, thereby preventing excessive failure determination or The availability in the system can be improved. In particular, when the corresponding task is processed asynchronously between the computers, the computer that is the leader does not need to wait to receive the above-mentioned message from the follower computer, and the use resources of each computer required for fault determination are made efficient. As a result, the availability of the system can be further improved.

1 fault tolerant system 102 terminal 110 computer node (computer)
110A leader node 110B, 110C follower node 111 external network 112 inter-node network 120 central processing unit 121 main storage unit 122 external communication interface 123 inter-node communication interface 124 auxiliary storage unit 130 basic software 131 FT middleware 132 application 133 task 140 task communication unit 141 Inter-node communication unit 142 Message processing unit 150 Approval message list 151 Notification message list 152 Message type table

Claims (8)

In each of a plurality of computers that form a majority redundant configuration,
A task communication unit for acquiring the processing contents from a program operating inside the computer;
The predetermined message indicating the processing content of the program running on the other computer is received from any other computer forming the majority voting redundant configuration, and the processing content indicated by the predetermined message and the processing obtained by the task communication unit In accordance with the result of determination of coincidence with the contents, a message processing unit that returns a followability message to the other computer,
The message processing unit of the computer that has become the other computer has a number of computers other than the other computer that are subject to reception of the follow-up / possibility message according to the processing content obtained by the task communication unit of the other computer. To further execute the process of changing
A fault-tolerant system. The message processing unit of the computer that has become the other computer is:
The processing content obtained by the task communication unit of the other computer is checked against a predetermined standard to determine whether the corresponding processing content corresponds to either synchronization or quasi-synchronization, and the processing content corresponds to synchronization If it is, for all computers other than the other computer, wait for reception of the followability message, and if the processing content corresponds to quasi-synchronization, the computer other than the other computer For at least half of the computers, waiting for receiving the follow-up message is executed.
The fault tolerant system according to claim 1. The message processing unit of the computer that has become the other computer is:
The processing content obtained by the task communication unit of the other computer is checked against a predetermined standard to determine whether the corresponding processing content corresponds to one of synchronous, semi-synchronous, and asynchronous, and the processing content is synchronized. If it is compatible, execute waiting for reception of the follow-up enable / disable message for all computers other than the other computer, and if the processing content corresponds to semi-synchronization, If at least half of the other computers are targeted for receiving the follow-up success / failure message, and the processing content is asynchronous, the follow-up from a computer other than the other computer It does not execute the waiting for receiving the message of availability.
The fault tolerant system according to claim 1. The message processing unit of the computer that has become the other computer is:
The processing content obtained by the task communication unit of the other computer is checked against a predetermined standard, it is determined whether the corresponding processing content corresponds to either synchronous or asynchronous, and the processing content corresponds to synchronization. If there is, execute the waiting for reception of the followability message for all the computers other than the other computer, and if the processing content is asynchronous, if from the computer other than the other computer It does not execute reception waiting for the follow-up message.
The fault tolerant system according to claim 1. In each of a plurality of computers that form a majority redundant configuration,
Processing to acquire the processing contents from a program operating inside the computer;
A predetermined message indicating the processing content of a program operating on the other computer is received from any other computer forming the majority voting redundant configuration, and the processing content indicated by the predetermined message and the processing content obtained by the processing In accordance with the result of the match determination, a process of returning a followability message to the other computer is executed.
Furthermore, in the other computer, a process of changing the number of computers other than the other computer, which is subject to reception of the followability message, according to the processing content obtained from the program operating on the other computer. Perform further,
A fault tolerant system control method characterized by the above. The message processing unit of the computer that has become the other computer,
The processing content obtained by the task communication unit of the other computer is checked against a predetermined standard to determine whether the corresponding processing content corresponds to either synchronization or quasi-synchronization, and the processing content corresponds to synchronization If it is, for all computers other than the other computer, wait for reception of the followability message, and if the processing content corresponds to quasi-synchronization, the computer other than the other computer Waiting for reception of the followability message for at least half of the computers
The fault tolerant control method according to claim 5. The message processing unit of the computer that has become the other computer,
The processing content obtained by the task communication unit of the other computer is checked against a predetermined standard to determine whether the corresponding processing content corresponds to one of synchronous, semi-synchronous, and asynchronous, and the processing content is synchronized. If it is compatible, execute waiting for reception of the follow-up enable / disable message for all computers other than the other computer, and if the processing content corresponds to semi-synchronization, If at least half of the other computers are targeted for receiving the follow-up success / failure message, and the processing content is asynchronous, the follow-up from a computer other than the other computer Do not wait for receipt of acceptable messages,
The fault tolerant control method according to claim 5. The message processing unit of the computer that has become the other computer,
The processing content obtained by the task communication unit of the other computer is checked against a predetermined standard, it is determined whether the corresponding processing content corresponds to either synchronous or asynchronous, and the processing content corresponds to synchronization. If there is, execute the waiting for reception of the followability message for all the computers other than the other computer, and if the processing content is asynchronous, if from the computer other than the other computer Do not wait to receive the follow-up message.
The fault tolerant control method according to claim 5.

Images