CN106446300A - Transaction processing method and system based on shared storage pool - Google Patents

Abstract

The invention discloses a transaction processing method and system based on a shared storage pool, which is used in a distributed file system, including that each participant feeds back its own feedback message to the coordinator, and records a local log, and the local log is updated through the ID of the transaction Mark; the coordinator judges the status of the transaction according to whether the feedback message sent by all participants is received within the first preset time and the feedback message of all participants in the case of yes, and records a corresponding ID and status of the transaction the global log to the shared storage pool; when the participant does not receive the operation command sent by the coordinator within the second preset time, the participant searches the global log in the shared storage pool according to the ID of the transaction and according to the found global log Determine the status of the transaction, and take appropriate actions based on the status of the transaction. The invention enables the participants to determine the status of the transaction when they do not receive the operation command sent by the coordinator in time, thereby improving the fault tolerance.

Description

A transaction processing method and system based on a shared storage pool

technical field

The invention relates to the technical field of distributed file systems, in particular to a transaction processing method and system based on a shared storage pool.

Background technique

With more and more scenarios such as high concurrent access and massive data processing of large websites, how to achieve the goals of website scalability becomes more and more important. In order to solve this series of problems, distributed file system came into being.

In a distributed file system, a two-phase commit protocol is usually used for transaction processing. The two-phase commit is in the field of computer networks and databases. In order to enable all nodes (metadata servers) based on the distributed file system architecture to submit transactions An algorithm designed to maintain consistency. In a distributed file system, although each node can know whether its own operation succeeds or fails, it cannot know whether other nodes' operations succeed or fail. When a transaction spans multiple nodes, in order to maintain the ACID characteristics of the transaction, it is necessary to select a node from the multiple nodes corresponding to the transaction as the coordinator to uniformly control the operation results of other nodes (called participants) and finally instruct these Whether the node actually submits the operation results (such as writing updated data to disk, etc.). Therefore, the algorithm idea of the two-phase commit can be summarized as follows: participants feed back information about the success or failure of their own operations to the coordinator, and then the coordinator decides whether each participant wants to commit or abort the operation based on the feedback messages from all participants.

Specifically, after participants feed back their own operation success or failure messages to the coordinator so that the coordinator can determine the status of the transaction, usually the participants will receive the operation command sent by the coordinator within a certain period of time. The operation command here is Execute commands or abort commands, but there are some situations in practical applications, such as in the case of network failure or coordinator errors, the participants cannot receive the operation commands sent by the coordinator, so that the participants cannot determine the status of the transaction at this time, and then The corresponding operation cannot be performed.

Therefore, how to provide a solution to the above technical problems is a problem that those skilled in the art need to solve at present.

Contents of the invention

The purpose of the present invention is to provide a transaction processing method based on a shared storage pool, which can enable the participants to determine the state of the transaction when they do not receive the operation command sent by the coordinator on time, and improve the fault tolerance; another aspect of the present invention One object is to provide a transaction processing system based on a shared storage pool.

In order to solve the above technical problems, the present invention provides a transaction processing method based on a shared storage pool, which is used in a distributed file system, including:

Step S11: Each participant feeds back its own feedback message to the coordinator, and records a local log, which is marked by the ID of the transaction;

Step S12: The coordinator judges the state of the transaction according to whether the feedback message sent by all participants is received within the first preset time and if so, the feedback message of all participants, and correspondingly records a message containing the transaction The global log of ID and status is sent to the shared storage pool;

Step S13: When the participant does not receive the operation command sent by the coordinator within the second preset time, the participant searches the global log in the shared storage pool according to the ID of the transaction And determine the state of the transaction according to the found global log, and perform corresponding operations according to the state of the transaction.

Preferably, the process of step S12 is specifically:

Step S121: The coordinator judges whether the feedback message sent by all participants is received within the first preset time, if yes, proceed to step S122, otherwise, proceed to step S124;

Step S122: Determine whether the feedback messages sent by all participants are ready messages, if yes, go to step S123, otherwise, go to step S124;

Step S123: the coordinator determines that the transaction is executable, and records a global log including the ID and status of the transaction to the shared storage pool, and sets the status as successful;

Step S124: The coordinator determines that the transaction is not executable, and records a global log including the ID and status of the transaction to the shared storage pool, and sets the status as failed.

Preferably, the process of step S13 is specifically:

When the participant does not receive the operation command sent by the coordinator within the second preset time, the participant searches the storage pool for the global log according to the ID of the transaction;

When the global log is found and the status of the transaction recorded in the global log is successful, the participant performs the corresponding metadata operation; when the status of the transaction recorded in the global log is failed , the participant performs a rollback operation.

Preferably, the local log includes operation type, file path and metadata server identifier.

Preferably, step S13 also includes:

A failure type is determined, where the failure type includes a network failure or an error of the coordinator.

In order to solve the above technical problems, the present invention also provides a transaction processing system based on a shared storage pool, which is used in a distributed file system, including:

The feedback unit is used for each participant to feed back its own feedback message to the coordinator, and record a local log, and the local log is marked by the ID of the transaction;

The judging unit is used for the coordinator to judge the status of the transaction according to whether the feedback message sent by all participants is received within the first preset time and the feedback message of all participants in the case of yes, and correspondingly records a message including the The global log of transaction ID and status to the shared storage pool;

A search unit, configured to search the shared storage pool for the participant according to the ID of the transaction when the participant does not receive the operation command sent by the coordinator within a second preset time. The global log determines the state of the transaction according to the found global log, and performs corresponding operations according to the state of the transaction.

Preferably, the judging unit specifically includes:

The first judging unit is used for the coordinator to judge whether the feedback message sent by all participants is received within the first preset time, if yes, trigger the second judging unit, otherwise, trigger the second determination unit;

A second judging unit, configured to judge whether the feedback messages sent by all participants are ready messages, if so, trigger the first determining unit, otherwise, trigger the second determining unit;

The first determination unit is used for the coordinator to judge that the transaction is executable, and record a global log including the ID and status of the transaction to the shared storage pool, and set the status as successful;

The second determination unit is used for the coordinator to determine that the transaction is not executable, and record a global log including the ID and status of the transaction to the shared storage pool, and set the status as failed.

Preferably, the search unit specifically includes:

The search subunit is used to search the storage pool according to the ID of the transaction when the participant does not receive the operation command sent by the coordinator within the second preset time. Describe the global log;

an execution unit, configured to execute the corresponding metadata operation by the participant when the global log is found and the status of the transaction recorded in the global log is successful; when the transaction recorded in the global log When the status of the transaction is failure, the participant performs a rollback operation.

Preferably, the local log includes operation type, file path and metadata server identifier.

Preferably, the search unit further includes:

A fault determining unit, configured to determine a fault type, where the fault type includes a network fault or an error of the coordinator.

The present invention provides a transaction processing method and system based on a shared storage pool, which is used in a distributed file system, including that each participant feeds back its own feedback message to the coordinator, and records a local log, and the local log uses the ID of the transaction to Mark; the coordinator judges the status of the transaction according to whether the feedback message sent by all participants is received within the first preset time and the feedback message of all participants in the case of yes, and records a corresponding ID and status of the transaction the global log to the shared storage pool; when the participant does not receive the operation command sent by the coordinator within the second preset time, the participant searches the global log in the shared storage pool according to the ID of the transaction and according to the found global log Determine the status of the transaction, and take appropriate actions based on the status of the transaction.

It can be seen that in the present invention, after the coordinator determines the state of the transaction, it will record the ID and status global log including the transaction into the shared storage pool. When the participant does not receive the operation sent by the coordinator within the second preset time When ordering, the participant searches the global log in the shared storage pool according to the transaction ID and determines the status of the transaction according to the found global log, and performs corresponding operations according to the status of the transaction, so that the participant can not receive the coordinator on time The status of the transaction can still be determined when the operation command is sent, which improves the fault tolerance.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the prior art and the accompanying drawings that need to be used in the embodiments. Obviously, the accompanying drawings in the following description are only some of the present invention. Embodiments, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

Fig. 1 is a flow chart of the process of a transaction processing method based on a shared storage pool provided by the present invention;

FIG. 2 is a schematic structural diagram of a transaction processing system based on a shared storage pool provided by the present invention.

detailed description

The core of the present invention is to provide a transaction processing method based on a shared storage pool, which can enable participants to determine the state of the transaction when they do not receive the operation command sent by the coordinator on time, and improve fault tolerance; another aspect of the present invention A core is to provide a transaction processing system based on a shared storage pool.

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Please refer to FIG. 1. FIG. 1 is a flow chart of the process of a transaction processing method based on a shared storage pool provided by the present invention. The method is used in a distributed file system, including:

Step S11: Each participant feeds back its own feedback message to the coordinator, and records a local log, which is marked by the ID of the transaction;

Specifically, the client submits an operation request, and the distributed file system selects the corresponding metadata server and assigns different roles, that is, selects a coordinator, and the rest are transaction participants.

In the prepare phase, the coordinator sends a prepare command to each participant, informing it to check the operating conditions, and waits for the participant's response. After receiving the prepare command, each participant judges whether it meets the conditions for executing the transaction, that is, whether it can perform the commit operation, and if so, sends a ready feedback message to the coordinator, otherwise, sends an abort feedback message to the coordinator information. In addition, participants also record local log files and mark them with distributed transaction IDs.

The local log is used to record the metadata operation information related to the local node in the transaction.

Preferably, the local log includes operation type, file path and metadata server identifier.

Of course, the local log here may also include other metadata operation information, which is not specifically limited in the present invention and is determined according to actual conditions.

Step S12: The coordinator judges the status of the transaction according to whether the feedback message sent by all participants is received within the first preset time and if so, the feedback message of all participants, and records a corresponding transaction ID and A global log of state to the shared storage pool;

Specifically, the global log is used to record the ID of the transaction and the status of the transaction. Each metadata server has its own local log, but only one copy of the global log is maintained in the distributed file system.

As preferably, the process of step S12 is specifically:

Step S121: The coordinator judges whether the feedback messages sent by all participants have been received within the first preset time, if yes, proceed to step S122, otherwise, proceed to step S124;

Step S122: Determine whether the feedback messages sent by all participants are ready messages, if yes, go to step S123, otherwise, go to step S124;

Step S123: the coordinator determines that the transaction is executable, and records a global log including the transaction ID and status to the shared storage pool, and sets the status as successful;

Step S124: The coordinator determines that the transaction is not executable, and records a global log including the transaction ID and status to the shared storage pool, and sets the status as failed.

In the judgment stage, the coordinator judges whether the feedback messages sent by all participants are received within the first preset time, if yes, the status of the transaction is determined at this time, and then judges that the feedback messages sent by all participants are ready The message is still an abort message. If the feedback messages sent by all participants are ready messages, it is judged that the transaction is executable, and a global log including the transaction ID and status is recorded to the shared storage pool, and the status is set to success. If If the feedback message from some or all of the participants is an abort message, the transaction is judged to be unexecutable, and a global log including the transaction ID and status is recorded to the shared storage pool, and the status is set as failure. Because the coordinator is prone to malfunction while waiting for the feedback message from the participant, it is easy to cause the coordinator to fail to receive the participant's message in time. If the coordinator does not receive feedback from some or all of the participants within the first preset time, all failures will be attributed to timeout errors, and the transaction will be judged as unexecutable, and a record including the ID and status of the transaction will be recorded global log to the shared storage pool and set the status to failed.

Step S13: When the participant does not receive the operation command sent by the coordinator within the second preset time, the participant searches the global log in the shared storage pool according to the transaction ID and determines the status of the transaction according to the found global log, And perform corresponding operations according to the status of the transaction.

After the preparation phase and the judgment phase are completed, the transaction enters the execution phase, and the coordinator broadcasts operation commands to each participant, where the operation commands include execution commands and cancellation commands. In the case of no failure, the participants start their own metadata operations after receiving the execution command. If they receive the cancel command, they will not perform the operation of the distributed transaction and perform the rollback operation.

When the participant does not receive the operation command sent by the coordinator within the second preset time, the catering combination can search in the shared storage pool according to the ID of the transaction. When the global log is found, according to the record information in the global log Determine the status of the transaction, and then perform corresponding operations according to the status of the transaction.

Preferably, the process of step S13 is specifically:

When the participant does not receive the operation command sent by the coordinator within the second preset time, the participant searches the global log in the storage pool according to the ID of the transaction;

When the global log is found and the status of the transaction recorded in the global log is successful, the participant performs the corresponding metadata operation. When the status of the transaction recorded in the global log is failed, the participant performs a rollback operation.

As preferably, step S13 also includes:

Determine the failure type, which includes network failure or coordinator error.

Specifically, at this time, the fault type can be narrowed down to a network fault or a coordinator error.

In addition, if the participant does not find the global log in the shared storage pool, the fault needs to be determined by the time when the coordinator writes the global log and the second preset time. If the time when the coordinator writes the global log is greater than the second preset time , then the participant did not find the global log at this time, it may be because the coordinator has not written it yet, then the participant can wait a little longer, if it still cannot be found, it may be that the coordinator cannot write or the network failure. If the time for the coordinator to write the global log is less than the second preset time, if the participant does not find the global log at this time, it means that the coordinator cannot write or the network is faulty.

Because the shared storage pool is very reliable, it can guarantee participants' correct access to the global log and avoid uncertainty caused by errors.

The invention provides a transaction processing method based on a shared storage pool, which is used in a distributed file system, including that each participant feeds back its own feedback message to the coordinator, and records a local log, and the local log is marked by the ID of the transaction; The coordinator judges the status of the transaction according to whether the feedback message sent by all participants is received within the first preset time, and if so, the feedback message of all participants, and records a global transaction ID and status correspondingly. log to the shared storage pool; when the participant does not receive the operation command sent by the coordinator within the second preset time, the participant searches the global log in the shared storage pool according to the transaction ID and determines the transaction according to the found global log state, and perform corresponding operations according to the state of the transaction.

It can be seen that in the present invention, after the coordinator determines the state of the transaction, it will record the ID and status global log including the transaction into the shared storage pool. When the participant does not receive the operation sent by the coordinator within the second preset time When ordering, the participant searches the global log in the shared storage pool according to the transaction ID and determines the status of the transaction according to the found global log, and performs corresponding operations according to the status of the transaction, so that the participant can not receive the coordinator on time The status of the transaction can still be determined when the operation command is sent, which improves the fault tolerance.

Compatible with the above method embodiment, please refer to FIG. 2, which is a schematic structural diagram of a transaction processing system based on a shared storage pool provided by the present invention. The system is used in a distributed file system, including:

The feedback unit 1 is used for each participant to feed back its own feedback message to the coordinator, and record the local log, and the local log is marked by the ID of the transaction;

The judging unit 2 is used for the coordinator to judge the state of the transaction according to whether the feedback message sent by all participants is received within the first preset time and the feedback message of all participants in the case of yes, and correspondingly records a message including transaction The ID and status of the global log to the shared storage pool;

The search unit 3 is used for when the participant does not receive the operation command sent by the coordinator within the second preset time, the participant searches the global log in the shared storage pool according to the ID of the transaction and determines the transaction according to the found global log state, and perform corresponding operations according to the state of the transaction.

As preferably, the judging unit 2 specifically includes:

The first judging unit is used for the coordinator to judge whether the feedback message sent by all participants is received within the first preset time, if so, trigger the second judging unit, otherwise, trigger the second determining unit;

The second judging unit is used to judge whether the feedback messages sent by all participants are ready messages, if yes, trigger the first determining unit, otherwise, trigger the second determining unit;

The first determining unit is used for the coordinator to judge that the transaction is executable, and records a global log including the ID and status of the transaction to the shared storage pool, and sets the status as success;

The second determination unit is used for the coordinator to determine that the transaction is not executable, and record a global log including the ID and status of the transaction to the shared storage pool, and set the status as failure.

As preferably, the search unit 3 specifically includes:

The search subunit is used to search the global log in the storage pool according to the ID of the transaction when the participant does not receive the operation command sent by the coordinator within the second preset time;

The execution unit is used to execute the corresponding metadata operation when the global log is found and the status of the transaction recorded in the global log is successful, and the participant rolls back when the status of the transaction recorded in the global log is failed operate.

Preferably, the local log includes operation type, file path and metadata server identifier.

As preferably, the search unit 3 also includes:

The fault determining unit is configured to determine a fault type, and the fault type includes a network fault or a coordinator error.

For the introduction of the transaction processing system based on the shared storage pool provided by the present invention, please refer to the above method embodiment, and the present invention will not repeat it here.

The present invention provides a transaction processing system based on a shared storage pool, which is used in a distributed file system, including that each participant feeds back its own feedback message to the coordinator, and records a local log, and the local log is marked by the ID of the transaction; The coordinator judges the status of the transaction according to whether the feedback message sent by all participants is received within the first preset time, and if so, the feedback message of all participants, and records a global transaction ID and status correspondingly. log to the shared storage pool; when the participant does not receive the operation command sent by the coordinator within the second preset time, the participant searches the global log in the shared storage pool according to the transaction ID and determines the transaction according to the found global log state, and perform corresponding operations according to the state of the transaction.

It can be seen that in the present invention, after the coordinator determines the state of the transaction, it will record the ID and status global log including the transaction into the shared storage pool. When the participant does not receive the operation sent by the coordinator within the second preset time When ordering, participants search the global log in the shared storage pool according to the ID of the transaction, determine the status of the transaction according to the found global log, and perform corresponding operations according to the status of the transaction, which improves fault tolerance.

It should be noted that in this specification, relative terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations Any such actual relationship or order exists between. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. a kind of transaction methods based on shared storage pool, for distributed file system it is characterised in that including：

Step S11：The feedback message of itself is fed back to coordinator by each participant, and records local daily record, described local day Will is by the ID of affairs come labelling；

Step S12：Coordinator is according to whether receive the feedback message of all participants' transmissions and be in the first Preset Time In the case of the feedback message of all participants judge the state of affairs, and the ID of correspondingly one described affairs of inclusion of record And the overall daily record of state is to shared storage pool；

Step S13：When described participant is not received by the operational order that described coordinator sends in the second Preset Time, Described participant goes to search the daily record of the described overall situation in described shared storage pool and according to the institute finding according to the ID of described affairs State the state that overall daily record determines described affairs, and operated accordingly according to the state of described affairs.

2. transaction methods as claimed in claim 1 are it is characterised in that the process of step S12 is specially：

Step S121：Described coordinator judges whether that the feedback receiving all participants' transmissions in the first Preset Time disappears Breath, if it is, entering step S122, otherwise, enters step S124；

Step S122：Judge whether that the feedback message that all participants send is ready message, if it is, entering step S123, otherwise, enters step S124；

Step S123：Described coordinator judges that described affairs are executable, and records ID and the state of the described affairs of inclusion Overall daily record to shared storage pool, and to arrange described state be successfully；

Step S124：Described coordinator judges described affairs for not can perform, and records ID and the shape of the described affairs of inclusion The overall daily record of state is to shared storage pool, and to arrange described state be unsuccessfully.

3. transaction methods as claimed in claim 2 are it is characterised in that the process of step S13 is specially：

When described participant is not received by the operational order that described coordinator sends in the second Preset Time, described participation Person goes to search according to the ID of described affairs and searches the daily record of the described overall situation in described storage pool；

Find described the overall situation daily record and described the overall situation daily record in record described affairs state be successfully when, described participation Person execute corresponding metadata operation, when described the overall situation daily record in record described affairs state be unsuccessfully when, described participation Person carries out rolling back action.

4. transaction methods as claimed in claim 1 are it is characterised in that described local daily record includes action type, file Path and meta data server mark.

5. the transaction methods as described in any one of claim 1-4 are it is characterised in that step S13 also includes：

Determine fault type, described fault type includes network failure or described coordinator error.

6. a kind of transacter based on shared storage pool, for distributed file system it is characterised in that including：

The feedback message of itself is fed back to coordinator for each participant by feedback unit, and records local daily record, described Ground daily record is by the ID of affairs come labelling；

Judging unit, for coordinator according to whether receiving the feedback message that all participants send in the first Preset Time And the feedback message of all participants in the case of being judges the state of affairs, and correspondingly one described affairs of inclusion of record ID and state overall daily record to shared storage pool；

Searching unit, for being not received by, as described participant, the operation life that described coordinator sends in the second Preset Time When making, described participant goes to search the daily record of the described overall situation in described shared storage pool and according to finding according to the ID of described affairs The daily record of the described overall situation determine the state of described affairs, and operated accordingly according to the state of described affairs.

7. transacter as claimed in claim 6 is it is characterised in that described judging unit specifically includes：

First judging unit, judges whether to receive what all participants sent in the first Preset Time for described coordinator Feedback message, if it is, triggering described second judging unit, otherwise, triggers described second determining unit；

Second judging unit, the feedback message for judging whether all participants' transmissions is ready message, if it is, touching Send out the first determining unit described, otherwise, trigger described second determining unit；

For described coordinator, first determining unit, judges that described affairs are executable, and record the described affairs of inclusion The overall daily record of ID and state is to shared storage pool, and to arrange described state be successfully；

Second determining unit, judges described affairs for not can perform for described coordinator, and records the described affairs of inclusion ID and state overall daily record to shared storage pool, and to arrange described state be unsuccessfully.

8. transacter as claimed in claim 7 is it is characterised in that described searching unit specifically includes：

Search subelement, for being not received by, as described participant, the operation that described coordinator sends in the second Preset Time During order, described participant goes to search according to the ID of described affairs and searches the daily record of the described overall situation in described storage pool；

Performance element, for being in the state finding the described affairs of record in the daily record of the described overall situation and the daily record of the described overall situation During work(, described participant executes corresponding metadata operation, when the state of the described affairs of record in the daily record of the described overall situation is to lose When losing, described participant carries out rolling back action.

9. transacter as claimed in claim 6 is it is characterised in that described local daily record includes action type, file Path and meta data server mark.

10. the transacter as described in any one of claim 6-9 is it is characterised in that described searching unit also includes：

Fault determining unit, for determining fault type, described fault type includes network failure or described coordinator error.