WO2024239992A1 - Data synchronization method for distributed cluster and related device therefor - Google Patents

Abstract

The present application relates to a data synchronization method for a distributed cluster and a related device therefor. The method comprises: on the basis of the master-slave state switching of a memory management unit group in a distributed cluster, determining a synchronization time slice; generating a data operation log of a master memory management unit in the memory management unit group within the synchronization time slice; synchronizing the data operation log to a slave memory management unit in the memory management unit group, so that the slave memory management unit performs data synchronization on the basis of the data operation log. The present invention solves problems of service unavailability and incremental data synchronization consistency due to downtime migration between different isolated sites, allows for zero-downtime data migration between the isolated sites, and implements incremental data synchronization between the isolated sites.

Description

Distributed cluster data synchronization method and related equipment

This application claims the priority of the Chinese patent application filed with the China Patent Office on May 19, 2023, with application number 202310576529.7 and application name “Data synchronization method and related equipment for distributed clusters”, the entire contents of which are incorporated by reference in this application.

Technical Field

The present application relates to the field of computer technology, and in particular to a data synchronization method for a distributed cluster and related equipment.

Background Art

This section is intended to provide a background or context to the embodiments of the invention recited in the claims. No admission is made that the description herein is prior art by its inclusion in this section.

The product warehouse needs to be redundantly backed up at different sites and synchronized between different sites, so that when regional problems occur, master-slave switching can be performed to achieve high availability. At the same time, network delays caused by regional factors can also be solved through multi-site deployment. In addition, there is also a need for product synchronization between different deployment environments. Unverified products need to be verified in the development environment first, and then synchronized to the stable production environment through the site synchronization function after passing the verification.

Different sites of the product warehouse are isolated from each other, so conventional online data synchronization methods cannot achieve data synchronization between different sites of the product warehouse. In the related art, data migration between isolated different sites usually adopts the method of downtime migration, which will cause the service to be temporarily unavailable, and it is impossible to achieve real-time synchronization for incremental data.

Summary of the invention

The distributed cluster data synchronization method and related devices provided by the embodiments of the present invention at least solve the problem of service unavailability caused by shutdown migration between isolated different sites.

A distributed cluster data synchronization method, comprising:

Determine the synchronization time slice based on the master-slave state switching of the storage management unit group in the distributed cluster;

Generating a data operation log of a primary storage management unit in the storage management unit group within a synchronization time slice;

The data operation log is synchronized to a slave storage management unit in the storage management unit group, so that the slave storage management unit performs data synchronization based on the data operation log.

In some embodiments, the method further comprises:

Generate metadata of the master storage management unit so that when synchronizing the data operation log to the slave storage management unit in the storage management unit group, the data operation log to be synchronized in the data operation log is determined based on the metadata, wherein the metadata includes a synchronization status of the data operation log;

The metadata is updated based on the synchronization result of the data operation log.

In some embodiments, generating a data operation log of a primary storage management unit in the storage management unit group within a synchronization time slice includes:

Persisting data operation information of the cluster in the distributed cluster according to the execution order of the data operation, wherein the data operation information includes the operation object, the operation content and the information of the storage management unit that performs the data operation;

According to the data operation information of the cluster, a data operation log of the primary storage management unit in the synchronization time slice is obtained, wherein the data operation log is sorted according to the execution order of the data operation.

In some of the embodiments, determining the synchronization time slice based on the master-slave state switching of the storage management unit group in the distributed cluster includes:

The master-slave state switching logic clock of the storage management unit group is obtained, and the synchronization time slice and the sequence number of the synchronization time slice are determined according to the master-slave state switching logic clock.

In some embodiments, the method further comprises:

When the master-slave state of the storage management unit of the cluster in the distributed cluster switches, the cluster reports the master-slave state switching logical clock to the storage system, and the storage system sequentially increases the sequence number of the synchronization time slice according to the master-slave state switching logical clock and sends it to the distributed cluster.

In some embodiments, the method further comprises:

The distributed cluster periodically obtains the sequence number of the synchronization time slice from the storage system.

In some embodiments, synchronizing the data operation log to a slave storage management unit in the storage management unit group includes:

According to the index number of the data operation log, the index number of the submitted data operation log recorded in the metadata and the sequence number of the synchronization time slice, the data operation log to be synchronized in the main storage management unit is determined, and the data operation log to be synchronized is synchronized to the slave storage management unit.

In some of the embodiments, when the sequence numbers of the synchronization time slices of the primary storage management unit and the secondary storage management unit are the same, the data operation logs to be synchronized are determined and synchronized.

In some of the embodiments, in an initial state, the storage management units in the master cluster in the distributed cluster are all master storage management units, the storage management units in the slave clusters in the distributed cluster are all slave storage management units, and the master-slave switching of the storage management unit group is managed by a preset controller.

In some of the embodiments, the preset controller sets the storage management unit with the identifier of the storage management unit as the main storage management unit by adding the identifier of the storage management unit to the gray list, so as to realize the master-slave state switching of the storage management units in the storage management unit group in the distributed cluster.

In some of the embodiments, when there is data to be written into the storage management unit group, the primary storage management unit in the storage management unit group is determined by querying the preset controller, and the data is written into the primary storage management unit.

In some embodiments, the method further comprises:

The data storage state of the storage management unit is used as a finite state machine, and a leader node and at least one follower node are respectively elected in the main storage management unit and the slave storage management unit. The data operation log performs copy synchronization of the finite state machine within the storage management unit group.

A distributed cluster system comprises a plurality of clusters and a storage system, wherein the plurality of clusters comprises a master cluster and at least one slave cluster, and the storage system is connected to the plurality of clusters respectively;

The storage system is used to store the serial number of the synchronization time slice; the multiple clusters all perform data synchronization through the above-mentioned data synchronization method.

In some of the embodiments, the distributed cluster system further includes a preset controller, which is used to control the master-slave state switching of the storage management unit and record the master-slave state of the storage management unit.

An electronic device comprises: a processor and a memory storing a program, wherein the program comprises instructions, and when the instructions are executed by the processor, the processor executes the above-mentioned data synchronization method.

A non-transitory machine-readable medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the above-mentioned data synchronization method.

The data synchronization method and related equipment of a distributed cluster provided by an embodiment of the present invention determine the synchronization time slice by switching the master-slave state of a storage management unit group in the distributed cluster; generate a data operation log of the master storage management unit in the storage management unit group within the synchronization time slice; synchronize the data operation log to the slave storage management unit in the storage management unit group, so that the slave storage management unit synchronizes data based on the data operation log, thereby solving the problems of service unavailability and incremental data synchronization consistency caused by downtime migration between different isolated sites, realizing non-stop data migration between isolated sites, and realizing incremental data synchronization between isolated sites.

The details of one or more embodiments of the invention are set forth in the following drawings and description so that other features, objects, and advantages of the invention are more readily apparent.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other embodiments can be obtained based on these drawings without creative work.

FIG1 is a flow chart of a distributed cluster data synchronization method according to an embodiment of the present invention.

FIG. 2 is a schematic diagram of generating a data operation log and metadata of a storage management unit according to an embodiment of the present invention.

FIG. 3 is a flow chart of a storage management unit sending a data operation log request according to an embodiment of the present invention.

FIG. 4 is a flow chart of a storage management unit receiving a data operation log request according to an embodiment of the present invention.

FIG5 is a flow chart of a storage management unit serving as a Master sending a data snapshot according to an embodiment of the present invention.

FIG. 6 is a schematic diagram of synchronous control of multiple storage management units according to an embodiment of the present invention.

FIG. 7 is a schematic diagram of synchronous task processing based on a distributed consistency algorithm according to an embodiment of the present invention.

FIG. 8 is a schematic diagram of the structure of a distributed cluster according to an embodiment of the present invention.

FIG. 9 is a schematic structural diagram of an electronic device according to this embodiment.

DETAILED DESCRIPTION

Embodiments of the present embodiment will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present embodiment are shown in the accompanying drawings, it should be understood that the present embodiment can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein, which are instead provided for a more thorough and complete understanding of the present embodiment. It should be understood that the drawings and embodiments of the present embodiment are only for exemplary purposes and are not intended to limit the scope of protection of the present embodiment.

As the name implies, the artifact library is a warehouse for artifacts. Artifacts are the fruitful products of software delivery, usually in executable binary form, so the artifact library is often also called a binary artifact warehouse. During the development phase, the artifact library provides a unique entry point for dependency resolution for microservice developers using various development languages. During the build phase, it provides a unique dependency resolution source and a unified artifact management library for various languages for build tools in various languages. After entering the test phase, all test environment deployment tools pull artifacts that meet the test conditions from the artifact library for deployment. After the test is completed, the test result data is fed back to the artifact library and associated with the artifact. During the deployment phase, the artifact is checked based on the quality level to see if it meets the deployment conditions. If it does, the deployment tool pulls the artifact from the artifact library to the environment for deployment.

In order to ensure the security of data synchronization, the real-time data synchronization solution requires a unified synchronized clock between different devices to avoid data synchronization errors and ensure data rollback. However, the product warehouse uses multiple regions for isolation. Since there is no synchronized clock between the isolated regions, the conventional real-time data synchronization method in related technologies cannot be used, and only the shutdown migration method can be used.

For example, in the related technology, Jforg's repository replication data migration solution uploads products through http calls by calling the interface of the existing product system that does not have a product system. However, this method cannot avoid the security and stability of data synchronization under non-Byzantine problems. It only supports product migration but not data migration in other formats, and it cannot isolate data in multiple regions.

In the related art, the open source data migration solution of Rsync establishes an SSH connection with the target system for data transmission and calls the data interface on the remote system for data transmission. This method requires manual migration and cannot automatically synchronize incremental data. It is only a simple file synchronization and cannot establish a connection between files and systems.

The distributed cluster system of this embodiment includes a master cluster and at least one slave cluster. Each cluster includes multiple nodes, which are divided into multiple storage management units isolated from each other according to regions. The storage management unit is also called a region. The master storage management unit and the slave storage management unit for data synchronization are regarded as a storage management unit group. There is only one master storage management unit and at least one or more slave storage management units in the storage management unit group. The master-slave status of the storage management units in the storage management unit group can be switched. In the initial state, the regions in the master cluster are all in the master control role, that is, in the Master state, and the regions in the slave cluster are all in the slave control role, that is, in the Slaver state. The master cluster acts as the cluster that initiates data synchronization during the data synchronization process; the slave cluster acts as the cluster that receives data synchronization during the data synchronization process. However, after the master-slave status of the regions in the master-slave clusters is switched, data synchronization may also be initiated by the slave cluster and received by the master cluster.

For example, during data synchronization, in the initial state, data is written to the region that serves as the master in the master cluster and then synchronized to the region that serves as the slave in the slave cluster. When the master-slave status of two or more regions that are mutually master-slave is switched, data is written to the region that serves as the master in the slave cluster and then synchronized to the region that serves as the slave in the master cluster. region.

In order to achieve data synchronization between isolated sites and avoid service downtime and unavailability, an embodiment of the present invention provides a distributed cluster data synchronization method. FIG1 is a flow chart of a distributed cluster data synchronization method according to an embodiment of the present invention. As shown in FIG1 , the process includes the following steps:

Step S101, determining a synchronization time slice based on the master-slave state switching of a storage management unit group in a distributed cluster.

Step S102: Generate a data operation log of the primary storage management unit in the storage management unit group within the synchronization time slice.

Step S103: synchronize the data operation log to the slave storage management unit in the storage management unit group, so that the slave storage management unit performs data synchronization based on the data operation log.

Through the above steps, based on the master-slave state switching of the storage management unit group in the distributed cluster, the synchronization time slice is determined, and the data storage state of the storage management unit within the synchronization time slice is synchronized based on the data operation log, which solves the problem of real-time data synchronization between isolated multiple storage management units. This data synchronization method can not only realize real-time data synchronization of existing data, but also realize real-time data synchronization of incremental data, without downtime migration, and is also applicable to any type of data synchronization.

Among them, the synchronization status of the data operation log, for example, the synchronized data operation log, the data operation log submitted to the slave storage management unit and other information, this information can be saved in the metadata, and updated according to the synchronization result of the data operation log to determine the data storage status of each storage management unit when synchronizing data. In some embodiments thereof, the method further includes: generating metadata of the master storage management unit so that when synchronizing the data operation log to the slave storage management unit in the storage management unit group, the data operation log to be synchronized in the data operation log is determined based on the metadata, wherein the metadata includes the synchronization status of the data operation log; based on the synchronization result of the data operation log, the metadata is updated.

In some of the embodiments, the data storage state of the storage management unit is used as a finite state machine, so that data synchronization of the storage management unit can be achieved based on the replica synchronization mechanism of the finite state machine. In the replica synchronization mechanism of the finite state machine, replica synchronization is performed based on a synchronization log. In this embodiment, the data operation log is used as a synchronization log for replica synchronization of the finite state machine.

In this embodiment, the predefined data objects include cluster objects, node objects and finite state machines. Among them, the cluster object includes the status of each cluster node. The node object includes the metadata of the node, which is used to record the status information of the cluster during data synchronization, including the sequence number (term) of the synchronization time slice of the storage management unit, the index (logIndex) of the data operation log for marking the location information of the data operation log, the index number (snapshotIndex) of the last data operation log contained in the snapshot, the sequence number (snapshotTerm) of the synchronization time slice of the last data operation log contained in the snapshot, and the index number (preLogIndex) of the maximum log that has been applied to the slave. The node object also includes node location information (endpoint), which is used to represent the coordinates of the node, which is generally the IP address of the terminal in a network environment. The node object can also include the status of the master node that initiates the data synchronization cluster during data synchronization. The finite state machine includes the executed data object and the executed action (i.e., data operation). In this embodiment, the storage management unit with a master-slave relationship is regarded as a finite state machine.

The data storage status includes the existing data, as well as the data added, deleted, and modified based on the existing data. Operation. The stock data is usually stored in the form of data snapshots, and the data operations are recorded in the operation log. If the stock data between the storage management units in a master-slave relationship has been synchronized, that is, the stock data of each storage management unit is the same, then only the data operations on the stock data can be used to represent the data storage status of the storage management unit.

A finite state machine has a finite number of states. The state machine starts at a given start state (such as an initial state, or a state with the same stock data). Each input received by the state machine generates a new state and corresponding output through transition equations and output equations. This new state will remain until the next input arrives, and the generated output will be delivered to the corresponding recipient. Multiple copies of the same state machine can be synchronized between multiple storage management units based on a replicated state machine. Multiple copies of the same state machine start with a start state, and receiving the same input in the same order will reach the same state that has generated the same output, i.e., a replicated state machine.

The term of the synchronization time slice is used as the synchronization clock of all clusters in the distributed cluster. The term of the synchronization time slice is strictly and sequentially incremented from a set value (eg, 0). The term of the synchronization time slice will be cached in a storage system with a high-performance storage medium.

In the above steps, the synchronization time slice of the finite state machine is updated by the master-slave state switching of any group of storage management units. Since the update of the sequence number of the synchronization time slice is marked by the master-slave state switching of the storage management unit, the state of the finite state machine of any storage management unit after the master-slave state switching falls into at least one synchronization time slice, that is, a corresponding relationship can be established between the finite state machine of the storage management unit and the sequence number of the synchronization time slice. Therefore, by obtaining the master-slave state switching logic clock of any group of storage management units in the cluster, the cluster can determine the sequence number of the synchronization time slice of the synchronization time slice that the storage management unit enters from the master-slave state switching logic clock. Since the sequence number of the synchronization time slice is also affected by the master-slave state switching of the storage management units of other groups in the distributed cluster and increases automatically, for a certain storage management unit, the state change of the finite state machine may span multiple synchronization time slices, corresponding to the sequence numbers of multiple synchronization time slices. In this embodiment, the data operation log and metadata generated in each synchronization time slice are marked with the corresponding synchronization time slice sequence number.

When the master-slave state of any group of storage management units in any cluster in the distributed cluster is switched, any cluster reports the master-slave state switching logic clock to the storage system, and the sequence number of the synchronization time slice in the storage system is incremented by one. The storage system will sequentially increment the sequence number of the synchronization time slice according to the master-slave state switching logic clock and then send it to the distributed cluster. In addition, the clusters in the distributed cluster will periodically pull the sequence number of the synchronization time slice to prevent the failure of sending the sequence number of the synchronization time slice.

In this embodiment, the input data of the finite state machine is data operation information. In order to meet the same input requirement of the replication state machine, the data operation information not only needs to have the same operation content, but also needs to ensure the same execution order. Therefore, in any cluster of the distributed cluster, the data operation information of the cluster in the distributed cluster is persisted according to the execution order of the data operation, wherein the data operation information includes the operation object, the operation content, and the information of the storage management unit that executes the data operation.

Specifically, in the cluster that receives data writes (usually the Master cluster), you can attach a point record to each write operation to record the operation object, operation content, and information about the storage management unit that performs the data operation. After encapsulation, it is used as the incremental operation log (pendingLog) generated by the data synchronization initiator, and then the incremental operation log is persisted. The incremental operation logs are stored in the database in the order of execution.

Because in the cluster that receives data writes, the incremental operation logs executed by all storage management units are stored together in the order of execution. In order to synchronize data in units of storage management units, the incremental operation logs can be further grouped and processed according to the storage management units. For example, as shown in Figure 2, a node for processing synchronization tasks is selected in the cluster that receives data writes to start a thread to batch read incremental operation logs from the database, and then divide them according to the storage management units. In the same storage management unit, each log is marked with the serial number of the synchronization time slice and the index number (logIndex) of the data operation log to generate a data operation log for data synchronization.

If the current storage management unit generates a synchronization log (log) for the first time, the index number of the data operation log will be automatically incremented from 0, and the metadata (metadata) of the storage management unit will be initialized at the same time. The data snapshot index number (snapshotIndex) in the metadata is a random value n (for example, it can be any integer between 1-100). The metadata of the storage management unit also includes data used for data synchronization management, for example, the index number of the last data operation log of the storage management unit that has been applied from the node (lastAppliedIndex), and the index number of the data operation log that has been submitted by the storage management unit (committedIndex). During initialization, lastAppliedIndex and committedIndex are set to the same value n as snapshotIndex. The serial number of the synchronization time slice in the metadata is set to the serial number of the current synchronization time slice. The data operation log and metadata are persisted, and the data preparation work for data synchronization is completed.

If the current storage management unit is not generating a synchronization log for the first time, that is, there are already historical data operation logs (including data operation logs that have been synchronized and data operation logs that have not been synchronized), the index number of the newly generated data operation log is increased by one from the index number of the last data operation log generated last time, and the snapshotIndex, lastAppliedIndex, and committedIndex in the metadata are also increased by one from the snapshotIndex, lastAppliedIndex, and committedIndex generated last time, and the sequence number of the synchronization time slice is also set to the sequence number of the current synchronization time slice. Persisting the data operation log and metadata completes the data preparation work for data synchronization.

When data synchronization is performed, data synchronization is implemented based on replica synchronization of the finite state machine. For the storage management unit, the cluster receiving the data write will determine the data operation log to be synchronized in the primary storage management unit according to the index number of the data operation log, the index number of the submitted data operation log recorded in the metadata, and the sequence number of the synchronization time slice, and synchronize the data operation log to be synchronized to the secondary storage management unit.

Referring to Figure 3, the main storage management unit selects a node for processing synchronization tasks, and the node processes the synchronization tasks and selects the storage management units that need to be synchronized. In the synchronization task, the next log index number that needs to be synchronized is first obtained in the cache. If it does not exist, the index number of the last submitted log in the metadata is queried, and then the index number preLogIndex of the previous data operation log is set to the submitted index number minus one. By judging the size of preLogIndex and snapshotIndex, it is determined whether to send a snapshot (the current data snapshot of the finite state machine). If preLogIndex is not less than snapshotIndex, it means that the stock data has been migrated. Otherwise, a batch of data operation logs that need to be synchronized will be taken out from the database and sent to the secondary storage management unit for processing.

The process of processing the data operation log from the storage management unit is shown in FIG4 , and a node for processing data synchronization is also selected from the storage management unit. The node is responsible for processing the data operation log.

In some embodiments, in the same storage management unit group, the master storage management unit and the slave storage management unit When the sequence numbers of the synchronization time slices of the units are the same, the data operation logs to be synchronized are determined and the synchronization of the data operation logs to be synchronized is performed. If the sequence numbers of the synchronization time slices are different, a retry is performed.

If the sequence number of the synchronization time slice of the received data operation log is inconsistent with the sequence number of the synchronization time slice of the current node, it may be that the sequence number of the synchronization time slice of the current node has not been updated or the synchronization request of the data operation log has expired. At this time, the node will return the sequence number of the synchronization time slice of the current node to the main storage management unit. If the sequence numbers of the synchronization time slices are consistent, it means that the synchronization request of the data operation log is valid, and the current storage management unit is checked again to see if it is in the slave role (Slaver) state in the cluster. If it is in the Slaver state, log verification is performed.

The state of the current FSM of the slave storage management unit is determined by the committedIndex of the committed data operation log and the sequence number of the synchronization time slice. If it can match the synchronization request, it only needs to directly apply the log sent by the main storage management unit, update the current FSM state and metadata information, and return the result to the main storage management unit. Otherwise, it is necessary to return the current FSM state to the main storage management unit in the hope that the main storage management unit will send a matching log in the next synchronization request. The main storage management unit may not receive the return value from the slave storage management unit due to a response timeout. In this case, it will retry. If it receives the return value from the slave storage management unit, it will update the next synchronization request sent according to the state returned by the slave storage management unit.

The above synchronization method can be used for the synchronization of incremental data, and can also be used for the synchronization of stock data. Continuing to refer to Figure 3, when the data operation log request is sent for the first time, there is no data operation log of the storage management unit written in the slave storage management unit, and the conflict position returned to the main storage management unit is 0. The main storage management unit continues to send the data operation log after updating the nextLogIndex. At this time, the preLogIndex of the data operation log must be less than the randomly generated snapshotIndex, so the main storage management unit will send the snapshot to migrate the stock data. Snapshot is a snapshot of the FSM of the current storage management unit generated by the main storage management unit. The snapshot stores the indexes of all data (such as products) and related data. In some embodiments, the snapshot sent by the main storage management unit can be the URL of the snapshot, and the actual generated snapshot will be uploaded to the file storage system, which can reduce the data size of the snapshot request. The process of sending snapshots by the main storage management unit is shown in Figure 5. First, the storage management unit that needs to migrate inventory is determined in the set of snapshots that need to be sent, and the FSM snapshot of the storage management unit is generated. After sending an installation snapshot request to the slave storage management unit, the installation status of the slave storage management unit is determined. If the sequence number of the request and the synchronization time slice of the slave storage management unit do not match or the installation of the snapshot from the storage management unit fails, the installation snapshot request will be added to the set of snapshots to be sent for retry.

In some of the embodiments, in the initial state, the storage management units in the master cluster in the distributed cluster are all master storage management units, and the storage management units in the slave clusters in the distributed cluster are all slave storage management units, and the master-slave switching of the storage management units is managed by a preset controller. The preset controller sets the storage management unit with the identifier as the master storage management unit by adding the identifier of the storage management unit to the gray list, so as to realize the master-slave state switching of the storage management units in the distributed cluster.

For example, referring to Figure 6, set the master-slave relationship of different storage management units in different clusters and save it in the master-slave controller that can be queried. Set the master-slave controller for unified access. When the storage management unit is not in the gray list, the storage management units of cluster 1 (for example, the master cluster) are all in the Master state, and the storage management units of cluster 2 (for example, the slave cluster) are all in the Master state. All management units are in the Slaver state, and the data flow direction is from cluster 1 to cluster 2. If the storage management unit is in the gray list, the storage management unit of cluster 2 is in the Master state, and data flows from cluster 2 to cluster 1. Due to bidirectional data writing, the finite state machine (FSM) of the master storage management unit and the slave storage management unit in the same storage management unit group always maintains a consistent state in the safe state. At this time, by controlling whether the storage management unit is in the gray list, the roles of different storage management units in different clusters are controlled to control the flow of data synchronization. The data added by the user in any cluster will not be lost, and the new data will be synchronized in cluster 1 and cluster 2 in the form of synchronization logs. Among them, the above-mentioned safe state means that the FSM state of the master storage management unit and the slave storage management unit in the same storage management unit is consistent, and there is no data operation log that has been applied to the master storage management unit but not yet applied to the slave storage management unit.

In the case where a preset controller is set for querying the master-slave status of the storage management unit, in some embodiments, when new data is written to the storage management unit, the master storage management unit is determined by querying the preset controller, and the new data is written to the master storage management unit. For example, when a user needs to perform a data operation, the user can directly access cluster 1 or cluster 2; the user queries the identification of the storage management unit stored in the master-slave controller to confirm whether the access is the master node or the slave node.

In order to ensure the reliability of data synchronization and avoid data synchronization failure caused by a single node failure, in some embodiments, a leader node and at least one follower node are respectively elected in the main storage management unit and the slave storage management unit, and the leader node in each storage management unit performs the copy synchronization of the finite state machine of the storage management unit with a master-slave relationship. Referring to Figure 7, in this embodiment, based on raft (a distributed consistency algorithm), a leader node can be elected in each cluster or storage management unit to process the synchronization task; at the same time, multiple follower nodes are configured to be able to take over the leader node to continue to process the synchronization task after the leader node fails, ensuring that the data will not be lost and can be synchronized to the slave storage management unit. When the leader node of the main storage management unit crashes or other abnormal situations occur, the data in the main storage management unit has been persisted so that it will not be lost. After selecting a new leader in the follower node, it can continue to synchronize data to the slave storage management unit according to the persisted log and metadata, so the slave storage management unit can still be consistent with the main storage management unit.

The embodiment of the present invention, based on the raft log synchronization technology, can realize the stock data migration and incremental data synchronization of multiple storage management units, and can be applied to files of any format and is not limited to Jforg product files. At the same time, raft is used to ensure the security and reliability of data synchronization under non-Byzantine problems. All synchronized data is applied to the system through FSM, and the association relationship between data can also be synchronized between different systems. By isolating data through multiple storage management units, the synchronization direction of different storage management units between different sites can be controlled.

This embodiment also provides a distributed cluster system. FIG8 is a schematic diagram of the distributed cluster system of this embodiment. As shown in FIG8 , the system includes a cluster 81, at least one cluster 82, and a storage system 83. The cluster 81 serves as a master control role, the cluster 82 serves as a slave control role, and the storage system 83 is connected to the above-mentioned multiple clusters respectively.

The storage system 83 is used to store the serial numbers of the synchronization time slices; the clusters all perform data synchronization through the above-mentioned distributed cluster data synchronization method.

In some embodiments, the cluster switches the master-slave state of the storage management unit group in the distributed cluster to ensure generating a data operation log of the master storage management unit in the storage management unit group within the synchronization time slice; synchronizing the data operation log to the slave storage management unit in the storage management unit group, so that the slave storage management unit performs data synchronization based on the data operation log.

In some embodiments, the cluster generates metadata of the master storage management unit so that when synchronizing the data operation log to the slave storage management unit in the storage management unit group, the data operation log to be synchronized in the data operation log is determined based on the metadata, wherein the metadata includes the synchronization status of the data operation log; based on the synchronization result of the data operation log, the metadata is updated

In some of the embodiments, the cluster persists data operation information of the cluster in the distributed cluster according to the execution order of the data operations, wherein the data operation information includes information of the operation object, operation content, and the storage management unit that performs the data operation; based on the data operation information of the cluster, the data operation log of the main storage management unit within the synchronization time slice is obtained, wherein the data operation log is sorted according to the execution order of the data operations.

In some of the embodiments, the cluster obtains the master-slave state switching logic clock of the storage management unit group, and determines the synchronization time slice and the sequence number of the synchronization time slice according to the master-slave state switching logic clock.

In some of the embodiments, when the master-slave state of the storage management unit of the cluster in the distributed cluster is switched, the cluster reports the master-slave state switching logical clock to the storage system, and the storage system sequentially increments the sequence number of the synchronization time slice according to the master-slave state switching logical clock and sends it to the distributed cluster.

In some of the embodiments, the distributed cluster periodically obtains the sequence number of the synchronization time slice from the storage system.

In some of the embodiments, the cluster determines the data operation log to be synchronized in the primary storage management unit based on the index number of the data operation log, the index number of the submitted data operation log recorded in the metadata, and the sequence number of the synchronization time slice, and synchronizes the data operation log to be synchronized to the secondary storage management unit.

In some of the embodiments, the cluster determines the data operation logs to be synchronized and synchronizes the data operation logs to be synchronized when the sequence numbers of the synchronization time slices of the primary storage management unit and the secondary storage management unit are the same.

In some of the embodiments, the distributed cluster system further includes a preset controller, which is used to control the master-slave state switching of the storage management unit and record the master-slave state of the storage management unit.

In some of the embodiments, in an initial state, the storage management units in the master cluster in the distributed cluster are all master storage management units, and the storage management units in the slave clusters in the distributed cluster are all slave storage management units, and the master-slave switching of the storage management unit group is managed by a preset controller.

In some of the embodiments, the preset controller sets the storage management unit with the identifier of the storage management unit as the main storage management unit by adding the identifier of the storage management unit to the gray list, so as to realize the master-slave state switching of the storage management units in the storage management unit group in the distributed cluster.

In some of the embodiments, when there is data to be written into the storage management unit group, a primary storage management unit in the storage management unit group is determined by querying a preset controller, and the data is written into the primary storage management unit.

In some of the embodiments, the data storage state of the storage management unit is used as a finite state machine. A leader node and at least one follower node are respectively elected in the main storage management unit and the slave storage management unit. The leader node synchronizes the copies of the finite state machine within the storage management unit group based on the data operation log.

An embodiment of the present invention further provides an electronic device, comprising: at least one processor; and a memory in communication with the at least one processor. The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the electronic device executes the method of the embodiment of the present invention.

An embodiment of the present invention further provides a non-transitory machine-readable medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method of the embodiment of the present invention.

An embodiment of the present invention further provides a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to enable the computer to execute the method of the embodiment of the present invention.

With reference to Figure 9, the structural block diagram of the electronic device that can be used as the server or client of an embodiment of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer equipment, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the present invention described herein and/or required.

As shown in FIG9 , the electronic device includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In RAM 903, various programs and data required for the operation of the electronic device can also be stored. The computing unit 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input/output (I/O) interface 905 is also connected to the bus 904.

Multiple components in the electronic device are connected to the I/O interface 905, including: an input unit 906, an output unit 907, a storage unit 908, and a communication unit 909. The input unit 906 can be any type of device that can input information to the electronic device, and the input unit 906 can receive input digital or character information, and generate key signal input related to user settings and/or function control of the electronic device. The output unit 907 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video/audio output terminal, a vibrator, and/or a printer. The storage unit 908 can include but is not limited to a disk, an optical disk. The communication unit 909 allows the electronic device to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and/or a chipset, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and/or the like.

The computing unit 901 may be a variety of general and/or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a CPU, a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above. For example, in some embodiments, the method embodiments of the present invention may be implemented as a computer program, which is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and/or installed on an electronic device via the ROM 902 and/or the communication unit 909. In some embodiments, the computing unit 901 may be a computer program that is tangibly contained in a machine-readable medium, such as a storage unit 908. Element 901 may be configured to execute the above method in any other appropriate manner (eg, by means of firmware).

The computer programs for implementing the methods of the embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer programs are executed by the processor or controller, the functions/operations specified in the flow chart and/or block diagram are implemented. The computer programs may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

In the context of an embodiment of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable signal medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

It should be noted that the term "including" and its variations used in the embodiments of the present invention are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The modifications of "one" and "multiple" mentioned in the embodiments of the present invention are illustrative rather than restrictive. Those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more".

The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

The various steps described in the method implementation methods provided in the embodiments of the present invention may be performed in different orders and/or in parallel. In addition, the method implementation methods may include additional steps and/or omit the steps shown. The scope of protection of the present invention is not limited in this respect.

The term "embodiment" in this specification refers to specific features, structures or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. The various embodiments in this specification are described in a related manner, and the same or similar parts between the various embodiments refer to each other. In particular, for the device, equipment, and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment.

The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of patent protection. It should be pointed out that for ordinary technicians in this field, Without departing from the concept of the present invention, several modifications and improvements may be made, which all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the appended claims.

Claims (16)

A distributed cluster data synchronization method, comprising: Determine the synchronization time slice based on the master-slave state switching of the storage management unit group in the distributed cluster; Generating a data operation log of a primary storage management unit in the storage management unit group within a synchronization time slice; The data operation log is synchronized to a slave storage management unit in the storage management unit group, so that the slave storage management unit performs data synchronization based on the data operation log. The method according to claim 1, wherein the method further comprises: Generate metadata of the master storage management unit so that when synchronizing the data operation log to the slave storage management unit in the storage management unit group, the data operation log to be synchronized in the data operation log is determined based on the metadata, wherein the metadata includes a synchronization status of the data operation log; The metadata is updated based on the synchronization result of the data operation log. The method according to claim 1, wherein generating a data operation log of a primary storage management unit in the storage management unit group within a synchronization time slice comprises: Persisting data operation information of the cluster in the distributed cluster according to the execution order of the data operation, wherein the data operation information includes the operation object, the operation content and the information of the storage management unit that performs the data operation; According to the data operation information of the cluster, a data operation log of the primary storage management unit in the synchronization time slice is obtained, wherein the data operation log is sorted according to the execution order of the data operation. The method according to claim 1, wherein determining the synchronization time slice based on the master-slave state switching of the storage management unit group in the distributed cluster comprises: The master-slave state switching logic clock of the storage management unit group is obtained, and the synchronization time slice and the sequence number of the synchronization time slice are determined according to the master-slave state switching logic clock. The method according to claim 1, wherein the method further comprises: When the master-slave state of the storage management unit of the cluster in the distributed cluster switches, the cluster reports the master-slave state switching logical clock to the storage system, and the storage system sequentially increases the sequence number of the synchronization time slice according to the master-slave state switching logical clock and sends it to the distributed cluster. The method according to claim 5, wherein the method further comprises: The distributed cluster periodically obtains the sequence number of the synchronization time slice from the storage system. The method according to claim 2, wherein synchronizing the data operation log to a slave storage management unit in the storage management unit group comprises: According to the index number of the data operation log, the index number of the submitted data operation log recorded in the metadata and the sequence number of the synchronization time slice, the data operation log to be synchronized in the main storage management unit is determined, and the data operation log to be synchronized is synchronized to the slave storage management unit. According to the method of claim 7, wherein, when the serial numbers of the synchronization time slices of the primary storage management unit and the secondary storage management unit are the same, the data operation log to be synchronized is determined and the synchronization of the data operation log to be synchronized is performed. According to the method according to claim 1, wherein, in an initial state, the storage management units in the master cluster in the distributed cluster are all master storage management units, the storage management units in the slave cluster in the distributed cluster are all slave storage management units, and the master-slave switching of the storage management unit group is managed by a preset controller. According to the method according to claim 9, the preset controller sets the storage management unit with the identifier of the storage management unit as the main storage management unit by adding the identifier of the storage management unit to the gray list, so as to realize the master-slave state switching of the storage management units in the storage management unit group in the distributed cluster. The method according to claim 9, wherein, in the case where there is data to be written into the storage management unit group, a primary storage management unit in the storage management unit group is determined by querying the preset controller, and the data is written into the primary storage management unit. The method according to claim 1, wherein the method further comprises: The data storage state of the storage management unit is used as a finite state machine. A leader node and at least one follower node are respectively selected in the main storage management unit and the slave storage management unit. The leader node synchronizes the copies of the finite state machine within the storage management unit group based on the data operation log. A distributed cluster system comprises a plurality of clusters and a storage system, wherein the plurality of clusters comprises a master cluster and at least one slave cluster, and the storage system is connected to the plurality of clusters respectively; The storage system is used to store the serial number of the synchronization time slice; the multiple clusters all perform data synchronization through the method described in any one of claims 1 to 12. According to the distributed cluster system of claim 13, the distributed cluster system further comprises a preset controller, which is used to control the master-slave state switching of the storage management unit and record the master-slave state of the storage management unit. An electronic device comprises: a processor and a memory storing a program, wherein the program comprises instructions, and when the instructions are executed by the processor, the processor is caused to perform the method according to any one of claims 1 to 12. A non-transitory machine-readable medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 12.