CN113138879A - Method and system for hybrid edge replication - Google Patents

Abstract

Methods and systems for hybrid edge replication are disclosed. The present invention is applied in edge cloud scenarios. For any edge cloud deployment, data processing at the edge is a key feature. For this processing, data needs to be shared across other edge nodes or central clouds, and also needs to be replicated by other edge nodes or central clouds. Therefore, the present invention plays a key role in all edge use cases as it involves processing data from the device or from the cloud. It also helps to handle offline and failure scenarios in edge cloud. Through selective replication, better resource utilization is ensured and the availability of edge cloud services is ensured.

Description

Method and system for hybrid edge replication

Technical Field

The subject matter described herein relates generally to replication databases and, more particularly, to methods and systems for hybrid edge replication.

Background

Replication involves storing copies of data in multiple storage devices. There is only one logical copy of the data and updates to one copy will be reflected on all other copies of the data. This technique offers the following advantages, listed below:

a) performance enhancement: replication allows more requests to be accepted and processed since the workload is distributed across multiple processes.

b) Delay reduction: since the data is copied closer to the user, the copy delay is reduced.

c) And (3) fault reduction: replication allows for a service to overcome a single server failure, as there are other servers that can provide the same service.

d) Fault tolerance: replication also ensures that the correct data is provided to the client.

The technology finds its application in all user devices that continuously synchronize data between the user device and a social media platform. In particular, it allows users to continuously synchronize data between multiple devices without explicitly saving, reloading, and exchanging shared data. This is not limited to distributed versions of mobile applications or social media platforms or desktop applications. In distributed systems, replication is mainly employed to provide fault tolerance. It is a process that entities are replicated. In a distributed system, there are two types of replication policies used, namely active replication and passive replication.

Fig. 1 illustrates a typical edge cloud topology, where each edge node needs to communicate both vertically (north-south) and horizontally (east-west). Data sharing across edge nodes and edge nodes to the cloud is required. High Availability (HA) data replication is required at the cloud and at the edge cloud. Based on data processing and computing location, selective data communication across edges and edge-to-cloud is required.

In passive replication, the primary server processes client requests, updates state on multiple backup servers, and returns responses to the client. If the primary server fails, one of the backup servers will take its place. They can also be used for non-deterministic processes. Fig. 2 shows a block diagram of a passive copy view. This may be referred to as a primary-secondary replication model, where the initial write always occurs on the primary node and then is replicated to the secondary nodes. The disadvantages of passive replication are: the response may be delayed in case of a failure. In particular, the problems associated with edge cloud scenarios are as follows:

a) the primary server cannot be owned at all edges. Since there is a single master server, it can be either at one edge or in the cloud. However, to write at all edges, we need multiple primary servers at the edge;

b) data writes to the edge still occur on the primary server in the cloud. Since there is only one primary server ideally located in the cloud, all write operations at the edge will occur on the primary server in the cloud and then be replicated to the edge's secondary nodes;

c) all data is replicated at all edges because there is only one primary server ideally located in the cloud and secondary servers are located at the edges, all data being replicated from the primary servers in the cloud to the secondary servers at the edges;

d) edge offline scene failures. When there is no connection between the cloud and the edge, the secondary node of the edge will operate by itself. Since the particular number of primary nodes (quorum) selected by the secondary node is minimal, a single secondary node at the edge will not be able to continue to operate. This does not meet the basic requirements of edge calculation (offline mode).

Therefore, passive replication cannot handle selective sharing, replication, and offline scenarios. In passive replication, replication occurs in one direction: from master to replica.

In active replication, all servers process each client request. It requires that the processes hosted by the server are deterministic, i.e. all processes will produce the same sequence of responses and end up in the same final state, given the same initial state and sequence of requests. FIG. 3 illustrates a block diagram of an actively replicated view. This may be referred to as a multi-master model, where all nodes act as master nodes. The disadvantages of active replication are: most real-world servers are non-deterministic. In particular, the problems associated with edge cloud scenarios are as follows:

a) when the main control node is arranged at the edge, all data are copied among all edges;

b) due to the large number of edges, synchronization between copies will be a problem;

c) edge offline scenes are difficult to handle.

Therefore, selective sharing or copying is not handled. In addition, dynamic partitioning or selective properties are missing. In active (also known as cross-centric data replication (XDCR)) replication, a client application performs read/write operations on any participating cluster, and then replicates and applies changes in one database to all other databases.

Distributed in-memory key value repositories, such as memcached, aerospike, voltdb, support passive replication and active replication (also known as cross-centric data replication, or XDCR) to achieve high availability and disaster recovery.

In an edge computing environment, the use of in-memory repositories to store metadata about edge nodes, devices, and edge groups speeds local client write speeds. The replication requirement is slightly different from that provided by the key value store in available memory. Since connection to the cloud cannot always be guaranteed, it is crucial that the client can write secondary edge copies. The secondary edge replica can copy this data to the cloud as long as the cloud connection is available. Passive replication will not meet this requirement.

This can still be achieved with XDCR. In addition, however, the edge nodes may selectively publish some common metadata to all other edge nodes in the same edge group. By default, this functionality is not provided in any key-value store or kv store.

In XDCR, data to be copied between nodes in a cluster is controlled by dividing the data, allowing the data to be divided by edge nodes. However, there is no mechanism to handle the selective replication of data between a set of edge nodes. Thus, in this case, additional logic to be written to the top of the database tier to selectively replicate data, the replication logic provided by the key-value store itself is not actually used, as the logic is error-prone, more complex, and difficult to maintain.

For some prior art reference is made to US 9239767B 2, which discloses selective replication of databases operating in a cluster. The replication task is triggered by the primary server, which builds a federated table in the primary database (db) to update the secondary database. This achieves a loose coupling and facilitates asynchronous replication. A disadvantage associated with this technique is that they employ a master replication mechanism. The join table is constructed for updating the secondary database server, and is not suitable for the edge computing environment in which the edge copy is of the type of the secondary database server and can be written directly by the client.

Reference is also made to US 8315977B 2, which discloses data synchronization between a data center environment and a cloud computing environment. A database manager component running on the data center can detect the change in the data record and transmit it to the cloud. A cloud replicator running in the cloud listens for database record updates and applies the changes to the cloud database. It allows for automatic synchronization of data between the data center and the cloud. The cloud replicator has logic to detect database update changes, convert data to a format specific to the cloud database, and store. A drawback associated with this technique is the need to develop custom database manager and replicator modules to perform the synchronization.

Reference is also made to US 7177886B 2, which discloses an apparatus and method for coordinating logical data replication with high availability data replication. In particular, asynchronous data replication between a primary database server and a backup database server. In the event of a failure of a central database master server, it avoids data inconsistencies between remote servers and provides an asynchronous replication function that is robust against master database failures. It also provides fail-safe recovery through a high availability replication system, while retaining the broad functionality of data distribution through asynchronous replication. A drawback associated with this technique is that it only supports the primary replication mechanism, and the backup server cannot accept write requests.

A problem associated with the state of the art is that the distributed in-memory storage library operates in a passive mode, i.e., writes occur only through the master node, mode or active mode, where all nodes can accept read/write requests and synchronize. In edge computing, an edge node needs to accept local writes and copy data into the cloud, so passive copying cannot meet this requirement. Active replication requires a certain number of pending reads/writes, which cannot be guaranteed in an offline scenario, and therefore active replication also cannot meet the edge requirements. Furthermore, for security reasons, some data of the edge nodes should not be made available to other edge nodes connected to the cloud, i.e. selective sharing across edge nodes. Furthermore, existing built-in replication mechanisms in kv repositories do not provide customization, and therefore additional work needs to be done at the platform level to handle selective data replication.

Therefore, there is a need to selectively copy edge data to a cloud/edge group, selectively share one edge data across edge nodes, and support offline scenarios of edges for data copying and sharing. In particular, hybrid edge replication in key-value stores for edge-computing scenarios is required.

The above-described need for hybrid edge replication of edge computing scenarios is intended only to provide an overview of some of the shortcomings of conventional systems/mechanisms/techniques, and is not intended to be exhaustive. Other problems/disadvantages of conventional systems/mechanisms/techniques and the corresponding benefits of various non-limiting embodiments described herein will become more apparent upon reading the following description.

Disclosure of Invention

This summary is provided to introduce concepts related to a method and system for hybrid edge replication, which is further described below in the detailed description. This summary is not intended to identify essential features of the claimed subject matter, nor is it intended to be used to determine or limit the scope of the claimed subject matter.

It is an object of the present invention to provide a mechanism to handle selective replication, selective sharing between edge nodes in an edge group, and between an edge node and a cloud.

It is another object of the present invention to support offline scenarios of edges for data replication and sharing.

It is yet another object of the present invention to provide a Hybrid Edge Replication Protocol (HERP) for edge computing scenarios in a key-value store.

In particular, the present invention allows an edge client to write to an edge node that will replicate the corresponding data in the cloud, i.e., the master/designated master node and from south to north. The cloud client will be able to write to the master/master server, copying it to the corresponding edge node by providing a new id (north to south). The edge nodes will be able to replicate data between them belonging to the same group, i.e. a southbound binding copy. It supports dynamic selective data replication and supports dynamic selective data sharing.

According to a first aspect of the present invention, there is provided a method for selectively replicating data to a plurality of data nodes. The method comprises the following steps: the user equipment sends a write-in request for writing data in a first data node; wherein the write request includes data and metadata, the metadata including an issue bit and an endpoint bit; the first data node determines whether to: to at least one data node, or to a group of data nodes associated with a group, or to all of the data nodes associated with one or more groups; or to one or more data nodes and groups; each data node comprises a group member relation table; the first data node copies the data to one or more data nodes.

In a first possible implementation form of the method according to the first aspect, the data node is an edge node; and a group includes a plurality of data nodes.

In a second possible implementation form of the method according to the first aspect, the method further comprises the steps of: and the user equipment writes data into the first data node.

In a third possible implementation form of the method according to the first aspect, the method further comprises the steps of: the first data node stores an operation log op _ num to record the data written by the user equipment in the first data node; the first data node increments the commit number commit _ num in the first data node by one count.

In a fourth possible implementation of the method according to the first aspect, each data node comprises Op _ log comprising a number of edge views to determine whether the data node is located at an edge of the network.

In a fifth possible implementation form of the method according to the first aspect, the first data node copies and shares the data when the issue bit is y; wherein, when the issue bit is s, the first data node does not copy and share the data; wherein when the issue bit is r, the first data node replicates the data; and wherein the first data node shares the data when the issue bit is s.

In a sixth possible implementation form of the method according to the first aspect, the end point bits comprise at least one node ID, or at least one group ID, or a combination of a node ID and a group ID.

In a seventh possible implementation form of the method according to the first aspect, the group membership table comprises at least one group ID of a group; and at least one node ID of a data node associated with the group.

In a seventh possible implementation of the method according to the first aspect, the first data node is configured to copy and share the data to at least one data node when the distribution bit is y and the endpoint bit comprises the node ID associated with the data node.

In a ninth possible implementation of the method according to the first aspect, when the issue bit is y and the endpoint bit comprises the group ID and a node ID of a group of data nodes associated with a group, the first data node copies and shares the data to the group of data nodes associated with the group.

In a tenth possible implementation of the method according to the first aspect, when the issue bit is y and the endpoint bit comprises the group ID associated with the group, the first data node copies and shares the data to all of the data nodes associated with one or more groups.

In an eleventh possible implementation of the method according to the first aspect, when the issue bit is y and the endpoint bit comprises one or more node IDs associated with one or more data nodes and the group ID associated with a group, the first data node copies and shares the data to the data nodes and the group.

In a twelfth possible implementation form of the method according to the first aspect, the method further comprises the steps of: the user equipment registering a new group ID associated with a new group with the first data node; the user device registers one or more node IDs associated with one or more data nodes with the new group.

In a thirteenth possible implementation form of the method according to the first aspect, the method further comprises the steps of: the user device deregisters a group ID associated with a group from the first data node; the user device deregisters one or more node IDs associated with one or more data nodes associated with a group.

According to a second aspect of the present invention, there is provided a method for selectively replicating data from a cloud network to a plurality of data nodes. The method comprises the following steps: the method comprises the steps that user equipment sends a write-in request for writing data in a main node of the cloud network; wherein the write request includes data and metadata, the metadata including an issue bit and an endpoint bit; the master node determines whether to: to at least one data node, or to a group of data nodes associated with a group, or to all of the data nodes associated with one or more groups; or to one or more data nodes and groups; wherein each data node comprises a group membership table and the master node comprises a client table; the master node copies the data to one or more data nodes.

In a first possible implementation form of the method according to the second aspect, the cloud network comprises at least one master node and one or more cloud nodes; wherein the data node is an edge node; and a group includes a plurality of data nodes.

In a second possible implementation form of the method according to the second aspect, the method further comprises the steps of: the user equipment writes data to the master node.

In a third possible implementation form of the method according to the second aspect, the method further comprises the steps of: when no node ID is indicated in the endpoint data, the master node determines the data node associated with a group by looking up the group membership table.

In a fourth possible implementation form of the method according to the second aspect, the method further comprises the steps of: the main node stores an operation log op _ num so as to record the data written by the main node in a data node; the master node increments the commit _ num number by one count in the data node.

In a fifth possible implementation of the method according to the second aspect, each data node comprises Op _ log comprising a number of edge views to determine whether the data node is located at an edge of the network.

In a sixth possible implementation form of the method according to the second aspect, when the issue bit is y, the primary node copies and shares the data; when the release position is s, the main node does not copy and share the data; when the release bit is r, the main node copies the data; and wherein the master node shares the data when the issue bit is s.

In a seventh possible implementation form of the method according to the second aspect, the end point bits comprise at least one node ID, or at least one group ID, or a combination of a node ID and a group ID.

In a eighth possible implementation form of the method according to the second aspect, the group membership table comprises at least one group ID of a group; and at least one node ID of a data node associated with the group.

In a ninth possible implementation of the method according to the second aspect, the master node copies and shares the data to at least one data node when the issue bit is y and the endpoint bit comprises the node ID associated with the data node.

In a tenth possible implementation of the method according to the second aspect, when the issue bit is y and the endpoint bit comprises the group ID and a node ID of a group of data nodes associated with a group, the master node copies and shares the data to the group of data nodes associated with the group.

In an eleventh possible implementation of the method according to the second aspect, when the issue bit is y and the endpoint bit comprises the group ID associated with the group, the master node replicates and shares the data to all of the data nodes associated with one or more groups.

In a twelfth possible implementation of the method according to the second aspect, when the issue bit is y and the endpoint bit comprises one or more node IDs associated with one or more data nodes and a group ID associated with a group, the master node replicates and shares the data to the data nodes and the group.

In a thirteenth possible implementation form of the method according to the second aspect, the method further comprises the steps of: the user equipment registering a new group ID associated with a new group with the data node; the user device registers one or more node IDs associated with one or more data nodes with the new group.

In a fourteenth possible implementation form of the method according to the second aspect, the method further comprises the steps of: the user device deregisters a group ID associated with a group from the data node; the user device deregisters one or more node IDs associated with one or more data nodes associated with a group.

According to a third aspect of the present invention, there is provided a system for selectively replicating data to a plurality of data nodes. The system includes a user device that sends a write request to write data at a first data node, wherein the write request includes data and metadata, the metadata including an issue bit and an endpoint bit; the first data node determines whether to: to at least one data node, or to a group of data nodes associated with a group, or to all of the data nodes associated with one or more groups; or to one or more data nodes and groups, wherein each data node comprises a group membership table; the data is replicated to one or more data nodes.

In a first possible implementation form of the system according to the third aspect, the data node is an edge node; and a group includes a plurality of data nodes.

In a second possible implementation form of the system according to the third aspect, the user equipment writes data to the first data node.

In a third possible implementation form of the system according to the third aspect, the first data node stores an operation log op _ num to record the data written by the user equipment in the first data node. And incrementing the commit _ num in said first data node by a count.

In a fourth possible implementation of the system according to the third aspect, each data node comprises Op _ log comprising a number of edge views to determine whether the data node is located at an edge of the network.

In a fifth possible implementation form of the system according to the third aspect, the first data node replicates and shares the data when the issue bit is y; wherein, when the issue bit is s, the first data node does not copy and share the data; wherein when the issue bit is r, the first data node replicates the data; and wherein the first data node shares the data when the issue bit is s.

In a sixth possible implementation form of the system according to the third aspect, the end point bits comprise at least one node ID, or at least one group ID, or a combination of a node ID and a group ID.

In a seventh possible implementation form of the system according to the third aspect, the group membership table comprises at least one group ID of a group; and at least one node ID of a data node associated with the group.

In an eighth possible implementation of the system according to the third aspect, the first data node copies and shares the data to at least one data node when the issue bit is y and the endpoint bit comprises the node ID associated with the data node.

In a ninth possible implementation form of the system according to the third aspect, when the issue bit is y and the endpoint bit comprises the group ID and a node ID of a group of data nodes associated with a group, the first data node copies and shares the data to the group of data nodes associated with the group.

In a tenth possible implementation of the system according to the third aspect, when the issue bit is y and the endpoint bit includes the group ID associated with the group, the first data node copies and shares the data to all of the data nodes associated with one or more groups.

In an eleventh possible implementation of the system according to the third aspect, when the issue bit is y and the endpoint bit comprises one or more node IDs associated with one or more data nodes and a group ID associated with a group, the first data node copies and shares the data to the data nodes and the group.

In a twelfth possible implementation form of the system according to the third aspect, the user equipment registers a new group ID associated with the new group with the first data node; registering one or more node IDs associated with one or more data nodes with the new group.

In a thirteenth possible implementation form of the system according to the third aspect, the user equipment deregisters a group ID associated with a group from the first data node; deregisters one or more node IDs associated with one or more data nodes associated with a group.

According to a fourth aspect of the present invention, there is provided a system for selectively replicating data from a cloud network to a plurality of data nodes. The system comprises: the cloud network comprises user equipment and a server, wherein the user equipment sends a write request for writing data in a main node of the cloud network, the write request comprises data and metadata, and the metadata comprises an issue bit and an endpoint bit; and the master node, the master node determining whether to: to at least one data node, or to a group of data nodes associated with a group, or to all of the data nodes associated with one or more groups; or to one or more data nodes and groups, wherein each data node comprises a group membership table and the master node comprises a client table; and the master node copies the data to one or more data nodes.

In a first possible implementation form of the system according to the fourth aspect, the cloud network comprises at least one master node and one or more cloud nodes; wherein the data node is an edge node; and a group includes a plurality of data nodes.

In a second possible implementation form of the system according to the fourth aspect, the user equipment writes data to the master node.

In a third possible implementation of the system according to the fourth aspect, the master node determines the data node associated with a group by looking up the group membership table when no node ID is indicated in the endpoint data.

In a fourth possible implementation form of the system according to the fourth aspect, the master node stores an operation log op _ num to record the data written by the master node in a data node; the commit number commit num is incremented by one count in the data node.

In a fifth possible implementation of the system according to the fourth aspect, each data node comprises Op _ log comprising a number of edge views to determine whether the data node is located at an edge of the network.

In a sixth possible implementation form of the system according to the fourth aspect, the primary node copies and shares the data when the issue bit is y. When the release position is s, the main node does not copy and share the data; when the release bit is r, the main node copies the data; and wherein the master node shares the data when the issue bit is s.

In a seventh possible implementation form of the system according to the fourth aspect, the end point bits comprise at least one node ID, or at least one group ID, or a combination of a node ID and a group ID.

In a eighth possible implementation form of the system according to the fourth aspect, the group membership table comprises at least one group ID of a group; and at least one node ID of a data node associated with the group.

In a ninth possible implementation form of the system according to the fourth aspect, the master node copies and shares the data to at least one data node when the issue bit is y and the endpoint bit comprises the node ID associated with the data node.

In a tenth possible implementation of the system according to the fourth aspect, when the issue bit is y and the endpoint bit comprises the group ID and a node ID of a group of data nodes associated with a group, the master node copies and shares the data to the group of data nodes associated with the group.

In an eleventh possible implementation of the system according to the fourth aspect, the master node copies and shares the data to all of the data nodes associated with one or more groups when the issue bit is y and the endpoint bit includes the group ID associated with the group.

In a twelfth possible implementation form of the system according to the fourth aspect, when the issue bit is y and the endpoint bit comprises one or more node IDs associated with one or more data nodes and a group ID associated with a group, the master node replicates and shares the data to the data nodes and the group.

In a thirteenth possible implementation form of the system according to the fourth aspect, the user equipment registers a new group ID associated with the new group with the data node; registering one or more node IDs associated with one or more data nodes with the new group.

In a fourteenth possible implementation form of the system according to the fourth aspect, the user equipment deregisters a group ID associated with a group from the data node; deregisters one or more node IDs associated with one or more data nodes associated with a group.

Other aspects, advantages and salient features of the invention will become apparent to those skilled in the art from the following detailed description, which, taken in conjunction with the annexed drawings, discloses exemplary embodiments of the invention.

Drawings

The detailed description is described with reference to the accompanying drawings. In the drawings, the number of a reference numeral identifies the drawing in which the reference numeral first appears. The same numbers are used throughout the drawings to reference like features of components.

FIG. 1 illustrates a block diagram of a typical edge cloud topology as prior art for the present invention.

Fig. 2 shows a block diagram of a passive copy view as prior art of the present invention.

FIG. 3 shows a block diagram of an actively replicated view as prior art for the present invention.

FIG. 4 illustrates a block diagram of a HERP in a passive replication model in accordance with an embodiment of the present invention.

FIG. 5 illustrates a block diagram of a HERP in an active replication model in accordance with another embodiment of the present invention.

Fig. 6 illustrates edge replication based on a set of block diagrams in a HERP according to another embodiment of the present invention.

Fig. 7 shows a block diagram of an offline copy according to the present invention.

FIG. 8 illustrates a block diagram of indirect copying of shared group data in accordance with the present invention.

Fig. 9 shows a block diagram of a master node or master node failure condition according to the present invention.

FIG. 10 illustrates a block diagram of an edge node failure scenario in accordance with the present invention.

FIG. 11 illustrates a block diagram of a HERP view in an edge cloud topology in accordance with the present invention.

Fig. 12 illustrates a block diagram of a HERP location in an edge platform architecture in accordance with the present invention.

FIG. 13 illustrates a block diagram of a HERP membership table in accordance with the present invention.

FIG. 14 illustrates a block diagram of a new metadata for HERP in accordance with the present invention.

FIG. 15 illustrates a block diagram of data being written to an edge and copied to a cloud in accordance with the present invention.

FIG. 16 illustrates a block diagram of data written to an edge and copied to the cloud and published to the edge in accordance with the present invention.

FIG. 17 illustrates a block diagram of data writing without publishing in accordance with the present invention.

FIG. 18 illustrates a block diagram of not publishing data across groups in accordance with the present invention.

FIG. 19 illustrates a block diagram of cloud data writing in accordance with the present invention.

FIG. 20 illustrates a block diagram of data writing to a cloud and replication to a group in accordance with the present invention.

FIG. 21 illustrates a block diagram of data replication across edges in the group, in accordance with the present invention.

FIG. 22 illustrates a block diagram of copying to a particular edge in accordance with the present invention.

FIG. 23 illustrates a block diagram of a fault condition when the cloud is offline in accordance with the present invention.

Fig. 24 shows a block diagram of a failure scenario when a cloud is offline and a partial peer node fails (1) according to the present invention.

Fig. 25 shows a block diagram of a failure case where the cloud is offline and part of the peers fail (2) according to the present invention.

FIG. 26 illustrates a block diagram of a failure condition for a cloud secondary replica failure in accordance with the present invention.

FIG. 27 illustrates a block diagram of a fault condition in which an edge node is being restarted in accordance with the present invention.

Fig. 28 shows a block diagram of a fault situation in which an edge node fails according to the present invention.

FIG. 29 illustrates a block diagram of data coherency processing-time stamp ordering (1) according to the present invention.

FIG. 30 illustrates a block diagram of data coherency processing-timestamp ordering (2) according to the present invention.

FIG. 31 shows a block diagram of data coherency processing-time stamp ordering (3) according to the invention.

FIG. 32 illustrates a block diagram of data coherency processing-time stamp ordering (4) according to the present invention.

FIG. 33 illustrates a block diagram of an ownership based conflict resolution sample in accordance with the present invention.

Fig. 34 shows a block diagram of an Owner Only Publish (OOPub) method according to the present invention.

FIG. 35 illustrates a flow chart of a method for selectively replicating data across a plurality of data nodes in accordance with the invention.

FIG. 36 illustrates a flow chart of a method for selectively replicating data to a cloud network in accordance with the present invention.

FIG. 37 illustrates a flow diagram of a method for selectively copying data to a plurality of data nodes in accordance with a first embodiment of the present invention.

Fig. 38 illustrates a flow chart of a method for selectively replicating data from a cloud network to a plurality of data nodes according to a second embodiment of the invention.

FIG. 39 illustrates a block diagram of a system for selectively replicating data to a plurality of data nodes, in accordance with a third embodiment of the invention.

Fig. 40 illustrates a block diagram of a system for selectively replicating data from a cloud network to a plurality of data nodes, in accordance with a fourth embodiment of the invention.

It is to be understood that the drawings are for purposes of illustrating the concepts of the invention and may not be to scale.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly described below with reference to the drawings in the embodiments of the present invention. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The invention can be implemented in numerous ways, including as a process, an apparatus, a system, a composition of matter, a computer readable medium such as a computer readable storage medium or a computer network wherein program instructions are sent over optical or electronic communication links. In this specification, these embodiments, or any other form that the invention may take, may be referred to as techniques. In general, the order of the steps of disclosed processes may be altered within the scope of the invention.

The following provides a detailed description of one or more embodiments of the invention and accompanying drawings that illustrate the principles of the invention. The invention is described in connection with these embodiments, but the invention is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention encompasses numerous alternatives, modifications and equivalents. Numerous specific details are set forth in the following description in order to provide a thorough understanding of the invention. These details are provided for the purpose of example and the invention may be practiced according to the claims without some or all of these specific details. For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the invention is not unnecessarily obscured.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of various embodiments of the invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the present invention.

Although embodiments of the invention are not limited in this regard, discussions utilizing terms such as, for example, "processing," "computing," "calculating," "determining," "establishing", "analyzing", "checking", or the like, may refer to operation(s) and/or process (es) of a computer, a computing platform, a computing system, or other electronic computing device, that manipulate and/or transform data represented as physical (e.g., electronic) quantities within the computer's registers and/or memories into other data similarly represented as physical quantities within the computer's registers and/or memories or other information non-transitory storage medium that may store instructions for performing the operations and/or processes.

Although embodiments of the present invention are not limited in this respect, the terms "plurality" and "a plurality" as used herein may include, for example, "multiple" or "two or more". The terms "plurality" or "a plurality" may be used throughout the specification to describe two or more components, devices, elements, units, parameters and the like. Unless explicitly stated, the method embodiments described herein are not limited to a particular order or sequence. Additionally, some of the method embodiments or elements thereof may occur or be performed simultaneously, at the same point in time, or concurrently.

In the present invention, "edge cloud topology" refers to a manner in which various edge nodes are deployed.

In the present invention, an "edge node" refers to a node connected at an edge cloud. The edge node may be connected to various passive devices (sensors/actuators) or another edge node.

In the present invention, an "edge client" refers to any application or service request, or writing data to a node. The same is true for clouds and edges.

In the present invention, "edge copy" refers to data copied at the edge in accordance with the configuration explained in the present invention. Replication is a standard process and no distinction is made.

In the present invention, "passive replication" refers to replication that occurs in one direction (i.e., from master to replica).

In the present invention, "active/XDCR replication" refers to replication in which a client application performs read/write operations on any participating cluster, and copies and applies changes in one database to all other databases.

In the present invention, a "kv repository" refers to a key-value repository, which is a key building block for many systems (e.g., databases and large web sites).

In the present invention, "metadata" refers to metadata information of actual data.

Methods and systems for hybrid edge replication are disclosed. Although aspects of Hybrid Edge Replication Protocol (HERP) in a key-value (kv) store for edge-computing scenarios are described, the present invention may be implemented in any number of different computing systems, environments, and/or configurations, which are described in the context of the following exemplary systems, devices/nodes/apparatus, and methods.

Embodiments of the disclosure are explained hereinafter with the aid of exemplary figures and one or more examples. However, such exemplary figures and examples are provided for a better understanding of the present disclosure and should not be construed to limit the scope of the present disclosure.

The invention provides a method and system for hybrid edge replication.

A problem associated with the state of the art is that the distributed in-memory storage library operates in a passive mode, i.e., writes occur only through the master node or active mode, where all nodes can accept read/write requests and synchronize. In edge computing, an edge node needs to accept local writes and copy data into the cloud, so passive copying cannot meet this requirement. Active replication requires a certain number of pending reads/writes, which cannot be guaranteed in an offline scenario, and therefore active replication also cannot meet the edge requirements.

Furthermore, for security reasons, some data of the edge nodes should not be made available to other edge nodes connected to the cloud, i.e. selective sharing across edge nodes. Furthermore, existing built-in replication mechanisms in kv repositories do not provide customization, and therefore additional work needs to be done at the platform level to handle selective data replication. Therefore, there is a need to selectively copy edge data to a cloud/edge group, selectively share one edge data across edge nodes, and support offline scenarios of edges for data copying and sharing.

Accordingly, the present invention discloses a method and system for hybrid edge replication in a kv repository of edge computation scenarios. In particular, the present invention allows an edge client to write to an edge node that will replicate the corresponding data in the cloud, i.e., the master/designated master node and from south to north. The cloud client will be able to write to the master/master server, copying it to the corresponding edge node by providing a new id (north to south). The edge nodes will be able to replicate data between them belonging to the same group, i.e. a southbound binding copy. It supports dynamic selective data replication and supports dynamic selective data sharing. The hybrid edge replication protocol has a corresponding interface in the cloud.

It is clear that edge calculation can handle large amounts of data and perform maximum processing at the edges. Thus, data processing needs to be highly configurable for replication and sharing. The invention provides highly configurable and extensible data processing for edge computing scenarios. Selective data filtering across edge-to-cloud and edge-to-edge is crucial for efficient and intelligent edge computing solutions. However, existing implementations are based on custom built modules. The present invention provides an immutable kv repository protocol to handle selective data sharing and replication.

The present invention supports complete southbound (edge) binding recovery and HA, i.e. processed offline, shutdown/restart conditions. It also provides selective sharing of data across edges or edge to cloud to implement specific use cases or tasks as needed. The edge nodes can easily share public/private data among each other or with the cloud in the group, providing flexibility and higher security. The master independent model facilitates easily extensible and extensible data processing, including replication, sharing, and the like. It facilitates data coherency handling and is compatible with any standard data conflict resolution method, such as sorting based on time stamps or ownership. Configurable and dynamic selective data replication and sharing facilitates efficient processing of a variety of different edge scenarios without any additional development.

In the present invention, each edge group communicates with one master replica in the cloud. This allows to maintain simplicity and scalability of the replication. If the primary copy fails, the edge group need only switch to the newly elected leader. This scheme is easy to scale if more edge nodes are added to the group or more groups are added to the cluster. The present invention HAs excellent fault tolerance characteristics for recovery and HA. Migration of devices from edge to edge or from group to group can be achieved very easily. It is highly supportive to data analysis at the edge. The present invention may also be implemented for specific kv repositories and other databases.

The present invention can also support intelligent edge networks (IEFs) to handle edge cloud scenarios and competing features by using HERP:

a) dynamic selective data replication and sharing;

b) HA, recovery and scalability (offline and online scenarios);

c) equipment migration (large future demand); and

d) dynamically configurable data management and analysis.

Fig. 4 shows a block diagram of the HERP in the passive replication model. Notably, in the following figures, C represents a client; e represents an edge; p represents the primary replica and S represents the secondary replica. The primary and secondary replicas will reside in the cloud. The edge node will support a new edge replica with HERP with a corresponding interface in the master replica in the cloud. The edge client can write in the edge copy, the edge copy copies corresponding data to the primary copy in the cloud, and the primary copy further copies the corresponding data to the secondary copy in the cloud. The cloud client may write to the primary copy in the cloud, and in addition to copying to the secondary copy in the cloud, the primary copy will also copy to the corresponding edge node based on the edge ID. Furthermore, edge copies belonging to the same group may replicate data among themselves, as detailed in [ 00105 ] below. As explained in [ 00133-. Thus, under the passive replication model, selective data replication and selective data sharing are achieved using the HERP.

FIG. 5 illustrates a block diagram of a HERP in an active replication model. The master copy will reside in the cloud. The edge node will support a new edge copy with the HERP with a corresponding interface in the master copy in the cloud. The edge client can write in the edge copy, the edge copy copies corresponding data to a corresponding master copy in the cloud, and the master copy further copies the data to other master copies in the cloud. The cloud client can write the master copy in the cloud, and the master copy is copied to the corresponding edge node based on the edge ID besides being copied to the master copy in the cloud. Furthermore, edge copies belonging to the same group may replicate data among themselves, as detailed in [ 00105 ] below. As explained below [ 00133-. Thus, under the active replication model, selective data replication and selective data sharing are achieved using the HERP.

FIG. 6 illustrates a block diagram of group-based edge replication in HERP. Edges may be grouped for selectively replicating data between edge replicas, which avoids dependency on clouds during offline scenarios. The edge client may write to the edge copy, which copies the data to the corresponding edge copy in the group. Replication to the cloud will occur when the cloud is online or through indirect replication (as detailed in [ 00107 ]).

Fig. 7 shows a block diagram of offline replication. Notably, CC stands for cloud client, EC for edge client, E for edge, and P/M for master/primary node. The edge nodes may be offline from the cloud and/or from other edges of the group. When the cloud is online, edge data that does not correspond to any group will be copied directly to the cloud. The edge data corresponding to a certain group may be copied to other edges in the group, either directly (when additional edges are in an available online state) or indirectly in an offline state, as detailed in [ 00107 ].

FIG. 8 illustrates a block diagram of indirect replication of shared group data. Some or all of the edges in the group may be offline from the cloud. Some or all of the edge nodes corresponding to a group may also be offline from the edge. Edge data corresponding to a group may be replicated to another offline edge in the group (either indirectly through the cloud when the edge is online with the cloud or through another edge in the group that is online with the offline edge).

Fig. 9 shows a block diagram of a master node or master node failure condition. In a passive replication scenario, all master/slave nodes contain edge and group information. When a master node fails, the new master node indicates the same to the edge nodes and data replication will occur with the new master node. In the active replication scenario, the edge nodes contain information of all master nodes. And when the current main control node fails, the edge node selects a new main control node. Thus, even if a PM node in the cloud fails, data replication is handled between the cloud and the edge.

FIG. 10 illustrates a block diagram of an edge node failure scenario. Notably, CC stands for cloud client, EC for edge client, E for edge, and P/M for master/primary node. In the event of an edge node failure, the corresponding edge data is replicated in the P/M in the cloud and in the other edges corresponding to its group. When an edge node recovers from a failure, the edge data is replicated from the P/M and other edge nodes corresponding to its group. Thus, even if an edge node fails, data replication is handled between the cloud and the edge.

The replication mechanism in the cloud can still continue to perform replication using its selected replication protocol. The HERP protocol will not change this replication behavior in the cloud. This ensures that the HERP protocol is loosely coupled and only edge-side replication is of concern. FIG. 11 illustrates a block diagram of a HERP view in an edge cloud topology. The HERP allows for seamless support of multi-cluster, multi-group deployment scenarios, ensures that the default cloud replication model is not impacted, and can be inserted as an extension to existing replication protocols in Kv repositories, providing easy integration. By using the HERP, edge-to-edge, edge-to-cloud copying and data sharing are realized through dynamic selective support.

Fig. 12 illustrates a block diagram of HERP positioning in an edge platform architecture. In this architecture, each edge and cloud cluster node has a kv repository that supports HERP. Data replication and sharing will follow the HERP protocol, except for the existing replication support of the underlying kv repository.

FIG. 13 illustrates a block diagram of a HERP membership table. The primary replica registers edge nodes belonging to a certain group in an "edge-group-membership" table.

FIG. 14 illustrates a data set operation flow. The client writes to edge node e 1. pub ═ y "indicates that the write needs to be replicated across all edges in the set. Group membership metadata is only copied to the edges belonging to the group. Further, endpoints may be specified for selective sharing/copying when publishing data.

FIG. 15 illustrates a data set operation flow. The edge node e1 records the write operation in the operation log and updates the commit number by 1. Thus, it ensures that writes performed by the client are now persisted and accepted.

FIG. 16 illustrates a block diagram of data writing to an edge and copying to the cloud and publishing to the edge. The primary replica responds with a reply e1 message. Since the request includes publishing data to all edge numbers, e1 looks up the edge group membership table and sends a publish message to all edge nodes in the group.

FIG. 17 shows a block diagram of data writing without publishing. If the request does not include publication to all group members, e1 will not publish the message.

FIG. 18 illustrates a block diagram of data not being published across groups. Since e1 and e2 are in different groups, e1 does not publish a message to e 2.

FIG. 19 illustrates a block diagram of cloud data writing. The client writes to the primary replica and specifies the details of the group and edge nodes to which the data needs to be copied or shared.

FIG. 20 illustrates a block diagram of data being written to the cloud and copied to the group. If the edge id is not provided in the request, the master node searches the edge group membership table to find the edge node belonging to the group, records the edge id in the client table, updates the operation log, sends a preparation message to the subsequent part of the operation log, and finally executes a write request after the number of submissions is increased each time.

FIG. 21 illustrates a block diagram of data replication across edges in a group. If the original request contains pub ═ y ", the master node issues the write request to all edge nodes belonging to the group (after looking up from the edge-group-membership table), passing the original client-id it received in the request.

FIG. 22 shows a block diagram of copying to a particular edge. The client writes to the primary replica in the cloud, specifying the group and edge nodes to which the data is to be replicated, but does not wish to be published to other edge nodes in the group. Data is copied to a specified edge node in a group but is not shared with other edge nodes in the same group.

A failure may occur when the edge processes read/write requests. One such failure scenario may be the presence of intermittent/cloudless connections. In the following, the invention explains how to handle faults:

cloud offline: edge operations are not affected; synchronization occurs to the edge peers; synchronization to the cloud occurs while the cloud is in an online state to obtain selective expected data. FIG. 23 illustrates a block diagram of a fault condition when the cloud is offline.

Cloud offline and some peer node failures

Cloud offline and some peer nodes fail: edge operations are not affected; synchronizing to available edge peers; synchronization to the cloud and other peers occurs while both the cloud and other peers are online.

Fig. 24 shows a block diagram of a failure case when the cloud is offline and part of the peers fail (1). The client writes to edge node e 1. e1 sends write requests to the master replica in the cloud and to the peer nodes in the group. e1 issues write requests to other edge nodes available in the group. Each time the cloud connection is restored, e1 sends a publish message to the cloud. Whenever other offline peer nodes become online, data is shared between them. Data is only published to the cloud and other peers when it is online, if it is dedicated to the cloud and peers. The transaction view and number of submissions will be used.

Fig. 25 shows a block diagram of a failure case when the cloud is offline and part of the peers fail (2). The client writes the data to the cloud. The cloud sends the request to the edge node, but since the node is offline, a publish operation will occur when the node comes back online. According to selective sharing, the publishing operation occurs at a particular edge for a particular data.

FIG. 26 illustrates a block diagram of a failure condition when a cloud secondary fails. The client writes the data to the cloud. The cloud sends the request but cannot access the edge node. When the node is recovered to be on-line, writing in the cloud end normally. According to selective sharing, the publishing operation occurs at a particular edge for a particular data.

FIG. 27 illustrates a block diagram of a fault condition in which an edge node is being restarted. The replicated data is available in the cloud and with the peers in the group. The restarting edge node sends a group broadcast for recovery. The peer nodes in the group respond with data. The edge node may restore the data shared between all the group members and may start running from the last stopped position.

Fig. 28 shows a block diagram of a fault condition for an edge node fault.

The recovery possibilities provided by the present invention are indicated below:

a) if the system supports a new edge node to provide e1 credentials, it can recover the data from e2 and the cloud and work properly. (same sequence as restart scene)

b) If e1 has a backup system, it can simply take over and recover data from e2 and the cloud.

c) If 1 and 2 are not possible, a recovery method can be implemented from the cloud to get any failed edge nodes through the replicated group data replicated by the HERP. This security credential and set-up recovery are out of our scope. However, the e1 data copy will be available for recovery.

HERP: new API (draft)

// edge copy group management

Registration group (group Id string) error

Deregistration group (group Id string) errors

Group Id: name of group to be registered/deregistered

// edge node management

Add node to group (group Id string, node Id string) error

Removing node (group Id string, node Id string) errors from a group

Group Id: group name (Add or remove)

Node Id: ID of edge node or cloud cluster node

Broadcast message to group/edge

Post (group Id string, endpoint [ ] string, message string, post option [ ] string) error

Group Id: name of group to be published to

End point: specific endpoint to which data is to be published

Message: payload

And (4) issuing an option: based on the deployment model, the publication options may include a publication flag option, timestamp information/ownership information, and the like.

And issuing a mark option:

y-shared and replicated

not shared and not duplicated

r-copy only

s-shared only

Rejection (reject message string, endpoint string)

Rejection message: reason for rejection

End point: endpoint name

// write data to group/edge

Write (group Id string, endpoint [ ] string, message string) error

Group Id: name of group to be published to

End point: specific endpoint to which data is to be published

Message: payload

// recovery management

Recovery of (group Id string, nonce integer, recovery option [ ] string) errors, recovery response

Group Id: group name (Add or remove)

Temporary value: unique identifier for identifying sequence of recovery requests (avoid duplicate messages)

And (4) recovery options: recovery options that may include endpoint id

And (3) recovering response: the recovery contains the oplog, number of commits, response.

Note that: all dynamic options can be set to static configurations. If no dynamic metadata is given, the system will follow the static configuration.

Data consistency handling in HERP. The present invention provides consistency, availability and partitioning. In particular, it provides availability and dynamic real-time partitioning, which ultimately results in consistency. The final consistency is achieved based on the existing method and is not affected by the invention. This involves two key methods, namely, a) timestamp ordering, and b) ownership. The method may be determined based on edge cloud deployment and user requirements. HERP also provides some methods to facilitate standard methods to work better in edge scenes like OOPub (Owner Only publication).

Method 1-sort with timestamp (1).

FIG. 29 shows a block diagram of data coherency processing — timestamp ordering (1). The network timestamp server provides a unique timestamp for each data transaction. To resolve the conflict, the transaction with the latest timestamp is performed by the network timestamp server. Edge nodes e1 and e2 update the same attribute "a". They obtain timestamps t2 and t3 from the network timestamp server. Let t1< t2< t 3. Therefore, t3 is the latest transaction. The client table of the respective edge node also maintains a write timestamp (w _ ts) and a read timestamp (r _ ts) for each key-value pair.

FIG. 30 shows a block diagram of data coherency processing — timestamp ordering (2). Both edge nodes e1 and e2 issue write requests to each other with their timestamps.

Fig. 31 shows a block diagram of data consistency processing-time stamp sorting (3). Since t2< t3, edge node e2 retains the write value, and e1 receives a reject response from e 2. E1 rolls back (rollback) its write operations and reduces its number of operations and number of commits.

FIG. 32 shows a block diagram of data coherency processing-time stamp sorting (4). The publish request from e2 needs to be processed by e 1. Since t1< t3, e1 accepts issue requests from e2 and updates "a" with the new value.

Method 2-ownership based conflict resolution. This is the standard method used in most standard replication systems. This is common in deploying systems with multiple master or write nodes.

a) Systems based on single ownership or shared ownership may be used. Basically, conflicts will be resolved based on ownership of each data.

b) Dynamic ownership is also used in some shared ownership systems to manage update workflows based on deployment scenarios.

FIG. 33 illustrates a block diagram of a sample ownership-based conflict resolution. A variety of ownership-based data conflict management methods may be used, and any of these methods may be applied to the present invention. Since there are no dependencies, ownership can be configured based on the deployment. In an edge scenario, the data source may be set as the owner, dynamic ownership techniques may be used as needed, and token-based techniques or similar techniques may be applied to handle ownership. Notably, an owner-only publish (OOPub) algorithm can be added to the HERP, which can be configured as desired.

Ownership data conflict resolution is facilitated — owner only publish (OOPub) approach. In this scenario, only the owner can update the data, while others can only read the data. This approach may be used for edge-to-cloud or edge-to-edge scenes. Typically, the device data is owned by the device in the respective edge node. The configurable metadata may be owned by the user role (cloud or user modification interface).

FIG. 34 illustrates a block diagram of an "owner only publish" (OOPub) approach. FIG. 35 illustrates a flow chart of a method for selectively replicating data across a plurality of data nodes in accordance with the invention. FIG. 36 illustrates a flow chart of a method for selectively replicating data to a cloud network in accordance with the present invention.

FIG. 37 illustrates a method for selectively copying data to multiple data nodes (12) according to a first embodiment of the invention₁、12₂、12₃、……、12_n) A flow chart of the method of (1). The method comprises the following steps:

a user equipment (10) sending (S101) a write request to write data at a first data node (11); wherein the write request includes data and metadata, the metadata including an issue bit and an endpoint bit;

the first data node (11) determines (S102) whether to: is replicated to at least one data node (12)₁) Or copied to and group (G)₁) An associated set of data nodes (12)₁、12₃、12₄) Or to one or more groups (G)₁、G₂、……、G_n) All of the data nodes that are associated; or to one or more data nodes (12)₁、12₄) And group (G)₂)；

Wherein each data node (11, 12)₁、12₂、……、12_n) Including a group membership table;

the first data node (11) copying (S103) the data to one or more data nodes (12)₁、12₂、12₃、……、12_n)。

In the method illustrated in FIG. 37, a data node (12)₁、12₂、12₃、……、12_n) Are edge nodes. A group comprises a plurality of data nodes (12)₁、12₂、12₃、……、12_n)。

The method shown in fig. 37 further comprises the steps of: the user equipment (10) writes data to the first data node (11). The method further comprises the following steps: the first data node (11) stores an operation log op _ num to record the data written by the user equipment (10) in the first data node (11); the first data node (11) increments the commit number commit _ num in the first data node (11) by one count. The method further comprises the following steps: -the user equipment (10) registering a new group ID associated with a new group with the first data node (11); the user equipment (10) registers one or more node IDs associated with one or more data nodes with the new group. The method further comprises the following steps: the user equipment (10) deregisters a group ID associated with a group from the first data node (11); the user equipment (10) de-registers one or more node IDs associated with one or more data nodes associated with a group.

Notably, when the issue bit is y, the first data node (11) replicates and shares the data. When the issue bit is s, the first data node (11) does not replicate and share the data. When the issue bit is r, the first data node (11) replicates the data. When the issue bit is s, the first data node (11) shares the data. Further, each data node includes an Op _ log including a number of edge views to determine the data node (12)₁、12₂、12₃、……、12_n) Whether located at the edge of the network. The end bits include at least one node ID, or at least one group ID, or a combination of node ID and group ID. The group membership table includes a group (G)₁、G₂、……、G_n) At least one group ID of; and a data node (12) associated with the group₁、12₂、12₃、……、12_n) At least one node ID.

Obviously, in the method illustrated in fig. 37, the first data node (11):

when the issue bit is y and the endpoint bit comprises the data node (12)₁) Upon associating the node ID, copying and sharing the data to at least one data node (12)₁)；

When the issue bit is y and the endpoint bit comprises an AND group (G)₁) An associated set of data nodes (12)₁、12₃、12₄) When the group ID and the node ID are present, copying and sharing the data to the group (G)₁) The associated set of data nodes (12)₁、12₃、12₄)；

When it is at homeWhen the issue bit is y and the endpoint bit includes the group ID associated with the group, copying and sharing the data to one or more groups (G)₁、G₂、……、G_n) All of the data nodes that are associated;

when the issue bit is y and the endpoint bit comprises the data node (12)₁、12₄) Associated one or more node IDs and a group (G)₂) When the group ID is associated, the data is copied and shared to one or more data nodes (12)₁、12₄) And said group (G)₂)。

FIG. 38 illustrates a method for selectively replicating data from a cloud network to a plurality of data nodes (11, 12) according to a second embodiment of the invention₁、12₂、……、12_n) A flow chart of the method of (1). The method comprises the following steps:

a user equipment (20) sending (S201) a write request to write data at a master node (21) of the cloud network, wherein the write request comprises data and metadata, the metadata comprising an issue bit and an endpoint bit;

the master node (21) determining (S202) whether to: is replicated to at least one data node (12)₁) Or copied to and group (G)₁) An associated set of data nodes (12)₁、12₃、12₄) Or to one or more groups (G)₁、G₂、……、G_n) All of the data nodes that are associated; or to one or more data nodes (12)₁、12₄) And group (G)₂) Wherein each data node (11, 12)₁、12₂、……、12_n) Comprises a group membership table and the master node (21) comprises a client table;

the master node (21) replicating (S203) the data to one or more data nodes (12)₁、12₂、12₃、……、12_n)。

The cloud network comprises at least one master node (21) and one or more cloud nodes; wherein the data sectionDot (12)₁、12₂、12₃、……、12_n) Is an edge node and a group includes a plurality of data nodes. Further, each data node includes an Op _ log that includes a number of edge views to determine whether the data node is located at an edge of the network.

The method as illustrated in fig. 38 further comprises the steps of: the user equipment (20) writes data to the master node (21). The method further comprises the following steps: when no node ID is indicated in the endpoint data, the master node (21) determines the data node associated with a group by looking up the group membership table. The method further comprises the following steps: the master node (21) stores an operation log op _ num to record the data written by the master node (21) in a data node; the master node (21) increments the commit number commit _ num in the data node by one count.

The method as illustrated in fig. 38 further comprises the steps of: the user equipment (20) registering a new group ID associated with a new group with the data node; the user equipment (20) registers one or more node IDs associated with one or more data nodes with the new group. The method further comprises the following steps: the user equipment (20) deregisters a group ID associated with a group from the data node; the user equipment (20) de-registers one or more node IDs associated with one or more data nodes associated with a group.

Obviously, when the issue bit is y, the master node (21) copies and shares data. When the issue bit is s, the master node (21) does not copy and share data. When the issue bit is r, the master node (21) copies the data. When the issue bit is s, the master node (21) shares data. Further, the end point bits include at least one node ID, or at least one group ID, or a combination of a node ID and a group ID. The group membership table includes a group (G)₁、G₂、……、G_n) At least one group ID of; and a data node (12) associated with the group₁、12₂、12₃、……、12_n) At least one node ID.

Notably, in the method illustrated in fig. 37, the master node (21):

when the issue bit is y and the endpoint bit comprises the data node (12)₁) Upon associating the node ID, copying and sharing the data to at least one data node (12)₁)；

When the issue bit is y and the endpoint bit comprises an AND group (G)₁) An associated set of data nodes (12)₁、12₃、12₄) When the group ID and the node ID are present, copying and sharing the data to the group (G)₁) The associated set of data nodes (12)₁、12₃、12₄)；

When the issue bit is y and the endpoint bit includes the group ID associated with the group, copying and sharing the data to one or more groups (G)₁、G₂、……、G_n) All of the data nodes that are associated;

when the issue bit is y and the endpoint bit comprises the data node (12)₁、12₄) Associated one or more node IDs and a group (G)₂) When the group ID is associated, the data is copied and shared to one or more data nodes (12)₁、12₄) And said group (G)₂)。

FIG. 39 illustrates a method for selectively copying data to multiple data nodes (12) according to a third embodiment of the present invention₁、12₂、12₃、……、12_n) A block diagram of a system of (1). The system comprises a user equipment (10), a first data node (11) and a plurality of data nodes (12)₁、12₂、12₃、……、12_n). The system is adapted to perform the method illustrated in fig. 35 and 37.

FIG. 40 illustrates a method for selectively replicating data from a cloud network to a plurality of data nodes (12) according to a fourth embodiment of the invention₁、12₂、12₃、……、12_n) A block diagram of a system of (1). The system comprises a user equipment (20), a main node (21) and a data sectionDot (12)₁、12₂、12₃、……、12_n). The system is adapted to perform the method illustrated in fig. 36 and 38.

Referring to table 1, the HERP of the present invention is compared with the viewstampedreplication (vr) Protocol, while in table 2, the present invention is compared with the Open Source redispersion Protocol (Open Source redispersion Protocol). The HERP differs from both protocols such that it allows for secondary write operations at edge nodes, can work in an offline scenario because it does not involve master leader elections, and it supports group-based selective sharing, which is not available in both other protocols.

Table 1 lists the HERP and ViewStamped Replication (VR) protocols

Table 2 lists the HERP and open source Redis replication protocols

Implementation with different kv repository/database

a) Easily integrated with etcd (raft) or such kv banks;

b) easy integration with standard Replication protocols (experiments were performed using the ViewStamped Replication protocol illustrated by Diamond);

c) it is based on this architecture and can be integrated with standard database replication methods (however, possible innovations can be made thereto if desired).

Example (c): required modifications in Redis

a) Enterprise Redis uses CRDT and vector clocks to implement active replication.

b) A synchronizer performing CRDT peer-to-peer replication would need to be modified to implement selective replication using the publishing API

Performance impact of HERP

a) Very little metadata, and therefore minimal conditional validation during any transaction. Can be implemented without impacting system performance (since it is very consistent with the standard metadata of kv repository).

b) Since the metadata remains minimal, it can be implemented in an optimized manner to handle the performance and footprint requirements of the HERP.

c) publish () will replace write (), so there is no additional messaging between nodes.

d) Even in the case of system expansion (more devices/more edge nodes), the search requirement across nodes is very low since we only publish some feature by owner. Even if all edges publish data (not just the owner), it will be bound to the edge nodes. Again, this may be an edge cloud based hierarchical deployment.

e) The worst case performance: if the same data in one of the other edges in the group is updated at the same time, the requests issued to the edges in the group will be affected. The reject message needs to be processed by the source edge node and the roll-back needs to be performed by all other edges in the group that do not report a conflict. This is only the case when there is no OOPub and concurrent updates occur to all edges in the group. This is less likely.

The invention finds its application in edge cloud scenarios. For any edge cloud deployment, data processing of the edge is a key feature. For this process, data needs to be shared between and replicated by other edge nodes or central clouds. Thus, the present invention plays a crucial role in all edge use cases, as it involves data processing from devices or from the cloud. It also helps to handle offline and failure scenarios in the edge cloud. By selective replication, better resource utilization is ensured and the availability of edge cloud services is ensured.

Some non-limiting advantages and future implementations of the invention are indicated below:

it provides highly configurable and scalable data processing for edge computing scenarios.

Its selective data filtering across edge-to-cloud and edge-to-edge is crucial for efficient and intelligent edge computing solutions.

It provides a complete southbound (edge) binding recovery and HA (handled offline, shutdown/restart case).

It provides selective sharing of data across edges or edge to cloud to implement specific use cases or tasks as needed.

It provides flexibility and higher security since edge nodes can easily share public/private data between each other in a group or with a cloud.

Its master independent model provides data processing (including replication, sharing, etc.) that can be easily extended and extensible.

It facilitates data consistency handling and is compatible with any standard data conflict resolution method (like sorting or ownership based on timestamps).

Can be configured and movedThe selective data replication and sharing of states allows for efficient processing of a variety of different edge scenarios without any additional development.

This is simple and extensible, and when the primary replica fails, the edge group simply switches to the newly elected leader.

It provides excellent fault tolerance features for recovery and HA.

The protocol runs seamlessly even if the edge clusters grow or shrink dynamically.

It provides easy device migration from edge to edge or from group to group.

It is highly supportive to data analysis at the edge.

It provides dynamic selective data replication and sharing.

It provides HA, recovery and scalability in both offline and online scenarios.

It provides dynamically configurable data management andand (6) analyzing.

It finds future application in particular kv repositories and other databases.

Those skilled in the art will appreciate that any known or new algorithm may be used to implement the present invention. It should be noted, however, that the present invention provides a method and system for hybrid edge replication to achieve the benefits and technological advances described above, regardless of the use of any known or new algorithms.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps described in connection with the examples disclosed in the embodiments disclosed herein may be embodied in electronic hardware or in a combination of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

It will be clearly understood by those skilled in the art that for the sake of convenience and simplicity of description, reference may be made to corresponding processes in the foregoing method embodiments for the detailed working of the foregoing systems, devices and units, and details will not be described herein.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus, method, or system may be implemented in other ways. For example, the described apparatus embodiments are merely exemplary. For example, the cell partitions are merely logical functional partitions and may be other partitions in an actual implementation. For example, various elements or components may be combined or integrated in another system or portions of features may be omitted, or not implemented. Further, the shown or discussed mutual coupling or direct coupling or communicative connection may be achieved through some interfaces. Direct coupling or communicative connection between devices or units may be achieved through electrical, mechanical, or other means.

These functions may be stored in a computer-readable storage medium when they are implemented in the form of software functional units and sold or used as separate products. The solution according to the invention may be implemented substantially as such or as constituting part of the prior art or part of the solution according to the invention in the form of a software product. A computer software product is stored in a storage medium and includes instructions for instructing a computer device (which may be a personal computer, a server, or a network device) to perform all or part of the steps of the method described in the embodiments of the present invention. Any medium that can store program code, such as a USB disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

Devices that are in communication with each other need not be in continuous communication with each other, unless expressly specified otherwise. In addition, devices that are in communication with each other may communicate directly or indirectly through one or more intermediate devices.

Although a single device or article is described herein, it will be readily apparent that more than one device/article, whether or not they cooperate, may be used in place of a single device/article. Similarly, while more than one device or article of manufacture is described herein (whether or not they cooperate), it will be apparent that a single device/article of manufacture may be used in place of the more than one device or article of manufacture, or a different number of devices/articles of manufacture may be used than the number of devices or programs illustrated. The functionality and/or the features of a device may be alternatively embodied by one or more other devices which are not explicitly described as having such functionality/features. Thus, other embodiments of the present invention need not include the device itself.

Finally, the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. Accordingly, it is intended that the scope of the invention be limited not by this detailed description, but rather by any claims appended hereto as applied based on this disclosure. Accordingly, the disclosure of the embodiments of the invention is intended to be illustrative, but not limiting, of the scope of the invention, which is set forth in the following claims.

With respect to the use of any plural and/or singular terms herein, those having skill in the art may translate from the plural to the singular and/or from the singular to the plural as is appropriate to the context and/or application. For the sake of clarity, various singular/plural permutations may be expressly set forth herein.

Although implementations of the method and system for hybrid edge replication have been described in language specific to structural features and/or methods, it is to be understood that the appended claims are not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as examples of implementations of a hybrid edge replication protocol in kv-stores for edge computation scenarios.

Claims (58)

1. A method and apparatus for selectively copying data to a plurality of data nodes (12)₁、12₂、12₃、……、12_n) The method of (1), the method comprising:

a user equipment (10) sending (S101) a write request to write data at a first data node (11);

wherein the write request includes data and metadata, the metadata including an issue bit and an endpoint bit;

the first data node (11) determines (S102) whether to: is replicated to at least one data node (12)₁) Or copied to and group (G)₁) An associated set of data nodes (12)₁、12₃、12₄) Or to one or more groups (G)₁、G₂、……、G_n) All of the data nodes that are associated; or to one or more data nodes (12)₁、12₄) And group (G)₂)；

Wherein each data node (11, 12)₁、12₂、……、12_n) Including a group membership table;

the first data node (11) copying (S103) the data to one or more data nodes (12)₁、12₂、12₃、……、12_n)。

2. The method of claim 1, wherein the data node (12)₁、12₂、12₃、……、12_n) Is an edge node; and a group includes a plurality of data nodes.

3. The method of claim 1, further comprising the steps of:

the user equipment (10) writes data to the first data node (11).

4. The method of claim 1, further comprising the steps of:

the first data node (11) stores an operation log op _ num to record the data written by the user equipment (10) in the first data node (11);

the first data node (11) increments the commit number commit _ num in the first data node (11) by one count.

5. The method of claim 1, wherein each data node comprises an Op _ log comprising a number of edge views to determine the data node (12)₁、12₂、12₃、……、12_n) Whether located at the edge of the network.

6. The method of claim 1, wherein the first and second light sources are selected from the group consisting of a red light source, a green light source, and a blue light source,

wherein when the issue bit is y, the first data node (11) replicates and shares the data;

wherein when the issue bit is s, the first data node (11) does not replicate and share the data;

wherein the first data node (11) replicates the data when the issue bit is r; and is

Wherein the first data node (11) shares the data when the issue bit is s.

7. The method of claim 1, wherein the endpoint bits comprise at least one node ID, or at least one group ID, or a combination of node IDs and group IDs.

8. The method of claim 1, wherein the group membership table comprises a group (G)₁、G₂、……、G_n) At least one group ID of; and a data node (12) associated with the group₁、12₂、12₃、……、12_n) At least one node ID.

9. The method of any of the preceding claims, wherein when the issue bit is y and the endpoint bit comprises the data node (12)₁) The first data node (11) copies and shares the data to at least one data node (12) upon the node ID being associated therewith₁)。

10. Method according to any of the preceding claims, characterized in that when the issue bit is y and the end point bit comprises an AND group (G)₁) An associated set of data nodes (12)₁、12₃、12₄) The first data node (11) copies and shares the data to and with the group (G)₁) The associated set of data nodes (12)₁、12₃、12₄)。

11. The method according to any of the preceding claims, wherein when the issue bit is y and the endpoint bit comprises the group ID associated with the group, the first data node (11) copies and shares the data to and from one or more groups (G)₁、G₂、……、G_n) All of the data nodes associated.

12. The method of any of the preceding claims, wherein when the issue bit is y and the endpoint bit comprises the data node (12)₁、12₄) Associated one or more node IDs and a group (G)₂) The first data node (11) copies and shares the data to one or more data nodes (12) upon the group ID being associated therewith₁、12₄) And said group (G)₂)。

13. The method of claim 1, further comprising the steps of:

-the user equipment (10) registering a new group ID associated with a new group with the first data node (11);

the user equipment (10) registers one or more node IDs associated with one or more data nodes with the new group.

14. The method of claim 1, further comprising the steps of:

the user equipment (10) deregisters a group ID associated with a group from the first data node (11);

the user equipment (10) de-registers one or more node IDs associated with one or more data nodes associated with a group.

15. A method for selectively replicating data from a cloud network to a plurality of data nodes, the method comprising:

a user equipment (20) sending (S201) a write request to write data at a master node (21) of the cloud network;

wherein the write request includes data and metadata, the metadata including an issue bit and an endpoint bit;

the master node (21) determining (S202) whether to: is replicated to at least one data node (12)₁) Or copied to and group (G)₁) An associated set of data nodes (12)₁、12₃、12₄) Or to one or more groups (G)₁、G₂、……、G_n) All of the data nodes that are associated; or to one or more data nodes (12)₁、12₄) Andgroup (G)₂)；

Wherein each data node (11, 12)₁、12₂、……、12_n) Comprises a group membership table and the master node (21) comprises a client table;

the master node (21) replicating (S203) the data to one or more data nodes (12)₁、12₂、12₃、……、12_n)。

16. The method of claim 15, wherein the cloud network comprises at least one master node (21) and one or more cloud nodes; wherein the data node (12)₁、12₂、12₃、……、12_n) Is an edge node; and a group includes a plurality of data nodes.

17. The method of claim 15, further comprising the steps of: the user equipment (20) writes data to the master node (21).

18. The method of claim 15, further comprising the steps of: when no node ID is indicated in the endpoint data, the master node (21) determines the data node associated with a group by looking up the group membership table.

19. The method of claim 18, further comprising the steps of:

the master node (21) stores an operation log op _ num to record the data written by the master node (21) in a data node;

the master node (21) increments the commit number commit _ num in the data node by one count.

20. The method of claim 15, wherein each data node comprises Op _ log comprising a number of edge views to determine whether the data node is located at an edge of the network.

21. The method of claim 15, wherein the step of selecting the target,

wherein when the issue bit is y, the master node (21) copies and shares the data;

wherein when the release bit is s, the master node (21) does not copy and share the data;

wherein the master node (21) replicates the data when the issue bit is r; and is

Wherein the master node (21) shares the data when the issue bit is s.

22. The method of claim 15, wherein the endpoint bits comprise at least one node ID, or at least one group ID, or a combination of a node ID and a group ID.

23. The method of claim 15, wherein the group membership table comprises a group (G)₁、G₂、……、G_n) At least one group ID of; and a data node (12) associated with the group₁、12₂、12₃、……、12_n) At least one node ID.

24. The method of any of the preceding claims, wherein when the issue bit is y and the endpoint bit comprises the data node (12)₁) The master node (21) copies and shares the data to at least one data node (12) upon the node ID being associated therewith₁)。

25. Method according to any of the preceding claims, characterized in that when the issue bit is y and the end point bit comprises an AND group (G)₁) An associated set of data nodes (12)₁、12₃、12₄) The master node (21) copies and shares the data to and with the group (G)₁) The associated set of data nodes (12)₁、12₃、12₄)。

26. The method according to any one of the preceding claims, wherein when the issue bit is y and the endpoint bit comprises the group ID associated with the group, the master node (21) copies and shares the data to and with one or more groups (G)₁、G₂、……、G_n) All of the data nodes associated.

27. The method of any of the preceding claims, wherein when the issue bit is y and the endpoint bit comprises the data node (12)₁、12₄) Associated one or more node IDs and a group (G)₂) The master node (21) copies and shares the data to one or more data nodes (12) upon the group ID being associated therewith₁、12₄) And said group (G)₂)。

28. The method of claim 15, further comprising the steps of:

the user equipment (20) registering a new group ID associated with a new group with the data node;

the user equipment (20) registers one or more node IDs associated with one or more data nodes with the new group.

29. The method of claim 15, further comprising the steps of:

the user equipment (20) deregisters a group ID associated with a group from the data node;

the user equipment (20) de-registers one or more node IDs associated with one or more data nodes associated with a group.

30. A method and apparatus for selectively copying data to a plurality of data nodes (12)₁、12₂、12₃、……、12_n) The system of (a), the system comprising:

a user equipment (10) sending (S101) a write request for writing data at a first data node (11), characterized in that the write request comprises data and metadata, the metadata comprising an issue bit and an endpoint bit;

the first data node (11) determining (S102) whether to: is replicated to at least one data node (12)₁) Or copied to and group (G)₁) An associated set of data nodes (12)₁、12₃、12₄) Or to one or more groups (G)₁、G₂、……、G_n) All of the data nodes that are associated; or to one or more data nodes (12)₁、12₄) And group (G)₂) Characterized in that each data node (11, 12)₁、12₂、……、12_n) Including a group membership table; replicating (S103) the data to one or more data nodes (12)₁、12₂、12₃、……、12_n)。

31. The system of claim 30, wherein the data node (12)₁、12₂、12₃、……、12_n) Is an edge node; and a group includes a plurality of data nodes.

32. The system according to claim 30, characterized in that the user equipment (10) writes data to the first data node (11).

33. The system according to claim 30, characterized in that the first data node (11) stores an operation log op _ num to record the data written by the user equipment (10) in the first data node (11); -incrementing the commit number commit _ num in said first data node (11) by one count.

34. The system of claim 30, wherein each data nodeIncludes an Op _ log including a number of edge views to determine the data node (12)₁、12₂、12₃、……、12_n) Whether located at the edge of the network.

35. The system of claim 30, wherein the first and second sensors are arranged in a single unit,

wherein when the issue bit is y, the first data node (11) replicates and shares the data;

wherein when the issue bit is s, the first data node (11) does not replicate and share the data;

wherein the first data node (11) replicates the data when the issue bit is r;

wherein the first data node (11) shares the data when the issue bit is s.

36. The system of claim 30, wherein the endpoint bits comprise at least one node ID, or at least one group ID, or a combination of node IDs and group IDs.

37. The system of claim 30, wherein the group membership table comprises a group (G)₁、G₂、……、G_n) At least one group ID of; and a data node (12) associated with the group₁、12₂、12₃、……、12_n) At least one node ID.

38. The system of any of the preceding claims, wherein when the issue bit is y and the endpoint bit comprises the data node (12)₁) The first data node (11) copies and shares the data to at least one data node (12) upon the node ID being associated therewith₁)。

39. The system according to any of the preceding claims, wherein when the issue bit is y and the endpoint bit comprises an AND group (G)₁) Phase (C)An associated set of data nodes (12)₁、12₃、12₄) The first data node (11) copies and shares the data to and with the group (G)₁) The associated set of data nodes (12)₁、12₃、12₄)。

40. The system of any of the preceding claims, wherein the first data node (11) copies and shares the data to and from one or more groups (G) when the issue bit is y and the endpoint bit comprises the group ID associated with the group₁、G₂、……、G_n) All of the data nodes associated.

41. The system of any of the preceding claims, wherein when the issue bit is y and the endpoint bit comprises the data node (12)₁、12₄) Associated one or more node IDs and a group (G)₂) The first data node (11) copies and shares the data to one or more data nodes (12) upon the group ID being associated therewith₁、12₄) And said group (G)₂)。

42. The system of claim 30, characterized in that the user equipment (10) registers a new group ID associated with a new group with the first data node (11); registering one or more node IDs associated with one or more data nodes with the new group.

43. The system according to claim 30, wherein the user equipment (10) de-registers a group ID associated with a group from the first data node (11); deregisters one or more node IDs associated with one or more data nodes associated with a group.

44. A system for selectively replicating data from a cloud network to a plurality of data nodes, the system comprising:

a user equipment (20) sending (S201) a write request to write data at a master node (21) of the cloud network, characterized in that the write request comprises data and metadata, the metadata comprising an issue bit and an endpoint bit;

the master node (21), the master node determining (S202) whether to: is replicated to at least one data node (12)₁) Or copied to and group (G)₁) An associated set of data nodes (12)₁、12₃、12₄) Or to one or more groups (G)₁、G₂、……、G_n) All of the data nodes that are associated; or to one or more data nodes (12)₁、12₄) And group (G)₂) Characterized in that each data node (11, 12)₁、12₂、……、12_n) Comprises a group membership table and the master node (21) comprises a client table;

the master node (21) replicating (S203) the data to one or more data nodes (12)₁、12₂、12₃、……、12_n)。

45. The system of claim 44, wherein the cloud network comprises at least one master node (21) and one or more cloud nodes; wherein the data node (12)₁、12₂、12₃、……、12_n) Is an edge node; and a group includes a plurality of data nodes.

46. The system of claim 44, wherein the user device (20) writes data to the master node (21).

47. The system of claim 44, wherein the master node (21) determines the data node associated with a group by looking up the group membership table when no node ID is indicated in the endpoint bit.

48. The system according to claim 47, characterized in that the master node (21) stores an operation log op _ num to record the data written by the master node (21) in a data node; the commit number commit num is incremented by one count in the data node.

49. The system of claim 44, wherein each data node comprises Op _ log comprising a number of edge views to determine whether the data node is located at an edge of the network.

50. The system of claim 44, wherein said at least one sensor is a sensor,

wherein when the issue bit is y, the master node (21) copies and shares the data;

wherein when the release bit is s, the master node (21) does not copy and share the data;

wherein the master node (21) replicates the data when the issue bit is r;

wherein the master node (21) shares the data when the issue bit is s.

51. The system of claim 44, wherein the endpoint bits comprise at least one node ID, or at least one group ID, or a combination of node IDs and group IDs.

52. The system of claim 44, wherein the group membership table comprises a group (G)₁、G₂、……、G_n) At least one group ID of; and a data node (12) associated with the group₁、12₂、12₃、……、12_n) At least one node ID.

53. The system of any of the preceding claims, wherein when the issue bit is y and the endpoint bit comprises the data node (12)₁) The master node (21) copies and shares the data to at least one data node (12) upon the node ID being associated therewith₁)。

54. The system according to any of the preceding claims, wherein when the issue bit is y and the endpoint bit comprises an AND group (G)₁) An associated set of data nodes (12)₁、12₃、12₄) The master node (21) copies and shares the data to and with the group (G)₁) The associated set of data nodes (12)₁、12₃、12₄)。

55. The system according to any one of the preceding claims, wherein when said issue bit is y and said endpoint bit comprises said group ID associated with said group, said master node (21) copies and shares said data to and from one or more groups (G)₁、G₂、……、G_n) All of the data nodes associated.

56. The system of any of the preceding claims, wherein when the issue bit is y and the endpoint bit comprises the data node (12)₁、12₄) Associated one or more node IDs and a group (G)₂) The master node (21) copies and shares the data to one or more data nodes (12) upon the group ID being associated therewith₁、12₄) And said group (G)₂)。

57. The system of claim 44 wherein the user device (20) registers a new group ID associated with a new group with the data node; registering one or more node IDs associated with one or more data nodes with the new group.

58. The system of claim 44, wherein the user device (20) de-registers a group ID associated with a group from the data node; deregisters one or more node IDs associated with one or more data nodes associated with a group.

Images