CN115102968B - Data synchronization method, device, system and storage medium - Google Patents
Data synchronization method, device, system and storage medium Download PDFInfo
- Publication number
- CN115102968B CN115102968B CN202210682589.2A CN202210682589A CN115102968B CN 115102968 B CN115102968 B CN 115102968B CN 202210682589 A CN202210682589 A CN 202210682589A CN 115102968 B CN115102968 B CN 115102968B
- Authority
- CN
- China
- Prior art keywords
- message
- data
- synchronous
- messages
- synchronization
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 59
- 230000001360 synchronised effect Effects 0.000 claims abstract description 209
- 230000005540 biological transmission Effects 0.000 claims abstract description 20
- 238000012986 modification Methods 0.000 claims description 23
- 230000004048 modification Effects 0.000 claims description 23
- 238000004590 computer program Methods 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 15
- 238000012217 deletion Methods 0.000 claims description 5
- 230000037430 deletion Effects 0.000 claims description 5
- 230000000694 effects Effects 0.000 claims description 2
- 238000004891 communication Methods 0.000 abstract description 5
- 230000009286 beneficial effect Effects 0.000 abstract description 3
- 230000003993 interaction Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000012546 transfer Methods 0.000 description 3
- 230000002457 bidirectional effect Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 230000002045 lasting effect Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
- H04L67/141—Setup of application sessions
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to the field of communication, and provides a data synchronization method, a device, a system and a storage medium. The method comprises the following steps: the first device and/or the second device send a synchronous message to the message pool, wherein the synchronous message comprises a data identifier; when the second device and/or the first device get the synchronization message from the message Chi La, the second device obtains the data identifier in the synchronization message; and the second equipment determines data to be synchronized which need to be synchronized through data identification comparison, and establishes a data transmission link between the first equipment and the second equipment to transmit the data to be synchronized. Compared with a synchronous mode based on an http protocol, the method and the device can simplify the flow of data synchronization between the first device and the second device, can complete synchronization of data such as a plurality of pictures at one time, and are beneficial to improving data synchronization efficiency.
Description
Technical Field
The present application belongs to the field of communications, and in particular, relates to a data synchronization method, apparatus, system, and storage medium.
Background
Data synchronization is a method of achieving data consistency between two or more devices. By means of data synchronization, two or more devices can be enabled to possess the same data, including the same pictures, videos and the like, so that the same data processing or displaying can be achieved on the two or more devices conveniently.
Currently, for a synchronization mode containing larger data (including pictures, face feature values, audio, etc.), the synchronization mode is mainly implemented based on an http (chinese full called hypertext transfer protocol, english full called Hyper Text Transfer Protocol) protocol. Namely, the server provides url (English is called Universal Resource Locator fully, chinese is called uniform resource locator) for downloading the picture, and the client initiates an http request to download the picture through the url. The synchronous mode has more interaction times, and one picture can be usually synchronized only by one interaction, so that the synchronization efficiency is low.
Disclosure of Invention
In view of the above, the embodiments of the present application provide a data synchronization method, apparatus and system, and a storage medium, so as to solve the problem in the prior art that when data synchronization is performed, the number of interactions is more, synchronization efficiency is lower, and synchronization errors are easy to occur.
A first aspect of an embodiment of the present application provides a data synchronization method, where the data synchronization method is used to synchronize a first device and a second device, where the first device and the second device are connected by a message pool, and the method includes:
The first device and/or the second device send a synchronous message to the message pool, wherein the synchronous message comprises a data identifier;
When the second device and/or the first device get the synchronization message from the message Chi La, the second device obtains the data identifier in the synchronization message;
and the second equipment determines data to be synchronized which need to be synchronized through data identification comparison, and establishes a data transmission link between the first equipment and the second equipment to transmit the data to be synchronized.
With reference to the first aspect, in a first possible implementation manner of the first aspect, the synchronization message further includes a message type, after the first device and/or the second device send the synchronization message to the message pool, before the second device and/or the first device fetches the synchronization message to the message Chi La, the method further includes:
And the message pool carries out merging processing on the synchronous messages according to the message type.
With reference to the first possible implementation manner of the first aspect, in a second possible implementation manner of the first aspect, the message type includes a record type and a sub-record type under a directory of the record type;
the message pool carries out merging processing on the synchronous messages according to the message types, and the method comprises the following steps:
When the synchronous message received by the message pool comprises synchronous messages with the same record type and record subtype, combining two or more synchronous messages with the same record type and record subtype into one synchronous message, and adding the data identification of the received synchronous message into the combined synchronous message.
With reference to the first possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, the message type includes an operation type, where the operation type includes an add operation, a modify operation, and a delete operation;
And combining the synchronous messages according to the message types, wherein the combining comprises the following steps:
When the synchronous messages received by the message pool comprise the same data identification and the message types are operation types, determining effective synchronous messages according to the influence of two or more operation types on the data.
With reference to the third possible implementation manner of the first aspect, in a fourth possible implementation manner of the first aspect, when a synchronization message received by a message pool includes the same data identifier and a message type is an operation type, determining an effective synchronization message according to an effect of two or more operation types on the data includes:
When the message types of the synchronous messages with the same data identification are the adding operation and the deleting operation and are all the same-direction messages, the synchronous messages with the attribute of the adding operation are emptied, the synchronous messages with the attribute of the deleting operation are not saved, or the synchronous messages with the attribute of the deleting operation are emptied, and the synchronous messages with the attribute of the adding operation are not saved;
The message types of the synchronous messages with the same data identification are the adding operation and the modifying operation in sequence and are all the same-direction messages, the content of the synchronous message with the attribute of the modifying operation is added to the synchronous message with the same data identification and the attribute of the adding operation, and the synchronous message with the attribute of the adding operation is not saved;
The message types of the synchronous messages with the same data identification are modification operation and deletion operation in sequence, the deletion attribute is the synchronous message of the modification operation, and the storage attribute is the synchronous message of the deletion operation;
The message types of the synchronous messages with the same data identification are all modification operations and are all homodromous messages, the message contents of the synchronous messages with the same data identification are combined, the combined message contents are modified into the synchronous messages with the stored attribute of modification operations, and the newly received synchronous messages are not saved;
and if the message types of the synchronous messages with the same data identifiers are modification operations and are different-direction messages, merging the message contents of the synchronous messages with the same data identifiers, and respectively storing the merged message contents into the different-direction synchronous messages.
With reference to the first possible implementation manner of the first aspect, in a fifth possible implementation manner of the first aspect, the message type includes a control type, and the control type includes full-volume update and restart message synchronization;
And combining the synchronous messages according to the message types, wherein the combining comprises the following steps:
when receiving the synchronous message with the attribute of full quantity update, emptying the synchronous message in the message pool, not receiving the new synchronous message, and controlling the message pool to enter a full quantity synchronous state;
When the full volume update of the message pool is completed and a synchronization message with the attribute of restarting is received, the synchronization message with the attribute of full volume synchronization is cleared and new incremental messages are re-allowed to be received.
With reference to the first aspect, the first possible implementation manner of the first aspect, the second possible implementation manner of the first aspect, the third possible implementation manner of the first aspect, the fourth possible implementation manner of the first aspect, or the fifth possible implementation manner of the first aspect, in a sixth possible implementation manner of the first aspect, the establishing a data transmission link between the first device and the second device, and transmitting the data to be synchronized includes:
and establishing a data transmission link between the first equipment and the second equipment through a private protocol, and actively pushing the data to be synchronized through the private protocol.
With reference to the first aspect, in a seventh possible implementation manner of the first aspect, the data to be synchronized includes one or more segments of data streams, where the data streams include a plurality of target objects, and the number of target objects in the data streams is the same as the number of target objects in the data to be synchronized.
A second aspect of an embodiment of the present application provides a data synchronization method, where the data synchronization method is used to synchronize a first device and a second device, where the first device and the second device are connected by a message pool, and the method includes:
receiving a synchronous message sent by first equipment and/or second equipment, wherein the synchronous message comprises a data identifier;
When a synchronization request of pulling the synchronization message by the second device is received, sending a data identifier in the received synchronization message to the second device, wherein the data identifier is used for comparing the data identifier of the second device and/or the data identifier of the first device with the data identifier of the synchronization message in a message pool, determining data to be synchronized, which need to be synchronized, and establishing a data transmission link of the first device and the second device to transmit the data to be synchronized.
A third aspect of an embodiment of the present application provides a data synchronization apparatus, where the data synchronization apparatus is configured to synchronize a first device and a second device, where the first device and the second device are connected by a message pool, and the apparatus includes:
A synchronous message sending unit, configured to send a synchronous message to the message pool by the first device and/or the second device, where the synchronous message includes a data identifier;
A data identifier obtaining unit, configured to obtain, when the second device and/or the first device obtain the synchronization message from the message Chi La, a data identifier in the synchronization message;
The data synchronization unit is used for comparing the data identification of the second equipment and/or the first equipment with the data identification of the synchronous message in the message pool, determining the data to be synchronized which needs to be synchronized, establishing a data transmission link between the first equipment and the second equipment, and transmitting the data to be synchronized.
A third aspect of an embodiment of the present application provides a data synchronization system comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the method according to any one of the first aspects when the computer program is executed.
A fourth aspect of the embodiments of the present application provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the steps of the method according to any of the first aspects.
Compared with the prior art, the embodiment of the application has the beneficial effects that: according to the embodiment of the application, the data identification of the data to be synchronized of the first equipment is received through the message pool, the synchronization message is pulled at the second equipment, and the data identification is sent to the second equipment, so that the second equipment determines the data to be synchronized which needs to be synchronized according to the comparison of the data identification, and the data to be synchronized is transmitted by establishing the data transmission link between the first equipment and the second equipment.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of an implementation scenario of a data synchronization method according to an embodiment of the present application;
fig. 2 is a schematic diagram of an implementation flow of a data synchronization method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a data synchronization device according to an embodiment of the present application;
fig. 4 is a schematic diagram of a data synchronization system according to an embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
In order to illustrate the technical scheme of the application, the following description is made by specific examples.
Fig. 1 is a schematic diagram of an implementation scenario of a data synchronization method according to an embodiment of the present application. As shown in fig. 1, the implementation scenario includes a first device, a second device, and a message pool. The first device and the second device have data receiving and transmitting capability and data storage capability. The message pool is not limited to a specific form, and may be a server which exists independently, or may be provided in any device for synchronization. The message pool can be used as a transfer station of data information to assist the first equipment and the second equipment to perform data synchronization.
In the embodiment of the application, the first device and the second device for synchronization can be used for bidirectional synchronization, namely, the data synchronization from the first device to the second device and the data synchronization from the second device to the first device can be realized. Based on the bidirectional synchronization protocol, data synchronization between the first device and the second device is achieved. The second device to first device data synchronization is substantially the same as the first device to second device data synchronization flow.
In a possible implementation manner of the present application, an implementation scenario in which a system performs data synchronization with a plurality of devices may also be included. I.e. one system (first device) connects multiple devices (second device), one device can only connect to a single system. The single system is bound with a plurality of devices, so that the problem of data confusion caused by accessing a plurality of systems can be effectively avoided. When the equipment is accessed to the system, the two negotiates to determine the IP address of the message pool, the message theme, the transmission protocol and the like, and the equipment can be synchronized once in full. After that, no matter the modification is performed at the system end or the modification is performed at the equipment end, the incremental synchronous message can be sent to the message pool according to the message format specified by the scheme, and the message pool can perform certain merging processing on the incremental synchronous message. When the device side or the system side pulls the increment message, the data identification of the data to be synchronized is obtained from the message pool, and is compared with the data identification of the local data, the data identification to be synchronized, which needs to be synchronized, is determined, and a data link for synchronizing the data is established for data synchronization.
Fig. 2 is a schematic implementation flow chart of a data synchronization method according to an embodiment of the present application, where the synchronization method is used for a first device and a second device shown in fig. 1, where the first device and the second device are connected by a message pool, and as shown in fig. 2, the method includes:
In S201, the first device and/or the second device send a synchronization message to the message pool, where the synchronization message includes a data identifier.
In the embodiment of the application, according to the first device and the second device which are connected by the message pool, the first device can send synchronous data to the second device, and the second device can also send synchronous data to the first device. The first device or the second device may send two or more synchronization messages to the message pool. If the sender of two or more synchronous messages is the same, it means that two or more synchronous messages are the same-direction messages. For example, two or more synchronous messages are sent by the first device to the message pool, and the two or more synchronous messages are the same-direction messages. Accordingly, if the sender of two synchronization messages is different, for example, one of the two synchronization messages is sent by the first device and the other one is sent by the second device, it means that the two synchronization messages are an outgoing synchronization message or an outgoing synchronization message.
When the first device and/or the second device send the synchronization message to the message pool, for example, when the first device and/or the second device access the data synchronization system, or incremental data appears in the first device and/or the second device, for example, pictures and face features are added in the first device and/or the second device to wait for the synchronization data.
Before the first device and/or the second device synchronize the data, marking the data to be synchronized in the first device and/or the second device, and determining the data identification of the data to be synchronized in the first device and/or the second device. The data identification is used to globally uniquely identify the data. The data identification can be determined through encryption algorithms such as hash calculation and the like.
In the embodiment of the application, when the first device and/or the second device synchronize the data to other devices, the data identification of the data to be synchronized can be sent first, so that the message pool can acquire one or more data identifications of the data to be synchronized with the first device and/or the second device through less data volume.
In S202, when the second device and/or the first device fetches the synchronization message from the message Chi La, the data identifier in the synchronization message is fetched.
When the message pool is connected with the first device and the second device, the synchronous message received by the message pool can comprise the synchronous message sent by the first device, the synchronous message sent by the second device or the synchronous message sent by the first device and the second device. When the synchronization message is pulled, the first device pulling the synchronization message, the second device pulling the synchronization message, or the first device and the second device pulling the synchronization message may be included.
The pool of messages may merge synchronization messages based on the type of message received before the second device and/or the first device fetches synchronization messages for the message Chi La. For example, several or tens of synchronous messages can be combined into one synchronous message, when the second device and/or the first device pull the synchronous message, unnecessary redundant messages in a message pool can be reduced, and the workload of the second device for synchronization is reduced.
The message pool internal structure can adopt a Reactor mode, and comprises a receiving thread group (Acceptor) and a processing thread group (Handler). The receiving thread group is used for monitoring network connection and generating a unique message identification (MSGID) of the received message, and then the received message is sent to the processing thread group for subsequent processing, including merging the identified synchronous messages, maintaining a message list, lasting synchronous messages and the like.
When the synchronous messages are combined, the synchronous messages can be combined according to the types of the messages, so that redundant synchronous messages in a message pool are eliminated as much as possible, and the situation of modifying the same attribute for many times is prevented. Thus, the type of synchronization message may be determined prior to the synchronization message merge. The type of the synchronization message may be determined according to a predetermined correspondence between the synchronization message and the type. The determined type of synchronization message may include a record class, an operation class, a control class, and the like.
For each type of synchronization message, a primary message type (fixed) and a secondary message type (optional) may be included. Wherein the primary message type may be fixed by a protocol, and each system or device may only use the primary message type specified by the protocol. The secondary message type can be designed to provide expansion and adaptation for each system or device, and the system or device can define own message content. Thus, the merging of synchronization messages may be based on the primary message type.
The record type message may include a first-level message type record, which may represent all the invariable records including a face snapshot record, an attendance record, a vehicle entry record, etc.
The operation type message includes three kinds of messages for variable data, including adding, modifying and deleting, which respectively represent operations for executing the new adding, modifying and deleting of the data.
The control type message is more aimed at the message synchronization flow, and the plan includes primary message types of full-volume synchronization, stop synchronization, restart, connection and the like, which respectively indicate the meanings of needing to perform full-volume synchronization, stop message synchronization, restart message synchronization and negotiate connection modes on data.
In embodiments of the present application, a field representing the direction of propagation of the message may be included in the message to distinguish from which device the message is delivered to which device. Or may also include a message topic field that can be used to identify which topic the message belongs to.
When the embodiment of the application combines the synchronous messages, a quick look-up table can be established based on the Key value pair mapping table, the message type and the theme are used as a first layer Key (Key), and for the record type and the control type messages, the value of the first layer is the actual value. For operation type information, a unique information mark can be used as a second layer key, and the value of the second layer key is the increment data which is to be effective finally; after the processing thread in the message pool receives the messages, the processing thread scans the quick list, combines the messages conforming to the combination, and updates the quick list.
The message merging in the embodiment of the application can comprise merging of the synchronous messages of the record type, merging of the synchronous messages of the operation type and merging of the messages of the control type. Wherein:
1. Merging of record class messages.
And when the primary message type is record and the subject of the synchronous message is the same and the secondary message type is sub-record type under the record type, the two synchronous messages can be considered to be combined, and the combined result is that one record message is added in the value of the item and used for recording the data identification needing to be synchronized, and the batch is synchronized at one time later.
2. Merging operation type messages.
2.1 Add-delete type, which only occurs in the same direction message, if the first level message is deleted and the operation type message with the same data identifier exists, the combination result is to empty the existing add message and the newly added delete message is not saved.
2.2, Adding-modifying type, which only appears in the same-direction message, if the type of the first-level message is modification and there is an adding message with the same data identification, combining the two, and as a result, modifying the field attribute in the adding message, changing it into the attribute of modified content, and the newly added modified message is not saved.
2.3, Modify-delete type, which message type is not limited in direction, and is applicable to the following operations: if a delete type message is received and there is a modification message identified with the data, the merged structure is to delete the "modification" message and retain the "delete" message.
2.4, Modifying-modifying, when receiving a message of modifying type, namely modifying the message, if there is the same data identification, the same direction modifying message, combine the message content of the two, modify to the existing "modifying" message, the new "modifying" message will not keep; however, if there are different "modification" messages with the same data identifier, the two message contents need to be combined, and then the combined message contents are respectively stored in the two different synchronous messages.
3. Merging control class messages.
And 3.1, the full-quantity updating type, when receiving the message of the full-quantity updating type, directly clearing all the quick list tables, and entering a state of waiting for full-quantity synchronization without receiving a new message.
The restart type, when the full synchronization is completed, the party receiving the data is responsible for sending a message of the "restart" type to the message pool, at which time the message of the "full update" type can be eliminated and new incremental messages can be re-allowed to be received.
In S203, the data identifier of the second device and/or the first device is compared with the data identifier of the synchronization message in the message pool, so as to determine the data to be synchronized that needs to be synchronized, and a data transmission link between the first device and the second device is established to transmit the data to be synchronized.
And when the second device and/or the first device performs data pulling, the data processed by the message pool can be obtained. The message pool can combine the received synchronous messages, so that redundant synchronous messages can be effectively removed, the interaction times of data synchronization are greatly reduced, the flow is shorter, and the error probability is lower. And the application can match and transmit data based on private protocol, and can improve the security of data synchronization. According to the method and the device for synchronizing the data, the synchronous information and the synchronous data are processed separately, and the second device and/or the first device can acquire a plurality of data identifiers to be synchronized at one time when pulling the data, so that the second device and/or the first device can acquire a plurality of synchronous data through a data transmission link, the method and the device are beneficial to simplifying the synchronous flow and improving the data synchronization efficiency.
The data link established by the first device and the second device may be a direct data link or an indirect data link. The data link can establish a data transmission link based on a private protocol, and the device needing synchronization can actively push data based on the data transmission link. According to the private protocol, the synchronization operation is enabled to complete the synchronization of various types of data (including pictures, video, audio and the like) in one interaction. But are not limited thereto, and may include data links established by other communication protocols. When an indirect data link is established, data requiring synchronization can be forwarded through the message pool.
The proprietary protocol can be privately expanded based on the RTSP protocol, and compared with the current http protocol interaction mode, the proprietary protocol has the advantages of fewer interaction times, shorter flow and lower error probability. The risk of being deciphered is relatively smaller and the security is higher based on the proprietary protocol compared with the RFC standard protocol. By adopting the format of intelligent data, multiple pieces of data of different types, such as pictures, videos, audios and the like, can be intelligently identified and transmitted, and attribute information corresponding to the data type can be determined based on the identified data type, so that the data transmission requirements of different types are customized and met. The same system relates to NVR, IPC and other devices, and the system can better keep the integrity by utilizing a private protocol.
In addition, in the data stream of the private protocol in the embodiment of the present application, multiple target objects (such as different types of target objects or multiple target objects of the same type) may be transmitted on the top layer of the data stream, that is, a section of the data stream includes a data stream header and one or more target objects, where the number of target objects in the data stream is the same as the number of target objects in the data to be synchronized, so as to reduce the number of interactions when multiple target objects are transmitted. The format of the private protocol can be TLV (type-length-value, chinese) format, so that the private protocol has good expansibility and is convenient for adding the attribute of the target object. That is, the length identification of the TLV format can define the data length of the information such as the data stream, the target object, the attribute and the like, and based on the defined data length, the specific information of the data stream, the target object or the attribute can be conveniently and completely determined. And when the specific information is changed, the defined data length is correspondingly adjusted. Based on the private protocol format, the requirement of multi-section data synchronization can be effectively met, and the data interaction efficiency is improved.
For directly established data links, including data links based on HTTP, FTP protocols, etc. The data link may be established based on a message pool, i.e. indirectly through the message pool, without being limited thereto. Or the application can also calculate the data identification based on the message pool, compare the data identifications through the message pool, determine the synchronous data which needs to be sent to the second equipment, and combine the synchronous messages through the message pool.
It can be understood that when the execution body is a message pool, the synchronization method can receive a synchronization message sent by the first device and/or the second device through the message pool, where the synchronization message includes a data identifier; when a synchronization request of pulling the synchronization message by the second device and/or the first device is received, comparing the data identification of the second device and/or the first device with the data identification of the synchronization message in the message pool, and determining the data to be synchronized which needs to be synchronized so as to establish a data transmission link of the first device and the second device and transmit the data to be synchronized. This synchronization method corresponds to the synchronization method shown in fig. 2.
The data synchronized in the embodiment of the application can comprise data such as photos, facial features, fingerprint features and the like.
It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present application.
Fig. 3 is a schematic structural diagram of a data synchronization device according to an embodiment of the present application, as shown in fig. 3, the device includes:
a synchronous message sending unit 301, configured to send, by the first device and/or the second device, a synchronous message to the message pool, where the synchronous message includes a data identifier;
A data identifier obtaining unit 302, configured to obtain, when the second device and/or the first device obtain the synchronization message from the message Chi La, a data identifier in the synchronization message;
And the data synchronization unit 303 is configured to compare the data identifier of the second device and/or the data identifier of the first device with the data identifier of the synchronization message in the message pool, determine data to be synchronized that needs to be synchronized, establish a data transmission link between the first device and the second device, and transmit the data to be synchronized.
The data synchronization apparatus shown in fig. 3 corresponds to the data synchronization method shown in fig. 2.
Fig. 4 is a schematic diagram of a data synchronization system according to an embodiment of the present application. As shown in fig. 4, the data synchronization system 4 of this embodiment includes: a processor 40, a memory 41 and a computer program 42, such as a data synchronization program, stored in the memory 41 and executable on the processor 40. The steps of the various data synchronization method embodiments described above are implemented by the processor 40 when executing the computer program 42. Or the processor 40, when executing the computer program 42, performs the functions of the modules/units of the apparatus embodiments described above.
Illustratively, the computer program 42 may be partitioned into one or more modules/units that are stored in the memory 41 and executed by the processor 40 to complete the present application. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions for describing the execution of the computer program 42 in the data synchronization system 4.
The data synchronization system may include, but is not limited to, a processor 40, a memory 41. It will be appreciated by those skilled in the art that fig. 4 is merely an example of the data synchronization system 4 and does not constitute a limitation of the data synchronization system 4, and may include more or fewer components than shown, or may combine certain components, or different components, e.g., the data synchronization system may further include input and output devices, network access devices, buses, etc.
The Processor 40 may be a central processing unit (Central Processing Unit, CPU), other general purpose Processor, digital signal Processor (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 41 may be an internal storage unit of the data synchronization system 4, such as a hard disk or a memory of the data synchronization system 4. The memory 41 may also be an external storage device of the data synchronization system 4, such as a plug-in hard disk, a smart memory card (SMART MEDIA CARD, SMC), a Secure Digital (SD) card, a flash memory card (FLASH CARD) or the like, which are provided on the data synchronization system 4. Further, the memory 41 may also include both an internal storage unit and an external storage device of the data synchronization system 4. The memory 41 is used for storing the computer program as well as other programs and data required by the data synchronization system. The memory 41 may also be used for temporarily storing data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, the specific names of the functional units and modules are only for distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other manners. For example, the apparatus/terminal device embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical function division, and there may be additional divisions in actual implementation, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated modules/units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on this understanding, the present application may also be implemented by implementing all or part of the procedures in the methods of the above embodiments, and the computer program may be stored in a computer readable storage medium, where the computer program when executed by a processor may implement the steps of the respective method embodiments. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth. It should be noted that the computer readable medium may include content that is subject to appropriate increases and decreases as required by jurisdictions in which such content is subject to legislation and patent practice, such as in certain jurisdictions in which such content is not included as electrical carrier signals and telecommunication signals.
The above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application.
Claims (12)
1. A data synchronization method, wherein the data synchronization method is used for synchronizing a first device and a second device, the first device and the second device are connected through a message pool, the method comprises:
when incremental data occurs in the first device and/or the second device, the first device and/or the second device sends a synchronization message to the message pool, wherein the synchronization message comprises a data identifier, and the data identifier is used for globally unique identification data;
The message types of the synchronous messages with the same data identification are all modification operations and are all synchronous messages, the message contents of the synchronous messages with the same data identification are combined, the combined message contents are modified into the stored synchronous messages with the modification operations, the newly received synchronous messages are not saved, and the synchronous messages with the same synchronous message types are synchronous messages with the same sender;
The message types of the synchronous messages with the same data identification are modification operation and are different-direction messages, the message contents of the synchronous messages with the same data identification are combined, the combined message contents are respectively stored into the different-direction synchronous messages, and the different-direction messages are synchronous messages of different senders;
when the second device and/or the first device get the synchronization message from the message Chi La, the second device and/or the first device obtain a data identifier in the synchronization message in the message pool;
Comparing the data identification of the second device and/or the first device with the data identification of the synchronous message in the message pool, determining the data to be synchronized which needs to be synchronized, establishing a data transmission link between the first device and the second device, and transmitting the data to be synchronized.
2. The method of claim 1, wherein the synchronization message further comprises a message type, and wherein after the first device and/or second device sends the synchronization message to the message pool, before the second device and/or first device fetches the synchronization message to the message Chi La, the method further comprises:
And the message pool carries out merging processing on the synchronous messages according to the message type.
3. The method of claim 2, wherein the message type comprises a record type, and a sub-record type under a directory of the record type;
the message pool carries out merging processing on the synchronous messages according to the message types, and the method comprises the following steps:
When the synchronous message received by the message pool comprises synchronous messages with the same record type and record subtype, combining two or more synchronous messages with the same record type and record subtype into one synchronous message, and adding the data identification of the received synchronous message into the combined synchronous message.
4. The method of claim 2, wherein the message type comprises an operation type, the operation type comprising an add operation, a modify operation, and a delete operation;
And combining the synchronous messages according to the message types, wherein the combining comprises the following steps:
When the synchronous messages received by the message pool comprise the same data identification and the message types are operation types, determining effective synchronous messages according to the influence of two or more operation types on the data.
5. The method of claim 4, wherein when the synchronization messages received by the message pool include the same data identifier and the message type is an operation type, determining a valid synchronization message based on the effect of two or more operation types on the data, comprises:
When the message types of the synchronous messages with the same data identification are the adding operation and the deleting operation and are all the same-direction messages, the synchronous messages with the attribute of the adding operation are emptied, the synchronous messages with the attribute of the deleting operation are not saved, or the synchronous messages with the attribute of the deleting operation are emptied, and the synchronous messages with the attribute of the adding operation are not saved;
The message types of the synchronous messages with the same data identification are the adding operation and the modifying operation in sequence and are all the same-direction messages, the content of the synchronous message with the attribute of the modifying operation is added to the synchronous message with the same data identification and the attribute of the adding operation, and the synchronous message with the attribute of the adding operation is not saved;
and when the message types of the synchronous messages with the same data identification are the modification operation and the deletion operation in sequence, deleting the synchronous message with the attribute of the modification operation, and storing the synchronous message with the attribute of the deletion operation.
6. The method of claim 2, wherein the message type comprises a control type, the control type comprising full volume update and restart message synchronization;
And combining the synchronous messages according to the message types, wherein the combining comprises the following steps:
when receiving the synchronous message with the attribute of full quantity update, emptying the synchronous message in the message pool, not receiving the new synchronous message, and controlling the message pool to enter a full quantity synchronous state;
When the full volume update of the message pool is completed and a synchronization message with the attribute of restarting is received, the synchronization message with the attribute of full volume synchronization is cleared and new incremental messages are re-allowed to be received.
7. The method according to any of claims 1-6, wherein establishing a data transmission link between the first device and the second device, transmitting the data to be synchronized, comprises:
and establishing a data transmission link between the first equipment and the second equipment through a private protocol, and actively pushing the data to be synchronized through the private protocol.
8. The method of claim 7, wherein the data to be synchronized comprises one or more data streams, the data streams comprising a plurality of target objects, the number of target objects in the data streams being the same as the number of target objects in the data to be synchronized.
9. A data synchronization method, wherein the data synchronization method is used for synchronizing a first device and a second device, the first device and the second device are connected through a message pool, the method comprises:
When incremental data appear in the first equipment and/or the second equipment, receiving a synchronous message sent by the first equipment and/or the second equipment, wherein the synchronous message comprises a data identifier, and the data identifier is used for globally uniquely identifying data;
The message types of the synchronous messages with the same data identification are all modification operations and are all synchronous messages, the message contents of the synchronous messages with the same data identification are combined, the combined message contents are modified into the stored synchronous messages with the modification operations, the newly received synchronous messages are not saved, and the synchronous messages with the same synchronous message types are synchronous messages with the same sender;
The message types of the synchronous messages with the same data identification are modification operation and are different-direction messages, the message contents of the synchronous messages with the same data identification are combined, the combined message contents are respectively stored into the different-direction synchronous messages, and the different-direction messages are synchronous messages of different senders;
When a synchronization request of pulling the synchronization message by the second device is received, comparing the data identification of the second device and/or the data identification of the first device with the data identification of the synchronization message in the message pool, and determining the data to be synchronized which needs to be synchronized so as to establish a data transmission link of the first device and the second device and transmit the data to be synchronized.
10. A data synchronization apparatus for synchronizing a first device and a second device, the first device and the second device being connected by a message pool, the apparatus comprising:
A synchronous message sending unit, configured to send, when incremental data occurs in the first device and/or the second device, a synchronous message of data to be synchronized to the message pool by the first device and/or the second device, where the synchronous message includes a data identifier, the data identifier is used for globally uniquely identifying data, message types of synchronous messages with the same data identifier are all modification operations and are all same-direction messages, and then merge message contents of synchronous messages with the same data identifier, and modify the merged message contents into a synchronous message with a stored attribute being the modification operation, where the newly received synchronous message is not saved, and the same-direction message is a synchronous message with the same sender; the message types of the synchronous messages with the same data identification are modification operation and are different-direction messages, the message contents of the synchronous messages with the same data identification are combined, the combined message contents are respectively stored into the different-direction synchronous messages, and the different-direction messages are synchronous messages of different senders;
A data identifier obtaining unit, configured to obtain, when the second device and/or the first device obtain a synchronization message in the message pool from the message Chi La, a data identifier in the synchronization message;
The data synchronization unit is used for comparing the data identification of the second equipment and/or the first equipment with the data identification of the synchronous message in the message pool, determining the data to be synchronized which needs to be synchronized, establishing a data transmission link between the first equipment and the second equipment, and transmitting the data to be synchronized.
11. A data synchronization system comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any one of claims 1 to 8 when the computer program is executed by the processor.
12. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210682589.2A CN115102968B (en) | 2022-06-16 | 2022-06-16 | Data synchronization method, device, system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210682589.2A CN115102968B (en) | 2022-06-16 | 2022-06-16 | Data synchronization method, device, system and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115102968A CN115102968A (en) | 2022-09-23 |
| CN115102968B true CN115102968B (en) | 2024-08-30 |
Family
ID=83290838
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210682589.2A Active CN115102968B (en) | 2022-06-16 | 2022-06-16 | Data synchronization method, device, system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115102968B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115776494A (en) * | 2022-11-04 | 2023-03-10 | 深圳市欧瑞博科技股份有限公司 | Node data efficient synchronization method and device, computer equipment and storage medium |
| CN116150266A (en) * | 2022-12-16 | 2023-05-23 | 中国联合网络通信集团有限公司 | A data synchronization method and device |
| CN119728708A (en) * | 2023-09-27 | 2025-03-28 | 华为技术有限公司 | Method, electronic device and system for transmitting data |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104168335A (en) * | 2014-09-02 | 2014-11-26 | 东软熙康健康科技有限公司 | Data synchronization method and device |
| CN104184756A (en) * | 2013-05-21 | 2014-12-03 | 阿里巴巴集团控股有限公司 | Data synchronization method, device and system |
| CN112612799A (en) * | 2020-12-08 | 2021-04-06 | 福建天泉教育科技有限公司 | Data synchronization method and terminal |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101430517B1 (en) * | 2008-01-31 | 2014-08-19 | 삼성전자주식회사 | A method of data synchronization between a plurality of data communication apparatuses |
| CN105187475B (en) * | 2015-06-09 | 2018-10-12 | 深圳金蝶账无忧网络科技有限公司 | The method and system that data synchronize |
| CN110493298B (en) * | 2018-05-15 | 2021-10-08 | 阿里巴巴（中国）有限公司 | Information synchronization method and device |
| US11201918B2 (en) * | 2020-03-03 | 2021-12-14 | Snap Inc. | Minimizing number of synchs |
| CN113868228A (en) * | 2021-09-29 | 2021-12-31 | 北京沃东天骏信息技术有限公司 | Data synchronization method, device, equipment and computer readable storage medium |
-
2022
- 2022-06-16 CN CN202210682589.2A patent/CN115102968B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104184756A (en) * | 2013-05-21 | 2014-12-03 | 阿里巴巴集团控股有限公司 | Data synchronization method, device and system |
| CN104168335A (en) * | 2014-09-02 | 2014-11-26 | 东软熙康健康科技有限公司 | Data synchronization method and device |
| CN112612799A (en) * | 2020-12-08 | 2021-04-06 | 福建天泉教育科技有限公司 | Data synchronization method and terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115102968A (en) | 2022-09-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115102968B (en) | Data synchronization method, device, system and storage medium | |
| CN101217431B (en) | A method and system of photos in synchronous mobile terminal network TV and network album | |
| CN109951546B (en) | Transaction request processing method, device, equipment and medium based on intelligent contract | |
| CN108710681A (en) | File acquisition method, device, equipment and storage medium | |
| CN101820439A (en) | Display method, device and client for files | |
| US20090119388A1 (en) | Content relaying device and content relaying method | |
| CN113709250B (en) | Cross-domain user data synchronization method based on subscription transmission mode | |
| CN108494875A (en) | A kind of method and apparatus of feedback resources file | |
| US5490088A (en) | Method of handling data retrieval requests | |
| US8745101B2 (en) | Terminal and method for identifying contents | |
| CN103327026B (en) | A kind of data-updating method and the system of renewal | |
| CN113239121B (en) | Data synchronization method and device | |
| CN113379542B (en) | Block chain transaction query method, device, medium and electronic equipment | |
| CN117041321B (en) | Service processing system and method | |
| CN119276896A (en) | Medical Internet of Things data processing system and method based on edge container | |
| JPH10289172A (en) | Data transfer system | |
| CN113553206B (en) | Data event execution method and device, electronic equipment and computer readable medium | |
| CN113076380B (en) | Data synchronization method, device, system, equipment and storage medium | |
| CN113612811B (en) | Method, system, equipment and medium for client mounting in multiple channels | |
| CN104615784B (en) | A kind of method for accessing data | |
| CN109688204B (en) | File downloading method, node and terminal based on NDN (named data networking) | |
| EP3913879A1 (en) | Message parsing method, data sending end, data receiving end, and system | |
| JP2000222273A (en) | Document downloading system | |
| CN116320088B (en) | Method and device for realizing AAA forwarding | |
| CN110069303A (en) | A kind of screen layout's update method and server, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |