[go: up one dir, main page]

CN112714149A - Data synchronization method and device, computer equipment and storage medium - Google Patents

Data synchronization method and device, computer equipment and storage medium Download PDF

Info

Publication number
CN112714149A
CN112714149A CN202011364714.2A CN202011364714A CN112714149A CN 112714149 A CN112714149 A CN 112714149A CN 202011364714 A CN202011364714 A CN 202011364714A CN 112714149 A CN112714149 A CN 112714149A
Authority
CN
China
Prior art keywords
data synchronization
synchronization file
data
node
identification information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011364714.2A
Other languages
Chinese (zh)
Inventor
谈宜华
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Feixun Digital Technology Co ltd
Original Assignee
Beijing Feixun Digital Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Feixun Digital Technology Co ltd filed Critical Beijing Feixun Digital Technology Co ltd
Priority to CN202011364714.2A priority Critical patent/CN112714149A/en
Publication of CN112714149A publication Critical patent/CN112714149A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1095Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/06Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1097Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The embodiment of the invention discloses a data synchronization method, a data synchronization device, computer equipment and a storage medium. The method comprises the following steps: acquiring a data synchronization file, wherein the data synchronization file is generated according to the editing data; sending the version identification information corresponding to the data synchronization file to a connected node so that the connected node judges whether to acquire the data synchronization file according to the version identification information; and when receiving data synchronization file acquisition information sent by the connected node, sending the data synchronization file to the connected node so that the connected node carries out data synchronization according to the data synchronization file. The embodiment of the invention can synchronize the updating data to the connected nodes in real time and accurately execute the cross-level operation of the multi-level node network.

Description

Data synchronization method and device, computer equipment and storage medium
Technical Field
The embodiment of the invention relates to the field of data processing, in particular to a data synchronization method, a data synchronization device, computer equipment and a storage medium.
Background
In a multi-level node network, a superior-inferior relation exists among nodes, and each node stores private data.
When a certain node needs to operate in a cross-stage mode, the node needs to acquire data of other nodes. For example: when the upper node requests the cameras of the lower nodes in a cross-level mode, the data of the lower nodes about the cameras need to be found at the upper node. The data of the camera is searched for at the upper node, and then is forwarded to the lower node, and the camera is requested by the lower node.
If the data of the camera in the lower node is modified and the upper node does not know, the upper node cannot acquire the modified data, so that the cross-level operation cannot be correctly executed.
Disclosure of Invention
Embodiments of the present invention provide a data synchronization method, apparatus, computer device, and storage medium, which can synchronize update data to connected nodes in real time and accurately perform a cross-level operation of a multi-level node network.
In a first aspect, an embodiment of the present invention provides a data synchronization method, including:
acquiring a data synchronization file, wherein the data synchronization file is generated according to the editing data;
sending the version identification information corresponding to the data synchronization file to a connected node so that the connected node judges whether to acquire the data synchronization file according to the version identification information;
and when receiving data synchronization file acquisition information sent by the connected node, sending the data synchronization file to the connected node so that the connected node carries out data synchronization according to the data synchronization file.
In a second aspect, an embodiment of the present invention provides a data synchronization apparatus, including:
the data synchronization file acquisition module is used for acquiring a data synchronization file, and the data synchronization file is generated according to the editing data;
the version identification information sending module is used for sending the version identification information corresponding to the data synchronization file to a connected node so that the connected node judges whether to acquire the data synchronization file or not according to the version identification information;
and the data synchronization file sending module is used for sending the data synchronization file to the connected node when receiving the data synchronization file acquisition information sent by the connected node so as to enable the connected node to carry out data synchronization according to the data synchronization file.
In a third aspect, an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the data synchronization method according to any one of the embodiments of the present invention when executing the computer program.
In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the data synchronization method according to any one of the embodiments of the present invention.
According to the embodiment of the invention, the data synchronization file generated according to the editing data is obtained, the version identification information of the data synchronization file is sent to the connected node, and when the connected node determines that the data synchronization file needs to be obtained according to the version identification information, the data synchronization file obtaining information fed back by the connected node is received, so that the data synchronization file is sent to the connected node, the connected node performs data synchronization according to the editing data in the data synchronization file, the problem that the modified data of different-stage nodes cannot be updated synchronously in the prior art is solved, the updating of the editing data among the multi-stage nodes is realized, and the execution accuracy of the cross-stage operation is improved.
Drawings
FIG. 1a is a flow chart of a data synchronization method according to a first embodiment of the present invention;
fig. 1b is a schematic diagram of a multi-level node network according to a first embodiment of the present invention;
FIG. 2 is a flowchart of a data synchronization method according to a second embodiment of the present invention;
fig. 3 is a schematic structural diagram of a data synchronization apparatus according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of a computer device in the fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1a is a flowchart of a data synchronization method in a first embodiment of the present invention, where this embodiment is applicable to a situation where data synchronization is performed in a multi-level node network, and the method may be executed by a data synchronization apparatus provided in the first embodiment of the present invention, where the apparatus may be implemented in a software and/or hardware manner, and may generally be integrated into a computer device. As shown in fig. 1a, the method of this embodiment specifically includes:
as shown in fig. 1B, the multi-level node network has a structure as shown in fig. 1B, and neighboring nodes have a hierarchical relationship, for example, node a is an upper level of node B. Each node may have a plurality of databases, for example, node a is configured with database a1 and database a2, and databases a1 and a2 belong to a master database and a slave database, wherein the master database and the slave database are of the same type, the master database provides database services, the slave database is synchronized with the master database, and when the master database fails, the slave database is operated to provide normal database services instead of the master database; or there may be multiple different databases per node, for example, database a1 is a mySql database and database a2 is a MongoDB database.
S110, acquiring a data synchronization file, wherein the data synchronization file is generated according to the editing data.
The data synchronization file is used for synchronizing the carried node editing data to other nodes. Data synchronization files may be transmitted between two nodes that are connected. The data synchronization file includes edit data. The edited data is transmitted between two nodes connected as data to be synchronized. The edit data may refer to data associated with an operation on at least one database of the node, the operation including at least one of: addition, deletion, modification, and the like.
S120, sending the version identification information corresponding to the data synchronization file to a connected node, so that the connected node judges whether to acquire the data synchronization file according to the version identification information.
The version identification information is used to distinguish the data synchronization file from the historical data synchronization file to identify the version of the data synchronization file. Each data synchronization file is configured with version identification information. In fact, the nodes need to update the edit data continuously and generate new data synchronization files, and correspondingly, the nodes needing to be synchronized need to perform data synchronization continuously according to the new data synchronization files, and edition identification information is used for distinguishing edit data in different stages.
A connected node refers to a node that is connected to the current node. And the current node establishes connection with the connected node, and transmits version identification information, data synchronization files and the like through the established connection. And the connected node acquires the same data in the node corresponding to the data synchronization file and synchronizes the data in the associated database so as to realize the data synchronization of the connected node and the corresponding node.
The connected node can detect whether the version of the data in the associated database is the same as the version of the data synchronization file according to the version identification information so as to judge whether the data synchronization file needs to be acquired for data synchronization.
For example, for a plurality of pieces of edit data of one node, the connected node may record version identification information corresponding to each data synchronization file of historical synchronization, and if the connected node does not record information that is the same as the version identification information sent by the current node, or if the highest version of the data in the connected node is lower than the version of the data synchronization file sent by the current node, it is determined that the connected node needs to acquire the data synchronization file; and if the connected node records the version identification information which is the same as the version identification information sent by the current node, or the version of the data in the connected node is higher than or equal to the version of the data synchronization file sent by the current node, determining that the connected node does not need to acquire the data synchronization file.
The data synchronization file may be identified by time and generation node in addition to the version identification information. Optionally, the name of the data synchronization file includes node identification information, time information, and version identification information, where the node identification information is used to determine a node corresponding to the edited data; the time information is used for determining the generation time of the data synchronization file; the version identification information is used for determining a version number corresponding to the editing data; the data synchronization file is also used for restoring the editing data.
The node identification information is used to determine which node the data synchronization file was generated, i.e., which node the edit data is. The node identification information is used for distinguishing data synchronization files generated by different nodes. The version identification information is used for distinguishing different data synchronization files generated by one node and distinguishing the different data synchronization files according to the generated time sequence. The time information is used to determine when the data synchronization file is generated. The node identification information, the time information, the version identification information and the like can be combined to form the name of the data synchronization file, the corresponding relation between the data synchronization file and the corresponding identification information is quickly established, and meanwhile, the storage space which is independently allocated for the identification information of the data synchronization file can be saved.
In fact, the multi-level node network may be a network in which any two nodes intercommunicate, or a network in which only adjacent upper and lower level nodes intercommunicate. The transmission modes of the data synchronization files of different networks are different.
In a network in which any two nodes are intercommunicated, optionally, the sending the version identification information to the connected node includes: broadcasting the version identification information in a multi-level node network, wherein any two nodes in the multi-level node network are mutually communicated.
Any two nodes in the multi-level node network are communicated with each other, which indicates that the current node is communicated with any one node in the multi-level node network. Thus, the current node can broadcast the version identification information directly to the nodes. After receiving the version identification information, each node judges whether a data synchronization file corresponding to the version identification information is needed, and if so, feeds back data synchronization file acquisition information to the current node. When the current node receives the data synchronization file acquisition information sent by a certain node, the current node sends the data synchronization file to the node. After receiving the data synchronization file, any node analyzes and synchronizes the data, and after the data synchronization is completed, deletes the data synchronization file and releases the storage resource. And the data synchronization file is stored in the current node all the time.
In a specific example, the node N performs data modification on the associated database, generates a data synchronization file, and places the data synchronization file in the local backup folder. And then acquiring Internet Protocol (IP) information of all the superior and subordinate nodes of the user, broadcasting the version number of the data synchronization file required to be uploaded by the user through the IP, and informing all the superior and subordinate nodes of acquiring the data synchronization file. After receiving the message, all the superior nodes acquire corresponding data synchronization files from the node N according to the version numbers, store the data synchronization files in the upgrade folder, and write data after analyzing the data synchronization files; and deleting the data synchronization file in the local upgrade folder after the data writing is finished.
The synchronization mode is a Point-to-Point (P2P) mode for data synchronization. The synchronization efficiency is high, the speed is high, but the network communication requirement is high.
By broadcasting the version identification information to each node in the network where any two nodes are communicated, the synchronization speed can be increased, and the synchronization efficiency can be improved.
If a certain node is in an off-line state in a certain period of time, when the node is restarted, updating detection can be carried out by searching all subordinate information of the node to inquire version information, and when the node is updated, data synchronization is carried out to ensure that the node is synchronized in time.
Optionally, after offline startup, a target data synchronization file corresponding to each subordinate connected node is respectively requested from each subordinate connected node, where the target data synchronization file includes unsynchronized edited data between the current node and the subordinate connected node; and receiving each target data synchronization file and carrying out data synchronization.
The offline start may be a power-on start or a restart. The current node requests to acquire the target data synchronization file corresponding to the subordinate connected node, which may be understood as the current node requesting to acquire the local data synchronization file that is not synchronized in the subordinate connected node. And the target data synchronization file is used for data synchronization of the current node. The target data synchronization file comprises edit data which is not synchronized with the subordinate connected node by the current node, the target data synchronization file corresponds to the subordinate connected node, and the edit data is data which is operated by the subordinate connected node on the associated database.
When the current node is offline, the version identification information broadcasted by other nodes cannot be acquired, and the version identification information can be inquired for each subordinate node after the current node is started, so that the data in the current node can be updated.
In the nodes of the upper and lower relations, the upper node may generally acquire data of the lower node, and the lower node may not acquire data of the upper node, so that it is possible to ensure that the current node can acquire data of each lower node by inquiring of the lower node.
For example, the current node requests each subordinate connected node to obtain corresponding version identification information, where the version identification information may refer to a version number of at least one data synchronization file corresponding to the subordinate connected node. For each subordinate connected node, after the current node acquires any one version identification information, whether the version identification information is the target version identification information or not is judged, and the target version identification information corresponds to the target data synchronization file. If so, the current node feeds back data synchronization file acquisition information to the subordinate connected node, and correspondingly, the subordinate connected node sends the target data synchronization file matched with the target version identification information to the current node. If not, the current node does not feed back the data synchronization file acquisition information.
After offline online, the data synchronization file which is not synchronized is requested from each subordinate communication node, and data synchronization is carried out, so that the condition that the synchronized data is not updated timely due to offline can be avoided, the real-time performance of data synchronization is improved, and the stability of network data synchronization is improved.
In a network where only adjacent upper and lower nodes are intercommunicated, optionally, the sending the version identification information to a connected node includes: and sending the version identification information to adjacent connected nodes in a multi-level node network, wherein the adjacent two nodes in the multi-level node network are mutually connected, and the non-adjacent nodes are not connected.
In the network only with the intercommunication of the adjacent upper and lower nodes, any two adjacent upper and lower nodes are intercommunicated and can communicate, and the network between two nodes crossing one or more nodes is not intercommunicated. For example, in fig. 1B, node a and node B are interworking and node B and node N are not interworking, wherein at least one node exists between node B and node N.
And the adjacent connected node is adjacent to the current node and is in network connection. The current node can only send the version identification information and the data synchronization file to the adjacent connected nodes, and each adjacent connected node sends the version identification information and the data synchronization file to the adjacent connected nodes, and so on. The synchronization mode is one-way one-forward spreading data synchronization. Low efficiency and low speed, but low network requirements.
In one specific example, node A is in communication with node B, node B is in communication with node C, node C is in communication with node D, and node D is in communication with node E. And the node C modifies the data of the associated database, generates a data synchronization file and puts the data synchronization file into a local backup folder. The directly subordinate upper node B and the directly subordinate lower node D are notified of the version numbers, respectively. After receiving the notification, the node B and the node D judge whether the corresponding data synchronization file is needed according to the version number, and when needed, the node B and the node D acquire the data synchronization file of the node C, download the data synchronization file to a local upgrade folder, and perform data entry synchronization; then node B informs node A, node D informs node E; and after receiving the notification, the node A and the node E respectively receive the data synchronization file of the node C from the node B and the node D and store the data synchronization file into the upgrade file folder of the current level. The following synchronization process is analogized to the uppermost node and the lowermost node. After the data synchronization of each node is completed,
after receiving the data synchronization file, any node analyzes and synchronizes the data, and after the data synchronization is completed, deletes the data synchronization file corresponding to each version identification information before the latest version identification information, and releases the storage resource. And each node always stores the data synchronization file with the latest version number. The node that generates the data synchronization file always stores the data synchronization file.
By transmitting the version identification information to each node in a chained mode in a network only with the intercommunication of adjacent nodes, the version identification information can be accurately transmitted to nodes of the whole network, and the data synchronization is accurately realized.
Optionally, after offline startup, a target data synchronization file is respectively requested from each adjacent connected node, where the target data synchronization file is a data synchronization file stored by the adjacent connected node, and the target data synchronization file includes unsynchronized edit data between a current node and the adjacent connected node; and receiving each target data synchronization file and carrying out data synchronization.
The target synchronization file can be a data synchronization file generated and stored by an adjacent connected node and corresponds to the adjacent connected node; or the target synchronization file may be a data synchronization file generated by other nodes stored by adjacent connected nodes, and the adjacent connected nodes do not correspond to the target data synchronization file.
The current node requests to acquire the target data synchronization file stored in the adjacent connected node, which can be understood as the current node requesting to acquire the local data synchronization file which is not synchronized and stored in the adjacent connected node. And the target data synchronization file is used for data synchronization of the current node. The target data synchronization file comprises edit data which is not synchronized with the connected nodes by the current node, the edit data is data operated by the target node on the associated database, and the target node can be the same as or different from the adjacent connected nodes.
When the current node is offline, the version identification information broadcast by other nodes cannot be acquired, and the version identification information can be inquired from each adjacent connected node after the current node is started, so that the data in the current node can be updated.
Illustratively, the current node queries its own neighboring connected nodes respectively, and the neighboring connected nodes include neighboring upper nodes and/or neighboring lower nodes. And the adjacent connected nodes directly send the current node if the latest data synchronization file exists in the local upgrade folder, and the current node informs the adjacent superior node and/or the adjacent subordinate node after the synchronization is finished. And after the synchronization of the current node is finished, deleting the historical data synchronization file matched with the target data synchronization file, wherein the node identification information corresponding to the target data synchronization file is the same as the node identification information corresponding to the historical data synchronization file, the corresponding version identification information is different, and the version number corresponding to the target data synchronization file is higher than the version number corresponding to the historical data synchronization file.
The current node inquires its own adjacent connected nodes respectively, and the adjacent connected nodes directly send the current node if there is the latest data synchronization file in the local upgrade folder, which may be: the current node sends the version identification information of the current node to the adjacent connected node, the adjacent connected node inquires the version identification information of the data synchronization file corresponding to the same node in the data synchronization file stored in the local upgrade folder, and if the version identification information of the target data synchronization file is higher than the received version identification information, the target data synchronization file is sent to the current node.
After offline online, the unsynchronized data synchronization file is requested from each adjacent connected node, and data synchronization is performed, so that the condition that the synchronized data is not updated timely due to offline can be avoided, the real-time performance of data synchronization is improved, and the stability of network data synchronization is improved.
Optionally, the acquiring the data synchronization file includes: and receiving the data synchronization file sent by the adjacent connected nodes, and performing data synchronization.
In a network where only adjacent upper and lower nodes are intercommunicated, the current node may be a generation node of a data synchronization file or a forwarding node of the data synchronization file. Besides the adjacent connected node, the adjacent connected node also exists, the version identification information can be forwarded to the adjacent connected node, and when the data synchronization file acquisition information sent by the adjacent connected node is received, the data synchronization file is forwarded to the adjacent connected node
In addition, in the network where any two nodes are intercommunicated, the current node is a generation node of the data synchronization file.
S130, when receiving the data synchronization file acquisition information sent by the connected node, sending the data synchronization file to the connected node so that the connected node carries out data synchronization according to the data synchronization file.
The data synchronization file acquisition information is used for indicating the current node to send the data synchronization file to the connected node. And the data synchronization file acquisition information is determined according to the version identification information. And when the connected node acquires the version identification information and determines that the data synchronization file is needed, feeding back data synchronization file acquisition information in the connection established with the current node. And the current node receives the data synchronization file acquisition information sent by the connected node and sends the data synchronization file to the connected node. The data synchronization by the connected node according to the data synchronization file may include: adding the edited data in the data synchronization file to the corresponding database, deleting the same data as the edited data in the corresponding database, or modifying (replacing) the matched data with the edited data, and the like.
According to the embodiment of the invention, the data synchronization file generated according to the editing data is obtained, the version identification information of the data synchronization file is sent to the connected node, and when the connected node determines that the data synchronization file needs to be obtained according to the version identification information, the data synchronization file obtaining information fed back by the connected node is received, so that the data synchronization file is sent to the connected node, the connected node performs data synchronization according to the editing data in the data synchronization file, the problem that the modified data of different-stage nodes cannot be updated synchronously in the prior art is solved, the updating of the editing data among the multi-stage nodes is realized, and the execution accuracy of the cross-stage operation is improved.
Example two
Fig. 2 is a flowchart of a data synchronization method according to a second embodiment of the present invention, which is embodied based on the above embodiments. The method of the embodiment specifically includes:
s210, determining the version identification information according to the historical version identification information.
The generation node of the data synchronization file corresponding to the historical version identification information is the same as the generation node of the data synchronization file corresponding to the version identification information, and the generation time of the corresponding data synchronization file is different.
The version identification information is continuously accumulated, and can be accumulated on the historical version identification to calculate the version identification information. Illustratively, the historical version identification information is v1.0, and the version identification information may be v2.0, or may be v 1.1.
Reference may be made to the foregoing embodiments for describing embodiments of the invention without more specificity.
The current node in this embodiment is a generation node of the data synchronization file.
S220, acquiring the editing data of the associated at least one database, and generating a data synchronization file corresponding to the version identification information. The data synchronization file is an XML format file, and corresponds to the current node.
As in the previous example, the number of the databases is at least one, and when the number of the databases is at least two, the databases may include a master database or a slave database or several different types of databases.
And converting the data of the database into XML format data, storing the XML format data into an XML file, and determining the XML file as a data synchronization file. The XML file is adopted, so long as the same database name and data table are adopted, the database compatibility is improved without the limitation of different types of the databases of all nodes.
In the current node, after static data is modified each time, a user can manually click a data uploading button, the current node generates a file according to detailed setting rules of XML content, the current node names the generated XML file according to node identification information, time information (such as current time) and version identification information (such as current data version number), generates a data synchronization file, and places the data synchronization file into a backup folder. The node identification information, the current time and the current data version are used to record which node is the data change of which version number at which point in time.
Optionally, the data synchronization file includes: the data table comprises identification information of a data table, fields, field types corresponding to the fields and data corresponding to the fields, and the edited data comprises edited data in at least one database. The data table identification information is used for distinguishing data of different data tables in different databases. Fields are used as parameters, and correspondingly, data corresponding to the fields can refer to attribute values of the parameters. The field type may refer to a data type of the data.
Illustratively, the data format of an XML file is as follows:
<?XML version="1.0"encoding="UTF-8"?>
<TableMsg>
<TableName>
<ColumnType>
<Column>columnType</Column>
<Column>columnType</Column>
<Column>columnType</Column>
</ColumnType>
<Record>
<Column>columnValue</Column>
<Column>columnValue</Column>
<Column>columnValue</Column>
</Record>
<Record>
<Column>columnValue</Column>
<Column>columnValue</Column>
<Column>columnValue</Column>
</Record>
</TableName>
</TableMsg>
the first behavior file header information, version ═ 1.0, is irrelevant to the version identification information of the data synchronization file, and the content information is data-encapsulated by adopting the standard of UTF-8. The label and field meanings are: TableMsg may refer to a root node, which represents an XML file for data synchronization, and as a system identifier, the content between < TableMsg > and </TableMsg > is edit data; the TableName tag is a table name tag, namely data table identification information, and the content between < TableName > and </TableName > is the edit data of TableName (a certain table); the value of the ColumnType tag indicates the type of the field in which the data table is recorded, and the content between < ColumnType > and </ColumnType > is all the fields included in the edit data in the data table; columnType is a field type, e.g., date, text, or numeric value, etc.; the Record label indicates that data of a complete table is recorded, the content between the Record and the Record is a piece of data, the content between the Record and the Record corresponds to the content between the ColumType and the ColumType, the ColumType and the ColumType can be understood as a first line field in the excel table, and the content between the Record and the Record can be understood as a line of data arranged corresponding to the first line field in the excel table, wherein the field is a parameter name and the data is a parameter name corresponding attribute value; columnValue is the value of each field of each piece of data; the Column tag is a field name tag, and < Column > and </Column > represent a field or data. Optionally, the columns included in one Record correspond to the columns included in the Column type one by one.
S230, sending the version identification information corresponding to the data synchronization file to a connected node, so that the connected node judges whether to acquire the data synchronization file according to the version identification information.
S240, when receiving the data synchronization file acquisition information sent by the connected node, sending the data synchronization file to the connected node so that the connected node performs data synchronization according to the data synchronization file.
Generating the data synchronization file may refer to converting data of each database type into data in an XML format. Before data synchronization, the data synchronization file needs to be analyzed, and data recorded by the data synchronization file is acquired and stored in a database. Parsing the data synchronization file and performing data synchronization to the database may refer to converting XML formatted data into database-type data.
Optionally, the data synchronization method further includes: storing the data synchronization file; when a data reduction instruction is received, obtaining a reduced data synchronization file matched with the data reduction instruction from a plurality of pre-stored data synchronization files, wherein the version identification information matched with each data synchronization file is different, and the version identification information matched with the reduced data synchronization file is matched with the reduced version identification information appointed by the data reduction instruction; and performing data reduction according to the reduced data synchronization file.
The data synchronization file is stored locally, which is actually the process of data backup. The data reduction instruction is used for reducing the data into data of historical time. For example, the modified data is restored to the data before modification. The data restore instruction specifies the version number to be restored, i.e., specifies the restore version identification information. The restore version identification information may refer to historical version identification information, i.e., the restore version number is lower than or equal to the latest version number. The version identification information of each data synchronization file can be inquired, the data synchronization file matched with the reduced version identification information is determined as a reduced data synchronization file, analysis is carried out, and data synchronization is carried out, so that the data is reduced into the edited data included in the reduced data synchronization file.
The data synchronization file is stored as a data backup basis, when a data recovery instruction is received, the data synchronization file is determined to be recovered from the prestored data synchronization file, and the data recovery is carried out, so that the data backup and recovery can be realized, when the data is lost in the node database, the lost data can be acquired in time, the data safety is improved, and the fault tolerance of the node is improved.
The embodiment of the invention generates the XML format file by format conversion of the edited data, can be compatible with a plurality of database types, and can perform data synchronization on data of different database types only according to the same data file, thereby improving the efficiency of different types of data synchronization.
EXAMPLE III
Fig. 3 is a schematic diagram of a data synchronization apparatus according to a third embodiment of the present invention. The third embodiment is a corresponding apparatus for implementing the data synchronization method provided by the foregoing embodiments of the present invention, and the apparatus may be implemented in a software and/or hardware manner, and may be generally integrated in a computer device, for example, in a node of a multi-level node network.
Accordingly, the apparatus of the present embodiment may include:
a data synchronization file obtaining module 310, configured to obtain a data synchronization file, where the data synchronization file is generated according to the edit data;
the version identification information sending module 320 is configured to send the version identification information corresponding to the data synchronization file to a connected node, so that the connected node determines whether to acquire the data synchronization file according to the version identification information;
the data synchronization file sending module 330 is configured to send the data synchronization file to the connected node when receiving the data synchronization file acquisition information sent by the connected node, so that the connected node performs data synchronization according to the data synchronization file.
According to the embodiment of the invention, the data synchronization file generated according to the editing data is obtained, the version identification information of the data synchronization file is sent to the connected node, and when the connected node determines that the data synchronization file needs to be obtained according to the version identification information, the data synchronization file obtaining information fed back by the connected node is received, so that the data synchronization file is sent to the connected node, the connected node performs data synchronization according to the editing data in the data synchronization file, the problem that the modified data of different-stage nodes cannot be updated synchronously in the prior art is solved, the updating of the editing data among the multi-stage nodes is realized, and the execution accuracy of the cross-stage operation is improved.
Further, the version identification information sending module 320 is specifically configured to: broadcasting the version identification information in a multi-level node network, wherein any two nodes in the multi-level node network are mutually communicated.
Further, the data synchronization apparatus further includes: the system comprises an offline synchronization module, a data synchronization module and a data synchronization module, wherein the offline synchronization module is specifically used for respectively requesting a target data synchronization file corresponding to each subordinate connected node from each subordinate connected node after offline starting, and the target data synchronization file comprises unsynchronized editing data between a current node and the subordinate connected node; and receiving each target data synchronization file and carrying out data synchronization.
Further, the version identification information sending module 320 is specifically configured to: and sending the version identification information to adjacent connected nodes in a multi-level node network, wherein the adjacent two nodes in the multi-level node network are mutually connected, and the non-adjacent nodes are not connected.
Further, the data synchronization apparatus further includes: the offline synchronization module is specifically used for respectively requesting a target data synchronization file from each adjacent connected node after offline startup, wherein the target data synchronization file is a data synchronization file stored by the adjacent connected node, and the target data synchronization file comprises unsynchronized edit data between a current node and the adjacent connected node; and receiving each target data synchronization file and carrying out data synchronization.
Further, the data synchronization file obtaining module 310 is specifically configured to: and receiving the data synchronization file sent by the adjacent connected nodes, and performing data synchronization.
Further, the data synchronization file obtaining module 310 is specifically configured to: determining version identification information according to the historical version identification information; acquiring edit data of at least one associated database, and generating a data synchronization file corresponding to the version identification information; the data synchronization file is an XML format file, and corresponds to the current node.
Further, the data synchronization apparatus further includes: the data backup and restoration module is used for storing the data synchronization file; when a data reduction instruction is received, obtaining a reduced data synchronization file matched with the data reduction instruction from a plurality of pre-stored data synchronization files, wherein the version identification information matched with each data synchronization file is different, and the version identification information matched with the reduced data synchronization file is matched with the reduced version identification information appointed by the data reduction instruction; and performing data reduction according to the reduced data synchronization file.
The data synchronization device can execute the data synchronization method provided by the first embodiment of the invention, and has the corresponding functional modules and beneficial effects of the executed data synchronization method.
Example four
Fig. 4 is a schematic structural diagram of a computer device according to a fourth embodiment of the present invention. FIG. 4 illustrates a block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention. The computer device 12 shown in FIG. 4 is only one example and should not bring any limitations to the functionality or scope of use of embodiments of the present invention.
As shown in FIG. 4, computer device 12 is in the form of a general purpose computing device. The components of computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 and the processing unit 16. The computer device 12 may be a device that is attached to a high-speed industrial control bus.
Bus 18 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, such architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.
Computer device 12 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by computer device 12 and includes both volatile and nonvolatile media, removable and non-removable media.
The system memory 28 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)30 and/or cache memory 32. Computer device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 34 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 4, and commonly referred to as a "hard drive"). Although not shown in FIG. 4, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a Compact disk Read-Only Memory (CD-ROM), Digital Video disk (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to bus 18 by one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the invention.
A program/utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 42 generally carry out the functions and/or methodologies of the described embodiments of the invention.
Computer device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), with one or more devices that enable a user to interact with computer device 12, and/or with any devices (e.g., network card, modem, etc.) that enable computer device 12 to communicate with one or more other computing devices. Such communication may be through an Input/Output (I/O) interface 22. Also, computer device 12 may communicate with one or more networks (e.g., Local Area Network (LAN), Wide Area Network (WAN)) via Network adapter 20. As shown, Network adapter 20 communicates with other modules of computer device 12 via bus 18. it should be understood that although not shown in FIG. 4, other hardware and/or software modules may be used in conjunction with computer device 12, including without limitation, microcode, device drivers, Redundant processing units, external disk drive Arrays, (Redundant Arrays of Inesponsive Disks, RAID) systems, tape drives, data backup storage systems, and the like.
The processing unit 16 executes various functional applications and data processing by executing programs stored in the system memory 28, for example, implementing a data synchronization method provided by any of the embodiments of the present invention.
EXAMPLE five
Fifth embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the data synchronization method provided in all the inventive embodiments of the present application:
that is, the program when executed by the processor implements: acquiring a data synchronization file, wherein the data synchronization file is generated according to the editing data; sending the version identification information corresponding to the data synchronization file to a connected node so that the connected node judges whether to acquire the data synchronization file according to the version identification information; and when receiving data synchronization file acquisition information sent by the connected node, sending the data synchronization file to the connected node so that the connected node carries out data synchronization according to the data synchronization file.
Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a RAM, a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM), a flash Memory, an optical fiber, a portable CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, Radio Frequency (RF), etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a LAN or a WAN, or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims (10)

1. A method of data synchronization, comprising:
acquiring a data synchronization file, wherein the data synchronization file is generated according to the editing data;
sending the version identification information corresponding to the data synchronization file to a connected node so that the connected node judges whether to acquire the data synchronization file according to the version identification information;
and when receiving data synchronization file acquisition information sent by the connected node, sending the data synchronization file to the connected node so that the connected node carries out data synchronization according to the data synchronization file.
2. The method of claim 1, wherein sending version identification information corresponding to the data synchronization file to a connected node comprises:
broadcasting the version identification information in a multi-level node network, wherein any two nodes in the multi-level node network are mutually communicated.
3. The method of claim 2, further comprising:
after offline starting, respectively requesting a target data synchronization file corresponding to each subordinate connected node from each subordinate connected node, wherein the target data synchronization file comprises unsynchronized edited data between a current node and the subordinate connected node;
and receiving each target data synchronization file and carrying out data synchronization.
4. The method of claim 1, wherein sending version identification information corresponding to the data synchronization file to a connected node comprises:
and sending the version identification information to adjacent connected nodes in a multi-level node network, wherein the adjacent two nodes in the multi-level node network are mutually connected, and the non-adjacent nodes are not connected.
5. The method of claim 4, further comprising:
after offline starting, respectively requesting a target data synchronization file from each adjacent connected node, wherein the target data synchronization file is a data synchronization file stored by the adjacent connected node and comprises unsynchronized edit data between the current node and the adjacent connected node;
and receiving each target data synchronization file and carrying out data synchronization.
6. The method of claim 1, wherein the obtaining the data synchronization file comprises:
determining version identification information according to the historical version identification information;
acquiring edit data of at least one associated database, and generating a data synchronization file corresponding to the version identification information;
the data synchronization file is an XML format file, and corresponds to the current node.
7. The method of claim 1, further comprising:
storing the data synchronization file;
when a data reduction instruction is received, obtaining a reduced data synchronization file matched with the data reduction instruction from a plurality of pre-stored data synchronization files, wherein the version identification information matched with each data synchronization file is different, and the version identification information matched with the reduced data synchronization file is matched with the reduced version identification information appointed by the data reduction instruction;
and performing data reduction according to the reduced data synchronization file.
8. A data synchronization apparatus, comprising:
the data synchronization file acquisition module is used for acquiring a data synchronization file, and the data synchronization file is generated according to the editing data;
the version identification information sending module is used for sending the version identification information corresponding to the data synchronization file to a connected node so that the connected node judges whether to acquire the data synchronization file or not according to the version identification information;
and the data synchronization file sending module is used for sending the data synchronization file to the connected node when receiving the data synchronization file acquisition information sent by the connected node so as to enable the connected node to carry out data synchronization according to the data synchronization file.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the data synchronization method according to any of claims 1-7 when executing the program.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the data synchronization method according to any one of claims 1 to 7.
CN202011364714.2A 2020-11-27 2020-11-27 Data synchronization method and device, computer equipment and storage medium Pending CN112714149A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011364714.2A CN112714149A (en) 2020-11-27 2020-11-27 Data synchronization method and device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011364714.2A CN112714149A (en) 2020-11-27 2020-11-27 Data synchronization method and device, computer equipment and storage medium

Publications (1)

Publication Number Publication Date
CN112714149A true CN112714149A (en) 2021-04-27

Family

ID=75542483

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011364714.2A Pending CN112714149A (en) 2020-11-27 2020-11-27 Data synchronization method and device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN112714149A (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013071428A1 (en) * 2011-11-04 2013-05-23 Dassault Systemes Geovia Inc., Dba Gemcom Software International Inc. System and method for data synchronization over a network
US20160070717A1 (en) * 2014-09-10 2016-03-10 Salesforce.Com, Inc. Synchronization and strategic storage of multiple versions of a file
CN106817387A (en) * 2015-11-28 2017-06-09 成都华为技术有限公司 A kind of method of data synchronization, device and system
CN109684414A (en) * 2018-12-26 2019-04-26 百度在线网络技术(北京)有限公司 Synchronous method, device, equipment and the storage medium of block data
CN110071964A (en) * 2019-03-26 2019-07-30 罗克佳华科技集团股份有限公司 File synchronisation method, device, file sharing network, file are total to system and storage medium
CN110572450A (en) * 2019-09-05 2019-12-13 腾讯科技(深圳)有限公司 Data synchronization method and device, computer readable storage medium and computer equipment

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013071428A1 (en) * 2011-11-04 2013-05-23 Dassault Systemes Geovia Inc., Dba Gemcom Software International Inc. System and method for data synchronization over a network
US20160070717A1 (en) * 2014-09-10 2016-03-10 Salesforce.Com, Inc. Synchronization and strategic storage of multiple versions of a file
CN106817387A (en) * 2015-11-28 2017-06-09 成都华为技术有限公司 A kind of method of data synchronization, device and system
CN109684414A (en) * 2018-12-26 2019-04-26 百度在线网络技术(北京)有限公司 Synchronous method, device, equipment and the storage medium of block data
CN110071964A (en) * 2019-03-26 2019-07-30 罗克佳华科技集团股份有限公司 File synchronisation method, device, file sharing network, file are total to system and storage medium
CN110572450A (en) * 2019-09-05 2019-12-13 腾讯科技(深圳)有限公司 Data synchronization method and device, computer readable storage medium and computer equipment

Similar Documents

Publication Publication Date Title
CN111143382B (en) Data processing method, system and computer readable storage medium
KR101485613B1 (en) A method and system for synchronizing Web syndication items, a method for updating web syndication channels,
CN112214547B (en) Data processing method, data server, electronic device and storage medium
RU2471227C2 (en) Peer-to-peer synchronisation assisted with service unit
CN110377577B (en) Data synchronization method, device, system and computer readable storage medium
US8572161B2 (en) Simplifying synchronization of copies of same data used by multiple applications
CN103164525B (en) WEB application dissemination method and device
CN105183860B (en) Method of data synchronization and system
CN113761016B (en) Data query method, device, equipment and storage medium
CN111061498B (en) Configuration information management system
CN111104450B (en) Target data importing method, medium, device and computing equipment
CN106874281A (en) Realize the method and apparatus that data base read-write is separate
CN106326352A (en) Universal interface design method and system in butt joint with plurality of service systems
CN105991720A (en) Configuration change method and device
CN111104125B (en) Intelligent integrated version management method, device, medium and electronic equipment
JP2022006160A (en) Data processing methods and equipment, electronic devices, storage media and computer programs
CN113886485A (en) Data processing method, apparatus, electronic device, system and storage medium
CN111104409A (en) Database processing method and device, storage medium and electronic equipment
CN112783842B (en) Log collection method and device
CN112714149A (en) Data synchronization method and device, computer equipment and storage medium
CN111522688B (en) Data backup method and device for distributed system
CN111124542A (en) Configuration information management system
CN112181937A (en) Data transmission method and device
CN113037797B (en) Data processing method and device
CN112416975A (en) Message configuration method, electronic device and readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20210427

RJ01 Rejection of invention patent application after publication