CN120144596A - A distributed graph database attribute data updating method and system - Google Patents
A distributed graph database attribute data updating method and system Download PDFInfo
- Publication number
- CN120144596A CN120144596A CN202311713338.7A CN202311713338A CN120144596A CN 120144596 A CN120144596 A CN 120144596A CN 202311713338 A CN202311713338 A CN 202311713338A CN 120144596 A CN120144596 A CN 120144596A
- Authority
- CN
- China
- Prior art keywords
- attribute
- layer
- graph database
- bitmap
- parties
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
- G06F16/9024—Graphs; Linked lists
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a distributed graph database attribute data updating method and system, which comprise the steps of synchronously transmitting attribute definition lists of two parties of a participant based on a RAFT distributed protocol, keeping consistent attribute definitions of the two parties of the participant, and updating the attribute data increment of a graph database by adopting a double-layer bitmap changing attribute coding mode based on the attribute definition lists of the two parties of the participant. The invention introduces consistency data updating capability based on a distributed graph database RAFT protocol, provides an incremental transmission mode for changing graph data attributes, provides an attribute technology for designating and transmitting changed attributes based on a bitmap mode of point and edge attributes, avoids transmission of redundant keys, and improves attribute data transmission efficiency.
Description
Technical Field
The invention relates to the field of distributed graph databases, in particular to a distributed graph database attribute data updating method and system.
Background
With the explosive development of the internet, the mobile internet, the social network, the internet of things and the related networks in the industrial fields such as the power network, the storage of the relationship graph and the network topology analysis and function analysis based on the relationship graph have great demands, and meanwhile, the development of the graph database is promoted.
The graph database is a data management system which takes points and edges as basic storage units and takes efficient storage and query graph data as design principles.
The distributed database management system is an integral database management system which is used for storing data in different local databases respectively, managing the data by different local database management systems, running on different machines, being connected together by different communication networks and being in a product form. The distributed database is logically a unified whole, and is physically stored on different physical nodes respectively. From the user's perspective, a distributed database system is logically identical to a centralized database system in that a user can execute a global application at any one site. As if those data were stored on the same computer, there was a single database management system (DBMS) managing that the user did not feel as different.
In addition to read-only queries that do not change the graph data, graph data manipulation is also a common data manipulation for distributed graph databases, including:
1) Creating point and edge data;
2) Updating point and edge data;
3) And deleting the point and side data.
For the change operation of the point and edge data of the graph example, one of the change operations relates to the operation of the point and edge elements, such as creating and deleting a point or an edge. The other type of the method involves the changing operation of the attributes of the point and edge elements, such as updating the value of the attributes for a certain point or edge. Together, these two operations form the underlying data operation of the graph database.
For the change operation of the point and side element attribute, besides the actual change operation, the change of attribute data related to the point and side element operation is created:
the values of certain attributes are definitely specified when creating the point and edge elements, and the attribute values belong to changed data;
The values of some of the attributes at the time of creation of the point, edge element may not be specified, and these "unspecified value" attributes may be considered NULL/NULL values (some databases will set default values for these attributes).
In a distributed system, a consistency algorithm allows a group of machines to operate as a whole, and to continue to operate even if some of the machines fail. As such, coherency algorithms (e.g., paxos, RAFT, etc.) play an important role in building a reliable large-scale distributed software system. The Raft protocol splits the core content of the coherence protocol into several key phases, including leader election (leader election) and log replication (log replication), thereby simplifying the flow and improving the understandability of the protocol.
A typical point-to-edge change operation of a distributed graph database will typically involve multiple server nodes in a distributed cluster, because the distributed graph database typically provides redundant backup of partitioned data via a consistency protocol, and thus point-to-edge update operations will involve the updating of multiple redundant backup node servers in the cluster.
When such an update is performed across network nodes after the point and edge attributes in the distributed graph database are updated, the following communication modes of attribute data are generally adopted when the attribute is changed for a specific point or edge and the attribute value is communicated between distributed server nodes so as to notify the change of the attribute value:
all the attributes of a certain point (or edge), including the updated attribute and the non-updated attribute of the current change operation, are sent from one node to another or a plurality of server nodes for updating;
values of one or more attributes to which a point (or edge) is changed are sent from one server node to another node or nodes in key-value pairs. In this case, the "key" of the key-value pair designates the attribute name to which the point (or edge) is changed, and the "value" designates the attribute value to which the point (or edge) is changed. This approach involves only some of the updated attributes when communicating between nodes, and no updated attributes are involved.
Analyzing the data communication between nodes when the attribute data of the above common distributed graph database is changed, we can see that the problems of these methods are:
For the communication mode that all the attributes of the point (or edge) are transmitted, the transmitted data comprise the data of the attribute changed by the operation and the data of the attribute which is not changed, and the values of the attribute which is not changed are not used finally, so that useless data are actually transmitted in the communication process of changing the attribute data;
For the transmission of properties to which a point (or edge) change is made by a "key-value pair", each of the properties involved requires the transmission of a "key" and a "value", where the "key" is the property name. In this case, for points (or edges) of the same Schema (i.e., of the same type), the same "key", i.e., attribute name, will be transmitted multiple times when multiple points, edges, change to the value of the same attribute. For database systems with data sizes of the order of billions, trillions and even trillions, the repeated transmission of "keys" (attribute names) of point (or edge) attributes brings about a great deal of redundancy and increases the data transmission cost.
Disclosure of Invention
In order to solve the problems that in the prior art, unchanged attribute data are transmitted in the communication process of attribute data change and attribute names are transmitted for a plurality of times, the invention provides a distributed graph database attribute data updating method, which comprises the following steps:
synchronously transmitting attribute definition lists of the two parties participating in the transmission based on the RAFT distributed protocol, and keeping the attribute definition of the two parties transmitting consistent;
and updating the attribute data increment of the graph database by adopting a double-layer bitmap changing attribute coding mode based on the attribute definition lists of the two parties.
Optionally, the updating the attribute data increment of the graph database by adopting a modified attribute coding mode of a double-layer bitmap based on the attribute definition lists of the two parties includes:
marking the changed attributes in an ordered attribute list in the map database Schema by adopting a first layer map;
Based on the changed attribute of the first layer bitmap mark, marking the attribute which is set as a null value or a default value in an ordered attribute list in the map database Schema by adopting a second layer bitmap;
Updating the attribute data of the points or edges of the graph database based on the changed attribute of the first layer graph mark and the attribute of the second layer graph mark.
Optionally, the marking, by using a first layer modification bitmap, the changed attribute in the ordered attribute list in the Schema of the graph database includes:
calculating the number of bits by using one bit to represent one attribute according to the number of all attributes of the Schema, and aligning by using one byte to obtain the minimum byte number representing the number of all attributes;
setting subscripts for each bit starting from 0 from left to right, each bit representing an attribute;
If a certain attribute has been changed during the update operation, the bit value of the certain attribute is 1, otherwise, it is 0.
Optionally, the marking, by using a second layer bitmap, the attribute set to a null value or a default value in the ordered attribute list in the map database Schema based on the changed attribute of the first layer bitmap mark includes:
For a first layer bitmap, introducing subscripts into each bit with the position of 1, wherein the subscripts are calculated from 0;
According to the number of bits of 1 in the first layer map, aligning the bits by 8 bits, and calculating the byte number as the byte number of the second layer map;
And when the update value of a certain bit is set to be a NULL/NULL value, setting the value of the bit corresponding to the subscript of the first layer bitmap in the second layer bitmap to be 1.
Optionally, the updating the attribute data of the point or the edge of the graph database based on the changed attribute of the first layer bitmap mark and the attribute of the second layer bitmap mark includes:
Only the change attribute marked by the first layer bitmap and the attribute data of the point or edge of the graph database which is not marked in the second layer bitmap are updated.
Optionally, the RAFT-based distributed protocol synchronously transmits attribute definition lists of both parties involved in the transmission, and keeps both parties involved in the transmission consistent with the attribute definition, including:
The distributed graph database acquires the attribute list of the Schema corresponding to a certain point or edge type through the synchronous operation of the attribute list in the RAFT distributed protocol;
the operation participation parties of the distributed graph database update the attribute list in a synchronous mode in the RAFT distributed protocol, and the transmission parties are kept to have consistent attribute definition.
In still another aspect, the present invention further provides a distributed graph database attribute data updating system, including:
The consistency definition module is used for synchronously transmitting attribute definition lists of the two parties involved in the transmission based on the RAFT distributed protocol, and keeping that the two parties of the transmission have consistent attribute definitions;
and the mark updating module is used for updating the attribute data increment of the graph database by adopting a double-layer bitmap changing attribute coding mode based on the attribute definition lists of the two parties.
Optionally, the mark updating module includes:
The first marking submodule is used for marking the changed attributes in the ordered attribute list in the map database Schema by adopting a first layer bitmap;
The second marking submodule is used for marking the attribute which is set as a null value or a default value in an ordered attribute list in the map database Schema by adopting a second layer bitmap based on the changed attribute of the first layer bitmap marking;
And the updating sub-module is used for updating the attribute data of the points or edges of the graph database based on the changed attribute of the first layer bitmap mark and the attribute of the second layer bitmap mark.
Optionally, the first marking submodule is specifically configured to:
calculating the number of bits by using one bit to represent one attribute according to the number of all attributes of the Schema, and aligning by using one byte to obtain the minimum byte number representing the number of all attributes;
setting subscripts for each bit starting from 0 from left to right, each bit representing an attribute;
If a certain attribute has been changed during the update operation, the bit value of the certain attribute is 1, otherwise, it is 0.
Optionally, the second marking submodule is specifically configured to:
For a first layer bitmap, introducing subscripts into each bit with the position of 1, wherein the subscripts are calculated from 0;
According to the number of bits of 1 in the first layer map, aligning the bits by 8 bits, and calculating the byte number as the byte number of the second layer map;
And when the update value of a certain bit is set to be a NULL/NULL value, setting the value of the bit corresponding to the subscript of the first layer bitmap in the second layer bitmap to be 1.
Optionally, the update submodule is specifically configured to:
Only the change attribute marked by the first layer bitmap and the attribute data of the point or edge of the graph database which is not marked in the second layer bitmap are updated.
Optionally, the consistency definition module is specifically configured to:
The distributed graph database acquires the attribute list of the Schema corresponding to a certain point or edge type through the synchronous operation of the attribute list in the RAFT distributed protocol;
the operation participation parties of the distributed graph database update the attribute list in a synchronous mode in the RAFT distributed protocol, and the transmission parties are kept to have consistent attribute definition.
In yet another aspect, the present application also provides a computing device comprising one or more processors;
a processor for executing one or more programs;
When the one or more programs are executed by the one or more processors, a distributed graph database attribute data update method as described above is implemented.
In yet another aspect, the present application further provides a computer readable storage medium having a computer program stored thereon, the computer program when executed implementing a distributed graph database attribute data updating method as described above.
Compared with the prior art, the invention has the beneficial effects that:
The invention provides a distributed graph database attribute data updating method which comprises the steps of synchronously transmitting attribute definition lists of two parties of a transmission based on a RAFT distributed protocol, keeping consistent attribute definitions of the two parties of the transmission, and updating attribute data increment of a graph database by adopting a double-layer bitmap changing attribute coding mode based on the attribute definition lists of the two parties of the transmission. The invention introduces consistency data updating capability based on a distributed graph database RAFT protocol, provides an incremental transmission mode for changing graph data attributes, provides an attribute technology for designating and transmitting changed attributes based on a bitmap mode of point and edge attributes, avoids transmission of redundant keys, and improves attribute data transmission efficiency.
Drawings
FIG. 1 is a flowchart of a method for updating attribute data of a distributed graph database according to the present invention;
FIG. 2 is a flow chart of a distributed graph database attribute data update method over RAFT of the present invention.
Detailed Description
The invention provides a distributed graph database attribute data updating method and system, and provides a consistent incremental updating mechanism for updating distributed graph database attribute data based on a RAFT protocol in a distributed environment; the two-layer map communication mode for updating the attribute data of the map database is also provided, and only the value data of the attribute with data change is transmitted in the communication mode for the multi-attribute map structure data updating scene, so that the transmission of the unchanged attribute data is avoided, the transmission of the attribute key/attribute name is also avoided, and the transmission of the attribute value set as NULL/NULL value is also avoided.
Example 1:
a distributed graph database attribute data updating method, as shown in fig. 1, includes:
step S1, synchronously transmitting attribute definition lists of two parties involved in transmission based on a RAFT distributed protocol, and keeping the attribute definitions of the two parties of transmission consistent;
And step S2, updating the attribute data increment of the graph database by adopting a double-layer bitmap changing attribute coding mode based on the attribute definition lists of the two parties.
The invention introduces a point and edge attribute data increment updating mechanism on the RAFT distributed protocol, which only transmits the changed attribute and does not transmit the attribute which is not changed, thus being a consistent attribute change increment transmission mode;
The method for designating the changed attribute in the transmission process comprises the step of introducing a designating mode of changing attribute based on a bitmap, wherein the designating mode is used for designating a list of changed attribute in the transmission process, and clear and consistent attribute change coding information is provided.
The invention is described in detail below in connection with fig. 2:
graph points, edge data consistency data operation and an incremental transmission mode based on the RAFT distributed protocol are introduced.
For database systems, ACID (a-atomicity, i.e., atomicity, C-consistency, I-isolation, D-durability, i.e., persistence) is a property that needs to be guaranteed when the system performs data operations, and data consistency and isolation operations are one of core requirements. The invention introduces incremental update of attribute data of a graph database based on a RAFT distributed protocol to provide processing characteristics conforming to consistency and isolation, in particular:
Introducing RAFT ensures consistency of data of a plurality of nodes. The introduction of RAFT provides the characteristics of orderly copying and executing among a plurality of nodes, when the initial states of the nodes are consistent, the states among the nodes are ensured to be consistent, and when the attribute data among different nodes of the partition are updated, the RAFT ensures the consistency of different operations among the nodes at a microscopic level;
On the basis of introducing RAFT, when the attribute data of the point-oriented and edge-oriented elements is updated on each single partition server, incremental updating is introduced:
On the same node, each time of point and edge attribute data updating is necessarily performed on the basis of the previous updating of the attribute data of the same point and edge, and the point and edge attribute data updating is ensured by RAFT;
For the "creation" of a point or edge, the "previous update" is conceptually a state of "no existence", so that on a single server, attribute operations of a single point or edge always exist in a data state corresponding to the so-called "previous update" operation;
in the case where such a "previous update" state exists, only the updated attribute need be sent, and no attribute that has not changed need be sent, when there is an update of the attribute data for a single point or edge. Further:
For a creation point, edge element, "updated properties" refers to all properties of this point, edge. In this case, some attribute values are not specified when the point (or edge) is created, and the corresponding value is a null value (or default value);
For the update operation, the end user specifies at the user interface level or at the application algorithm interface level which "updated properties" are, and thus it is clear that other properties that are not updated will retain their original values.
Based on the consistency of the RAFT distributed protocol, the operation logic facing the updated attribute based on the attribute data of the point and side elements in the graph instance based on the state of the point and side elements in the previous update is introduced, so that the data transmission of the incremental attribute of the point and side attribute data, namely, the attribute facing only the update, is possible in the distributed environment.
The following provides an explanation of the steps of the present invention:
Step S1, based on a RAFT distributed protocol, synchronously transmitting attribute definition lists of both parties involved in transmission, and keeping consistent attribute definition of both parties involved in transmission, specifically comprising:
The distributed graph database acquires the attribute list of the Schema corresponding to a certain point or edge type through the synchronous operation of the attribute list in the RAFT distributed protocol;
the operation participation parties of the distributed graph database update the attribute list in a synchronous mode in the RAFT distributed protocol, and the transmission parties are kept to have consistent attribute definition.
This step S1 is further described below:
In order for the updated "delta" attribute transfer to maintain consistent operational semantics between the two server nodes of the transfer, both sides of the transfer need to make an explicit determination of what delta attributes are point and edge elements to update before the transfer.
In this step, the distributed graph database learns the "attribute list" of the Schema corresponding to a certain point (or edge) type through the synchronization operation of the "attribute list". The attribute list herein refers to a set of a plurality of attributes that a specific Schema has and a sequential list of the attributes. The ordered list of attributes is identified by a subscript, the first attribute subscript being 0, the second being 1, and so on. The operation participation parties of the distributed graph database update the attribute list in a synchronous mode, so that each participating node accurately acquires all the attributes of all the schemes and the sequence and subscripts of the attributes. Specifically:
When creating the Schema of the point (or the edge), the attribute data updating operation of the graph database participates in the synchronous mode among the server nodes of each party to update the attribute list of the Schema;
When a Schema of points (or edges) is updated (e.g., new attributes are added or existing attribute definitions are deleted, which change the order of the attribute list), the parties involved in the distributed update operation synchronize the updated attribute list (including the subscript order information).
Through this step, each server participating node of the attribute data update of the distributed graph database system maintains the consistency of the operation period on the information of the attribute list and the sequence.
Step S2, updating the attribute data increment of the graph database by adopting a double-layer bitmap changing attribute coding mode based on the attribute definition list of the two parties, wherein the step comprises the following steps:
marking the changed attributes in an ordered attribute list in the map database Schema by adopting a first layer map;
Based on the changed attribute of the first layer bitmap mark, marking the attribute which is set as a null value or a default value in an ordered attribute list in the map database Schema by adopting a second layer bitmap;
Updating the attribute data of the points or edges of the graph database based on the changed attribute of the first layer graph mark and the attribute of the second layer graph mark.
Further, the marking, by using a first layer modification bitmap, the changed attribute in the ordered attribute list in the map database Schema includes:
calculating the number of bits by using one bit to represent one attribute according to the number of all attributes of the Schema, and aligning by using one byte to obtain the minimum byte number representing the number of all attributes;
setting subscripts for each bit starting from 0 from left to right, each bit representing an attribute;
If a certain attribute has been changed during the update operation, the bit value of the certain attribute is 1, otherwise, it is 0.
Further, the marking, by using a second layer bitmap, the attribute set to a null value or a default value in the ordered attribute list in the map database Schema based on the changed attribute of the first layer bitmap marking includes:
For a first layer bitmap, introducing subscripts into each bit with the position of 1, wherein the subscripts are calculated from 0;
According to the number of bits of 1 in the first layer map, aligning the bits by 8 bits, and calculating the byte number as the byte number of the second layer map;
And when the update value of a certain bit is set to be a NULL/NULL value, setting the value of the bit corresponding to the subscript of the first layer bitmap in the second layer bitmap to be 1.
Further, the updating the attribute data of the point or the edge of the graph database based on the changed attribute of the first layer graph mark and the attribute of the second layer graph mark includes:
Only the change attribute marked by the first layer bitmap and the attribute data of the point or edge of the graph database which is not marked in the second layer bitmap are updated.
Step S2 is further described below:
When the point and edge increment attribute modification value starts to be transmitted, a modification attribute coding mode based on a double-layer bitmap is introduced.
On the basis of the steps, each time the updating operation of the point and side data is carried out, the distributed graph database carries out the coding of the 'updated attribute' based on the 'bitmap':
for a specific graph database Schema, introducing a first layer 'change bitmap' based on an ordered attribute list thereof;
according to the number of all the attributes of the Schema, representing one attribute by one bit, calculating the number of bits, and aligning by one byte (8 bits), so as to obtain the minimum byte number capable of representing the number of all the attributes;
From left to right, each bit represents an attribute, with the leftmost bit (subscript of bit can be considered as 0) representing an attribute subscript of 0, the leftmost second bit (subscript of bit can be considered as 1) representing an attribute subscript of 1, and so on;
if an attribute has changed during an update operation, the bit representing the attribute has a value of 1, otherwise it is set to 0. Thus, only one Bit in this "bitmap" (i.e., 1 Bit,8 bits forming a Byte) identifies which attribute is a point (or edge) element that has been updated.
It should be noted that setting a NULL value (or NULL value in some databases, and some databases may set a default value for the attribute set to NULL value) is a special operation, and for this scenario, the present invention introduces a second layer "bitmap"
In the first layer bitmap, each bit with the position of "1" has the sequence, and a subscript is conceptually introduced for each bit with the position of "1" in the first layer bitmap and the subscript is calculated from 0;
According to the number of bits set as 1 in one layer of bitmap, aligning with 8 bits (1 byte), calculating the required byte number as the actual required byte number of the second layer of bitmap;
setting the value of the subscript of the second layer bitmap corresponding to the subscript of the first layer bitmap to be "1", that is, the subscript of the second layer bitmap corresponding to the "value of" 1 "in the first layer bitmap, if the update value of a certain bit is set to be a NULL/NULL value, wherein" set to be "1" in the second layer bitmap represents that the update value of the attribute is a NULL value;
Based on the above first and second horizon map data:
Only focusing on the attribute of 1 in the first horizon map, and only transmitting the corresponding attribute value, and not transmitting the attribute key (attribute name) of the attribute;
If the updated attribute of the point (or edge) is set to 1 in a layer bitmap and is also set to 1 in a corresponding layer bitmap, indicating that the attribute is set to a NULL value (or default value), the updated value of the attribute does not need to be transmitted;
the end result is that for the update operation of the point (or edge) attribute, only the attribute corresponding to the bit in the first layer bitmap with bit 1 and the bit in the second layer bitmap with bit not 1 is transmitted;
combining the two-layer bitmap and the attribute data transmission strategy corresponding to the attribute change of a certain point (or edge):
The transmitted body data includes first horizon map data, second horizon map data, binary data of values of one or more attributes of the modification (attributes modified to null have been removed and transmission of attribute names has been avoided)
And auxiliary information such as the number of bytes of the first layer bitmap, the number of bytes of the second layer bitmap, the byte length of the modified attribute binary data, etc
Based on the two-level graph technology in the step, when the attribute change data transmission of the point (or the edge) is carried out, only the changed attribute is required to be transmitted, the transmission of an attribute key/attribute name is avoided, and only the identification of the changed attribute expressed by a plurality of bits (one bit represents 1/8 byte) is introduced, so that the data amount of transmission is saved.
According to the distributed graph database attribute data updating method, based on an increment attribute transmission mechanism on a RAFT consistency protocol and distributed graph database attribute transmission coding based on a bitmap, transmission of unchanged attribute values is avoided in the attribute updating process of graph data points (or edges), transmission of redundant keys (attribute names) is avoided, and attribute data transmission efficiency is improved.
The invention introduces an increment consistency transmission mechanism of the graph point and edge attribute data based on the RAFT distributed protocol, avoids the transmission of unchanged attribute data, ensures the consistency of processing changed attributes by the two parties when the attribute changed data is transmitted by the double-layer bitmap based coding mode of changing the attribute.
Example 2:
The invention also provides a distributed graph database attribute data updating system based on the same inventive concept, which comprises:
The consistency definition module is used for synchronously transmitting attribute definition lists of the two parties involved in the transmission based on the RAFT distributed protocol, and keeping that the two parties of the transmission have consistent attribute definitions;
and the mark updating module is used for updating the attribute data increment of the graph database by adopting a double-layer bitmap changing attribute coding mode based on the attribute definition lists of the two parties.
Optionally, the mark updating module includes:
The first marking submodule is used for marking the changed attributes in the ordered attribute list in the map database Schema by adopting a first layer bitmap;
The second marking submodule is used for marking the attribute which is set as a null value or a default value in an ordered attribute list in the map database Schema by adopting a second layer bitmap based on the changed attribute of the first layer bitmap marking;
And the updating sub-module is used for updating the attribute data of the points or edges of the graph database based on the changed attribute of the first layer bitmap mark and the attribute of the second layer bitmap mark.
Optionally, the first marking submodule is specifically configured to:
calculating the number of bits by using one bit to represent one attribute according to the number of all attributes of the Schema, and aligning by using one byte to obtain the minimum byte number representing the number of all attributes;
setting subscripts for each bit starting from 0 from left to right, each bit representing an attribute;
If a certain attribute has been changed during the update operation, the bit value of the certain attribute is 1, otherwise, it is 0.
Optionally, the second marking submodule is specifically configured to:
For a first layer bitmap, introducing subscripts into each bit with the position of 1, wherein the subscripts are calculated from 0;
According to the number of bits of 1 in the first layer map, aligning the bits by 8 bits, and calculating the byte number as the byte number of the second layer map;
And when the update value of a certain bit is set to be a NULL/NULL value, setting the value of the bit corresponding to the subscript of the first layer bitmap in the second layer bitmap to be 1.
Optionally, the update submodule is specifically configured to:
Only the change attribute marked by the first layer bitmap and the attribute data of the point or edge of the graph database which is not marked in the second layer bitmap are updated.
Optionally, the consistency definition module is specifically configured to:
The distributed graph database acquires the attribute list of the Schema corresponding to a certain point or edge type through the synchronous operation of the attribute list in the RAFT distributed protocol;
the operation participation parties of the distributed graph database update the attribute list in a synchronous mode in the RAFT distributed protocol, and the transmission parties are kept to have consistent attribute definition.
Example 3:
Based on the same inventive concept, the invention also provides a computer device comprising a processor and a memory for storing a computer program comprising program instructions, the processor for executing the program instructions stored by the computer storage medium. The processor may be a central processing unit (Central Processing Unit, CPU), but may also be other general purpose processor, digital signal processor (DIGITAL SIGNAL Processor, DSP), application specific integrated circuit (Application SpecificIntegrated Circuit, ASIC), off-the-shelf Programmable gate array (Field-Programmable GATEARRAY, FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc., which are the computational core and control core of the terminal adapted to implement one or more instructions, in particular to load and execute one or more instructions within a computer storage medium to implement the corresponding method flow or corresponding functions to implement the steps of a distributed map database attribute data updating method in the above embodiments.
Example 4:
Based on the same inventive concept, the present invention also provides a storage medium, in particular, a computer readable storage medium (Memory), which is a Memory device in a computer device, for storing programs and data. It is understood that the computer readable storage medium herein may include both built-in storage media in a computer device and extended storage media supported by the computer device. The computer-readable storage medium provides a storage space storing an operating system of the terminal. Also stored in the memory space are one or more instructions, which may be one or more computer programs (including program code), adapted to be loaded and executed by the processor. The computer readable storage medium herein may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one magnetic disk memory. One or more instructions stored in a computer-readable storage medium may be loaded and executed by a processor to implement the steps of a distributed graph database attribute data update method in the above embodiments.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing is illustrative of the present invention and is not to be construed as limiting thereof, but rather as providing for the use of additional embodiments within the spirit and scope of the present invention.
Claims (10)
1. A distributed graph database attribute data updating method, comprising:
synchronously transmitting attribute definition lists of the two parties participating in the transmission based on the RAFT distributed protocol, and keeping the attribute definition of the two parties transmitting consistent;
and updating the attribute data increment of the graph database by adopting a double-layer bitmap changing attribute coding mode based on the attribute definition lists of the two parties.
2. The method of claim 1, wherein updating the attribute data delta of the graph database by using a modified attribute coding manner of a double-layer bitmap based on the attribute definition list of the two parties comprises:
marking the changed attributes in an ordered attribute list in the map database Schema by adopting a first layer map;
Based on the changed attribute of the first layer bitmap mark, marking the attribute which is set as a null value or a default value in an ordered attribute list in the map database Schema by adopting a second layer bitmap;
Updating the attribute data of the points or edges of the graph database based on the changed attribute of the first layer graph mark and the attribute of the second layer graph mark.
3. The method of claim 2, wherein marking the changed attributes in the ordered list of attributes in the graph database Schema with a first layer change bitmap comprises:
calculating the number of bits by using one bit to represent one attribute according to the number of all attributes of the Schema, and aligning by using one byte to obtain the minimum byte number representing the number of all attributes;
setting subscripts for each bit starting from 0 from left to right, each bit representing an attribute;
If a certain attribute has been changed during the update operation, the bit value of the certain attribute is 1, otherwise, it is 0.
4. The method of claim 3, wherein marking the attributes set to null or default values in the ordered list of attributes in the graph database Schema with a second layer bitmap based on the changed attributes of the first layer bitmap marking comprises:
For a first layer bitmap, introducing subscripts into each bit with the position of 1, wherein the subscripts are calculated from 0;
According to the number of bits of 1 in the first layer map, aligning the bits by 8 bits, and calculating the byte number as the byte number of the second layer map;
And when the update value of a certain bit is set to be a NULL/NULL value, setting the value of the bit corresponding to the subscript of the first layer bitmap in the second layer bitmap to be 1.
5. The method of claim 2, wherein the updating of the attribute data of the points or edges of the graph database based on the changed attribute of the first layer bitmap marking and the attribute of the second layer bitmap marking comprises:
Only the change attribute marked by the first layer bitmap and the attribute data of the point or edge of the graph database which is not marked in the second layer bitmap are updated.
6. The method of claim 1, wherein the synchronizing transmission of the list of attribute definitions of the two parties to the transmission based on the RAFT distributed protocol maintains the two parties to the transmission to have consistent attribute definitions, comprising:
The distributed graph database acquires the attribute list of the Schema corresponding to a certain point or edge type through the synchronous operation of the attribute list in the RAFT distributed protocol;
the operation participation parties of the distributed graph database update the attribute list in a synchronous mode in the RAFT distributed protocol, and the transmission parties are kept to have consistent attribute definition.
7. A distributed graph database attribute data update system, comprising:
The consistency definition module is used for synchronously transmitting attribute definition lists of the two parties involved in the transmission based on the RAFT distributed protocol, and keeping that the two parties of the transmission have consistent attribute definitions;
and the mark updating module is used for updating the attribute data increment of the graph database by adopting a double-layer bitmap changing attribute coding mode based on the attribute definition lists of the two parties.
8. The system of claim 7, wherein the tag update module comprises:
The first marking submodule is used for marking the changed attributes in the ordered attribute list in the map database Schema by adopting a first layer bitmap;
The second marking submodule is used for marking the attribute which is set as a null value or a default value in an ordered attribute list in the map database Schema by adopting a second layer bitmap based on the changed attribute of the first layer bitmap marking;
And the updating sub-module is used for updating the attribute data of the points or edges of the graph database based on the changed attribute of the first layer bitmap mark and the attribute of the second layer bitmap mark.
9. A computer device, comprising: one or more processors;
the processor is used for storing one or more programs;
A distributed graph database attribute data updating method as claimed in any one of claims 1 to 6 when said one or more programs are executed by said one or more processors.
10. A computer readable storage medium, having stored thereon a computer program which, when executed, implements a distributed graph database attribute data updating method according to any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311713338.7A CN120144596A (en) | 2023-12-13 | 2023-12-13 | A distributed graph database attribute data updating method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311713338.7A CN120144596A (en) | 2023-12-13 | 2023-12-13 | A distributed graph database attribute data updating method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN120144596A true CN120144596A (en) | 2025-06-13 |
Family
ID=95948005
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311713338.7A Pending CN120144596A (en) | 2023-12-13 | 2023-12-13 | A distributed graph database attribute data updating method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN120144596A (en) |
-
2023
- 2023-12-13 CN CN202311713338.7A patent/CN120144596A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Almeida et al. | Efficient state-based crdts by delta-mutation | |
| CN111386522B (en) | System and method for data storage | |
| US9418130B2 (en) | Data replication protocol with efficient update of replica machines | |
| CN111881223B (en) | Data management method, device, system and storage medium | |
| US11314717B1 (en) | Scalable architecture for propagating updates to replicated data | |
| US10936559B1 (en) | Strongly-consistent secondary index for a distributed data set | |
| US9081843B2 (en) | Data replication protocol with steady state data distribution and quorum formation | |
| US20220224610A1 (en) | Heartbeat information sending method and apparatus, and heartbeat sending node | |
| CN111506592B (en) | Database upgrading method and device | |
| CN113434500B (en) | Table connection method, apparatus, distributed database system, server and medium | |
| Krechowicz et al. | Highly scalable distributed architecture for NoSQL datastore supporting strong consistency | |
| CN104298598A (en) | Debugging method for RDFS (Resource Description Framework Schema) body in distributed environment | |
| CN116701414B (en) | Blockchain-based data processing method, device, equipment, and readable storage medium | |
| CN117131023B (en) | Data table processing method, device, computer equipment and readable storage medium | |
| CN117193671B (en) | Data processing method, device, computer equipment and computer readable storage medium | |
| CN120144596A (en) | A distributed graph database attribute data updating method and system | |
| CN117271530B (en) | A method and device for modeling pragmatic relationships of digital objects in digital networking | |
| CN111782619A (en) | A method, synchronization device and storage medium for incremental synchronization of documents between servers | |
| CN107832121B (en) | Concurrency control method applied to distributed serial long transactions | |
| CN117076470A (en) | Capacity expansion method, device, equipment, medium and program product for financial database | |
| US11036733B2 (en) | Method, apparatus, system, server, and storage medium for connecting tables stored at distributed database | |
| JP6180710B2 (en) | Data storage method and apparatus | |
| Rimey | Version headers for flexible synchronization and conflict resolution | |
| CN117390078B (en) | Data processing method, device, storage medium and computer equipment | |
| Zeng et al. | SLSM: an efficient strategy for lazy schema migration on shared-nothing databases |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |