CN118171321B - A method, device, equipment and medium for reading data from a database cluster - Google Patents
A method, device, equipment and medium for reading data from a database clusterInfo
- Publication number
- CN118171321B CN118171321B CN202410426479.9A CN202410426479A CN118171321B CN 118171321 B CN118171321 B CN 118171321B CN 202410426479 A CN202410426479 A CN 202410426479A CN 118171321 B CN118171321 B CN 118171321B
- Authority
- CN
- China
- Prior art keywords
- node
- request
- data
- read
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Storage Device Security (AREA)
Abstract
The invention discloses a data reading method, device, equipment and medium of a database cluster. The method comprises the steps that a request node initiates a data reading request to a data page to be read, a global sharing blocking request is initiated to a global cache service based on the data reading request, so that the global cache service initiates an authority recycling request to a modification node, when the modification node receives the authority recycling request, data information of the data page to be read is sent according to node information of the request node, first response information is sent to the global cache service, the global cache service receives the first response information and sends authorization response information to the request node, the request node receives the data information of the data page to be read and the authorization response information in parallel, and the data information of the data page to be read is correspondingly processed according to a receiving sequence. According to the technical scheme, the data information of the data page is transmitted and the response authority recovery and the authorization processing operation of the request node are executed in parallel, so that the total execution time of data remote reading is shortened.
Description
Technical Field
The present invention relates to the field of cluster technologies, and in particular, to a method, an apparatus, a device, and a medium for reading data of a database cluster.
Background
In a shared storage database cluster, all read-write operations require access to the latest data page, global S (share) blocking is required before the data page is read, global X (Exclusive) blocking is required before the data page is modified, and as long as the data page is modified at one node, other nodes will trigger permission recovery when accessing the data page, then initiate remote read operation, and transfer the latest data page content to the requesting node through the network.
Remote reading is an important component of the shared storage cluster for exchanging data pages, but if multiple nodes in the shared storage cluster frequently modify the same data page in parallel, remote reading actions are frequently triggered, the data reading time is long, and the performance of the shared storage cluster is reduced.
Disclosure of Invention
The invention provides a data reading method, a device, equipment and a medium of a database cluster, which further shorten the total execution time of data remote reading and improve the performance of the database cluster by sending data information of a data page and recovering response authority and authorizing processing operation to a request node to be executed in parallel.
According to one aspect of the invention, a data reading method of a database cluster is provided, the database cluster comprises a modification node and a request node, local cache service and global cache service are deployed on the modification node and the request node, wherein the global cache service is distributed to any node in the database cluster for management, and the method comprises the following steps:
the request node initiates a data reading request to a data page to be read, initiates a global sharing blocking request to the global cache service based on the data reading request, and enables the global cache service to initiate a permission recycling request to the modification node;
when the modification node receives the permission recovery request, sending data information of the data page to be read according to node information of the request node, processing the permission recovery request, and sending first response information to the global cache service;
the global cache service receives the first response information, gives the request node global sharing blocking authority to the data page to be read, and sends authorization response information to the request node;
The request node receives the data information of the data page to be read and the authorization response information in parallel, and correspondingly processes the data information of the data page to be read according to the receiving sequence of the data information and the authorization response information so as to complete the reading operation of the data page to be read.
Optionally, the request node receives the data information of the data page to be read and the authorization response information in parallel, and correspondingly processes the data information of the data page to be read according to the receiving sequence of the data information and the authorization response information, so as to complete the reading operation of the data page to be read, and the method includes:
If the request node receives the data information of the data page to be read before receiving the authorization response information, the request node stores the received data information of the data page to be read, waits for the request node to receive the authorization response information and initiates a local read request, and reads the data to be read;
And if the request node receives the data information of the data page to be read after receiving the authorization response information, the request node fills the data information of the data page to be read into a local memory area, and the reading operation of the data page to be read is completed.
Optionally, after the requesting node receives the authorization response information, the method further includes:
The local cache service of the request node obtains the global sharing blocking authority of the data page to be read, and updates the node information of the request node and records the node information of the modification node.
Optionally, initiating a global sharing lockout request to the global cache service based on the data read request, so that the global cache service initiates a rights reclamation request to the modification node based on the read request, including:
Initiating a global sharing blocking request to the global caching service through the local caching service of the requesting node;
and when the global cache service receives the global sharing blocking request, initiating a permission recycling request to the modification node.
Optionally, the node information of the request node includes a log serial number of the request node;
When the modification node receives the permission recovery request, sending the data information of the data page to be read according to the node information of the request node, wherein the data information comprises the following steps:
When the modification node receives the recovery permission request, comparing the log serial number of the request node with the local log serial number of the modification node;
and if the log serial number of the request node is smaller than the local log serial number of the modification node, sending the data information of the data page to be read to the request node.
Optionally, processing the rights recovery request includes:
and releasing the exclusive blocking authority of the local cache service of the modification node, and responding to the authority recycling request.
Optionally, before sending the data information of the data page to be read according to the node information of the requesting node, the method further includes:
acquiring a memory control structure of a data page to be read;
and determining the data information of the data page to be read according to the memory control structure.
According to another aspect of the present invention, there is provided a data reading apparatus of a database cluster, where the database cluster configures a modifying node and a requesting node, and local cache service and global cache service are deployed on the modifying node and the requesting node, where the global cache service is distributed to any node in the database cluster for management, and the apparatus includes:
The request initiating module is arranged at the request node and used for initiating a data reading request to a data page to be read, and initiating a global sharing blocking request to the global cache service based on the data reading request so that the global cache service initiates a permission recycling request to the modification node;
the request processing module is arranged on the modification node and is used for sending the data information of the data page to be read according to the node information of the request node when the modification node receives the permission recovery request, processing the permission recovery request and sending first response information to the global cache service;
The authorization response module is arranged on the global cache service, and is used for receiving the first response information, giving global sharing blocking authority to the data page to be read to the request node, and sending authorization response information to the request node;
the information receiving module is arranged at the request node and is used for receiving the data information of the data page to be read and the authorization response information in parallel, and correspondingly processing the data information of the data page to be read according to the receiving sequence of the data information and the authorization response information so as to complete the reading operation of the data page to be read.
According to another aspect of the present invention, there is provided an electronic apparatus including:
At least one processor, and
A memory communicatively coupled to the at least one processor, wherein,
The memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the method of reading data of a database cluster according to any one of the embodiments of the present invention.
According to another aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to implement a data reading method of a database cluster according to any embodiment of the present invention when executed.
According to the technical scheme, a data reading request is initiated to a data page to be read through a request node, a global sharing blocking request is initiated to a global cache service based on the data reading request, so that the global cache service initiates an authority recycling request to a modification node, wherein data information of the data page to be read is modified by the modification node, the authority recycling request comprises node information of the request node, when the modification node receives the authority recycling request, the data information of the data page to be read is sent according to the node information of the request node, the authority recycling request is processed, first response information is sent to the global cache service, the global cache service receives the first response information, gives the request node global sharing blocking of the data page to be read, and sends authorization response information to the request node, the request node receives the data information of the data page to be read and the authorization response information in parallel, and correspondingly processes the data information of the data page to be read according to the receiving sequence of the data information and the authorization response information, and the data page to be read is completed. According to the technical scheme, the data information of the data page is transmitted, the response authority recovery and the authorization processing operation of the request node are executed in parallel, so that the total execution time of data remote reading is further shortened, and the performance of the database cluster is improved.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a method for reading data from a database cluster according to a first embodiment of the present invention;
FIG. 2 is a flowchart of a method for reading data from a database cluster according to a second embodiment of the present invention;
FIG. 3 is a flowchart illustrating a method for reading data from a database cluster according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of a data reading device of a database cluster according to a fourth embodiment of the present invention;
Fig. 5 is a schematic structural diagram of an electronic device according to a fifth embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
Fig. 1 is a flowchart of a data reading method of a database cluster according to a first embodiment of the present invention, where the method may be applicable to a case of reading a latest data page in a shared storage database cluster, and the method may be performed by a data reading device of a database cluster, where the data reading device of a database cluster may be implemented in a form of hardware and/or software, and the data reading device of a database cluster may be configured in an electronic device with data processing capability. As shown in fig. 1, the method includes:
The database cluster in the embodiment comprises a modifying node and a requesting node, wherein local cache service and global cache service are deployed on the modifying node and the requesting node, and the global cache service is distributed to any node in the database cluster for management. The database cluster in this embodiment may be a shared storage database cluster. The shared storage database cluster may comprise at least two nodes, wherein a node modifying a certain data page in the database cluster may be considered as a modifying node, and the requesting node may be a node requesting access to a certain data page. The modifying node has global exclusive blocking authority of the modified data page. And the global cache service (Global Buffer Service, GBS) is used for globally managing the blocking request of the data pages, maintaining the data page blocking information and the latest data page distribution condition, wherein one GBS exists on each node in the database cluster, and the data pages are distributed to GBS management on any node according to the data page numbers. A local caching service (Local Buffer Service, LBS), which assists the GBS in handling data page lockout requests, maintains data page lockout information and up-to-date data page distribution locally at each node, one LBS on each node. In this embodiment, the global cache service may be allocated to a modifying node or a requesting node in the database cluster, or may be managed on a node other than the modifying node or the requesting node, which may be managed according to an actual situation.
S110, the request node initiates a data reading request to the data page to be read, and initiates a global sharing blocking request to the global cache service based on the data reading request, so that the global cache service initiates a permission recycling request to the modification node.
The data information of the data page to be read is modified by the modification node, and the permission recovery request comprises node information of the request node. The data page to be read is the latest data page modified by the modification node. The data read request is to access a data page to be read and read data information contained in the data page. The global share lockout request is sent by the requesting node to the global caching service. The rights reclamation request may be understood as reclaiming global exclusive blocking rights to the modifying node.
Because the modifying node has global exclusive blocking authority of the modified data page, the shared storage database cluster of the embodiment needs to initiate a global blocking request to the global cache service before reading the data page, so that the global cache service initiates an authority recovery request to the modifying node.
And S120, when the modification node receives the permission recycling request, sending data information of the data page to be read according to the node information of the request node, processing the permission recycling request, and sending first response information to the global cache service.
The node information of the requesting node may include a node number and a log serial number of the requesting node. The log sequence number (Log Sequence Number, LSN) corresponds to an LSN value that monotonically increases for each modified data page. The data information of the data page to be read may be the latest data information contained in the data page.
In this embodiment, whether to send the data information of the data page to be read may be compared and judged according to the node information sent by the requesting node and the data information of the modifying node. Processing the rights reclamation request in this embodiment may be an operation to cancel the exclusive lock right of the local caching service. The first response information is feedback information for the permission recycling request of the global cache service. The first response information may include a data page latest LSN value, a node number of the modified data page corresponding to the data page latest LSN value, and a node number of the transmitted data page. Wherein the data page latest LSN value is generated by the node that last modified the data page.
In this embodiment, when the modification node receives the permission recycling request sent by the global cache service, according to node information of the request node carried by the permission recycling request, it is determined whether to send data information of a data page to be read to the request node, and the modification node responds to the permission recycling request and sends first response information.
In this embodiment, optionally, processing the rights reclamation request includes releasing the exclusive blocking rights of the local cache service of the modifying node in response to the rights reclamation request.
The exclusive blocking authority is the authority of the modification node to the data page to be read. The specific operation of processing the rights recovery request in this embodiment may be to release the exclusive blocking rights of the local cache service of the modification node in response to the rights recovery request. Through the arrangement, the exclusive blocking authority can be recovered through the request of the global cache service, so that the request node can conveniently read the data page to be read, and the latest data content can be obtained.
In this embodiment, optionally, before sending the data information of the data page to be read according to the node information of the request node, the method further includes obtaining a memory control structure of the data page to be read, and determining the data information of the data page to be read according to the memory control structure.
The memory control structure may be a memory control structure in which a data page to be read is in a memory area buffer of the cluster.
In this embodiment, before the data information of the data page to be read is sent according to the node information of the requesting node, the memory control structure of the data page to be read may be obtained, and the specific position of the data information of the data page to be read may be determined according to the obtained memory control structure, so that the content of the data page to be read may be sent in advance before the local cache service of the modifying node responds to the permission recovery request of the global cache service, which is more convenient and faster.
In this embodiment, optionally, when the modification node receives the permission recycling request, the data information of the data page to be read is sent according to the node information of the request node, which includes comparing the log serial number of the request node with the local log serial number of the modification node when the modification node receives the recycling permission request, and if the log serial number of the request node is smaller than the local log serial number of the modification node, sending the data information of the data page to be read to the request node.
The node information of the request node includes a log serial number of the request node, and may further include information such as a node number of the request node. The log sequence number of the requesting node may be the LSN value of the data page of the requesting node in the local buffer. The local log sequence number may be the LSN value of the local memory region data page of the modifying node.
In this embodiment, when the modification node receives the permission recovery request, the log sequence number of the request node and the local log sequence number of the modification node are compared according to the permission recovery request, if the log sequence number of the request node is smaller than the local log sequence number of the modification node, it is indicated that the data page information of the current modification node is up-to-date, the data information of the data page to be read is sent to the request node in advance, and if the log sequence number of the request node is not smaller than the local log sequence number of the modification node, the data information of the data page to be read does not need to be sent in advance.
By such arrangement in this embodiment, the content of the data page can be determined to be sent in advance by comparing the sizes of the log sequence numbers, which is beneficial to improving the reading efficiency of the data page.
S130, the global cache service receives the first response information, gives the request node global sharing blocking authority for the data page to be read, and sends authorization response information to the request node.
The global sharing blocking authority may be an access limit set on a data page to be read, and it may be ensured that a node requesting access can access the data page to be read. The authorization response information can be response information giving global sharing blocking authority to the request node, and the authorization response information can carry node information for sending the data page.
In this embodiment, the global cache service receives the first response information sent by the modifying node, gives the requesting node global sharing blocking authority for the data page to be read, sends authorization response information to the requesting node, updates the data page information corresponding to the global cache service, and adjusts the node information including the latest data page in the local memory area buffer.
And S140, the request node receives the data information and the authorization response information of the data page to be read in parallel, and correspondingly processes the data information of the data page to be read according to the receiving sequence of the data information and the authorization response information so as to complete the reading operation of the data page to be read.
The receiving order may be a receiving order of both the data information and the authorization response information of the data page to be read. In this embodiment, the requesting node receives the data information and the authorization response information of the data page to be read in parallel, and may perform different processing on the data information of the data page to be read according to the sequence of receiving the authorization response information before and after the data information of the data page to be read, so as to complete the reading operation of the data page to be read.
According to the technical scheme, a data reading request is initiated to a data page to be read through a request node, a global sharing blocking request is initiated to a global cache service based on the data reading request, so that the global cache service initiates an authority recycling request to a modification node, wherein data information of the data page to be read is modified by the modification node, the authority recycling request comprises node information of the request node, when the modification node receives the authority recycling request, the data information of the data page to be read is sent according to the node information of the request node and is processed, first response information is sent to the global cache service, the global cache service receives the first response information, gives the global sharing blocking authority of the data page to be read to the request node, and sends authorization response information to the request node, and the request node receives the data information and the authorization response information of the data page to be read in parallel and correspondingly processes the data information of the data page to be read according to the receiving sequence of the data information and the authorization response information, so that the reading operation of the data page to be read is completed. According to the technical scheme, the data information of the data page is transmitted, the response authority recovery and the authorization processing operation of the request node are executed in parallel, so that the total execution time of data remote reading is further shortened, and the performance of the database cluster is improved.
Example two
Fig. 2 is a flowchart of a data reading method of a database cluster according to a second embodiment of the present invention, which is optimized based on the foregoing embodiment. The request node receives the data information and the authorization response information of the data page to be read in parallel, and correspondingly processes the data information of the data page to be read according to the receiving sequence of the data information and the authorization response information so as to complete the reading operation of the data page to be read; and if the request node receives the data information of the data page to be read and after receiving the authorization response information, the request node fills the data information of the data page to be read into the local memory area to finish the reading operation of the data page to be read. As shown in fig. 2, the method includes:
s210, the request node initiates a data reading request to the data page to be read, and initiates a global sharing blocking request to the global cache service based on the data reading request, so that the global cache service initiates a permission recycling request to the modification node.
The data information of the data page to be read is modified by the modification node, and the permission recovery request comprises node information of the request node.
In this embodiment, optionally, a global sharing lockout request is initiated to the global cache service based on the data read request, so that the global cache service initiates a permission recycling request to the modification node based on the read request, including initiating the global sharing lockout request to the global cache service through the local cache service of the requesting node, and initiating the permission recycling request to the modification node when the global cache service receives the global sharing lockout request.
In the embodiment, a request node initiates a global sharing blocking request to a global caching service through a local caching service to obtain the access authority of a data page to be read, and when the global caching service receives the global sharing blocking request sent by the local caching service of the request node, the request node initiates an authority recovery request to a modification node, and the authority recovery request carries a request node number and a log serial number of the request node. By such arrangement in this embodiment, the global cache service may send an authority recovery request to the modifying node according to the received global sharing blocking request, so as to recover the global exclusive blocking authority of the modifying node, so that the subsequent requesting node may read the data page.
S220, when the modification node receives the permission recovery request, sending data information of a data page to be read according to node information of the request node, processing the permission recovery request, and sending first response information to the global cache service;
s230, the global cache service receives the first response information, gives the request node global sharing blocking authority of the data page to be read, and sends authorization response information to the request node;
s240, if the request node receives the data information of the data page to be read before receiving the authorization response information, the request node stores the received data information of the data page to be read, waits for the request node to receive the authorization response information and initiates a local read request, and reads the data to be read.
The local read request may be a request initiated by the requesting node to read the received data page to be read. In this embodiment, since the request node receives the data information of the data page to be read and the authorization response information in parallel, if the request node receives the data information of the data page to be read before receiving the authorization response information, that is, when the request node receives the data information of the data page to be read and does not receive the authorization response information, it is indicated that the request node may store the received data information of the data page to be read before the request node does not obtain the authorization of the global cache service, and wait for the request node to initiate a local read request after receiving the authorization response information, and perform a read operation on the data to be read.
S250, if the request node receives the data information of the data page to be read and after receiving the authorization response information, the request node fills the data information of the data page to be read into the local memory area, and the reading operation of the data page to be read is completed.
The local memory area may be a memory control structure in a local memory area buffer of the requesting node. In this embodiment, if the data information of the data page to be read is received after the authorization response information is received, that is, the request node receives the authorization response information first, which indicates that the request node has been authorized by the global cache service, and after the data information of the data page to be read is received after the authorization, the request node may directly read the data information of the data page, and fill the data information into the local memory area of the request node, so as to complete the reading operation of the data page to be read.
By the arrangement, the request node can process the data information of the received data page in different modes before and after being authorized, so that the data reading efficiency is further improved, and the overall execution time of remote reading is shortened.
In this embodiment, optionally, after the requesting node receives the authorization response information, the method further includes the steps that the local cache service of the requesting node obtains global sharing blocking authority of the data page to be read, and updates node information of the requesting node and node information of the record modification node.
The node information may include node number, log sequence number, and node information of the local memory area including the latest data page. In this embodiment, after receiving the authorization response information, the local caching service obtains the global sharing blocking authority of the data page to be read, and updates the storage node information of the requesting node and registers the modified node for sending the data page in the local caching service. Through the arrangement, the request node can obtain the corresponding access right, and update each node information in the local cache service, so that the data content of the data page can be shared conveniently.
According to the technical scheme, a data reading request is initiated to a data page to be read through a request node, a global sharing blocking request is initiated to a global cache service based on the data reading request, so that the global cache service initiates an authority recycling request to a modification node, wherein the data information of the data page to be read is modified by the modification node, the authority recycling request comprises node information of the request node, when the modification node receives the authority recycling request, the data information of the data page to be read is sent according to the node information of the request node, the authority recycling request is processed, first response information is sent to the global cache service, the global cache service receives the first response information, gives the global sharing blocking authority of the data page to be read to the request node, and sends the authority response information to the request node, if the request node receives the data information of the data page to be read before receiving the authority response information, the request node stores the received data information of the data page to be read, and waits for the request node to receive the local reading request, and reads the data to be read after the authority response information is received, and the request node fills the data page to be read to the local memory area of the data page to be read. According to the technical scheme, the data information of the data page is transmitted, the response authority recovery and the authorization processing operation of the request node are executed in parallel, so that the total execution time of data remote reading is further shortened, and the performance of the database cluster is improved.
Example III
Fig. 3 is a flowchart illustrating a data reading method of a database cluster according to a third embodiment of the present invention. Note that, the present invention is not limited to the above-described embodiments. Log sequence numbers (Log Sequence Number, LSNs), one LSN value for each modified data page. disc LSN is the LSN value of the data page on disk. Page_ LSN is the LSN value of the data page in the local buffer. fresh_ LSN is the latest LSN value for the data page, generated by the node that last modified the data page. fresh_ep is the node number of the modified data page corresponding to fresh_ lsn. The access_map is node information of the latest data page contained in the local buffer. Global caching service (Global Buffer Service, GBS), global management of data page lockout requests, maintenance of data page lockout information and latest data page distribution, one GBS exists on each node, and data pages are allocated to GBS management on one node according to data page numbers. A local caching service (Local Buffer Service, LBS), which assists the GBS in handling data page lockout requests, maintains data page lockout information locally at nodes, and up-to-date data page distribution, one LBS on each node. The buf_ctl is a memory control structure in the memory area buffer for the data page. lbs_ctl is the memory control structure of the data page in LBS. The global S blocking authority is global sharing blocking authority. Global X lock is global exclusive lock authority.
As shown in fig. 3, the database cluster includes three nodes, EP0, EP1, and EP2, respectively. Disc_ lsn of data page P1 is 1000, EP0 has the X lock authority of data page P1, EP0 modifies data page P1, and page_ lsn is 2000. Assume that the GBS to which data page P1 belongs is on EP2. The modifying node EP0 modifies the data page P1, and requests the node EP1 to access the data page P1 to be read to initiate a remote read process as follows:
first, EP1 initiates a global S lockout request to the GBS through LBS.
Secondly, the GBS receives the LBS blocking request of the EP1, initiates a recovery authority request to the EP0, and appends the request node number EP1 and page_ lsn information of the request node data page in the message.
Third, EP0 cancels the local LBS lockout information after receiving the GBS permission recovery request, compares the size of EP 1's page_ lsn with EP 0's local page_ lsn, if EP 1's page_ lsn is smaller than EP 0's local page_ lsn, sends data pages to EP1 in advance, responds to GBS and sends first response information, appends fresh_ep (EP 0), fresh_ lsn (2000) information in the message, and appends node information (EP 1) of the sent data pages.
Fourth, the GBS updates fresh_ep (EP 0), fresh_ lsn (2000) and access_map (EP 0) after receiving the first response message of EP0, grants global S lock to the data page P1 by EP1, adjusts the access_map to (EP 0, EP 1), and transmits authorization response information to the requesting node EP1, wherein node information (EP 0) for transmitting the data page is added to the authorization response information.
Fifth, after receiving the GBS grant response message, EP1 updates fresh_ep (EP 0), fresh_ lsn (2000) and access_map (EP 0, EP 1), the local LBS obtains global S lockout grant of P1, and the local LBS registration of EP1 sends data page node information (EP 0).
Sixth, the EP1 receives the data page content sent by the EP0, receives the data page content sent by the EP0 and the authorization response information sent by the GBS in parallel, and after receiving the authorization response information as GBS authorization, does not receive the authorization response information before GBS authorization.
The embodiment processes the data page read request according to the condition before GBS authorization or after GBS authorization, specifically, before GBS authorization, the received data page content can be stored first, then the local buffer is waited for initiating the data page read request, after GBS authorization, the data page content is directly filled into the buf_ctl, and the read operation of the data page is completed.
In addition, in this embodiment, EP1 initiates a local buffer access, registers the transmission data page node EP0 through lbs_ctl, and waits for EP0 to transmit the data page content (sixth).
Since in the shared storage cluster, once a data page is modified, neither buf_ctl nor lbs_ctl of the data page is retired. The recovered authority node modifies lbs_ctl object of the data page in LBS of the node, buf_ctl object of the data page in buffer is necessarily present and contents of the data page in buffer are up to date. Therefore, in this embodiment, before the LBS responds to the GBS authority recovery request, the modifying node may acquire the buf_ctl object, and send the data page content to the requesting node in advance, and after the LBS lockout requesting node receives the GBS authority response message, it is not necessary to send a remote read request to the fresh_ep, and only needs to directly wait for the data page message and complete the IO. Through the optimization of the reading method in this embodiment, the network request for remotely reading the remote read is reduced to 5 times, and the data information (③ in fig. 3) of the data page and the response information (④ and ⑤ in fig. 3) for responding to the GBS authority recovery and the GBS authorization are executed in parallel, so that the overall execution time of the remote read is further shortened, and the performance of the database cluster is improved.
Example IV
Fig. 4 is a schematic structural diagram of a data reading device of a database cluster according to a fourth embodiment of the present invention. The database cluster configures a modifying node and a requesting node, local cache service and global cache service are deployed on the modifying node and the requesting node, wherein the global cache service is distributed to any node in the database cluster for management, and as shown in fig. 4, the device comprises:
The request initiating module 410 is configured to initiate a data reading request to a data page to be read, and initiate a global sharing blocking request to a global cache service based on the data reading request, so that the global cache service initiates a permission recycling request to a modifying node, wherein the data information of the data page to be read is modified by the modifying node, and the permission recycling request includes node information of the requesting node;
The request processing module 420 is configured to, when the modification node receives the permission recovery request, send data information of a data page to be read according to node information of the request node, process the permission recovery request, and send first response information to the global cache service;
The authorization response module 430 is configured to receive the first response information, give the request node global sharing blocking authority to the data page to be read, and send authorization response information to the request node;
The information receiving module 440 is disposed at the request node, and is configured to receive the data information and the authorization response information of the data page to be read in parallel, and correspondingly process the data information of the data page to be read according to the receiving sequence of the data information and the authorization response information, so as to complete the reading operation of the data page to be read.
Optionally, the information receiving module 440 is configured to be disposed at the requesting node, and specifically configured to store the received data information of the data page to be read if the data information of the data page to be read is received before the authorization response information is received, wait for the requesting node to receive the authorization response information and initiate a local read request, and perform a read operation on the data to be read, and fill the data information of the data page to be read into the local memory area if the data information of the data page to be read is received by the requesting node after the authorization response information is received, so as to complete the read operation on the data page to be read.
Optionally, the device further includes a rights item acquisition module, configured to be disposed at the requesting node, and configured to, after the requesting node receives the authorization response information, obtain global sharing blocking rights of the data page to be read by a local cache service of the requesting node, and update node information of the requesting node and record node information of the modifying node.
Optionally, the request initiating module 410 is configured to be disposed at the requesting node, and specifically configured to initiate a global sharing blocking request to the global cache service through a local cache service of the requesting node;
When the global cache service receives the global sharing lockout request, a permission recycling request is initiated to the modification node.
Optionally, the node information of the request node includes a log serial number of the request node, a request processing module 420, configured to, when the modification node receives the reclamation authority request, compare the log serial number of the request node with a local log serial number of the modification node, and if the log serial number of the request node is smaller than the local log serial number of the modification node, send data information of a data page to be read to the request node.
Optionally, the request processing module 420 is disposed at the modifying node, and is specifically configured to release the exclusive blocking authority of the local cache service of the modifying node, so as to respond to the authority recovery request.
Optionally, the data information determining module is arranged at the modifying node and is used for acquiring the memory control structure of the data page to be read before sending the data information of the data page to be read according to the node information of the requesting node, and determining the data information of the data page to be read according to the memory control structure.
The data reading device of the database cluster provided by the embodiment of the invention can execute the data reading method of the database cluster provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example five
Fig. 5 is a schematic structural diagram of an electronic device according to a fifth embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 5, the electronic device 10 includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, in which the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 may also be stored. The processor 11, the ROM 12 and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
Various components in the electronic device 10 are connected to the I/O interface 15, including an input unit 16, such as a keyboard, mouse, etc., an output unit 17, such as various types of displays, speakers, etc., a storage unit 18, such as a magnetic disk, optical disk, etc., and a communication unit 19, such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as the data reading method of a database cluster.
In some embodiments, the data reading method of the database cluster may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the data reading method of the database cluster described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the data reading method of the database cluster in any other suitable way (e.g. by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be a special or general purpose programmable processor, operable to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user, for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a Local Area Network (LAN), a Wide Area Network (WAN), a blockchain network, and the Internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (9)
1. A data reading method of a database cluster is characterized in that the database cluster comprises a modification node and a request node, wherein local cache service and global cache service are deployed on the modification node and the request node, and the global cache service is distributed to any node in the database cluster for management, and the method comprises the following steps:
the request node initiates a data reading request to a data page to be read, initiates a global sharing blocking request to the global cache service based on the data reading request, and enables the global cache service to initiate a permission recycling request to the modification node;
when the modification node receives the permission recovery request, sending data information of the data page to be read according to node information of the request node, processing the permission recovery request, and sending first response information to the global cache service;
the global cache service receives the first response information, gives the request node global sharing blocking authority to the data page to be read, and sends authorization response information to the request node;
The request node receives the data information of the data page to be read and the authorization response information in parallel, and correspondingly processes the data information of the data page to be read according to the receiving sequence of the data information and the authorization response information so as to complete the reading operation of the data page to be read;
the request node receives the data information of the data page to be read and the authorization response information in parallel, and correspondingly processes the data information of the data page to be read according to the receiving sequence of the data information and the authorization response information so as to complete the reading operation of the data page to be read, and the method comprises the following steps:
If the request node receives the data information of the data page to be read before receiving the authorization response information, the request node stores the received data information of the data page to be read, waits for the request node to receive the authorization response information and initiates a local read request, and reads the data to be read;
And if the request node receives the data information of the data page to be read after receiving the authorization response information, the request node fills the data information of the data page to be read into a local memory area, and the reading operation of the data page to be read is completed.
2. The method of claim 1, further comprising, after the requesting node receives the authorization response information:
The local cache service of the request node obtains the global sharing blocking authority of the data page to be read, and updates the node information of the request node and records the node information of the modification node.
3. The method of claim 1, wherein initiating a global share lockout request to the global cache service based on the data read request such that the global cache service initiates a rights reclamation request to the modifying node based on the read request comprises:
Initiating a global sharing blocking request to the global caching service through the local caching service of the requesting node;
and when the global cache service receives the global sharing blocking request, initiating a permission recycling request to the modification node.
4. The method of claim 1, wherein the node information of the requesting node comprises a log sequence number of the requesting node;
When the modification node receives the permission recovery request, sending the data information of the data page to be read according to the node information of the request node, wherein the data information comprises the following steps:
When the modification node receives the recovery permission request, comparing the log serial number of the request node with the local log serial number of the modification node;
and if the log serial number of the request node is smaller than the local log serial number of the modification node, sending the data information of the data page to be read to the request node.
5. The method of claim 1, wherein processing the rights reclamation request comprises:
and releasing the exclusive blocking authority of the local cache service of the modification node, and responding to the authority recycling request.
6. The method according to claim 1, further comprising, before transmitting the data information of the data page to be read according to the node information of the requesting node:
acquiring a memory control structure of a data page to be read;
and determining the data information of the data page to be read according to the memory control structure.
7. The data reading device of the database cluster is characterized in that the database cluster is provided with a modifying node and a requesting node, local cache service and global cache service are deployed on the modifying node and the requesting node, the global cache service is distributed to any node in the database cluster for management, and the device comprises:
The request initiating module is arranged at the request node and used for initiating a data reading request to a data page to be read, and initiating a global sharing blocking request to the global cache service based on the data reading request so that the global cache service initiates a permission recycling request to the modification node;
the request processing module is arranged on the modification node and is used for sending the data information of the data page to be read according to the node information of the request node when the modification node receives the permission recovery request, processing the permission recovery request and sending first response information to the global cache service;
The authorization response module is arranged on the global cache service, and is used for receiving the first response information, giving global sharing blocking authority to the data page to be read to the request node, and sending authorization response information to the request node;
The information receiving module is arranged at the request node and is used for receiving the data information of the data page to be read and the authorization response information in parallel, and correspondingly processing the data information of the data page to be read according to the receiving sequence of the data information and the authorization response information so as to complete the reading operation of the data page to be read;
the information receiving module is configured to, if the request node receives the data information of the data page to be read before receiving the authorization response information, store the received data information of the data page to be read, wait for the request node to receive the authorization response information and initiate a local reading request to read the data page to be read, and if the request node receives the data information of the data page to be read after receiving the authorization response information, fill the data information of the data page to be read into a local memory area by the request node, thereby completing the reading operation of the data page to be read.
8. An electronic device, the electronic device comprising:
At least one processor, and
A memory communicatively coupled to the at least one processor, wherein,
The memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the data reading method of the database cluster of any one of claims 1-6.
9. A computer readable storage medium storing computer instructions for causing a processor to perform the method of data reading of a database cluster according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410426479.9A CN118171321B (en) | 2024-04-10 | 2024-04-10 | A method, device, equipment and medium for reading data from a database cluster |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410426479.9A CN118171321B (en) | 2024-04-10 | 2024-04-10 | A method, device, equipment and medium for reading data from a database cluster |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118171321A CN118171321A (en) | 2024-06-11 |
| CN118171321B true CN118171321B (en) | 2025-10-21 |
Family
ID=91352692
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410426479.9A Active CN118171321B (en) | 2024-04-10 | 2024-04-10 | A method, device, equipment and medium for reading data from a database cluster |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118171321B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118860595A (en) * | 2024-07-03 | 2024-10-29 | 武汉达梦数据库股份有限公司 | A multi-page reading method, device, equipment, medium and product |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104573428A (en) * | 2013-10-12 | 2015-04-29 | 方正宽带网络服务股份有限公司 | Method and system for improving resource effectiveness of server cluster |
| CN108170757A (en) * | 2017-12-22 | 2018-06-15 | 郑州云海信息技术有限公司 | A kind of data file read method, device and equipment |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7457796B2 (en) * | 2004-07-08 | 2008-11-25 | International Business Machines Corporation | Method using virtual replicated tables in a cluster database management system |
| CN109063132B (en) * | 2018-08-02 | 2020-08-25 | 上海达梦数据库有限公司 | Latch request method, latch authorization method, device and terminal equipment |
| SG10201809815PA (en) * | 2018-11-05 | 2020-06-29 | Mastercard International Inc | Computer system and computer-implemented method for sharing information between at least one financial institution and an account information service provider |
| CN109408519A (en) * | 2018-11-08 | 2019-03-01 | 上海达梦数据库有限公司 | A kind of access method of data page, device, server and storage medium |
| CN111444147B (en) * | 2020-03-30 | 2023-05-12 | 上海达梦数据库有限公司 | Data page creation method and device, terminal equipment and storage medium |
| CN115237968B (en) * | 2022-07-19 | 2025-11-25 | 上海达梦数据库有限公司 | Node management methods, devices, equipment and storage media in database systems |
| CN117596298A (en) * | 2023-11-24 | 2024-02-23 | 南方电网数字电网集团信息通信科技有限公司 | Data processing method and device, electronic equipment and storage medium |
-
2024
- 2024-04-10 CN CN202410426479.9A patent/CN118171321B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104573428A (en) * | 2013-10-12 | 2015-04-29 | 方正宽带网络服务股份有限公司 | Method and system for improving resource effectiveness of server cluster |
| CN108170757A (en) * | 2017-12-22 | 2018-06-15 | 郑州云海信息技术有限公司 | A kind of data file read method, device and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118171321A (en) | 2024-06-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109491928A (en) | Buffer control method, device, terminal and storage medium | |
| CN111444147B (en) | Data page creation method and device, terminal equipment and storage medium | |
| US8918795B2 (en) | Handling and reporting of object state transitions on a multiprocess architecture | |
| CN114036195A (en) | Data request processing method, device, server and storage medium | |
| CN118171321B (en) | A method, device, equipment and medium for reading data from a database cluster | |
| CN110162395B (en) | Memory allocation method and device | |
| CN117596298A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN115237968B (en) | Node management methods, devices, equipment and storage media in database systems | |
| CN117725115A (en) | Database sequence processing method, device, equipment and storage medium | |
| US8341368B2 (en) | Automatic reallocation of structured external storage structures | |
| US9063858B2 (en) | Multi-core system and method for data consistency by memory mapping address (ADB) to hash table pattern associated with at least one core | |
| CN115858104B (en) | Transaction commit sequence number snapshot acquisition method, device, equipment and storage medium | |
| CN118193485A (en) | Concurrency control method and device, electronic equipment and storage medium | |
| CN119311683A (en) | Distributed sequence number generation method, device and equipment based on database row lock | |
| EP4148577A2 (en) | Method and apparatus of responding to client fault detection with update operation, electronic device, and storage medium | |
| CN115114612A (en) | Access processing method, apparatus, electronic device, and storage medium | |
| CN116594764B (en) | Application program updating method, device, electronic device and storage medium | |
| CN115374131B (en) | Request processing method and system, storage medium and electronic equipment | |
| CN118916307B (en) | Descriptor transmission method, device, equipment and storage medium | |
| CN116431561B (en) | Data synchronization method, device, equipment and medium based on heterogeneous many-core accelerator card | |
| CN114090321B (en) | Fault handling methods, database nodes, and storage media | |
| CN114253476B (en) | Storage space expansion method, device, equipment and computer readable storage medium | |
| CN120528980A (en) | Method, device, electronic device and storage medium for preventing repeated submission of service requests | |
| CN118860595A (en) | A multi-page reading method, device, equipment, medium and product | |
| CN120406853A (en) | Request processing method, device, electronic device, storage medium and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |