CN116303754A - Database transaction snapshot generation method, device, equipment and storage medium - Google Patents

Abstract

The application provides a method, a device, equipment and a storage medium for generating transaction snapshots of a database, and relates to the field of databases. The transaction snapshot generation method of the database comprises the following steps: alternately acquiring transaction snapshots generated by the first global transaction management component and the second global transaction management component at different moments respectively; generating a transaction snapshot corresponding to the first global transaction management component according to the transaction number in an active state in the transaction snapshots of two adjacent times acquired from the first global transaction management component; generating a transaction snapshot corresponding to the second global transaction management component according to the transaction number in an active state in the transaction snapshots of two adjacent times acquired from the second global transaction management component; and generating a global transaction snapshot according to the transaction snapshot corresponding to the first global transaction management component and the transaction snapshot corresponding to the second global transaction management component. The method is applied to a distributed database system, and the method can accurately acquire the global consistency transaction snapshot.

Description

Database transaction snapshot generation method, device, equipment and storage medium

technical field

The embodiments of the present application relate to the field of databases, and in particular to a method, device, device, and storage medium for generating transaction snapshots of databases.

Background technique

In a distributed database, multiple global transaction managers are usually deployed to solve the problem of high concurrency pressure on each component or node in a load balancing manner. However, while deploying multiple global transaction managers to share the concurrency pressure of the entire system, it also makes Multiple global transaction managers have difficulty producing globally consistent snapshots.

Contents of the invention

The main purpose of the embodiments of the present application is to provide a database transaction snapshot generation method, device, device and storage medium, so as to achieve accurate global consistent snapshot acquisition.

In order to achieve the above purpose, an embodiment of the present application provides a database transaction snapshot generation method, comprising: alternately obtaining the transaction snapshots generated by the first global transaction management component and the second global transaction management component at different times; according to the The transaction number in the active state in the two adjacent transaction snapshots obtained by the first global transaction management component generates a transaction snapshot corresponding to the first global transaction management component; according to the transaction snapshot obtained from the second global transaction management component The transaction number in the active state in two adjacent transaction snapshots generates a transaction snapshot corresponding to the second global transaction management component; according to the transaction snapshot corresponding to the first global transaction management component and the second global transaction The transaction snapshot corresponding to the management component generates a global transaction snapshot.

In order to achieve the above purpose, the embodiment of the present application also proposes a database transaction snapshot generation device, including:

An acquisition module, configured to alternately acquire transaction snapshots generated by the first global transaction management component and the second global transaction management component at different times;

A first generating module, configured to generate a transaction snapshot corresponding to the first global transaction management component according to the transaction number in an active state in two consecutive transaction snapshots obtained from the first global transaction management component;

The second generation module is configured to generate a transaction snapshot corresponding to the second global transaction management component according to the transaction number in the active state in two adjacent transaction snapshots obtained from the second global transaction management component;

The third generating module is configured to generate a global transaction snapshot according to the transaction snapshot corresponding to the first global transaction management component and the transaction snapshot corresponding to the second global transaction management component.

To achieve the above purpose, an embodiment of the present application also provides an electronic device, including: at least one processor; and a memory connected to the at least one processor in communication; wherein, the memory stores information that can be used by the at least one processor An instruction executed by a processor, the instruction is executed by the at least one processor, so that the at least one processor can execute the above-mentioned method for generating a database transaction snapshot.

In order to achieve the above purpose, the embodiment of the present application also proposes a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above-mentioned method for generating a database transaction snapshot is implemented.

A transaction snapshot generation method, device, device and storage medium of a database proposed by the present application obtain transaction snapshots of multiple global transaction management components at different times alternately, and generate a global transaction snapshot according to multiple transaction snapshots, so that the final The obtained global transaction snapshot can correct the transaction status that has changed within the generation time difference of multiple transaction snapshots. The whole process can quickly and accurately obtain globally consistent snapshots.

Description of drawings

Fig. 1a is a schematic diagram of the transaction state in the distributed database system provided by the embodiment of the present application;

Fig. 1b is a schematic diagram of a transaction snapshot in a distributed database system provided by an embodiment of the present application;

Fig. 2 is the flow chart of the transaction snapshot generation method of the database that the embodiment of the present application provides;

FIG. 3 is a schematic structural diagram of a database transaction snapshot generation device provided by an embodiment of the present application;

FIG. 4 is a schematic structural diagram of an electronic device provided in an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below with reference to the accompanying drawings. However, those of ordinary skill in the art can understand that in each embodiment of the application, many technical details are provided for readers to better understand the application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in this application can also be realized. The division of the following embodiments is for the convenience of description, and should not constitute any limitation to the specific implementation of the present application, and the embodiments can be combined and referred to each other on the premise of no contradiction.

The database transaction snapshot generation method of the present application is aimed at distributed databases. A distributed database is logically a unified whole, and physically stored on different physical nodes. An application can access databases distributed in different geographical locations through a network connection. Distributed databases usually expand multiple computing nodes and storage nodes to handle massive data tasks. During data processing, multiple nodes or components processing tasks are prone to high concurrency pressure problems, and multiple global transaction managers are usually deployed. server and use load balancing to reduce the concurrency pressure of each node or component. Deploying multiple global transaction managers brings the problem of difficulty in generating globally consistent snapshots.

It should be noted that the meaning of a global snapshot is the global state at a certain moment, while consistency means that for any event saved in the global snapshot, the events that occurred before this event should be saved in this snapshot. Simply put, the combination of the states of all transactions at the same time is a globally consistent snapshot.

However, the current global snapshot generation method has the following problems: as shown in Figure 1a, if two global transaction managers gtm1 and gtm2 are deployed in a distributed database system, the first global transaction manager (i.e. the first global transaction management component ) gtm1 is responsible for managing transaction 8, transaction 9, transaction 10 and transaction 11, the second global transaction manager gtm2 is responsible for managing transaction 91, transaction 92, transaction 93 and transaction 94, before T0 time, transaction 8, transaction 10 and transaction 91 Transaction 9, Transaction 11, Transaction 92, Transaction 93, and Transaction 94 are active transactions that have not been completed. If gtm1 receives a snapshot generation request at time T0, the snapshot obtained by gtm1 is shown in Figure 1b. Similarly, when gtm2 receives a snapshot generation request at T1, the snapshot obtained by gtm2 is shown in Figure 1b. Merge these two snapshots into a global transactional snapshot of the entire system. However, combined with Figure 1a, it can be seen that due to the time difference between gtm1 and gtm2 to generate snapshots, during the time period from T0 to T1, the transaction status managed by gtm1 has changed, that is, transaction 9 has changed from an active transaction that has never ended to a completed transaction. Ended inactive transaction. In this way, the final obtained global transaction snapshot does not meet the consistency requirements.

Furthermore, if a global transaction snapshot that does not meet the consistency requirements is used for data backup, it is easy to cause data inconsistency on the data shards. It should be noted that in a distributed database system, multiple operations contained in a transaction are usually distributed across multiple data shards, and a transaction should have four attributes: atomicity, consistency, isolation, and durability. Among them, atomicity means that a transaction is an indivisible unit of work, and all operations included in the transaction are either executed or not executed. Consistency means that the transaction must change the database from one consistent state to another consistent state. However, the data inconsistency problem on the data fragmentation makes the corresponding transaction not meet the transaction attribute requirements.

In addition, through globally consistent snapshots, it is also possible to check and recover from faults in the system, and it is also possible to check whether there is a deadlock problem in the current application. All of these need to be realized on the basis of obtaining accurate global consistent snapshots.

However, this application alternately obtains the transaction snapshots of multiple global transaction management components at different times, and generates a global transaction snapshot according to the multiple transaction snapshots, so that the finally obtained global transaction snapshot can correct the changes that occur within the generation time difference of multiple transaction snapshots. transaction status.

The embodiment of the present application relates to a transaction snapshot generation method of a database, as shown in Figure 2, comprising:

Step 201, alternately acquire transaction snapshots generated by the first global transaction management component and the second global transaction management component at different times.

In this embodiment, step 201 specifically includes: sending a first snapshot generation request to the first global transaction management component, receiving the first transaction snapshot generated at the first moment fed back by the first global transaction management component; The management component sends a second snapshot generation request, receives the second transaction snapshot generated at the second moment fed back by the second global transaction management component; sends the third snapshot generation request to the first global transaction management component, and receives the first global transaction management component Feedback of the third transaction snapshot generated at the third moment; sending a fourth snapshot generation request to the second global transaction management component, and receiving the fourth transaction snapshot generated at the fourth moment fed back by the second global transaction management component. For example, taking two global transaction management components gtm1 and gtm2 as an example, gtm1 generates a transaction snapshot at time T0, gtm2 generates a transaction snapshot at time T1, gtm1 generates another snapshot at time T2, and gtm2 generates another snapshot at time T3. In this way, alternate Get snapshots of transactions at different times. Of course, the method for generating transaction snapshots in this application is not limited to two global transaction management components, but can also target multiple global transaction management components.

It should be noted that the interval between sending the third snapshot generation request and the second snapshot generation request is greater than or equal to the time required for the second snapshot generation request to be sent to the second global transaction management component; The sending interval of the generation request is greater than or equal to the time required for the third snapshot generation request to be sent to the first global transaction management component. In this way, it is guaranteed that the global transaction manager receives the next transaction snapshot generation request after obtaining the previous snapshot generation request.

In addition, when the first global transaction management component and the second global transaction management component are deployed on the same physical device, the sending interval between the third snapshot generation request and the second snapshot generation request is the same as that between the fourth snapshot generation request and the third snapshot generation request. Snapshot generation requests are sent at equal intervals. In this way, while ensuring accurate acquisition of transaction snapshots, the acquisition efficiency is improved.

Step 202: Generate a transaction snapshot corresponding to the first global transaction management component according to an active transaction number in two consecutive transaction snapshots obtained from the first global transaction management component.

In this embodiment, step 202 specifically includes: performing a logical AND operation on the transaction numbers in the active state in two consecutive transaction snapshots obtained from the first global transaction management component to obtain the transaction number corresponding to the first global transaction management component Transactional snapshots. That is to say, only when the same transaction is in the active state that has not ended in the two snapshots, is it confirmed that the transaction is in the active state that has not ended.

Step 203: Generate a transaction snapshot corresponding to the second global transaction management component according to an active transaction number in two consecutive transaction snapshots obtained from the second global transaction management component.

In this embodiment, step 203 specifically includes: performing a logical AND operation on the transaction numbers in the active state in two consecutive transaction snapshots obtained from the second global transaction management component to obtain the transaction number corresponding to the second global transaction management component Transactional snapshots.

Step 204: Generate a global transaction snapshot according to the transaction snapshot corresponding to the first global transaction management component and the transaction snapshot corresponding to the second global transaction management component.

In this embodiment, step 204 specifically includes: numbering the transaction number in the active state in the transaction snapshot corresponding to the first global transaction management component, and the transaction number in the active state in the transaction snapshot corresponding to the second global transaction management component Merge to get a global transaction snapshot.

In addition, after step 204, it also includes: backing up the data in each data fragment according to the global transaction snapshot. In this embodiment, the data in each data fragment can be backed up according to the state of each transaction in the global transaction snapshot. Those skilled in the art can understand that multiple operations included in a transaction are usually executed on multiple data shards.

A database transaction snapshot generation method proposed by this application, by alternately obtaining transaction snapshots of multiple global transaction management components at different times, generating a global transaction snapshot according to multiple transaction snapshots, so that the final obtained global transaction snapshot can be corrected Transaction states that have changed within the time difference between multiple transaction snapshots. The whole process can quickly and accurately obtain globally consistent snapshots.

In addition, it should be understood that the division of steps in the above methods is only for clarity of description, and may be combined into one step or split into multiple steps during implementation. As long as the same logical relationship is included, all Within the scope of protection of this patent; adding insignificant modifications to the process or introducing insignificant designs without changing the core design of the process are all within the scope of protection of this patent.

The embodiment of the present application relates to a database transaction snapshot generation device, as shown in Figure 3, comprising:

An acquisition module 301, configured to alternately acquire transaction snapshots generated by the first global transaction management component and the second global transaction management component at different times;

The first generation module 302 is configured to generate the transaction snapshot corresponding to the first global transaction management component according to the transaction number in the active state in two adjacent transaction snapshots obtained from the first global transaction management component;

The second generation module 303 is configured to generate a transaction snapshot corresponding to the second global transaction management component according to the transaction number in the active state in two consecutive transaction snapshots obtained from the second global transaction management component;

The third generating module 304 is configured to generate a global transaction snapshot according to the transaction snapshot corresponding to the first global transaction management component and the transaction snapshot corresponding to the second global transaction management component.

It should be noted that this embodiment of the device is mainly aimed at the description of the database transaction snapshot generation method provided by the method embodiment at the level of software implementation, and its implementation also needs to rely on hardware support. For example, the functions of related modules can be deployed to On the processor, so that the processor runs to implement corresponding functions, especially, the data generated by the running can be stored in the memory for subsequent inspection and use.

It is worth mentioning that all the modules involved in this embodiment are logical modules, and a logical unit may be a physical unit, or a part of a physical unit, or may be realized by a combination of multiple physical units. In addition, in order to highlight the innovative part of the present invention, units that are not closely related to solving the technical problem proposed by the present invention are not introduced in this embodiment, but this does not mean that there are no other units in this embodiment.

It is not difficult to find that this embodiment is an apparatus embodiment corresponding to the embodiment of the method for generating a transaction snapshot of a database, and this embodiment can be implemented in cooperation with the foregoing embodiments. The relevant technical details mentioned in the foregoing embodiments are still valid in this embodiment, and will not be repeated here in order to reduce repetition. Correspondingly, the relevant technical details mentioned in this embodiment can also be applied to the above method embodiments.

The embodiment of the present invention relates to an electronic device, as shown in FIG. 4 , including: at least one processor 401; Instructions executed by the at least one processor 401, the instructions are executed by the at least one processor 401, so that the at least one processor 401 can execute the data packet capture method in the foregoing implementation manner.

Wherein, the memory and the processor are connected by a bus, and the bus may include any number of interconnected buses and bridges, and the bus connects one or more processors and various circuits of the memory together. The bus may also connect together various other circuits such as peripherals, voltage regulators, and power management circuits, all of which are well known in the art and therefore will not be further described herein. The bus interface provides an interface between the bus and the transceivers. A transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing means for communicating with various other devices over a transmission medium. The data processed by the processor is transmitted on the wireless medium through the antenna, further, the antenna also receives the data and transmits the data to the processor.

The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interface, voltage regulation, power management, and other control functions. Instead, memory can be used to store data that the processor uses when performing operations.

Embodiments of the present invention relate to a computer-readable storage medium storing a computer program. When the computer program is executed by the processor, the above-mentioned database transaction snapshot generation method is realized.

That is, those skilled in the art can understand that all or part of the steps in the method of the above-mentioned embodiments can be completed by instructing related hardware through a program, the program is stored in a storage medium, and includes several instructions to make a device ( It may be a single-chip microcomputer, a chip, etc.) or a processor (processor) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes. .

Those of ordinary skill in the art can understand that the above-mentioned embodiments are specific embodiments for realizing the present application, and in practical applications, various changes can be made to it in form and details without departing from the spirit and spirit of the present application. scope.

Claims (10)

1. A method for generating a transaction snapshot of a database, comprising:

alternately acquiring transaction snapshots generated by the first global transaction management component and the second global transaction management component at different moments respectively;

generating a transaction snapshot corresponding to the first global transaction management component according to the transaction number in an active state in the transaction snapshots of two adjacent times acquired from the first global transaction management component;

generating a transaction snapshot corresponding to the second global transaction management component according to the transaction number in an active state in the transaction snapshots of two adjacent times acquired from the second global transaction management component;

and generating a global transaction snapshot according to the transaction snapshot corresponding to the first global transaction management component and the transaction snapshot corresponding to the second global transaction management component.

2. The method for generating a transaction snapshot of a database according to claim 1, wherein the generating a transaction snapshot corresponding to the first global transaction management component according to a transaction number in an active state in two adjacent transaction snapshots acquired from the first global transaction management component includes:

performing logic AND operation on the transaction numbers in the active state in the transaction snapshots of two adjacent times acquired from the first global transaction management component to obtain the transaction snapshot corresponding to the first global transaction management component;

the generating the transaction snapshot corresponding to the second global transaction management component according to the transaction numbers in the active state in the transaction snapshots of two adjacent times acquired from the second global transaction management component comprises the following steps:

and carrying out logic AND operation on the transaction numbers in the active state in the transaction snapshots of the adjacent two times acquired from the second global transaction management component to obtain the transaction snapshot corresponding to the second global transaction management component.

3. The method for generating a transaction snapshot of a database according to claim 2, wherein generating a global transaction snapshot according to the transaction snapshot corresponding to the first global transaction management component and the transaction snapshot corresponding to the second global transaction management component includes:

and merging the transaction number in the active state in the transaction snapshot corresponding to the first global transaction management component with the transaction number in the active state in the transaction snapshot corresponding to the second global transaction management component to obtain the global transaction snapshot.

4. A method of generating transaction snapshots of a database according to any one of claims 1 to 3, wherein the alternately taking transaction snapshots generated by the first global transaction management component and the second global transaction management component at different times, respectively, comprises:

a first snapshot generation request is sent to the first global transaction management component, and a first transaction snapshot generated at a first moment and fed back by the first global transaction management component is received;

sending a second snapshot generation request to the second global transaction management component, and receiving a second transaction snapshot generated at a second moment and fed back by the second global transaction management component;

sending a third snapshot generation request to the first global transaction management component, and receiving a third transaction snapshot generated at a third moment and fed back by the first global transaction management component;

and sending a fourth snapshot generation request to the second global transaction management component, and receiving a fourth transaction snapshot generated at a fourth time, which is fed back by the second global transaction management component.

5. The method of claim 4, wherein the third snapshot generation request is sent at a time interval from the second snapshot generation request that is greater than or equal to a time required for the second snapshot generation request to be sent to the second global transaction management component;

the transmission interval time between the fourth snapshot generation request and the third snapshot generation request is greater than or equal to the time required by the third snapshot generation request to be transmitted to the first global transaction management component.

6. The method according to claim 5, wherein in a case where the first global transaction management component and the second global transaction management component are disposed on the same physical device, a transmission interval time between the third snapshot generation request and the second snapshot generation request is equal to a transmission interval time between the fourth snapshot generation request and the third snapshot generation request.

7. A method of generating a transaction snapshot of a database according to any of claims 1 to 3, further comprising, after said generating a global transaction snapshot:

and backing up the data in each data partition according to the global transaction snapshot.

8. A transaction snapshot generating device of a database, comprising:

the acquisition module is used for alternately acquiring transaction snapshots generated by the first global transaction management component and the second global transaction management component at different moments;

the first generation module is used for generating a transaction snapshot corresponding to the first global transaction management component according to the transaction number in an active state in the transaction snapshots of two adjacent times acquired from the first global transaction management component;

the second generation module is used for generating a transaction snapshot corresponding to the second global transaction management component according to the transaction number in an active state in the transaction snapshots of two adjacent times acquired from the second global transaction management component;

and the third generation module is used for generating a global transaction snapshot according to the transaction snapshot corresponding to the first global transaction management component and the transaction snapshot corresponding to the second global transaction management component.

9. An electronic device, comprising:

at least one processor; the method comprises the steps of,

a memory communicatively coupled to the at least one processor; wherein,,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the transaction snapshot generation method of the database of any of claims 1 to 7.

10. A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the method of transaction snapshot generation of a database according to any one of claims 1 to 7.

Images