[go: up one dir, main page]

CN112749156B - Data processing method, database management system and data processing device - Google Patents

Data processing method, database management system and data processing device Download PDF

Info

Publication number
CN112749156B
CN112749156B CN201911040327.0A CN201911040327A CN112749156B CN 112749156 B CN112749156 B CN 112749156B CN 201911040327 A CN201911040327 A CN 201911040327A CN 112749156 B CN112749156 B CN 112749156B
Authority
CN
China
Prior art keywords
transaction
stored procedure
execution
branch
exception
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911040327.0A
Other languages
Chinese (zh)
Other versions
CN112749156A (en
Inventor
于巍
李国良
张金玉
董亚辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN201911040327.0A priority Critical patent/CN112749156B/en
Publication of CN112749156A publication Critical patent/CN112749156A/en
Application granted granted Critical
Publication of CN112749156B publication Critical patent/CN112749156B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • G06F16/2365Ensuring data consistency and integrity

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Security & Cryptography (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本申请实施例公开了一种数据处理的方法,用于存储过程执行失败重试。该方法包括:响应于存储过程请求,执行存储过程,所述存储过程包括一个或多个有序的事务,当所述一个或多个有序的事务中的第一事务执行结束时,记录所述第一事务的标记信息;基于执行所述存储过程中出现异常,且满足重试所述存储过程的条件,根据所述第一事务的标记信息重试所述存储过程中所述第一事务之后的事务。

The embodiment of the present application discloses a data processing method for retrying a failed stored procedure. The method comprises: executing a stored procedure in response to a stored procedure request, wherein the stored procedure comprises one or more ordered transactions, and when the first transaction in the one or more ordered transactions is executed, recording the marking information of the first transaction; based on an exception occurring during the execution of the stored procedure and satisfying the condition for retrying the stored procedure, retrying the transaction after the first transaction in the stored procedure according to the marking information of the first transaction.

Description

Data processing method, database management system and data processing device
Technical Field
The present application relates to the field of database technologies, and in particular, to a data processing method, a database management system, and a data processing device.
Background
A stored procedure (stored procedure) is a set of statements stored in a database system that can perform a specific function, and the stored procedure is called again after the first compilation without need for recompilation, and the user can run the stored procedure by specifying the name of the stored procedure. The retry of the storage process is a retry mechanism when the execution of the storage process fails, so that interruption of user service caused by hardware failure or software failure can be avoided. A transaction (transaction) is a sequence of operations performed as a single logical unit of work in a database system, either all or none, an indivisible unit of work. For example, in a relational database, a transaction may be a structured query language (structured query language, SQL) statement, a set of SQL statements, or an entire program. The stored procedure may contain one or more transactions.
In the prior art, a database driver calls a designated storage process according to a work request sent by a user to respond to the request of the user, and when the execution of the storage process fails and the restorability is abnormal, the database driver sends a retry request to a Database (DB) kernel, acquires a correct processing result and then responds to the user. Therefore, part of hardware and software errors can be shielded, and user experience is improved.
Because in the prior art, the database driver acquires whether the called storage process is successfully executed, when the execution of the storage process fails, the whole storage process can only be re-executed once again by retry. When a transaction commit instruction exists in the storage process, the whole storage process retries, which results in the repeated execution of part of the operations that have been committed, thereby resulting in data errors.
Disclosure of Invention
The embodiment of the application provides a data processing method which is used for retrying the execution failure of a storage process and can avoid the repeated execution of executed transactions in the storage process.
A first aspect of the embodiment of the application provides a data processing method, which comprises the steps of responding to a storage process request, executing a storage process, wherein the storage process comprises one or more ordered transactions, recording mark information of a first transaction in the one or more ordered transactions when the execution of the first transaction is finished, and retrying the transaction after the first transaction in the storage process according to the mark information of the first transaction based on the condition that the abnormality occurs in the execution of the storage process and the condition of retrying the storage process is met.
The storage process may contain one or more ordered transactions, and during execution of the storage process, the tag information of the executed transaction in the storage process, such as the location where the transaction ended, is recorded. Therefore, when the execution of the storage process fails to retry, only sentences after the executed transaction can be retried, and the executed transaction does not need to be re-executed, so that the execution efficiency of the retry is improved, and errors caused by repeatedly executing the submitted transaction during the retry are avoided.
In a possible implementation manner of the first aspect, the method further includes recording a request statement that invokes the stored procedure.
The data processing method provided by the embodiment of the application records the request statement for calling the storage process, and can be used for positioning the retry execution position through the request statement and the mark information of the first transaction.
In a possible implementation manner of the first aspect, the marking information includes a line number of an end statement of the first transaction.
The data processing method provided by the embodiment of the application provides a specific implementation mode for recording the marking information of the first transaction, and enhances the operability of the scheme.
In a possible implementation manner of the first aspect, the method further includes recording intermediate state information of the storage procedure, where the intermediate state information includes one or more of an execution statement result, error prompt information, input parameters, output parameters, global variables, and temporary variables of the first transaction.
According to the data processing method provided by the embodiment of the application, the database management system can also record the execution statement result or error prompt information, the input parameter, the output parameter, the global variable and the temporary variable of the first transaction, retry is performed according to the intermediate state information when the first transaction is finished, and the consistency of data can be met.
In a possible implementation manner of the first aspect, the intermediate state information includes an execution statement result or error prompt information of the first transaction, and the method further includes caching the execution statement result or error prompt information of the first transaction, and sending the execution statement result or error prompt information to the user equipment until the first transaction is over.
According to the data processing method provided by the embodiment of the application, the execution statement result or the error prompt information of the first transaction is recorded and then sent to the user equipment after the first transaction is ended, so that the situation that the same execution statement result or the error prompt information is repeatedly sent to the user equipment during retry can be avoided.
In a possible implementation manner of the first aspect, the intermediate state information includes an execution statement result or error prompt information of the first transaction, and the method further includes caching the execution statement result or error prompt information of the first transaction until the storing process is finished, and then sending the execution statement result or error prompt information to the user equipment.
The data processing method provided by the embodiment of the application provides another way for sending the execution statement result or the error prompt information to the user, namely, sending the user after the storage process is finished, so that the same execution statement result or error prompt information can be prevented from being repeatedly sent to the user equipment during retry.
In a possible implementation manner of the first aspect, the first transaction is located in a branch of a branch statement in the storage process, the method further includes recording intermediate state information of the storage process, the intermediate state information including a branch execution state, the branch execution state being used to identify a branch in which the first transaction is located, and retrying the transaction after the first transaction in the storage process includes executing the transaction after the first transaction in the branch in which the first transaction is located based on the branch execution state.
In the data processing method provided by the embodiment of the application, the intermediate state information may further include a branch execution state, where the branch execution state is used to identify a branch of the first transaction in a branch statement, and the branch statement includes an if branch statement or a case branch statement. And when retrying, the branch can be continuously executed according to the branch execution state, so that retry errors are avoided.
In a possible implementation manner of the first aspect, the branch execution state includes a conditional determination result of executing the branch statement in the storing process, where the conditional determination result is used to determine to execute the target branch in the branch statement.
According to the data processing method provided by the embodiment of the application, one specific record content of the branch execution state is the condition judgment result of the branch statement, so that the feasibility of the scheme is enhanced.
In one possible implementation manner of the first aspect, the method further includes the step of retrying the storage process, wherein the condition of the storage process is that the storage process exception is a restorability exception, and the restorability exception includes a network exception, a memory use exception, an input-output exception, a lock wait timeout exception, or a two-phase transaction commit exception.
According to the data processing method provided by the embodiment of the application, if the abnormality is restorable abnormality, retry can be executed.
In a possible implementation manner of the first aspect, the retrying the transaction after the first transaction in the storing process includes retrying the transaction after the first transaction in the storing process after waiting a preset first time period.
According to the data processing method provided by the embodiment of the application, the retry is performed after the scenes such as the main and standby switching wait for the preset duration, so that the success rate of the retry can be improved.
In one possible implementation manner of the first aspect, the method further includes recording tag information of a second transaction of the one or more ordered transactions when execution of the second transaction ends, the second transaction being a transaction executed after the first transaction, and retrying the transaction after the first transaction in the storage includes retrying the transaction after the second transaction in the storage based on the tag information of the second transaction, the second transaction being a last executed transaction in the executed transaction when a condition for retrying the storage is met.
In the data processing method provided by the embodiment of the application, if the storage process comprises a plurality of transactions, the marking information of each transaction can be recorded, and when retrying, the transaction after the last executed transaction is executed, so that the resource waste can be further reduced.
A second aspect of the embodiment of the application provides a database management system, which is characterized by comprising an execution unit, a recording unit and a retry unit, wherein the execution unit is used for responding to a storage process request and executing a storage process, the storage process comprises one or more ordered transactions, the recording unit is used for recording the marking information of a first transaction in the one or more ordered transactions when the execution of the first transaction is finished, and the retry unit is used for retrying the transaction after the first transaction in the storage process according to the marking information of the first transaction based on the condition that the abnormality occurs in the execution of the storage process and the condition of retrying the storage process is met.
In a possible implementation manner of the second aspect, the recording unit is further configured to record a request statement that invokes the stored procedure.
In a possible implementation manner of the second aspect, the marking information includes a line number of an end statement of the first transaction.
In a possible implementation manner of the second aspect, the database management system further includes a recording unit, configured to record intermediate state information of the storage procedure, where the intermediate state information includes one or more of an execution statement result, error prompt information, an input parameter, an output parameter, a global variable, and a temporary variable of the first transaction.
In a possible implementation manner of the second aspect, the intermediate state information includes an execution statement result or error prompt information of the first transaction, and the database management system further includes a buffer unit, configured to buffer the execution statement result or error prompt information of the first transaction, and send the execution statement result or error prompt information to the user device until the first transaction ends.
In a possible implementation manner of the second aspect, the intermediate state information includes an execution statement result or an error prompt information of the first transaction, and the database management system further includes a buffer unit, configured to buffer the execution statement result or the error prompt information of the first transaction, and send the execution statement result or the error prompt information to the user device until the storage process is finished.
In a possible implementation manner of the second aspect, the first transaction is located in a branch of a branch statement in the storage process, and the recording unit is further configured to record intermediate state information of the storage process, where the intermediate state information includes a branch execution state, and the branch execution state is used to identify a branch in which the first transaction is located, and the retry unit is specifically configured to execute a transaction after the first transaction in the branch in which the first transaction is located based on the branch execution state.
In a possible implementation manner of the second aspect, the branch execution state includes a conditional determination result of executing the branch statement in the storing process, where the conditional determination result is used to determine to execute the target branch in the branch statement.
In one possible implementation manner of the second aspect, the retry of the storage procedure is conditional on the storage procedure being a restorable exception, the restorable exception including a network exception, a memory use exception, an input-output exception, a lock wait timeout exception, or a two-phase transaction commit exception.
In a possible implementation manner of the second aspect, the retry unit is specifically configured to retry a transaction after the first transaction in the storing process after waiting for a preset first duration.
In a possible implementation manner of the second aspect, the recording unit is further configured to record, when execution of a second transaction of the one or more ordered transactions ends, marking information of the second transaction, where the second transaction is a transaction executed after the first transaction, and the retry unit is specifically configured to retry a transaction after the second transaction in the storage process according to the marking information of the second transaction, where the second transaction is a transaction executed last in the executed transaction when a condition for retrying the storage process is satisfied.
A third aspect of the embodiments of the present application provides a data processing apparatus comprising a memory for storing instructions and a processor for executing the instructions in the memory, causing the data processing apparatus to perform the steps of the foregoing first aspect or implementations of the first aspect.
A fourth aspect of the embodiments of the present application provides a computer program product, characterized in that the computer program product comprises instructions which, when run on a computer, cause the computer to perform the steps of the foregoing first aspect or implementations of the first aspect.
A fifth aspect of the embodiments of the present application provides a computer readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the steps of the foregoing first aspect or implementations of the first aspect.
A sixth aspect of the embodiment of the present application provides a database system, which is characterized by comprising a network interface for receiving a storage procedure request sent by a user equipment, and a data processing device for executing the method according to the foregoing first aspect or any one of the implementation manners of the first aspect in response to the storage procedure request.
According to the technical scheme, the embodiment of the application has the advantages that the data processing method provided by the embodiment of the application realizes the execution failure retry of the storage process, supports the execution failure retry of the storage process containing the transaction, improves the execution efficiency of the retry because the finished transaction does not need to be re-executed, and avoids errors caused by repeatedly executing the executed transaction during the retry.
Drawings
FIG. 1a is a schematic diagram of a single database system according to an embodiment of the present application;
FIG. 1b is a schematic diagram of a clustered database system according to an embodiment of the present application;
FIG. 1c is a schematic diagram of another architecture of a clustered database system in accordance with an embodiment of the present application;
FIG. 2 is a schematic diagram of a database server according to an embodiment of the present application;
FIG. 3 is a schematic diagram of an embodiment of a data processing method according to the present application;
FIG. 4 is a schematic diagram of one embodiment of creating a stored procedure in an embodiment of the present application;
FIG. 5 is a flow chart of a process of storing a process execution in an embodiment of the application;
FIG. 6 is a schematic diagram of a system architecture of a cluster scenario in an embodiment of the present application;
FIG. 7 is a diagram illustrating a retry of a storage procedure in a clustered scenario in accordance with an embodiment of the present application;
FIG. 8 is a diagram of another embodiment of a retry of a storage procedure in a clustered scenario in accordance with an embodiment of the present application;
FIG. 9 is a schematic diagram of a system architecture of a method for retrying a failed execution of a storage process according to an embodiment of the present application;
FIG. 10 is a schematic diagram of one embodiment of a database management system in an embodiment of the application;
FIG. 11 is a schematic diagram of another embodiment of a database management system in an embodiment of the present application;
FIG. 12 is a schematic diagram of an embodiment of a data processing apparatus in an embodiment of the present application.
Detailed Description
The embodiment of the application provides a data processing method which is used for retrying the execution failure of a storage process and can avoid repeated execution of transactions in the storage process.
Embodiments of the present application will now be described with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the present application. As one of ordinary skill in the art can know, with the development of technology and the appearance of new scenes, the technical scheme provided by the embodiment of the application is also applicable to similar technical problems.
The terms first, second and the like in the description and in the claims and in the above-described figures, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or modules is not necessarily limited to those steps or modules that are expressly listed or inherent to such process, method, article, or apparatus. The naming or numbering of the steps in the present application does not mean that the steps in the method flow must be executed according to the time/logic sequence indicated by the naming or numbering, and the execution sequence of the steps in the flow that are named or numbered may be changed according to the technical purpose to be achieved, so long as the same or similar technical effects can be achieved.
The data processing method, the database management system and the database server provided by the embodiment of the application can be applied to a stand-alone database system or a cluster database system, and are not limited in particular. A database system (database system) is a data processing system developed to accommodate the data processing requirements, and referring to fig. 1a-1c, a schematic diagram of the architecture of the database system according to an embodiment of the present application is shown below, and briefly described below:
FIG. 1a is a schematic diagram of a stand-alone database system according to an embodiment of the present application, where the stand-alone database system includes a database management system and a data store (data store), and the database management system is used to provide services such as query and modification of a database, and the database management system stores data in the data store. In a stand-alone database system, the database management system and data storage are typically located on a single server, such as a Symmetric Multiprocessor (SMP) server. The SMP server includes a plurality of processors, all of which share resources such as buses, memory, and i/o systems. The functions of the database management system may be implemented by one or more processors executing programs in memory.
Fig. 1b is a schematic diagram of an architecture of a clustered database system according to an embodiment of the present application, where the clustered database system is a clustered database system adopting a shared-storage architecture, and includes a plurality of nodes (e.g., nodes 1 to N in the figure), each node is configured with a database management system, which provides services such as querying and modifying a database for a user, respectively, and the plurality of database management systems store shared data in a shared data storage, and perform read-write operations on the data in the data storage through a switch. The shared data storage may be a shared disk array. Nodes in the clustered database system may be physical machines, such as database servers, or virtual machines running on abstract hardware resources. If the node is a physical machine, the switch is a storage area network (storage area network, SAN) switch, ethernet switch, fabric switch, or other physical switching device. If the node is a virtual machine, the switch is a virtual switch.
Fig. 1c is a schematic diagram of another architecture of a clustered database system according to an embodiment of the present application, where the clustered database system is a clustered database system adopting a shared-nothing architecture, and includes a plurality of nodes (e.g., nodes 1 to N in the figure), each node having a hardware resource (e.g., a data storage device), an operating system, and a database that are individually and independently shared, and the nodes communicate with each other via a network. Under the system, data are distributed to each node according to the database model and the application characteristics, the query task is divided into a plurality of parts, the query task is executed in parallel on all nodes, the query task is calculated in cooperation with each other, the database service is provided as a whole, and all communication functions are realized on a high-broadband network interconnection system. As with the shared-storage architecture clustered database system described in clustered database system 1, the nodes herein may be either physical machines or virtual machines.
The nodes in the cluster database system shown in fig. 1b and fig. 1c may be set as a coordinating node (coordinator) or a data node (data node) according to actual use requirements, where the coordinating node is configured to receive the SQL sent by the client or store a process request, and after analysis, may send the SQL to the data node that needs to participate in data processing. The data node finishes data extraction, the query result is returned to the coordination node, and the coordination node gathers the data and sends the data to the client.
In all embodiments of the application, the data store (data store) of the database system includes, but is not limited to, a Solid State Disk (SSD), a disk array, or other type of non-transitory computer readable medium. Although the database is not shown in fig. 1a-1c, it should be understood that the database is stored in a data store. Those skilled in the art will appreciate that a database system may include fewer or more components than those shown in fig. 1a-1c, or may include different components than those shown in fig. 1a-1c, with fig. 1a-1c only showing components more relevant to the implementations disclosed in this embodiment. For example, while 4 nodes have been described in the two clustered database systems of fig. 1b and 1c, one skilled in the art will appreciate that one clustered database system may contain any number of nodes. The database management system functions of each node may be implemented by an appropriate combination of software, hardware, and/or firmware running on each node, respectively.
For ease of understanding and description, by way of example and not limitation, the solution of an embodiment of the present application is described below with reference to a shared-nothing architecture, clustered database system, such as a database server. It will be apparent to those skilled in the art from this disclosure that the methods of embodiments of the present application are equally applicable to clustered database systems of shared-storage architecture, stand-alone database systems, and any type of relational database system.
Please refer to fig. 2, which is a schematic diagram illustrating a database server according to an embodiment of the present application.
As shown in FIG. 2, an embodiment of the present application provides a database server 100 comprising at least one processor 104, a non-transitory computer readable medium (non-transitory computer-readable medium) 106 storing executable code, and a database management system 108. The executable code, when executed by the at least one processor 104, is configured to implement the components and functions of the database management system 108. The non-transitory computer readable medium 106 may include one or more non-volatile memories including, by way of example, semiconductor memory devices such as erasable programmable read-only memory (erasable programmable read only memory, EPROM), electrically erasable read-only memory (ELECTRICALLY ERASABLE PROGRAMMABLE READ ONLY MEMORY, EEPROM) and flash memory (flash memory), magnetic disks such as internal hard disk (INTERNAL HARD DISK) or removable magnetic disk (removable disk), magneto-optical disk (magneto optical disk), and CD-ROMs and DVD-ROMs. In addition, the non-transitory computer-readable medium 106 may also include any device configured as a main memory (main memory). The at least one processor 104 may comprise any type of general purpose computing circuit or special purpose logic circuit, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). The at least one processor 104 may also be one or more processors, such as a CPU, coupled to one or more semiconductor substrates.
The database management system 108 may be a relational database management system (relational database MANAGEMENT SYSTEM, RDBMS). The database management system 108 supports a structured query language (structured query language, SQL). Generally, SQL refers to a specialized programming language that is used exclusively to manage data stored in a relational database. SQL may refer to various types of data-dependent languages including, for example, data definition languages and data manipulation languages, where the scope of SQL may include data insertion, querying, updating and deleting, schema creation and modification, and data access control. Further, in some examples, SQL may include descriptions related to various language elements.
A query is a request to view, access and/or manipulate data stored in a database. Database management system 108 may receive queries in SQL format (referred to as SQL queries) from database clients 102. Typically, the database management system 108 generates query results corresponding to the query by accessing relevant data from the database and manipulating the relevant data, and returns the query results to the database client 102. A database is a collection of data organized, described, and stored in a mathematical model, and may include one or more database structures or formats, such as row and column stores. The database is typically stored in a data store, such as external data store 120 in FIG. 2, or non-transitory computer readable medium 106. When the database is stored on the non-transitory computer readable medium 106, the database management system 108 is an in-memory database management system.
Database client 102 may include any type of device or application configured to interact with database management system 108. In some examples, database client 102 includes one or more application servers.
The database management system 108 includes an SQL engine 110, an execution engine 122, and a storage engine 134. The SQL engine 110 generates a corresponding execution plan from SQL statements, e.g., queries (Query), submitted by the client 102, and the execution engine 122 operates according to the execution plan of the statements to generate Query results. The storage engine 134 is responsible for managing the table data, the index, and the actual content on the file system, and also managing the Cache, buffer, transaction, log, and other data at runtime. For example, storage engine 134 may write the execution results of execution engine 122 to data store 120 via physical I/O. The SQL engine 110 comprises a parser (not shown) for performing syntactic and semantic analysis of SQL statements, expanding and partitioning the views in the query into small query blocks, and an optimizer (not shown). The optimizer generates a set of execution plans for the sentence that may be used, estimates the cost of each execution plan, compares the costs of the plans, and finally selects an execution plan with the smallest cost.
The following describes important concepts related to the embodiments of the present application:
A stored procedure (stored procedure) is a set of statements stored in a database system that can perform a specific function, and the stored procedure is called again after the first compilation without need for recompilation, and the user can run the stored procedure by specifying the name of the stored procedure.
A transaction is a sequence of database operations defined by a user in a database system, which operations are either all or none, and are an integral unit of work. Transactions have four properties, atomic, consistency, isolation, and persistence (durability). In a relational database, a transaction may be a structured query language (structured query language, SQL) statement, a set of SQL statements, or an entire program. Statements defining transactions in SQL include BEGIN (BEGIN), COMMIT (COMMIT), ROLLBACK (ROLLBACK).
Transactions typically BEGIN with BEGIN and end with COMMIT or ROLLBACK. COMMIT indicates that all operations of the transaction are committed, specifically, all updates to the database in the transaction are written to disk, and the transaction is normally ended. ROLLBACK represents a ROLLBACK, i.e., some failure occurs during the execution of a transaction, the transaction cannot continue, and the system rolls back all completed operations to the database in the transaction (UNDO) to the state at the beginning of the transaction.
In the prior art, a database server executes a storage process according to a request of a client, and when the execution of the storage process fails, the whole storage process can only be retried completely, and the control granularity is the storage process level.
When the stored procedure includes one or more transactions, if the stored procedure is in error, then there is a transaction that ends with a COMMIT, which represents that all updates to the database in the transaction have been written to disk. When retrying the storage process, the transaction is repeatedly executed, which will cause data errors. If a transaction ends with a ROLLBACK before the storage process is in error, which means that a fault occurs in the process of running the transaction, the transaction cannot be continuously executed, and when the storage process is retried, the ROLLBACK transaction is repeatedly executed, so that resources are wasted.
In addition, prior art retries need to take into account the driving support of different systems, such as open database connection (open database connectivity, ODBC), java database connection (Java database connectivity, JDBC) object relational database management system (object-relational database MANAGEMENT SYSTEM, orbms) postgresql c language application interface Libpq, etc.
In order to solve the above-mentioned problems, an embodiment of the present application provides a data processing method, which is used for a retry process of an execution failure of a storage process, and is described below with reference to a specific embodiment.
Referring to fig. 3, an embodiment of a data processing method according to an embodiment of the application is shown.
301. Executing a storage process;
The database management system receives a request for a stored procedure sent by a client, the stored procedure can be created by the client, refer to fig. 4, and for an embodiment of creating a stored procedure, the stored procedure is created by sending a statement to create the stored procedure, and the statement is registered in the database, illustratively by the SQL client. Thereafter, the stored procedure may be invoked directly, e.g., by the statement "CALL STPTEST1 ()", a stored procedure request is sent to effect the invocation, and the stored procedure is executed in the database. The storage includes one or more ordered transactions.
302. Recording first position and intermediate state information of ending of a first transaction;
During execution of the stored procedure, when execution of a transaction ends, the database management system may record the tag information of the transaction, optionally, the database management system records location information of the statement that the transaction ended, or an identification of the transaction. Specifically, the database management system records the line number of the transaction end statement. The location at the end of the transaction, including the location of the COMMIT or the location of the ROLLBACK. For example, as shown in fig. 4, if the fourth line is in error, the position of the last COMMIT before the fourth line, i.e., the second line (line 2), is recorded;
in addition, the database management system also records intermediate state information of the stored procedures.
The intermediate state information includes various parameters or variables generated during the execution of the process, such as execution statement results, error prompt information generated during execution, temporary variables, global variables, input parameters, output parameters, branch execution states, and the like. It will be appreciated that intermediate state information is not satisfactory for data consistency if missing the intermediate state information during a retry, as is necessary for retries.
Optionally, the database management system caches the execution statement result or error prompt information generated in execution. In the storage process, if the intermediate result is needed to be returned before the COMMIT, the intermediate result is cached and is not sent to the user equipment until the COMMIT is performed, or the intermediate result is uniformly sent when the storage process is finished. By way of example, the term "track non-word 'hello world';" indicates that the screen print prompt message 'hello world' is sent directly before the end of the transaction, and is resent again when retried, so that the same message is printed on the screen multiple times, and therefore, the result of the execution statement is saved through data buffering, and is sent until the end of the execution of the storage process or the end of the transaction, so that repeated operations can be avoided.
Optionally, the database management system records values of variables of the stored procedure, including temporary variable execution results. For example, after the execution of the second row of COMMIT in FIG. 4, the value of variable i at that time is recorded.
Optionally, the database management system records input parameters and/or output parameters of the stored procedure, e.g., input parameters x, y of the stored procedure in fig. 4.
Optionally, the first transaction is located in a branch of the branch statement in the storing process, and the database management system records a branch execution state, where the branch execution state is used to identify the branch in which the first transaction is located. The branch execution state includes a conditional branch execution state or a case branch execution state, etc. Optionally, the branch execution state is a conditional determination result of the branch statement. Fig. 5 is a flow chart of a process of storing a process execution procedure according to an embodiment of the application. For example, as shown in fig. 5, if an if branch is included in the execution of the storage procedure, a branch determination result needs to be recorded, for example, for a branch determination if (select= =6), if select is equal to 6, a first branch is entered, if select is not equal to 6, a second branch is entered, the branch determination result is recorded to be equal to 6, and may be used to determine to execute the first branch, the branch determination result is recorded to be not equal to 6, and may be used to determine to execute the second branch, and when the storage procedure is retried, a branch determined according to the branch determination result may be directly entered without re-executing the if determination statement.
For a stored procedure that contains multiple transactions, there is corresponding intermediate state information for the end position of each transaction. For example, if the storage procedure includes 3 transactions, namely, an a transaction, a B transaction and a C transaction according to an execution sequence, when the a transaction is ended in the execution procedure of the storage procedure, corresponding first intermediate state information is recorded, when the B transaction is ended, corresponding second intermediate state information is recorded, at this time, optionally, the first intermediate state information may be released, if an error occurs in the C transaction in the execution procedure, when retrying, the already ended a transaction and B transaction may be skipped, and according to the second intermediate state information, retrying may be started after the position where the B transaction is ended.
Optionally, a request statement sent by the client to call the stored procedure, such as "CALL STPTEST1 ()", the request statement and the transaction end statement line number may be used together to determine the location of the retry start.
303. Retrying the stored procedure based on the first location and intermediate state information at which the first transaction ended;
based on the condition that the exception occurs in the process of executing the storage and the condition of retrying the storage process is met, retrying the transaction after the first transaction in the storage process according to the marking information of the first transaction.
Conditions for retrying the stored procedure include that the exception is a restorable exception. The types of exceptions that cause failure in the storage process execution are various, such as network exceptions, e.g., network flashovers, memory usage exceptions, e.g., memory application failures, input/output (I/O) exceptions, e.g., read exceptions due to busy hard disks, lock wait timeout exceptions, exceptions that occur during the two-phase transaction execution, including two-phase transaction commit exceptions. Specific reasons for failure of execution of the stored procedure are not limited herein. If the exception is a restorable exception that can be resolved by retry, a retry of the storage procedure will occur.
The database management system can realize exception capture, identify exception types, judge whether the exception types are restorable exceptions or not so as to determine whether retry is needed or not, and initiate retry logic at the server side if retry is needed. The specific process is not described here.
It should be noted that, the data processing aspect of the embodiment of the present application may be applicable to retry of execution failure of a storage process in different database usage scenarios, including application scenarios such as a stand-alone database, a cluster database, and a master/slave database.
The retry includes one retry or multiple retries, and the retry time may be a direct retry, a retry after waiting for a preset period of time, or a retry when a preset condition is satisfied, and the specific number of retries and the retry time are not limited herein.
Retrying the transaction following the first transaction in the storage according to the marking information of the first transaction. When the database management system retries, through the previously recorded execution intermediate state, the executed and submitted statement is skipped, the consistency state after the previous submission is restored, and the transaction after the first transaction is retried.
It will be appreciated that one or more transactions may be included in a storage process, and if there are multiple transactions in a storage process, when the execution of the storage process fails, a retry may be performed from the last transaction completed, e.g., for a storage process containing 3 transactions, if there is an error in the execution of the 3 rd transaction, a retry may be performed from the end of the 2 nd transaction.
For example, as shown in fig. 4, if the fourth line is in error, the sentence before the second line of COMMIT is skipped during retry, and the sentence after the second line of COMMIT is continuously executed according to the variable i recorded in step 302, the parameters x and y are input, and the intermediate result generated by execution is executed.
If the retry execution is successful, the correct execution result can be returned to the user.
The data processing method provided by the embodiment of the application realizes the execution failure retry of the storage process, supports the execution failure retry of the storage process containing the transaction, reduces the retry granularity to the transaction level because the finished transaction does not need to be re-executed, improves the execution efficiency of the retry, and avoids the error caused by repeatedly executing the submitted part during the retry.
Fig. 6 is a schematic diagram of a system architecture of a cluster scenario in an embodiment of the application.
In the database cluster scenario, including a coordinating node (coordinator) and a plurality of data nodes (data nodes), the data nodes shown in fig. 6 are deployed in a primary-backup mode, and when a primary data node fails, a backup data node takes over a query request. When a fault occurs, the coordination node decides whether to retry, retry strategy and the like according to the retry decision module. The embodiment of the application can process the retry of the primary and standby switching scenes and cope with various situations such as planned shutdown, unplanned shutdown and the like. Typically, the method comprises the steps of retrying the execution failure of the storage process and retrying the execution failure scene caused by the switching of the master node and the standby node. The following description will be made respectively:
1. In the cluster scenario, the storage process fails to execute a retry.
Referring to fig. 7, a diagram of one embodiment of a retry of a storage procedure in a clustered scenario is shown in an embodiment of the present application.
701. The business application sends a storage process request to the coordination node;
In a cluster database scenario, a business application sends a stored procedure request to a coordinating node.
702. The coordination node issues the request of the storage process;
And the coordination node executes the stored procedure request, and sends SQL to the data node according to the actual demand or preset rules, and the data node executes part of SQL.
The coordination node can acquire intermediate state information of a storage process and can also capture an abnormality in an execution process.
703. Synchronizing the master and the slave;
if the data nodes are configured in a master-slave mode, SQL received by the group data nodes is synchronized to the slave data nodes.
Step 703 is an optional step, and may or may not be performed, which is not limited herein.
704. The coordination node acquires failure information;
the storage process execution failure includes a coordination node execution failure or a data node execution failure.
Optionally, the types of exceptions that may cause failure in execution of the storage process include network exceptions such as network flashovers, memory usage exceptions such as memory application failures, input/output (I/O) exceptions such as read exceptions due to busy hard disk, lock wait timeout exceptions, and exceptions that occur during execution of the two-phase transaction. Specific reasons for failure of execution of the stored procedure are not limited herein. If the anomaly is a restorable anomaly, a retry of the stored procedure will be initiated.
Optionally, the data node executes SQL and returns the execution result to the coordination node, if the data node executes error, the coordination node can obtain failure information of the storage process.
The specific implementation process refers to step 302 in the corresponding embodiment of fig. 3, which is not described herein.
705. The coordination node checks the failure reason;
the coordination node identifies whether the reason of the failure of the execution of the data node is restorability abnormality which can be restored by retry, if so, the retry is initiated. Optionally, the coordination node decision module judges whether to retry or not and the time for delaying retry according to the current active/standby situation.
706. The storage process is retried.
The coordination node sends a retry request according to the recorded intermediate state information of the storage process, and the main data node or the standby data node retries according to the intermediate state information of the storage process, and the specific implementation process refers to step 303 in the embodiment corresponding to fig. 3, which is not described herein again.
2. In the cluster scene, the master-slave node is switched to cause the retry of the execution failure.
Referring to fig. 8, another embodiment of a retry of a storage procedure in a clustered scenario is shown in an embodiment of the present application.
801. The business application sends a storage process request to the coordination node;
802. The coordination node issues the request of the storage process;
803. synchronizing the master and the slave;
Step 803 is an optional step, and may or may not be performed, which is not limited herein.
Steps 801 to 803 are similar to steps 701 to 703 in the corresponding embodiment of fig. 7, and are not described here again.
804. When the data node fails to execute, the master-slave switching is performed;
When the data node performs the storing process, the coordinating node carries the record data with the intermediate state information of the storing process, and the specific implementation process refers to step 302 in the embodiment corresponding to fig. 3, which is not described herein again.
When the data node storage process fails to execute, an error is reported to the coordination node.
Optionally, there are various types of exceptions that cause the data node to store the process execution failure, and specific reasons for the store process execution failure are not limited herein.
If the main data node crashes and is down, the working state of the standby data node is normal, the background cluster management module senses the situation, and the coordination node indicates to perform main and standby switching. The switching of the master and slave nodes is generally controlled by a cluster management system, the cluster management system is mainly responsible for starting and stopping the whole cluster, and collects the running state of each node, and when the running abnormality is found, the master and slave switching is controlled by sending a command to the data node.
805. The storage process is retried.
The coordination node retries according to the recorded intermediate state information of the storage process, including sending the execution task to the backup data node for retrying, and the specific implementation process refers to step 303 in the embodiment corresponding to fig. 3, which is not described herein again.
Optionally, the coordination node may synchronize data with other coordination nodes in a synchronous manner, or save the variable in the execution result set before the execution failure of the storage process in a disk-drop manner, so that even if the coordination node crashes, the retry of the execution failure of the storage process can be realized.
Fig. 9 is a schematic diagram of a system architecture of a method for retrying a failed execution of a storage process according to an embodiment of the application.
The database management system stores the position of the executed transaction end statement and the intermediate state information when the transaction ends in the execution of the storage process, and after the database management system captures the abnormality, the database management system resends the storage process request to realize retry. Because the data processing method of the embodiment records the position when the transaction is ended and the corresponding intermediate state information, fine-granularity retry can be realized, and the customer experience when the software and the hardware are abnormal is improved.
The data processing method for implementing the retry of the execution failure of the storage process was described above, and the data processing apparatus implementing the method will be described below.
Referring to FIG. 10, a database management system according to an embodiment of the present application is shown.
The database management system provided by the embodiment of the application can be deployed in a stand-alone database system or nodes in various cluster database systems, and the nodes can run in physical machines or virtual machines, and are not limited in particular.
In practical applications, the data processing device for deploying the database management system may be a database server, or a dedicated data processing device, which is not limited herein.
The database management system 1000 provided in the embodiment of the present application includes:
an execution unit 1001 for executing a stored procedure, the stored procedure comprising one or more ordered transactions, in response to a stored procedure request;
A recording unit 1002 configured to record tag information of a first transaction when execution of the first transaction is ended among the one or more ordered transactions;
and a retry unit 1003, configured to retry a transaction after the first transaction in the storage process according to the tag information of the first transaction, based on that an exception occurs in the execution of the storage process and that a condition for retrying the storage process is satisfied.
The recording unit 1002 is further configured to record a request statement for invoking the stored procedure.
The tag information includes a line number of an end statement of the first transaction.
The apparatus further comprises:
A recording unit 1002, configured to record intermediate state information of the storage procedure, where the intermediate state information includes one or more of an execution statement result of the first transaction, error prompt information, an input parameter, an output parameter, a global variable, and a temporary variable.
The intermediate state information includes an execution statement result or error hint information of the first transaction, the apparatus further comprising:
The buffer unit 1004 is configured to buffer an execution statement result or error prompt information of the first transaction until the first transaction is ended, and then send the execution statement result or error prompt information to the user equipment.
The intermediate state information includes an execution statement result or error hint information of the first transaction, the apparatus further comprising:
the buffer unit 1004 is configured to buffer the execution statement result or the error prompt information of the first transaction until the storage process is finished, and then send the execution statement result or the error prompt information to the user equipment.
The first transaction is located at a branch of a branch statement in the storage process, the recording unit 1002 is further configured to:
recording intermediate state information of the stored procedure, the intermediate state information including a branch execution state, the branch execution state being used to identify a branch in which the first transaction is located;
the retry unit 1003 is specifically configured to execute, based on the branch execution status, a transaction subsequent to the first transaction in the branch where the first transaction is located.
The branch execution state includes a conditional determination of the branch statement in the execution of the store.
The condition for retrying the storage process is that the storage process exception is a restorability exception, including a network exception, a memory use exception, an input-output exception, a lock wait timeout exception, or a two-phase transaction commit exception.
The retry unit 1003 is specifically configured to retry a transaction after the first transaction in the storage process after waiting for a preset first duration.
The recording unit 1002 is further configured to record, when execution of a second transaction of the one or more ordered transactions ends, tag information of the second transaction, where the second transaction is a transaction executed after the first transaction;
the retry unit 1003 is specifically configured to retry a transaction after the second transaction in the storage procedure according to the tag information of the second transaction, where the second transaction is a last executed transaction in the executed transactions when the condition of retrying the storage procedure is satisfied.
Referring to FIG. 11, another embodiment of a database management system according to the present application is shown.
The database management system may include a plurality of software functional modules, including a database server framework (DB framework), a database compiler, a database executor, and a database exception handling module, where the newly added submodule implements retry logic. Wherein:
1. And the database server framework is used for analyzing the information sent by the user to determine what kind of information is, such as SQL information, PBE information and the like. The method is also used for storing original request sentences, preparing for retry, storing intermediate state information and the like.
2. And the database compiler is used for limiting the inspection of the application scene, and setting a non-retry mark if a non-support scene is met.
3. And the database executor is used for storing and storing the process variables and parameters, storing the positions of the transaction, including the positions of COMMIT and ROLLBACK, storing the intermediate state information of the transaction, including the first process parameters, the branch execution state, the execution results of the first variables and the like.
4. The storage process exception handling module is used for capturing exceptions, checking whether retries are needed or not, and can also be used for rollback transactions.
The database server can comprise a re-execution module used for starting a new transaction, skipping over a COMMIT statement according to the stored COMMIT position information when the database server is executed last time and recovering the current state comprising variables and parameters and result set information when the database server is executed last time according to information of a COMMIT point when the database server is executed last time.
It should be noted that, the preconditions include that the data is not sent to the client before the storage process is executed, and that the failure is a processable failure, and that the retry of the internal failure of the program is meaningless.
5. The abnormal retry module is used for restarting a new transaction, skipping over submitted sentences according to intermediate state information such as stored COMMIT position information, if branch judging results and the like in the last execution, and recovering the current state including variables, parameters and the like according to intermediate state information corresponding to the COMMIT points in the last execution.
Referring to FIG. 12, a schematic diagram of an embodiment of a data processing apparatus is shown.
The data processing device provided in this embodiment may be a server or a dedicated data processing device, for example, a database server, which is not limited in the embodiment of the present application.
The data processing apparatus 1200 may vary considerably in configuration or performance and may include one or more processors 1201 and memory 1202, with programs or data stored in the memory 1202.
Wherein the memory 1202 may be volatile storage or nonvolatile storage. Processor 1201 is optionally one or more central processing units (central processing unit, CPU), which may be a single-core CPU or a multi-core CPU, or a special purpose processor. The processor 1201 may be in communication with the memory 1202 to execute a series of instructions in the memory 1202 on the data processing apparatus 1200.
The data processing apparatus 1200 also includes one or more wired or wireless network interfaces 1203, such as an ethernet interface.
Optionally, although not shown in FIG. 12, the data processing device 1200 may also include one or more power supplies, one or more input/output interfaces, which may be used to connect a display, mouse, keyboard, touch screen device, or sensing device, etc., with or without optional components, and is not limited herein.
The flow executed by the processor 1201 in the data processing apparatus 1200 in this embodiment may refer to the method flow described in the foregoing method embodiment, and will not be described herein.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.
In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. The storage medium includes a U disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, an optical disk, or other various media capable of storing program codes.
While the application has been described in detail with reference to the foregoing embodiments, it will be understood by those skilled in the art that the foregoing embodiments may be modified or equivalents may be substituted for some of the features thereof, and that the modifications or substitutions do not depart from the spirit and scope of the embodiments of the application.

Claims (24)

1.一种数据处理的方法,其特征在于,包括:1. A method for data processing, comprising: 响应于存储过程请求,执行存储过程,所述存储过程包括一个或多个有序的事务;In response to a stored procedure request, executing a stored procedure, the stored procedure comprising one or more ordered transactions; 当所述一个或多个有序的事务中的第一事务执行结束时,记录所述第一事务的标记信息,所述标记信息包括所述第一事务的结束语句的行号;When a first transaction in the one or more ordered transactions is executed to completion, recording marking information of the first transaction, the marking information including a line number of an end statement of the first transaction; 基于执行所述存储过程中出现异常,且满足重试所述存储过程的条件,根据所述第一事务的标记信息重试所述存储过程中所述第一事务之后的事务。Based on an exception occurring during the execution of the stored procedure and the condition for retrying the stored procedure being met, a transaction subsequent to the first transaction in the stored procedure is retried according to the marking information of the first transaction. 2.根据权利要求1所述的方法,其特征在于,所述方法还包括:2. The method according to claim 1, characterized in that the method further comprises: 记录调用所述存储过程的请求语句。Record the request statement that calls the stored procedure. 3.根据权利要求1中任一项所述的方法,其特征在于,所述方法还包括:3. The method according to any one of claims 1, characterized in that the method further comprises: 记录所述存储过程的中间状态信息,所述中间状态信息包括以下一个或多个:所述第一事务的执行语句结果、错误提示信息、输入参数、输出参数、全局变量和临时变量。The intermediate state information of the stored procedure is recorded, wherein the intermediate state information includes one or more of the following: the execution statement result, error prompt information, input parameters, output parameters, global variables and temporary variables of the first transaction. 4.根据权利要求3所述的方法,其特征在于,所述中间状态信息包括第一事务的执行语句结果或错误提示信息,所述方法还包括:4. The method according to claim 3, wherein the intermediate state information includes an execution statement result or error prompt information of the first transaction, and the method further comprises: 缓存所述第一事务的执行语句结果或错误提示信息,直至所述第一事务结束之后,再将所述执行语句结果或错误提示信息发送给用户设备。Cache the execution statement result or error prompt information of the first transaction until the first transaction ends, and then send the execution statement result or error prompt information to the user device. 5.根据权利要求3所述的方法,其特征在于,所述中间状态信息包括第一事务的执行语句结果或错误提示信息,所述方法还包括:5. The method according to claim 3, wherein the intermediate state information includes an execution statement result or error prompt information of the first transaction, and the method further comprises: 缓存所述第一事务的执行语句结果或错误提示信息,直至所述存储过程结束之后,再将所述执行语句结果或错误提示信息发送给用户设备。The execution statement result or error prompt information of the first transaction is cached until the storage process ends, and then the execution statement result or error prompt information is sent to the user device. 6.根据权利要求1中任一项所述的方法,其特征在于,所述第一事务位于所述存储过程中的分支语句的一个分支,所述方法还包括:6. The method according to any one of claims 1, characterized in that the first transaction is located in a branch of a branch statement in the stored procedure, and the method further comprises: 记录所述存储过程的中间状态信息,所述中间状态信息包括分支执行状态,所述分支执行状态用于标识所述第一事务所在的分支;Recording intermediate state information of the stored procedure, wherein the intermediate state information includes a branch execution state, and the branch execution state is used to identify the branch where the first transaction is located; 重试所述存储过程中所述第一事务之后的事务包括:基于所述分支执行状态,执行所述第一事务所在的分支中第一事务之后的事务。Retrying the transaction after the first transaction in the stored procedure includes: executing the transaction after the first transaction in the branch where the first transaction is located based on the branch execution status. 7.根据权利要求6所述的方法,其特征在于,7. The method according to claim 6, characterized in that 所述分支执行状态包括执行所述存储过程中所述分支语句的条件判断结果。The branch execution status includes a conditional judgment result of executing the branch statement in the stored procedure. 8.根据权利要求1至7中任一项所述的方法,其特征在于,所述重试所述存储过程的条件为所述存储过程异常为可恢复性异常,所述可恢复性异常包括:网络异常、内存使用异常、输入输出异常、锁等待超时异常或两阶段事务提交异常。8. The method according to any one of claims 1 to 7 is characterized in that the condition for retrying the stored procedure is that the stored procedure exception is a recoverable exception, and the recoverable exception includes: network exception, memory usage exception, input and output exception, lock wait timeout exception or two-phase transaction commit exception. 9.根据权利要求1至7中任一项所述的方法,其特征在于,所述重试所述存储过程中所述第一事务之后的事务包括:9. The method according to any one of claims 1 to 7, characterized in that the retrying of the transaction after the first transaction in the storage process comprises: 等待预设的第一时长后,重试所述存储过程中所述第一事务之后的事务。After waiting for a preset first time period, the transaction after the first transaction in the storage process is retried. 10.根据权利要求1至7中任一项所述的方法,其特征在于,所述方法还包括:10. The method according to any one of claims 1 to 7, characterized in that the method further comprises: 当所述一个或多个有序的事务中的第二事务执行结束时,记录所述第二事务的标记信息,所述第二事务为在所述第一事务之后执行的事务;When a second transaction in the one or more ordered transactions is executed, record marking information of the second transaction, where the second transaction is a transaction executed after the first transaction; 所述重试所述存储过程中所述第一事务之后的事务包括:The retrying of the transaction after the first transaction in the stored procedure includes: 根据所述第二事务的标记信息重试所述存储过程中所述第二事务之后的事务,所述第二事务为满足重试所述存储过程的条件时,已执行事务中最后被执行的事务。The transaction after the second transaction in the stored procedure is retried according to the marking information of the second transaction, where the second transaction is the last transaction executed among the executed transactions when the condition for retrying the stored procedure is met. 11.一种数据库管理系统,其特征在于,包括:11. A database management system, comprising: 执行单元,用于响应于存储过程请求,执行存储过程,所述存储过程包括一个或多个有序的事务;An execution unit, configured to execute a stored procedure in response to a stored procedure request, wherein the stored procedure includes one or more ordered transactions; 记录单元,用于当所述一个或多个有序的事务中的第一事务执行结束时,记录所述第一事务的标记信息,所述标记信息包括所述第一事务的结束语句的行号;a recording unit, configured to record, when a first transaction in the one or more ordered transactions is finished executing, marking information of the first transaction, wherein the marking information includes a line number of an end statement of the first transaction; 重试单元,用于基于执行所述存储过程中出现异常,且满足重试所述存储过程的条件,根据所述第一事务的标记信息重试所述存储过程中所述第一事务之后的事务。A retry unit is used to retry a transaction after the first transaction in the stored procedure according to the marking information of the first transaction if an exception occurs during the execution of the stored procedure and a condition for retrying the stored procedure is met. 12.根据权利要求11所述的数据库管理系统,其特征在于,所述记录单元还用于:12. The database management system according to claim 11, characterized in that the recording unit is further used for: 记录调用所述存储过程的请求语句。Record the request statement that calls the stored procedure. 13.根据权利要求11所述的数据库管理系统,其特征在于,所述记录单元还用于:13. The database management system according to claim 11, characterized in that the recording unit is further used for: 记录所述存储过程的中间状态信息,所述中间状态信息包括以下一个或多个:所述第一事务的执行语句结果、错误提示信息、输入参数、输出参数、全局变量和临时变量。The intermediate state information of the stored procedure is recorded, wherein the intermediate state information includes one or more of the following: the execution statement result, error prompt information, input parameters, output parameters, global variables and temporary variables of the first transaction. 14.根据权利要求13所述的数据库管理系统,其特征在于,所述中间状态信息包括第一事务的执行语句结果或错误提示信息,所述数据库管理系统还包括:14. The database management system according to claim 13, wherein the intermediate state information includes the execution statement result or error prompt information of the first transaction, and the database management system further includes: 缓存单元,用于缓存所述第一事务的执行语句结果或错误提示信息,直至所述第一事务结束之后,再将所述执行语句结果或错误提示信息发送给用户设备。The cache unit is used to cache the execution statement result or error prompt information of the first transaction until the first transaction ends, and then send the execution statement result or error prompt information to the user device. 15.根据权利要求13所述的数据库管理系统,其特征在于,所述中间状态信息包括第一事务的执行语句结果或错误提示信息,所述数据库管理系统还包括:15. The database management system according to claim 13, wherein the intermediate state information includes the execution statement result or error prompt information of the first transaction, and the database management system further comprises: 缓存单元,用于缓存所述第一事务的执行语句结果或错误提示信息,直至所述存储过程结束之后,再将所述执行语句结果或错误提示信息发送给用户设备。The cache unit is used to cache the execution statement result or error prompt information of the first transaction until the storage process is completed, and then send the execution statement result or error prompt information to the user device. 16.根据权利要求11所述的数据库管理系统,其特征在于,所述第一事务位于所述存储过程中的分支语句的一个分支,所述记录单元还用于:16. The database management system according to claim 11, wherein the first transaction is located in a branch of a branch statement in the stored procedure, and the recording unit is further used for: 记录所述存储过程的中间状态信息,所述中间状态信息包括分支执行状态,所述分支执行状态用于标识所述第一事务所在的分支;Recording intermediate state information of the stored procedure, wherein the intermediate state information includes a branch execution state, and the branch execution state is used to identify the branch where the first transaction is located; 所述重试单元,具体用于基于所述分支执行状态,执行所述第一事务所在的分支中第一事务之后的事务。The retry unit is specifically configured to execute a transaction subsequent to the first transaction in the branch where the first transaction is located based on the branch execution status. 17.根据权利要求16所述的数据库管理系统,其特征在于,所述分支执行状态包括执行所述存储过程中所述分支语句的条件判断结果,所述条件判断结果用于确定执行所述分支语句的哪一个分支。17. The database management system according to claim 16, characterized in that the branch execution status includes a conditional judgment result of executing the branch statement in the stored procedure, and the conditional judgment result is used to determine which branch of the branch statement is executed. 18.根据权利要求11至17中任一项所述的数据库管理系统,其特征在于,所述重试所述存储过程的条件为所述存储过程异常为可恢复性异常,所述可恢复性异常包括:网络异常、内存使用异常、输入输出异常、锁等待超时异常或两阶段事务提交异常。18. The database management system according to any one of claims 11 to 17, characterized in that the condition for retrying the stored procedure is that the stored procedure exception is a recoverable exception, and the recoverable exception includes: network exception, memory usage exception, input and output exception, lock wait timeout exception or two-phase transaction commit exception. 19.根据权利要求11至17中任一项所述的数据库管理系统,其特征在于,所述重试单元具体用于:19. The database management system according to any one of claims 11 to 17, characterized in that the retry unit is specifically used for: 等待预设的第一时长后,重试所述存储过程中所述第一事务之后的事务。After waiting for a preset first time period, the transaction after the first transaction in the storage process is retried. 20.根据权利要求11至17中任一项所述的数据库管理系统,其特征在于,20. The database management system according to any one of claims 11 to 17, characterized in that: 所述记录单元还用于,当所述一个或多个有序的事务中的第二事务执行结束时,记录所述第二事务的标记信息,所述第二事务为在所述第一事务之后执行的事务;The recording unit is further configured to record, when a second transaction in the one or more ordered transactions is completed, marking information of the second transaction, where the second transaction is executed after the first transaction; 所述重试单元具体用于:The retry unit is specifically used for: 根据所述第二事务的标记信息重试所述存储过程中所述第二事务之后的事务,所述第二事务为满足重试所述存储过程的条件时,已执行事务中最后被执行的事务。The transaction after the second transaction in the stored procedure is retried according to the marking information of the second transaction, where the second transaction is the last transaction executed among the executed transactions when the condition for retrying the stored procedure is met. 21.一种数据处理设备,其特征在于,包括:21. A data processing device, comprising: 存储器,用于存储指令;A memory for storing instructions; 处理器,用于执行所述存储器中的指令,使得所述数据处理设备执行权利要求1至10中任一项所述的方法。A processor, configured to execute instructions in the memory so that the data processing device executes the method according to any one of claims 1 to 10. 22.一种计算机程序产品,其特征在于,所述计算机程序产品包括指令,当所述指令在计算机上运行时,使得所述计算机执行权利要求1至10中任一项所述的方法。22. A computer program product, characterized in that the computer program product comprises instructions, and when the instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 10. 23.一种计算机可读存储介质,其特征在于,所述计算机可读存储介质存储指令,当所述指令在计算机上运行时,使得所述计算机执行权利要求1至10中任一项所述的方法。23. A computer-readable storage medium, characterized in that the computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the method according to any one of claims 1 to 10. 24.一种数据库系统,其特征在于,包括:网络接口,用于接收用户设备发送的存储过程请求;数据处理设备,用于响应于所述存储过程请求,执行如权利要求1至10中任一项所述的方法。24. A database system, characterized in that it comprises: a network interface for receiving a stored procedure request sent by a user device; and a data processing device for executing the method according to any one of claims 1 to 10 in response to the stored procedure request.
CN201911040327.0A 2019-10-29 2019-10-29 Data processing method, database management system and data processing device Active CN112749156B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911040327.0A CN112749156B (en) 2019-10-29 2019-10-29 Data processing method, database management system and data processing device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911040327.0A CN112749156B (en) 2019-10-29 2019-10-29 Data processing method, database management system and data processing device

Publications (2)

Publication Number Publication Date
CN112749156A CN112749156A (en) 2021-05-04
CN112749156B true CN112749156B (en) 2025-02-18

Family

ID=75640201

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911040327.0A Active CN112749156B (en) 2019-10-29 2019-10-29 Data processing method, database management system and data processing device

Country Status (1)

Country Link
CN (1) CN112749156B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114501057B (en) * 2021-12-17 2024-06-14 阿里巴巴(中国)有限公司 Data processing method, storage medium, processor and system

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104508663A (en) * 2012-07-27 2015-04-08 微软公司 Automatic transaction retry after session failure
CN108205464A (en) * 2016-12-20 2018-06-26 阿里巴巴集团控股有限公司 A kind of processing method of database deadlocks, device and Database Systems

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106033439B (en) * 2015-03-13 2019-10-22 阿里巴巴集团控股有限公司 A kind of distributed transaction processing method and system
CN107506384A (en) * 2017-07-25 2017-12-22 北京供销科技有限公司 The discovery modification method and system of a kind of Data Consistency

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104508663A (en) * 2012-07-27 2015-04-08 微软公司 Automatic transaction retry after session failure
CN108205464A (en) * 2016-12-20 2018-06-26 阿里巴巴集团控股有限公司 A kind of processing method of database deadlocks, device and Database Systems

Also Published As

Publication number Publication date
CN112749156A (en) 2021-05-04

Similar Documents

Publication Publication Date Title
US11914572B2 (en) Adaptive query routing in a replicated database environment
US10795881B2 (en) Table replication in a database environment
Plattner et al. Ganymed: Scalable replication for transactional web applications
US10248709B2 (en) Promoted properties in relational structured data
EP3391244B1 (en) Replication control among redundant data centers
WO2019109854A1 (en) Data processing method and device for distributed database, storage medium, and electronic device
US11226985B2 (en) Replication of structured data records among partitioned data storage spaces
US10235406B2 (en) Reminder processing of structured data records among partitioned data storage spaces
US12210505B2 (en) Operation request processing method, apparatus, device, readable storage medium, and system
US12093241B2 (en) Method for replaying log on data node, data node, and system
CN112148436B (en) Decentralized TCC transaction management method, device, equipment and system
US20250200070A1 (en) Consensus Protocol For Asynchronous Database Transaction Replication With Fast, Automatic Failover, Zero Data Loss, Strong Consistency, Full SQL Support And Horizontal Scalability
US9612921B2 (en) Method and system for load balancing a distributed database providing object-level management and recovery
US20240311170A1 (en) System and method supporting highly-available replicated computing applications using deterministic virtual machines
CN118733604B (en) Database updating method and device, electronic device and storage medium
CN112749156B (en) Data processing method, database management system and data processing device
US20240126781A1 (en) Consensus protocol for asynchronous database transaction replication with fast, automatic failover, zero data loss, strong consistency, full sql support and horizontal scalability
US12360961B2 (en) Hybrid database implementations
US12259891B2 (en) Hybrid database implementations
US20240256551A1 (en) Execution tracing for node cluster
US20250021572A1 (en) Hybrid database implementations
CN114637763A (en) An error information processing method and module based on OpenGauss
CN120067210A (en) Database synchronization method, system and medium based on data synchronization
CN119149620A (en) ETL full-increment dynamic mapping data integration system and method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant