CN119513082A

CN119513082A - Data processing method, electronic device and storage medium

Info

Publication number: CN119513082A
Application number: CN202311076689.1A
Authority: CN
Inventors: 李源涛; 方佩文
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2023-08-24
Filing date: 2023-08-24
Publication date: 2025-02-25

Abstract

The application provides a data processing method, electronic equipment and a computer readable storage medium, and relates to the technical field of databases. In the application, when the DDL operation is carried out on the table in the database, the table is not locked, and the DML operation can be carried out on the table at the moment. In the process of performing the DDL operation on the table, when performing the DML operation on the table, the data to be DML operated in the table may be first subjected to reconstruction processing, so that the data to be DML operated satisfies the structure of the table after being DDL operated, and then the data to be DML operated in the table is subjected to the DML operation. Thus, the consistency of the data in the table can be ensured, and the blocking of the DDL operation to the DML operation can be avoided.

Description

Data processing method, electronic device and storage medium

Technical Field

The present application relates to the field of database technologies, and in particular, to a data processing method, an electronic device, and a computer readable storage medium.

Background

With the development of business, a large amount of data in an application system needs to be stored in a database. The data in the database may be stored in the form of tables, and a user may query the data in the database via a data query language (data query language, DQL) provided by a database management system (database MANAGEMENT SYSTEM, DBMS), may perform DDL operations on the tables in the database via a data definition language (DDL, data definition language) provided by the DBMS system, such as modifying the structure of the tables, and may perform DML operations on the tables in the database via a data manipulation language (data manipulation language, DML) provided by the DBMS system, such as modifying the data in the tables.

In the prior art, in order to ensure the consistency of data, when a certain table in a database is operated by a DDL, the table is locked, and at the moment, the table can not be operated by the DML any more, so that the DDL operation blocks the DML operation, thereby reducing the efficiency of data processing. Therefore, how to avoid the DDL operation from blocking the DML operation becomes a technical problem to be solved.

Disclosure of Invention

Some embodiments of the application provide a data processing method, an electronic device, a computer readable storage medium, and a computer program product. The technical scheme of the application can avoid the DDL operation from blocking the DML operation.

In a first aspect, the application provides a data processing method for electronic equipment, which is used for receiving a first instruction in the process of performing first change operation on a first table, wherein the first instruction is used for indicating that first data in the first table is changed into second data, determining that the first change operation does not adjust the format of the first data in the first table to meet the format requirement of the first table after the first change operation, adjusting the format of the first data in the first table to meet the format requirement of the first table after the first change operation, and modifying the first data in the first table to the second data.

The first change operation may be a DDL operation mentioned in the present application, and the first instruction may be a DML instruction mentioned in the present application. The first table may be a table a mentioned in the present application, specifically, for example, a student information table or the like.

When the DDL operation is carried out on the table in the database, the data processing method does not lock the table, and the DML operation can be carried out on the table. In the process of performing the DDL operation on the table, when the DML operation is performed on the table, the data to be DML operated in the table may be subjected to reconstruction processing first, so that the data to be DML operated satisfies the structure of the table after being DDL operated, that is, the format of the data to be DML operated satisfies the format requirement of the table after being DDL operated, and then the data to be DML operated in the table is subjected to the DML operation. Thus, the consistency of the data in the table can be ensured, and the blocking of the DDL operation to the DML operation can be avoided.

In some embodiments, determining that the first change operation has not adjusted the format of the first data in the first table to satisfy the format requirement of the first table after the first change operation includes determining that the first change operation has not adjusted the format of the first data to satisfy the format requirement of the first table after the first change operation in response to determining that the format of the first data does not match the format parameters of the first table after the first change operation.

Wherein the format parameter may be metadata mentioned in the present application. Metadata includes a database name, a database character set, a table name, a table size, a recording line number of a table, a character set of a table, a field of a table, an index of a table, a description of a table, a type of a field, precision of a field, description of a field, and the like.

In some embodiments, adjusting the format of the first data in the first table to satisfy the format requirement of the first table after the first change operation includes invoking a data reconfiguration thread to adjust the format of the first data so that the format of the first data matches the format parameters of the first table after the first change operation.

In the process of performing DDL operation on the table A in the database, the format of the data in the first table A of the data reconstruction thread can be called to be adjusted, so that the format of the data is matched with the metadata of the table A after the DDL operation.

Illustratively, the first behavior column name of the student information table is "name", "age", "class", and the corresponding metadata is "name, age, class" in this order. Student information corresponding to the fourth action of the student information table is 'Wangwu, 13 and Jiuban' in sequence. After the metadata "name, age, class" is updated to "name, age, class", a column of "class" is added between the second column and the third column of the student information table. At this time, the fourth line data "wang five, 13, jiu ban" of the student information table needs to be modified to "wang five, 13, null, jiu ban" so that the fourth line data matches the modified metadata.

In some embodiments, invoking the data reconstruction thread to adjust the format of the first data includes invoking the data reconstruction thread to parse a second instruction for indicating the first change operation to determine a format parameter of the first table after the first change operation, and invoking the data reconstruction thread to adjust the format of the first data based on the format parameter of the first table after the first change operation.

Wherein the second instruction may be a DML instruction.

After the data in the table is operated by DDL, the structure of the table is changed, and if DML is directly operated on the data in the table, the modified data is not matched with the structure of the table, that is, the modified data is matched with metadata of the table. Before the electronic device performs the DML operation on the data in the table, the data reconstruction thread is called to adjust the format of the data (i.e., the first data) to be subjected to the DML operation, so that the format of the data to be subjected to the DML operation can be matched with the structure of the table, and the situation that the data of the table is inconsistent after the data to be subjected to the DML operation is performed is avoided.

In some embodiments, the first instruction comprises a data manipulation language DML instruction and the second instruction comprises a data definition language DDL instruction.

In some embodiments, the method further comprises performing a first change operation on the first table to obtain a second table, storing the second table, receiving a first request sent by the first application of the first version, wherein the first request is used for requesting binding with the second table, and binding the first application with the second table.

Wherein the first application may be a student information management application. The first table may be a table before the DDL operation and the second table may be a table after the DDL operation.

Illustratively, after the V1.1 version of table a (as an example of the first table) is updated to the V1.2 version of table a (as an example of the second table) after being ddled, after the V1.2 version of APP sends a first request to the database, the database may bind the V1.2 version of table a with the V1.2 version of APP such that the V1.2 version of APP may use the V1.2 version of table a. Thus, applications of different versions can use tables of different versions, and applications of an old version can continue to use tables of an old version without using tables of a latest version, i.e., without adapting to tables of a latest version. Thus, the continuity of service is ensured, and the workload can be reduced.

In some embodiments, the method further comprises binding the second application with the first table, receiving a third request sent by the second application, the third request being used for indicating that the second application is switched to be bound with the second table, unbinding the second application from the first table, and binding the second application with the second table.

The second application may be a pre-update application of the first application, such as the first application is version V1.2 APP, the second application is version V1.1 APP, and when version V1.1 APP is bound to version V1.1 table a, after version V1.1 table a is updated to version V1.2 table a, version V1.1 APP may send a third request to the database. The request database unbinds the V1.1 version of APP from the V1.1 version of table a, and then binds the V1.1 version of APP to the V1.2 version of table a, so that the V1.1 version of APP can use the V1.2 version of table a, i.e., can use the updated table a. Therefore, the application program can switch different tables for use, and has stronger flexibility.

In some embodiments, the method further comprises deleting the first table in response to determining that all of the data in the first table meets the format requirements of the first table and that the first table is unbound with the application.

When it is determined that the metadata update of the first table is completed, the data in the table is completely reconstructed, and the first table is not bound with any application program, it is indicated that the first table is not used any more, and at this time, the first table can be deleted to save storage space.

In a second aspect, an embodiment of the present application provides an electronic device, including a memory, configured to store instructions for execution by one or more processors of the electronic device, where the processor, when executing the instructions in the memory, may cause the electronic device to perform a method according to any embodiment of the first aspect of the present application. The advantages achieved by the second aspect may refer to the advantages provided by any embodiment of the first aspect, and are not described herein.

In a third aspect, embodiments of the present application provide a computer-readable storage medium having stored thereon instructions that, when executed on a computer, cause the computer to perform the method according to any of the embodiments of the first aspect. The advantages achieved by the third aspect may refer to the advantages provided by any embodiment of the first aspect, and are not described herein.

In a fourth aspect, embodiments of the present application provide a computer program product comprising a computer program/instruction which, when executed, causes a computer to perform the method of any of the embodiments of the first aspect. The advantages achieved by the fourth aspect may refer to the advantages provided by any embodiment of the first aspect, and are not described herein.

Drawings

FIG. 1 illustrates an exemplary application scenario of the present application;

FIG. 2A is a schematic illustration of DDL operation provided by some embodiments;

FIG. 2B is a schematic illustration of the operation of a DML provided by some embodiments;

FIG. 2C is a schematic illustration of the operation of a DML according to an embodiment of the present application;

FIG. 3 is a flowchart illustrating a data processing method according to an embodiment of the present application;

FIG. 4A is an exemplary diagram of a table update provided by an embodiment of the present application;

FIG. 4B is an exemplary diagram of an application of metadata multi-version management according to an embodiment of the present application;

FIG. 5 is a flowchart illustrating DDL operations provided by an embodiment of the present application;

FIG. 6 is a flowchart illustrating a data reconstruction process according to an embodiment of the present application;

FIG. 7 is a flowchart illustrating a DML operation according to an embodiment of the present application;

FIG. 8 is a flowchart illustrating an example of the DQL operation provided by an embodiment of the present application;

FIG. 9 is a flowchart illustrating metadata cleaning according to an embodiment of the present application;

FIG. 10 is a diagram illustrating an exemplary architecture of a database according to an embodiment of the present application;

FIG. 11 is a flowchart illustrating an online DDL provided by an embodiment of the present application;

FIG. 12 is a diagram illustrating an exemplary architecture of a database according to an embodiment of the present application;

FIG. 13 is a flowchart illustrating a data reconstruction process according to an embodiment of the present application;

fig. 14 shows a block diagram of an electronic device provided by an embodiment of the application.

Detailed Description

The embodiment of the application is used for providing a data processing method. The method provided by the application supports concurrent execution of DDL operation and DML operation, and avoids the DDL operation from blocking the DML operation.

In order to facilitate understanding of the technical solution of the present application, the following first explains some terms related to the present application.

Database Metadata (Metadata) is some data belonging to the database itself, including database name, database character set, table name, table size, number of record lines of table, character set of table, field of table, index of table, description of table, type of field, precision of field, description of field, etc. The "database metadata" will be referred to herein simply as "metadata".

DDL operations are operations that define, modify, and adjust database schemas (schemas) of a database. schema is a structural description of a database. Within a relational database, schema defines tables, fields in tables, relationships between tables and fields, indexes of tables, column names of tables, and the like. DDL operations may be understood in the present application as modifications to metadata.

DML operations are operations that add, delete, modify, etc. data in a table.

The DQL operation is a query operation on data in a database.

Fig. 1 shows an exemplary application scenario of the present application. As shown in fig. 1, in the case where a certain service (for example, an e-commerce service, a student information management service, an office service) has a lot of data, the data of the service may be stored in a database of a database server (refer to a server where the database is disposed), and then a terminal device such as a computer or a mobile phone may access the database of the database server to obtain the data. However, as the service progresses, the data stored in the database may need to be modified, and at this time, the user may send corresponding data modification instructions, such as DDL instructions, DML instructions, etc., to the database server through a terminal device connected to the database server. After the database receives the data modification instruction, a DBMS system for managing the database can be called to modify the data in the database.

In the application, the terminal device can be electronic devices such as a notebook computer, a desktop computer, a tablet computer, a mobile phone and the like, and is not limited to the above. The databases may be structured query language (structured query language, SQL), non-relational databases (not only structuredquery language, noSQL), oracle, mySQL, openGauss, etc. types of databases. The database of the present application may be deployed in not only a server, but also electronic devices such as a cloud platform, an edge device (e.g., an edge server, a router, a switch, an internet of things device, etc.), a mobile device (e.g., a notebook computer, a tablet computer, a mobile phone, etc.), and the like, which is not limited thereto.

In some embodiments, when the user a wants to change the table a in the database, for example, modify the structure of the table a, the user a may send a corresponding DDL instruction to the database server through a terminal device such as a computer, so as to instruct the database to perform DDL operation on the table a to modify the structure of the table a. In order to ensure the consistency of the data, after performing DDL operation on the table a, the database needs to reconstruct the data (such as user data) in the table a, so that the data in the table a meets the format requirement after the table a is changed. In addition, before the data in the table A is reconstructed, the database locks the table A so as to avoid that other users modify the data in the table A, so that the data in the table A does not meet the format requirement after the table A is changed. In the case where table a is locked, the DML instruction will be put aside when it is received by the database. The DML instruction is executed to modify the reconstructed data in Table A only after the structural modification of Table A is completed and all the data in Table A is reconstructed. That is, the DDL instruction and the DML instruction cannot be executed concurrently, and the DDL operation may block the DML operation. Thus, the modification efficiency of the table is reduced.

Illustratively, as shown in fig. 2A, when the user a wants to add a column (whose column name is "grade") before the third column of the student information table (as an example of table a), a corresponding DDL instruction may be transmitted to the database through the terminal device to instruct the database to perform a corresponding DDL operation, such as modifying metadata { name, age, class } of the student information table to metadata { name, age, grade, class }. Thus, a column named "rank" may be added before the third column of the student information table. Then, the database may call a data reconstruction thread to reconstruct the data in the student information table, so that the reconstructed data matches with the structure of the student information table after modification. For example, the data of the first row can be "Zhang three, 13, seven-shift" reconstructed into data "Zhang three, 13, NULL, seven-shift", where NULL indicates NULL. During the data reconstruction of the DDL operation, the student information table needs to be locked, so that other users are prevented from modifying the data in the student information table, and the consistency of the data is ensured. If the student information table is unlocked during data reconstruction of DDL operations, data anomalies may occur when other users modify the data in the student information table.

For example, as shown in fig. 2B, if the student information table is not locked during DDL operation on the student information table, when data "wang five, 13, nine shift" in the third row of the student information table is modified to data "wang five, 13, seven shift" it causes "seven shift" in the data to appear in the column of "grade", resulting in that the data does not correspond to the column name, that is, the data is inconsistent. Thus, during DDL operations on student information tables, locking of the student information tables is required. But locking the student information table may cause the DDL operation to block the DML operation, thereby reducing the processing efficiency of data.

In order to solve the technical problems, the application provides a data processing method. In the embodiment of the application, in the process that the database performs DDL operation on the table in the database based on the DDL instruction (for example, changes the structure of the table), when the database receives the DML instruction, the database judges whether the data in the table meets the format requirement of the table after being operated by the DDL based on the DDL instruction. When the database determines that the data does not meet the format requirement of the table after being subjected to the DDL operation, a data reconstruction thread is called to reconstruct the data so that the data meets the corresponding format requirement, and then the data is subjected to the DML operation (such as modification). Thus, the consistency of the data in the table can be ensured under the condition that the table is not locked. And the database does not need to wait until the DDL operation is executed, and then execute the DML operation, so that the DDL operation can be prevented from blocking the DML operation, and the data processing efficiency is improved.

Illustratively, as shown in FIG. 2C, the student information table is not locked in the course of the database performing the DDL operation on the student information table, in which case the database may perform the DML operation on the student information table. For example, if the third row of the student information table has not been reconstructed yet, and the user wants to modify the class of the student "wangwu" in the third row from "jiuban" to "qiban", the database may first invoke the data reconstruction thread to reconstruct the data "wangwu, 13, jiuban" of the third row into "wangwu, 13, null, jiuban", and then perform the DML operation on the reconstructed data "wangwu, 13, null, jiuban" to "wangwu, 13, null, qiban". Thus, not only can the mismatch between the data and the structure of the table be avoided, but also the blocking can be avoided.

Specific embodiments of the present application are described below.

Fig. 3 is a flowchart illustrating a data processing method according to an embodiment of the present application. As shown in fig. 3, the method comprises the steps of:

s101, the database receives a DDL instruction aiming at a table A.

When a user wants to change a table a of a database in the database, for example, the structure of the table a is changed, a DDL instruction corresponding to the table a can be sent to the database through a terminal device connected with the database to instruct the server to perform DDL operation on the table a.

For example, as shown in fig. 2A, the student information table includes four rows and three columns. The first behavior column name includes "name", "age", "class", and the corresponding metadata includes "name, age, class". Student information of second row to fourth row, wherein the second row comprises 'Zhang Sang', '13', 'seven classes', the third row comprises 'Lisi', '12', 'eight classes', and the fourth row comprises 'Wang Wu', '13', 'nine classes'. When a user wants to insert a column named "grade" before the third column "class" of the student information table (as an example of table a) in the database, a corresponding DDL instruction may be transmitted to the database through a terminal device connected to the database to instruct the database to insert a column in front of the third column of the student information table and its column name is "grade".

S102, the database performs DDL operation on the table A in the database.

After receiving the DDL instruction for table a, the database performs DDL operations on table a in the database, such as modifying the metadata of table a to alter the structure of table a, etc.

In order to ensure the data consistency of the table a, the database may reconstruct the data in the table a after the structure of the table a is changed, so that the data in the table a is matched with the modified structure of the table a, that is, the format of the data in the table a meets the format requirement of the table a. After the database has completed reconstructing all of the data in table a, the entire DDL operation process ends.

For example, as shown in fig. 2A, after receiving the DDL instruction for the student information table, the database may modify the metadata "name, age, class" of the student information table into "name, age, grade, class", and thus a column named "grade" may be inserted before the third column of the student information table. After the structure of the student information table is modified by the database, the data reconstruction thread can be called to reconstruct the data in the student information table, so that the data in the student information table is matched with the structure of the student information table, namely, the format of the data in the student information table meets the format requirement of the student information table.

It will be appreciated that each time the database performs a DDL operation on table a, table a will be updated. In some cases, after the table a is updated, if an application program (APP) using the table a (such as a student information management application) does not perform corresponding adaptation based on the updated table a, the updated table a is still parsed according to a parsing manner before updating, and parsing abnormality may occur, so that data read by the APP is abnormal, and thus corresponding business is abnormal.

For example, as shown in fig. 4A, after the database performs DDL operation on the student information table, a column of "grade" is inserted between the second column "age" and the third column "class" of the original student information table, so that the student information table is updated from three columns to four columns. At this time, if the student information management application analyzes the updated student information table according to the analysis mode before updating, only the first three columns of data of the student information table can be read, but the fourth column of data cannot be read, and class information of the students cannot be queried, so that the business of the student information management application is abnormal.

Based on this, after each DDL operation of table a by the database, the APP using table a needs to be adapted accordingly. However, during the adaptation period, the APP cannot use the table a, so that the service of the APP is interrupted, the continuity of the service is poor, and the problem that the service is difficult to smoothly evolve exists. Also, the versions of APP (such as student information management application) installed on the terminal devices of different users may be different, such as V1.1 version APP installed on the terminal device of user a and V1.2 version APP installed on the terminal device of user B. The V1.2 version of APP needs to use an updated table a (e.g., an updated student information table), and the V1.1 version of APP only needs to use an updated table a. However, since the table a is updated, the V1.1 version APP can only use the updated table a, and the V1.1 version APP needs to be adapted to the updated table a to avoid the analysis exception of the V1.1 version APP to the updated table a. Thus, a certain amount of work is increased.

In order to solve the problem that the service is difficult to evolve smoothly, in some embodiments, after each DDL operation performed on the table a in the database by the database, a metadata Snapshot (snap shot) of the table a (referring to a static view of metadata at a certain point in time) may be created and stored immediately, so as to obtain metadata of a corresponding version. Wherein each version of metadata corresponds to a version of the table. The version of the metadata snapshot is referred to herein as a "Schema Version (SV)". Thus, after the database performs DDL operations on the table a for a plurality of times, the table a corresponding to a plurality of SVs can be obtained. In this case, the old version of APP can use the old version of table a without adapting to the new version of table a, so that continuity of service can be ensured.

Illustratively, as shown in FIG. 4B, after adding column c2 to the database in form A of version V0.0, form A of version V1.1 may be obtained. After the database adds column c3 to form A of version V1.1, form A of version V1.2 can be obtained. After the database adds column c2 to form A of version V1.2, form A of version V1.3 can be obtained. Wherein, the table A of V1.1 version, the table A of V1.2 version and the table A of V1.3 version can be stored in the database to provide for the use of APP of different versions. Thus, even if the table A is updated, the use of the APP is not affected. For example, version V1.1 of APP can be bound to version V1.1 of SV, which corresponds to establishing a user link on version V1.1 of SV, which corresponds to version V1.1 of APP, so that version V1.1 of APP can use version V1.1 of Table A. Similarly, version V1.2 of APP can be bound to SV with version name V1.2, so that version V1.2 of APP can use version V1.2 of Table A. And version V1.3 APP can be bound to SV with version name V1.3 such that version V1.3 APP can use version V1.3 table a.

In the example of fig. 4B, when the table a is updated from the V1.2 version to the V1.3 version, the V1.2 version of APP can still use the V1.2 version of table a, and additional adaptation for the V1.3 version of table a is not required, so that interruption of the service of the V1.2 version of APP can be avoided, continuity of the service is ensured, and workload is reduced. And after the version of the APP is updated from the V1.2 version to the V1.3 version by the user, the V1.3 version of the APP is already adapted to the V1.3 version of the table A, so that the V1.3 version of the table A can be directly used, and the service of the APP can be smoothly migrated (evolved).

S103, the database receives a DML instruction aiming at the table A.

In the process of performing DDL operation on the table A by the database, the table A is not locked, and at this time, a user can send a DML instruction aiming at the table A to the database through the terminal equipment so as to perform operations of adding, deleting, changing and the like on data in the table A.

For example, as shown in fig. 2C, when the user wants to modify the class of the student "wang five" from "nine classes" to "seven classes", a corresponding DML instruction may be sent to the database through the terminal device to instruct the database to modify the class of the student "wang five" in the student information table from "nine classes" to "seven classes".

S104, the database judges whether the data to be subjected to the DML operation is reconstructed. If not, step S105 is performed. If yes, go to step S106.

After receiving the DML instruction, the database can judge whether the data to be subjected to the DML operation in the table A is completely reconstructed, if not, the data to be subjected to the DML operation is required to be reconstructed first, then the data is subjected to the DML operation, and if so, the data to be subjected to the DML operation is not required to be reconstructed, and the data can be directly subjected to the DML operation.

S105, the database calls a data reconstruction thread to reconstruct the data to be subjected to the DML operation.

When the database judges that the data to be subjected to the DML operation is not complete in reconstruction, the data reconstruction thread can be called to determine the modified structure of the table A, namely the modified metadata, based on the DDL instruction. And then reconstructing the data to be subjected to the DML operation based on the modified structure, so that the data is matched with the structure of the table A, namely, the format of the data meets the format requirement of the table A.

For example, as shown in fig. 2C, when the third line data of the student information table (as an example of data to be DML operated) is not completed to reconstruct, the database may call the data reconstruction thread to reconstruct the third line data of the student table "wang five, 13, nine shift" into "wang five, 13, null, nine shift" according to the metadata "name, age, grade, class" modified by the student information table.

It should be noted that, the data reconstruction of the DML operation and the data reconstruction of the DDL operation may be performed concurrently, and the data that has been reconstructed may not be reconstructed again. Because the data reconstruction of the DDL operation is to reconstruct the data of the whole table, and the DML operation is to reconstruct part of the data in the table, the data reconstruction of the DML operation can accelerate the data reconstruction of the DDL operation, thereby accelerating the execution of the DDL operation.

S106, the database performs DML operation on the data to be subjected to the DML operation.

After the data to be DML operated is reconstructed, the data to be DML operated meets the format requirement of the table a, and at this time, the database may execute the DML instruction to perform the corresponding DML operation on the data.

For example, as shown in fig. 2C, after the third row of data "wang five, 13, nine shift" of the student information table is reconstructed into "wang five, 13, null, nine shift", the database may execute the DML instruction to modify the reconstructed data "wang five, 13, null, nine shift" into "wang five, 13, null, seven shift". In the embodiment of the application, the database can simultaneously carry out DDL operation and DML operation on the tables in the database without blocking. This is because the embodiment of the present application decouples metadata update and data reconstruction, and the database can reconstruct not only data in a table when performing DDL operations, but also data in a table when performing DML operations. Therefore, when the database executes the DML operation, the data to be subjected to the DML operation in the table can be directly reconstructed without waiting for the completion of the DDL operation, namely without completing the reconstruction of the data of the whole table, so that the data to be subjected to the DML operation is matched with the structure of the table. And then performing DML operation on the data to be subjected to the DML operation, so that the situation of inconsistent data can be avoided. Based on this, DDL operations and DML operations can be performed concurrently, thereby improving data processing efficiency.

The following describes a specific procedure for DDL operation.

Fig. 5 is a flowchart illustrating a DDL operation according to an embodiment of the present application.

As shown in fig. 5, the DDL operation includes the steps of:

s201, analyzing the DDL instruction.

When a user wants to modify the structure of a certain table (e.g., a student information table) in the database, a corresponding DDL instruction can be sent to the database through the terminal device. After receiving the DDL instruction, the database can call the DBMS system to analyze the DDL instruction, and a corresponding analysis result is obtained.

And S202, updating metadata.

After the database analyzes the DDL instruction, the metadata of the table can be updated according to the analysis result.

For example, if the metadata "name, age, class" of the student information table is updated to "name, age, class" at the time of the parsing result of the DDL instruction, the database may modify the metadata "name, age, class" of the student information table to "name, age, class".

S203, writing a log.

In the embodiment of the application, the log generated before the metadata is updated can be written into the database, so that the information such as the updating process and the updating time of the metadata can be determined according to the log.

And S204, creating and storing a metadata snapshot.

After the metadata of the table is updated, the database can create and store the metadata snapshot of the table to obtain the corresponding SV, so that the APP with different versions can use the tables with different SVs, thereby realizing the metadata multi-version management of the database and ensuring the continuity of the service.

And S205, caching the updated metadata.

After updating the metadata of the table, the database may Buffer the metadata into a Buffer.

S206, starting an asynchronous thread to reconstruct data.

After the database updates the metadata of the table, the structure of the table is changed, and at this time, the database needs to start an asynchronous thread (namely a data reconstruction thread) to reconstruct the data in the table, so that the data of the table is matched with the updated structure of the table.

In the embodiment of the application, the database can update the metadata of the table according to the DDL instruction, and after the update is completed, the metadata snapshot of the table is stored, so that the metadata multi-version management of the database is realized, and the continuity of the service is ensured.

It will be appreciated that after the database performs a DDL operation on the tables in the database, the structure of the tables is changed. In order to avoid that the data in the table is not matched with the structure of the table, the data in the table needs to be reconstructed so that the data in the table meets the structural requirement of the table. The process of data reconstruction is described below.

Fig. 6 is a flowchart illustrating a data reconstruction process according to an embodiment of the present application.

As shown in fig. 6, the flow includes the steps of:

S301, starting a data reconstruction thread to scan the table Page by Page.

Tables in a database may be stored in pages, and during execution of DDL operations, the database may invoke a data reconstruction thread to scan a table (e.g., table a) in the database in units of pages.

S302, acquiring a Page version.

Page version refers to the version of the data in each Page of the table.

In some embodiments, each time the data of each Page table is updated, the data version (Page version) information for that Page may be stored. In this case, the database may obtain per Page table data version information to determine the Page version of the Page table. The data version information may include information of version number, version name, etc.

In other embodiments, each time data in the table is updated, the database generates a corresponding log to record update information of the data, such as update time, update result, and the like. In this case, the update time of the data in each Page table can be determined by querying the log, and then the Page version of the Page table is determined according to the update time.

And S303, newly creating or acquiring a reconstruction task.

In some embodiments, after the database obtains the Page version of a certain Page table, the database may determine the latest update time of the Page version of the Page table by querying the log corresponding to the Page table, and then determine whether the Page version is the latest version according to the latest update time. If yes, the data in the page table is completely reconstructed, and reconstruction is not needed. If not, the data in the Page table is not complete to reconstruct, and the database can call the DBMS system to create a reconstruction task for subsequent execution according to the data difference between the current Page version (source Page version) and the latest Page version (destination Page version). In some cases, if the DBMS system has previously created a corresponding reconstruction task, the database may directly retrieve the reconstruction task for execution in a subsequent step without the need to recreate the reconstruction task.

S304, reconstructing the multiple data in the Page.

A Tuple is a collection of a set of ordered objects, such as a row of data in a table.

In some embodiments, the database may invoke a data reconstruction thread to perform a reconstruction task, reconstructing each row of data of each page of the table in a row order.

For example, as shown in fig. 2A, when the database reconstructs data in the student information table, each row of data in the student information table may be reconstructed from the first row. Taking the first line of data as an example, the database may "reconstruct" the first line of data (the current Page version of data) into "Zhang three, 13, seven shifts" of data (the latest Page version of data).

In other embodiments, the data in the table may be reconstructed in other ways, for example, each column of data of the table may be reconstructed in units of columns starting from the first column of the table.

S305, updating the Page version.

After the database updates the data in the Page table, the Page version corresponding to the Page table is updated.

S306, completing Page reconstruction.

After the database is updated and the Page version of the Page table is updated, the data of the Page table is reconstructed, that is, the Page reconstruction is completed. In the embodiment of the application, the database can reconstruct each page of data of the table in such a way that all data in the table are matched with the structure of the table, and after the reconstruction of all data in the table is completed, the reconstruction task is ended.

In the embodiment of the application, when the structure of the table is changed, the database can reconstruct the data in the table, so that the data in the table is matched with the changed structure of the table, and the condition of inconsistent data is avoided.

In the embodiments described above with respect to fig. 5 and 6, the database does not lock the table during DDL operations on the table and reconstruction of the data in the table. In this case, the database may perform other operations on the table, such as a DML operation, a DQL operation, and the like.

The procedure for DML operation is described below.

Fig. 7 is a flowchart illustrating a DML operation according to an embodiment of the present application.

As shown in fig. 7, the flow of DML operations includes the steps of:

s401, analyzing a DML instruction.

In the case where the database performs DDL operations on a certain table in the database and the table is not locked, when a user wants to modify a certain page of data in the table, a corresponding DML instruction may be transmitted to the database through the terminal device. After receiving the DML instruction, the database can call the DBMS system to analyze the DML instruction to obtain a corresponding analysis result.

S402, determining a target Page.

After the database analyzes the DML instruction, the target Page to be modified can be determined according to the analysis result.

S403, reconstructing the target Page.

After determining the target Page, the database may invoke a data reconstruction thread to execute the DML instruction to reconstruct the data in the target Page, e.g., row-by-row, or column-by-column, etc.

For example, as shown in fig. 2C, the database may "reconstruct" wang five, 13, null, nine shift "from the third row data" wang five, 13, nine shift "in the student information table.

S404, executing DML operation update data.

After the reconstruction of the data in the target Page is completed, the database may execute the DML operation to perform the DML operation on the data in the target Page.

For example, as shown in fig. 2C, the database reconstructs the third row of data "wang wu, 13, nine shift" in the student information table into "wang wu, 13, null, nine shift", and then DML operation may be performed on the reconstructed data "wang wu, 13, null, nine shift" and the reconstructed data "wang wu, 13, null, nine shift" may be modified into "wang wu, 13, null, seven shift".

S405, finishing the DML operation.

And after the database finishes modifying the data in the target Page, finishing the DML operation.

In the embodiment of the application, the database firstly reconstructs data of a certain page in the table, and then performs DML operation on the data of the page, namely, performs the DML operation by taking the page as a unit. In other embodiments, the DML operations may also be performed in units of rows, columns, or a single data.

For example, the database may reconstruct a certain data in the table and then perform DML operation on the data.

For another example, the database may reconstruct a column of data in the table before performing DML operations on the column of data.

For another example, the database may reconstruct certain data in the table before performing DML operations on the data.

In the embodiment of the application, the database reconstructs the data in the table before performing the DML operation on the data, so that the data is matched with the structure of the table. Therefore, when the database performs DML operation on the data in the table, even if the structure of the table is changed, the abnormal phenomenon that the data in the table is inconsistent is not caused. In addition, the data in the table is required to be reconstructed during DDL operation, and the data in the table is firstly reconstructed before DML operation, so that the execution of DDL operation can be quickened.

It can be appreciated that by adding the data reconstruction operation before the DML operation, the table can be not locked any more during the DDL operation, and the DDL operation can be prevented from blocking the DML operation, thereby improving the execution efficiency of the DML operation.

Since the database does not lock a table when performing DDL operations on the table in the database, in this case, the database may perform DQL operations on not only the table but also the table. The procedure for DQL operation is described below.

Fig. 8 is a flowchart illustrating an DQL operation according to an embodiment of the application.

As shown in fig. 8, the flow of DQL operation includes the steps of:

S501, analyzing the DQL instruction.

When a user wants to query data in the database, the DQL instruction can be sent to the database through the terminal equipment, after the database receives the DQL instruction, the DBMS system can be called to analyze the DQL instruction to obtain a corresponding analysis result, and the analysis result can comprise information such as target data to be queried, page information (e.g. page header, page number, etc.) of a page where the target data is located, and version of a table where the target data is located.

S502, acquiring a request version.

In the embodiment of the application, after the database modifies the metadata of a certain table in the database, the latest metadata snapshot is stored to obtain the latest version of metadata. Thus, after the metadata of the table is modified for a plurality of times, metadata of multiple versions, that is, multiple versions of the table, are obtained.

In some embodiments, an APP (e.g., student information management application) on the terminal device may be bound to a designated SV when the terminal device is connected to the database. In this case, the designated SV may be taken as the requested version. In other embodiments, the latest SV may be the requested version if the APP (e.g., student information management application) on the terminal device is not bound to the designated SV while the terminal device is connected to the database.

S503, determining a target page.

After the database determines the table to which the target data to be queried belongs, the page to which the target data to be queried belongs can be determined from a plurality of pages of the target table as a target page according to the page information of the page to which the target data is located in the analysis result of the DQL instruction.

S504, inquiring target data of the target page.

And after the database determines the target page, inquiring target data from the target page according to the analysis result of the DQL instruction.

S505, returning the target data.

And after the database inquires the target data, returning the target data to the terminal equipment of the user. Thus, the user can inquire the target data through the terminal equipment.

In the embodiment of the application, the DQL operation only queries the data from the table, the data in the table is not modified, and the table is not locked. Accordingly, the DDL operation does not block the DQL operation, which can be normally performed.

Because the database can store one metadata snapshot every time the metadata of the table is modified, more versions of metadata can be stored, and the stored metadata needs to be cleaned in order to save storage space. The process of metadata cleaning is described below.

Fig. 9 is a flowchart illustrating metadata cleaning according to an embodiment of the present application.

As shown in fig. 9, the process of metadata cleaning includes the steps of:

and S601, deleting the metadata version.

Compared with the traditional database, the metadata cleaning method has the advantage that the metadata multi-version management scheme is adopted, and the metadata cleaning method is required to be correspondingly adjusted. The cleaning of the metadata is changed into version driving, namely, when the metadata corresponding to the version is confirmed to be not used any more, the metadata cleaning flow can be started.

In some embodiments, it may be determined that metadata is no longer being used when the metadata meets both conditions.

And on the condition one, no user data of a corresponding version exists, namely the data in the table after the metadata is updated is subjected to data reconstruction.

And on the condition II, no user link of a corresponding version exists, namely, the table of the SV is not bound with the APP.

When the metadata satisfies the first condition and the second condition at the same time, the metadata version is not used any more, and the metadata version can be deleted.

After metadata that is no longer used is determined based on the above condition one and condition two, the SV corresponding to the metadata may be deleted. In a database management system (i.e., DBMS system), each transaction has a unique transaction commit sequence number (commit sequencenumber, CSN), which is typically represented as an incremented integer or other unique identifier, that is used to determine the commit order of the transactions. Each time a metadata modification transaction is executed, the CSN may be submitted once to the transaction manager to determine the order of the metadata modification transactions and thus the SV based on the CSN. It is understood that one SV corresponds to one CSN.

S602, acquiring a metadata version CSN.

In some embodiments, after determining an SV corresponding to metadata that is no longer being used, a CSN corresponding to the SV may be obtained from the transaction manager.

S603, truncating the metadata log by taking CSN as Forzen points.

The metadata log is a log in which metadata is recorded.

In some embodiments, SVs satisfying the above condition one and condition two may be determined, then, among those SVs, the SV having the largest CSN is determined, the CSN of the SV is taken as a point Forzen (intercept point), and then, the metadata log is subjected to an intercept process according to the point Forzen, such as deleting the metadata log before the point Forzen.

S604, completing metadata cleaning.

When the metadata log before the Forzen point is deleted, deleting the corresponding metadata, and completing the cleaning of the metadata.

According to the embodiment of the application, the metadata which is not used any more can be cleaned, so that more storage space is released, and the data storage pressure is reduced. It will be appreciated that when an SV is cleaned, the corresponding table is also cleaned.

FIG. 10 is an exemplary diagram of a DBMS architecture of a database according to an embodiment of the present application. The database in the present application may be SQL, noSQL, oracle, mySQL, openGauss types of databases, and will be described below using an SQL database as an example.

As shown in FIG. 10, the architecture includes an advanced command line tool, a Process Manager module, a client connection Manager (Client Communications Manager) module, a relationship query processing (Relational Query Processor) module, a transactional store Manager (Transactional Storage Manager) module, a shared component, and a utility (Shared Componentsand Utilities) module.

The high-level command line tools include a pt-online-schema-change (a tool developed by percona company), a pt-show-grants, and the like. The pt-online-schema-change is used to modify the table online without locking the table. The pt-show-grants is used to print out the relevant information of database authorization in a standardized way.

The client connection management module is used for processing connection of the client, maintaining connection state and relevant operation when data are transmitted between the client and the database server.

The process management module includes a Scheduling (DISPATCH AND Scheduling) module and an entry control (Admission Control) module. The calling module is used for managing and coordinating system resources so as to effectively execute and distribute tasks, processes or requests. The admission control module is used for managing and controlling new requests, tasks or resources to join the system.

The relational Query processing module comprises an SQL parsing and Authorization (Query PARSING AND Authorization) module, a Query Rewrite (Query Rewrite) module, a Query optimization (Query optimization) module, a plan execution (Plan Executor) module and a DDL execution (DDL and Utility Processing) module.

The SQL analysis and authorization module is used for analyzing the database query language, verifying the user authority and deciding whether to allow the query operation to be executed. The query rewrite module is used for optimizing and rewriting the query submitted by the user so as to improve the query performance, reduce the execution cost or realize other optimization targets. The query optimization module is used for analyzing and optimizing query sentences submitted by users to generate an optimal execution plan, so that the optimal query performance is realized. The plan execution module is used for actually executing database query operation according to the execution plan generated by the query optimizer and returning the query result to the user or the application program. The DDL execution module is used for processing structure definition, metadata management and various utility operations of the database, and is responsible for managing creation, modification and deletion of database objects and executing various tasks related to database structure and management.

The transactional memory management module comprises an Access Method (Access Method) module, a Buffer Manager (Buffer Manager) module, a Lock Manager (Lock Manager) module and a Log Manager (Log Manager) module. The access method module is used for managing and processing the storage and access modes of the data in the database. The buffer management module is used for storing metadata, logs and other data. The lock management module is used for managing and accessing the data of the database concurrently so as to ensure isolation and consistency among transactions. The log management module is used for recording database operations and changes to realize the durability, the recoverability and the consistency of data, and is responsible for recording the operation log of the transaction so as to carry out data recovery and rollback operations when the system fails or crashes.

The sharing components and tool modules include a Catalog management (Catalog Manager) module, a Memory Manager module, a management, monitoring and utility module (Monitoring & Utilities) module, and a batch tool (Batch Utilities) module.

The catalog management module is used for managing metadata information of the database, and comprises definitions and attributes of database objects, tables, columns, indexes, views, users, rights and the like. The memory management module is used for managing allocation, use and release of the system memory so as to support database operation and query execution. The management, monitoring and utility module is used to manage, monitor and perform various database management tasks and utility operations. The batch tool module is used for executing batch processing tasks and utility operations.

For the database shown in FIG. 10 described above, the tables may be subjected to online DDL operations using the advanced command line tool pt-online-schema-change in the database. During online DDL operation of a table using pt-online-schema-change, DML operation of the data table is allowed. The online DDL supports the definition, modification, and adjustment of schema at runtime without stopping database service or restarting. That is, structural changes may be made to tables, indexes, columns, etc. while the database is active. Database schema is a structural description of a database. Within a relational database, database schema defines tables, fields in tables, relationships between tables and fields.

The procedure by which the pt-online-scheme-change performs online DDL operations is described below.

FIG. 11 is a flowchart illustrating an example of a pt-online-scheme-change operation performed on-line DDL. As shown in fig. 11, the flow includes the steps of:

S701, receiving an online DDL instruction.

When a user wants to perform online DDL operations on a table in a database, online DDL instructions can be sent to the database.

S702, creating a copy table XXX_new.

The database, upon receiving the online DDL instruction, may invoke the copying of the structure of the table with table name "XXX" to the copy table with table name "xxx_new".

And S703, performing DDL operation on XXX_new.

After creating the replication table XXX_new, the database may call pt-online-schema-change to perform DDL operations on the replication table XXX_new, e.g., add fields, change field types, etc.

S704, creating Update, insert and Delete triggers.

The database may invoke a pt-online-schema-change to create an Update trigger, insert trigger, delete trigger in the original table.

S705, starting data replication and log synchronization.

After the Update trigger, the Insert trigger and the Delete trigger are created in the original table, the data in the original table can be copied into the copy table through the Update trigger, the Insert trigger and the Delete trigger.

In order to ensure the security of data, a database is generally deployed in a master-slave database mode so as to achieve the purpose of data backup. In order to ensure data synchronization of the master database and the slave database, when the database calls pt-online-schema-change to perform DDL operation on the tables, the logs of the master database (master) can be configured, and then the logs of the master database are synchronized into the slave database (slave) to copy the changes of the tables into the slave database, so that the slave database can adjust the tables in the slave database according to the synchronized logs to ensure consistency of the master database and the slave database.

S706, completing data replication and log synchronization.

After all the data in the original table is copied to the copy table, the data copying is completed. After synchronizing the log of the primary database server to the secondary database server, the log synchronization is completed.

S707 replaces XXX with xxx_new.

After completing the data replication and log synchronization, the database may call pt-online-schema-change to replace the replication table with the original table.

S708, deleting XXX.

The database calls pt-online-schema-change to delete the original table.

S709, completing online DDL.

After the database call pt-online-schema-change completes the DDL operation on the replication table, the online DDL is completed.

Because the pt-online-schema-change creates a copy table (secondary table) for the primary table (primary table), and an Update trigger, an Insert trigger and a Delete trigger are added in the primary table, the triggers can synchronize the data change of the table, and when the DDL operation is performed on the table, the DML operation can be performed on the table, and concurrent execution of the DDL operation and the DML operation is supported, so that blocking can be avoided. Creating a duplicate table, however, doubles the storage space and increases the pressure on data storage. And if the data of the table is large, the data synchronization time of the primary table and the secondary table is long. Therefore, the pt-online-scheme-change scheme is difficult to apply to a scene with increased data volume, and has the defects of larger memory space consumption and longer data synchronization time.

In some embodiments, an Instant algorithm may also be used to perform online DDL operations, where the algorithm only updates metadata, does not reconstruct the data immediately, and the DDL instruction is performed very fast, and does not block DML operations and DQL operations. But the Instant algorithm only supports a small number of DDL type operations such as adding columns, and will not support the Instant algorithm if the reserved columns are not sufficient. Thus, the Instant algorithm has certain limitations.

On the basis of the database system shown in fig. 10, as shown in fig. 12, the technical scheme of the application improves a client connection management module, an SQL analysis and authorization module, a catalog management module, a DDL execution module and a log management module in the database system.

For example, the client connection management module adds the capability of version connection management based on the existing capability. The SQL analysis and authorization module increases the SV management command line analysis capability based on the existing functions. The catalog management module adds multi-version metadata management capabilities, such as multi-version metadata caching capability, version metadata query capability and version metadata updating capability, based on the existing capabilities. The log management module adds metadata history version management capability and garbage collection based on existing capability.

The DDL operation process of the present application is described below in connection with an exemplary diagram of a system architecture for the data shown in FIG. 12.

FIG. 13 is a flowchart illustrating a DDL operation process according to an embodiment of the present application.

As in fig. 13, ddl operations include the steps of:

S801, analyzing DDL instructions by the SQL analyzing and authorizing module.

When a user wants to modify the structure of table a in the database of the database, a corresponding DDL instruction can be sent to the database through the terminal device. After receiving the DDL instruction, the database can call an SQL analysis and authorization module in the DBMS system to analyze the DDL instruction, so as to obtain a corresponding analysis result.

S802, the DDL executing module executes the metadata updating operation.

After the SQL analysis and authorization module analyzes the DDL instruction, the DDL execution module executes the DDL operation to update the metadata.

And S803, the buffer management module updates the metadata.

After the DDL execution module performs the metadata update operation, the buffer management module stores the updated metadata into the buffer.

S804, the log management module writes the log.

After the buffer management module stores the updated metadata in the buffer, the log management module can generate a corresponding log according to the update process, update time and other information of the metadata, and write the log into the database. The log may be used to determine the update process and update time of the metadata.

S805 the buffer management module creates and stores a metadata snapshot.

The buffer management module may create and store a snapshot of the metadata of the table.

S806, the catalog management module updates the metadata cache.

The catalog management module stores the updated metadata in the buffer.

S807, the process management module starts an asynchronous thread to complete data reconstruction.

After the database updates the metadata of the table, the structure of the table is changed, and at the moment, the database can call the process management module to start a starting thread (namely a data reconstruction thread) to reconstruct the data in the table, so that the data of the table is matched with the updated structure of the table.

In the embodiment of the application, the DDL execution module does not lock the table when executing the DDL operation, so that the table can be subjected to the DML operation. In this case, when the database receives the DML operation instruction, the process management module may call the data reconstruction thread to reconstruct the data to be DML operated so that the data to be DML operated satisfies the structural change of the table, and then perform the DML operation on the data to be DML operated. Thus, neither data inconsistency nor congestion occurs.

It can be understood that the data processing method of the present application is to add the data reconstruction operation before the DML operation, solve the blocking problem, and not create a copy table, and compared with the pt-online-schema-change scheme, the present application does not add additional storage space, and can be suitable for the scene with larger data volume. And compared with the limitation of an Instant algorithm, the data processing method disclosed by the application can be suitable for various DDL operations, is not limited by a reserved space, and has better applicability.

The application provides a metadata multi-version management scheme which is completely compatible with the use mode of a traditional database. Meanwhile, in order to enable the database to have the capability of providing multi-version user data services based on the same data, the application provides a complete set of interfaces for users to manage metadata versions, and the users can decide whether to use the interfaces according to own requirements.

The interface provided by the present application is described below.

(1) Creation of an SV

After a database administrator (database administrator, DBA) invokes a database to perform a particular DDL operation, the database generates a corresponding metadata snapshot, which is stored in a metadata snapshot system table (pg_category_snapshot) as an internal SV. After the database finishes DDL operation each time, the SV creation interface provided by the application can be called to create a corresponding SV.

The interface to create an SV will be described using SQL as an example.

The SQL syntax is shown in Table 1:

TABLE 1

The interface parameters are shown in table 2:

TABLE 2

As shown in table 2, in this interface, parameter SCHEMAVERSION is an SV name, which can be expressed as version_name. Parameters SCHEMAVERSION are unique within the same CDB or ADB and do not allow repetition. The data type can be varchar type, and the data length is 1-64 bytes. The parameter commants is an optional parameter for storing version description information and distinguishing SVs. The data type is varchar type, and the data length is 0-256 bytes.

SQL examples are shown in Table 3:

TABLE 3 Table 3

Illustratively, as shown in FIG. 4B, the version name of the initial SV is V0.0, corresponding to the initial Table A of the V0.0 version. After the database performs DDL operation on the V0.0 version of the initial table a, for example, column c2 is added to the V0.0 version of the table a, the SV with version name V1.1 created by the SV creation interface may be called, and the SV is bound to the V1.1 version of APP, which is equivalent to creating a link of the V1.1 version of APP on the V with version name V1.1, so that the V1.1 version of APP may use the V1.1 version of table a. Similarly, after the data server performs DDL operations on version V1.1 of table a, such as adding column c3, the SV creation interface may be invoked to create an SV with version name V1.2 and bind the SV to version V1.2 of APP, so that version V1.2 of APP may use version V1.2 of table a. After the data server performs DDL operations on the V0.0 version of the table, such as adding column c2, the SV creation interface may be invoked to create an SV with version name V1.3 and bind the SV to the V1.3 version of APP, so that the V1.3 version of APP may use the V1.3 version of table a.

(2) Querying SV

When the database receives the SV query request, the SV query interface provided by the application can be called to query the information of the appointed SV.

The SV query interface is described using SQL as an example.

The SQL syntax is shown in Table 4:

TABLE 4 Table 4

The returned results are shown in Table 5:

TABLE 5

schema	version_name	create_time	comments
				schema1	V1.1	2021-09-24 09:15:42.099642+08	For App V1.1
schema1	V1.2	2021-09-25 09:15:42.099642+08	For App V1.2

The return result parameter specification is shown in table 6:

TABLE 6

As shown in table 6, in this interface, the parameter schema (database schema) is the current schema name, such as schema1 in table 5. The parameter version_name is the SV name, such as V1.1 and V1.2 in Table 5. The parameter create_time is the SV corresponding timestamp, such as 2021-09-24 09:15:42.099642+08 and 2021-09-25 09:15:42.099642+08 in Table 5. The parameter create_time is automatically generated by the DBMS system and can be displayed in Unix timestamp format. The parameter comments are version description information For distinguishing SVs, such as For App V1.1 and For App V1.2 in table 5.

Illustratively, as shown in fig. 4B, when the database receives the SV query request, the SV query interface may be invoked to query the SV information (i.e., the SV-bound APP information) such as the schema name, the SV name, the timestamp corresponding to the SV, and the version description information as shown in table 5 above.

(3) Deleting a designated SV

When the database receives the SV deleting instruction, the SV deleting interface provided by the application can be called to delete the designated SV so as to delete the table corresponding to the SV.

The SV delete interface is described using SQL as an example.

The SQL syntax is shown in Table 7:

TABLE 7

The interface parameters are shown in table 8:

TABLE 8

As shown in Table 8, in this interface, parameter SCHEMAVERSION represents the SV name to be deleted, the data type is the varchar type, and the data length is 1-64 bytes.

Illustratively, as shown in fig. 4B, when the database receives a delete instruction, the SV delete interface may be invoked to delete the specified SV, such as SV with version name V0.0, to delete table a of V0.0 version.

(4) Querying definition of tables in SV

When the database receives a table definition query instruction, the table definition query interface provided by the application can be called to query the definition of the designated table.

The table definition query interface is described using SQL as an example.

The SQL syntax is shown in Table 9:

TABLE 9

The interface parameters are shown in table 10:

Table 10

As shown in TABLE 10, the parameter TABLE represents the TABLE name to be queried. Parameter SCHEMAVERSION indicates the SV name corresponding to the name of the table to be queried, the data type is the varchar type, and the data length is 1-64 bytes.

The returned results are shown in Table 11:

TABLE 11

column	type	modifiers	Storage	Description
					id	integer	PRIMARY KEY not null	plain
name	character varying(50)	extended	extended
					gender	character varying(20)		extended

The return result parameter specification is shown in table 12:

table 12

Parameters (parameters)	Description of the invention
		column	Table field name
type	Field type
		modifiers	Field modifier
Storage	Field storage mode
		Description	Field description information

As shown in Table 12, the parameter column indicates a field name, such as id, name, gene in Table 11. The parameter type indicates a field type, such as integer (integer) in table 11, CHARACTER VARYING (50), CHARACTER VARYING (20). Wherein CHARACTERVARYING (50) represents a character string that can store a maximum of 50 characters in length, and CHARACTER VARYING (20) represents a character string that can store a maximum of 20 characters in length. Parameters modifiers represent field modifiers, such as PRIMARY KEY non null, extended in table 11. Wherein PRIMARY KEY non null indicates a primary key non-null constraint. extended indicates that off-line storage is allowed, but compression is allowed. The parameter Storage indicates a field Storage manner, such as play and extended in table 11. Wherein play means avoiding compression and off-line storage. The Description represents field Description information.

Illustratively, as shown in fig. 4B, when the database receives a V1.3 version of APP send table definition query instruction, the table definition query interface may be invoked to query for table definition information corresponding to SV with version name V1.3. For example, the query results shown in table 11 may be queried.

(5) Querying all metadata information in a given SV

When the database receives the metadata information inquiry instruction, the metadata information inquiry interface provided by the application can be called to inquire all metadata information in the appointed SV.

The metadata information query interface is described using SQL as an example.

The SQL syntax is shown in Table 13:

TABLE 13

The interface parameters are shown in table 14:

TABLE 14

Parameters (parameters)	Description of the invention
		SCHEMA	Schema name to be queried
SCHEMAVERSION	SV name to be queried

As shown in table 14, in this interface, the parameter SCHEMA represents the SCHEMA name to be queried. Parameters (parameters)

SCHEMAVERSION denotes the name of the data version to be queried.

The returned results are shown in, for example, table 15List of table, table 16List of index, table 17List of view, table 18List of functions, table 19List of sequence:

table 15List of tables

Name	Type	Owner	Storage
				table1	table	gauss	{orientation=row,compression=no}
table2	table	gauss	{orientation=row,compression=no}
				table3	table	gauss	{orientation=row,compression=no}

In table 15, parameter Name represents a table Name, for example, table1, table2, table3. The parameter Type indicates a data Type, and when the Type is table, the data Type is table. The parameter Owner represents the data source, and when the Owner is gauss (gaussian database), the representation table is derived from gauss. The parameter Storage indicates a Storage manner, for example, when Storage is { organization=row, and expression=no } indicates that a Storage manner of a table is a line Storage manner and compression is not performed.

Table 16List of index (index Table)

Name	Type	Owner	Table	Storage
					index1	index	gauss	t1
index2	index	gauss	t2
					index3	index	gauss	t3

In table 16, the parameter Name indicates index names, such as index1, index2, index3. The parameter Type indicates a data Type, and when the Type is index, indicates that the data Type is index. The parameter Owner represents the data source, and when the Owner is gauss, the representation table is derived from gauss. The parameter Table represents a Table name, for example, t1, t2, t3. The parameter Storage represents the Storage mode.

Table 17List of view (View Table)

Name	Type	Owner	Storage
				view1	view	gauss
view2	view	gauss
				view3	view	gauss

In table 17, a parameter Name indicates view names, such as view1, view2, view3. The parameter Type indicates a data Type, and when the Type is view, the data Type is view. The parameter Owner represents the data source, and when the Owner is gauss, the representation table is derived from gauss. The parameter Storage represents the Storage mode.

Watch 18List of functions (functional watch)

Name	Result data type	Argument data types	Type	Fencedmode
					fun1	text	id character varying	normal	f
fun2	integer		normal	f
					fun 3	text		normal	f

In table 18, the parameter Name indicates the function Name to be queried, that is, the function Name, for example, fun1, fun2, fun3. The parameter Result data Type indicates a data Result Type, indicates that the data Type is text when the Type is text, and indicates that the data Type is an integer when the Type is integer. Parameters Argument DATA TYPES represent parameter data types, such as ID CHARACTER VARYING represent ID character changes. The parameter Type indicates a function mode, and when the Type is normal, indicates a normal function mode. Parameter Fencedmode is fence mode, when Fencedmode is f, indicating that the corresponding function is not on.

Table 19List of sequence (sequence Listing)

Name	Type	Owner	Storage
				seq1	sequence	gauss
seq2	sequence	gauss
				seq3	sequence	gauss

In table 19, the parameter Name indicates the sequence Name, e.g., seq1, seq2, seq3. The parameter Type indicates a data Type, and when the Type is sequence, indicates that the data Type is sequence. The parameter Owner represents the data source, and when the Owner is gauss, the representation table is derived from gauss. The parameter Storage represents the Storage mode.

For example, as shown in fig. 4B, when the database receives the V1.3 version APP send metadata information query instruction, the metadata information query interface may be invoked to query all metadata object information, such as tables, indexes, views, stored procedures (function parameters), sequences, etc., under SV with version name V1.3.

(6) Querying user links established on a specified SV

When the database receives a user link inquiry instruction, the user link inquiry interface provided by the application can be called to inquire the user link information established on the appointed SV.

The user link query interface is described using SQL as an example.

The DBA needs to manage all user links, acquire all user links established on a specific SV by calling this interface, and then manage the user links.

The query user links show the results of the system view pg_stat_activity query, each record representing a worker thread. The system view pg_stat_activity displays some information about the active thread of the current session, such as the state and queries of the current session, etc.

An example of viewing user link information through the system view pg_stat_activity is shown in table 20:

Table 20

In table 20, parameters datid represent data IDs, such as 15140. Parameters datname represent data names such as postgres. The parameter schema represents a database schema name, such as scehma a1. Parameter shemaversion represents an SV name, such as V1.1.pid indicates a process controller, such as 14035566254125,14035566254126. The parameter Owner represents the data source, and when the Owner is gauss, the representation table is derived from gauss. The parameter backend _start represents a time stamp of the process start time, i.e., the time at which the user link was established at the specified SV, e.g., 2021-09-24 09:15:42.099642+08,2021-09-25 09:15:42.099642+08.

The SQL syntax is shown in Table 21:

Table 21

When the APP requests to establish the user link, the request contains schema information, and corresponding SV information can also be specified. The client and the server can store corresponding link information. The query linked list may modify the definition of the system view pg_stat_activity on the existing implementation of openGauss, and add the schema field and the schema_version field to the definition.

The interface parameters are shown in table 22:

Table 22

Parameters (parameters)	Description of the invention
		SCHEMA	Database schema name corresponding to user link
SCHEMAVERSION	SV name corresponding to user link

And (3) displaying the query result of the current system view pg_stat_activity.

As shown in table 22, in this interface, the parameter SCHEMA represents the database SCHEMA name to which the user link corresponds. Parameters SCHEMAVERSION represent the SV name to which the user link corresponds.

Illustratively, as shown in FIG. 4B, when the server receives a user link query request sent by version V1.1 of APP, the user link query interface may be invoked to query for user link information on SV with version name V1.1, such as the information shown in Table 20.

(7) Switching SV

When the SV switching instruction is received by the database, the SV switching interface provided by the application can be called to switch the SV bound by the APP.

The SV switch interface will be described using SQL as an example.

The SQL syntax is shown in Table 23:

Table 23

The interface parameters are shown in table 24:

Table 24

Parameters (parameters)	Description of the invention
		SCHEMAVERSION	Target SV name

In this interface, as shown in table 24, parameters SCHEMAVERSION represent the target SV name.

The returned results are shown in Table 25:

Table 25

Returning a result scenario	Returning information
		Execution success	SUCCESS
Execution failure	Failure cause

As shown in table 25, when the return result is that the execution is successful, that is, the SV handover fails, the return information is "SUCCESS". When the return result is that the execution fails, namely the SV switching is successful, the return information comprises the failure reason.

When the APP is connected with the database, if the APP is not already bound with the SV, the database can call the SV switching interface to bind the APP with the appointed SV, so that the APP can use a table corresponding to the appointed SV. For example, as shown in fig. 4B, when version V1.2 APP is first connected to the database, the database may call the SV switch interface to bind version V1.2 APP to SV with version name V1.2, so that version V1.2 APP may use version V1.2 table a. At this time, when the database receives the SV switching command sent by the V1.2 version of APP, the SV switching interface may be invoked to switch the SV bound by the APP, for example, the APP is unbound with the SV with the version name V1.2, and then the APP is bound with the SV with the version name V1.3, so that the APP may use the table a with the version V1.3. Therefore, the database can switch the SV bound by the APP by calling the SV switching interface, so that the flexibility and the continuity of the service are improved.

Management of metadata is described below.

(1) Metadata version SV

After the metadata is updated, a new metadata snapshot is automatically formed, which is consistent with the version concept in multi-version concurrency control (multi-versionconcurrency control, MVCC). The application marks the version by using the metadata snapshot, and has the main advantage that the metadata query based on the snapshot can be quickly realized by means of the existing MVCC algorithm in the in-situ update storage engine (Ustore) without repeated development. In the running process of the system, after DDL operation is executed each time, corresponding metadata snapshot can be recorded and stored in a system table. In the actual processing, since the metadata acted by part of DDL instructions does not need to be subject to multi-version management, such as index, constraint and other related DDL operations, not all DDL operations record corresponding metadata snapshots.

Ustore storage engine, also called In-place Update storage engine (In-place Update), is a storage mode that the openGauss kernel adds. Ustore the storage engine stores the latest version of "valid data" separately from the historical version of "junk data". The latest version of effective data is stored on the data page, and a section of Undo space is opened up independently for unified management of historical version of garbage data, so that the data space cannot be expanded due to frequent updating, and the centralized recovery efficiency of the garbage data is higher.

(2) User linked version management

In the application, the user request analysis and the request execution all need the support of the SV, and the SV corresponding to the user request is determined by the user link. Thus, after using the metadata multi-version property, when the user link is established, the connection manager needs to be responsible for managing the correspondence of the user link to the SV.

If the user designates the SV when initiating the database connection request through the APP, the APP is bound with the designated SV, so that the APP can use the designated SV corresponding table. During the connection duration, if the SV is not switched by calling the switching SV interface, the SV corresponding to the binding is kept unchanged. If the SV is not specified when the database connection request is initiated, the user request corresponding to the database connection is processed according to the latest metadata in real time, and can be directly obtained by inquiring the system table. Therefore, the user service version management capability can be provided while the operation habit of the traditional database is completely compatible, and support is provided for smooth evolution of the user service.

It can be appreciated that DDL operations are always performed based on the most up-to-date metadata, and target metadata can be obtained directly by querying the system table without considering version issues.

(3) Historical metadata management

Different databases, different storage engines may manage the historical data in different ways. In the additional update storage engine (Astore), historical version data and the latest data are mixed and stored in the Page, the Page in the table is scanned by a special garbage collector, the version of the tuple in the Page is identified, and the history version tuple which is not used any more is cleaned so as to release the storage space. Ustore in the storage engine, only the latest version of data is stored in the Page, and the historical version of data is stored in a centralized way by an Undo Log (Undo Log), which is a mechanism for recording the historical version information of the transaction operation. When garbage (metadata) is cleaned, a Frozen point can be determined, and then the logs are truncated according to the Frozen point so as to keep the corresponding logs (such as the latest logs) and complete the corresponding garbage cleaning.

In the application, the historical version of the metadata can be used for a long time, and for Astore, the scanning of the system table is skipped in the working process of the garbage collector. Historical version data clearing is triggered by SV delete operations. In Ustore, since the historical version of the metadata and the historical version of the user data are stored in the revocation Block (Undo Block) (a basic unit for storing the transaction historical version data) in a mixed manner, the truncation algorithm adopted by the existing log cleaning algorithm is to keep the historical version of the metadata, which causes that a large amount of Undo logs (a logic unit for storing the transaction historical version data) cannot be cleaned for a long time, and occupies a large amount of storage space. It is therefore necessary for Ustore to store the corresponding Undo Record of the system table separately, the cleaning of which is also triggered by the SV delete operation.

(4) Metadata query

When the metadata multi-version characteristic is not enabled, the latest metadata corresponding to the database processing request can be obtained by directly inquiring the system table, and the method is realized by adopting the MVCC technology in specific implementation. When the metadata multi-version characteristic is used, a user is connected to a database through a designated SV, a DML instruction and a DQL instruction which are sent to the database through the connection are processed according to metadata corresponding to the SV, and corresponding metadata acquisition is achieved through snapshot query of the metadata database designated in the SV.

User data (data stored in the table) management is described below.

In the conventional database, since the metadata multi-version is not supported, when user data is reconstructed, user data reconstruction can only be performed in units of tables. In the process of reconstructing the version of the user data, the user request cannot accurately sense the version state of the user data, and the consistency of the data can be ensured only by locking or master copy switching. Locking user data can ensure that the data is consistent during evolution, but can block other user requests in the database system. The primary-copy switching mode does not block the user request but consumes a large amount of storage space, and for updating intensive applications, the primary-copy synchronization is difficult to realize and long in time consumption.

The application completes asynchronous reconstruction of the user data by means of user data version identification and data version conversion. The proposal can ensure that DDL operation and DML operation are not blocked mutually, realize online DDL and simultaneously ensure that the database has the capability of providing database services for the requests of different versions of metadata based on the same user data.

In the traditional database, the user data storage mainly solves the problems of data storage format, free space management, storage space recovery and the like. The application mainly describes the problem of user data version introduced by metadata multi-version, and mainly comprises user data version identification, user data version conversion and user data reconstruction.

(1) User data version identification

In the application, in order to realize online DDL, the whole table is not locked in the process of user data reconstruction, and simultaneously, the user data version update and the metadata update are asynchronously carried out, so that when a user operates based on different SVs, the coexistence of a plurality of user data of different versions in one table appears in a database. So to properly parse the user data, a scheme for accurately identifying the version of the user data must be provided.

In consideration of the fact that the transaction information related to the user data is managed in the storage engine, the time sequence of data generation can be confirmed through the transaction information, and then the data version can be identified through the transaction information, so that adjustment of the storage structure of the user data is not needed. Taking Ustore as an example, when identifying the user data version, transaction information corresponding to the tuple can be acquired first, and then the SV corresponding to the tuple can be judged according to the transaction information.

(2) User data version conversion

When performing DQL operation or DML operation on user data, when the user data version is inconsistent with the target data version, version conversion of the user data is required. User data version conversion is divided into two types, version evolution and version rollback. Version evolution occurs when the data version is lower than the user requested version, and conversely, version rollback is required when the data version is higher than the requested version.

User data in the database is always evolving from a low version to a high version. Therefore, reorgTask (reorganization task) corresponding to the table can be managed in a linked list form, and the specific content of the version conversion task can be quickly determined through tblOid (table object identifier), srcVersion (source version of data) and destVersion (target version of data) when the version conversion is carried out later. In particular implementation, reorgTask may be stored in a linked list form in a Table version corresponding to a relationship cache (RelCache) to implement global sharing. When the reconstruction task is inquired, when the corresponding task does not exist in the cache, the reconstruction task creation interface is called to create and cache the corresponding task.

In some embodiments, upon performing UHeapTupleData (a data representing UHeap tuples) version conversion processing, toDeleteColList (list of columns to be deleted), toAddColList (list of columns to be added) may be generated from metadata differences of different versions. The current IsNull field of the original tuple is then obtained and a determination IsNull is made as to whether the field is in toDeleteColList. If so, then the IsNull field is deleted, and the processing of that field is skipped. If not, the corresponding bit in the original IsNull is copied to the new IsNull. If the current field is not empty, the original field value is copied.

In a database (e.g., a cloud-native database), newly added fields can only be stored after existing fields. For the newly added field, the corresponding bitmap value may not be stored, and the newly added field value is a default value, and no padding is required (except for the case that the user explicitly assigns a default value through Insert, update). The fields in toAddColList are therefore not processed at reconstruction.

For user data version rollback, user data version rollback can be realized by only exchanging toDeleteColList with the content in toAddColList.

(3) Transaction execution (DML/DQL)

When the transaction is executed, the request version and the user data version need to be confirmed first, and when the two versions are inconsistent, version conversion is needed, and then the transaction is processed according to the conventional database transaction execution flow.

Since DML operations change the transaction information corresponding to the data, the transaction information change may cause the version information corresponding to the data to change. Therefore, a strong constraint must be added to the management of user data, that is, when DML operations are performed to update data, the updated version of user data must be the latest SV in the current database to ensure that the user data corresponds to the true version. That is, when updating user data, the user data must be subjected to version evolution no matter whether the SV corresponding to the request is the latest version or not, and the user data must be stored in the latest SV-corresponding structure, so that it is ensured that the SV corresponding to the data can be identified by the transaction information corresponding to the data. Because the inquiry does not change the transaction information of the user data, version evolution of the user data is not needed in the inquiry process.

User data reconstruction is described below.

In the application, when the DDL updates the user data structure, the version evolution of the user data is needed. In the specific implementation process, the method is embodied as user data reconstruction. Taking the adding and deleting data column as an example, the adding and deleting operation is performed on the field corresponding to the target column in the user data in the reconstruction process. And starting a data reconstruction thread, and completing a corresponding reconstruction task by the thread.

The data reconstruction thread acquires pages in the user table one by one in a full table scanning (SeqScan) mode, and reconstructs tuples in the pages one by one. In order to record the reconstruction progress, a field needs to be added in a header (PAGEHEADER) for recording the CSN number corresponding to the SV when the previous reconstruction of the current page is completed, so that after the reconstruction operation is failed and recovered, the starting page after the reconstruction is recovered can be quickly determined. When the data in the table is reconstructed, the designated page can be obtained directly through a Buffer interface provided by a Buffer Pool, and then the user data in the page is reconstructed and the corresponding index is updated as necessary.

User data reconstruction in a database is a process of converting and storing user data from a low version to a high version, and can be equivalent to modification operations on data in a page.

Because the database cleans up the deleted columns (columns) in the original tuple when reconstructing the tuple, the newly added columns do not occupy extra storage space. The entire reconstruction will only be free of storage space and will not take up additional space.

When the tuple in a certain page of the table is reconstructed, the CSN corresponding to the table can be acquired, and the data reconstruction task can be acquired from the cache according to the CSN. If the data reconstruction task does not exist, the data reconstruction task is constructed through inquiring a system table, and the data reconstruction task is cached. And then, carrying out version conversion on the tuple in the page according to the data reconstruction task, storing the reconstructed tuple next to the last tuple in the page, and updating the corresponding RowPtr (line pointer), wherein the corresponding index does not need to be updated. After reconstruction of all tuples in the page is completed, the reconstruction field in the header is updated, and the corresponding free space size in the free space table (FSM, FREE SPACE MAP) is updated.

Ustore in the storage engine, the deleted tuple is marked as deleted only, and the real clean-up of the corresponding tuple must wait for all transactions whose CSN is smaller than that of the deleted tuple to be executed to completion. The specific reason is to ensure that long transactions can still acquire the tuple through the old snapshot. Thus, when reconstructing data within a page, there may be user data within the page that is deleted but not cleaned. For the part of user data, the user data can be directly skipped during reconstruction, and the original tuple is reserved. The specific reason is that the record value in the page can not be used any more, and when the historical value of the tuple is queried by the old transaction, the old transaction can be traced back through the version chain of the time sequence database (TIME SERIES database slot, TD slot).

Fig. 14 shows a schematic structural diagram of the electronic device 100. The electronic device 100 may be a database server as referred to in the present application. As shown in fig. 14, the electronic device 100 includes one or more processors 1032, a communication interface 1033, a memory 1031, a bus system 1034, and one or more programs. Wherein the communication interface 1033, the processor 1032, and the memory 1031 are interconnected by a bus system 1034, which may be a peripheral component interconnect standard bus or an extended industry standard architecture bus, among others. The buses may be classified as address buses, data buses, control buses, etc. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus. Wherein the one or more programs are stored in the memory 1031, the one or more programs include instructions, which when executed by the electronic device 100, cause the electronic device 100 to perform the relevant methods of any of figures 3, 5, 6, 7, 8, 9, 11, 13.

The present application also provides a computer storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by the electronic device 100, cause the electronic device 100 to perform the relevant method of any of figures 3, 5, 6, 7, 8, 9, 11, 13.

The present application also provides a computer program product comprising instructions which, when run on an electronic device 100, cause the electronic device 100 to perform the relevant method of any of the figures 3, 5, 6, 7, 8, 9, 11, 13.

The electronic device, the computer storage medium or the computer program product provided by the present application are used to execute the corresponding method provided above, and therefore, the advantages achieved by the present application may refer to the advantages in the corresponding method provided above, and will not be described herein.

It should be understood that, in various embodiments of the present application, the sequence numbers of the foregoing processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and internal logic thereof, and should not constitute any limitation on the implementation process of the embodiments of the present application.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.

In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interface, indirect coupling or communication connection of devices or units, electrical, mechanical, or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using a software program, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in accordance with embodiments of the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, database server, or data center to another website, computer, database server, or data center by a wired (e.g., coaxial cable, fiber optic, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device including one or more database servers, data centers, etc. that can be integrated with the medium. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium, or a semiconductor medium (e.g., solid state disk (Solid STATE DISK, SSD)), etc.

The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A data processing method, applied to an electronic device, characterized in that the method comprises:

During a first change operation on a first table, a first instruction is received, wherein the first instruction is used to instruct to modify first data in the first table to second data;

determining that the first change operation has not yet adjusted the format of the first data in the first table to: meet the format requirement of the first table after the first change operation;

adjusting the format of the first data in the first table to meet the format requirement of the first table after the first change operation;

The first data in the first table is modified into the second data.

2. The method according to claim 1, wherein determining that the first change operation has not yet adjusted the format of the first data in the first table to meet the format requirements of the first table after the first change operation comprises:

Corresponding to determining that the format of the first data does not match the format parameters of the first table after the first change operation, it is determined that the first change operation has not adjusted the format of the first data to meet the format requirement of the first table after the first change operation.

3. The method according to claim 2, wherein the step of adjusting the format of the first data in the first table to meet the format requirements of the first table after the first change operation comprises:

The data reconstruction thread is called to adjust the format of the first data so that the format of the first data matches the format parameters of the first table after the first change operation.

4. The method according to claim 3, characterized in that the calling of the data reconstruction thread to adjust the format of the first data comprises:

calling the data reconstruction thread to parse a second instruction indicating the first change operation to determine a format parameter of the first table after the first change operation;

The data reconstruction thread is called to adjust the format of the first data based on the format parameters of the first table after the first change operation.

5 . The method according to claim 4 , wherein the first instruction comprises a data manipulation language DML instruction, and the second instruction comprises a data definition language DDL instruction.

6. The method according to claim 1, characterized in that the method further comprises:

Performing the first change operation on the first table to obtain a second table, and saving the second table;

receiving a first request sent by a first application, wherein the first request is used to request binding with the second table;

Bind the first application to the second table.

7. The method according to claim 6, characterized in that the method further comprises:

The second application is bound to the first table;

receiving a third request sent by the second application, wherein the third request is used to instruct to switch the second application to be bound to the second table;

Unbinding the second application from the first table;

Bind the second application to the second table.

8. The method according to claim 7, characterized in that the method further comprises:

Corresponding to determining that the formats of all data in the first table meet the format requirement of the first table and the first table is not bound to an application, the first table is deleted.

9. An electronic device comprising:

a memory for storing instructions executed by one or more processors of the electronic device;

A processor, when the processor executes the instructions in the memory, can enable the electronic device to execute the data processing method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that instructions are stored on the computer-readable storage medium, and when the instructions are executed on a computer, the computer executes the data processing method according to any one of claims 1 to 8.