CN104504012A - Database data migration method based on inverse topology - Google Patents

Abstract

The invention discloses a database data migration method based on inverse topology, which comprises the following steps: according to the database to be migrated, obtaining its topological relational diagram; according to the topological relational diagram and a mode conversion model, obtaining the expansion order sequence in the migration process; Migrate data and schemas to NoSQL according to the sequence of extended sequences obtained. The present invention performs mode transfer at the same time as data transfer, so that the data of the dependent data table and the retrograde one are correctly integrated into the migrated NoSQL database, thereby ensuring any join in the pre-migration SQL database The query can find the correct result in a single table in the migrated NoSQL database, and no longer needs to access multiple tables, which effectively improves the query efficiency of the migrated NoSQL database. As a database data migration method based on inverse topology, the invention can be widely applied in the computer field.

Description

A Database Data Migration Method Based on Inverse Topology

technical field

The invention relates to the field of computer technology, in particular to a database data migration method based on inverse topology.

Background technique

At present, the existing related research work mainly includes the following three aspects: denormalization, data transfer and schema conversion. The denormalization work mainly refers to the design of the database by means of appropriate redundant storage to improve query efficiency and other methods to meet the normative constraints of traditional relational databases. The data transfer work mainly refers to the migration from SQL database to NoSQL database is only the relocation of data in the database, for example, the data of a certain table in the SQL database is simply dumped to a certain Collection in MongoDB data or a certain data in other NoSQL databases. Tables have not undergone schema conversion, and the dependencies of tables in the SQL database have not been migrated. Schema conversion mainly refers to not only dumping the data in the SQL database to the NoSQL database, but also migrating the corresponding schema.

In the data migration method from the traditional relational database to the NoSQL database, a lot of current research work is simply migrating the content of the table (data transfer) without schema conversion, that is, the data between the SQL database tables before the migration is lost during the migration process. relationship. Then the migrated NoSQL database cannot support the results of the join query statement of the original SQL database in a single table, and needs to be queried multiple times separately, resulting in low query efficiency.

Contents of the invention

In order to solve the above technical problems, the object of the present invention is to provide a database data migration method based on inverse topology that can perform schema conversion and improve query efficiency after migration.

The technical scheme adopted in the present invention is:

A database data migration method based on inverse topology, comprising the following steps:

A. According to the database to be migrated, obtain its topology diagram;

B. According to the topological relationship diagram and the mode conversion model, the expansion order sequence in the migration process is obtained;

C. Migrate data and schemas to NoSQL according to the obtained extended order sequence.

As a further improvement of the database data migration method based on reverse topology, the step B includes:

B1. Randomly select a node from the leaf node set in the topology diagram as the initial node;

B2. Process all the edges with the initial node as the arc tail node, and obtain the integration sequence of the corresponding edges;

B3. Move the initial node out of the leaf node set and put it into the isolated node set;

B4. Add the integrated sequence obtained in the processing process to the extended sequence sequence;

B5. Go back to step B1 until the set of leaf nodes is an empty set.

As a further improvement of the database data migration method based on reverse topology, the step B2 includes:

B21. Randomly take any edge with the initial node as the arc tail node, and extend and integrate the arc head node of the edge to obtain the integration order of the edge;

B22. Determine whether the arc head node is a leaf node without including the edge, if so, put the starting point into the leaf node set; otherwise, perform step B23;

B23. Return to step B21 until all edges whose initial node is an arc tail node are processed.

As a further improvement of the database data migration method based on reverse topology, the step C includes:

C1. Migrate the data of the nodes that only appear at the end of the arc to the corresponding data tables in the NoSQL database;

C2. Migrate the data of the remaining nodes to the corresponding data tables in the NoSQL database in sequence according to the extended order sequence according to the converted schema.

As a further improvement of the database data migration method based on reverse topology, the step C2 includes:

C21. Taking out the integration order in the extended order sequence sequentially;

C22. Integrating the information of the arc tail node in the extracted integration sequence into the arc head node and expanding the arc head node;

C23. Migrate the data of the expanded arc head node to the corresponding data table in the NoSQL database.

The beneficial effects of the present invention are:

A database data migration method based on inverse topology in the present invention performs schema transfer while data transfer is performed, so that the data of dependent data tables and the reverse one are correctly integrated into the migrated NoSQL database, thereby ensuring Any join query in the pre-migration SQL database can find the correct result in a single table in the migrated NoSQL database, eliminating the need to access multiple tables, effectively improving the query efficiency of the migrated NoSQL database .

Description of drawings

The specific embodiment of the present invention will be further described below in conjunction with accompanying drawing:

Fig. 1 is a flow chart of steps of a database data migration method based on inverse topology in the present invention;

FIG. 2 is a schematic diagram of mode conversion in Embodiment 1 of a database data migration method based on inverse topology in the present invention;

FIG. 3 is a schematic diagram of topology relationship in Embodiment 2 of a database data migration method based on inverse topology in the present invention.

Detailed ways

With reference to Fig. 1, a kind of database data migration method based on inverse topology of the present invention, comprises the following steps:

A. According to the database to be migrated, obtain its topology diagram;

B. According to the topological relationship diagram and the mode conversion model, the expansion order sequence in the migration process is obtained;

C. Migrate data and schemas to NoSQL according to the obtained extended order sequence.

As a further improvement of the database data migration method based on reverse topology, the step B includes:

B1. Randomly select a node from the leaf node set in the topology diagram as the initial node;

B2. Process all the edges with the initial node as the arc tail node, and obtain the integration sequence of the corresponding edges;

B3. Move the initial node out of the leaf node set and put it into the isolated node set;

B4. Add the integrated sequence obtained in the processing process to the extended sequence sequence;

B5. Go back to step B1 until the set of leaf nodes is an empty set.

As a further improvement of the database data migration method based on reverse topology, the step B2 includes:

B21. Randomly take any edge with the initial node as the arc tail node, and extend and integrate the arc head node of the edge to obtain the integration order of the edge;

B22. Determine whether the arc head node is a leaf node without including the edge, if so, put the starting point into the leaf node set; otherwise, perform step B23;

B23. Return to step B21 until all edges whose initial node is an arc tail node are processed.

Among them, the specific overview of step B is as follows:

(1) Randomly take a node u from the leaf node set P in the topology graph;

(2) Take out any edge <v, u> with the node u as the end of the arc, and expand the arc head node v of the edge, that is, insert all the information of the node u into the node v, and loop through all the following u is the edge of the arc tail;

(3) When the node v is expanded, that is, there is no edge with the point as the arc head, put the node into the leaf node set P. When the leaf node u is no longer dependent on any node, that is, the node is not used as the arc tail, the node is removed from the node set P and put into the isolated node set T;

(4) Process the leaf node set P in this way until the leaf node set P is an empty set.

The specific embodiment one of step B of the present invention is as follows:

Referring to Figure 2, for example, the relational database to be migrated includes tables T3, T2, and T1, and the dependencies include the foreign key dependence of T3 on T2, and the foreign key dependence of T2 on T1. The tables and dependencies of the database are expressed as a corresponding directed acyclic graph, and there are three nodes T3, T2, T1 and two tables <T3, T2>, <T2, T1>. Here are all the specific steps for this case:

S1. Randomly take out a node T1 from the leaf node set P={T1};

S2. Take out any edge <T2, T1> with the node T1 as the arc tail, and extend and integrate the arc head node T2 of the edge, that is, insert all the information of the node T1 into the node T2. After node T2 is expanded, that is, there is no edge with this point as the arc head, put the node into the leaf node set P, P={T1,T2}. When the leaf node T1 is no longer dependent on any node, that is, the node is not used as the arc tail, the node is removed from the node set P, P={T2}, and put into the isolated node set T, T={T1};

S3. Add the integration sequence <T2, T1> into the sequence S of the integration sequence result of the data table, S={<T2, T1>};

S4. Continue to take out a node T2 from the leaf node set P={T2};

S5. Take out any edge <T3, T2> with the node T2 as the arc tail, and extend and integrate the arc head node T3 of the edge, that is, insert all the information of the node T2 into the node T3. After node T3 is expanded, that is, there is no edge with this point as the arc head, put the node into the leaf node set P, P={T2,T3}. When the leaf node T2 is no longer dependent on any node, that is, the node is not used as the arc tail, the node is removed from the node set P, P={T3}, and put into the isolated node set T, T={T1,T2};

S6. Add the integration sequence <T3, T2> into the sequence S of the integration sequence result of the data table, S={<T2, T1>, <T3, T2>};

S7. Continue to take out a node T3 from the leaf node set P={T3}, without an edge with the node T2 as the arc tail, move this node out of the node set P, P={}, and put it into the isolated node set T, T={T1,T2,T3};

S8. At this time, P={}, P is an empty set;

S9. Outputting the sequence S of the integration sequence result of the data table, S={<T2, T1>, <T3, T2>}.

As a further improvement of the database data migration method based on reverse topology, the step C includes:

C1. Migrate the data of the nodes that only appear at the end of the arc to the corresponding data tables in the NoSQL database;

C2. Migrate the data of the remaining nodes to the corresponding data tables in the NoSQL database in sequence according to the extended order sequence according to the converted schema.

As a further improvement of the database data migration method based on reverse topology, the step C2 includes:

C21. Taking out the integration order in the extended order sequence sequentially;

C22. Integrating the information of the arc tail node in the extracted integration sequence into the arc head node and extending the arc head node;

C23. Migrate the data of the expanded arc head node to the corresponding data table in the NoSQL database.

The specific embodiment two of step C of the present invention is as follows:

Referring to Figure 3, where the extended order sequence:

S=[<T2,T1>,<T3,T1>,<T6,T5>,<T3,T2>,<T6,T2>].

1. First transfer the data in all leaf nodes T1 and T5 to tables T1’ and T5’ in the NoSQL database;

2. Take out the first tuple <T2, T1> of the tuples in S, integrate all the contents of T1 into T2, expand T2 into T2", and transfer the data of T2" to T2' in the NoSQL database;

3. Repeat the extension method in step 2 to process the remaining extension sequences [<T3,T1>,<T6,T5>,<T3,T2>,<T6,T2>] in S until S is an empty set.

The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the described embodiments, and those skilled in the art can also make various equivalent deformations or replacements without violating the spirit of the present invention. , these equivalent modifications or replacements are all within the scope defined by the claims of the present application.

Claims (5)

1. A database data migration method based on reverse topology, characterized in that: comprise the following steps: A. According to the database to be migrated, obtain its topology diagram; B. According to the topological relationship diagram and the mode conversion model, the expansion order sequence in the migration process is obtained; C. Migrate data and schemas to NoSQL according to the obtained extended order sequence. 2. a kind of database data migration method based on reverse topology according to claim 1, is characterized in that: described step B comprises: B1. Randomly select a node from the leaf node set in the topology diagram as the initial node; B2. Process all edges with the initial node as the arc tail node, and obtain the integration sequence of the corresponding edges; B3. Move the initial node out of the leaf node set and put it into the isolated node set; B4. Add the integrated sequence obtained in the processing process to the extended sequence sequence; B5. Go back to step B1 until the set of leaf nodes is an empty set. 3. a kind of database data migration method based on reverse topology according to claim 2, is characterized in that: described step B2 comprises: B21. Randomly take any edge with the initial node as the arc tail node, and extend and integrate the arc head node of the edge to obtain the integration order of the edge; B22. Determine whether the arc head node is a leaf node without including the edge, if so, put the starting point into the leaf node set; otherwise, perform step B23; B23. Return to step B21 until all edges whose initial node is an arc tail node are processed. 4. A kind of reverse topology-based database data migration method according to claim 1, characterized in that: said step C comprises: C1. Migrate the data of the nodes that only appear at the end of the arc to the corresponding data tables in the NoSQL database; C2. Migrate the data of the remaining nodes to the corresponding data tables in the NoSQL database in sequence according to the extended order sequence according to the converted schema. 5. A method for database data migration based on reverse topology according to claim 4, characterized in that: said step C2 comprises: C21. Taking out the integration order in the extended order sequence sequentially; C22. Integrating the information of the arc tail node in the extracted integration sequence into the arc head node and extending the arc head node; C23. Migrating the data of the expanded arc head node to the corresponding data table in the NoSQL database.