CN113111051A - Data migration method and device - Google Patents
Data migration method and device Download PDFInfo
- Publication number
- CN113111051A CN113111051A CN202110516630.4A CN202110516630A CN113111051A CN 113111051 A CN113111051 A CN 113111051A CN 202110516630 A CN202110516630 A CN 202110516630A CN 113111051 A CN113111051 A CN 113111051A
- Authority
- CN
- China
- Prior art keywords
- data
- data item
- content
- item
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/178—Techniques for file synchronisation in file systems
- G06F16/1794—Details of file format conversion
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data migration method and device, and relates to the field of automatic program design. One embodiment of the method comprises: determining a data item according to the set data migration range; acquiring corresponding data to be migrated from a first system according to the data items; processing the data to be migrated into the content of the data item according to a preset processing logic; the content of the data item is loaded to the second system. The implementation method can reduce the workload of workers, simplify the work content, improve the accuracy of data migration, improve the migration speed and reduce the migration time.
Description
Technical Field
The present invention relates to the field of automatic program design, and in particular, to a data migration method and apparatus.
Background
In the existing scheme for migrating data from an old system to a new system, due to the difference of database design, table structure design and overall architecture design between the old system and the new system, a worker needs to fully understand the related database design, database table structure and business logic of the old system and the new system, analyze the design difference of the old system and the new system, determine the specific scheme of data migration, and realize data migration.
In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art:
the workload of workers is large, the work content is complex, the accuracy of data migration is low, the migration speed is low, and the consumed time is long.
Disclosure of Invention
In view of this, embodiments of the present invention provide a data migration method and apparatus, which can reduce workload of workers, simplify work content, improve accuracy of data migration, improve migration speed, and reduce migration time.
To achieve the above object, according to an aspect of an embodiment of the present invention, a data migration method is provided.
A method of data migration, comprising: determining a data item according to the set data migration range; acquiring corresponding data to be migrated from a first system according to the data items; processing the data to be migrated into the content of the data item according to a preset processing logic; loading the content of the data item to a second system.
Optionally, the determining a data item according to the set data migration range includes: and determining the data items according to the pages and/or functions included in the set data migration range.
Optionally, the determining the data item according to the page and/or the function included in the set data migration range includes: splitting the set data migration range into data migration blocks according to the page and/or the function included in the set data migration range; determining the data items in each of the data migration blocks.
Optionally, the obtaining, according to the data item, the corresponding data to be migrated from the first system includes: acquiring a first mapping relation between the predetermined data item and a database table name and a database field name in the first system; and acquiring corresponding data to be migrated from the first system according to the database table name and the database field name in the first system according to the first mapping relation.
Optionally, the processing, according to a preset processing logic, the processing the data to be migrated into the content of the data item includes: unloading the data to be migrated into a first format file; converting the first format file to obtain a temporary file comprising the content of the data item to be processed; and processing the content of the data item to be processed in the temporary file according to the processing rule of the data item to obtain the content of the data item.
Optionally, the data corresponding to the data to be migrated in the first format file has a unique identifier, where the unique identifier is a data migration master serial number, and the data migration master serial number is obtained by encoding the data in the set data migration range.
Optionally, the converting the first format file to obtain a temporary file including the content of the data item to be processed includes: converting the first format file into an executable second format file; and executing the second format file to generate the temporary file comprising the content of the data item to be processed.
Optionally, the first format file is a file in an XML format or a Json format, and the second format file is a standard SQL file.
Optionally, after the processing the content of the data item to be processed in the temporary file according to the processing rule of the data item to obtain the content of the data item, the processing method includes: and checking the content of the data item according to a checking rule of the data item, and determining that the checking is passed.
Optionally, for each of the determined data items, the method further comprises: determining data rules in the data items, the number of the data items, the verification rules and the processing rules.
Optionally, the loading the content of the data item to a second system includes: acquiring a second mapping relation between the predetermined data item and a database table name and a database field name in the second system; and loading the content of the data item to a corresponding database table in the second system according to the second mapping relation and the name of the database table and the name of the database field in the second system.
According to another aspect of the embodiments of the present invention, a data migration apparatus is provided.
A data migration apparatus, comprising: the data item determining module is used for determining data items according to the set data migration range; the data acquisition module is used for acquiring corresponding data to be migrated from the first system according to the data items; the data processing module is used for processing the data to be migrated into the content of the data item according to a preset processing logic; a loading module for loading the content of the data item to a second system.
Optionally, the data item determination module is further configured to: and determining the data items according to the pages and/or functions included in the set data migration range.
Optionally, the data item determination module is further configured to: splitting the set data migration range into data migration blocks according to the page and/or the function included in the set data migration range; determining the data items in each of the data migration blocks.
Optionally, the data obtaining module is further configured to: acquiring a first mapping relation between the predetermined data item and a database table name and a database field name in the first system; and acquiring corresponding data to be migrated from the first system according to the database table name and the database field name in the first system according to the first mapping relation.
Optionally, the data processing module is further configured to: unloading the data to be migrated into a first format file; converting the first format file to obtain a temporary file comprising the content of the data item to be processed; and processing the content of the data item to be processed in the temporary file according to the processing rule of the data item to obtain the content of the data item.
Optionally, the data corresponding to the data to be migrated in the first format file has a unique identifier, where the unique identifier is a data migration master serial number, and the data migration master serial number is obtained by encoding the data in the set data migration range.
Optionally, the data processing module is further configured to: converting the first format file into an executable second format file; and executing the second format file to generate the temporary file comprising the content of the data item to be processed.
Optionally, the first format file is a file in an XML format or a Json format, and the second format file is a standard SQL file.
Optionally, the system further comprises a verification module, configured to: and checking the content of the data item according to a checking rule of the data item, and determining that the checking is passed.
Optionally, the data item determination module is further configured to: determining data rules in the data items, the number of the data items, the verification rules and the processing rules.
Optionally, the loading module is further configured to: acquiring a second mapping relation between the predetermined data item and a database table name and a database field name in the second system; and loading the content of the data item to a corresponding database table in the second system according to the second mapping relation and the name of the database table and the name of the database field in the second system.
According to yet another aspect of an embodiment of the present invention, an electronic device is provided.
An electronic device, comprising: one or more processors; a memory for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the data migration method provided by embodiments of the present invention.
According to yet another aspect of an embodiment of the present invention, a computer-readable medium is provided.
A computer-readable medium, on which a computer program is stored, which, when executed by a processor, implements the data migration method provided by an embodiment of the present invention.
One embodiment of the above invention has the following advantages or benefits: determining a data item according to the set data migration range; acquiring corresponding data to be migrated from a first system according to the data items; processing the data to be migrated into the content of the data item according to a preset processing logic; the content of the data item is loaded to the second system. The workload of workers can be reduced, the work content is simplified, the accuracy of data migration is improved, the migration speed is improved, and the migration time is shortened.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of the main steps of a data migration method according to one embodiment of the present invention;
FIG. 2 is a flow diagram of data migration according to one embodiment of the present invention;
FIG. 3 is a schematic diagram of the main modules of a data migration apparatus according to one embodiment of the present invention;
FIG. 4 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
fig. 5 is a schematic block diagram of a computer system suitable for use in implementing a terminal device or server of an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram of main steps of a data migration method according to an embodiment of the present invention.
As shown in fig. 1, the data migration method according to an embodiment of the present invention mainly includes the following steps S101 to S104.
Step S101: and determining the data items according to the set data migration range.
Determining the data item according to the set data migration range may include: and determining the data items according to the pages and/or functions included in the set data migration range.
Determining the data items according to the pages and/or functions included in the set data migration range may include: splitting the set data migration range into data migration blocks according to pages and/or functions included in the set data migration range; data items in each data migration block are determined.
For each determined data item, the embodiment of the present invention may further determine a data rule, a number of data items, a check rule, and a processing rule in the data item.
The data item is the minimum unit into which the data migration range can be divided.
Step S102: and acquiring corresponding data to be migrated from the first system according to the data items.
According to the data item, acquiring the corresponding data to be migrated from the first system may include: acquiring a first mapping relation between a predetermined data item and a database table name and a database field name in a first system; and acquiring corresponding data to be migrated from the first system according to the database table name and the database field name in the first system according to the first mapping relation.
Step S103: and processing the data to be migrated into the content of the data item according to preset processing logic.
According to the preset processing logic, processing the data to be migrated into the content of the data item may include: unloading the data to be migrated into a first format file; converting the first format file to obtain a temporary file comprising the content of the data item to be processed; and processing the content of the data item to be processed in the temporary file according to the processing rule of the data item to obtain the content of the data item.
The data corresponding to the data to be migrated (i.e., the data unloaded into the first format file) in the first format file has a unique identifier, the unique identifier is a data migration primary serial number, and the data migration primary serial number can be obtained by encoding the data in the set data migration range.
Converting the first format file to obtain a temporary file including the content of the data item to be processed may include: converting the first format file into an executable second format file; and executing the second format file to generate a temporary file comprising the content of the data item to be processed.
The first format file may be an XML (Extensible Markup Language) format file or a Json (java script object Notation) format file, and the second format file may be a standard SQL (Structured Query Language) file, where Json is a lightweight data exchange format and SQL is a database Query and programming Language.
Processing the content of the data item to be processed in the temporary file according to the processing rule of the data item, and after obtaining the content of the data item, the processing method may include: and checking the content of the data item according to the checking rule of the data item, and determining that the checking is passed. For example, checking the content of the data item may be checking whether the content of the data item meets a certain data rule in the data item, for example, checking whether the content of the data item meets a set length requirement, and the like.
Step S104: the content of the data item is loaded to the second system.
Loading the content of the data item to the second system may include: acquiring a second mapping relation between a predetermined data item and a database table name and a database field name in a second system; and loading the content of the data item to a corresponding database table in the second system according to the database table name and the database field name in the second system according to the second mapping relation.
The first system may be an old system, the second system may be a new system, and the data migration is to migrate data from the old system to the new system.
In one embodiment, the set data migration range is divided into data migration blocks according to the page and/or function included in the set data migration range, and each data migration block is refined to determine the data item. Cutting in from the user angle, and determining the whole range of data migration according to the pages (such as query pages, detail display pages and the like) at the user view angle and the functions (such as import, export, printing and the like) required by the user, so as to obtain the set data migration range; splitting the whole data migration range into data migration blocks according to pages and functions from the clear whole data migration range, wherein one data migration page or function can be divided into one or more data migration blocks; and refining each data migration block to determine data migration items (namely data items), wherein one data migration block can be refined into one or more data migration items.
In one embodiment, a data migration main serial number is obtained by encoding data in a set data migration range, the data migration main serial number is a unique identifier, and data corresponding to data to be migrated in a first format file has the unique identifier. And numbering the data of each page and/or each function in the definite data migration whole range to generate a data migration main serial number, wherein the data migration main serial number has global uniqueness, and the data to be migrated can be determined through the data migration main serial number and the migration data item.
In one embodiment, for each data item, a data rule, a number of data items, a validation rule, and a processing rule in the data item are determined. The data rules in the data item may include a single data rule and a plurality of data rules, wherein the single data rule is a data rule related to a single data item, such as whether the data item content may be empty, must be a number, whether the length meets requirements, data content filling requirements, and the like. The multiple data rules are data rules related to two or more data items, such as whether the corresponding relationship between the 2 nd digit of the identification number data item and the gender data item is correct or not. The processing rule and the verification rule can be determined by the data rule; a data item table, a data item content table, a data item processing table, and a data item check rule table may be respectively established for the data item, the data item content, the processing rule, and the check rule, as shown in table 1, table 2, table 3, and table 4, where the data item id is a unique identifier of the data item.
TABLE 1 data item Table
| Serial number | Data item id | Data item names |
Table 2 data item table of contents
TABLE 3 data item processing Table
| Serial number | Data item id | Rules of processing |
TABLE 4 data item verification table
| Serial number | Data item id | Checking rules | Error prompt |
In one embodiment, a first mapping relationship between a data item and a database table name and a database field name in a first system is predetermined, and the first mapping relationship is used for acquiring corresponding data to be migrated from the first system according to the database table name and the database field name in the first system. For each data migration item, the source of the data migration item in the old system (i.e., the first system) is predetermined, the source specifically includes a database table name and a database field name, and a mapping relationship table of the old system and the data item is generated, as shown in table 5, in which the first mapping relationship is recorded.
TABLE 5 old System and data item mapping Table
In one embodiment, a second mapping relationship between the data item and the database table name and the database field name in the second system is predetermined, and the second mapping relationship is used for loading the content of the data item to a corresponding database table in the second system according to the database table name and the database field name in the second system. For each data migration item, the destination of the data migration item in the new system (i.e. the second system) is predetermined, the destination specifically includes a database table name and a database field name, and a mapping relationship table of the new system and the data item is generated, as shown in table 6, in which the second mapping relationship is recorded.
TABLE 6 mapping relationship table of new system and data items
FIG. 2 is a flow diagram of data migration, according to one embodiment of the invention.
As shown in fig. 2, the embodiment of the present invention may include a data unloading unit, a converting unit, a processing unit, a verifying unit, and a loading unit.
In the number unloading unit, acquiring a first mapping relation between a predetermined data item and a database table name and a database field name in a first system; acquiring corresponding data to be migrated from the first system according to the database table name and the database field name in the first system according to the first mapping relation; and unloading (i.e. saving) the data to be migrated into the first format file. Specifically, the data of the old system is unloaded into a file (i.e., a first format file) in an XML format or a Json format according to the mapping relationship table of the old system and the data item.
In a conversion unit, converting the first format file into an executable second format file; and executing the second format file to generate a temporary file comprising the content of the data item to be processed. Specifically, the file in the XML format or the Json format is processed, converted into a standard SQL file (i.e., a file in the second format), and executed in the database to generate a temporary file, and if the conversion fails, the unloading unit is executed again after checking and adjusting.
And in the processing unit, processing the content of the data item to be processed in the temporary file according to the processing rule of the data item to obtain the content of the data item. Specifically, data processing is performed on the data item content to be processed in the temporary file according to the processing rule in the data item processing table, and if the processing fails, the adjustment is checked and the data unloading unit is executed again, where the processing may include standard data conversion, special symbol processing, and the like, and correspondingly, the processing rule may include relevant rules of standard data conversion, special symbol processing, and the like.
In the verification unit, the content of the data item is verified according to the verification rule of the data item, and verification is determined to pass (for example, the content of the data item is verified to meet the requirement of the verification rule). Specifically, the processed data item content is verified according to the verification rules in the data item verification rule table, if the verification fails, an error prompt is displayed, and the number unloading unit is executed again after checking and adjusting.
In the loading unit, acquiring a second mapping relation between a predetermined data item and a database table name and a database field name in a second system; and loading the content of the data item to a corresponding database table in the second system according to the database table name and the database field name in the second system according to the second mapping relation. Specifically, according to the mapping relation table of the new system and the data items, loading the processed and verified data item contents into the corresponding database table in the new system, and if the loading fails, re-executing the unloading unit after checking and adjusting.
In one embodiment, data migration from one or more first systems to one or more second systems may be accomplished by altering the first mapping and/or the second mapping of data items.
In one embodiment, the processing rule of the data item is determined by the staff of the old system, the verification rule is determined by the staff of the new system, the staff of the old system does not need to learn the content architecture of the new system, the staff of the new system does not need to learn the content architecture of the old system, and the staff of the two systems are respectively responsible for respective work content, so that the embodiment of the invention can greatly reduce the workload of the staff, simplify the work content and improve the efficiency and accuracy of data migration.
The data migration method of the embodiment of the invention is switched in from the perspective of a user, determines the range of the migrated data, determines the check rule of each data item, can process and check the data by the dimensionality of the migrated data item (namely, the data item), and design and development personnel (namely, working personnel) can be responsible for the parts of the respective systems, so that a large amount of contents of the systems of the other party do not need to be cross-learned, the learning contents are relatively less, the working personnel of the new system and the old system only need to complete the source and the destination of the respective data item, the accuracy of data migration is greatly improved, the complex data migration work can be completed in a short time, and the operability is strong.
FIG. 3 is a schematic diagram of the main modules of a data migration apparatus according to an embodiment of the present invention.
As shown in fig. 3, the data migration 300 according to an embodiment of the present invention mainly includes: a data item determining module 301, a data acquiring module 302, a data processing module 303 and a loading module 304.
A data item determining module 301, configured to determine a data item according to a set data migration range;
a data obtaining module 302, configured to obtain, according to a data item, corresponding data to be migrated from a first system;
a data processing module 303, configured to process data to be migrated into content of a data item according to a preset processing logic;
a loading module 304 for loading the content of the data item to the second system.
In one embodiment, the data item determination module is specifically configured to: and determining the data items according to the pages and/or functions included in the set data migration range.
In one embodiment, the data item determination module is specifically configured to: splitting the set data migration range into data migration blocks according to pages and/or functions included in the set data migration range; data items in each data migration block are determined.
In one embodiment, the data acquisition module is specifically configured to: acquiring a first mapping relation between a predetermined data item and a database table name and a database field name in a first system; and acquiring corresponding data to be migrated from the first system according to the database table name and the database field name in the first system according to the first mapping relation.
In one embodiment, the data processing module is specifically configured to: unloading the data to be migrated into a first format file; converting the first format file to obtain a temporary file comprising the content of the data item to be processed; and processing the content of the data item to be processed in the temporary file according to the processing rule of the data item to obtain the content of the data item.
In one embodiment, the data corresponding to the data to be migrated in the first format file has a unique identifier, the unique identifier is a data migration primary serial number, and the data migration primary serial number is obtained by encoding data in a set data migration range.
In one embodiment, the data processing module is specifically configured to: converting the first format file into an executable second format file; and executing the second format file to generate a temporary file comprising the content of the data item to be processed.
In one embodiment, the first format file is an XML format or Json format file, and the second format file is a standard SQL file.
In one embodiment, the system further comprises a verification module for: and checking the content of the data item according to the checking rule of the data item, and determining that the checking is passed.
In one embodiment, the data item determination module is specifically configured to: and determining data rules, the number of the data items, the check rules and the processing rules in the data items.
In one embodiment, the loading module is specifically configured to: acquiring a second mapping relation between a predetermined data item and a database table name and a database field name in a second system; and loading the content of the data item to a corresponding database table in the second system according to the database table name and the database field name in the second system according to the second mapping relation.
In addition, the detailed implementation of the data migration apparatus in the embodiment of the present invention has been described in detail in the above data migration method, so that repeated descriptions herein will not be repeated.
Fig. 4 illustrates an exemplary system architecture 400 to which the data migration method or the data migration apparatus of the embodiments of the present invention may be applied.
As shown in fig. 4, the system architecture 400 may include terminal devices 401, 402, 403, a network 404, and a server 405. The network 404 serves as a medium for providing communication links between the terminal devices 401, 402, 403 and the server 405. Network 404 may include various types of connections, such as wire, wireless communication links, or fiber optic cables, to name a few.
A user may use terminal devices 401, 402, 403 to interact with a server 405 over a network 404 to receive or send messages or the like. The terminal devices 401, 402, 403 may have installed thereon various communication client applications, such as shopping-like applications, web browser applications, search-like applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).
The terminal devices 401, 402, 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 405 may be a server providing various services, such as a background management server (for example only) providing support for shopping websites browsed by users using the terminal devices 401, 402, 403. The backend management server may analyze and perform other processing on the received data such as the product information query request, and feed back a processing result (for example, target push information, product information — just an example) to the terminal device.
It should be noted that the data migration method provided by the embodiment of the present invention is generally executed by the server 405, and accordingly, the data migration apparatus is generally disposed in the server 405.
It should be understood that the number of terminal devices, networks, and servers in fig. 4 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 5, a block diagram of a computer system 500 suitable for use with a terminal device or server implementing an embodiment of the invention is shown. The terminal device or the server shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 5, the computer system 500 includes a Central Processing Unit (CPU)501 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data necessary for the operation of the system 500 are also stored. The CPU 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
The following components are connected to the I/O interface 505: an input portion 506 including a keyboard, a mouse, and the like; an output portion 507 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The driver 510 is also connected to the I/O interface 505 as necessary. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as necessary, so that a computer program read out therefrom is mounted into the storage section 508 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 509, and/or installed from the removable medium 511. The computer program performs the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 501.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor comprises a data item determining module, a data acquiring module, a data processing module and a loading module. The names of these modules do not constitute a limitation to the module itself in some cases, and for example, the data item determination module may also be described as a "module for determining a data item according to a set data migration range".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise: determining a data item according to the set data migration range; acquiring corresponding data to be migrated from a first system according to the data items; processing the data to be migrated into the content of the data item according to a preset processing logic; the content of the data item is loaded to the second system.
According to the technical scheme of the embodiment of the invention, the data items are determined according to the set data migration range; acquiring corresponding data to be migrated from a first system according to the data items; processing the data to be migrated into the content of the data item according to a preset processing logic; the content of the data item is loaded to the second system. The workload of workers can be reduced, the work content is simplified, the accuracy of data migration is improved, the migration speed is improved, and the migration time is shortened.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (16)
1. A method of data migration, comprising:
determining a data item according to the set data migration range;
acquiring corresponding data to be migrated from a first system according to the data items;
processing the data to be migrated into the content of the data item according to a preset processing logic;
loading the content of the data item to a second system.
2. The method of claim 1, wherein determining the data items according to the set data migration range comprises:
and determining the data items according to the pages and/or functions included in the set data migration range.
3. The method according to claim 2, wherein the determining the data items according to the pages and/or functions included in the set data migration range comprises:
splitting the set data migration range into data migration blocks according to the page and/or the function included in the set data migration range;
determining the data items in each of the data migration blocks.
4. The method according to claim 1, wherein the obtaining the corresponding data to be migrated from the first system according to the data item comprises:
acquiring a first mapping relation between the predetermined data item and a database table name and a database field name in the first system;
and acquiring corresponding data to be migrated from the first system according to the database table name and the database field name in the first system according to the first mapping relation.
5. The method according to any one of claims 1 to 3, wherein the processing the data to be migrated into the content of the data item according to a preset processing logic comprises:
unloading the data to be migrated into a first format file;
converting the first format file to obtain a temporary file comprising the content of the data item to be processed;
and processing the content of the data item to be processed in the temporary file according to the processing rule of the data item to obtain the content of the data item.
6. The method according to claim 5, wherein the data in the first format file corresponding to the data to be migrated has a unique identifier, the unique identifier is a data migration primary serial number, and the data migration primary serial number is obtained by encoding data in the set data migration range.
7. The method of claim 5, wherein converting the first format file to obtain a temporary file including the content of the data item to be processed comprises:
converting the first format file into an executable second format file;
and executing the second format file to generate the temporary file comprising the content of the data item to be processed.
8. The method of claim 7, wherein the first format file is an XML format or Json format file and the second format file is a standard SQL file.
9. The method according to claim 5, wherein the processing the content of the data item to be processed in the temporary file according to the processing rule of the data item, after obtaining the content of the data item, comprises:
and checking the content of the data item according to a checking rule of the data item, and determining that the checking is passed.
10. The method of claim 9, wherein for each of the determined data items, the method further comprises: determining data rules in the data items, the number of the data items, the verification rules and the processing rules.
11. The method of claim 1, wherein loading the content of the data item to a second system comprises:
acquiring a second mapping relation between the predetermined data item and a database table name and a database field name in the second system;
and loading the content of the data item to a corresponding database table in the second system according to the second mapping relation and the name of the database table and the name of the database field in the second system.
12. A data migration apparatus, comprising:
the data item determining module is used for determining data items according to the set data migration range;
the data acquisition module is used for acquiring corresponding data to be migrated from the first system according to the data items;
the data processing module is used for processing the data to be migrated into the content of the data item according to a preset processing logic;
a loading module for loading the content of the data item to a second system.
13. The apparatus of claim 12, wherein the data item determination module is further configured to:
and determining the data items according to the pages and/or functions included in the set data migration range.
14. The apparatus of claim 12 or 13, wherein the data processing module is further configured to:
unloading the data to be migrated into a first format file;
converting the first format file to obtain a temporary file comprising the content of the data item to be processed;
and processing the content of the data item to be processed in the temporary file according to the processing rule of the data item to obtain the content of the data item.
15. An electronic device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-11.
16. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110516630.4A CN113111051A (en) | 2021-05-12 | 2021-05-12 | Data migration method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110516630.4A CN113111051A (en) | 2021-05-12 | 2021-05-12 | Data migration method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113111051A true CN113111051A (en) | 2021-07-13 |
Family
ID=76722418
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110516630.4A Pending CN113111051A (en) | 2021-05-12 | 2021-05-12 | Data migration method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113111051A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114281801A (en) * | 2021-12-23 | 2022-04-05 | 中国建设银行股份有限公司 | A data migration method and device |
| CN115344556A (en) * | 2022-08-16 | 2022-11-15 | 中国民生银行股份有限公司 | Data processing method, device, equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070179983A1 (en) * | 2006-01-31 | 2007-08-02 | International Business Machines Corporation | Method and program product for migrating data from a legacy system |
| CN109284293A (en) * | 2018-10-08 | 2019-01-29 | 湖南工学院 | A data migration method for upgrading the water company's operating charging system |
-
2021
- 2021-05-12 CN CN202110516630.4A patent/CN113111051A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070179983A1 (en) * | 2006-01-31 | 2007-08-02 | International Business Machines Corporation | Method and program product for migrating data from a legacy system |
| CN109284293A (en) * | 2018-10-08 | 2019-01-29 | 湖南工学院 | A data migration method for upgrading the water company's operating charging system |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114281801A (en) * | 2021-12-23 | 2022-04-05 | 中国建设银行股份有限公司 | A data migration method and device |
| CN115344556A (en) * | 2022-08-16 | 2022-11-15 | 中国民生银行股份有限公司 | Data processing method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113760948A (en) | Method and device for querying data | |
| CN113641633B (en) | File processing method, apparatus, electronic device, medium, and computer program | |
| CN112926008B (en) | Method, device, electronic equipment and storage medium for generating form page | |
| CN110689268B (en) | Method and device for extracting indexes | |
| CN112948138B (en) | A method and device for processing messages | |
| CN111125064B (en) | Method and device for generating database schema definition statement | |
| CN113641700B (en) | A data processing method and device based on Spring boot framework | |
| CN111104479A (en) | Data labeling method and device | |
| CN113626223A (en) | Interface calling method and device | |
| CN110705271B (en) | System and method for providing natural language processing service | |
| CN114356884A (en) | Data migration method and device | |
| CN113076153A (en) | Interface calling method and device | |
| CN113111051A (en) | Data migration method and device | |
| CN112947919A (en) | Method and device for constructing service model and processing service request | |
| CN113312900A (en) | Data verification method and device | |
| CN118194837A (en) | Report file generation method and device | |
| CN118133794B (en) | Table configuration method, apparatus, device and storage medium | |
| CN112783903B (en) | Method and device for generating update log | |
| CN113779122A (en) | Method and apparatus for exporting data | |
| CN109522211B (en) | Interface parameter transmission method and device, electronic equipment and storage medium | |
| CN113760714B (en) | Software testing method and device | |
| CN113590483B (en) | Use case operation method and device | |
| CN111523295A (en) | Data rendering method, device, medium and electronic equipment | |
| CN116204428A (en) | Test case generation method and device | |
| CN116069725A (en) | File migration method, device, apparatus, medium and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |