CN106503457B - Clinical data based on translational medicine analysis platform integrates technical data introduction method - Google Patents

Abstract

The invention relates to a clinical data integration technology data import method based on a translational medicine analysis platform, which belongs to the field of data integration technology; the method is used for the import of text-type data and the import of data in source databases: including preparing data files, column mapping Files, text replacement files, and files used to build a concept tree; build a concept tree, verify whether the data in the column mapping file meets the set rules, read and process the data for the first time, and add the custom variable logo again Construct a concept tree; import patient ID, visitor ID data, and patient information data into the platform database and perform relevant processing on data abnormalities, network abnormalities, or platform database system abnormalities. The characteristics of this method include building a concept tree in memory on the basis of tranSMART Batch, adding custom variables to directly read data from the database; adding an exception handling mechanism to make it have the characteristics of high automation and strong adaptability.

Description

Data import method of clinical data integration technology based on translational medicine analysis platform

technical field

The invention belongs to the technical field of data integration, and particularly relates to the realization of a tranSMART clinical data integration tool with high degree of automation and strong adaptability.

Background technique

In recent years, with the introduction of the concept of precision medicine and the vigorous development of related industries, clinical data and genetic data generated in the medical process have been gradually accumulated, and effective analysis of the medical information contained in these data has begun to appear in the medical field. more important.

tranSMART (translational medicine analysis platform) initially provides a data collaborative management platform for pharmaceutical companies and drug research and development organizations. Subsequently, tranSMART was gradually converted to open source software. And the tranSMART Foundation organization will take over for continuous development and maintenance. This platform is developed based on the I2B2 clinical medical data warehouse, so tranSMART can manage structured clinical data, such as demographic information and clinical pathological data.

tranSMART allows different groups to use the same data warehouse to integrate internal and external data resources. And it provides system-level tools to retrieve, view and analyze the data in the warehouse. tranSMART includes the following features:

Search functionality. By using the Solr search service, researchers can use search tools such as Google and Baidu to search for resource information of interest. The search function includes searching for a unique category, such as diseases, genes or clinical pathways, and other categories. Establish complex search conditions, you can customize the complex conditions of the type to be searched, and you can save them.

Dataset management functions. Based on the I2B2 design. It is possible to support self-defined subsets of studies. Conditions can be defined to compare the differences between two subsets. Featured features of the Data Manager include saving the criteria used to construct subsets. It can support data analysis with multiple analysis algorithms. For example, principal component analysis can be used to reduce the dimensionality of the data set.

Gene Imprinting Wizard function. tranSMART provides a wizard to assist in the creation and definition of gene signatures and gene lists. The saved tagged genes can be used in the analysis function of the dataset manager. The functions of the gene marker wizard include private gene imprinting, so that only you can access the private gene, and you can also change the gene to public so that all users can use it. An existing gene imprint can be cloned.

tranSMART is developed based on the Groovy language, and Groovy can be understood as a scripting language that can run on the Jvm. It and the Java language can call each other and integrate well. At the same time, it has the convenience and flexibility of scripting language. In particular, the concept of closure can make the program short and the readability will not be weakened. Many features of the Groovy language can greatly reduce the amount of code, especially the closure concept in Groovy, which is widely used in projects implemented with it. The Groovy language allows omitting parentheses and omitting semicolons. It can be seen that using the Groovy language can greatly improve the efficiency of programming. At the same time, the Groovy language can seamlessly call some frameworks and toolkits that Java already has. In fact, it combines very well with the Spring framework due to certain features.

The design of tranSMART is a typical N-Tier architecture, which is divided into data processing layer, business processing layer and display layer. The Grails framework used by tranSMART follows a principle that convention is greater than configuration. It only needs a few lines of configuration to distinguish between development, production, and test environments, and you can specify which environment it is when starting the system. Some Proven good presets and a large number of APIs are provided, so that the development can be smooth without paying too much attention to mature technical details. At the same time, Web applications generated by Grails are compatible with today's J2EE containers. The most commendable thing about Grails is that it has a wealth of plug-ins, a dedicated plug-in community, and this community has been growing. In addition, Grails also supports ORM and NoSQL.

There are several different implementations of tranSMART's existing data integration technology. Including the Kettle-based tranSMART Data tool, tranSMART ICE tool and scripting methods, as well as the method of using stored procedures. These tools only consider the one-time import of data when they are designed, but cannot meet the demand of adding data incrementally, that is, satisfy the updateability of data. And secondary development is more difficult. Finally, the tranSMART Batch project, which can provide data update support and is developed based on Spring Batch, was selected as the basis for building autonomous data integration tools.

tranSMART Batch is a data import program based on Spring Batch. Different types of programs can be imported. Such as one-time imported clinical scientific research data, genetic data, dimension data, I2B2 data, etc. suitable for temporary research. Among them, importing data in the form of I2B2 can satisfy data update and solve the problem of data dispersion to dimension tables.

The realization of tranSMART Batch adopts Groovy language. In terms of automated tool building, tranSMART Batch uses Gradle, an automated building tool based on the concepts of Ant and Maven. Gradle is based on Groovy's domain-specific language (DSL) to declare project settings. Compared with Maven and Ant, Gradle abandons the tedious XML configuration of Ant and Maven. There are many other Gradle features. For example, put the status of engineering dependence at a high level. And supports local construction. In addition, Gradle supports multiple ways of building dependencies. Including Maven repository, Nexus private server, Ivy repository, and jars or dirs of the local file system. Using Gradle to manage projects can achieve twice the result with half the effort. After configuring the database connection information, you need to execute the command ./GradlewSetupSchema to create the Schema in the database required by Spring Batch, which is used to record batch processing information for rollback, recording and other operations. tranSMART Batch imports data through the packaged jar package. By executing defined capsule tasks. You can get a jar package for importing data. Next, you need to configure the configuration information for the database connection. Is some regular jdbc configuration. Including the driver used, the location of the service, user name, password, etc.

The data import method using tranSMARTBatch integration technology is described as follows:

Step 1: Prepare documents. Prepared documents include:

11) Data file: the data file to be imported into tranSMART (translational medicine analysis platform, hereinafter referred to as the platform), the first line in the file is the header line, which is used to explain the data. Starting from the second line, each line represents a piece of data, and each specific data in the line is separated by a tab character ('/t').

12) Column mapping file: The column mapping file has six fixed columns. They are the file name of the data file, the data column number, whether the data can be null, data variable, data type (numeric data or text data or natural language type), data unit (such as g, ml and other units) . The data variable is used to describe the meaning of the data. The value of the data variable is some predefined symbols or a concept path representing a concept. The predefined symbols include PAT_ID, PAT_SEX, VISIT_ID and so on. They are used to describe the patient's ID, age, gender and visiting ID respectively. A concept consists of a concept path and a concept type. A concept path is a text that separates concepts with slashes, such as '\Open Research\Heart Disease Research\'.

13) Text replacement file: the function of the text replacement file is to clean up some data that does not meet the predefined specifications. So this file needs to specify which column in which data file needs to be replaced from what value to what value. This file consists of four columns, which are the file name, the column number of the data column, the original value, and the value to be replaced. For example, the data column is "Patient's basic examination file, 1, male, m". The representative means to replace all "male" data in the first column of the patient's basic examination file with "m". Step 2: Manually insert the path and type of the concept appearing in the column mapping file into the concept table in the tranSMART database. In addition to inserting the concept table, you also need to insert related content into the I2B2 table.

Step 3: Read the column mapping file: verify whether the data in the column mapping file meets the set rules, such as verifying whether the concepts given in the mapping file already exist in the concept table in the database, to ensure the correctness of the data of.

Step 4: Read and process data for the first time: read the data in the data file, first replace the data with the standard value according to the definition in the text replacement file; then verify the correctness of the data; supplement some missing data information. In the process of traversal, the patient's ID information is extracted, that is, the data variable of the column in the column mapping file is 'PAT_ID', and the ID data of the visitor, that is, the data variable of the column in the column mapping file is 'VIS_ID' data.

Step 5: Import the patient ID and visitor ID data into the platform database and read the data in the data file for the second time. Read this data row by row. According to the definition in the text replacement file, replace the data that does not meet the requirements of the defined column with the data that meets the requirements; according to the column serial number and data variable defined in the column mapping file, identify the specific representation of each data in each piece of data Meaning: Insert the corresponding patient information data and patient ID into the patient information table in the platform database, insert the corresponding visitor data and visitor ID into the visitor information table in the platform database, and identify the variables of the concept type in the data variables The data and concept path itself and other types of data units, data types and other related data defined in the column mapping file are inserted into the observation fact table in the platform database, that is, the data import is completed.

There are following problems in above-mentioned method:

1. The degree of automation is not high. In step two, a large amount of data needs to be manually edited. Cumbersome and error-prone.

2. The data organization form that supports import can only be in the form of the same concept for each column of data. According to the design of the existing column mapping file with variable type. It is not possible to integrate data organization situations where concepts and values exist in the same row.

3. The data source supports a single. In step 1, the data file to be imported needs to be prepared. This file is in text form. This method cannot extract data directly from the hospital database.

4. The exception handling mechanism is missing in the second traversal of the data in the step. Some exceptions that can be skipped will not be skipped, but will directly cause the program to crash.

Contents of the invention

The purpose of the present invention is to overcome the deficiencies of the prior art and propose a data import method based on the clinical data integration technology of the translational medicine analysis platform (tranSMART). The method has the characteristics of high degree of automation and strong adaptability.

A clinical data integration technology data import method based on a translational medicine analysis platform proposed by the present invention is characterized in that the method is used for text-based data import, and specifically includes the following steps:

Step 1: Prepare documents, the prepared documents include:

11) Data file: the data file to be imported to the platform. The first line in the file is the header line, which is used to explain the data; starting from the second line, each line represents a piece of data, and each specific data in the line is represented by the system. Table character ('/t') separated;

The data file also includes a column of data values corresponding to data of different concepts, but corresponding to data of different concepts; 12) column mapping file: the column mapping file has fixed six columns of content. They are the file name of the data file, the data column number, whether the data can be null, the data variable, the data type, and the data unit; the data variable is used to describe the meaning of the data. The value of the data variable is a predefined symbol or a concept path representing a concept; a concept is composed of a concept path and a concept type; a concept path is a text that separates concepts with slashes;

It also includes adding a custom logo to the data variable to reflect the hierarchical relationship of data variables with multiple concepts in one row;

13) Text replacement file: used to clean out some data that does not meet the predefined specifications; the file consists of four columns, which are the file name, the column number of the data column, the original value, and the value to be replaced;

14) The file used to build the concept tree: the file has two columns of concept path and concept category. The concept path divides the concept into different levels, and the different levels are separated by a slash "\". The concept category includes category type and Two types of numerical types; Step 2: Construct a concept tree according to the concept path and concept category hierarchy of the concept tree file constructed in step 1; insert the concept path and concept type in each node in the concept tree into the platform database In the concept table in , insert the concept path, concept type and the relative position of the node in the concept tree into the I2B2 table of the platform database;

Step 3: Read in the column mapping file, and verify whether the data in the column mapping file meets the set rules to ensure the correctness of the data;

Step 4: Read and process the data for the first time: first replace the data with a standardized value according to the definition in the text replacement file; then verify the correctness of the data; supplement the missing information of the data; extract the patient’s ID information and The data variable of this column in the column mapping file is the data of 'PAT_ID', and the ID data of the visit is the data of the data variable of the column in the column mapping file of 'VIS_ID';

Add the ID of the custom variable. Constructs a concept path from a custom-identified concept path variable. Constructs a concept type from a custom-identified concept type variable. Construct the concept tree again, and insert the concept tree into the table corresponding to the platform database;

Step 5: Import the patient ID and visitor ID data into the platform database and read the data in the data file for the second time: read the data by row, replace the data in the specified column that does not meet the requirements according to the definition in the text replacement file Replace with data that meets the requirements; identify the specific meaning of each data in each piece of data according to the column serial number and data variables defined in the column mapping file, and insert the patient information data and patient ID into the patient in the platform database In the information table, insert the visit data and the visit ID into the visit information table in the platform database, and compare the data identified by the variable of the concept type in the data variable with the concept path itself and other types of data defined in the column mapping file Insert relevant data such as units and data types into the observation fact table in the platform database; if there is an abnormality in the imported data, only make relevant records for later viewing; if it is a network abnormality or an abnormality in the platform database system, roll back the thing and record The current status of the import; that is, the data import is complete.

The invention also includes importing data in the source database.

Features and beneficial effects of the present invention:

The technical characteristics of the present invention are: on the basis of tranSMART Batch 1. by providing the concept tree file. Use the appropriate data structure to build a concept tree in memory, and automatically fill in the content of the concept table and I2B2 table in the tranSMART database; 2. By adding four custom variables in the column mapping file, and realize the function of processing data according to the custom variables , to complete the integration of the data organization form that cannot be integrated in the original method; 3. By adding the function of reading data from the database, the data can be read directly from the database; 4. By adding exception handling in the second traversal of the data in the step Mechanism, so that some exceptions that can be skipped can be skipped directly.

The beneficial effects of the present invention are:

This method can not only correctly and comprehensively import the clinical data in the form of text data into the tranSMART analysis platform, but also import the data in the source database, and then use various algorithms of tranSMART for data analysis.

This method does not lose the key medical information during the data import process, and each piece of clinical scientific research data can be traced. In the event of special circumstances, such as network exceptions and database system exceptions, you can record the success or failure of the import task, and choose whether to commit the transaction or roll back the transaction. Ensure the correctness of data import.

The data import of this method has a high degree of automation.

This method can choose different data source forms, such as text or database. There is a perfect exception handling mechanism. It can import a column of data values corresponding to different concepts of data, and has strong adaptability.

Description of drawings

Fig. 1 is an overall flow chart of the method of the present invention.

FIG. 2 is a schematic diagram of the present invention related to concept tree construction.

Detailed ways

The clinical data integration method based on the tranSMART platform proposed by the present invention will be described as follows in conjunction with the accompanying drawings and implementation methods:

The method of the present invention is implemented on the basis of tranSMART Batch with Spring Batch as the framework. The specific implementation method of the invention as shown in Figure 1, is used for the import of the data of text type, specifically comprises the following steps:

Step 1: Prepare documents, the prepared documents include:

11) Data file: the data file to be imported into the platform, the first line in the file is the header line, which is used to explain the data. Starting from the second line, each line represents a piece of data, and each specific data in the line is separated by a tab character ('/t').

The data file in this method may also include a column of data values corresponding to data of different concepts. But data corresponding to different concepts. For example, the concept corresponding to the value of 24.6 in Table 1 is prothrombin time activity, and the concept corresponding to the value of 30.2 is prothrombin time.

Table 1 Invention can be integrated sample data

12) Column mapping file: The column mapping file has six fixed columns. They are the file name of the data file, the data column number, whether the data can be null, data variable, data type (numeric data or text data or natural language type), data unit (such as g, ml and other units) . The data variable is used to describe the meaning of the data. The value of the data variable is some predefined symbols or a concept path representing a concept. The predefined symbols include PAT_ID, PAT_SEX, VISIT_ID and so on. They are used to describe the patient's ID, age, gender and visiting ID respectively. A concept consists of a concept path and a concept type. A concept path is a text that separates concepts with slashes, such as '\Open Research\Heart Disease Research\'.

This method adds a custom identifier to the data variable of the column mapping file, and this identifier reflects the hierarchical relationship of data variables with multiple concepts in a row. In order to solve the problem that the prior art cannot import data whose data values correspond to different concepts. The identifiers of data variables in the column mapping file in this embodiment are divided into identifiers of data variables representing concept types, identifiers of data variables representing numerical values, and identifiers of data variables representing numerical units.

13) Text replacement file: the function of the text replacement file is to clean up some data that does not meet the predefined specifications. So this file needs to specify which column in which data file needs to be replaced from what value to what value. This file consists of four columns, which are the file name, the column number of the data column, the original value, and the value to be replaced. For example, the data column is "Patient's basic examination file, 1, male, m". The representative means to replace all "male" data in the first column of the patient's basic examination file with "m".

14) The file used to build the concept tree: the file has two columns of concept path and concept category. The concept path divides the concept into different levels, and the different levels are separated by a slash "\". The concept category includes category type and Numerical types. Table 2 is an example of a concept tree file.

Table 2 Example of building a concept tree file

Step 2: Construct a concept tree according to the concept path and the hierarchy in the concept category of the concept tree file constructed in step 1. Figure 2 is an example of a concept tree constructed based on the data in Table 2. Insert the concept path and concept type in each node in the concept tree into the concept table in the platform database, and insert the concept path, concept type and the relative position of the node in the concept tree into the I2B2 table in the platform database.

Step 3: Read the column mapping file and verify whether the data in the column mapping file meets the set rules, such as verifying whether the concepts given in the mapping file already exist in the concept table in the database to ensure the correctness of the data of.

Step 4: Read and process data for the first time. First, replace the data with a standard value according to the definition in the text replacement file; then verify the correctness of the data; supplement some missing information of the data. The ID information of the patient is extracted, that is, the data variable of the column in the column mapping file is 'PAT_ID', and the ID data of the visitor is the data of the data variable of the column in the column mapping file, which is 'VIS_ID'.

When reading in, add the identifier of the custom variable. Constructs a concept path from a custom-identified concept path variable. Constructs a concept type from a custom-identified concept type variable. Then construct a concept tree in the form of FIG. 2 , and insert the concept tree into the corresponding table of the platform database.

Step 5: Import the patient ID and visitor ID data into the platform database and read the data in the data file for the second time: read the data by row, replace the data in the specified column that does not meet the requirements according to the definition in the text replacement file Replace with data that meets the requirements; identify the specific meaning of each data in each piece of data according to the column serial number and data variables defined in the column mapping file, and insert the patient information data and patient ID into the patient in the platform database In the information table, insert the visit data and the visit ID into the visit information table in the platform database, and compare the data identified by the variable of the concept type in the data variable with the concept path itself and other types of data defined in the column mapping file Units, data types and other relevant data are inserted into the observation fact table in the platform database; if there are abnormalities in the imported data, such as the absence of some data items, the duplication of some data items, only relevant records are made for later viewing; if it is If the network is abnormal or the platform database system is abnormal, the transaction will be rolled back, and the current import status will be recorded, including configuration information, time, abnormal conditions, etc.; that is, the data import will be completed.

The invention also includes the import of data in the source database:

When the data to be imported exists in the source database table, the stored procedure must first be written in the source database, and the stored procedure returns the cursor of the data to be imported; at the same time, the text data read-in unit is replaced with the stored procedure from the database The read-in unit that reads the data, and ensures that the result is the same as that of the text read-in unit; the specific import steps are as follows:

Step 1: Prepare documents: The prepared documents include:

11) Column mapping file: The column mapping file has six fixed columns. They are the predefined stored procedure name in the source database, data column number, whether the data can be null, data variable, data type (numeric data or text data or natural language type), data unit (such as g, ml and other units). The data variable is used to describe the meaning of the data. The value of the data variable is some predefined symbols or a concept path representing a concept. The predefined symbols include PAT_ID, PAT_SEX, VISIT_ID and so on. They are used to describe the patient's ID, age, gender and visiting ID respectively. A concept consists of a concept path and a concept type. A concept path is a slash-separated text, such as '\Open Research\Heart Disease Research\'.

This method adds a custom identifier to the data variable of the column mapping file, and this identifier reflects the hierarchical relationship of data variables with multiple concepts in a row. In order to solve the problem that the prior art cannot import data whose data values correspond to different concepts. The identifiers of data variables in the column mapping file in this embodiment are divided into identifiers of data variables representing concept types, identifiers of data variables representing numerical values, and identifiers of data variables representing numerical units.

12) Text replacement file: the function of the text replacement file is to clean up some data that does not meet the predefined specifications. So this file needs to specify which column in which data file needs to be replaced from what value to what value. This file consists of four columns, which are the file name, the column number of the data column, the original value, and the value to be replaced. For example, the data column is "Patient's basic examination file, 1, male, m". The representative means to replace all "male" data in the first column of the patient's basic examination file with "m".

13) Source database connection configuration file: including information such as the user name and password of the source database, whether to automatically reconnect, etc.

14) The file used to build the concept tree: the file has two columns of concept path and concept category. Concepts are divided into different levels, and the different levels are separated by a slash "\". kind. Table 2 is an example of a concept tree file.

Table 2 Example of building a concept tree file

Step 2: Construct a concept tree according to the concept path and the hierarchy in the concept category of the concept tree file constructed in step 1. Figure 2 is an example of a concept tree constructed based on the data in Table 2. Insert the concept path and concept type in each node in the concept tree into the concept table in the platform database, and insert the concept path, concept type and the relative position of the node in the concept tree into the I2B2 table in the platform database.

Step 3: Read the column mapping file and verify whether the data in the column mapping file meets the set rules, such as verifying whether the concepts given in the mapping file already exist in the concept table in the database to ensure the correctness of the data of.

Step 4: Traverse the data in the source database to be imported for the first time. First, replace the data with a standard value according to the definition in the text replacement file; then verify the correctness of the data; supplement some missing information of the data. In the process of traversal, the patient's ID information is extracted, that is, the data variable of the column in the column mapping file is 'PAT_ID', and the ID information of the visitor, that is, the data variable of the column in the column mapping file is 'VIS_ID' data.

When reading in, add the identifier of the custom variable. Constructs a concept path from a custom-identified concept path variable. Constructs a concept type from a custom-identified concept type variable. Then construct a concept tree in the form of Figure 2, and insert the tree into the table corresponding to the platform database.

Step 5: Import the patient ID and visitor ID data into the platform database and traverse the data in the source database for the second time: read the data in the source database by row, and replace the specified columns with those that do not meet the requirements according to the definition in the text replacement file Replace the data with the data that meets the requirements; identify the specific meaning of each data in each piece of data according to the column serial number and data variables defined in the column mapping file, and insert the patient information data and patient ID into the platform database correspondingly In the patient information table, the visitor data and the visitor ID are inserted into the visitor information table in the platform database correspondingly, and the data identified by the variable of the concept type in the data variable is related to the concept path itself and other types defined in the column mapping file Insert relevant data such as the data unit and data type into the observation fact table in the platform database; if there is an abnormality in the imported data, such as the absence of some data items or the duplication of some data items, only relevant records will be made for later viewing; if If it is a network abnormality or an abnormality in the platform database system, roll back the transaction and record the current import status, including configuration information, time, abnormal conditions, etc.; that is, the data import is completed.

Claims (2)

1. A clinical data integration technology data import method based on translation medicine analysis platform, it is characterized in that, the method is used for the import of text-type data, specifically comprises the following steps: Step 1: Prepare documents, the prepared documents include: 11) Data file: the data file to be imported to the platform. The first line in the file is the header line, which is used to explain the data; starting from the second line, each line represents a piece of data, and each specific data in the line is represented by the system. The table symbol '/t' separates; The data file also includes a column of data values corresponding to data of different concepts, but corresponding to data of different concepts; 12) Column mapping file: The column mapping file has fixed six columns of content; respectively, the file name of the data file, the data column number, whether this data can be a null value, the data variable, the data type, and the data unit; where the data variable is used To describe the meaning of the data; the value of the data variable is a predefined symbol or a concept path representing a concept; a concept is composed of a concept path and a concept type; a concept path is a text that separates concepts with slashes; It also includes adding a custom logo to the data variable to reflect the hierarchical relationship of data variables with multiple concepts in one row; 13) Text replacement file: used to clean out some data that does not meet the predefined specifications; the file consists of four columns, which are the file name, the column number of the data column, the original value, and the value to be replaced; 14) The file used to build the concept tree: the file has two columns of concept path and concept category. The concept path divides the concept into different levels, and the different levels are separated by a slash "\". The concept category includes category type and numerical type two types; Step 2: Construct a concept tree according to the concept path and concept category hierarchy of the concept tree file constructed in step 1; insert the concept path and concept type in each node in the concept tree into the concept table in the platform database , inserting the concept path, concept type and the relative position of the node in the concept tree into the I2B2 table of the platform database; Step 3: Read the column mapping file, and verify whether the data in the column mapping file meets the set rules to ensure the correctness of the data; Step 4: Read and process the data for the first time: first replace the data with a standardized value according to the definition in the text replacement file; then verify the correctness of the data; supplement the missing information of the data; extract the patient’s ID information and The data variable of this column in the column mapping file is the data of 'PAT_ID', and the ID data of the visit is the data of the data variable of the column in the column mapping file of 'VIS_ID'; Add the identity of the custom variable; construct the concept path according to the concept path variable of the custom identity; construct the concept type according to the concept type variable of the custom identity; construct the concept tree again, and insert the concept tree into the corresponding table of the platform database; Step 5: Import the patient ID and visitor ID data into the platform database and read the data in the data file for the second time: read the data by row, replace the data in the specified column that does not meet the requirements according to the definition in the text replacement file Replace with data that meets the requirements; identify the specific meaning of each data in each piece of data according to the column serial number and data variables defined in the column mapping file, and insert the patient information data and patient ID into the patient in the platform database In the information table, insert the visit data and the visit ID into the visit information table in the platform database, and compare the data identified by the variable of the concept type in the data variable with the concept path itself and other types of data defined in the column mapping file Data related to units and data types is inserted into the observation fact table in the platform database; if there is an abnormality in the imported data, only relevant records are made for later viewing; The status of the import; that is, the data import is complete. 2. data import method as claimed in claim 1, is characterized in that, also comprises the import that is used for the data in source database: at first will write stored procedure in this source database, and stored procedure returns the cursor that wants to import data; Simultaneously Replace the text data read-in unit with the read-in unit that reads data from the stored procedure in the database, and ensure that the result is the same as that of the text read-in unit; specifically, the following steps are included: Step 1: Prepare documents: The prepared documents include: 11) Column mapping file: The column mapping file has fixed six columns of content; they are the predefined stored procedure name in the source database, data column number, whether this data can be null, data variable, data type, and data unit; The data variable is used to describe the meaning of the data; the value of the data variable is a predefined symbol or a concept path representing a concept; a concept is composed of a concept path and a concept type; the concept path is a text separated by a slash; It also includes adding a custom logo to the data variable to reflect the hierarchical relationship of data variables with multiple concepts in one row; 12) Text replacement file: used to clean some data that does not meet the predefined specifications; the file consists of four columns, which are the file name, the column number of the data column, the original value, and the value to be replaced; 13) Source database connection configuration file: including the source database user name, password, and whether to automatically reconnect information; 14) The file used to build the concept tree: the file has two columns of concept path and concept category. Concepts are divided into different levels, and different levels are separated by slashes "\". The concept categories include category type and numerical type; Step 2: Construct a concept tree according to the concept path and concept category hierarchy of the concept tree file constructed in step 1; insert the concept path and concept type in each node in the concept tree into the concept table in the platform database , inserting the concept path, concept type and the relative position of the node in the concept tree into the I2B2 table of the platform database; Step 3: Read the column mapping file, and verify whether the data in the column mapping file meets the set rules to ensure the correctness of the data; Step 4: Traverse the data in the source database to be imported for the first time: first, replace the data with the standard value according to the definition in the text replacement file; then verify the correctness of the data; supplement the missing information of the data; extract the patient The ID information of the column is the data whose data variable is 'PAT_ID' in the column mapping file, and the ID information of the visit is the data whose data variable of the column is 'VIS_ID' in the column mapping file; Add the identification of the custom variable; construct the concept path according to the concept path variable of the custom identification; construct the concept type according to the concept type variable of the custom identification; then construct the concept tree, and insert the tree into the corresponding table of the platform database; Step 5: Import the patient ID and visitor ID data into the platform database and traverse the data in the source database for the second time: read the data in the source database by row, and replace the specified columns with those that do not meet the requirements according to the definition in the text replacement file Replace the data with the data that meets the requirements; identify the specific meaning of each data in each piece of data according to the column serial number and data variables defined in the column mapping file, and insert the patient information data and patient ID into the platform database correspondingly In the patient information table, the visitor data and the visitor ID are inserted into the visitor information table in the platform database correspondingly, and the data identified by the variable of the concept type in the data variable is related to the concept path itself and other types defined in the column mapping file The relevant data of the data unit and data type are inserted into the observation fact table in the platform database; if there is an exception in the imported data, only related records are made for later viewing; The current status of the import; that is, the data import is complete.