CN114925074A - A data unloading method, device, device and medium - Google Patents

Abstract

The invention relates to the technical field of data processing, in particular to a data unloading method, a data unloading device, data unloading equipment and a data unloading medium. The method specifically comprises the following steps: generating a database table list to be unloaded according to a plurality of database tables, wherein the database table list comprises respective indexes of the plurality of database tables and a plurality of field information, and the plurality of field information comprises field information corresponding to each database table in the plurality of database tables; generating unloading parameters according to field information of the database table list, wherein the unloading parameters are used for unloading a plurality of database tables; and importing the unloading parameters into an unloading script to generate an unloading file, wherein the unloading script is used for unloading the plurality of database tables. The method can dynamically configure the unloading time of each database table, thereby unloading the data in different data sources simultaneously and improving the unloading efficiency.

Description

A data unloading method, device, device and medium

technical field

The present invention relates to the technical field of data processing, in particular to a data unloading method, device, device and medium.

Background technique

In recent years, with the rapid growth of the amount of information, a large amount of data is generated along with daily life. For example, when the sponsor conducts a marketing campaign, there is data indicating the participation in the event, data indicating the development of the event, and data indicating the distribution of prizes, and so on. Statistical analysis of these data is usually required. Due to the different sources of data, for statistical analysis of these data, generally, these data are first stored in the database to generate database tables. Send the database table to the data analysis system for analysis. The most common method is to use Oracle to generate unload files for unloading database tables, and then send them to the data analysis system. In this way, the data analysis system receives the unloading file, and reloads the data in the database table according to the unloading file.

Currently, only one database table is supported for unloading (that is, processing, such as extracting partial data, etc.). In scenarios with multiple data sources, only one database table can be unloaded at a time, and the unloading efficiency is low. And since different data sources have different business execution dates, unloading different data sources uniformly may unload unnecessary data.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide a data unloading method, apparatus, device and medium. When different data sources exist, data in different data sources can be unloaded at the same time, thereby improving the efficiency of data unloading.

In a first aspect, an embodiment of the present invention provides a data unloading method. The method can be executed by an electronic device with computing processing capability, such as a personal computer. The method includes:

The electronic device generates a list of database tables to be uninstalled according to a plurality of database tables, the database table list includes respective indexes of the plurality of database tables and a plurality of field information, and the plurality of field information also includes the corresponding database tables in the plurality of database tables respectively field information. Then, the electronic device generates an unloading parameter according to the field information of the database table list, where the unloading parameter is used to unload a plurality of database tables. After that, the electronic device imports the uninstallation parameters into an uninstallation script to generate an uninstallation file, where the uninstallation script is used to uninstall a plurality of database tables. In the scenario of multiple data sources, when uninstalling data from different data sources, different data sources have different uninstall dates, and unified uninstallation of different data sources may uninstall unnecessary data. Therefore, in this embodiment of the present invention, the unloading time of the data source may be dynamically configured, for example, the unloading time may be the same for different data sources. When uninstalling, a database table list is generated according to the database tables of different data sources to be uninstalled, and an uninstall file is generated according to the uninstall parameters generated by the database table list. Therefore, by running the uninstallation file, each database table in the database table list can be uninstalled in sequence according to the uninstallation parameters, that is, simultaneous uninstallation of multiple data sources is realized, and the uninstallation efficiency is improved.

Optionally, the multiple field information of the database table list includes uninstallation mode information, the uninstallation mode information includes the uninstallation mode, the uninstallation mode includes the full uninstallation mode or the incremental uninstallation mode, and the full uninstallation mode indicates that the uninstallation database table list includes all the database tables included. The incremental unloading mode indicates unloading data added within a preset time interval in multiple database tables, and generating the unloading parameters according to the field information of the database table list includes: the electronic device obtains the unloading mode information, and according to the unloading mode information The uninstall parameters are generated. In this way, when there are multiple database tables in the database table list, the user can configure different unloading modes for each database table according to actual needs. Correspondingly, the electronic device generates the unloading parameters according to the unloading mode configured by the user, which can realize the flexibility of the database table. Uninstall.

Optionally, the multiple field information of the database table list also includes unloading time information, the unloading time information indicates the unloading time of the database table in the incremental unloading mode, and the unloading parameters are generated according to the field information of the database table list, including: electronic The device obtains the unloading time information, and generates the unloading parameters according to the unloading time information, wherein the unloading parameters include the ith unloading time and the preset time interval, and the preset time interval and the ith unloading time are used to determine the ith+1 th unloading time . It can be understood that, compared with the full offload mode, the incremental offload mode is to offload data within a preset time interval. Therefore, the unload time of individual database tables can be dynamically configured. Different database tables can be set to the same unload time, so that multiple database tables can be unloaded within the same period of time. In addition, the unloading time may include the i-th unloading time and the i+1-th unloading time. Among them, the i+1-th unloading time can be calculated by the i-th unloading time and the preset time interval, that is, the i+1-th unloading time is indirectly configured through the preset time interval, and there is no need to configure the i+1-th unloading time separately. It can reduce the number of times to configure the i+1 unloading time, which is more convenient.

Optionally, after the uninstallation parameters are imported into the uninstallation script to generate the uninstallation file, the method further includes: the electronic device updates the i+1th uninstallation time to the ith uninstallation time, and then obtains the ith uninstallation time and the preset time. Time interval to generate a new i+1 unloading time. The preset time intervals corresponding to different database tables are different. This solution can update the i+1 th unloading time to the i th unloading time, thereby realizing the purpose of cyclically setting the unloading time of the database table. That is, by setting the i-th unloading time and the preset time interval once, the i-th unloading time and the i+1-th unloading time can be set cyclically. At the same time, the unloading of each database table can be achieved over time.

Optionally, the multiple field information of the database table list includes an uninstallation identifier, and the uninstallation identifier is used to indicate whether the uninstallation of the database table is completed, and the method further includes: the electronic device obtains the uninstallation identifier corresponding to the first database table, if the uninstallation identifier indicates After the unloading of the first database table is completed, push information is obtained, where the push information includes a storage path of the first database table and a target file path. Thus, the electronic device acquires the first database table from the storage path, and pushes the first database table to the target file path. In this embodiment of the present invention, after the electronic device unloads the database table, it can also push the unloaded database table. For example, the electronic device can determine which database tables are to be uninstalled according to the uninstall identifier, and then select the push time according to the requirements. When the push time is reached, the electronic device can obtain the database table to be pushed according to the push information, and push the database table to the target file path.

Optionally, the push information further includes a push mode identification, the push mode identification indicates a push mode, and the electronic device pushes the first database to the target file path, including: the electronic device obtains the push mode identification, and according to the push mode identification indicates the push mode. Push the first database table to the target file path. In this embodiment of the present invention, there are multiple push modes, including, for example, a jfserver push mode, an ftp push mode, and an httpclient push mode. In different application scenarios, a matching push mode can be selected, and the application range is wider.

Optionally, after the uninstallation parameters are imported into the uninstallation script to generate the uninstallation file, the method further includes: the electronic device receives the input uninstallation times, and uninstalls the database table that reaches the uninstallation time according to the uninstallation times. Users can set the number of unloading according to actual needs. Since the electronic device only unloads the data within a preset time interval of the database table at a time, by setting the number of unloading, the electronic device can unload the data within multiple preset time intervals, thereby improving the electronic performance. The unloading efficiency of the device. At the same time, by setting the number of uninstalls, when a database table fails to be uninstalled at one time, it can be uninstalled next time, effectively avoiding data omission caused by the failure of uninstallation.

In a second aspect, an embodiment of the present invention provides a data unloading apparatus, and the data changing apparatus can be used to implement the data unloading method provided in the first aspect. The data unloading device includes a list generating module, a parameter generating module, and an unloading module. The list generation module is configured to generate a list of database tables to be uninstalled according to multiple database tables, the database table list includes respective indexes of the multiple database tables and multiple field information, and the multiple field information includes the Field information corresponding to each database table in the multiple database tables. The parameter generation module is configured to generate an unloading parameter according to the field information of the database table list, where the unloading parameter is used to unload the plurality of database tables. The uninstallation module is configured to import the uninstallation parameters into an uninstallation script to generate an uninstallation file, and the uninstallation script is used to uninstall the multiple database tables.

Optionally, the parameter generation module is specifically configured to: acquire the uninstallation mode information, and generate the uninstallation parameter according to the uninstallation mode information.

Optionally, the parameter generation module is further configured to: obtain the unloading time information, and generate the unloading parameters according to the unloading time information, wherein the unloading parameters include the ith unloading time and the preset time interval, and the preset time interval and the ith unloading time. Used to determine the i+1th unloading time.

Optionally, the parameter generation module is further configured to: update the i+1 th unloading time to the i th unloading time. Obtain the i-th unloading time and the preset time interval, and generate a new i+1-th unloading time, wherein the preset time intervals corresponding to different database tables are different.

Optionally, the data unloading device is further configured to: obtain the unloading identifier corresponding to the first database table, and if the unloading identifier indicates that the unloading of the first database table is completed, obtain push information, where the push information includes the storage path of the first database table and the target file path. . Acquire the first database table from the storage path, and push the first database table to the target file path.

Optionally, the data unloading apparatus is further configured to: obtain the push mode identifier, and push the first database table to the target file path according to the push mode indicated by the push mode identifier.

Optionally, the parameter generation module is further configured to: receive the input number of unloading, and unload the database table that reaches the unloading time according to the number of unloading.

In a third aspect, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored on the memory that can be executed by the processor. When the processor executes the computer program, the first aspect and various options are implemented. the method.

In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, implements the steps described in the first aspect and various options.

In a fifth aspect, the present invention also provides a computer program product, the computer program product includes: computer program code, when the computer program code is run on the computer, the computer is made to execute the above-mentioned steps as described in the method of the first aspect and each optional item .

Description of drawings

1 is a schematic flowchart of a data unloading method according to an embodiment of the present invention;

2 is a schematic structural diagram of a data unloading apparatus provided by an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. In the technical solution of the present invention, the collection, transmission, and use of data all meet the requirements of relevant national laws and regulations.

The purpose of the embodiments of the present invention is to realize the dynamic configuration of offloading and pushing multiple data sources. In order to better understand the solutions in the embodiments of the present invention, some concepts involved in the embodiments of the present invention are first introduced below.

1) The database table is a warehouse that organizes, stores and manages data according to the data structure. Typically, one or more database tables reside in a database. A database table corresponds to a data source, for example, data representing activity participation, data representing activity development, and data representing prize distribution. Database tables from different data sources can belong to one database. Each database table contains a plurality of field information (herein, it may also be referred to as a field for short), and each field represents an attribute of the database table. For example, a database table includes an unload time field, then the unload time field indicates the unload time of the database table.

2) Data unloading is a general term for data processing. For example, the data in the database table is extracted according to the rules, and the unloaded data is stored in another form. In the embodiment of the present invention, the data unloading is realized by importing the unloading script according to the unloading parameters to generate the unloading file. The uninstall script can be understood as a tool used to execute the uninstall process, and the uninstall parameter is the input parameter when using the uninstall script. Among them, the uninstall parameter is generated through the field information in the database table. That is to say, the process of data unloading can be understood as generating unloading parameters according to the field information in the database table, importing the unloading parameters into the unloading script, generating the unloading file, and running the unloading file to realize the unloading of the data.

3) Data unloading mode, including full unloading mode and incremental unloading mode. Full unloading refers to unloading all the data in the database table at one time. It can be understood that, regardless of whether the data volume of the database table increases or decreases, all data in the database table is unloaded during unloading. Incremental unloading refers to unloading data from database tables at preset time intervals. It is understandable that, in a period of time, some data may be entered or deleted in the database table. For example, in a period of time, new prize settings may be added, and the newly added prize data will be entered. Incremental unloading can process data within a preset time interval, for example, it can unload newly entered data within a preset time interval, which is more flexible.

4) Data push refers to pushing (sending) the data to the data analysis system to analyze the data through the data analysis system. For example, for different activities, data from one or more data sources may be pushed to the data analysis system, eg, data indicative of activity participation is sent to the data analysis system. The data analysis system analyzes these data to obtain the activity participation situation, so as to adjust the activity plan according to the current activity participation situation.

Statistical analysis is performed on the data according to the source of the data, usually by storing the data of the same data source in the form of a database table. An unload file for unloading database tables can be generated by a database tool, such as Oracle, and sent to the data analysis system. In this way, the data analysis system receives the unloading file, and reloads the data in the database table according to the unloading file. However, currently only one database table is supported for unloading (that is, processing, such as extracting partial data, etc.). In scenarios with multiple data sources, only one database table can be unloaded at a time, and the unloading efficiency is low. And since different data sources have different business execution dates, unloading different data sources uniformly may unload unnecessary data.

In view of this, the solution of the embodiment of the present invention is provided, in which the unloading time of each database table (data of different data sources) can be dynamically configured, so that the unloading time of data of different data sources may be the same, so that more than one data can be unloaded at one time. A database table to improve unloading efficiency. Moreover, since the unloading time of each database table is dynamically configured according to requirements, it is also avoided that the multiple database tables to be unloaded at the same time include database tables that do not need to be unloaded. In addition, after the database table is unloaded, the unloaded database table can also be pushed to the data analysis system, and the push time can also be dynamically configured, which is more flexible.

To facilitate understanding of the solutions provided by the embodiments of the present invention, the data unloading method provided by the embodiments of the present invention is described below with reference to the accompanying drawings.

Referring to FIG. 1 , it is a schematic flowchart of a data unloading method according to an embodiment of the present invention. The method may be performed by an electronic device with processing functions, such as a mobile phone, a tablet computer (PAD), a personal computer (PC), and the like. Of course, the type of the electronic device in the embodiment of the present invention is not limited to this. In the following description, the method is performed by an electronic device as an example.

S101, the electronic device generates a list of database tables to be uninstalled according to a plurality of database tables.

Database tables can represent data from a data source. Depending on the data source, the number and type of database tables vary. Taking a marketing campaign as an example, the data source may include data representing activity participation, data representing activity development, data representing prize distribution, and so on. The data of each data source exists in the form of database tables. Each database table may set one or more fields (also referred to as field information) to identify corresponding attribute information. For example, you can set an unload time field for a database table to indicate when the database table is unloaded. For another example, an unloading mode field can be set for a database table to indicate the unloading mode adopted for unloading the data table. Of course, the database table may also include more fields, which will not be repeated here.

According to actual needs, it may be necessary to analyze the data of some or all of the data sources. Following the above example, it may be necessary to analyze the data of the activity participation (ie, a certain database table) to obtain the actual activity participation, so as to facilitate subsequent adjustment of the activity strategy according to the actual activity participation. Before analyzing the data, the data can be unloaded, and then the unloaded data can be sent to the data analysis system. Unloading data means extracting data from database tables according to rules. For example, there are multiple database tables, and the electronic device can process the multiple database tables to generate a list of database tables to be unloaded.

The database table list may include the unloaded plurality of database tables, the database table list including a plurality of field information including the field information of the plurality of database tables, and more other fields. For example, the database table list further includes index fields of multiple database tables and multiple field information, wherein the multiple field information includes a database table name field, an uninstallation mode field, an incremental uninstallation mode field, an uninstallation time field, and the like. The uninstallation time field may include a last uninstallation time field, a next uninstallation time field, an uninstallation time interval field, and the like. The last unloading time field indicates the time when the database table was last unloaded, and the next unloading time field indicates the time when the database table is unloaded next, where the next unloading time of the database table is determined by the last unloading time of the database table and the unloading interval field The indicated time is determined. It should be noted that the fields included in the database table list above are only examples, and more fields may also be included, such as push information fields, which will be described below.

S102, the electronic device generates an uninstall parameter according to the field information in the database table list.

After generating the list of database tables to be uninstalled, the electronic device performs the uninstallation according to the field information of the database table list. The electronic device may acquire some or all of the field information in the database table list to generate unloading parameters, so as to unload one database table or multiple database tables in the database table list according to the unloading parameters.

For example, the electronic device may generate the uninstallation parameter representing the uninstallation mode according to the uninstallation mode information in the database table list. The offload mode information may indicate a mode of offload data. In this embodiment of the present invention, the mode of offloading data includes a full offloading mode and an incremental offloading mode. If the unloading mode is full unloading mode, all data from the start date of collecting data to the unloading date will be unloaded. It can also be understood that when the set unloading time is reached, all the data in the database table is unloaded. The unloading time of different database tables can be the same, so that the purpose of unloading multiple database tables at the same time can be realized. Compared with the full offload mode, the incremental offload mode refers to offloading the data added to the database table within a preset time interval. Different preset time intervals can be set for different database tables, so that the unloading time of each database table is independent of each other.

The preset time interval can be dynamically set according to the unloading requirements, and refers to the preset time interval from the last unloading time. For convenience of description, the current unloading time is recorded as the i-th unloading time, and the next unloading time is recorded as the i+1-th unloading time. The i+1 unloading time is the sum of the ith unloading time and the preset time interval. The field information included in the database table list may include unloading time information, for example, the database table list includes an i-th unloading time field, an i+1-th unloading time field, and a time interval field. The time interval field may indicate a preset time interval. In the incremental unloading mode, the electronic device can obtain unloading time information and generate unloading parameters. The unloading parameters include the ith unloading time and time interval. In addition, the electronic device may also determine the i+1 th unloading parameter according to the ith unloading time and the time interval, for example, the i+1 th unloading time.

It should be noted that, in the full offloading mode, since there is no relationship with the preset time interval, in the full offloading mode, the electronic device does not need to determine the offloading time according to the time interval field.

S103, the electronic device imports the determined uninstallation parameters into an uninstallation script to generate an uninstallation file.

After generating the uninstallation parameters according to the field information, the electronic device can import the uninstallation parameters of one or more database tables in the database table list to be uninstalled into the uninstallation script, and generate the uninstallation files corresponding to these database tables. The uninstall script can be a script edited by Shell or a script edited by JAVA.

Running the uninstall file can uninstall one or more database tables in the database table list according to the uninstall parameters. Take the database table category including database table A, database table B, and database table C, and the uninstallation parameter including the uninstallation time as an example. The ith unloading time of database table A is d1, and the preset time interval is t1; the ith unloading time of database table B is d2, and the preset time interval is t2; the ith unloading time of database table C is d3, The preset time interval is t3. Among them, d1, d2, and d3 can be the same, so that database table A, database table B, and database table C can be unloaded at one time. Compared with the current situation where only one database table can be unloaded at a time, the unloading efficiency is obviously improved. For the incremental unloading mode, even if d1, d2 and d3 are different from each other, d1+t1, d2+t2 and d3+t3 may be the same, so for the incremental unloading mode, it is also possible to unload multiple Database Table. In addition, t1, t2 and t3 can be the same or different, and users can set t1, t2 and t3 according to actual unloading requirements, which is more flexible. Even if the number of database tables is large, by dynamically configuring the i-th unloading time and the preset time interval, each database table can be unloaded independently and in an orderly manner.

The electronic device can run the uninstall file through the scheduling tool. In this embodiment of the present invention, multiple scheduling tools, such as crontab, springbatch, etc., may be set. For different application scenarios, you can choose to use matching scheduling tools, which have a wide range of applications. Specifically, taking crontab as an example, generally speaking, the running time of the crontab tool is in hours, and the running interval of the crontab tool can be called the scheduling interval. For any database table, the user can set the i-th uninstall time through crontab, and the scheduling interval of crontab can be set to be different from the preset time interval. For example, the user sets the i-th unloading time as d4 and the scheduling interval as 1 hour. Then the electronic device executes the crontab command, reaches d4, and unloads the corresponding database table. After the unloading is completed, the unloading time is updated to d5 according to the unloading time d4 and the preset time interval. In this way, the electronic device unloads and completes the data between the database tables d4 and d5. After that, at an interval of 1 hour, unload the database data within the preset time interval starting from d5. Following the above example, assuming that d1=d2=d4, then when d4 is reached, database table A and database table B are unloaded. Assuming d3=d4+3 days, then on d4+3, database table C is unloaded.

After the electronic device runs the uninstallation file, the i+1 th uninstall time can be updated to the i th uninstall time, and a new i+1 th uninstall time can be generated according to the i th uninstall time and the preset time interval. The preset time intervals corresponding to different database tables may be different. Following the above example, the user sets the number of uninstalls to 3 through crontab and the scheduling interval to one hour. Taking database table A as an example, the first unloading time is d1, and the preset time interval is 1 day. Before the first unloading, the i-th unloading time is d1, and the i+1-th unloading time is d1+1. When d1 is reached, the first unloading is performed. After the first unloading, the i-th unloading time is d1+1, and the i+1-th unloading time is d1+2; when d1+1 is reached, the second unloading is performed. After the second unloading, the i-th unloading time is d1+2, and the i+1-th unloading time is d1+3; when d1+2 is reached, the third unloading is performed. After the third unloading, the i-th unloading time is d1+3, and the i+1-th unloading time is d1+4.

It can be seen that the embodiment of the present invention can dynamically configure the unloading parameters of the unloading database table, such as the unloading time, the unloading interval time, etc., so that multiple database tables can be unloaded at one time. In addition, the number of unloading can be dynamically configured, and the database tables that are not actually unloaded during the unloading cycle can be continued to be unloaded, so as to avoid unloading the database tables.

After unloading the database table, the electronic device can also push the database table, that is, send the database table to the data analysis system, and analyze the data through the data analysis system. Usually, the database table pushed to the data analysis system is the database table after unloading. Therefore, before pushing the database table, the electronic device may determine that the unloading of the database table has been completed.

In a possible implementation manner, an uninstall flag is set in the database table list to indicate whether the corresponding database table is uninstalled. For example, the uninstall flag is "0", indicating that the uninstall is complete, and the uninstall flag is "1", indicating that the uninstall is not completed. It should be understood that, for any database table, the uninstallation flag can be reset by the electronic device after uninstalling the database table.

Before pushing the database table, the electronic device may obtain field information representing the uninstallation identifier in the database table, and then determine the database table for which the uninstallation is completed. It is assumed that the database after uninstallation is the first database table as an example. The user can obtain the push information of the first database table, and push the first database table according to the push information. The push information includes a storage path and a target file path. The storage path refers to the path to store the first database table, and the target file path refers to the path to which the first database table is to be pushed, for example, the path to store the first database table in the data analysis system. Wherein, the storage path may be an uninstallation file corresponding to the database table where the uninstallation is completed. Pushing the first database table by the electronic device can also be understood as sending the uninstall file corresponding to the stored first database table to the target file path.

The embodiments of the present invention provide a variety of modes for pushing database tables, which can also be understood as the modes for pushing database tables, which are referred to as push modes for short. The push mode, for example, includes the mode of pushing database tables by means of jfserver, ftp or httpclient. For the interface of the data analysis system, the electronic device can choose a matching push mode, which is applicable to a wider range. Take the predefined push modes including jfserver push mode, ftp push mode and httpclient push mode as an example. The push modes of different database table configurations can be the same or different. Before pushing the first database table, the electronic device may obtain the push mode identifier corresponding to the first database table, so as to push the first database table according to the push mode indicated by the push mode identifier. Taking the push mode identifier indicating the ftp push mode as an example, the electronic device obtains the uninstall file of the database table A according to the storage path, and then sends the uninstall file to the target file path in the ftp push mode.

Similar to the uninstall time, the user can also set the push time of the database table. The push time can be different for different database tables. The user can also set the push interval of the database table, indicating that the push is not performed immediately after the uninstallation is completed, but after the push interval. The push interval for different database tables can also be different. Similarly, users can also set the push times of database tables to avoid missing pushes of database tables as much as possible.

The following describes the device provided by the embodiments of the present invention with reference to the accompanying drawings.

Referring to FIG. 2 , based on the same inventive concept, an embodiment of the present invention provides a data unloading apparatus 200 , and the data unloading apparatus 200 can implement the process shown in FIG. 1 . The data unloading apparatus 200 includes a list generating module 201 , a parameter generating module 202 , and an unloading module 203 . The list generation module 201 is configured to generate a list of database tables to be uninstalled according to multiple database tables, where the database table list includes respective indexes of the multiple database tables and multiple field information, and the multiple field information includes all Field information corresponding to each database table in the multiple database tables is described. The parameter generating module 202 is configured to generate an unloading parameter according to the field information of the database table list, where the unloading parameter is used to unload the plurality of database tables. The uninstallation module 203 is configured to import the uninstallation parameters into an uninstallation script to generate an uninstallation file, and the uninstallation script is used to uninstall the multiple database tables.

Optionally, the parameter generation module 202 generates the uninstallation parameter according to the field information of the database table list, and is specifically configured to: acquire the uninstallation mode information, and generate the uninstallation parameter according to the uninstallation mode information.

Optionally, the parameter generation module 202 generates the unloading parameters according to the field information of the database table list, and is also used for: obtaining the unloading time information, and generating the unloading parameters according to the unloading time information, wherein the unloading parameters include the ith unloading time and the preset time. interval, the preset time interval and the i-th unloading time are used to determine the i+1-th unloading time.

Optionally, after the uninstallation module 203 imports the uninstallation parameters into the uninstallation script to generate the uninstallation script, it is also used to: update the i+1th uninstallation time to the ith uninstallation time, and then obtain the ith uninstallation time and the preset time. interval to generate a new i+1 unloading time. The preset time intervals corresponding to different database tables are different.

Optionally, the data unloading apparatus 200 is further configured to: obtain the unloading identifier corresponding to the first database table, and obtain push information if the unloading identifier indicates that the unloading of the first database table is completed. It should be noted that the push information includes the storage path of the first database table and the target file path. Then, the electronic device acquires the first database table from the storage path, and pushes the first database table to the target file path.

Optionally, the data unloading apparatus 200 is further configured to: acquire the push mode identifier, and push the first database table to the target file path according to the push mode indicated by the push mode identifier.

Optionally, the data unloading apparatus 200 is further configured to: receive the input unloading times, and unload the database table that reaches the unloading time according to the unloading times.

Referring to FIG. 3 , based on the same inventive concept, an embodiment of the present invention provides an electronic device 300 , and the electronic device 300 can implement the process shown in FIG. 1 . The electronic device 300 includes at least one processor 301 and a memory 302 communicatively connected to the at least one processor 301 , and the processor 301 implements any of the above-mentioned data offloading methods when executing a computer program.

Wherein, at least one processor 301 may be configured to generate a list of database tables to be uninstalled according to multiple database tables, where the database table list includes respective indexes of multiple database tables and multiple field information, and multiple field information includes The corresponding field information of each database table. And according to the field information of the database table list, unloading parameters are generated, the unloading parameters are used to unload multiple database tables; the unloading parameters are imported into the unloading script to generate the unloading file, and the unloading script is used to unload the multiple database tables.

The electronic device 300 can be used to execute the method provided by the embodiment shown in FIG. 1 , such as the aforementioned electronic device. Therefore, regarding the functions that can be implemented by each functional module in the electronic device 300, reference may be made to the corresponding descriptions in the embodiment shown in FIG. 1, and details are not repeated here.

An embodiment of the present invention further provides a computer storage medium, wherein the computer storage medium stores computer instructions, and when the computer instructions are executed on the computer, the computer is made to execute the method described in FIG. 1 .

An embodiment of the present invention also provides a computer program product, the computer program product includes: computer program code, when the computer program code is run on the computer, the computer is made to execute the method described in FIG. 1 .

Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims (14)

1. A method of data offloading, comprising:

generating a database table list to be unloaded according to a plurality of database tables, wherein the database table list comprises respective indexes and a plurality of field information of the plurality of database tables, and the plurality of field information comprises the field information corresponding to each database table in the plurality of database tables;

generating unloading parameters according to the field information of the database table list, wherein the unloading parameters are used for unloading the plurality of database tables;

and importing the unloading parameters into an unloading script to generate an unloading file, wherein the unloading script is used for unloading the plurality of database tables.

2. The method of claim 1, wherein the plurality of field information of the database table list includes unload mode information, the unload mode information including an unload mode, the unload mode including a full unload mode or an incremental unload mode, the full unload mode indicating to unload data in all of the database tables included in the database table list, the incremental unload mode indicating to unload data in the plurality of database tables added within a preset time interval; generating unloading parameters according to the field information of the database table list, wherein the unloading parameters comprise:

and acquiring the unloading mode information, and generating the unloading parameters according to the unloading mode information.

3. The method of claim 2 wherein the plurality of field information of the database table list further includes unload time information indicating an unload time of the database table in the incremental unload mode, generating unload parameters from the field information of the database table list including:

and acquiring the unloading time information, and generating the unloading parameters according to the unloading time information, wherein the unloading parameters comprise ith unloading time and a preset time interval, and the preset time interval and the ith unloading time are used for determining the (i + 1) th unloading time.

4. The method of claim 3, wherein after importing the uninstall parameters into an uninstall script to generate an uninstall file, the method further comprises:

updating the i +1 th unloading time to the i-th unloading time;

and acquiring the ith unloading time and a preset time interval, and generating new (i + 1) th unloading time, wherein the preset time intervals corresponding to different database tables are different.

5. The method of any one of claims 1-4, wherein the plurality of fields of the database table list include an unload flag, the unload flag to indicate whether the unloading of the database table is complete, the method further comprising:

acquiring the unloading identifier corresponding to a first database table, and acquiring push information if the unloading identifier indicates that the unloading of the first database table is finished, wherein the push information comprises a storage path and a target file path of the first database table;

and acquiring the first database table from the storage path, and pushing the first database table to the target file path.

6. The method of claim 5, wherein the push information further includes a push mode identification, the push mode identification indicating a push mode, pushing the first database table to the target file path, comprising:

acquiring the pushing mode identification;

and pushing the first database table to the target file path according to the pushing mode indicated by the pushing mode identifier.

7. The method of claim 3 or 4, further comprising:

receiving input uninstalling times;

and unloading the database table reaching the unloading time according to the unloading times.

8. A data offloading device, comprising:

the database table list comprises indexes and field information of the database tables, and the field information comprises the field information corresponding to each database table in the database tables;

the parameter generating module is used for generating unloading parameters according to the field information of the database table list, and the unloading parameters are used for unloading the plurality of database tables;

and the unloading module is used for importing the unloading parameters into an unloading script to generate an unloading file, and the unloading script is used for unloading the plurality of database tables.

9. The apparatus of claim 8, wherein the plurality of field information of the database table list includes offload mode information, the offload mode information including an offload mode, the offload mode including a full offload mode or an incremental offload mode, the full offload mode indicating to offload data in all database tables included in the database table list, the incremental offload mode indicating to offload data in the plurality of database tables that is increasing within a preset time interval, the parameter generation module specifically to:

and acquiring the unloading mode information, and generating the unloading parameters according to the unloading mode information.

10. The apparatus of claim 9, wherein the plurality of field information of the database table list further includes unload time information indicating an unload time of the database table in the incremental unload mode, the parameter generation module further to:

and acquiring the unloading time information, and generating the unloading parameters according to the unloading time information, wherein the unloading parameters comprise ith unloading time and a preset time interval, and the preset time interval and the ith unloading time are used for determining the (i + 1) th unloading time.

11. The apparatus of claim 10, wherein the offload module is specifically configured to:

updating the i +1 th unloading time to the i-th unloading time;

and acquiring the ith unloading time and a preset time interval, and generating new (i + 1) th unloading time, wherein the preset time intervals corresponding to different database tables are different.

12. An electronic device, comprising:

a memory for storing program instructions;

a processor for calling program instructions stored in said memory and for executing the method according to any one of claims 1-7 in accordance with the obtained program instructions.

13. A computer-readable storage medium, characterized in that it stores a computer program comprising program instructions which, when executed by a computer, cause the computer to carry out the steps of the method according to any one of claims 1 to 7.

14. A computer program product, the computer program product comprising: computer program code for causing a computer to carry out the steps of the method as claimed in any one of claims 1 to 7 when said computer program code is run on a computer.

Images