CN105512239A - Database data push method and device - Google Patents
Database data push method and device Download PDFInfo
- Publication number
- CN105512239A CN105512239A CN201510860527.6A CN201510860527A CN105512239A CN 105512239 A CN105512239 A CN 105512239A CN 201510860527 A CN201510860527 A CN 201510860527A CN 105512239 A CN105512239 A CN 105512239A
- Authority
- CN
- China
- Prior art keywords
- data
- pushing
- file
- database
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a database data push method and device; the method and device pre-creates a uniform and universal data push demand template, and the template integrates each attribute configuration item corresponding to different data push demands of each data sheet in the database; in subsequent database data pushing, template information configured by the user and aiming at each data sheet can be read and parsed so as to obtain the push demands of each data sheet, thus carrying out corresponding data pushing process for each data sheet, and realizing data filing. The method and device need not to do different developing works for each sheet in the database, thus reducing massive work load of the developing people in respective developing works.
Description
Technical field
The invention belongs to Database Systems application, particularly relate to a kind of database data method for pushing and device.
Background technology
At present, the mechanisms such as bank are after per next day, egress changed, the database data of each system need be collected corresponding data integration platform to file, particularly, such as, after per next day, egress changed, need will comprise contract table, the data-pushing of the financial database P8 of the multiple types of data such as interest table, table on exchange rates table is filed etc. to data integration platform P9.
The database of each system of bank often comprises many tables of data, as P8 can comprise the contract table of some simultaneously, multiple tables of data such as the interest table of some and the table on exchange rates of some, when to data integration platform propelling data, different tables of data often has different propelling movement demands, different several mode and file generated rule etc. is unloaded as used, still for P8, for unloading several mode, some table need send same day incremental data to P9, some table need send full dose data to P9, some table requires the order of the field of writing in files (pushing files is to P9 after showing data writing in files) and former literary name section sequence consensus, some table then requires that the order of the field of writing in files does certain adjustment, some table requires to adopt field A to screen, some table requires to adopt field B to screen etc., for file generated rule, some table requires to generate DAT file, and some table then requires not generate DAT file etc.
Based on this feature, prior art is when fulfillment database data are to the propelling movement of data integration platform, in advance for each table in database, realize different demand codes respectively, as realized different unloading several mode and file generated rule etc. respectively, namely prior art need carry out corresponding development respectively for each tables of data, thus finally for developer brings huge workload.
Summary of the invention
In view of this, the object of the present invention is to provide a kind of database data method for pushing and device, being intended to by providing a kind of unification, general data-pushing requirement templet, reducing developer in prior art and developing the huge workload of work for each tables of data respectively.
For this reason, the present invention's openly following technical scheme:
A kind of database data method for pushing, comprising:
Determine current pending target matrix in target database;
Obtain the data-pushing demand information of described target matrix; Wherein, described data-pushing demand information is in advance based on the information that data-pushing requirement templet configures; Described data-pushing requirement templet is integrated with in advance: each attribute configuration item corresponding to different pieces of information propelling movement demand of each tables of data in described target database;
Based on described data-pushing demand information, corresponding data-pushing process is carried out to described target matrix, to realize the corresponding data of described target matrix to push to target location.
Said method, preferably, describedly determine that in target database, current pending target matrix comprises:
Based on the mode of multi-thread concurrent process, determine current pending multiple target matrixs in target database.
Said method, preferably, the data-pushing demand information of the described target matrix of described acquisition comprises:
In corresponding thread, read the data-pushing demand information of each described target matrix.
Said method, preferably, described data-pushing demand information comprise unload several mode and file generated rule, described target location is default data integration platform, then described based on described data-pushing demand information, corresponding data-pushing process is carried out to described target matrix and comprises:
In corresponding thread, from each described target matrix, read out target data to be pushed based on corresponding several mode of unloading;
In corresponding thread, based on corresponding file generated rule, corresponding written document process is carried out to the target data of each described target matrix, generates the pushing files of each described target matrix;
After each tables of data for described target database generates corresponding pushing files, pushing files described in each is sent to described data integration platform.
Said method, preferably, described pushing files comprises data file, list structure file and control documents, then the described target data to each described target matrix is carried out corresponding written document process and comprised:
Based on database caches technology, the target data of each described target matrix is write in corresponding data file;
Data file for each described target matrix writes corresponding list structure file and control documents; Wherein, described list structure file comprises field name and the field type of each field included by the file name of described data file, described data file; Described control documents comprises the record number that the size of described data file, rise time and described data file comprise.
Said method, preferably, is describedly sent to described data integration platform by pushing files described in each and comprises:
Pushing files described in each is carried out divide into groups, pack, obtain the pushing files bag of corresponding number;
Based on the mode of multi-thread concurrent process, pushing files bag described in each is sent to described data integration platform and files.
Said method, preferably, also comprises:
With the beginning delivery time of described pushing files for initial time is waited for after preset duration, whether the process of transmitting detecting each pushing files completes, if completed, then terminates; If do not completed, then report an error.
A kind of database data pusher, comprising:
Determination module, for determining current pending target matrix in target database;
Acquisition module, for obtaining the data-pushing demand information of described target matrix; Wherein, described data-pushing demand information is in advance based on the information that data-pushing requirement templet configures, and described data-pushing requirement templet is integrated with: each attribute configuration item corresponding to different pieces of information propelling movement demand of each tables of data in described target database;
Push processing module, for based on described data-pushing demand information, corresponding data-pushing process is carried out to described target matrix, to realize the corresponding data of described target matrix to push to target location.
Said apparatus, preferably, described determination module comprises:
Determining unit, for the mode based on multi-thread concurrent process, determines current pending multiple target matrixs in target database.
Said apparatus, preferably, described acquisition module comprises:
First reading unit, in corresponding thread, reads the data-pushing demand information of each described target matrix.
Said apparatus, preferably, described data-pushing demand information comprise unload several mode and file generated rule, described target location is default data integration platform, then described propelling movement processing module comprises:
Second reading unit, in corresponding thread, reads out target data to be pushed based on corresponding several mode of unloading from each described target matrix;
Written document unit, in corresponding thread, based on corresponding file generated rule, carries out corresponding written document process to the target data of each described target matrix, generates the pushing files of each described target matrix;
File transmitting element, for after each tables of data for described target database generates corresponding pushing files, is sent to described data integration platform by pushing files described in each.
Said apparatus, preferably, described pushing files comprises data file, list structure file and control documents, then described written document unit comprises:
First written document subelement, for based on database caches technology, writes in corresponding data file by the target data of each described target matrix;
Second written document subelement, for writing corresponding list structure file and control documents for the data file of each described target matrix; Wherein, described list structure file comprises field name and the field type of each field included by the file name of described data file, described data file; Described control documents comprises the record number that the size of described data file, rise time and described data file comprise.
Said apparatus, preferably, described file transmitting element comprises:
Grouping packing subelement, divides into groups for being carried out by pushing files described in each, packs, obtain the pushing files bag of corresponding number;
Send subelement, for the mode based on multi-thread concurrent process, pushing files bag described in each is sent to described data integration platform and files.
Said apparatus, preferably, also comprises:
Detection module, for the beginning delivery time of described pushing files for initial time, wait for after preset duration, whether the propelling movement process detecting each pushing files completes, if completed, then terminates; If do not completed, then report an error.
From above scheme, database data method for pushing disclosed in the present application and device, be pre-created a kind of unification, general data-pushing requirement templet, this template is integrated with each attribute configuration item corresponding to the different pieces of information propelling movement demand of each tables of data in database; Follow-up when carrying out database data and pushing, by reading and resolving the Template Information that user configures for each tables of data, know the propelling movement demand of each tables of data, and then corresponding data-pushing process can be carried out to each tables of data on this basis, realize data filing.Visible, the application is without the need to carrying out different developments respectively for each table in database in application, decreases the huge workload that developer develops respectively.
Accompanying drawing explanation
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, be briefly described to the accompanying drawing used required in embodiment or description of the prior art below, apparently, accompanying drawing in the following describes is only embodiments of the invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to the accompanying drawing provided.
Fig. 1 is the database data method for pushing process flow diagram that the embodiment of the present invention one provides;
Fig. 2 is the processing flow chart unloading several stage that the embodiment of the present invention one provides;
Fig. 3 is the database data method for pushing process flow diagram that the embodiment of the present invention two provides;
Fig. 4-Fig. 5 is the structural representation of the database data pusher that the embodiment of the present invention three provides.
Embodiment
For the purpose of quoting and know, the technical term hereinafter used, to write a Chinese character in simplified form or summary of abridging is explained as follows:
Unload number: downloading data from database table is also write in file.
Below in conjunction with the accompanying drawing in the embodiment of the present invention, be clearly and completely described the technical scheme in the embodiment of the present invention, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, belong to the scope of protection of the invention.
Embodiment one
The embodiment of the present invention one discloses a kind of database data method for pushing, and with reference to figure 1, described method can comprise the following steps:
S101: determine current pending target matrix in target database.
S102: the data-pushing demand information obtaining described target matrix; Wherein said data-pushing demand information is in advance based on the information that data-pushing requirement templet configures; Described data-pushing requirement templet is integrated with in advance: each attribute configuration item corresponding to data-pushing demand of each tables of data in described target database.
S103: based on described data-pushing demand information, carries out corresponding data-pushing process to described target matrix, to realize the corresponding data of described target matrix to push to target location.
The application is by being pre-created a kind of unification, general data-pushing requirement templet, solve the different pieces of information propelling movement demand of prior art for tables of data each in database, the development amount caused because corresponding development need be carried out respectively this problem large.
Wherein, described data-pushing requirement templet is integrated with each attribute configuration item corresponding to the different pieces of information propelling movement demand of each tables of data in database, such as concrete accessible site table name, for garbled data field name, unload the multiple attribute configuration item such as several mode (or being called inquiry mode), file generated rule (or being called written document mode).In actual application environment, can adopt the form of configuration file that described data-pushing requirement templet is showed user, thus, user can according to the actual propelling movement demand of tables of data each in database, corresponding configuration is carried out to the value of each attribute configuration item, the configuration file needed for the generation of each tables of data can be embodied as on this basis.
When developing, realizing described data-pushing requirement templet, the scope of application of described template can be defined in advance according to actual demand, such as specifically can according to actual demand, described template is defined as the data-pushing demand configuration being only applicable to each tables of data in a certain database (as financial database P8), or is defined as the data-pushing demand configuration etc. being applicable to each tables of data in multiple disparate databases; On this basis, can determine according to the defined scope of application and integrated each required attribute configuration item, the data-pushing requirement templet needed for final formation.
Next, the present embodiment describes in detail the application's method for the financial database P8 of banking institution.First, the present embodiment provides a kind of data-pushing requirement templet example being applicable to each tables of data of P8, this template be specially one comprise table name, for garbled data field name, unload the configuration file of multiple attribute configuration item such as several mode and file generated rule, each attribute configuration item included by this configuration file and the value menu of correspondence thereof specifically can with reference to following tables 1.
Table 1
For often opening tables of data in P8, as different contract table, interest table, table on exchange rates etc., by being configured to the concrete value of upper each attribute item, for the configuration file needed for its generation, follow-up system by read, resolve configuration file, know often open tables of data unload several mode, file generated rule etc. data-pushing demand.
Often open in for database after tables of data configuration generates corresponding required configuration file, the propelling movement flow process of database data can be started.Particularly, with reference to figure 2, can first reading database treat propelling movement table inventory, afterwards for table inventory in often open tables of data, carry out follow-up data-pushing process according to its configuration file.The data-pushing process of the present embodiment is divided into unloads number and file sends two stages, and wherein, unloading several stage comprises reading database and write pushing files two links.
Unloading in several process, in order to raise the efficiency, the present embodiment adopts the mode of multi-thread concurrent process to unload number process to each tables of data, as shown in Figure 2, corresponding tool is adopted to generate n thread, each thread processes multiple task evenly, each task comprises reading (or inquiry) tables of data and the table data genaration pushing files based on reading, wherein, the corresponding corresponding one group of pushing files of each tables of data, in the present embodiment, often organize pushing files to comprise data file and (DAT file may be generated according to the actual requirements, also DAT file may not be generated, also DAT empty file etc. may be generated), list structure file and control documents.
Particularly, unload in the process of several task in each thread process of use, first each thread need determine current pending target matrix in database, read afterwards and resolve the configuration file of this tables of data, and generate a SQL (StructuredQueryLanguage based on the information such as several modes of unloading in analysis result, Structured Query Language (SQL)) query statement, from this tables of data, inquire about satisfactory data set according to this SQL query statement afterwards.
On this basis, continue the table data write data file will read according to the file generated rule in analysis result, several efficiency is unloaded for improving, the present embodiment specifically adopts database caches technology (memory cache), by the table data write data file read, thus realize the generation of data file, unload in several process and do not need to carry out data interaction with disk, thus performance is more excellent.Afterwards, continue to generate corresponding list structure file and control documents according to the data file generated.Wherein, list structure file comprises the information such as field name and field type of each field included by the file name of described data file, described data file; Described control documents comprises the record number etc. that the size of described data file, rise time and described data file comprise.Follow-up, the common pushing files as tables of data is sent to data integration platform and files by data file, list structure file and control documents.
After each tables of data all completes and unloads the corresponding pushing files of number generation in a database, can continue to carry out transmission processing to each pushing files, realize filing.
Particularly, because each number of files sent of current file transfer suggestion does not surpass 20, therefore, in the present embodiment, under default setting the pushing files (10-15 file) of every 5 tables of data form a list group carry out packing send, in practical application, the number of files that can comprise according to actual demand configuration modification list group, each pushing files is being divided into groups, on the basis of packing, can by packaging file and corresponding node number, file type, the reference informations such as transport-type are supplied to respective transmissions interface simultaneously, as NFT (NextFileTransportion, file transfer component of future generation) asynchronous transmission interface, realize data-pushing.
Wherein, described node number is used for target-marking data integration platform, described file type should match by integrated platform with adopted target data, for data integration platform P9, described file type need be set to the discernible types value of P9, as DATA2P9 etc., when P9 receives the file of this type, automatically can trigger corresponding filing process to file file, described transport-type comprises compression transmission and does not do compression transmission etc.
At transmission document stage, for raising the efficiency, the mode of multi-thread concurrent process can be adopted equally, realizing each pushing files to the transmission of the integrated platform of target data and transmitting.
From above scheme, database data method for pushing disclosed in the present application, has been pre-created a kind of unification, general data-pushing requirement templet, and this template is integrated with each attribute configuration item corresponding to the different pieces of information propelling movement demand of each tables of data in database; Follow-up when carrying out database data and pushing, by reading and resolving the Template Information that user configures for each tables of data, know the propelling movement demand of each tables of data, and then corresponding data-pushing process can be carried out to each tables of data on this basis, realize data filing.Visible, the application is without the need to carrying out different developments respectively for each table in database in application, decreases the huge workload that developer develops respectively.
Embodiment two
In the present embodiment, with reference to figure 3, described database data method for pushing can also comprise the following steps:
S104: with the beginning delivery time of described pushing files for initial time is waited for after preset duration, whether the process of transmitting detecting each pushing files completes, if completed, then terminates; If do not completed, then report an error.
The present embodiment pushes process for database data, there is provided corresponding and detect the function that reports an error, particularly, all tables of data complete and unload the corresponding pushing files of number generation in a database, and after startup file transfer process, start timing to wait for, and after wait preset duration, whether the process of transmitting detecting each pushing files completes, if completed, the database then terminating this pushes process, if do not completed, then characterize and push extremely, report an error, thus user can be facilitated to understand the implementation status of propelling movement task in time, and then can when pushing abnormal, carry out corresponding debugging process in time.
Embodiment three
The present embodiment three discloses a kind of database data pusher, and described device is corresponding with database data method for pushing disclosed in above each embodiment.
Corresponding to embodiment one, with reference to figure 4, described database data pusher comprises determination module 100, acquisition module 200 and pushes processing module 300.
Determination module 100, for determining current pending target matrix in target database.
Described determination module comprises determining unit, for the mode based on multi-thread concurrent process, determines current pending multiple target matrixs in target database.
Acquisition module 200, for obtaining the data-pushing demand information of described target matrix; Wherein, described data-pushing demand information is in advance based on the information that data-pushing requirement templet configures, and described data-pushing requirement templet is integrated with: each attribute configuration item corresponding to different pieces of information propelling movement demand of each tables of data in described target database.
Described acquisition module 200 comprises the first reading unit, in corresponding thread, reads the data-pushing demand information of each described target matrix.
Push processing module 300, for based on described data-pushing demand information, corresponding data-pushing process is carried out to described target matrix, to realize the corresponding data of described target matrix to push to target location.
Described propelling movement processing module 300 comprises the second reading unit, written document unit and file transmitting element.
Second reading unit, in corresponding thread, reads out target data to be pushed based on corresponding several mode of unloading from each described target matrix;
Written document unit, in corresponding thread, carries out corresponding written document process based on corresponding file generated rule to the target data of each described target matrix, generates the pushing files of each described target matrix;
File transmitting element, for after each tables of data for described target database generates corresponding pushing files, is sent to described data integration platform by pushing files described in each.
Wherein, described written document unit comprises the first written document subelement and the second written document subelement.First written document subelement, for based on database caches technology, writes in corresponding data file by the target data of each described target matrix; Second written document subelement, for writing corresponding list structure file and control documents for the data file of each described target matrix; Wherein, described list structure file comprises field name and the field type of each field included by the file name of described data file, described data file; Described control documents comprises the record number that the size of described data file, rise time and described data file comprise
Described file transmitting element comprises grouping packing subelement and sends subelement.Grouping packing subelement, divides into groups for being carried out by pushing files described in each, packs, obtain the pushing files bag of corresponding number; Send subelement, for the mode based on multi-thread concurrent process, pushing files bag described in each is sent to described data integration platform and files.
Corresponding to embodiment two, with reference to figure 5, described database data pusher can also comprise detection module 400, for with the beginning delivery time of described pushing files for initial time, wait for after preset duration, whether the propelling movement process detecting each pushing files completes, if completed, then terminate; If do not completed, then report an error.
For database data pusher disclosed in the embodiment of the present invention three, because it is corresponding with database data method for pushing disclosed in embodiment one to embodiment two, so description is fairly simple, relevant similarity refers to the explanation of database data pusher method part in embodiment one to embodiment two, no longer describes in detail herein.
It should be noted that, each embodiment in this instructions all adopts the mode of going forward one by one to describe, and what each embodiment stressed is the difference with other embodiments, between each embodiment identical similar part mutually see.
For convenience of description, various module or unit is divided into describe respectively with function when describing above system or device.Certainly, the function of each unit can be realized in same or multiple software and/or hardware when implementing the application.
As seen through the above description of the embodiments, those skilled in the art can be well understood to the mode that the application can add required general hardware platform by software and realizes.Based on such understanding, the technical scheme of the application can embody with the form of software product the part that prior art contributes in essence in other words, this computer software product can be stored in storage medium, as ROM/RAM, magnetic disc, CD etc., comprising some instructions in order to make a computer equipment (can be personal computer, server, or the network equipment etc.) perform the method described in some part of each embodiment of the application or embodiment.
Finally, also it should be noted that, in this article, the relational terms of such as first, second, third and fourth etc. and so on is only used for an entity or operation to separate with another entity or operational zone, and not necessarily requires or imply the relation that there is any this reality between these entities or operation or sequentially.And, term " comprises ", " comprising " or its any other variant are intended to contain comprising of nonexcludability, thus make to comprise the process of a series of key element, method, article or equipment and not only comprise those key elements, but also comprise other key elements clearly do not listed, or also comprise by the intrinsic key element of this process, method, article or equipment.When not more restrictions, the key element limited by statement " comprising ... ", and be not precluded within process, method, article or the equipment comprising described key element and also there is other identical element.
The above is only the preferred embodiment of the present invention; it should be pointed out that for those skilled in the art, under the premise without departing from the principles of the invention; can also make some improvements and modifications, these improvements and modifications also should be considered as protection scope of the present invention.
Claims (14)
1. a database data method for pushing, is characterized in that, comprising:
Determine current pending target matrix in target database;
Obtain the data-pushing demand information of described target matrix; Wherein, described data-pushing demand information is in advance based on the information that data-pushing requirement templet configures; Described data-pushing requirement templet is integrated with in advance: each attribute configuration item corresponding to different pieces of information propelling movement demand of each tables of data in described target database;
Based on described data-pushing demand information, corresponding data-pushing process is carried out to described target matrix, to realize the corresponding data of described target matrix to push to target location.
2. method according to claim 1, is characterized in that, describedly determines that in target database, current pending target matrix comprises:
Based on the mode of multi-thread concurrent process, determine current pending multiple target matrixs in target database.
3. method according to claim 2, is characterized in that, the data-pushing demand information of the described target matrix of described acquisition comprises:
In corresponding thread, read the data-pushing demand information of each described target matrix.
4. method according to claim 3, it is characterized in that, described data-pushing demand information comprise unload several mode and file generated rule, described target location is default data integration platform, then described based on described data-pushing demand information, corresponding data-pushing process is carried out to described target matrix and comprises:
In corresponding thread, from each described target matrix, read out target data to be pushed based on corresponding several mode of unloading;
In corresponding thread, based on corresponding file generated rule, corresponding written document process is carried out to the target data of each described target matrix, generates the pushing files of each described target matrix;
After each tables of data for described target database generates corresponding pushing files, pushing files described in each is sent to described data integration platform.
5. method according to claim 4, is characterized in that, described pushing files comprises data file, list structure file and control documents, then the described target data to each described target matrix is carried out corresponding written document process and comprised:
Based on database caches technology, the target data of each described target matrix is write in corresponding data file;
Data file for each described target matrix writes corresponding list structure file and control documents; Wherein, described list structure file comprises field name and the field type of each field included by the file name of described data file, described data file; Described control documents comprises the record number that the size of described data file, rise time and described data file comprise.
6. method according to claim 4, is characterized in that, describedly pushing files described in each is sent to described data integration platform comprises:
Pushing files described in each is carried out divide into groups, pack, obtain the pushing files bag of corresponding number;
Based on the mode of multi-thread concurrent process, pushing files bag described in each is sent to described data integration platform and files.
7. the method according to claim 4-6 any one, is characterized in that, also comprises:
With the beginning delivery time of described pushing files for initial time is waited for after preset duration, whether the process of transmitting detecting each pushing files completes, if completed, then terminates; If do not completed, then report an error.
8. a database data pusher, is characterized in that, comprising:
Determination module, for determining current pending target matrix in target database;
Acquisition module, for obtaining the data-pushing demand information of described target matrix; Wherein, described data-pushing demand information is in advance based on the information that data-pushing requirement templet configures, and described data-pushing requirement templet is integrated with: each attribute configuration item corresponding to different pieces of information propelling movement demand of each tables of data in described target database;
Push processing module, for based on described data-pushing demand information, corresponding data-pushing process is carried out to described target matrix, to realize the corresponding data of described target matrix to push to target location.
9. device according to claim 8, is characterized in that, described determination module comprises:
Determining unit, for the mode based on multi-thread concurrent process, determines current pending multiple target matrixs in target database.
10. device according to claim 9, is characterized in that, described acquisition module comprises:
First reading unit, in corresponding thread, reads the data-pushing demand information of each described target matrix.
11. devices according to claim 10, is characterized in that, described data-pushing demand information comprise unload several mode and file generated rule, described target location is default data integration platform, then described propelling movement processing module comprises:
Second reading unit, in corresponding thread, reads out target data to be pushed based on corresponding several mode of unloading from each described target matrix;
Written document unit, in corresponding thread, based on corresponding file generated rule, carries out corresponding written document process to the target data of each described target matrix, generates the pushing files of each described target matrix;
File transmitting element, for after each tables of data for described target database generates corresponding pushing files, is sent to described data integration platform by pushing files described in each.
12. devices according to claim 11, is characterized in that, described pushing files comprises data file, list structure file and control documents, then described written document unit comprises:
First written document subelement, for based on database caches technology, writes in corresponding data file by the target data of each described target matrix;
Second written document subelement, for writing corresponding list structure file and control documents for the data file of each described target matrix; Wherein, described list structure file comprises field name and the field type of each field included by the file name of described data file, described data file; Described control documents comprises the record number that the size of described data file, rise time and described data file comprise.
13. devices according to claim 11, is characterized in that, described file transmitting element comprises:
Grouping packing subelement, divides into groups for being carried out by pushing files described in each, packs, obtain the pushing files bag of corresponding number;
Send subelement, for the mode based on multi-thread concurrent process, pushing files bag described in each is sent to described data integration platform and files.
14. devices according to claim 11-13 any one, is characterized in that, also comprise:
Detection module, for the beginning delivery time of described pushing files for initial time, wait for after preset duration, whether the propelling movement process detecting each pushing files completes, if completed, then terminates; If do not completed, then report an error.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510860527.6A CN105512239A (en) | 2015-11-30 | 2015-11-30 | Database data push method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510860527.6A CN105512239A (en) | 2015-11-30 | 2015-11-30 | Database data push method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105512239A true CN105512239A (en) | 2016-04-20 |
Family
ID=55720221
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510860527.6A Pending CN105512239A (en) | 2015-11-30 | 2015-11-30 | Database data push method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105512239A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108170462A (en) * | 2017-12-26 | 2018-06-15 | 深圳豪客互联网有限公司 | A kind of function recommendation tables group maintenance method and system |
| CN108628909A (en) * | 2017-03-24 | 2018-10-09 | 北京京东尚科信息技术有限公司 | Information-pushing method and device |
| CN109542408A (en) * | 2018-11-27 | 2019-03-29 | 深圳市关运通科技有限公司 | A kind of template intelligent configuration method and device |
| CN109617941A (en) * | 2018-10-22 | 2019-04-12 | 中国平安财产保险股份有限公司 | Data push method, device, computer equipment and computer readable storage medium |
| CN111666745A (en) * | 2020-06-03 | 2020-09-15 | 中国建设银行股份有限公司 | File downloading method, device, server and medium |
| CN115391445A (en) * | 2022-08-30 | 2022-11-25 | 中国银行股份有限公司 | A data push method and related device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050228728A1 (en) * | 2004-04-13 | 2005-10-13 | Microsoft Corporation | Extraction, transformation and loading designer module of a computerized financial system |
| CN101364891A (en) * | 2007-08-10 | 2009-02-11 | 中兴通讯股份有限公司 | System for collecting performance data by single point in distributed telecommunication network management and implementing method |
| CN102004744A (en) * | 2009-09-02 | 2011-04-06 | 中国银联股份有限公司 | Data extraction system and method from one source table to table of at least one object database |
| CN102938793A (en) * | 2012-11-29 | 2013-02-20 | 深圳中兴网信科技有限公司 | Information push device and information push method |
| CN103186639A (en) * | 2011-12-31 | 2013-07-03 | 腾讯科技（北京）有限公司 | Data generation method and system |
| CN103368902A (en) * | 2012-03-27 | 2013-10-23 | 湖南亲安网络科技有限公司 | Data interaction method |
-
2015
- 2015-11-30 CN CN201510860527.6A patent/CN105512239A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050228728A1 (en) * | 2004-04-13 | 2005-10-13 | Microsoft Corporation | Extraction, transformation and loading designer module of a computerized financial system |
| CN101364891A (en) * | 2007-08-10 | 2009-02-11 | 中兴通讯股份有限公司 | System for collecting performance data by single point in distributed telecommunication network management and implementing method |
| CN102004744A (en) * | 2009-09-02 | 2011-04-06 | 中国银联股份有限公司 | Data extraction system and method from one source table to table of at least one object database |
| CN103186639A (en) * | 2011-12-31 | 2013-07-03 | 腾讯科技（北京）有限公司 | Data generation method and system |
| CN103368902A (en) * | 2012-03-27 | 2013-10-23 | 湖南亲安网络科技有限公司 | Data interaction method |
| CN102938793A (en) * | 2012-11-29 | 2013-02-20 | 深圳中兴网信科技有限公司 | Information push device and information push method |
Non-Patent Citations (2)
| Title |
|---|
| 刘莉等: "《大学计算机基础教程》", 31 August 2015 * |
| 李和平: "《《企业档案工作规范》实施指南》", 30 April 2010 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108628909A (en) * | 2017-03-24 | 2018-10-09 | 北京京东尚科信息技术有限公司 | Information-pushing method and device |
| CN108628909B (en) * | 2017-03-24 | 2022-04-12 | 北京京东尚科信息技术有限公司 | Information pushing method and device |
| CN108170462A (en) * | 2017-12-26 | 2018-06-15 | 深圳豪客互联网有限公司 | A kind of function recommendation tables group maintenance method and system |
| CN108170462B (en) * | 2017-12-26 | 2020-09-22 | 深圳豪客互联网有限公司 | Function recommendation table group maintenance method and system |
| CN109617941A (en) * | 2018-10-22 | 2019-04-12 | 中国平安财产保险股份有限公司 | Data push method, device, computer equipment and computer readable storage medium |
| CN109542408A (en) * | 2018-11-27 | 2019-03-29 | 深圳市关运通科技有限公司 | A kind of template intelligent configuration method and device |
| CN109542408B (en) * | 2018-11-27 | 2022-04-26 | 深圳市关运通科技有限公司 | Intelligent template configuration method and device |
| CN111666745A (en) * | 2020-06-03 | 2020-09-15 | 中国建设银行股份有限公司 | File downloading method, device, server and medium |
| CN115391445A (en) * | 2022-08-30 | 2022-11-25 | 中国银行股份有限公司 | A data push method and related device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105512239A (en) | Database data push method and device | |
| US7913052B2 (en) | Method and apparatus for reducing the amount of data in a storage system | |
| US11586359B1 (en) | Tracking storage consumption in a storage array | |
| CN101777017B (en) | Quick recovery method of continuous data protection system | |
| CN102200972A (en) | Database maintenance method and database maintenance equipment | |
| CN112131214B (en) | Data writing, data query method, system, device and storage medium | |
| CN103440301A (en) | Data multi-duplicate hybrid storage method and system | |
| CN103020227A (en) | Data processing method and system in computer equipment | |
| CN104750744B (en) | A kind of compressed data synchronous method and device based on oracle database | |
| US20160019247A1 (en) | Building a metadata index from source metadata records when creating a target volume for subsequent metadata access from the target volume | |
| CN110019169B (en) | A method and device for data processing | |
| CN118860751A (en) | Data backup and recovery method and device based on anomaly detection | |
| WO2014144033A1 (en) | Multiple schema repository and modular data procedures | |
| CN102467528A (en) | deduplication operating system | |
| CN117171129A (en) | Data processing method, device, electronic equipment and computer readable storage medium | |
| US10242025B2 (en) | Efficient differential techniques for metafiles | |
| CN113434613A (en) | Associated data block processing method and device | |
| CN115552391B (en) | Zero-copy optimization of Select queries | |
| WO2015105493A1 (en) | Support data deduplication | |
| Kersten et al. | On the symbiosis of a data mining environment and a DBMS | |
| Kimball | Data wrangling | |
| US20180373726A1 (en) | File and Move Data to Shrink Datafiles | |
| CN119046398A (en) | Method, system, equipment and storage medium for realizing text entering into HBase library in configuration mode | |
| CN115617570A (en) | Backup method and device of incremental data, electronic equipment and storage medium | |
| CN118118030A (en) | Data compression method, data query method, device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160420 |