CN112783887A - Data processing method and device based on data warehouse - Google Patents
Data processing method and device based on data warehouse Download PDFInfo
- Publication number
- CN112783887A CN112783887A CN201911083470.8A CN201911083470A CN112783887A CN 112783887 A CN112783887 A CN 112783887A CN 201911083470 A CN201911083470 A CN 201911083470A CN 112783887 A CN112783887 A CN 112783887A
- Authority
- CN
- China
- Prior art keywords
- data
- data table
- virtual
- layer
- warehouse
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data processing method and device based on a data warehouse, and relates to the technical field of computers. One embodiment of the method comprises: obtaining a data request, wherein the data request indicates a first data table to be accessed and stored in an application layer of the data warehouse; determining a plurality of second data tables which are related to the first data table and store second data from a base layer of the data warehouse according to the hierarchical incidence relation of the data warehouse; the hierarchical incidence relation indicates the corresponding relation and the calculation relation between the data table in the data warehouse and the data table of the non-same layer; according to the calculation relationship indicated by the hierarchical incidence relationship and the second data respectively stored in the plurality of second data tables, calculating first data used for responding to the data request in the first data table. This embodiment reduces the storage resources consumed by the data warehouse.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a data processing method and device based on a data warehouse.
Background
Data warehouses have evolved over decades, essentially forming the following hierarchical architecture: a base layer, an integration layer, a summary layer and an application layer; wherein, the base layer is used for storing long-term base data; the integration layer integrates the basic data and forms integrated data in a wide table form; the summarizing layer summarizes and processes the summarized data to reduce the data volume; the application layer processes the summarized data according to specific application requirements, and the processed data can be directly displayed and used by a front-end report tool or pushed to other systems to be used as related data support.
In a conventional data warehouse, data of each level is stored in the data warehouse, for example, a base layer stores 100T order type basic data, an integration layer integrates the 100T basic data to form 80T integrated data, and a summary layer integrates and summarizes the integrated data to form about 20T summary data, so that the original 100T basic data occupies about 200T storage resources, and thus the data warehouse consumes a large amount of storage resources.
Disclosure of Invention
In view of this, embodiments of the present invention provide a data processing method based on a data warehouse, which can reduce storage resources consumed by the data warehouse.
To achieve the above object, according to an aspect of an embodiment of the present invention, a data processing method based on a data warehouse is provided.
The data processing method based on the data warehouse comprises the following steps:
obtaining a data request, wherein the data request indicates a first data table to be accessed and stored in an application layer of the data warehouse;
determining a plurality of second data tables which are related to the first data table and store second data from a base layer of the data warehouse according to the hierarchical incidence relation of the data warehouse; the hierarchical incidence relation indicates the corresponding relation and the calculation relation between the data table in the data warehouse and the data table of the non-same layer;
according to the calculation relationship indicated by the hierarchical incidence relationship and the second data respectively stored in the plurality of second data tables, calculating first data used for responding to the data request in the first data table.
Alternatively,
the data warehouse further comprises a virtual entity layer storing at least one virtual data table; the hierarchical association relationship indicates a corresponding relationship and a calculation relationship between the at least one virtual data table and the first data table and the second data table respectively;
the determining, from the base layer of the data warehouse, a second data table that is related to the first data table and stores second data, comprising:
according to the corresponding relation between the at least one virtual data table and the first data table and the second data table respectively, determining a plurality of virtual data tables related to the first data table from a virtual physical layer of the data warehouse, and determining a plurality of second data tables corresponding to the plurality of virtual data tables respectively from the base layer.
Alternatively,
the calculating first data in the first data table for responding to the data request comprises:
respectively calculating third data corresponding to each virtual data table according to the second data stored in the plurality of second data tables and the calculation relationship between the virtual data table and the second data table;
and calculating the first data according to the third data and the calculation relationship between the virtual data table and the first data table.
Optionally, the method further comprises:
and correspondingly storing the first data and the first data table in the application layer.
Alternatively,
after the calculating the first data in the first data table for responding to the data request, further comprising: and deleting the third data.
Optionally, the virtual entity layer comprises: a virtual integration layer and/or a virtual summary layer.
To achieve the above object, according to still another aspect of the embodiments of the present invention, there is provided a data processing apparatus based on a data warehouse.
The data processing device based on the data warehouse in the embodiment of the invention comprises: the device comprises a request acquisition module, a determination module and a calculation module; wherein,
the request acquisition module is used for acquiring a data request which indicates a first data table to be accessed and stored in an application layer of the database;
the determining module is used for determining a plurality of second data tables which are related to the first data table and store second data from a base layer of the data warehouse according to the hierarchical incidence relation of the data warehouse; the hierarchical incidence relation indicates the corresponding relation and the calculation relation between the data table in the data warehouse and the data table of the non-same layer;
the calculation module is configured to calculate, according to the calculation relationship indicated by the hierarchical association relationship and the second data stored in the plurality of second data tables, first data used for responding to the data request in the first data table.
Alternatively,
the data warehouse further comprises a virtual entity layer storing at least one virtual data table, wherein the hierarchical association relationship indicates a corresponding relationship and a calculation relationship between the at least one virtual data table and the first data table and the second data table respectively;
the determining module is configured to determine, from a virtual entity layer of the data warehouse, a plurality of virtual data tables related to the first data table according to correspondence between the at least one virtual data table and the first data table and the second data table, and determine, from the base layer, the plurality of second data tables corresponding to the plurality of virtual data tables, respectively.
Alternatively,
the calculation module is configured to calculate third data corresponding to each of the virtual data tables according to the second data stored in the plurality of second data tables and a calculation relationship between the virtual data table and the second data table; and calculating the first data according to the third data and the calculation relationship between the virtual data table and the first data table.
Optionally, the apparatus further comprises: a storage module; wherein,
the storage module is further configured to correspondingly store the first data and the first data table in the application layer.
Optionally, the apparatus further comprises: a deletion module; wherein,
and the deleting module is used for deleting the third data.
To achieve the above object, according to still another aspect of the embodiments of the present invention, there is provided an electronic device for data processing.
An electronic device for data processing according to an embodiment of the present invention includes: one or more processors; a storage device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement a data warehouse-based data processing method according to an embodiment of the present invention.
To achieve the above object, according to still another aspect of embodiments of the present invention, there is provided a computer-readable storage medium.
A computer-readable storage medium of an embodiment of the present invention stores thereon a computer program that, when executed by a processor, implements a data warehouse-based data processing method of an embodiment of the present invention.
One embodiment of the above invention has the following advantages or benefits: when the data request is acquired, a second data table related to the first data table of the application layer indicated by the data request can be determined from the base layer of the data warehouse according to the hierarchical association relationship stored in the data warehouse, and then the first data used for responding to the data request in the first data table is calculated according to the calculation relationship indicated by the hierarchical association relationship and the second data in the second data table. Therefore, in the data warehouse, the stored hierarchical incidence relation replaces the data stored in the integration layer and the summary layer, and when a data request is received, the data used for responding to the data request in the application layer is calculated according to the hierarchical incidence relation and the data stored in the base layer, namely, the integration layer and the summary layer of the data warehouse do not store the integration data and the summary data corresponding to the base data any more, so that the storage resources consumed by the data warehouse are reduced. And the base layer and the application layer in the data warehouse store corresponding data, so that the base layer is used as a data root of the data warehouse, and data is reserved for a long time to facilitate data backtracking.
Further, for the integration layer and/or the summarization layer, the virtual entity layer in the form of a storage virtual data table stores a correspondence relationship and a calculation relationship between the virtual data table and a first data table of the application layer and a second data table of the base layer, and when calculating first data in the first data table for responding to the data request, the virtual data table corresponding to the second data table may be determined from the virtual entity layer first according to the correspondence relationship and the calculation relationship between the virtual data table in the virtual entity layer and the first data table and second data table, and second data corresponding to the second data table may be calculated according to the second data in the second data table, and then the first data for responding to the data request may be calculated according to the correspondence relationship and the calculation relationship between the virtual data table and the first data table and the calculated second data. Therefore, the data architecture of the integration layer and the summary layer in the original data warehouse and the corresponding relation and the calculation relation between the virtual data table and the data tables in the application layer and the basic layer are reserved in the form of the virtual entity layer, and when data calculation is carried out, calculation can be carried out layer by layer according to the stored corresponding relation and the calculation relation, so that the efficiency and the accuracy of the data calculation are improved.
And after the first data is generated, the second data is deleted from the virtual data layer, so that the virtual data layer does not store entity data, and a large amount of storage resources can be saved.
In addition, after the first data for responding to the data request is calculated, the first data and the first data table can be correspondingly stored in the application layer, so that when a service request for the first data is received, the first data table in which the first data is stored can be directly output from the application layer, thereby improving the data use efficiency and responding to the service requirement more effectively and more quickly.
Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of a hierarchical architecture of a data warehouse in the prior art;
FIG. 2 is a schematic diagram of the main steps of a data warehouse-based data processing method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a hierarchical architecture of a data warehouse, according to an embodiment of the invention;
FIG. 4 is a diagram of a hierarchical association relationship according to an embodiment of the invention;
FIG. 5 is a schematic diagram of the main steps of another data warehouse-based data processing method according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of the major modules of a data warehouse-based data processing apparatus, according to an embodiment of the present invention;
FIG. 7 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;
fig. 8 is a schematic structural diagram of a computer system suitable for implementing a terminal device or a server according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
It should be noted that the embodiments of the present invention and the technical features of the embodiments may be combined with each other without conflict.
Data warehouses have evolved over decades, essentially forming the following hierarchical architecture: a base layer, an integration layer, a summary layer and an application layer; wherein, the base layer is used for storing long-term base data; the integration layer integrates the basic data and forms integrated data in a wide table form; the summarizing layer summarizes and processes the summarized data to reduce the data volume; the application layer processes the summarized data according to specific application requirements, and the processed data can be directly displayed and used by a front-end report tool or pushed to other systems to be used as related data support.
For example, the hierarchical structure of the data warehouse corresponding to the service platform a and the type of data stored in each hierarchy are shown in table 1:
TABLE 1
As can be seen from table 1, the data link of a conventional data warehouse may be as shown in fig. 1, which sequentially constructs data through a base layer, an integration layer, a summary layer and an application layer, and retains a large amount of entity data, including historical data, at each level. Due to the fact that data architecture and levels are complex and data growth is rapid, data warehouse can store processing data of each level in the ground after data are collected and stored at each level, a large amount of redundant storage is occupied by each data level as time passes, and the bottleneck of data storage becomes a barrier of enterprise-level data development of many companies. For example, the base layer stores 100T order type base data, the integration layer integrates the 100T base data to form 80T integrated data, in general, in order to improve the usability of the data, the dimension data is appropriately redundant and occupies about 25% of the redundant data, the integration layer occupies about 100T space, and the summary layer integrates and summarizes the 100T integrated data to form about 20T summary data, so that the original 100T space is enlarged by 2.2 times and stored to 220T, and thus the data warehouse consumes a large amount of storage resources.
With the continuous development of big data technology, various types (cache-based or memory-based) of computing frameworks are derived, and data computing is no longer a bottleneck in big data and data warehouses. On the contrary, because long-term historical data of each level is reserved in a big data and data warehouse system, and the situation that storage resources are insufficient frequently occurs, the embodiment of the invention provides a data processing method based on a data warehouse, only second data of a base layer fdm and first data of an application layer data app are reserved in the base layer (fdm), an integration layer (gdm), a summary layer (adm) and the application layer (app) based on the data warehouse stored by virtual data, and data of the integration layer gdm and the summary layer adm are replaced by a virtual entity in combination with a level association relationship and are not actually reserved on the ground, so that the storage resources are greatly saved, the storage resource consumption of the data warehouse is reduced, and the construction of the data warehouse of an enterprise is facilitated to save resources.
Fig. 2 is a schematic diagram of main steps of a data warehouse-based data processing method according to an embodiment of the present invention.
As shown in fig. 2, a data processing method based on a data warehouse according to an embodiment of the present invention mainly includes the following steps:
step S201: obtaining a data request indicating a first data table stored at an application layer of the data warehouse to be accessed.
The data request may indicate a first data table in the application layer, for example, the first data table indicated by the data request select from app. app1 is app 1. The data request can be initiated for a terminal, or initiated according to a set time, for example, the set time can be night or time with less traffic of other service platforms, so that the data processing method based on the data warehouse provided by the embodiment of the invention can be performed at night or when the traffic of other service platforms is less, to form first data for responding to the service requirement in the application layer in advance, and to respond to the service requirement more quickly and effectively when the service requirement is received.
Step S202: determining a plurality of second data tables which are related to the first data table and store second data from a base layer of the data warehouse according to the hierarchical incidence relation of the data warehouse; the hierarchical association relationship indicates a correspondence relationship and a calculation relationship between the data table in the data warehouse and the data table of the non-layer.
In order to improve the accuracy of searching the second data table related to the first data table and preserve the model design architecture of the data warehouse, in one embodiment of the present invention, the data warehouse further comprises a virtual entity layer storing at least one virtual data table; the hierarchical association relationship indicates a corresponding relationship and a calculation relationship between the at least one virtual data table and the first data table and the second data table respectively; a plurality of virtual data tables associated with the first data table may then be determined from a virtual entity layer of the data warehouse based on the correspondence between the at least one virtual data table and the first data table and the second data table, respectively, and the plurality of second data tables corresponding to the plurality of virtual data tables, respectively, may be determined from the base layer.
The virtual entity layer may be a virtual integration layer and/or a virtual summary layer. When the virtual entity layer is a virtual integration layer, the summary layer of the data warehouse is the same as that of the conventional data warehouse, that is, when the virtual entity layer is a virtual integration layer, the summary layer of the data warehouse stores entity data. Similarly, when the pseudo-physical layer is a pseudo-aggregation layer, the integration layer of the data warehouse is the same as the integration layer of the conventional data warehouse, that is, when the pseudo-physical layer is a pseudo-integration layer, the integration layer of the data warehouse stores the physical data. In addition, when the virtual entity layer is a virtual integration layer and a virtual summary layer, neither the virtual integration layer nor the virtual summary layer of the data warehouse stores entity data.
It is to be understood that, the levels (the virtual integration layer and/or the virtual summary layer) corresponding to the virtual entity layer are different, and only indicate that the levels storing the entity data in the data warehouse are different, and the hierarchical architecture of the data warehouse itself is not affected, so for convenience of description, the data processing method based on the data warehouse provided by the embodiment of the present invention will be described in detail below by taking the levels corresponding to the virtual entity layer as the virtual integration layer and the virtual summary layer as an example. In this example, neither the virtual integration layer nor the virtual summary layer stores entity data, and the corresponding data link may be as shown in fig. 3.
The virtual data table may be a blank data table in which identification information is associated with the virtual entity layer but entity data is not stored, or may be a concept of a data table in which only identification information exists, that is, a real data table corresponding to the identification information of the data table does not exist in the virtual entity layer, and identification information corresponding to different data tables is defined only in the virtual entity layer. For example, at the virtual integration layer, the order integration virtual data table is defined as gdm _ ord (gdm1), wherein gdm1 is the identification information of the order integration virtual data table; similarly, a product integrated virtual data table is defined as gdm _ sku (gdm2), a user integrated virtual data table is defined as gdm _ user (gdm3), and a delivery integrated virtual data table is defined as gdm _ ship (gdm 4). Similar design can be made in the virtual summary layer, for example, the user order commodity summary virtual data report is defined as adm _ user _ ord _ sku (adm1), and the user delivery summary virtual entity adm _ user _ ship (adm2) is defined.
Although the entity data is not stored in the virtual entity layer, during the construction of the data warehouse, the model design work needs to be performed on the virtual integration layer and the virtual summary layer, and accordingly, the level association relationship corresponding to the virtual integration layer and the virtual summary layer is also maintained. In the embodiment of the invention, the hierarchical association relationship between the data tables of each hierarchy in the data warehouse can be uniformly stored and managed through the data calculation rule device, and the hierarchical association relationship indicates the corresponding relationship and the calculation relationship between the data tables in the data warehouse and the data tables of the non-same layer. In the data calculation rule device, the hierarchical association relationship may be stored in various ways such as a path diagram, an index or a table. For example, the data corresponding to the order and commodity aggregated virtual data report adm1 is obtained according to the data corresponding to the order integrated virtual data table gdm1, the commodity integrated virtual data table gdm2 and the user integrated virtual data table gdm3 in the integration layer, and the hierarchical association relationship indicates the correspondence and calculation relationship between adm1 and gdm1, gdm2 and gdm3, for example, as adm 1-gdm 1-gdm 2-gdm 3. In other words, the hierarchical association relationship indicates the parent-child relationship corresponding to each data table in the data warehouse and the calculation relationship between the data tables, for example, adm1 is derived from gdm1, gdm2 and gdm3, then the child nodes corresponding to adm1 are gdm1, gdm2 and gdm3, and correspondingly, the parent nodes corresponding to gdm1, gdm2 and gdm3 are adm 1.
In addition, the hierarchical association relationship of each hierarchy in the data warehouse may also be stored hierarchically, for example, the correspondence and calculation relationship between each virtual data table and the first data table and the second data table are stored in the virtual entity layer. In the base layer, not only the correspondence and calculation relationship between the second data table and the virtual data table in the virtual integration layer, but also the second data (entity data) is stored in the second data table. In the application layer, not only the corresponding relation and the calculation relation between the first data table and the virtual data table in the virtual summary layer are stored, but also the first data (entity data) is stored in the first data table.
Regardless of the manner in which the hierarchical association relationship is stored, when a data request is received, a second data table associated with the first data table indicated by the data request can be determined according to the hierarchical association relationship. The following takes the hierarchical association relationship stored in the data warehouse in a hierarchical manner as an example, and the second data table related to the first data table is determined by combining the specific association relationship of each layer in the data warehouse.
In this example, as shown in fig. 4, the application layer stores a hierarchical association relationship corresponding to the user full link application request app1 (data of app1 is derived from adm1, adm2 and a calculation relationship among apps 1, adm1 and adm 2). The virtual summary layer stores the corresponding hierarchical association relationship of the user order commodity summary virtual data table adm1 (the data of adm1 is derived from gdm1, gdm2 and gdm3 and the calculation relationship between adm1, gdm1, gdm2 and gdm3), and the corresponding hierarchical association relationship of the user delivery summary virtual data table gdm2 (the data of adm2 is derived from gdm3 and gdm4 and the calculation relationship between adm2, gdm3 and gdm 4). The virtual integration layer stores therein hierarchical associations corresponding to the order integration virtual data table gdm1 (gdm1 is derived from the second data table fdm1 and the second data table fdm2, and the calculation relationships between gdm1, fdm1 and fdm 2), hierarchical associations corresponding to the commodity integration virtual data table gdm2 (gdm2 is derived from the second data table fdm3 and the second data table fdm4, and the calculation relationships between gdm2, fdm3 and fdm 4), hierarchical associations corresponding to the user integration virtual data table gdm3 (gdm3 is derived from the second data table fdm5, and the calculation relationships between gdm3 and fdm 5), and hierarchical associations corresponding to the delivery integration virtual data table gdm4 (gdm4 is derived from the second data table fdm6, and the calculation relationships between gdm4 and fdm 6).
When the first data table of the application layer indicated by the data request is the app1, the second data tables related to the apps 1 may be determined to be fdm1, fdm2, fdm3, fdm4, fdm5 and fdm6 according to the hierarchical association relationship as shown in fig. 4.
It is understood that, when storing the hierarchical association relationship, the corresponding relationship and the calculation relationship between the first data table in the application layer and the second data table in the base layer may be directly stored, for example, if the data of the storage app1 is derived from the calculation relationship between fdm1, fdm2, fdm3, fdm4, fdm5 and fdm6, and app1, fdm1, fdm2, fdm3, fdm4, fdm5 and fdm6, when acquiring the data request indicating the first data table app1, the second data table related to the app1 is determined according to the stored association relationship. That is, in one embodiment of the present invention, the virtual entity layer may be omitted from the data warehouse, and the corresponding hierarchical association relationship and entity data are stored directly through the application layer and the base layer.
In order to clearly express the corresponding relation and the calculation relation between the data tables of all the hierarchies, in a preferred embodiment of the invention, a virtual entity layer is adopted to keep the hierarchy of an original integration layer and a summary layer between a base layer and an application layer, and the virtual entity layer is used to record the hierarchy association relation between the virtual data table and a first data table and a second data table, so that the data table related to the first data table can be conveniently searched layer by layer, and the efficiency and the accuracy of searching the data table are improved.
Step S203: according to the calculation relationship indicated by the hierarchical incidence relationship and the second data respectively stored in the plurality of second data tables, calculating first data used for responding to the data request in the first data table.
When the data warehouse includes a virtual entity layer, third data corresponding to each of the virtual data tables may be calculated according to the second data stored in the plurality of second data tables and the calculation relationship between the virtual data table and the second data table, and then the first data may be calculated according to the third data and the calculation relationship between the virtual data table and the first data table.
When recording data association relations among data tables of various levels in the data warehouse by using the path graph, after determining a second data table and a virtual data table related to a first data table, obtaining a calculation path graph related to the first data table, the virtual data table and the second data table from a total path graph recording all data tables, wherein the calculation path graph is shown in fig. 4, then sequentially calculating third data corresponding to the virtual data tables in the virtual integration layer and the virtual summary layer according to the second data table of the base layer in the calculation path graph shown in fig. 4, and finally calculating first data corresponding to the first data table in the application layer according to the third data.
Of course, in an embodiment of the present invention, the hierarchical association relationship between the hierarchical data tables may also be recorded in other forms such as an index or a table, and after the virtual data table and the second data table related to the first data table indicated by the data request are determined, the calculation relationship diagram shown in fig. 4 is generated according to the correspondence relationship among the first data table, the virtual data table, and the second data table, so as to calculate the first data corresponding to the first data table according to the calculation relationship diagram shown in fig. 4 at a later stage, thereby improving the calculation efficiency of the first data.
In order to save the storage resources consumed by the data warehouse, in one embodiment of the present invention, after the first data in the first data table for responding to the data request is calculated, the third data corresponding to the virtual data table in the virtual entity layer is deleted.
Specifically, in the process of calculating the third data and the first data layer by layer in the order of the virtual integration layer, the virtual summary layer and the application layer according to the corresponding relationship and the calculation relationship among the first data table, the virtual data table and the second data table, after each layer calculates the corresponding data, the data layer on the upper layer is notified, so that the upper data layer continues to calculate according to the calculated data, for example, after the virtual integration layer calculates corresponding third data according to the second data stored in the second data table, informing the virtual summary layer of the upper layer, calculating and calculating by the virtual summary layer according to the third data calculated by the virtual integration layer, after the virtual summary layer calculates the third data corresponding to the corresponding virtual data table, the application layer is notified, and the application layer calculates first data corresponding to the first data table according to the third data calculated by the virtual summary layer.
After the first data is finally generated, the application layer notifies the virtual summary layer and the virtual integration layer to release corresponding third data, namely, the third data is deleted, so that the data integrity in the calculation process can be ensured, and the efficiency and the accuracy of calculating the first data can be improved.
Further, since the application layer is a layer designed directly facing the business requirements, the layer is actually grounded as a data storage entity table, so as to respond to the business requirements more effectively and quickly. Therefore, after the first data is calculated, the first data and the first data table are correspondingly stored in the application layer, that is, the application layer stores entity data without virtualization, so that when a service demand is received, the corresponding first data is fed back quickly.
In summary, the processing method based on the data warehouse provided by the embodiment of the present invention may include the steps as shown in fig. 5:
step S501: obtaining a data request indicating a first data table stored at an application layer of the data warehouse to be accessed.
Referring to fig. 4, the data request indicates a first data table app1 at the application layer.
Step S502: and determining a first virtual data table related to the first data table from the virtual summary layer of the data warehouse according to the hierarchical incidence relation of the data warehouse.
As shown in fig. 4, the hierarchical association relationship indicates a correspondence relationship and a calculation relationship between the data table in the virtual data warehouse and the data table in the non-peer layer, for example, a correspondence relationship and a calculation relationship between the virtual data table in the virtual integration layer and the second data table in the base layer, such as a correspondence relationship and a calculation relationship between the virtual data table in the virtual summary layer and the first data table in the application layer. In this example, referring to FIG. 4, the first virtual data tables associated with the first data table app1 are adm1 and adm 2.
Step S503: and determining a second virtual data table related to the first virtual data table from the virtual integration layer of the data warehouse according to the hierarchical association relationship of the data warehouse.
In this example, referring to fig. 4, the second virtual data tables related to the first virtual data table adm1 are gdm1, gdm2 and gdm3, and the second virtual data tables related to the first virtual data table adm2 are gdm2 and gdm 4.
Step S504: and determining a second data table related to the second virtual data table from the base layer of the data warehouse according to the hierarchical association relationship of the data warehouse.
In this example, referring to fig. 4, the second data tables associated with the second virtual data table gdm1 are fdm1 and fdm2, the second data tables associated with the second virtual data table gdm2 are fdm3 and fdm4, the second data table associated with the second virtual data table gdm3 is fdm5, and the second data tables associated with the second virtual data table gdm4 are fdm 6.
Step S505: and calculating third data corresponding to the second virtual data table according to the calculation relationship indicated by the hierarchy incidence relationship and the second data in the second data table, and calculating fourth data corresponding to the first virtual data table according to the third data.
Step S506: and calculating first data corresponding to the first data table of the application layer according to the fourth data corresponding to the first virtual data table.
Step S507: and correspondingly storing the first data and the first data table in an application layer of the data warehouse, and deleting the third data of the virtual integration layer and the fourth data of the virtual summary layer.
When the data is correspondingly stored, the first data can be filled into the first data table, and then the first data table is stored.
According to the data processing method based on the data warehouse, when the data request is acquired, the second data table related to the first data table of the application layer indicated by the data request can be determined from the base layer of the data warehouse according to the hierarchical association relation stored in the data warehouse, and then the first data used for responding to the data request in the first data table is calculated according to the calculation relation indicated by the hierarchical association relation and the second data in the second data table. Therefore, in the data warehouse, the stored hierarchical incidence relation replaces the data stored in the integration layer and the summary layer, and when a data request is received, the data used for responding to the data request in the application layer is calculated according to the hierarchical incidence relation and the data stored in the base layer, namely, the integration layer and the summary layer of the data warehouse do not store the integration data and the summary data corresponding to the base data any more, so that the storage resources consumed by the data warehouse are reduced. And the base layer and the application layer in the data warehouse store corresponding data, so that the base layer is used as a data root of the data warehouse, and data is reserved for a long time to facilitate data backtracking. And the application layer is designed directly facing to the business requirement, and the layer is also actually grounded as a data storage entity table and correspondingly stores the first data table and the first data (entity data) without virtualization, so that the business requirement is responded more effectively and quickly.
Fig. 6 is a schematic diagram of the main modules of a data warehouse-based data processing apparatus according to an embodiment of the present invention.
As shown in fig. 6, a data warehouse-based data processing apparatus 600 according to an embodiment of the present invention includes: a request acquisition module 601, a determination module 602 and a calculation module 603; wherein,
the request obtaining module 601 is configured to obtain a data request, where the data request indicates a first data table to be accessed and stored in an application layer of the database;
the determining module 602 is configured to determine, according to the hierarchical association relationship of the data warehouse, a plurality of second data tables that are related to the first data table and store second data from a base layer of the data warehouse; the hierarchical incidence relation indicates the corresponding relation and the calculation relation between the data table in the data warehouse and the data table of the non-same layer;
the calculating module 603 is configured to calculate, according to the calculation relationship indicated by the hierarchical association relationship and the second data stored in the plurality of second data tables, first data used for responding to the data request in the first data table.
In one embodiment of the present invention, the data warehouse further comprises a virtual entity layer storing at least one virtual data table, wherein the hierarchical association relationship indicates a correspondence relationship and a calculation relationship between the at least one virtual data table and the first data table and the second data table, respectively;
the determining module 602 is configured to determine, from a virtual entity layer of the data warehouse, a plurality of virtual data tables related to the first data table according to the correspondence between the at least one virtual data table and the first data table and the second data table respectively, and determine, from the base layer, the plurality of second data tables corresponding to the plurality of virtual data tables respectively.
In an embodiment of the present invention, the calculating module 603 is configured to calculate third data corresponding to each of the virtual data tables according to the second data stored in each of the plurality of second data tables and the calculation relationship between the virtual data table and the second data table; and calculating the first data according to the third data and the calculation relationship between the virtual data table and the first data table.
With continued reference to fig. 6, in one embodiment of the invention, the data processing apparatus may further include: a storage module 604; wherein,
the storage module 604 is further configured to correspondingly store the first data and the first data table in the application layer.
With continued reference to fig. 6, in one embodiment of the invention, the data processing apparatus may further include: a delete module 605; the deleting module 605 is configured to delete the third data.
According to the data processing device based on the data warehouse, when a data request is acquired, a second data table related to a first data table of an application layer indicated by the data request can be determined from a base layer of the data warehouse according to the hierarchical association relation stored in the data warehouse, and then first data used for responding to the data request in the first data table can be calculated according to the calculation relation indicated by the hierarchical association relation and the second data in the second data table. Therefore, in the data warehouse, the stored hierarchical incidence relation replaces the data stored in the integration layer and the summary layer, and when a data request is received, the data used for responding to the data request in the application layer is calculated according to the hierarchical incidence relation and the data stored in the base layer, namely, the integration layer and the summary layer of the data warehouse do not store the integration data and the summary data corresponding to the base data any more, so that the storage resources consumed by the data warehouse are reduced. And the base layer and the application layer in the data warehouse store corresponding data, so that the base layer is used as a data root of the data warehouse, and data is reserved for a long time to facilitate data backtracking. And the application layer is designed directly facing to the business requirement, and the layer is also actually grounded as a data storage entity table and correspondingly stores the first data table and the first data (entity data) without virtualization, so that the business requirement is responded more effectively and quickly.
Fig. 7 shows an exemplary system architecture 700 of a data warehouse-based data processing method or a data warehouse-based data processing apparatus to which embodiments of the present invention may be applied.
As shown in fig. 7, the system architecture 700 may include terminal devices 701, 702, 703, a network 704, and a server 705. The network 704 serves to provide a medium for communication links between the terminal devices 701, 702, 703 and the server 705. Network 704 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
A user may use the terminal devices 701, 702, 703 to interact with a server 705 over a network 704, to receive or send messages or the like. Various communication client applications, such as shopping applications, web browser applications, search applications, instant messaging tools, mailbox clients, social platform software, and the like, may be installed on the terminal devices 701, 702, and 703.
The terminal devices 701, 702, 703 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 705 may be a server that provides various services, such as a background management server that supports shopping websites browsed by users using the terminal devices 701, 702, and 703. The background management server may analyze and perform other processing on the received data such as the product information query request, and feed back a processing result (e.g., target push information and product information) to the terminal device.
It should be noted that the data processing method based on the data warehouse provided by the embodiment of the present invention is generally executed by the server 705, and accordingly, the data processing apparatus based on the data warehouse is generally disposed in the server 705.
It should be understood that the number of terminal devices, networks, and servers in fig. 7 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 8, shown is a block diagram of a computer system 800 suitable for use with a terminal device implementing an embodiment of the present invention. The terminal device shown in fig. 8 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 8, the computer system 800 includes a Central Processing Unit (CPU)801 that can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)802 or a program loaded from a storage section 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data necessary for the operation of the system 800 are also stored. The CPU 801, ROM 802, and RAM 803 are connected to each other via a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
The following components are connected to the I/O interface 805: an input portion 806 including a keyboard, a mouse, and the like; an output section 807 including a signal such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 808 including a hard disk and the like; and a communication section 809 including a network interface card such as a LAN card, a modem, or the like. The communication section 809 performs communication processing via a network such as the internet. A drive 810 is also connected to the I/O interface 805 as necessary. A removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 810 as necessary, so that a computer program read out therefrom is mounted on the storage section 808 as necessary.
In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 809 and/or installed from the removable medium 811. The computer program executes the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 801.
It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes a request acquisition module, a determination module, and a calculation module. The names of these modules do not in some cases constitute a limitation on the module itself, for example, a requesting module may also be described as a "module that obtains a data request.
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise: obtaining a data request, wherein the data request indicates a first data table to be accessed and stored in an application layer of the data warehouse; determining a plurality of second data tables which are related to the first data table and store second data from a base layer of the data warehouse according to the hierarchical incidence relation of the data warehouse; the hierarchical incidence relation indicates the corresponding relation and the calculation relation between the data table in the data warehouse and the data table of the non-same layer; according to the calculation relationship indicated by the hierarchical incidence relationship and the second data respectively stored in the plurality of second data tables, calculating first data used for responding to the data request in the first data table.
According to the technical scheme of the embodiment of the invention, when the data request is acquired, the second data table related to the first data table of the application layer indicated by the data request can be determined from the base layer of the data warehouse according to the hierarchical incidence relation stored in the data warehouse, and then the first data used for responding to the data request in the first data table is calculated according to the calculation relation indicated by the hierarchical incidence relation and the second data in the second data table. Therefore, in the data warehouse, the stored hierarchical incidence relation replaces the data stored in the integration layer and the summary layer, and when a data request is received, the data used for responding to the data request in the application layer is calculated according to the hierarchical incidence relation and the data stored in the base layer, namely, the integration layer and the summary layer of the data warehouse do not store the integration data and the summary data corresponding to the base data any more, so that the storage resources consumed by the data warehouse are reduced. And the base layer and the application layer in the data warehouse store corresponding data, so that the base layer is used as a data root of the data warehouse, and data is reserved for a long time to facilitate data backtracking. And the application layer is designed directly facing to the business requirement, and the layer is also actually grounded as a data storage entity table and correspondingly stores the first data table and the first data (entity data) without virtualization, so that the business requirement is responded more effectively and quickly.
The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (13)
1. A data processing method based on a data warehouse, comprising:
obtaining a data request, wherein the data request indicates a first data table to be accessed and stored in an application layer of the data warehouse;
determining a plurality of second data tables which are related to the first data table and store second data from a base layer of the data warehouse according to the hierarchical incidence relation of the data warehouse; the hierarchical incidence relation indicates the corresponding relation and the calculation relation between the data table in the data warehouse and the data table of the non-same layer;
according to the calculation relationship indicated by the hierarchical incidence relationship and the second data respectively stored in the plurality of second data tables, calculating first data used for responding to the data request in the first data table.
2. The method of claim 1, wherein the data warehouse further comprises a virtual entity layer storing at least one virtual data table; the hierarchical association relationship indicates a corresponding relationship and a calculation relationship between the at least one virtual data table and the first data table and the second data table respectively;
the determining, from the base layer of the data warehouse, a second data table that is related to the first data table and stores second data, comprising:
according to the corresponding relation between the at least one virtual data table and the first data table and the second data table respectively, determining a plurality of virtual data tables related to the first data table from a virtual physical layer of the data warehouse, and determining a plurality of second data tables corresponding to the plurality of virtual data tables respectively from the base layer.
3. The method of claim 2, wherein said computing the first data in the first data table for responding to the data request comprises:
respectively calculating third data corresponding to each virtual data table according to the second data stored in the plurality of second data tables and the calculation relationship between the virtual data table and the second data table;
and calculating the first data according to the third data and the calculation relationship between the virtual data table and the first data table.
4. The method of claim 1, further comprising:
and correspondingly storing the first data and the first data table in the application layer.
5. The method of claim 3, further comprising, after said computing first data in the first data table for responding to the data request:
and deleting the third data.
6. The method of claim 2,
the virtual entity layer includes: a virtual integration layer and/or a virtual summary layer.
7. A data processing apparatus based on a data warehouse, comprising: the device comprises a request acquisition module, a determination module and a calculation module; wherein,
the request acquisition module is used for acquiring a data request which indicates a first data table to be accessed and stored in an application layer of the database;
the determining module is used for determining a plurality of second data tables which are related to the first data table and store second data from a base layer of the data warehouse according to the hierarchical incidence relation of the data warehouse; the hierarchical incidence relation indicates the corresponding relation and the calculation relation between the data table in the data warehouse and the data table of the non-same layer;
the calculation module is configured to calculate, according to the calculation relationship indicated by the hierarchical association relationship and the second data stored in the plurality of second data tables, first data used for responding to the data request in the first data table.
8. The apparatus of claim 7,
the data warehouse further comprises a virtual entity layer storing at least one virtual data table, wherein the hierarchical association relationship indicates a corresponding relationship and a calculation relationship between the at least one virtual data table and the first data table and the second data table respectively;
the determining module is configured to determine, from a virtual entity layer of the data warehouse, a plurality of virtual data tables related to the first data table according to correspondence between the at least one virtual data table and the first data table and the second data table, and determine, from the base layer, the plurality of second data tables corresponding to the plurality of virtual data tables, respectively.
9. The apparatus of claim 8,
the calculation module is configured to calculate third data corresponding to each of the virtual data tables according to the second data stored in the plurality of second data tables and a calculation relationship between the virtual data table and the second data table; and calculating the first data according to the third data and the calculation relationship between the virtual data table and the first data table.
10. The apparatus of claim 8, further comprising: a storage module; wherein,
the storage module is further configured to correspondingly store the first data and the first data table in the application layer.
11. The apparatus of claim 9, further comprising: a deletion module; wherein,
and the deleting module is used for deleting the third data.
12. An electronic device for data processing, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-6.
13. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911083470.8A CN112783887B (en) | 2019-11-07 | 2019-11-07 | Data processing method and device based on data warehouse |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911083470.8A CN112783887B (en) | 2019-11-07 | 2019-11-07 | Data processing method and device based on data warehouse |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112783887A true CN112783887A (en) | 2021-05-11 |
| CN112783887B CN112783887B (en) | 2024-08-16 |
Family
ID=75747991
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911083470.8A Active CN112783887B (en) | 2019-11-07 | 2019-11-07 | Data processing method and device based on data warehouse |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112783887B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115422169A (en) * | 2022-11-04 | 2022-12-02 | 暨南大学 | Data warehouse construction method and device based on commercial scene |
| CN115934746A (en) * | 2023-02-28 | 2023-04-07 | 江西省旅游集团文旅科技有限公司 | Travel data processing method and system, electronic device and readable storage medium |
| CN116662414A (en) * | 2023-07-27 | 2023-08-29 | 腾讯科技（深圳）有限公司 | Data processing method, apparatus, device, storage medium, and program product |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7461086B1 (en) * | 2006-01-03 | 2008-12-02 | Symantec Corporation | Run-time application installation application layered system |
| CN101563929A (en) * | 2006-12-22 | 2009-10-21 | 高通股份有限公司 | Multimedia data reorganization between base layer and enhancement layer |
| CN103019691A (en) * | 2012-11-20 | 2013-04-03 | 北京思特奇信息技术股份有限公司 | Transformation method for extract, transform and load (ETL) operation relation graph and implementation system thereof |
| CN103902582A (en) * | 2012-12-27 | 2014-07-02 | 中国移动通信集团湖北有限公司 | Data warehouse redundancy reduction method and device |
| US9075635B1 (en) * | 2010-07-26 | 2015-07-07 | Symantec Corporation | Systems and methods for merging virtual layers |
| CN106875110A (en) * | 2017-02-06 | 2017-06-20 | 泰康保险集团股份有限公司 | Business index layered calculation method and device, distributed calculation method and system |
| US20180157731A1 (en) * | 2016-12-05 | 2018-06-07 | Business Objects Software Ltd. | Hierarchy member selections in queries based on relational databases |
| CN108427711A (en) * | 2018-01-31 | 2018-08-21 | 北京三快在线科技有限公司 | Real-time data warehouse, real-time data processing method, electronic equipment and storage medium |
| CN108628894A (en) * | 2017-03-21 | 2018-10-09 | 阿里巴巴集团控股有限公司 | Data target querying method in data warehouse and device |
| CN110245270A (en) * | 2019-05-09 | 2019-09-17 | 重庆天蓬网络有限公司 | Data genetic connection storage method, system, medium and equipment based on graph model |
-
2019
- 2019-11-07 CN CN201911083470.8A patent/CN112783887B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7461086B1 (en) * | 2006-01-03 | 2008-12-02 | Symantec Corporation | Run-time application installation application layered system |
| CN101563929A (en) * | 2006-12-22 | 2009-10-21 | 高通股份有限公司 | Multimedia data reorganization between base layer and enhancement layer |
| US9075635B1 (en) * | 2010-07-26 | 2015-07-07 | Symantec Corporation | Systems and methods for merging virtual layers |
| CN103019691A (en) * | 2012-11-20 | 2013-04-03 | 北京思特奇信息技术股份有限公司 | Transformation method for extract, transform and load (ETL) operation relation graph and implementation system thereof |
| CN103902582A (en) * | 2012-12-27 | 2014-07-02 | 中国移动通信集团湖北有限公司 | Data warehouse redundancy reduction method and device |
| US20180157731A1 (en) * | 2016-12-05 | 2018-06-07 | Business Objects Software Ltd. | Hierarchy member selections in queries based on relational databases |
| CN106875110A (en) * | 2017-02-06 | 2017-06-20 | 泰康保险集团股份有限公司 | Business index layered calculation method and device, distributed calculation method and system |
| CN108628894A (en) * | 2017-03-21 | 2018-10-09 | 阿里巴巴集团控股有限公司 | Data target querying method in data warehouse and device |
| CN108427711A (en) * | 2018-01-31 | 2018-08-21 | 北京三快在线科技有限公司 | Real-time data warehouse, real-time data processing method, electronic equipment and storage medium |
| CN110245270A (en) * | 2019-05-09 | 2019-09-17 | 重庆天蓬网络有限公司 | Data genetic connection storage method, system, medium and equipment based on graph model |
Non-Patent Citations (1)
| Title |
|---|
| 拜亚萌;张燕玲;: "虚拟化技术在智慧校园云服务平台IaaS层中的应用", 焦作大学学报, no. 01 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115422169A (en) * | 2022-11-04 | 2022-12-02 | 暨南大学 | Data warehouse construction method and device based on commercial scene |
| CN115422169B (en) * | 2022-11-04 | 2023-07-14 | 暨南大学 | Data warehouse construction method and device based on commercial advertisement scene |
| CN115934746A (en) * | 2023-02-28 | 2023-04-07 | 江西省旅游集团文旅科技有限公司 | Travel data processing method and system, electronic device and readable storage medium |
| CN116662414A (en) * | 2023-07-27 | 2023-08-29 | 腾讯科技（深圳）有限公司 | Data processing method, apparatus, device, storage medium, and program product |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112783887B (en) | 2024-08-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109189835B (en) | Method and device for generating data wide table in real time | |
| CN107229718B (en) | Method and device for processing report data | |
| CN109614402B (en) | Multidimensional data query method and device | |
| US20140324917A1 (en) | Reclamation of empty pages in database tables | |
| CN110689268B (en) | Method and device for extracting indexes | |
| CN111753019B (en) | Data partitioning method and device applied to data warehouse | |
| CN113760521B (en) | Virtual resource allocation method and device | |
| CN112783887B (en) | Data processing method and device based on data warehouse | |
| US12386837B2 (en) | Memory graph query engine with persisted storage | |
| US10873552B2 (en) | Large data management in communication applications through multiple mailboxes | |
| CN108985805B (en) | Method and device for selectively executing push task | |
| CN107609038B (en) | Data cleaning method and device | |
| CN119336731A (en) | Data storage method and device | |
| CN113704242B (en) | Data processing method and device | |
| CN113760860A (en) | Data reading method and device | |
| CN112699116A (en) | Data processing method and system | |
| CN112988857B (en) | Service data processing method and device | |
| CN113760965B (en) | Data query method and device | |
| CN113778318B (en) | Data storage method and device | |
| CN113362097B (en) | User determination method and device | |
| CN115454971A (en) | Data migration method and device, electronic equipment and storage medium | |
| CN110473035B (en) | A method and device for determining the duration of order deletion | |
| CN113722548A (en) | Method and device for processing reference relationship in business system | |
| CN112783914A (en) | Statement optimization method and device | |
| CN112711572A (en) | Online capacity expansion method and device suitable for sub-warehouse and sub-meter |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |