[go: up one dir, main page]

CN119201946A - Management and query method of the same indicator in multiple fact tables based on indicator center - Google Patents

Management and query method of the same indicator in multiple fact tables based on indicator center Download PDF

Info

Publication number
CN119201946A
CN119201946A CN202411718956.5A CN202411718956A CN119201946A CN 119201946 A CN119201946 A CN 119201946A CN 202411718956 A CN202411718956 A CN 202411718956A CN 119201946 A CN119201946 A CN 119201946A
Authority
CN
China
Prior art keywords
query
indicator
source table
index
dimension
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202411718956.5A
Other languages
Chinese (zh)
Inventor
石凯
严林刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Guanyuan Data Co ltd
Original Assignee
Hangzhou Guanyuan Data Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Guanyuan Data Co ltd filed Critical Hangzhou Guanyuan Data Co ltd
Priority to CN202411718956.5A priority Critical patent/CN119201946A/en
Publication of CN119201946A publication Critical patent/CN119201946A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2282Tablespace storage structures; Management thereof
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2453Query optimisation
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/283Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application relates to a management and query method for realizing the same index of multiple fact tables based on an index center, which is characterized in that the same index of each source table is defined in caliber, the source tables are arranged from small to large according to the data size, the query dimension is obtained according to the index query condition input by a user, the source table matched with the query dimension is found and used as a target source table, and then index query is carried out based on the target source table, and the corresponding index calculation result is returned. According to the screening dimension range of index query conditions, dynamically matching available analysis dimensions on each source table at a time, and selecting a current source table to perform data query after successful matching, so as to find an optimal source table meeting service requirements, and improve index query efficiency. The index platform does not need to create a plurality of indexes to map different fact tables respectively, uses one index as mapping, positions the query table according to the query condition and the preset table acceleration sequence, has simpler index management and more intelligent index analysis.

Description

Method for realizing management and query of same index of multiple fact tables based on index center
Technical Field
The disclosure relates to the field of table lookup technology, and in particular relates to a method for managing and querying the same index of a multi-fact table based on an index center, a device for managing and querying the same index of the multi-fact table based on the index center, and electronic equipment.
Background
As shown in FIG. 1, in general, a DWD or a DWS processed based on a detail table will have a plurality of fact tables during the construction of a plurality of bins. The fact tables differ only in dimension range, but the index list is identical.
Although the table lookup management can be performed on each fact table based on different indexes, the following technical defects still exist in practical application:
firstly, in the process of generating a plurality of fact tables based on detail table processing, an index platform needs to create a plurality of indexes and map different fact tables respectively, which causes excessively complicated index data, slow subsequent table searching process and low table searching efficiency;
The index design of the fact tables does not have unified relevance, so that the index design can only query according to the designated table, and therefore, ordered acceleration query service cannot be realized, if the fact table with large data size is encountered and no index data is needed, the subsequent table look-up progress is delayed, or the fact table with small data size and the same index is preferentially queried to be the optimal path, but in reality, the user cannot accurately find the optimal path.
Disclosure of Invention
In order to solve the problems, the application provides a management and query method for realizing the same index of a multi-fact table based on an index center, a management and query device for realizing the same index of the multi-fact table based on the index center and electronic equipment.
In one aspect of the present application, a method for managing and querying the same index of a multi-fact table based on an index center is provided, including the following steps:
S1, performing caliber definition on the same index of each source table, and arranging each source table from small to large according to the data size;
s2, acquiring a query dimension of an index query condition input by a user, traversing and finding the source table matched with the query dimension, and taking the source table as a target source table;
and S3, inquiring the index based on the target source table, and returning a corresponding index calculation result.
As an optional embodiment of the present application, optionally, S1 defines the caliber of the same index of each source table, and arranges each source table from small to large according to the data size, including:
Setting index names and service apertures;
searching from a plurality of bins to obtain each source table corresponding to the index name and the service caliber;
grouping each of the source tables according to an analysis dimension;
and arranging the source tables after grouping according to the data size from small to large.
As an optional embodiment of the present application, optionally, S1 defines the caliber of the same index of each source table, and arranges each source table from small to large according to the data size, and further includes:
And after the source tables are grouped and arranged from small to large according to the data size, respectively marking the sequence numbers of the source tables arranged from small to large.
As an optional embodiment of the present application, optionally, S1 defines the caliber of the same index of each source table, and arranges each source table from small to large according to the data size, and further includes:
binding the marked sequence number with the corresponding source table;
and generating and storing a corresponding source form according to the source form marked by the sequence numbers from small to large.
As an optional embodiment of the present application, optionally, S2, obtaining a query dimension of an index query condition input by a user, traversing and finding the source table matched with the query dimension, and taking the source table as a target source table, including:
acquiring index query conditions input by a user;
Analyzing query dimensions to obtain the query dimensions in the index query conditions;
Traversing a source form, orderly identifying analysis dimensions of each source form in the source form, and judging whether the analysis dimensions contain the query dimensions or not:
if yes, finishing traversing, and marking the source table containing the query dimension in the analysis dimension as the target source table;
otherwise, the traversal is continued.
As an optional embodiment of the present application, optionally, when setting the index name and the service aperture, the method further includes:
and constructing the index name and the service caliber corresponding to the query dimension according to the query dimension.
As an optional embodiment of the present application, optionally, S3, performing index query based on the target source table, and returning a corresponding index calculation result, including:
constructing a query request according to the query dimension;
sending the query request to a plurality of bins;
And responding to the query request through a plurality of bins, performing index query and index calculation based on the target source table, and returning corresponding index calculation results to the user.
In another aspect of the present application, a management and query device for implementing the same index of a multi-fact table based on an index center is provided, which is used for implementing the above management and query method for implementing the same index of a multi-fact table based on an index center, including:
the index definition module is used for defining the same index of each source table in caliber and arranging the source tables from small to large according to the data size;
The execution plan matching module is used for acquiring the query dimension of the index query condition input by the user, traversing and finding the source table matched with the query dimension, and taking the source table as a target source table;
and the index query module is used for performing index query based on the target source table and returning a corresponding index calculation result.
In another aspect, the present application further provides an electronic device, including:
A processor;
a memory for storing processor-executable instructions;
The processor is configured to implement the method for managing and querying the same index of the multi-fact table based on the index center when executing the executable instructions.
The invention has the technical effects that:
The method comprises the steps of firstly defining the same index of each source table in caliber, arranging each source table from small to large according to the data size, secondly obtaining query dimension according to index query conditions input by a user, traversing each source table with the data size from small to large, finding the source table matched with the query dimension, taking the source table as a target source table, finally querying the index based on the target source table, and returning a corresponding index calculation result. Therefore, the available analysis dimensions on each source table can be dynamically matched once according to the screening dimension range of the index query condition, and the current source table is selected for data query after successful matching, so that the optimal source table meeting the service requirement is found, and the index query efficiency is improved. The index platform can be used for mapping different fact tables without creating a plurality of indexes, one index is used for mapping, the query table is positioned according to the query condition and the preset table acceleration sequence, the index management is simpler, and the index analysis is more intelligent.
Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features and aspects of the present disclosure and together with the description, serve to explain the principles of the disclosure.
FIG. 1 is a diagram showing a mapping relationship between a conventional detail table and a fact table;
FIG. 2 shows a schematic flow chart of an embodiment of the invention;
FIG. 3 is a schematic diagram showing a source table arranged from small to large according to the data gauge module of the present invention;
FIG. 4 is a flow chart of index query according to the present invention;
FIG. 5 is a schematic view showing the structure of the device of the present invention;
fig. 6 shows a schematic application diagram of the electronic device of the invention.
Detailed Description
Various exemplary embodiments, features and aspects of the disclosure will be described in detail below with reference to the drawings. In the drawings, like reference numbers indicate identical or functionally similar elements. Although various aspects of the embodiments are illustrated in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
The word "exemplary" is used herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.
In addition, numerous specific details are set forth in the following detailed description in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, well known means, elements, and circuits have not been described in detail so as not to obscure the present disclosure.
Description of related terms and principles:
1. DWD (Data Warehouse Detail) and DWS (Data Warehouse Summary) are important links of data processing in a data warehouse (Data Warehouse, abbreviated as a number of bins) or a large data processing flow. The DWD layer is generally referred to as a detail data layer, which holds the most raw, detailed data, while the DWS layer is a layer that aggregates the data of the DWD layer into a light aggregate for more efficient analysis and querying.
The process of obtaining multiple fact tables from a DWD or DWS layer typically involves the following steps:
‌ data preparation ‌:
Data is loaded from a data source (e.g., database, file, API, etc.) to the DWD layer.
Ensuring the integrity, accuracy and consistency of data may require data cleansing, deduplication, format conversion, and the like.
‌ Build DWD layer ‌:
In the DWD layer, data is typically organized in terms of business processes, each business process corresponding to one or more tables of detail.
These tables contain all the detailed data generated during the business process and are the basis for subsequent analysis.
‌ Build DWS layer ‌:
in the DWS layer, data of the DWD layer are summarized and aggregated according to service requirements.
The granularity of aggregation may vary depending on business needs, e.g., by day, by month, by division, etc.
The data of the DWS layer is typically used to generate reports, perform data analysis, or as input to other data processing flows.
‌ Generates fact table ‌:
based on the data of the DWD or DWS layer, a plurality of fact tables are generated according to analysis requirements.
Fact tables typically contain metrics (e.g., sales, quantity, costs, etc.) and keys associated with dimension tables (e.g., time dimension, product dimension, customer dimension, etc.).
The fact table design should satisfy classical dimension modeling principles such as star model or snowflake model, so as to perform efficient analysis and query.
‌ Optimizing and storing ‌:
the generated fact table is optimized, such as creating indexes, partitions, etc., to improve query performance.
The optimized fact table is stored in a data warehouse or a big data platform for subsequent analysis and query.
‌ Data management and maintenance ‌:
and the data of the DWD layer and the DWS layer are updated regularly, so that the timeliness and the accuracy of the data are ensured.
2. The step of index inquiry from the bins can be divided into:
‌ select query means ‌:
queries using the SQL language, BI tools, or predefined data modeling are selected according to requirements.
‌ Write an SQL query (e.g., using SQL) ‌:
The SELECT statement is used to specify the table name, field name, and conditions that need to be queried.
Multiple tables are concatenated using JOIN operations to obtain a comprehensive view of the data.
The data is grouped BY a GROUP BY clause and an aggregation function is applied.
‌ Execute query ‌:
the compiled SQL query is run in the data warehouse system or the query is triggered by the BI tool.
‌ Optimize query performance ‌:
physical I/O and repeated SQL parsing are reduced as much as possible using variable binding techniques.
The logic reading or executing times of a single statement are reduced, and the query efficiency is improved.
‌ Obtain and process results ‌:
and checking the query result, and performing further data processing or analysis according to the requirement.
In this embodiment, the steps of index query based on the input query condition from the number bin and the number bin are not repeated for the number bin based on the DWD processed by the detail table or the number of fact tables stored in the DWS.
Example 1
As shown in fig. 2, in one aspect of the present application, a method for managing and querying the same index of a multi-fact table based on an index center is provided, including the following steps:
S1, performing caliber definition on the same index of each source table, and arranging each source table from small to large according to the data size;
s2, acquiring a query dimension of an index query condition input by a user, traversing and finding the source table matched with the query dimension, and taking the source table as a target source table;
and S3, inquiring the index based on the target source table, and returning a corresponding index calculation result.
The invention firstly carries out multi-table mapping management of index definition, when the index calculates caliber definition, the index can carry out grouping definition according to a plurality of source tables, the analysis dimension is selected, and the index is arranged from small to large according to the scale of the table data volume.
When the subsequent multi-fact table query execution under the same index is carried out, the table acceleration sequence based on each table can be realized, and the management and query of the same index of the multi-fact table are realized, so that the table lookup efficiency is improved, and the defect of table lookup confusion caused by disorder among the tables is avoided. According to the invention, table lookup can be performed according to the data set scale, and each fact table can be queried in sequence from small to large until the fact table matched with the query dimension is found, so that the table lookup efficiency is improved and the table lookup logic is enhanced.
Embodiments of the present invention will be further described below.
As shown in fig. 3, as an alternative embodiment of the present application, optionally, S1 defines the caliber of the same index of each source table, and arranges each source table from small to large according to the data size, including:
Setting index names and service apertures;
searching from a plurality of bins to obtain each source table corresponding to the index name and the service caliber;
grouping each of the source tables according to an analysis dimension;
and arranging the source tables after grouping according to the data size from small to large.
The index name and service caliber of the source table are firstly required to be defined in the index center, so that the grouping definition is facilitated.
The index name and the service caliber are combined, and a source table meeting the index dimension of the time and meeting the index data query and calculation under the dimension is found from a plurality of bins.
The index name and the service caliber can be defined according to the index query dimension required by the user. The index name and the service caliber corresponding to the query dimension can be constructed according to the query dimension. Such as:
index name, trade amount;
and the service caliber is obtained by summarizing and calculating transaction amount based on the transaction records.
In this embodiment, the query dimension is set according to the design requirement of the index system.
The number of source tables searched from a plurality of bins is large, the selection is needed, and the source tables are grouped according to the required analysis dimension. The source tables are divided into tables of different analytical dimensions. The analysis dimensions are different, and the corresponding data size and the subsequent index calculation amount are also different. Therefore, after grouping, the data amount sizes of the tables (i.e., the calculation scale of the index or the size of the index data required for the analysis dimension) are sorted in groups, and the data amount sizes are arranged from the bottom to the top. The amount of index data in each table may be checked by reading the table (the amount of index data recorded in the table may be checked by reading the table).
The LLM large language model can be used for searching the source tables corresponding to the index names and the service apertures from the bins based on the defined index names and the service apertures, so that intelligent bin data table searching is realized, automatic table acceleration source table construction is realized, and efficiency is improved. Specific:
The method comprises the steps of setting large language model prompt words based on index names and service apertures, respectively setting the large language model prompt words corresponding to the different index names and service apertures according to different index query dimensions and inputting a preset LLM large language model, wherein the LLM large language model can be called in an API calling mode (such as hundred degree center-to-center) or is deployed and quoted based on the LLM large language model for custom construction.
And the prompting words can be added with data gauge template reading information, so that the LLM large language model can read the data volume of each retrieved source list, and the construction of the subsequent list acceleration sequence number is facilitated.
And the LLM large language model searches the source tables corresponding to the index names and the service apertures from a plurality of bins based on prompt words, and groups the source tables according to analysis dimension. After the corresponding source table is automatically retrieved from the plurality of bins and the corresponding data size is read through the LLM large language model, the retrieved source table can be automatically ranked according to the data size, and a ranking result is output.
And then index inquiry is carried out from the source table with the minimum data volume, if the source table with the minimum data volume contains the dimension of the index calculation, the current source table is directly selected for data inquiry, and other source tables are not required to be inquired, so that the inquiry volume is greatly reduced, and the inquiry efficiency is improved.
As an optional embodiment of the present application, optionally, S1 defines the caliber of the same index of each source table, and arranges each source table from small to large according to the data size, and further includes:
And after the source tables are grouped and arranged from small to large according to the data size, respectively marking the sequence numbers of the source tables arranged from small to large.
The source tables are sorted according to the size of the data amount and are marked with corresponding sequence numbers, for example, "sequence number: 1" represents the source table 1 of the regrouped arrangement.
And a serial number is marked, so that the follow-up list and the list in the list according to the serial number are facilitated, and the list looking-up management, the binding management of list data and the like are facilitated.
As an optional embodiment of the present application, optionally, S1 defines the caliber of the same index of each source table, and arranges each source table from small to large according to the data size, and further includes:
binding the marked sequence number with the corresponding source table;
and generating and storing a corresponding source form according to the source form marked by the sequence numbers from small to large.
The method and the device store each ordered source list in a preset blank list according to the serial numbers to generate the source list, integrate each ordered source list, and are convenient for users to use. The index data in each source table can be written into each table in the table in sequence, and each content in the table can be read in sequence subsequently, so that the time is saved.
As shown in fig. 4, as an optional embodiment of the present application, optionally, S2, obtaining a query dimension of an index query condition input by a user, traversing and finding the source table matched with the query dimension, and taking the source table as a target source table, including:
acquiring index query conditions input by a user;
Analyzing query dimensions to obtain the query dimensions in the index query conditions;
Traversing source forms, sequentially identifying analysis dimensions of each source form (the analysis dimensions of the source forms are stored in a database and can be read), and judging whether the analysis dimensions comprise the query dimensions:
if yes, finishing traversing, and marking the source table containing the query dimension in the analysis dimension as the target source table;
otherwise, the traversal is continued.
An Excel-like form structure, wherein a horizontal form may be used to store index data for each source form written, and a vertical form may represent each index (and also the analysis dimension), respectively. And the source form fills the index data of the source form with the sequence number from small to large into the transverse form of each row according to the accelerating direction of the form.
And subsequently, when traversing the source form, sequentially identifying the analysis dimension of each source form in the source form, reading index data of each form from small to large in the form by using the LLM large language model, and acquiring information in each form item of the transverse axis. Therefore, the LLM large language model can be utilized to quickly traverse, and the source table with the minimum data size of the matching analysis dimension can be found according to the table acceleration direction.
Here, the index query flow is described by taking the query index of trade website= "Ningbo financial area branch" & trade channel= "self-service machine" as an example.
Firstly, the index query conditions are defined;
And secondly, analyzing the query dimension, and analyzing from the conditions to obtain the query dimension which needs index query at this time. For example, the query dimension is a trade site and a trade channel;
After the query dimension is found, matching search is needed to be carried out from all the sorted source tables, the source table matched with the query dimension is found and used as the target table of the data query, and thus the query and calculation of the final index data are carried out.
Traversing the query dimension information of each table in the list, identifying and judging whether the analysis dimension contains the query dimension, if so, traversing to a source table containing the query dimension, and recognizing that the source table matches the current query dimension, and can be used as an index database pointed by the query dimension and used for final index data retrieval.
If not, the identification of the next source table is performed.
For example, if the source table marked by the sequence number 1 fails to match, the source table marked by the sequence number 2 is entered for matching. Through matching, if the analysis dimension of the source table marked by the sequence number 2 contains the query dimension for index query at this time, the source table marked by the sequence number 2 is determined to be the target source table designated by the query dimension at this time, and index data required by the query dimension at this time is contained. At this time, the matching is successful, a query request is constructed, and the query and calculation of target index data are performed based on the target source table.
The specific index calculation conditions and the like may be preset in the target source table, and are not limited in this regard.
As an optional embodiment of the present application, optionally, S3, performing index query based on the target source table, and returning a corresponding index calculation result, including:
constructing a query request according to the query dimension;
sending the query request to a plurality of bins;
And responding to the query request through a plurality of bins, performing index query and index calculation based on the target source table, and returning corresponding index calculation results to the user.
When the index is queried, a query request of SQL can be generated and sent to the number of bins, and the number of bins is requested to execute the request and feed back a corresponding index calculation result. The user can view the index calculation result under the query index.
Specifically, the method for automatically constructing the query request and requesting the data warehouse (the data warehouse) to perform data retrieval of the corresponding dimension according to the query dimension by calling LLM (Large Language Model) the large language model comprises the following steps:
1. Understanding query dimensions
‌ Input parsing ‌ the LLM first receives query dimensions entered by the user, which dimensions may be trade sites and trade channels, etc.
‌ Dimension identification ‌. By natural language processing technique, LLM identifies and extracts key dimension information in the query.
2. Building query requests
‌ Query templates ‌ LLM embeds extracted dimension information into query statements according to predefined query templates or rules. These templates may be based on SQL, noSQL, or other query languages. For example, using an SQL query template, an SQL query request can be generated. The SQL query templates may be self-defined or built upon invocation.
‌ Dynamically assembling ‌. According to different query dimensions, LLM can dynamically assemble query sentences to ensure the accuracy and completeness of query.
3. Request number bin
‌ Connect several bins ‌ the LLM establishes a connection with the data warehouse through an API, database connection pool, or other means.
‌ Execute query ‌ to send the constructed query request to the data warehouse and wait for several bins to respond.
4. Processing bin responses
‌ Result analysis ‌, namely receiving the data result returned by the plurality of bins and analyzing the data result for subsequent processing or display.
‌ Exception handling ‌ if exceptions occur during the query (e.g., connection failure, query timeout, etc.), the LLM needs to be able to capture these exceptions and perform corresponding processing such as retries, logging, or notifying the user.
5. Displaying or processing results
‌ Data display ‌, which displays the parsed data results in a user-friendly manner, such as tables, charts and the like.
‌ Post-processing ‌, further processing or analyzing the data according to the business requirements, such as data aggregation, trend prediction, and the like.
The LLM large predictive model feeds back the result corresponding to the request query to the front end of the user, and automatic query is realized.
6. Optimization and iteration
‌ Performance optimization ‌, optimizing query sentences, connection pool configuration and the like according to the execution condition of the query and feedback of a user, and improving query accuracy.
Therefore, the application dynamically matches the available analysis dimension on each source table at one time according to the screening dimension range of the index query condition, and the matching is successful, namely the current source table is selected for data query, and the data table with the minimum data scale can be selected for acceleration query.
By adopting the method, different query scenes can be compatible, and the defect that the multiple fact tables cannot be compatible with query due to inconsistent query dimensions is avoided.
It should be apparent to those skilled in the art that implementing all or part of the above-described embodiments may be accomplished by computer programs to instruct related hardware, and the programs may be stored in a computer readable storage medium, which when executed may include the processes of the embodiments of the controls described above. It will be appreciated by those skilled in the art that implementing all or part of the above-described embodiments may be accomplished by computer programs to instruct related hardware, and the programs may be stored in a computer readable storage medium, which when executed may include the processes of the embodiments of the controls described above. The storage medium may be a magnetic disk, an optical disc, a Read-only memory (ROM), a random access memory (RandomAccessMemory, RAM), a flash memory (flash memory), a hard disk (HARDDISKDRIVE, abbreviated as HDD), a Solid state disk (Solid-state disk-STATEDRIVE, SSD), or the like, and the storage medium may further include a combination of the above types of memories.
Example 2
As shown in fig. 5, based on the implementation principle of embodiment 1, another aspect of the present application provides a management and query device for implementing multiple fact tables with the same index based on an index center, which is used to implement the management and query method for implementing multiple fact tables with the same index based on the index center described in the above embodiment 1, including:
the index definition module is used for defining the same index of each source table in caliber and arranging the source tables from small to large according to the data size;
The execution plan matching module is used for acquiring the query dimension of the index query condition input by the user, traversing and finding the source table matched with the query dimension, and taking the source table as a target source table;
The index query module is used for performing index query based on the target source table and returning a corresponding index calculation result;
The index definition module is in communication connection with the execution plan matching module, and the execution plan matching module is in communication connection with the index query module.
The module functions and the interactive application principle of the device are understood by combining with embodiment 1, and the description of this embodiment is omitted.
The modules or steps of the invention described above may be implemented in a general-purpose computing device, they may be centralized in a single computing device, or distributed across a network of computing devices, or they may alternatively be implemented in program code executable by a computing device, such that they may be stored in a memory device and executed by a computing device, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps within them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
Example 3
As shown in fig. 6, in still another aspect, the present application further provides an electronic device, including:
A processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to implement the method for managing and querying the same index of the multiple fact table based on the index center described in embodiment 1 when executing the executable instructions.
Embodiments of the present disclosure provide for an electronic device that includes a processor and a memory for storing processor-executable instructions. The processor is configured to implement any of the aforementioned methods for managing and querying the same index of the multiple fact table based on the index center when executing the executable instructions.
Here, it should be noted that the number of processors may be one or more. Meanwhile, in the electronic device of the embodiment of the disclosure, an input device and an output device may be further included. The processor, the memory, the input device, and the output device may be connected by a bus, or may be connected by other means, which is not specifically limited herein.
The memory is used as a computer readable storage medium for storing software programs, computer executable programs and various modules, such as a program or a module corresponding to a method for managing and querying the same index of the multiple fact tables based on an index center in the embodiments of the present disclosure. The processor executes various functional applications and data processing of the electronic device by running software programs or modules stored in the memory.
The input device may be used to receive an input number or signal. Wherein the signal may be a key signal generated in connection with user settings of the device/terminal/server and function control. The output means may comprise a display device such as a display screen.
The foregoing description of the embodiments of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the technical improvement of the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims (9)

1.一种基于指标中心实现多事实表同一指标的管理和查询方法,其特征在于,包括如下步骤:1. A method for managing and querying the same indicator of multiple fact tables based on an indicator center, characterized in that it includes the following steps: S1、对各个来源表相同的指标进行口径定义,并按照数据量规模从小到大排列各个所述来源表;S1. Define the caliber of the same indicators of each source table, and arrange the source tables in ascending order according to the data volume; S2、获取用户输入的指标查询条件的查询维度,遍历并找到与所述查询维度相匹配的所述来源表,将其作为目标来源表;S2, obtaining the query dimension of the indicator query condition input by the user, traversing and finding the source table matching the query dimension, and using it as the target source table; S3、基于所述目标来源表进行指标查询,并返回对应的指标计算结果。S3. Perform an indicator query based on the target source table and return the corresponding indicator calculation result. 2.根据权利要求1所述基于指标中心实现多事实表同一指标的管理和查询方法,其特征在于,S1、对各个来源表相同的指标进行口径定义,并按照数据量规模从小到大排列各个所述来源表,包括:2. The method for managing and querying the same indicator of multiple fact tables based on the indicator center according to claim 1 is characterized in that S1, defining the caliber of the same indicator of each source table, and arranging each source table from small to large according to the data volume, including: 设定指标名称和业务口径;Set indicator names and business scope; 从数仓中查找得到对应所述指标名称和所述业务口径的各个所述来源表;Searching and obtaining each of the source tables corresponding to the indicator name and the business scope from the data warehouse; 按照分析维度,对各个所述来源表进行分组;According to the analysis dimension, each of the source tables is grouped; 对分组后的各个所述来源表,按照数据量规模从小到大排列。The grouped source tables are arranged from small to large according to the data volume. 3.根据权利要求2所述基于指标中心实现多事实表同一指标的管理和查询方法,其特征在于,S1、对各个来源表相同的指标进行口径定义,并按照数据量规模从小到大排列各个所述来源表,还包括:3. The method for managing and querying the same indicator of multiple fact tables based on the indicator center according to claim 2 is characterized by: S1, defining the caliber of the same indicator of each source table, and arranging each source table from small to large according to the data volume, and further comprising: 在对分组后的各个所述来源表,按照数据量规模从小到大排列之后,对从小到大排列的各个所述来源表分别进行序号标记。After arranging the grouped source tables from small to large according to the data volume, each source table arranged from small to large is marked with a serial number. 4.根据权利要求3所述基于指标中心实现多事实表同一指标的管理和查询方法,其特征在于,S1、对各个来源表相同的指标进行口径定义,并按照数据量规模从小到大排列各个所述来源表,还包括:4. The method for managing and querying the same indicator of multiple fact tables based on the indicator center according to claim 3 is characterized by: S1, defining the caliber of the same indicator of each source table, and arranging each source table from small to large according to the data volume, and further comprising: 将所标记的所述序号与对应的所述来源表进行绑定;Binding the marked serial number with the corresponding source table; 按照序号从小到大所标记的所述来源表,生成相应的来源表表单并保存。According to the source tables marked by serial numbers from small to large, a corresponding source table form is generated and saved. 5.根据权利要求2所述基于指标中心实现多事实表同一指标的管理和查询方法,其特征在于,S2、获取用户输入的指标查询条件的查询维度,遍历并找到与所述查询维度相匹配的所述来源表,将其作为目标来源表,包括:5. According to claim 2, the method for managing and querying the same indicator of multiple fact tables based on the indicator center is characterized in that S2, obtaining the query dimension of the indicator query condition input by the user, traversing and finding the source table matching the query dimension, and using it as the target source table, comprises: 获取用户输入的指标查询条件;Get the indicator query conditions entered by the user; 进行查询维度解析,获取所述指标查询条件中的所述查询维度;Perform query dimension analysis to obtain the query dimension in the indicator query condition; 遍历来源表表单,有序识别所述来源表表单中各个所述来源表的分析维度,判断所述分析维度中是否包含所述查询维度:Traverse the source table form, identify the analysis dimensions of each source table in the source table form in order, and determine whether the analysis dimension contains the query dimension: 若是,则遍历结束,将所述分析维度中包含所述查询维度的所述来源表,标记为所述目标来源表;If so, the traversal ends, and the source table that includes the query dimension in the analysis dimension is marked as the target source table; 反之,继续遍历。Otherwise, continue traversing. 6.根据权利要求5所述基于指标中心实现多事实表同一指标的管理和查询方法,其特征在于,在设定指标名称和业务口径之时,还包括:6. The method for managing and querying the same indicator of multiple fact tables based on the indicator center according to claim 5 is characterized in that when setting the indicator name and business scope, it also includes: 根据所述查询维度,构建对应所述查询维度下的所述指标名称和所述业务口径。According to the query dimension, construct the indicator name and the business scope corresponding to the query dimension. 7.根据权利要求5所述的一种基于指标中心实现多事实表同一指标的管理和查询方法,其特征在于,S3、基于所述目标来源表进行指标查询,并返回对应的指标计算结果,包括:7. According to claim 5, a method for managing and querying the same indicator of multiple fact tables based on an indicator center is characterized in that S3, querying the indicator based on the target source table and returning the corresponding indicator calculation result, comprises: 根据所述查询维度,构建查询请求;Constructing a query request according to the query dimension; 将所述查询请求发送至数仓;Sending the query request to a data warehouse; 通过数仓响应所述查询请求,基于所述目标来源表进行指标查询和指标计算,并返回相应的指标计算结果至用户。The data warehouse responds to the query request, performs indicator query and indicator calculation based on the target source table, and returns the corresponding indicator calculation results to the user. 8.一种基于指标中心实现多事实表同一指标的管理和查询装置,用于实现权利要求1-7中任一项所述基于指标中心实现多事实表同一指标的管理和查询方法,其特征在于,包括:8. A device for managing and querying the same indicator of multiple fact tables based on an indicator center, used to implement the method for managing and querying the same indicator of multiple fact tables based on an indicator center as claimed in any one of claims 1 to 7, characterized in that it comprises: 指标定义模块,用于对各个来源表相同的指标进行口径定义,并按照数据量规模从小到大排列各个所述来源表;An indicator definition module is used to define the same indicators of various source tables and arrange the source tables in ascending order according to the data volume; 执行计划匹配模块,用于获取用户输入的指标查询条件的查询维度,遍历并找到与所述查询维度相匹配的所述来源表,将其作为目标来源表;An execution plan matching module is used to obtain the query dimension of the indicator query condition input by the user, traverse and find the source table matching the query dimension, and use it as the target source table; 指标查询模块,用于基于所述目标来源表进行指标查询,并返回对应的指标计算结果。The indicator query module is used to perform indicator query based on the target source table and return the corresponding indicator calculation results. 9.一种电子设备,其特征在于,包括:9. An electronic device, comprising: 处理器;processor; 用于存储处理器可执行指令的存储器;a memory for storing processor-executable instructions; 其中,所述处理器被配置为执行所述可执行指令时实现权利要求1-7中任一项所述的一种基于指标中心实现多事实表同一指标的管理和查询方法。The processor is configured to implement a method for managing and querying the same indicator of multiple fact tables based on an indicator center as described in any one of claims 1 to 7 when executing the executable instructions.
CN202411718956.5A 2024-11-28 2024-11-28 Management and query method of the same indicator in multiple fact tables based on indicator center Pending CN119201946A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202411718956.5A CN119201946A (en) 2024-11-28 2024-11-28 Management and query method of the same indicator in multiple fact tables based on indicator center

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202411718956.5A CN119201946A (en) 2024-11-28 2024-11-28 Management and query method of the same indicator in multiple fact tables based on indicator center

Publications (1)

Publication Number Publication Date
CN119201946A true CN119201946A (en) 2024-12-27

Family

ID=94055242

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202411718956.5A Pending CN119201946A (en) 2024-11-28 2024-11-28 Management and query method of the same indicator in multiple fact tables based on indicator center

Country Status (1)

Country Link
CN (1) CN119201946A (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060161544A1 (en) * 2005-01-18 2006-07-20 International Business Machines Corporation System and method for planning and generating queries for multi-dimensional analysis using domain models and data federation
CN106776834A (en) * 2016-11-28 2017-05-31 中通服公众信息产业股份有限公司 A kind of data analysis based on index is from access method and system
CN110674228A (en) * 2019-09-23 2020-01-10 阿里巴巴集团控股有限公司 Data warehouse model construction and data query method, device and equipment
WO2022121560A1 (en) * 2020-12-11 2022-06-16 腾讯科技(深圳)有限公司 Data query method and apparatus, device, and computer-readable storage medium
WO2023138213A1 (en) * 2022-01-19 2023-07-27 广州新科佳都科技有限公司 Data warehouse data query method and apparatus based on subway station system
CN117217933A (en) * 2023-09-06 2023-12-12 中国人民财产保险股份有限公司 Data multidimensional analysis method and device for insurance industry
CN117312354A (en) * 2023-06-30 2023-12-29 平安银行股份有限公司 Data processing method, device, storage medium and equipment
CN117827876A (en) * 2023-12-29 2024-04-05 广电运通集团股份有限公司 Index management system and index query method
CN118964589A (en) * 2024-06-21 2024-11-15 数野科技(深圳)有限公司 A method for constructing an intelligent search engine based on a large language model
CN119003527A (en) * 2024-08-09 2024-11-22 山大地纬软件股份有限公司 Medical insurance service index data storage and query method and system

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060161544A1 (en) * 2005-01-18 2006-07-20 International Business Machines Corporation System and method for planning and generating queries for multi-dimensional analysis using domain models and data federation
CN106776834A (en) * 2016-11-28 2017-05-31 中通服公众信息产业股份有限公司 A kind of data analysis based on index is from access method and system
CN110674228A (en) * 2019-09-23 2020-01-10 阿里巴巴集团控股有限公司 Data warehouse model construction and data query method, device and equipment
WO2022121560A1 (en) * 2020-12-11 2022-06-16 腾讯科技(深圳)有限公司 Data query method and apparatus, device, and computer-readable storage medium
WO2023138213A1 (en) * 2022-01-19 2023-07-27 广州新科佳都科技有限公司 Data warehouse data query method and apparatus based on subway station system
CN117312354A (en) * 2023-06-30 2023-12-29 平安银行股份有限公司 Data processing method, device, storage medium and equipment
CN117217933A (en) * 2023-09-06 2023-12-12 中国人民财产保险股份有限公司 Data multidimensional analysis method and device for insurance industry
CN117827876A (en) * 2023-12-29 2024-04-05 广电运通集团股份有限公司 Index management system and index query method
CN118964589A (en) * 2024-06-21 2024-11-15 数野科技(深圳)有限公司 A method for constructing an intelligent search engine based on a large language model
CN119003527A (en) * 2024-08-09 2024-11-22 山大地纬软件股份有限公司 Medical insurance service index data storage and query method and system

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
LI, YC 等: "COLTR: Semi-Supervised Learning to Rank With Co-Training and Over-Parameterization for Web Search", IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 13 March 2024 (2024-03-13) *
宋爱波;万雨桐;贡欢;薛荧荧;: "海量多维数据的存储与查询研究", 计算机工程与应用, no. 13, 5 April 2016 (2016-04-05) *
李志义;黄子风;许晓绵;: "基于表示学习的跨模态检索模型与特征抽取研究综述", 情报学报, no. 04, 24 April 2018 (2018-04-24) *
黄学文;孔祥波;惠靖;马雪丽;: "基于自动累加表的查询优化技术", 软件工程师, no. 11, 15 November 2010 (2010-11-15) *

Similar Documents

Publication Publication Date Title
CN112199366B (en) Data table processing method, device and equipment
US9141665B1 (en) Optimizing search system resource usage and performance using multiple query processing systems
US9218379B2 (en) Method, apparatus, and computer-readable medium for efficiently performing operations on distinct data values
US9652498B2 (en) Processing queries using hybrid access paths
US10162855B2 (en) Systems and methods for optimizing data analysis
KR102034679B1 (en) A data input/output system using grid interface
US20050091199A1 (en) Method and system for generating SQL joins to optimize performance
US12292887B2 (en) Columnar cache query using hybrid query execution plan
US20240221039A1 (en) Auto-mated price performance offers for cloud database systems
CN1987861A (en) System and method for processing database query
CN114116773A (en) Structured query language SQL text review method and device
CN113360517A (en) Data processing method and device, electronic equipment and storage medium
CN117009417B (en) Analysis report generation method, device, system and storage medium
CN114564621A (en) A method, apparatus, device and readable storage medium for associating data
CN120743914B (en) Management methods, devices, and storage media for coverage-based indexed databases
CN109388659A (en) Date storage method, device and computer readable storage medium
CN114416750B (en) A modeling storage method, device and medium for electronic spreadsheet
RU2711717C1 (en) Recruitment system using machine learning and downsizing of multidimensional data and a method for recruiting personnel using machine learning and lowering the dimension of multidimensional data
CN117648339B (en) Data exploration method and device, server and storage medium
JP5033322B2 (en) Information management method and apparatus using connection relation information
CN119201946A (en) Management and query method of the same indicator in multiple fact tables based on indicator center
CN109766388B (en) Method and system for storing data into HBase
CN115809268B (en) Adaptive query method and device based on fragment index
CN113625967B (en) Data storage method, data query method and server
CN115544163A (en) Cluster database sub-database management method, device, computer terminal and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination