CN113688151B - Data access method, device, system, equipment and medium based on virtual database - Google Patents
Data access method, device, system, equipment and medium based on virtual database Download PDFInfo
- Publication number
- CN113688151B CN113688151B CN202110980369.3A CN202110980369A CN113688151B CN 113688151 B CN113688151 B CN 113688151B CN 202110980369 A CN202110980369 A CN 202110980369A CN 113688151 B CN113688151 B CN 113688151B
- Authority
- CN
- China
- Prior art keywords
- data
- query
- virtual database
- sql
- preset rules
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
- G06F16/2445—Data retrieval commands; View definitions
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
- G06F16/24554—Unary operations; Data partitioning operations
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/256—Integrating or interfacing systems involving database management systems in federated or virtual databases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本公开提供了一种基于虚拟数据库的数据访问方法、装置、系统、设备、介质和程序产品，涉及数据库技术领域。该方法包括通过接收用户层的SQL查询请求；对SQL查询请求进行语句分析，基于语句分析的结果调用规则库中的预设规则将SQL查询请求分解为多个局部查询语句，并将局部查询语句路由分发到对应的数据库以获取相应的查询数据；以及对查询数据进行组装后向用户层输出查询结果。本公开中的基于虚拟数据库的数据访问方法中解决了现有技术中针对不同的数据库编写不同的代码逻辑完成相应的连接和访问操作而导致代码编写量大，请求端业务逻辑复杂，维护困难的问题。
The present disclosure provides a data access method, device, system, equipment, medium and program product based on a virtual database, and relates to the field of database technology. The method includes receiving a SQL query request from a user layer; performing statement analysis on the SQL query request, calling a preset rule in a rule library based on the result of the statement analysis to decompose the SQL query request into multiple local query statements, and routing and distributing the local query statements to the corresponding database to obtain corresponding query data; and assembling the query data and outputting the query results to the user layer. The data access method based on a virtual database in the present disclosure solves the problem that in the prior art, different code logic is written for different databases to complete the corresponding connection and access operations, resulting in a large amount of code writing, complex request-side business logic, and difficult maintenance.
Description
Technical Field
The present disclosure relates to the field of database technologies, and more particularly, to a method, apparatus, system, device, medium, and program product for accessing data based on a virtual database.
Background
The internal logic of the mainstream large-scale enterprise-level application is increasingly complex, and the types of used databases are also relatively large due to history or upgrading reasons, such as mysql, oracle, informix and the like. If the data requesting party is to realize the connection access to the heterogeneous databases, different code logic is written in the application program for different databases to complete the corresponding connection and access operations. The code writing amount is large, the service logic of the request end is complex, and the maintenance is difficult.
Disclosure of Invention
In view of the foregoing, the present disclosure provides a data access method, apparatus, system, device, medium, and program product for a virtual database.
According to a first aspect of the present disclosure, there is provided a data access method of a virtual database, applied to a virtual database layer, including: receiving an SQL query request of a user layer; performing statement analysis on the SQL query request, calling a preset rule in a rule base based on the statement analysis result to decompose the SQL query request into a plurality of local query statements, and distributing the local query statements to a corresponding database by routes to acquire corresponding query data; and outputting a query result to the user layer after assembling the query data.
In certain embodiments, further comprising: generating a series of preset rules by adopting a deep learning algorithm; and creating a rule base based on a series of the preset rules.
In some embodiments, said generating a series of said preset rules using a deep learning algorithm comprises: creating a structured data table; extracting data information in heterogeneous data sources and storing the data information in the structured data table; performing aggregation matching on the data information in the structured data table by adopting a deep learning algorithm, and determining the similarity among different data tables; and performing corresponding rule definition according to the similarity to form the preset rule.
In certain embodiments, the data information includes a library name, a table name, and table structure information including column names, column attributes, constraints, and table data in a data table.
In some embodiments, the preset rule includes associating isomorphic table operations of heterogeneous data sources with homoattribute data extraction association operations of non-isomorphic tables.
In certain embodiments, further comprising: and checking the similarity by adopting a manual error marking method, wherein the checking the similarity by adopting the manual error marking method comprises the following steps:
Uploading the similarity and the relationship view corresponding to the similar field to a control center; and judging the accuracy of the similarity according to the relationship view corresponding to the similarity field by an administrator of the control center.
In certain embodiments, further comprising: and writing the differentiated SQL grammar rules of the heterogeneous data sources into the local rule base.
In some embodiments, the invoking the preset rules in the rule base based on the results of the statement analysis to decompose the SQL query request into a plurality of local query statements includes: matching the preset rule according to the statement analysis result; determining a data table in the heterogeneous data source to be accessed according to the preset rule obtained by matching; determining a corresponding differentiated SQL grammar rule according to the type of the heterogeneous data source; and decomposing the SQL query request into a plurality of local query statements based on the table structure information of the data table and the differentiated SQL grammar rules.
In some embodiments, the outputting the query result to the user layer after assembling the query data includes: and grouping and sequencing the queried data according to the constraint rule of the SQL query request to form the query result.
A second aspect of the present disclosure provides a virtual database-based data access apparatus, comprising: the receiving module is used for receiving the SQL query request of the user layer; the SQL statement distributing routing module is used for carrying out statement analysis on the SQL query request, calling a preset rule in a rule base based on the statement analysis result to decompose the SQL query request into a plurality of local query statements, and distributing the local query statements to a corresponding database in a routing way so as to acquire corresponding query data; and the data assembling module is used for outputting the query result to the user layer after assembling the query data.
In certain embodiments, further comprising: the wrapper module is used for generating a series of preset rules by adopting a deep learning algorithm; and the rule base module is used for writing the preset rule.
In certain embodiments, further comprising: and the control center module is used for checking the similarity by adopting a manual error marking method.
A third aspect of the present disclosure provides a virtual database-based data access system, comprising: a virtual database layer for executing the virtual database-based data access method as set forth in any one of the above; the user layer is used for sending an SQL query request to the virtual database layer; and the heterogeneous database layer is used for storing the data source and allowing the virtual database layer to access and inquire.
A fourth aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the virtual database-based data access method described above.
The fifth aspect of the present disclosure also provides a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the virtual database-based data access method described above.
A sixth aspect of the present disclosure also provides a computer program product comprising a computer program which, when executed by a processor, implements the virtual database based data access method described above.
The data access method based on the virtual database receives an SQL query request of a user layer; performing statement analysis on the SQL query request, calling a preset rule in a rule base based on the statement analysis result to decompose the SQL query request into a plurality of local query statements, and distributing the local query statements to a corresponding database by routes to acquire corresponding query data; and outputting the query result to the user layer after assembling the query data. According to the data access method based on the virtual database, data access to the bottom heterogeneous data source is completed in the virtual database layer, meanwhile, the result obtained by the access is assembled and fed back to the user layer in a unified mode, and the problems that in the prior art, the code writing amount is large, the service logic of a request end is complex and maintenance is difficult due to the fact that different code logics are written for different databases to complete corresponding connection and access operations are avoided.
Drawings
The foregoing and other objects, features and advantages of the disclosure will be more apparent from the following description of embodiments of the disclosure with reference to the accompanying drawings, in which:
FIG. 1 schematically illustrates an application scenario diagram of a virtual database-based data access method, apparatus, device, medium and program product according to an embodiment of the present disclosure;
FIG. 2 schematically illustrates a flow chart of a virtual database based data access method according to an embodiment of the present disclosure;
FIG. 3 schematically illustrates another flow diagram of a virtual database based data access method according to an embodiment of the present disclosure;
FIG. 4 schematically illustrates a block diagram of a virtual database based data access system in accordance with an embodiment of the present disclosure;
FIG. 5 schematically illustrates a block diagram of a virtual database based data access apparatus according to an embodiment of the present disclosure;
FIG. 6 schematically illustrates a block diagram of a computer system suitable for implementing a virtual database based data challenge method in accordance with an embodiment of the present disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is only exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.
Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a convention should be interpreted in accordance with the meaning of one of skill in the art having generally understood the convention (e.g., "a system having at least one of A, B and C" would include, but not be limited to, systems having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
Detailed background art may include other technical problems in addition to the technical problem that is solved by the exclusive right.
The embodiment of the disclosure provides a data access method based on a virtual database, which comprises the following steps: receiving an SQL query request of a user layer; performing statement analysis on the SQL query request, calling a preset rule in a rule base based on the statement analysis result to decompose the SQL query request into a plurality of local query statements, and distributing the local query statements to a corresponding database by routes to acquire corresponding query data; and outputting the query result to the user layer after assembling the query data.
It should be noted that, the data query preventing method of the virtual database provided by the embodiment of the disclosure may be used for the aspects related to the transmission of the micro-service data by the big data and the distributed technology, and may also be used in various fields other than the big data and the distributed technology, such as the financial field.
The data access method based on the virtual database in the embodiment of the disclosure is particularly suitable for storing the queried data in heterogeneous data sources when the user side performs data query, and can improve the data query efficiency when multiple types of databases to be accessed exist.
Fig. 1 schematically illustrates an application scenario diagram of a virtual database-based data access method according to an embodiment of the present disclosure.
As shown in fig. 1, the application scenario 100 according to this embodiment may include user layer terminal devices 101, 102, 103. The virtual database 104 is used to provide queries and output of data between the terminal devices 101, 102, 103 and the plurality of databases 105.
The user may interact with the server 105 through the virtual database 104 using the terminal devices 101, 102, 103 to query data or the like. Various communication client applications, such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only) may be installed on the terminal devices 101, 102, 103.
The terminal devices 101, 102, 103 may be a variety of electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The type of database 105 may be mysql, oracle, informix, with data stored inside the database 105 for review by the user.
It should be noted that, the virtual database-based data access method provided in the embodiments of the present disclosure may be generally performed by the virtual database layer 104. The virtual database layer receives the data query requests sent by the terminal devices 101, 102 and 103 of the user layer, acquires the data queried by the user in the multi-database 105 through the analysis processing of the query requests, assembles the queried data to form a query result, and returns the query result to the user layer for the user to check.
It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
The virtual database-based data access method according to the embodiment of the present disclosure will be described in detail below with reference to the scenario described in fig. 1, by referring to fig. 2 to 3.
Fig. 2 schematically illustrates a flow chart of a virtual database based data method according to an embodiment of the present disclosure.
As shown in fig. 2, the virtual database-based data access method of this embodiment includes operations S210 to S230.
In operation S210, an SQL query request of a user layer is received.
The user layer is the initiator of the data demand, a user in the user layer inputs a query request on a user operation interface in an application program according to the data query requirement, and the query request is processed and then an SQL query request based on a database management language is sent to the virtual database layer. The SQL query request may be to query a specific type of table structure, to query all columns in a specified data table, to query a specified column or row in a specified data table, to query a specified data table for empty/non-empty field contents or to order query results by field values, etc.
It will be appreciated that the user layer may be a personal mobile terminal device of a client, such as a cell phone, a notebook computer or a desktop computer, or may be a workstation of an enterprise client, or the like. And establishing a data transmission channel between the heterogeneous data sources of the user layer and the bottom layer based on a network. The type of network may be a wired network, a wireless communication network, etc. Related application programs of the user layer can access the virtual database through unified SQL and XML interfaces so as to realize the transmission of SQL query requests to the virtual database.
In operation S220, the SQL query request is subjected to statement analysis, a preset rule in a rule base is called based on the result of the statement analysis, the SQL query request is decomposed into a plurality of local query statements, and the local query statements are distributed to a corresponding database by routes to obtain corresponding query data.
Operations S221 to S222 also need to be performed before operation S220 is performed, as shown in fig. 3.
In operation S221: generating a series of preset rules by adopting a deep learning algorithm;
In the embodiment of the disclosure, the structured data table is created and serves as a data table for temporarily storing data, the mapping of the structured data table stored in the database is used for extracting data information in heterogeneous data sources, such as a common for loop, and a corresponding table is derived from the database. And storing the data information in a structured data table, the data information of the present disclosure including a library name, a table name, and table structure information including column names, column attributes, constraints, and table data in the data table, examples being: the data information of the table A1 is exported from the database A to a structured data table, the library name of the database A and the table name of the table A1 need to be copied, the table structure information in the table A1 is copied to the structured data table, and the structured data table is created with corresponding field names, field types, field lengths, data records and the like to classify and record the data information in the above. And carrying out aggregation matching on the data information in the structured data table by adopting a deep learning algorithm, and determining the similarity among different data tables. As described above, the data information of the data tables of different databases is stored in the structured data table, and the data relationship between different data tables can be determined through the process of aggregation matching. Examples are: the field names, field types, field lengths and the like of the data table A1 in the database A and the data table B1 in the database B are the same, only the recorded data contents are different, at this time, the data table A1 and the data table B1 are the same type through the process of collecting and matching, rules for associating isomorphic table operation of heterogeneous data sources can be created to associate the data table A1 with the data table B1, and when a user inquires the data tables with the table structures, the data table A1 and the data table B1 can be simultaneously obtained. Again, for example: the field names, field types, field lengths and the like of the data table A1 in the database A and the data table B1 in the database B are different, but the recorded data contents are the same, at the moment, the data table A1 and the data table B1 can be obtained through the process of collecting and matching, the data table A1 and the data table B1 can be associated by creating rules for associating heterogeneous data source different isomorphic table operations, and when a user inquires the data records in the data table A1 and the data record recorded in the data table B1 can be simultaneously obtained. In the judging process, different field names, field types, data records and the like are required to be matched one by one to confirm the similarity between two data tables, and then corresponding rule definition is carried out according to the similarity to form a preset rule.
It may be appreciated that, in order to ensure accuracy of the deep learning result, the embodiment of the disclosure further includes checking the similarity by using a manual error correction method, where the checking the similarity by using the manual error correction method includes: uploading the relationship view corresponding to the similarity and the similar field to a control center; and judging the accuracy of the similarity by an administrator of the control center according to the relationship view corresponding to the similarity field, displaying the names of the current two or more data tables and the database where the data tables are located on the relationship view, and simultaneously displaying a similarity result and obtaining the similarity field matched with the result so as to be used for the administrator to perform artificial similarity correction.
It can be understood that, as described above, data is obtained by querying a plurality of databases at the bottom layer, access grammar rules of different databases are different, in order to facilitate access of a virtual database layer to the databases, in an embodiment of the disclosure, differentiated SQL grammar rules of heterogeneous data sources are written into a rule base, and the grammar rules corresponding to the existing databases can be recorded into the rule base by counting the grammar rules of the databases with database names as identifiers, so that when it is determined that a data path needs to be accessed, the corresponding grammar rules of a caller are invoked, thereby improving the access efficiency.
In operation S222, a rule base is created based on a series of preset rules. It should be noted that, the rule base here corresponds to a small database built in the virtual database layer, and the creation process thereof is consistent with the creation process of the database, which is not described herein. The built-in rules are equivalent to the data tables in the database, and when the rules are called, the corresponding preset rules are queried in the rule base and used according to the statement analysis result as an index. The creation of the preset RULEs in the RULE base may be performed by various commands, such as CREATE RULE command, ENTERPRISE MANAGER command, etc. The created rule includes the name, creation time, owner, etc. of the rule to facilitate the invocation of the subsequent preset rule.
Returning to operation S220, performing statement analysis on the SQL query request, invoking a preset rule in the rule base based on a result of the statement analysis to decompose the SQL query request into a plurality of local query statements, and distributing the local query statements to the corresponding databases by routes to obtain corresponding query data, wherein the steps include:
According to the statement analysis result, the preset rules are matched, for example, the statement analysis result contains field names, field types and the like of data to be queried, and all the preset rules are traversed according to the field names and the field types to find out the matched preset rules. The tool for analyzing the sentences can adopt an SQL parser built in the database, which parses the SQL sentences into parse trees, and completes preprocessing of grammar sequences, keywords of query contents and the like.
And determining data tables in heterogeneous data sources to be accessed according to preset rules obtained by matching, wherein the data required to be queried for the SQL query request can be distributed in a plurality of data tables according to the preset rules, and the data tables can be in the same database or can exist in different databases. When the data table exists in different databases, the library names of the databases where the data table exists can be obtained according to preset rules, and corresponding differentiated SQL grammar rules are determined according to the library names of the databases. The SQL query request is then broken down into a plurality of local query statements based on the table structure information of the data table and the differentiated SQL grammar rules. The local query statement routes are then distributed to the corresponding databases to obtain the corresponding query data. The query data acquisition process is the same as the method of accessing the database by using SQL sentences, and will not be described in detail.
In operation S230, the query data is assembled and then the query result is output to the user layer. And assembling the data queried from the database according to the constraint rules in the SQL query request, and outputting a query result required by a user.
Based on the data access method based on the virtual database, the disclosure also provides a data access device of the virtual database. The device will be described in detail below in connection with fig. 5.
Fig. 5 schematically illustrates a block diagram of a virtual database-based data access apparatus according to an embodiment of the present disclosure.
As shown in fig. 5, the embodiment 300 includes a receiving module 301, an sql statement distribution routing module 302, and a data assembling module 303.
A receiving module 301, configured to receive an SQL query request of a user layer, and adapted to execute step S210 in the foregoing description;
The SQL statement distributing routing module 302 is configured to perform statement analysis on the SQL query request, invoke a preset rule in the rule base based on a result of the statement analysis to decompose the SQL query request into a plurality of local query statements, and route and distribute the local query statements to a corresponding database to obtain corresponding query data, which is suitable for executing step 220 in the foregoing description;
The data assembling module 303 is configured to output the query result to the user layer after assembling the query data, and is adapted to perform step 230.
For example, the virtual database-based data access method according to the embodiment of the present disclosure further includes: the wrapper module is configured to generate a series of preset rules using a deep learning algorithm, and is adapted to perform step 221.
For example, the virtual database-based data access method according to the embodiment of the present disclosure further includes: the rule base module is configured to write a preset rule, and is adapted to execute step 222.
For example, the virtual database-based data access method according to the embodiment of the present disclosure further includes: and the control center module is used for checking the similarity by adopting a manual error marking method.
The data access method based on the virtual database receives an SQL query request of a user layer; performing statement analysis on the SQL query request, calling a preset rule in a rule base based on the statement analysis result to decompose the SQL query request into a plurality of local query statements, and distributing the local query statements to a corresponding database by routes to acquire corresponding query data; and outputting the query result to the user layer after assembling the query data. According to the data access method based on the virtual database, data access to the bottom heterogeneous data source is completed in the virtual database layer, meanwhile, the result obtained through the access is assembled and then fed back to the user layer in a unified mode, and the problems that in the prior art, the code writing amount is large, the service logic of a request end is complex and maintenance is difficult due to the fact that different code logics are written for different databases to complete corresponding connection and inquiry preventing operations are avoided. Meanwhile, the data access method based on the virtual database has the following advantages: the access channel does not need to control the connection and access strategy of each heterogeneous data source, the logic complexity is reduced, and the maintainability is improved; the data access and integration strategies are uniformly put in the virtual database layer, so that the coupling degree of channels and downstream data sources is reduced; the unified management of data access is realized, the expansibility of external service is improved, and the virtual database layer can integrate and access the data of various heterogeneous data sources, so that a complete basic platform is provided for the subsequent business scene expansion based on big data.
According to embodiments of the present disclosure, any of the receiving module 301, the sql statement distribution routing module 302, and the data assembling module 303 may be incorporated in one module to be implemented, or any of the modules may be split into a plurality of modules. Or at least some of the functionality of one or more of the modules may be combined with, and implemented in, at least some of the functionality of other modules. According to embodiments of the present disclosure, at least one of the receiving module 301, the sql statement distribution routing module 302, and the data assembling module 303 may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in hardware or firmware in any other reasonable way of integrating or packaging the circuits, or in any one of or a suitable combination of any of the three. Or the receiving module 301, the sql statement distribution routing module 302, at least one of the data assembling modules 303 may be at least partially implemented as a computer program module which, when executed, may perform the corresponding functions.
Fig. 4 schematically illustrates a block diagram of a virtual database-based data access system according to an embodiment of the present disclosure.
As shown in fig. 4, this embodiment includes: a virtual database layer for executing the data access method based on the virtual database in the above embodiment; the virtual database layer in this embodiment includes an SQL statement distribution routing module, the SQL query request sent by the user layer is received by the SQL statement distribution routing module, the SQL statement distribution routing module performs statement analysis on the SQL query request, invokes a preset rule in a rule base according to a result of the statement analysis to decompose the SQL query into a plurality of local query statements, realizes access to a plurality of databases through the local query statements to return query data, the returned query-proof result enters the data assembly module from the SQL statement distribution routing module, and the data assembly module performs operations such as ordering on the query data according to constraint rules of the SQL query request to form a query result and sends the query result to the user layer. The preset rules are obtained by the packer module adopting a deep learning algorithm to perform aggregation matching of different bottom databases.
The user layer is used for sending an SQL query request to the virtual database layer;
And the heterogeneous database layer is used for storing the data source and providing the virtual database layer with access inquiry.
Fig. 6 schematically illustrates a block diagram of an electronic device adapted to implement a virtual database based data access method according to an embodiment of the present disclosure.
As shown in fig. 6, an electronic device 400 according to an embodiment of the present disclosure includes a processor 401 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 402 or a program loaded from a storage section 408 into a Random Access Memory (RAM) 403. The processor 401 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. Processor 401 may also include on-board memory for caching purposes. Processor 401 may include a single processing unit or multiple processing units for performing different actions of the method flows in accordance with embodiments of the disclosure.
In the RAM 403, various programs and data necessary for the operation of the electronic device 400 are stored. The processor 401, the ROM 402, and the RAM 403 are connected to each other by a bus 404. The processor 401 performs various operations of the method flow according to the embodiment of the present disclosure by executing programs in the ROM 402 and/or the RAM 403. Note that the program may be stored in one or more memories other than the ROM 402 and the RAM 403. The processor 401 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in one or more memories.
According to an embodiment of the present disclosure, electronic device 400 may also include an input/output (I/O) interface 405, with input/output (I/O) interface 405 also connected to bus 404. Electronic device 400 may also include one or more of the following components connected to I/O interface 405: an input section 406 including a keyboard, a mouse, and the like; an output portion 407 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker, and the like; a storage section 408 including a hard disk or the like; and a communication section 405 including a network interface card such as a LAN card, a modem, or the like. The communication section 405 performs communication processing via a network such as the internet. The drive 410 is also connected to the I/O interface 405 as needed. A removable medium 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed on the drive 410 as needed, so that a computer program read therefrom is installed into the storage section 408 as needed.
The present disclosure also provides a computer-readable storage medium that may be embodied in the apparatus/device/system described in the above embodiments; or may exist alone without being assembled into the apparatus/device/system. The computer-readable storage medium carries one or more programs which, when executed, implement methods in accordance with embodiments of the present disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example, but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, the computer-readable storage medium may include ROM 402 and/or RAM 403 and/or one or more memories other than ROM 402 and RAM 403 described above.
Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowcharts. The program code, when executed in a computer system, causes the computer system to implement the item recommendation method provided by embodiments of the present disclosure.
The above-described functions defined in the system/apparatus of the embodiments of the present disclosure are performed when the computer program is executed by the processor 401. The systems, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.
In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted, distributed in the form of signals over a network medium, and downloaded and installed via the communication part 405, and/or installed from the removable medium 411. The computer program may include program code that may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 405 and/or installed from the removable medium 411. The above-described functions defined in the system of the embodiments of the present disclosure are performed when the computer program is executed by the processor 401. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.
According to embodiments of the present disclosure, program code for performing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, such computer programs may be implemented in high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. Programming languages include, but are not limited to, such as Java, c++, python, "C" or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will appreciate that the features recited in the various embodiments of the disclosure and/or in the claims may be provided in a variety of combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the disclosure. In particular, the features recited in the various embodiments of the present disclosure and/or the claims may be variously combined and/or combined without departing from the spirit and teachings of the present disclosure. All such combinations and/or combinations fall within the scope of the present disclosure.
The embodiments of the present disclosure are described above. These examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the disclosure, and such alternatives and modifications are intended to fall within the scope of the disclosure.
Claims (14)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110980369.3A CN113688151B (en) | 2021-08-25 | 2021-08-25 | Data access method, device, system, equipment and medium based on virtual database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110980369.3A CN113688151B (en) | 2021-08-25 | 2021-08-25 | Data access method, device, system, equipment and medium based on virtual database |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113688151A CN113688151A (en) | 2021-11-23 |
| CN113688151B true CN113688151B (en) | 2024-11-26 |
Family
ID=78582397
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110980369.3A Active CN113688151B (en) | 2021-08-25 | 2021-08-25 | Data access method, device, system, equipment and medium based on virtual database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113688151B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115185979A (en) * | 2022-07-26 | 2022-10-14 | 中国工商银行股份有限公司 | Data access routing method, apparatus device, storage medium and program product |
| CN115423633A (en) * | 2022-09-29 | 2022-12-02 | 中国建设银行股份有限公司 | Transaction data processing method, device, electronic device and medium |
| CN117453731A (en) * | 2023-12-22 | 2024-01-26 | 北京宇信科技集团股份有限公司 | Multi-source data query system and multi-source data query method |
| CN119128004A (en) * | 2024-08-22 | 2024-12-13 | 中国电信股份有限公司技术创新中心 | Data caching method, system, device and medium |
| CN119719141A (en) * | 2025-02-24 | 2025-03-28 | 京东科技信息技术有限公司 | Data query method, device and system |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107491510A (en) * | 2017-08-03 | 2017-12-19 | 国网江苏省电力公司信息通信分公司 | One kind mixing heterogeneous data source unified query system and distributed enquiring method |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101290570A (en) * | 2007-04-17 | 2008-10-22 | 高丹 | Heterogeneous database data integration middleware in different language environment |
| CN102521254B (en) * | 2011-11-17 | 2013-04-10 | 广东电网公司电力科学研究院 | Uniform access method of isomeric database |
| CN104008135A (en) * | 2014-05-07 | 2014-08-27 | 南京邮电大学 | Multi-source heterogeneous database fusion system and data query method thereof |
| CN105786946A (en) * | 2015-12-24 | 2016-07-20 | 国云科技股份有限公司 | Virtualized data integration and query system and implementation method thereof |
-
2021
- 2021-08-25 CN CN202110980369.3A patent/CN113688151B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107491510A (en) * | 2017-08-03 | 2017-12-19 | 国网江苏省电力公司信息通信分公司 | One kind mixing heterogeneous data source unified query system and distributed enquiring method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113688151A (en) | 2021-11-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113688151B (en) | Data access method, device, system, equipment and medium based on virtual database | |
| US10007698B2 (en) | Table parameterized functions in database | |
| CN109492053B (en) | Method and apparatus for accessing data | |
| CN112148509A (en) | Data processing method, device, server and computer readable storage medium | |
| JP2021511582A (en) | Dimensional context propagation technology for optimizing SQL query plans | |
| US20100228764A1 (en) | Offline Validation of Data in a Database System for Foreign Key Constraints | |
| CN107291471B (en) | Meta-model framework system supporting customizable data acquisition | |
| US20150269234A1 (en) | User Defined Functions Including Requests for Analytics by External Analytic Engines | |
| CN105164674A (en) | Queries involving multiple databases and execution engines | |
| US20110125773A1 (en) | Logical Object Search Framework and Application Programming Interface | |
| US11275806B2 (en) | Dynamic materialization of feeds for enabling access of the feed in an online social network | |
| CN111427971A (en) | Business modeling method, device, system and medium for computer system | |
| US9652740B2 (en) | Fan identity data integration and unification | |
| CN112650526B (en) | Method, device, electronic equipment and medium for detecting version consistency | |
| CN116594683A (en) | Code annotation information generation method, device, equipment and storage medium | |
| US20250238744A1 (en) | System and method for leveraging a completeness graph | |
| US9330140B1 (en) | Transient virtual single tenant queries in a multi-tenant shared database system | |
| CN116414707A (en) | Test data execution method, device, equipment, medium and product based on API (application program interface) document | |
| CN117009397A (en) | Data query method, data query device, electronic equipment and storage medium | |
| US10289620B1 (en) | Reporting and data governance management | |
| US20140122158A1 (en) | License Meta-Model for Automatic License Reasoning | |
| US20150169675A1 (en) | Data access using virtual retrieve transformation nodes | |
| CN119149558B (en) | Distributed database shard key update method, device, equipment and medium | |
| CN111949259B (en) | Risk decision configuration method, system, electronic device and storage medium | |
| US9053151B2 (en) | Dynamically joined fast search views for business objects |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |