CN121326973A

CN121326973A - Method, device, equipment and medium for executing database operation statement

Info

Publication number: CN121326973A
Application number: CN202511845423.8A
Authority: CN
Inventors: 朱大治; 朱毓堃; 邹斌
Original assignee: iFlytek Co Ltd
Current assignee: iFlytek Co Ltd
Priority date: 2025-12-09
Filing date: 2025-12-09
Publication date: 2026-01-13

Abstract

This invention relates to the field of low-code platform technology, and provides a method, apparatus, device, and medium for executing database operation statements. The method includes: parsing the semantics of an original statement initiated by a low-code platform and identifying the business processing attributes corresponding to the original statement; distributing the original statement to a target database matching the business processing attributes, wherein the target database is a transactional database for transaction processing or an analytical database for data analysis; rewriting the original statement based on the syntax rules of the target database to generate a target statement adapted to the target database, and executing the target statement. This invention solves the performance bottleneck and single point of failure risk of a single database under large data volumes, and avoids complex analysis queries from dragging down core transaction businesses.

Description

Method, device, equipment and medium for executing database operation statement

Technical Field

The present invention relates to the field of low code platform technologies, and in particular, to a method, an apparatus, a device, and a medium for executing database operation statements.

Background

Current low code platforms typically support only a single data source or hide the data source internally, lacking flexible support for heterogeneous databases. In the face of complex business, a single online transaction (Online Transaction Processing, OLTP) database is often forced to assume both transaction and analysis tasks. When the data volume is increased rapidly, a single database is easy to become a performance bottleneck and even single-point faults occur, and the CPU is raised suddenly due to forced execution of complex analysis query on the OLTP database, so that the whole business system is towed down, and the real-time performance of transaction processing and the high-efficiency query requirement of online analysis processing (Online Analytical Processing, OLAP) cannot be considered.

Disclosure of Invention

The invention provides a method, a device, equipment and a medium for executing database operation sentences, which are used for solving the defects in the prior art.

The invention provides an execution method of database operation sentences, which comprises the following steps:

analyzing the semantics of an original sentence initiated by a low-code platform, and identifying the business processing attribute corresponding to the original sentence;

distributing the original sentence to a target database matched with the business processing attribute according to the business processing attribute, wherein the target database is a transaction type database for transaction processing or an analysis type database for data analysis;

And based on the grammar rule of the target database, carrying out grammar rewriting on the original sentence, generating a target sentence adapting to the target database, and executing the target sentence.

According to the method for executing the database operation statement provided by the invention, the semantic meaning of the original statement is analyzed, and the service processing attribute corresponding to the original statement is identified, which comprises the following steps:

Generating an abstract syntax tree of the original sentence;

traversing nodes in the abstract syntax tree, and identifying operation instruction types represented by each node;

Under the condition that the operation instruction type represented by any node is data change or single form query, determining the service processing attribute corresponding to the original statement as transaction operation;

and under the condition that the operation instruction type represented by any node is multidimensional aggregation, multi-table association or complex ordering, determining the service processing attribute corresponding to the original statement as analysis type query.

According to the method for executing the database operation statement provided by the invention, when the original statement is a data manipulation statement or a data query statement, the method for executing grammar rewriting on the original statement based on grammar rules of the target database to generate the target statement adapted to the target database comprises the following steps:

traversing the abstract syntax tree, and identifying source nodes to be converted which are incompatible with the syntax rules of the target database;

Retrieving a dialect mapping table of the target database, and acquiring a target equivalent node corresponding to the source node to be converted from the dialect mapping table, wherein the dialect mapping table comprises mapping relations between a general grammar node and a special grammar node of the target database;

and replacing the source node to be converted in the abstract syntax tree by the target equivalent node, and converting the replaced abstract syntax tree to generate the target sentence.

According to the method for executing the database operation statement provided by the invention, the method for replacing the source node to be converted in the abstract syntax tree by the target equivalent node comprises the following steps:

acquiring byte codes of component classes for executing the original sentence;

modifying the byte code, and inserting a node replacement instruction aiming at an abstract syntax tree into the processing logic of the component class;

And loading and running the modified byte code to trigger the execution of the node replacement instruction so as to update the source node to be converted in the abstract syntax tree into the target equivalent node.

According to the execution method of the database operation statement provided by the invention, before executing the original statement initiated by the receiving low-code platform, the execution method further comprises the following steps:

synchronizing the data of the dispersed heterogeneous transaction databases to an analytical database;

And establishing a mapping relation between a data model and a physical data source in a metadata base of the low-code platform, wherein the mapping relation comprises a first corresponding relation between a data model identification and a database instance of the transaction type database and a second corresponding relation between the data model identification and a data table of the analysis type database.

According to the method for executing the database operation statement provided by the invention, after the mapping relation between the data model and the physical data source is established in the metadata base of the low-code platform, the method further comprises the following steps:

Encrypting and storing the connection information and authentication credentials of the transaction database and the analysis database in a configuration center of the low-code platform;

Acquiring a change message of the connection information in real time;

And under the condition that the change message is acquired, dynamically updating the database connection instance and the mapping relation in the running memory of the low-code platform in the process that the low-code platform keeps the running state.

According to the execution method of the database operation statement provided by the invention, the execution of the target statement comprises the following steps:

obtaining connection parameters of the target database;

Determining whether a plurality of logic databases belonging to the same database instance exist in the connection configuration of the low-code platform based on the connection parameters;

In the case that a plurality of logic databases belonging to the same database instance exist in the connection configuration of the low-code platform, a shared connection pool for establishing communication with the database instance is established or multiplexed in a local connection manager;

And when the target sentence is executed, performing prefix limitation on the names of the data tables in the target sentence by utilizing a logic database identifier, and executing the limited target sentence through the shared connection pool.

According to the method for executing the database operation statement provided by the invention, when the original statement is a data definition statement, the grammar rule based on the target database rewrites the original statement in grammar to generate a target statement adapted to the target database, and the method comprises the following steps:

determining a database type of the target database;

Extracting dialect fragments corresponding to the database types from a preset persistent layer configuration file;

Invoking a specific implementation class of a preset database operation interface, wherein the specific implementation class is packaged with grammar implementation logic corresponding to the database type;

based on the dialect fragment, converting the original sentence into the target sentence adapting to the target database by utilizing the concrete implementation class.

According to the execution method of the database operation statement provided by the invention, the receiving of the original statement initiated by the low-code platform further comprises the following steps:

Identifying a source scene of the original sentence;

And under the condition that the source scene is a data view scene, determining that the service processing attribute is an analysis type query, and limiting the grammar rule of the original sentence to the grammar rule of the analysis type database.

The invention also provides an execution device of the database operation statement, which comprises the following modules:

the analysis module is used for analyzing the semantics of the original sentence initiated by the low-code platform and identifying the service processing attribute corresponding to the original sentence;

The distribution module is used for distributing the original statement to a target database matched with the business processing attribute according to the business processing attribute, wherein the target database is a transaction type database for transaction processing or an analysis type database for data analysis;

And the execution module is used for carrying out grammar rewriting on the original sentence based on the grammar rule of the target database, generating a target sentence adapting to the target database and executing the target sentence.

The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the execution method of any database operation statement is realized when the processor executes the program.

The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method of executing a database operation statement as described in any of the above.

The invention also provides a computer program product comprising a computer program which when executed by a processor implements a method of executing a database operating statement as described in any one of the above.

According to the method, the device, the equipment and the medium for executing the database operation statement, provided by the invention, the intelligent read-write separation of the transaction type load and the analysis type load is realized by acquiring and analyzing the semantics of the original statement to identify the service processing attribute, and the dialect rewriting is automatically completed by combining the grammar rule of the target database. Due to the fact that intelligent routing and transparent adaptation based on semantics are achieved, the performance bottleneck and single-point fault risk of a single database under large data volume are solved, complex analysis inquiry is avoided, core transaction business is dragged down, labor cost and technical threshold of low-code platform access multi-source heterogeneous database and domestic database adaptation are greatly reduced, and high availability and high performance of the system are achieved.

Drawings

In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.

Fig. 1 is a flow chart of a method for executing database operation sentences provided by the invention.

FIG. 2 is a schematic diagram of a database operation statement execution system according to the present invention.

Fig. 3 is a schematic diagram of an execution device of a database operation statement provided by the present invention.

Fig. 4 is a schematic structural diagram of an electronic device provided by the present invention.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

In the conventional application development mode, the data sources are generally selected and then accessed through corresponding software development kits (Software Development Kit, SDKs), and when a plurality of data sources are involved, a large number of SDKs are required to be accessed, so that the use process is complicated. In the field of low-code platforms, related technologies often do not support a multi-data source architecture, usually data sources are hidden inside, or supported data sources are single, and only data pulled to the low-code platform is supported for unified management.

The processing mode based on the single data source has the obvious defects that on one hand, for the service with larger data quantity, the single database is easy to become a performance bottleneck, an access interface is too slow, and a single point fault exists to cause the risk of global database paralysis, on the other hand, the system is difficult to consider different types of service loads, for an OLTP type database, the system is not suitable for executing complex analysis type structured query language (Structured Query Language, SQL) query, if the query is forced to run, the system responds slowly, the CPU occupation rate in the database is increased, and the whole system is dragged down, and for an OLAP type database, the transaction processing requirement good for the OLTP type database cannot be met rapidly. Users often require significant human effort to address the heterogeneous adaptation and performance isolation issues described above, and these scenarios are precisely essential for building complex applications.

In this regard, the invention provides a method for executing database operation sentences, which aims to automatically identify corresponding business processing attributes by receiving original sentences initiated by a low-code platform and analyzing the semantics thereof, intelligently distribute requests to a matched transactional database or an analytical database according to the attributes, and complete automatic grammar rewriting based on grammar rules of a target database, so as to realize a single-point problem that the low-code platform is separated from a data source, effectively shield the realization difference of a bottom multi-source heterogeneous database, reduce the access cost of users, and isolate transactions and analysis loads through intelligent diversion of sceneries, thereby reducing systematic risks.

Fig. 1 is a flow chart of a method for executing a database operation statement, as shown in fig. 1, where the method includes steps 110, 120 and 130.

And 110, analyzing the semantics of the original sentence initiated by the low-code platform and identifying the service processing attribute corresponding to the original sentence.

The low-code platform can be understood as a development environment for quickly constructing an application program through a graphical interface, a drag component, a model configuration and the like. The original statement initiated by the low-code platform generally refers to a structured query language or other data operation instruction generated by the platform back end according to the operation of a user on a front end interface, such as clicking a save button, loading a report view, etc., or by the background logic triggering. The original statement is typically generated based on a default database dialect preset by the platform, such as an SQL statement generated based on MySQL grammar.

As an alternative embodiment, the original statement may be received by setting an interceptor, proxy, or introducing a specialized multi-data source middleware component at the database connection layer of the low code platform.

Since the original sentence generated by the low-code platform may relate to various business intents, the direct execution may not be capable of taking both performance and stability into account, so that semantic analysis can be performed based on the original sentence to determine the type of operation that the sentence specifically wants to perform. And further identifying the business processing attribute corresponding to the original sentence on the basis of analyzing the semantics. The business processing attribute can be understood as the required characteristics of the operation statement on the computing resource, the data consistency and the response time, and is mainly used for distinguishing whether the statement belongs to daily transaction processing focusing on the data consistency or complex analysis processing focusing on mass data calculation.

For example, if the original statement involves only a single record of writes or state updates, the semantics of which are focused on atomic operations, the corresponding business process attributes tend to be transactional, while if the original statement involves grouping statistics, multi-table joint queries, or trend analysis of millions of rows of data, the semantics of which are focused on batch reads and computations, the corresponding business process attributes tend to be analytical.

Step 120, distributing the original sentence to a target database matched with the business processing attribute according to the business processing attribute, wherein the target database is a transaction type database for transaction processing or an analysis type database for data analysis.

In particular, transactional databases (OLTP) refer to databases that are adept at handling high concurrency, small data volumes, requiring strict atomicity, consistency, isolation, and persistence. Transactional databases typically include, but are not limited to, mySQL, oracle, and other relational databases. The transactional database is adapted to assume accounting responsibilities in the business system, such as order generation, user information modification, etc.

Analytical databases (OLAPs) refer to databases that are adept at handling massive amounts of data, complex aggregate computations, and multidimensional analytical queries. Analytical databases typically include, but are not limited to StarRocks, doris, clickHouse in-line storage or MPP architecture databases. The analysis type database is suitable for assuming audit or report responsibilities in the business system, and the execution speed of complex SQL is accelerated by utilizing the strong computing power of the analysis type database.

Further, considering that in the related art, if complex analysis query is forcedly run on the transaction type database, the database CPU will surge and even drag down the whole service system, and if the analysis type database is used for processing high-frequency transaction, the requirement of quick response cannot be met. Based on this, in order to achieve both high performance and high stability, the present embodiment performs intelligent offloading in combination with service processing attributes.

As an alternative embodiment, when the business processing attribute is identified as a transaction type operation (such as adding or deleting), the original sentence is distributed to a transaction type database so as to ensure the real-time writing and strong consistency of the business data.

As an alternative embodiment, when the business processing attribute is identified as an analysis type query (such as complex report statistics), the original sentence is distributed to an analysis type database, and the calculation advantage of the OLAP engine is utilized to avoid causing performance pressure on the main business library.

It should be noted that this distribution is transparent to the upper level applications of the low code platform, and the platform developer does not need to manually write code to determine which database should be connected, thus masking the complexity of the underlying implementation.

And 130, based on the grammar rule of the target database, carrying out grammar rewriting on the original sentence, generating a target sentence adapting to the target database, and executing the target sentence.

Specifically, the grammar rules of the target database refer to SQL dialects, function libraries and reserved word specifications specific to the target database. Since low code platforms typically support heterogeneous data sources, such as switching from MySQL to Oracle, or forking from MySQL to StarRocks, different types of databases tend to differ in syntax. For example, the DATE formatting function may be DATE_FORMAT in MySQL, TO_CHAR in Oracle, and a different expression in some OLAP databases.

If the original sentence is directly sent to the heterogeneous target database, the execution of the report error may be caused by language confusion.

Based on this, in order to enable cross-library collaboration and heterogeneous compatibility, this embodiment requires syntax rewriting of the original sentence before or during distribution. The rewrite process may include replacing a generic function in an original statement with a specialized function of a target database, adjusting a grammar structure of a paging query, handling implicit transformations of data types, and so forth. Through the grammar rewriting, a target sentence that can be correctly recognized and executed by the target database is generated.

As an alternative embodiment, a conversion mechanism can be established by means of an engine with SQL parsing capability or by using ASM byte code enhancement technology, source code coverage and other modes, so that SQL can be ensured to be converted into unified target database grammar. For example, for a scenario such as a data view, the target statement may also be generated directly with reference to the syntax specification of the OLAP database.

Finally, after the target statement is obtained, the target statement is executed through a driver of the target database, and an execution result is returned to the low-code platform.

In summary, according to the method for executing the database operation statement provided by the embodiment, by acquiring and analyzing the semantics of the original statement to identify the service processing attribute, the intelligent read-write separation of the transaction type load and the analysis type load is realized, and the dialect rewriting is automatically completed by combining the grammar rule of the target database. Due to the fact that intelligent routing and transparent adaptation based on semantics are achieved, the performance bottleneck and single-point fault risk of a single database under large data volume are solved, complex analysis inquiry is avoided, core transaction business is dragged down, labor cost and technical threshold of low-code platform access multi-source heterogeneous database and domestic database adaptation are greatly reduced, and high availability and high performance of the system are achieved.

Considering that simple keyword matching makes it difficult to accurately distinguish complex traffic scenarios, for example, a SELECT statement may only query a single row of configuration information, and may also contain complex nested sub-queries and full-table scans. If the routing is performed only according to the keywords, erroneous judgment is very easy to be caused, for example, delay is increased due to erroneous sending of a light-weight query of the running cache or the transaction library to the OLAP library, or system blocking is caused by erroneous sending of a heavy-weight query of the running OLAP library to the transaction library. In order to avoid the above problems, the present embodiment adopts a deep semantic analysis strategy based on an abstract syntax tree (Abstract Syntax Tree, AST) to realize accurate load type identification.

Specifically, the parsing the semantics of the original sentence in the step 110, and identifying the service processing attribute corresponding to the original sentence includes:

And step 111, generating an abstract syntax tree of the original sentence.

Here, an abstract syntax tree is an abstract representation of the source code syntax structure. In this embodiment, the mature SQL parsing engine may be used to perform lexical analysis and syntax analysis on the original sentence, and convert it into an AST object of a tree structure. The tree structure clearly shows the hierarchical relationship of SQL sentences, for example, the root node represents sentence types, and the child nodes respectively represent projection columns, filtering conditions, table scanning and the like.

Step 112, traversing the nodes in the abstract syntax tree, and identifying the operation instruction types represented by the nodes.

Specifically, the operation instruction type refers to a specific operation logic or function class represented by each syntax element in the SQL statement. The operation instruction types may include, but are not limited to, data manipulation (e.g., insert, update, delete), data query (e.g., single table scan, join, aggregate, sort), control flow instructions, and the like.

As an alternative embodiment, after generating the AST, the nodes in the AST are accessed one by one through a depth-first traversal algorithm or a breadth-first traversal algorithm, the specific meaning represented by each node is analyzed, and the operation instruction type represented by each node is determined according to a preset grammar rule dictionary or metadata definition.

And 113, determining that the business processing attribute corresponding to the original statement is transactional operation under the condition that the operation instruction type represented by any node is data change or single-table query.

Specifically, the data change refers to an operation of adding, deleting, or modifying data in the database, for example, INSERT, UPDATE, DELETE, or the like. A single table lookup refers to a read operation involving only a single data table and does not involve complex computational logic.

If the operation instruction type represented by any node is data change, the operation is indicated to generate lasting influence on the data state, strict transaction consistency guarantee is needed, and at the moment, the corresponding service processing attribute can be determined to be transaction operation. If the operation instruction type represented by any node is single-table query, the operation is indicated to belong to lightweight data retrieval, is generally used for real-time service interaction and is sensitive to response delay, and the corresponding service processing attribute can be determined to be transactional operation. Transactional operations, among other things, refer to operations that focus on OLTP, emphasizing the atomicity, consistency, isolation, and persistence of data, as well as the fast response capability under high concurrency.

For example, if the traversal finds that an AST includes an explicit data change instruction node, such as INSERT, UPDATE, DELETE, etc., then it is determined to be a transaction operation.

For example, if the AST is a query node, but the AST has a simple structure, for example, only involves a single table scan and does not include aggregation, grouping or ordering nodes, or if the query condition hits a primary key or a unique index, it is determined that the AST is a single table simple query, and the AST is also classified as a transactional operation.

Step 114, determining that the service processing attribute corresponding to the original sentence is an analysis type query under the condition that the operation instruction type represented by any node is multidimensional aggregation, multi-table association or complex ordering.

Specifically, most aggregation refers to an operation of grouping data according to one or more dimensions and calculating a statistical index, for example, an aggregation function such as GROUP BY (grouping), HAVING (grouping filtering), SUM (summing), AVG (average value), and the like. Multi-table association refers to an operation of concatenating data in two or more tables according to a specific condition. For example, INNER JOIN (interconnect), LEFT JOIN (LEFT interconnect), etc. Complex ordering refers to ordering or windowing computation operations performed under large amounts of data. For example, ORDER BY, LIMIT, RANK, etc.

If the operation instruction type represented by any node is multidimensional aggregation, the operation is indicated to involve the summarized calculation of large-scale data, the operation belongs to a computationally intensive task, and at the moment, the corresponding service processing attribute can be determined to be an analysis type query. If the operation instruction type represented by any node is multi-table association, the operation is indicated to need to perform a large number of Cartesian product operations or hash connection, and the memory consumption is large, so that the corresponding service processing attribute can be determined to be the analysis type query. If the operation instruction type represented by any node is complex ordering, the operation is indicated to involve the scanning and rearrangement of the full data, and at this time, the corresponding service processing attribute can be determined to be an analysis type query. The analysis type query is focused on OLAP, and operations of complex calculation, trend analysis and report generation capability of massive data are emphasized.

Through the refined identification based on AST, the embodiment can accurately distinguish service load characteristics, ensure reasonable distribution and load isolation of computing resources, namely, concentrate an OLTP database on transaction processing and concentrate an OLAP database on complex analysis, thereby maximizing the overall throughput and stability of the system.

Considering that there are significant grammatical differences between heterogeneous databases, for example, different databases differ in terms of date processing, string concatenation, paging queries, and implementation of specific functions. If the original sentence generated by the low-code platform is directly sent to the target database for execution, the execution failure is easily caused by incompatible grammar. In order to achieve seamless adaptation across database types, the present embodiment uses AST-based node replacement techniques for deep syntax re-writing.

Specifically, in the case that the original sentence is a data manipulation sentence or a data query sentence, the method includes, based on a grammar rule of a target database, performing grammar rewriting on the original sentence to generate a target sentence adapted to the target database, including:

Traversing the abstract syntax tree, and identifying source nodes to be converted, which are incompatible with the syntax rules of the target database;

retrieving a dialect mapping table of a target database, and acquiring a target equivalent node corresponding to a source node to be converted from the dialect mapping table, wherein the dialect mapping table comprises mapping relations between general grammar nodes and special grammar nodes of the target database;

and replacing the source node to be converted in the abstract syntax tree by using the target equivalent node, and converting the replaced abstract syntax tree to generate a target sentence.

Specifically, the source node to be converted refers to a node whose grammar structure, function name or keyword is not supported in the target database or has different expression modes in the AST generated by the original sentence.

As an alternative embodiment, a set of grammar rules for the target database may be preset. When traversing an AST, each node is checked one by one for compliance with the rule set. If any node accords with the rule set, the node is supported in the target database, conversion is not needed, and the node can be directly reserved. If any node does not conform to the rule set, the node is indicated to be a heterogeneous grammar node, and if the node is directly executed, grammar errors are caused, so that the node is marked as a source node to be converted.

Then, considering that specific function mapping or grammar correspondence rules exist among different databases, after identifying the source node to be converted, the embodiment can search in the dialect mapping table by taking the source node to be converted as a key to obtain a corresponding target equivalent node. Here, the dialect mapping table is a predefined configuration table or dictionary structure for storing correspondence between standard SQL and various target database dialects. Wherein the target equivalent node is an AST node object conforming to the target database grammar specification.

After the target equivalent node is obtained, the modification operation of the AST tree structure is executed, the original source node to be converted is removed from the tree, and the target equivalent node is inserted into the corresponding position, so that the structural reorganization of the grammar tree is completed.

After the replacement of all incompatible nodes is completed, the SQL generator can be called to traverse the modified AST and reversely analyze the AST into SQL sentences in the form of character strings, namely, the target sentences of the final adaptation target database are generated. In this way, the low code platform can automatically adapt to the switching of the bottom database without modifying the upper service code.

The cost of adapting each new database in view of directly modifying the source code of the database driver or middleware is extremely high and requires recompilation, packaging and deployment of the entire system, which is often difficult to implement in a production environment. In order to realize non-invasive and dynamic grammar compatibility enhancement to the existing system, the embodiment can flexibly inject AST node replacement logic on the premise of not changing the original code structure by dynamically modifying execution logic in a class loading stage or in running, thereby greatly improving the expansibility and maintenance efficiency of the system.

Specifically, the replacing the source node to be converted in the abstract syntax tree with the target equivalent node includes:

acquiring byte codes of component classes for executing the original sentence;

modifying byte codes, and inserting node replacement instructions aiming at abstract syntax trees into the processing logic of the component class;

And loading and running the modified byte codes to trigger an execution node replacement instruction to update the source node to be converted in the abstract syntax tree into a target equivalent node.

In particular, component classes refer to core Java classes that are responsible for database connections, SQL preprocessing or execution, or equivalent components in other languages.

As an alternative embodiment, the binary bytecode may be intercepted and retrieved when the class loader loads the component class using Java Agent technology or a bytecode operation framework (e.g., ASM, javassist, byte Buddy).

After the original byte code is obtained, the byte code structure can be analyzed by utilizing a byte code enhancement tool, the byte code structure is positioned at an entry or a parameter processing logic of a key method for executing the SQL sentence, and a node replacement instruction for the abstract syntax tree is constructed according to a predefined AST conversion logic template. The node replacement instruction is a compiled machine code or byte code sequence whose functional logic is equivalent to that described above for traversing an AST and replacing the code logic of the source node to be converted with the target equivalent node. Node replacement instructions can then be inserted into the byte code sequences of the component class critical methods to intercept and modify the semantic structure of the original SQL prior to its execution.

Considering that the modified bytecode must be read again by the Java virtual machine to be effective, if the modified bytecode is dynamically modified in the runtime, the class needs to be redefined or the original class definition needs to be replaced in the class loading stage. Therefore, the modified byte code is loaded and operated, and when the component class is called to execute database operation, an execution node replacement instruction is triggered, so that the source node to be converted in the abstract syntax tree is automatically updated to be a target equivalent node.

Because the logic is injected at the byte code level, the source code of the low code platform is not required to be modified, and the whole process is completely transparent to an upper layer caller, so that the non-sense grammar dialect adaptation is realized.

Considering that a low-code platform usually faces the problem of data islanding, namely business data are scattered in different physical databases, unified cross-database association query is difficult to perform, and business stability is affected by performing complex analysis directly on a transaction library. In order to solve the above-mentioned problems and provide a physical basis and a logical basis for the following intelligent routing, the present embodiment adopts a preprocessing scheme combining data synchronization and metadata mapping.

Specifically, before executing the original sentence initiated by the low code receiving platform, the method further includes:

And establishing a mapping relation between the data model and the physical data source in the metadata base of the low-code platform, wherein the mapping relation comprises a first corresponding relation between the data model identification and a database instance of the transactional database and a second corresponding relation between the data model identification and a data table of the analytical database.

In particular, the dispersed plurality of heterogeneous transactional databases may include database instances located at different IP addresses, different network segments, or even different regions.

As an alternative embodiment, the data of these heterogeneous transactional databases may be synchronized in real-time or near real-time into a unified analytical database by means of a synchronization tool or Connector (Connector) provided by an open source OLAP database engine.

The data of a plurality of distributed heterogeneous transaction databases are synchronized into the analysis type database, so that analysis load and transaction load are physically isolated, heterogeneous data are converged into the same OLAP engine, so that cross-database data which cannot be directly joined originally are changed into a logic table which can be locally associated in the analysis type database.

In addition, considering that the intelligent routing module needs to distribute SQL requests according to attribution of the data model, a mapping relation between the data model and the physical data source needs to be built in a metadata base of the low-code platform. Here, the metadata base of the low-code platform is a database for storing platform configuration information. The data model is an abstraction of a business entity in a low-code platform, such as an order model, a customer model, and the like.

To achieve accurate route distribution, the physical storage locations behind each data model need to be identified. Therefore, the present embodiment establishes a bidirectional mapping relationship, that is, a first correspondence relationship and a second correspondence relationship.

The first corresponding relation records the binding relation between the data model identification and the database instance of the transaction database, so that the correct transaction processing position can be found when the adding and deleting operation occurs.

The second corresponding relation records the binding relation between the data model identification and the data table of the analysis database, so that the correct analysis calculation position can be found when complex analysis inquiry occurs.

As an alternative embodiment, the mapping relationship may be configured in combination with service correlation. For example, the low code platform is isolated according to projects and tenants, and a plurality of transaction type database tables belonging to the same tenant or project and having associated requirements need to be ensured to be mapped into a logic space of the same analysis type database so as to facilitate subsequent cross-library associated queries. Thus, the mapping relationship in this scenario may include an association mapping of tenant IDs, project IDs, and analytics database instance IDs, ensuring that the associated data models fall physically or logically into the same OLAP library.

It is contemplated that in an actual production environment, the connection information of the database may be changed due to security policies, server migration, or capacity expansion. If the conventional hard-coded configuration or static configuration file mode is adopted, the low-code platform service needs to be restarted for each change, which causes service interruption and cannot meet the requirement of high availability. In addition, the plaintext storage database password also has a great potential safety hazard. To solve the configuration rigidification and security problems, the present embodiment employs a dynamic hot update mechanism based on a configuration center.

Specifically, after the mapping relationship between the data model and the physical data source is established in the metadata base of the execution low-code platform, the method further includes:

Acquiring a change message of the connection information in real time;

under the condition that the change message is obtained, dynamically updating the database connection instance and the mapping relation in the running memory of the low code platform in the process that the low code platform keeps the running state.

In particular, the connection information refers to network parameters and configuration parameters for locating and accessing database services. Authentication credentials refer to sensitive information used to verify the identity of a visitor, which typically includes a user name and password of a database. Because the connection information and the authentication credentials belong to high-sensitivity security data, if the plaintext storage is extremely easy to cause data leakage, the connection information and the authentication credentials of the transaction database and the analysis database are stored in the configuration center of the low-code platform in an encrypted manner, so that even if the configuration center is broken, an attacker cannot directly acquire the plaintext password.

As an alternative embodiment, the connection information can be encrypted by using Base64 coding or a simple confusion algorithm in consideration of the fact that the connection information is relatively public and still needs tamper resistance, and the authentication credentials can be encrypted by using strong encryption technologies such as advanced encryption standards or asymmetric encryption algorithms in consideration of the fact that the database password directly relates to the security of the core data asset.

Because of database expansion, disaster recovery switching or regular password rotation policy caused by service development, the connection information is changed generally, and the change message in this embodiment is notification data containing new changed connection parameters and version numbers. In order to ensure that the low code platform can timely sense and apply the latest configuration, the embodiment also needs to acquire the change message of the connection information in real time.

As an alternative embodiment, the change message of the connection information may be obtained in real time through a long polling mechanism of the configuration center. When the operation and maintenance personnel modify the connection information of a certain database and issue the connection information at the control console of the configuration center, the configuration center can actively push the change information to the low-code platform.

When the change message is received, the low-code platform executes hot update operation in the memory on the premise of keeping the running state, namely dynamically updates the database connection instance and the mapping relation in the running memory of the low-code platform, so that the low-code platform can be smoothly switched to the new database service at the bottom layer under the condition of not interrupting the user service access. Maintaining the running state here means that the low code platform does not stop the service and does not restart the process.

By the aid of the mechanism, seamless switching of database configuration is achieved, the problems of single-point failure and service shutdown of load balancing caused by configuration change are effectively avoided, and operation and maintenance efficiency and system stability are greatly improved.

Consider that under a multi-tenant or micro-service architecture, a low code platform typically needs to manage hundreds or thousands of logical databases. If a separate connection pool is established for each logical database in the conventional manner, the total number of connections is extremely large, even if each pool holds only a small number of connections. This not only consumes a lot of client memory and file handles, but also results in a exhaustion of the number of connections at the database server, severely affecting system performance and startup speed. In order to solve the problem of connection resource waste, the embodiment adopts a connection pool merging optimization technology based on physical examples.

Specifically, the executing the target sentence includes:

Obtaining connection parameters of a target database;

when executing the target sentence, performing prefix limitation on the names of the data tables in the target sentence by utilizing the logic database mark, and executing the limited target sentence through the shared connection pool.

Specifically, the connection parameters refer to configuration information required for establishing a database session, and include a physical access address, a service port, a logical database identifier, a user name, a password, and the like of the target database.

After the connection parameters are acquired, whether a logic database with the identical physical access address and service port exists can be identified by analyzing the connection parameters. If yes, the corresponding logical database is indicated to belong to the same database instance.

In this case, the network channels are reusable, considering that the logical databases actually run in the same database service process, and the connection merge policy can be adopted without creating a connection pool for the logical databases separately.

Specifically, a shared connection pool can be established or found in the local connection manager with physical instances as keys. The shared connection pool is typically connected to the root directory or default library of the database, with the purpose of establishing a TCP long connection channel with the physical server, rather than being tied to a specific certain service library.

When a specific target statement needs to be executed, a connection is obtained from the shared connection pool. Direct execution may report errors because the connection is not bound to a particular logical database. Therefore, the present embodiment dynamically rewrites the target sentence before execution.

Specifically, the logical database identifier is utilized to limit the prefix of the name of the data table in the target sentence, and the target sentence is rewritten into a format with a name space. Then, the defined target statement is executed by the shared connection pool. Thus, while all requests travel the same physical channel, the database engine can be accurately located into the respective logical libraries according to the prefixes.

In this way, the connection number is reduced from the order of magnitude of the logic library to the order of magnitude of the physical instance, so that system resources are greatly saved, and the concurrent processing capacity is improved. By the dynamic prefix adding mode, multiplexing access of a single physical connection to a plurality of logic libraries is realized, and the database connection overhead is obviously reduced.

The grammar differences between different database types are particularly great in view of the data definition statements. It is difficult to cover complex DDL structure differences with simple canonical substitutions or AST modifications alone. In order to realize unified support of the low-code platform to the table building and table changing capability of the multi-source heterogeneous database, the embodiment adopts an adapter mode based on interface polymorphism and configuration files.

Specifically, in the case that the original sentence is a data definition sentence, the method includes, based on a grammar rule of a target database, performing grammar overwriting on the original sentence to generate a target sentence adapted to the target database, including:

determining a database type of a target database;

Calling a specific implementation class of a preset database operation interface, wherein the specific implementation class is packaged with grammar implementation logic corresponding to the database type;

Based on the dialect fragments, converting the original sentences into target sentences of the adaptation target database by using the concrete implementation class.

Specifically, the database type of the target database is a vendor identifier of the database product, and the data type of the target database may be determined by reading ProductName in the database driver metadata, or may be determined by reading a data source type field stored in the system configuration center, which is not specifically limited in this embodiment.

The preset persistent layer configuration file refers to an XML file or a property file for defining the SQL mapping rule, and the configuration file includes SQL code blocks defined for different database IDs. Dialect fragments corresponding to a database type may be understood as keywords, data type definitions, or functional expressions specific to the database.

Considering that the syntax structure of different databases on DDL operations is completely different, it is necessary to determine dialect fragments corresponding to database types, so that the correct syntax building blocks can be provided for subsequent SQL assembly.

After the dialect fragment is determined, the general data model definition needs to be converted into a specific SQL execution statement, so that the embodiment calls a specific implementation class of a preset database operation interface, realizes logic by using grammar of a database type packaged in the specific implementation class, and performs character string splicing or template rendering in combination with the dialect fragment to convert an original statement into a target statement adapted to a target database.

It is contemplated that in low code platforms, some specific functional modules are designed at the beginning of the design for complex data analysis and presentation. Under these scenarios, the initiated database operation necessarily belongs to an analytical query. If the request of the explicit scene still executes the full-flow AST semantic analysis and automatic inference, not only unnecessary performance overhead is brought, but also the generic grammar conversion mechanism cannot cover the special advanced analysis functions of some OLAP databases, so that the exertion of the analysis capability is limited. In order to establish a fast path and support advanced analysis requirements, the present embodiment introduces a forced routing and grammar definition mechanism based on scene recognition.

Specifically, the receiving the original sentence initiated by the low code platform further includes:

Identifying a source scene of an original sentence;

In particular, a source scenario refers to a specific functional module or business context in a low code platform that triggers the database operation.

As an alternative embodiment, the scene identification may be carried in the request header or context object of the original sentence, so that the source scene may be identified by reading the identification.

A data view scenario refers to a scenario in which a user constructs a virtual data table by writing custom SQL or configuring aggregation logic in order to meet specific statistical requirements. If the source scene is a data view scene, considering that the operation intention in the scene is definitely analyzed for data, and often complex report logic is involved, the service processing attribute is not required to be determined according to the operation instruction type, and can be directly determined to be an analysis type query.

After determining that the service processing attribute is the analysis type query, considering that the general SQL rewrite mechanism may not support the high-level analysis grammar specific to the target OLAP database, the embodiment limits the grammar rule of the original sentence to the grammar rule of the analysis type database, so that the lossless calling of the native high-level analysis capability of the OLAP database can be realized under the condition of ensuring the execution efficiency.

FIG. 2 is a schematic diagram of a framework of a database operation statement execution system provided by the present invention, and as shown in FIG. 2, the workflow of the system is as follows:

First, the low code platform is used as a source of service initiation, generates an original sentence and sends a request. Then, the request is intercepted by a multi-data source component, and the multi-data source component is responsible for storing the information of each database of the OLTP and the OLAP in the database of the low-code platform and performing the encryption and decryption operations. Next, the data source component performs semantic parsing and route distribution on the original statement. If a transactional operation is identified, it is distributed to a transactional database.

Meanwhile, in order to solve the problem of cross-database query and analysis performance, the database connector is used for synchronizing the data in the transactional database to the analysis database, so that the difference of the bottom database is shielded, the SQL query mode is unified, and the data originally scattered in different physical instances can be logically associated at an OLAP layer, thereby solving the technical problem of cross-database joint query. When the multi-data source component recognizes an analytical query, the request is routed directly to the analytical database execution to accelerate SQL execution using its computing power.

The device for executing the database operation sentence provided by the invention is described below, and the device for executing the database operation sentence described below and the method for executing the database operation sentence described above can be referred to correspondingly.

Based on any of the above embodiments, fig. 3 is a schematic structural diagram of an execution device of a database operation statement provided by the present invention, and as shown in fig. 3, the device includes:

The parsing module 310 is configured to parse the semantics of the original sentence initiated by the low-code platform, and identify a service processing attribute corresponding to the original sentence;

The distributing module 320 is configured to distribute, according to the service processing attribute, the original sentence to a target database that matches the service processing attribute, where the target database is a transaction database for transaction processing or an analysis database for data analysis;

the execution module 330 is configured to rewrite the original sentence based on the grammar rule of the target database, generate a target sentence adapted to the target database, and execute the target sentence.

Fig. 4 is a schematic structural diagram of an electronic device according to the present invention, as shown in fig. 4, the electronic device may include a processor (processor) 410, a communication interface (Communications Interface) 420, a memory (memory) 430, and a communication bus 440, where the processor 410, the communication interface 420, and the memory 430 perform communication with each other through the communication bus 440. The processor 410 may call logic instructions in the memory 430 to execute the execution method of the database operation statement.

Further, the logic instructions in the memory 430 described above may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. The storage medium includes a U disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, an optical disk, or other various media capable of storing program codes.

In another aspect, the present invention also provides a computer program product, where the computer program product includes a computer program, where the computer program can be stored on a non-transitory computer readable storage medium, and when the computer program is executed by a processor, the computer can execute the method for executing the database operation statement provided by the above methods.

In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the method of executing the database operation statement provided by the methods described above.

The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.

It should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention, and not for limiting the same, and although the present invention has been described in detail with reference to the above-mentioned embodiments, it should be understood by those skilled in the art that the technical solution described in the above-mentioned embodiments may be modified or some technical features may be equivalently replaced, and these modifications or substitutions do not make the essence of the corresponding technical solution deviate from the spirit and scope of the technical solution of the embodiments of the present invention.

Claims

1. A method for executing database operation statements, comprising:

2. The method for executing the database operation sentence according to claim 1, wherein said parsing the semantics of the original sentence and identifying the business processing attribute corresponding to the original sentence includes:

Generating an abstract syntax tree of the original sentence;

traversing all nodes in the abstract syntax tree, and identifying operation instruction types represented by all nodes;

3. The method for executing the database operation sentence according to claim 2, wherein, in the case where the original sentence is a data manipulation sentence or a data query sentence, the performing, based on the grammar rule of the target database, the grammar rewriting of the original sentence to generate a target sentence adapted to the target database includes:

4. A method of executing a database operation statement according to claim 3, wherein said replacing the source node to be converted in the abstract syntax tree with the target equivalent node comprises:

acquiring byte codes of component classes for executing the original sentence;

5. The method of any one of claims 1 to 4, further comprising, prior to executing the original statement initiated by the receiving low-code platform:

6. The method according to claim 5, wherein after executing the mapping relationship between the data model and the physical data source established in the metadata base of the low-code platform, further comprising:

Acquiring a change message of the connection information in real time;

7. The method of executing a database operation statement according to any one of claims 1 to 4, wherein the executing the target statement includes:

obtaining connection parameters of the target database;

8. The method according to any one of claims 1 to 4, wherein, in the case where the original sentence is a data definition sentence, the performing, based on the grammar rule of the target database, the grammar overwriting of the original sentence to generate a target sentence adapted to the target database includes:

determining a database type of the target database;

9. The method of any one of claims 1 to 4, further comprising, after executing the original statement initiated by the receiving low-code platform:

Identifying a source scene of the original sentence;

10. An execution apparatus for database operation statements, comprising:

11. An electronic device comprising a memory, a processor and a computer program stored on the memory and running on the processor, characterized in that the processor implements the method of executing the database operating statement of any one of claims 1 to 9 when the computer program is executed.

12. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements a method of executing a database operation statement according to any of claims 1 to 9.