CN119576662B - Database middleware switching method and device - Google Patents
Database middleware switching method and deviceInfo
- Publication number
- CN119576662B CN119576662B CN202411916125.9A CN202411916125A CN119576662B CN 119576662 B CN119576662 B CN 119576662B CN 202411916125 A CN202411916125 A CN 202411916125A CN 119576662 B CN119576662 B CN 119576662B
- Authority
- CN
- China
- Prior art keywords
- middleware
- target
- database
- performance
- adaptation layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/16—Error detection or correction of the data by redundancy in hardware
- G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
- G06F11/202—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where processing functionality is redundant
- G06F11/2023—Failover techniques
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/2866—Architectures; Arrangements
- H04L67/288—Distributed intermediate devices, i.e. intermediate devices for interaction with other intermediate devices on the same level
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a method and a device for switching database middleware, wherein the method comprises the steps of constructing a dynamic adaptation layer, utilizing the dynamic adaptation layer to execute the following steps of acquiring a database operation request of target proxy service and configuration information of a plurality of database middleware, determining the target middleware in the plurality of database middleware according to request types and attribute information corresponding to the database operation request, acquiring target connection information corresponding to the target middleware in the plurality of connection information, switching to the target middleware according to the target connection information to realize connection between the target proxy service and the target middleware, monitoring performance data corresponding to the dynamic adaptation layer in real time in the process of executing the steps by the dynamic adaptation layer, analyzing the performance of the dynamic adaptation layer according to the performance data, determining corresponding processing strategies according to the performance problems, and processing the performance problems according to the processing strategies.
Description
Technical Field
The present application relates to the field of internet technologies, and in particular, to a method and an apparatus for switching a database middleware.
Background
Along with the rapid development of the internet of things technology, the MQTT (Message Queuing Telemetry Transport, message queue telemetry transport protocol) is widely applied to various internet of things scenes as a lightweight and efficient publish/subscribe message protocol. In these scenarios, the MQTT proxy service, as a core component of message transmission, needs to interact with database middleware to store and retrieve message data.
In application, due to the complexity of the internet of things system, the database middleware may be provided by different service providers, and there may be differences in technical standards and implementation among these service providers. When the MQTT proxy service needs to switch database middleware of different service providers, risks of complex configuration, service interruption, data loss and the like are often faced. Conventional solutions typically rely on manually modifying the configuration or redeploying the service.
However, this way of manually modifying the configuration or redeploying the service may not only be inefficient, but may also result in system instability.
Disclosure of Invention
The application provides a method and a device for switching database middleware, which are used for solving the problems that in the prior art, by manually modifying configuration or redeploying service, the efficiency is low and the system is possibly unstable.
In a first aspect, the present application provides a method for switching database middleware, including:
Constructing a dynamic adaptation layer, and executing the following steps by using the dynamic adaptation layer:
Acquiring a database operation request of a target proxy service and configuration information of a plurality of database middleware, wherein the configuration information comprises attribute information of each database middleware and connection information for connecting each database middleware;
Determining target middleware in a plurality of database middleware according to the request type corresponding to the database operation request and the attribute information;
acquiring target connection information corresponding to the target middleware from a plurality of pieces of connection information;
switching to the target middleware according to the target connection information so as to realize the connection between the target proxy service and the target middleware;
In the process that the dynamic adaptation layer executes the steps, monitoring performance data corresponding to the dynamic adaptation layer in real time, wherein the performance data are used for representing data affecting the performance of the dynamic adaptation layer;
Analyzing the performance of the dynamic adaptation layer according to the performance data to determine the performance problem of the dynamic adaptation layer;
and determining a corresponding processing strategy according to the performance problem, and processing the performance problem according to the processing strategy.
In one possible implementation manner, the determining, according to the request type corresponding to the database operation request and the attribute information, the target middleware in the plurality of database middleware includes:
determining target performance according to a request type corresponding to the database operation request, wherein the target performance refers to performance required by processing the request of the request type;
determining the priority of each database middleware according to the target performance and the attribute information of each database middleware;
and determining the database middleware with the highest corresponding priority as the target middleware.
In one possible embodiment, the method further comprises:
Taking each database middleware with the highest corresponding priority as a candidate middleware under the condition that at least two database middleware with the highest corresponding priority exist;
determining the corresponding load condition of each candidate middleware;
and determining the candidate middleware with the lowest corresponding load as the target middleware.
In one possible embodiment, the method further comprises:
In the process of connecting the target proxy service with the target middleware, performing fault monitoring on the target middleware;
And triggering a fault recovery mechanism in the case that the target middleware fails, wherein the fault recovery mechanism comprises reconnecting the target middleware or switching to a standby database middleware.
In one possible implementation manner, before the obtaining the database operation request of the target proxy service and the configuration information of the plurality of database middleware, the method further includes:
Constructing a dynamic adaptation layer, executing a database operation request for acquiring a target proxy service and configuration information of a plurality of database middleware by using the dynamic adaptation layer, wherein the configuration information comprises attribute information of each database middleware and connection information for connecting each database middleware, determining a target middleware in the plurality of database middleware according to a request type corresponding to the database operation request and the attribute information, acquiring target connection information corresponding to the target middleware in the plurality of database middleware, and switching to the target middleware according to the target connection information so as to realize connection between the target proxy service and the target middleware.
In one possible embodiment, the method further comprises:
And creating a connection pool in the dynamic adaptation layer, and managing connection between the proxy service and the database middleware by using the connection pool.
In one possible embodiment, the method further comprises:
determining operation content corresponding to the database operation request, wherein the operation content is used for representing an operation requested to be executed by the database operation request;
Determining whether the database operation request is a time-consuming request according to the operation content;
And under the condition that the database operation request is a time-consuming request, carrying out asynchronous processing on the database operation request by utilizing the dynamic adaptation layer.
In one possible embodiment, the method further comprises:
acquiring the query frequency of the target proxy service on each data in a database and the change frequency of each data;
Determining data with the corresponding query frequency larger than a first threshold value and the corresponding change frequency smaller than a second threshold value as target data;
And caching the target data by using the dynamic adaptation layer so as to respond by using the cached target data under the condition that the target proxy service inquires the target data next time.
In a second aspect, the present application provides a database middleware switching device, including:
the construction module is used for constructing a dynamic adaptation layer and executing the following steps by utilizing the dynamic adaptation layer:
The system comprises a request receiving module, a target agent service processing module and a storage module, wherein the request receiving module is used for acquiring a database operation request of the target agent service and configuration information of a plurality of database middleware, and the configuration information comprises attribute information of each database middleware and connection information used for connecting each database middleware;
The middleware determining module is used for determining target middleware in a plurality of database middleware according to the request type corresponding to the database operation request and the attribute information;
The information acquisition module is used for acquiring target connection information corresponding to the target middleware from a plurality of pieces of connection information;
the middleware switching module is used for switching to the target middleware according to the target connection information so as to realize the connection between the target proxy service and the target middleware;
The monitoring module is used for monitoring the performance data corresponding to the dynamic adaptation layer in real time in the process of executing the steps by the dynamic adaptation layer, wherein the performance data are used for representing the data affecting the performance of the dynamic adaptation layer;
The analysis module is used for analyzing the performance of the dynamic adaptation layer according to the performance data and determining the performance problem of the dynamic adaptation layer;
and the processing module is used for determining a corresponding processing strategy according to the performance problem and processing the performance problem according to the processing strategy.
In one possible embodiment, the middleware determination module is specifically configured to:
determining target performance according to a request type corresponding to the database operation request, wherein the target performance refers to performance required by processing the request of the request type;
determining the priority of each database middleware according to the target performance and the attribute information of each database middleware;
and determining the database middleware with the highest corresponding priority as the target middleware.
In one possible embodiment, the middleware determination module is further configured to:
Taking each database middleware with the highest corresponding priority as a candidate middleware under the condition that at least two database middleware with the highest corresponding priority exist;
determining the corresponding load condition of each candidate middleware;
and determining the candidate middleware with the lowest corresponding load as the target middleware.
In one possible embodiment, the apparatus further comprises a fault monitoring module for:
In the process of connecting the target proxy service with the target middleware, performing fault monitoring on the target middleware;
And triggering a fault recovery mechanism in the case that the target middleware fails, wherein the fault recovery mechanism comprises reconnecting the target middleware or switching to a standby database middleware.
In one possible embodiment, the apparatus further comprises a construction module for:
Constructing a dynamic adaptation layer, executing a database operation request for acquiring a target proxy service and configuration information of a plurality of database middleware by using the dynamic adaptation layer, wherein the configuration information comprises attribute information of each database middleware and connection information for connecting each database middleware, determining a target middleware in the plurality of database middleware according to a request type corresponding to the database operation request and the attribute information, acquiring target connection information corresponding to the target middleware in the plurality of database middleware, and switching to the target middleware according to the target connection information so as to realize connection between the target proxy service and the target middleware.
In one possible embodiment, the apparatus further comprises a creation module for:
And creating a connection pool in the dynamic adaptation layer, and managing connection between the proxy service and the database middleware by using the connection pool.
In a possible embodiment, the apparatus further comprises an asynchronous processing module for:
determining operation content corresponding to the database operation request, wherein the operation content is used for representing an operation requested to be executed by the database operation request;
Determining whether the database operation request is a time-consuming request according to the operation content;
And under the condition that the database operation request is a time-consuming request, carrying out asynchronous processing on the database operation request by utilizing the dynamic adaptation layer.
In one possible implementation manner, the apparatus further includes a buffer module, configured to:
acquiring the query frequency of the target proxy service on each data in a database and the change frequency of each data;
Determining data with the corresponding query frequency larger than a first threshold value and the corresponding change frequency smaller than a second threshold value as target data;
And caching the target data by using the dynamic adaptation layer so as to respond by using the cached target data under the condition that the target proxy service inquires the target data next time.
In a third aspect, the present application provides an apparatus comprising a processor and a memory, the processor being configured to execute a database middleware switching program stored in the memory, to implement the database middleware switching method of any one of the first aspects.
In a fourth aspect, the present application provides a storage medium storing one or more programs executable by one or more processors to implement the database middleware switching method of any one of the first aspects.
The method provided by the embodiment of the application has the advantages that firstly, a dynamic adaptation layer is constructed, and the dynamic adaptation layer is utilized to execute the following steps of acquiring a database operation request of a target proxy service and configuration information of a plurality of database middleware, wherein the configuration information comprises attribute information of each database middleware and connection information for connecting each database middleware, then, determining a target middleware in the plurality of database middleware according to a request type corresponding to the database operation request and the attribute information, further, acquiring target connection information corresponding to the target middleware in the plurality of connection information, and finally, switching to the target middleware according to the target connection information to realize connection of the target proxy service and the target middleware. And in the process of executing the steps by the dynamic adaptation layer, monitoring the performance data corresponding to the dynamic adaptation layer in real time, wherein the performance data are used for representing data affecting the performance of the dynamic adaptation layer, analyzing the performance of the dynamic adaptation layer according to the performance data to determine the performance problem of the dynamic adaptation layer, determining a corresponding processing strategy according to the performance problem, and processing the performance problem according to the processing strategy. Through the scheme, the automatic identification and adaptation of the database middleware of different service providers can be realized, and further, the automatic switching of the database middleware is realized, so that the service deployment efficiency and the flexibility and reliability of the system can be improved. And through carrying out real-time monitoring to the dynamic adaptation layer, potential performance bottlenecks or potential safety hazards can be found and processed in time, so that the stability of the system is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the embodiments of the application or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, and it will be obvious to a person skilled in the art that other drawings can be obtained from these drawings without inventive effort.
One or more embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements, and in which the figures of the drawings are not to be taken in a limiting sense, unless otherwise indicated.
Fig. 1 is a flowchart of an embodiment of a method for switching database middleware according to an embodiment of the present application;
FIG. 2 is a flowchart of an embodiment of determining a target middleware among a plurality of database middleware according to a request type corresponding to a database operation request and the attribute information according to the embodiment of the present application;
FIG. 3 is a flowchart of an embodiment of another method for switching database middleware according to an embodiment of the present application;
FIG. 4 is a flowchart of an embodiment of another method for switching database middleware according to an embodiment of the present application;
Fig. 5 is a block diagram of an embodiment of a database middleware switching device according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The following disclosure provides many different embodiments, or examples, for implementing different structures of the application. In order to simplify the present disclosure, components and arrangements of specific examples are described below. They are, of course, merely examples and are not intended to limit the application. Furthermore, the present application may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed.
In order to solve the technical problems that in the prior art, the efficiency is low and the system is possibly unstable due to the manual configuration modification or service redeployment, the application provides a database middleware switching method which can realize automatic identification and adaptation of database middleware of different service providers, and further realize automatic switching of the database middleware, thereby improving the service deployment efficiency and the flexibility and reliability of the system.
Fig. 1 is a flowchart of an embodiment of a method for switching database middleware according to an embodiment of the present application. As shown in fig. 1, the method comprises the steps of:
Step 101, constructing a dynamic adaptation layer, and acquiring a database operation request of a target proxy service and configuration information of a plurality of database middleware through the dynamic adaptation layer, wherein the configuration information comprises attribute information of each database middleware and connection information for connecting each database middleware.
And 102, determining target middleware in a plurality of database middleware according to the request type corresponding to the database operation request and the attribute information.
And step 103, acquiring target connection information corresponding to the target middleware from the plurality of connection information.
And 104, switching to the target middleware according to the target connection information so as to realize the connection between the target proxy service and the target middleware.
For ease of understanding, steps 101 to 104 are collectively described as follows:
the request type is used for characterizing the type of database operation request, such as query request, insertion request, update request, deletion request, batch operation request, etc.
Configuration information including attribute information of each database middleware and connection information for connecting each database middleware. Wherein attribute information such as performance information of read performance, write performance, load condition, etc., connection information such as connection address, port number, authentication information, database name, and other configuration parameters for connecting the database middleware.
In the application, a dynamic adaptation layer can be constructed by using programming languages such as Java or C++, and intelligent switching logic is integrated in the dynamic adaptation layer, so that the dynamic adaptation layer can be used as a bridge between MQTT (Message Queuing Telemetry Transport, message queue telemetry transport protocol) proxy service and database middleware.
Specifically, the dynamic adaptation layer is responsible for receiving database operation requests of the MQTT proxy service, then matching proper target middleware in a plurality of database middleware according to the request type and attribute information in the current configuration information through the intelligent switching logic, and finally switching to the target middleware by utilizing corresponding target connection information, thereby realizing interaction between the target proxy service and the target middleware.
In practical application, the consistency and the integrity of the data can be ensured through transaction management in the switching process, namely, the data is either completely successful or completely failed, or the consistency of the data can be ensured through a double-write mechanism, namely, in the switching process, the data is written into two databases at the same time and the read-write is separated, and in the switching process, the strategy of the read-write separation can be adopted, so that the consistency of the data is ensured.
And 105, monitoring performance data corresponding to the dynamic adaptation layer in real time in the process of executing the steps by the dynamic adaptation layer, wherein the performance data are used for representing data affecting the performance of the dynamic adaptation layer.
And 106, analyzing the performance of the dynamic adaptation layer according to the performance data to determine the performance problem of the dynamic adaptation layer.
And 107, determining a corresponding processing strategy according to the performance problem, and processing the performance problem according to the processing strategy.
For ease of understanding, steps 105 through 107 are collectively described as follows:
Performance data for characterizing data affecting performance of the dynamic adaptation layer, e.g., response time: response time per database operation, throughput: number of requests processed per unit time, connection pool status: number of current active connections, number of idle connections, number of waiting connections, etc., cache hit rate: number of hits and total number of accesses of the cache, SQL injection attack: detecting if there is malicious SQL injection attempt, unauthorized access: recording unauthorized access attempt, exception log: recording all exception and error information, etc.
In the embodiment of the application, the performance of the dynamic adaptation layer can be analyzed according to the performance data, so that the possible performance problem of the dynamic adaptation layer is determined, and the performance problem is processed according to the corresponding processing strategy.
For example, there may be performance bottlenecks for a certain operation if its response time is significantly higher than other operations, resource starvation or code problems for a throughput if the throughput of the system drops suddenly, connection leakage or improper connection pool size setting for a connection pool state if the number of active connections in the connection pool is high for a long period of time, searching for malicious SQL injection attempts by log analysis for SQL injection attacks, recording all unauthorized access attempts for unauthorized access, analyzing the source and frequency, checking for exception logs periodically, searching for potential security vulnerabilities.
Through the scheme, the adaptive layer can be monitored in real time, and potential performance bottlenecks or potential safety hazards can be found and processed in time, so that the stability of the system is improved.
In another embodiment of the application the method may further comprise the step of creating a connection pool in the dynamic adaptation layer, with which connection pool the connection between the proxy service and the database middleware is managed. In the scheme, the dynamic adaptation layer can use the connection pool to manage database connection, and the connection pool can multiplex the existing connection, so that frequent connection and disconnection operations can be avoided, and the cost of creating and destroying the connection is reduced.
In addition, in another embodiment of the present application, the method may further include the steps of performing fault monitoring on the target middleware during the process of connecting the target proxy service with the target middleware, and triggering a fault recovery mechanism in the case that the target middleware fails, where the fault recovery mechanism includes reconnecting the target middleware or switching to a standby database middleware.
The fault monitoring may include connection timeout monitoring, SQL execution monitoring, heartbeat detection, and network connection monitoring, among others. Specifically, the connection timeout monitoring process is to set a timeout time when attempting to connect to the database, and if the connection fails within a specified time, the connection is considered to fail, i.e. a failure occurs. The SQL execution monitoring process is to capture the exception when executing SQL statement. If SQL execution fails, the database is deemed unavailable, i.e., fails. The heartbeat monitoring process is to periodically send heartbeat requests and check the health state of the database. If multiple heartbeat requests fail, the database is deemed unavailable, i.e., failed. The network connection monitoring process is to check whether the network connection is normal or not, ensure that the database middleware can be accessed, and consider that the network connection is abnormal, if so, the fault occurs.
In application, the fault recovery mechanism may also include logging the fault and alerting the fault. Therefore, the user can conveniently and timely process faults.
Through the scheme, when the target middleware fails, the failure recovery processing can be timely carried out, so that the stability of the system is ensured.
The technical scheme provided by the embodiment of the application comprises the steps of firstly, constructing a dynamic adaptation layer, utilizing the dynamic adaptation layer to execute the following steps of acquiring a database operation request of a target proxy service and configuration information of a plurality of database middleware, wherein the configuration information comprises attribute information of each database middleware and connection information for connecting each database middleware, then, determining a target middleware in the plurality of database middleware according to a request type corresponding to the database operation request and the attribute information, further, acquiring target connection information corresponding to the target middleware in the plurality of connection information, and finally, switching to the target middleware according to the target connection information to realize connection of the target proxy service and the target middleware. And in the process of executing the steps by the dynamic adaptation layer, monitoring the performance data corresponding to the dynamic adaptation layer in real time, wherein the performance data are used for representing data affecting the performance of the dynamic adaptation layer, analyzing the performance of the dynamic adaptation layer according to the performance data to determine the performance problem of the dynamic adaptation layer, determining a corresponding processing strategy according to the performance problem, and processing the performance problem according to the processing strategy. Through the scheme, the automatic identification and adaptation of the database middleware of different service providers can be realized, and further, the automatic switching of the database middleware is realized, so that the service deployment efficiency and the flexibility and reliability of the system can be improved. And through carrying out real-time monitoring to the dynamic adaptation layer, potential performance bottlenecks or potential safety hazards can be found and processed in time, so that the stability of the system is improved.
Fig. 2 is a flowchart of an embodiment of determining a target middleware in a plurality of database middleware according to a request type corresponding to a database operation request and the attribute information according to an embodiment of the present application. The flow shown in fig. 2 is based on the flow shown in fig. 1, and comprises the following steps:
Step 201, determining target performance according to a request type corresponding to the database operation request, wherein the target performance refers to performance required when processing the request of the request type;
Step 202, determining the priority of each database middleware according to the target performance and the attribute information of each database middleware;
And 203, determining the database middleware with the highest corresponding priority as the target middleware.
For ease of understanding, steps 201 to 203 are collectively described below:
In applications, the requirements of different types of requests on performance are different, for example, for a read request, the performance is focused on as a read performance, for a write request, the performance is focused on as a write performance, and for a request containing both read and write operations, performance and stability are generally comprehensively considered.
Based on this, in the embodiment of the present application, first, the target performance of the database operation request is determined according to the request type corresponding to the database operation request, and then, for each database middleware, the database middleware with the best target performance or the higher performance matching degree is determined as the target middleware from the plurality of database middleware according to the description of the target performance in the attribute information (that is, the higher the priority is, the better the target performance or the higher the performance matching degree is, the database middleware with the target performance as the reference to the target performance in all the database middleware).
For example, for a read request, middleware with better read performance can be preferentially selected, for a write request, middleware with better write performance and stability can be preferentially selected, and for a request containing read-write operation, middleware with high comprehensive score of performance and stability can be preferentially selected.
In addition, in the application, a situation that a plurality of database middleware with highest corresponding priority exists may occur, and therefore, in a further embodiment of the present application, the method may further include the steps of, in the case that at least two database middleware with highest corresponding priority exists, determining, as candidate middleware, a load condition corresponding to each candidate middleware, and determining, as the target middleware, a candidate middleware with lowest corresponding load.
The load may include, among other things, CPU utilization, memory utilization, network bandwidth, number of requests, and response time.
As is clear from the above description, in this scheme, when there are a plurality of database middleware with highest corresponding priority, the database middleware with lowest corresponding load can be determined as the target middleware. Therefore, the load balance of the system can be ensured, and the system stability is improved.
Through the flow shown in fig. 2, the target performance of interest can be compared according to the database operation request, and the database middleware with the best corresponding target performance is determined from the plurality of database middleware to be the target middleware, so that the efficient processing of the database operation request can be ensured.
Fig. 3 is a flowchart of an embodiment of another method for switching database middleware according to an embodiment of the present application. The flow shown in fig. 3 is based on the flow shown in fig. 1, and comprises the following steps:
Step 301, determining operation content corresponding to the database operation request, wherein the operation content is used for representing an operation requested to be executed by the database operation request;
Step 302, determining whether the database operation request is a time-consuming request according to the operation content;
and 303, in the case that the database operation request is a time-consuming request, performing asynchronous processing on the database operation request by using the dynamic adaptation layer.
For ease of understanding, steps 301 to 303 are collectively described below:
The operation content is used for representing the operation requested to be executed by the database operation request, and some database operation requests with complex operations in the application take longer time and are considered as time-consuming requests.
In practical applications, there are several time-consuming requests, namely, a first, complex SQL query, especially a join operation SQL query involving a large amount of data or multiple tables, a second, calling external APIs or services, which typically require network communication with potentially large delays, a third, file operation, namely, reading and writing large files or processing large files, a fourth, computationally intensive operation, such as image processing, data analysis, etc., and a fifth, I/O operation, such as disk I/O operation, network I/O operation, etc.
In the embodiment of the application, firstly, the time-consuming request is identified according to the operation content, then, the dynamic adaptation layer can design the time-consuming request into an asynchronous task through an asynchronous programming model (such as a callback function, a coroutine and the like), and submits the asynchronous task into a task queue or an event loop for task scheduling, and the special thread or process processes the asynchronous task. After the asynchronous task is completed, the result is returned to the main thread through a callback function or other mechanisms, and the main thread can continue to process other tasks.
Through the flow shown in fig. 3, for the time-consuming database operation request, an asynchronous processing mechanism can be used for asynchronous processing, so that the blocking of a main thread is avoided, and the overall efficiency of the system is improved.
Fig. 4 is a flowchart of an embodiment of another method for switching database middleware according to an embodiment of the present application. The flow shown in fig. 4 is based on the flow shown in fig. 1, and comprises the following steps:
step 401, obtaining the query frequency of the target proxy service to each data in a database and the change frequency of each data;
step 402, determining data with the corresponding query frequency being greater than a first threshold and the corresponding change frequency being less than a second threshold as target data;
step 403, caching the target data by using the dynamic adaptation layer, so as to respond by using the cached target data when the target proxy service queries the target data next time.
For ease of understanding, steps 401 to 403 are collectively described below:
The first threshold is used for determining whether the data is frequently queried, that is, if the query frequency of the data is greater than the first threshold, the data is considered to be frequently queried.
The second threshold is used for determining whether the data is changed frequently, that is, if the change frequency of the data is smaller than the second threshold, the data is considered to be changed infrequently.
As can be seen from the above description, in the embodiment of the present application, for the target data that is frequently queried and not frequently changed, the dynamic adaptation layer may cache the target data, so when the target proxy service queries the target data, the cached target data may be used to respond, thereby reducing the number of accesses to the database.
Fig. 5 is a block diagram of an embodiment of a database middleware switching device according to an embodiment of the present application. As shown in fig. 5, the apparatus includes:
a construction module 51, configured to construct a dynamic adaptation layer, and perform the following steps using the dynamic adaptation layer:
A request receiving module 52, configured to obtain a database operation request of a target proxy service and configuration information of a plurality of database middleware, where the configuration information includes attribute information of each database middleware and connection information for connecting each database middleware;
a middleware determining module 53, configured to determine a target middleware from a plurality of database middleware according to a request type corresponding to the database operation request and the attribute information;
An information obtaining module 54, configured to obtain target connection information corresponding to the target middleware from a plurality of connection information;
The middleware switching module 55 is configured to switch to the target middleware according to the target connection information, so as to implement connection between the target proxy service and the target middleware;
The monitoring module 56 is configured to monitor, in real time, performance data corresponding to the dynamic adaptation layer during the process of executing the steps by the dynamic adaptation layer, where the performance data is used to characterize data affecting performance of the dynamic adaptation layer;
an analysis module 57, configured to analyze the performance of the dynamic adaptation layer according to the performance data, and determine a performance problem of the dynamic adaptation layer;
And the processing module 58 is configured to determine a corresponding processing policy according to the performance problem, and process the performance problem according to the processing policy.
In one possible embodiment, the middleware determination module is specifically configured to:
determining target performance according to a request type corresponding to the database operation request, wherein the target performance refers to performance required by processing the request of the request type;
determining the priority of each database middleware according to the target performance and the attribute information of each database middleware;
and determining the database middleware with the highest corresponding priority as the target middleware.
In one possible embodiment, the middleware determination module is further configured to:
Taking each database middleware with the highest corresponding priority as a candidate middleware under the condition that at least two database middleware with the highest corresponding priority exist;
determining the corresponding load condition of each candidate middleware;
and determining the candidate middleware with the lowest corresponding load as the target middleware.
In one possible embodiment, the apparatus further comprises a fault monitoring module for:
In the process of connecting the target proxy service with the target middleware, performing fault monitoring on the target middleware;
And triggering a fault recovery mechanism in the case that the target middleware fails, wherein the fault recovery mechanism comprises reconnecting the target middleware or switching to a standby database middleware.
In one possible embodiment, the apparatus further comprises a construction module for:
Constructing a dynamic adaptation layer, executing a database operation request for acquiring a target proxy service and configuration information of a plurality of database middleware by using the dynamic adaptation layer, wherein the configuration information comprises attribute information of each database middleware and connection information for connecting each database middleware, determining a target middleware in the plurality of database middleware according to a request type corresponding to the database operation request and the attribute information, acquiring target connection information corresponding to the target middleware in the plurality of database middleware, and switching to the target middleware according to the target connection information so as to realize connection between the target proxy service and the target middleware.
In one possible embodiment, the apparatus further comprises a creation module for:
And creating a connection pool in the dynamic adaptation layer, and managing connection between the proxy service and the database middleware by using the connection pool.
In a possible embodiment, the apparatus further comprises an asynchronous processing module for:
determining operation content corresponding to the database operation request, wherein the operation content is used for representing an operation requested to be executed by the database operation request;
Determining whether the database operation request is a time-consuming request according to the operation content;
And under the condition that the database operation request is a time-consuming request, carrying out asynchronous processing on the database operation request by utilizing the dynamic adaptation layer.
In one possible implementation manner, the apparatus further includes a buffer module, configured to:
acquiring the query frequency of the target proxy service on each data in a database and the change frequency of each data;
Determining data with the corresponding query frequency larger than a first threshold value and the corresponding change frequency smaller than a second threshold value as target data;
And caching the target data by using the dynamic adaptation layer so as to respond by using the cached target data under the condition that the target proxy service inquires the target data next time.
In one possible embodiment, the apparatus further comprises a monitoring module for:
monitoring performance data corresponding to the dynamic adaptation layer in real time, wherein the performance data are used for representing data affecting the performance of the dynamic adaptation layer;
Analyzing the performance of the dynamic adaptation layer according to the performance data to determine the performance problem of the dynamic adaptation layer;
and determining a corresponding processing strategy according to the performance problem, and processing the performance problem according to the processing strategy.
As shown in fig. 6, an embodiment of the present application provides an apparatus including a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 perform communication with each other through the communication bus 114,
A memory 113 for storing a computer program;
In one embodiment of the present application, the processor 111 is configured to implement the method for switching the database middleware provided in any one of the foregoing method embodiments when executing the program stored in the memory 113, where the method includes:
Constructing a dynamic adaptation layer, and executing the following steps by using the dynamic adaptation layer:
Acquiring a database operation request of a target proxy service and configuration information of a plurality of database middleware, wherein the configuration information comprises attribute information of each database middleware and connection information for connecting each database middleware;
Determining target middleware in a plurality of database middleware according to the request type corresponding to the database operation request and the attribute information;
acquiring target connection information corresponding to the target middleware from a plurality of pieces of connection information;
switching to the target middleware according to the target connection information so as to realize the connection between the target proxy service and the target middleware;
In the process that the dynamic adaptation layer executes the steps, monitoring performance data corresponding to the dynamic adaptation layer in real time, wherein the performance data are used for representing data affecting the performance of the dynamic adaptation layer;
Analyzing the performance of the dynamic adaptation layer according to the performance data to determine the performance problem of the dynamic adaptation layer;
and determining a corresponding processing strategy according to the performance problem, and processing the performance problem according to the processing strategy.
The embodiment of the application also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the steps of the database middleware switching method provided in any one of the method embodiments described above.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
From the above description of embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus a general purpose hardware platform, or may be implemented by hardware. Based on such understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the related art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform the method described in the respective embodiments or some parts of the embodiments.
It is to be understood that the terminology used herein is for the purpose of describing particular example embodiments only, and is not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms "comprises," "comprising," "includes," "including," and "having" are inclusive and therefore specify the presence of stated features, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and/or groups thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring their performance in the particular order described or illustrated, unless an order of performance is explicitly stated. It should also be appreciated that additional or alternative steps may be used.
The foregoing is only a specific embodiment of the application to enable those skilled in the art to understand or practice the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (8)
1. A method for switching database middleware, the method comprising:
Constructing a dynamic adaptation layer, and executing the following steps by using the dynamic adaptation layer:
Acquiring a database operation request of a target proxy service and configuration information of a plurality of database middleware, wherein the configuration information comprises attribute information of each database middleware and connection information for connecting each database middleware;
Determining target middleware in a plurality of database middleware according to the request type corresponding to the database operation request and the attribute information;
acquiring target connection information corresponding to the target middleware from a plurality of pieces of connection information;
switching to the target middleware according to the target connection information so as to realize the connection between the target proxy service and the target middleware;
In the process that the dynamic adaptation layer executes the steps, monitoring performance data corresponding to the dynamic adaptation layer in real time, wherein the performance data are used for representing data affecting the performance of the dynamic adaptation layer;
Analyzing the performance of the dynamic adaptation layer according to the performance data to determine the performance problem of the dynamic adaptation layer;
and determining a corresponding processing strategy according to the performance problem, and processing the performance problem according to the processing strategy.
2. The method according to claim 1, wherein determining the target middleware among the plurality of database middleware according to the request type corresponding to the database operation request and the attribute information comprises:
determining target performance according to a request type corresponding to the database operation request, wherein the target performance refers to performance required by processing the request of the request type;
determining the priority of each database middleware according to the target performance and the attribute information of each database middleware;
and determining the database middleware with the highest corresponding priority as the target middleware.
3. The method according to claim 2, wherein the method further comprises:
Taking each database middleware with the highest corresponding priority as a candidate middleware under the condition that at least two database middleware with the highest corresponding priority exist;
determining the corresponding load condition of each candidate middleware;
and determining the candidate middleware with the lowest corresponding load as the target middleware.
4. The method according to claim 1, wherein the method further comprises:
In the process of connecting the target proxy service with the target middleware, performing fault monitoring on the target middleware;
And triggering a fault recovery mechanism in the case that the target middleware fails, wherein the fault recovery mechanism comprises reconnecting the target middleware or switching to a standby database middleware.
5. The method according to claim 1, wherein the method further comprises:
And creating a connection pool in the dynamic adaptation layer, and managing connection between the proxy service and the database middleware by using the connection pool.
6. The method according to claim 1, wherein the method further comprises:
determining operation content corresponding to the database operation request, wherein the operation content is used for representing an operation requested to be executed by the database operation request;
Determining whether the database operation request is a time-consuming request according to the operation content;
And under the condition that the database operation request is a time-consuming request, carrying out asynchronous processing on the database operation request by utilizing the dynamic adaptation layer.
7. The method according to claim 1, wherein the method further comprises:
acquiring the query frequency of the target proxy service on each data in a database and the change frequency of each data;
Determining data with the corresponding query frequency larger than a first threshold value and the corresponding change frequency smaller than a second threshold value as target data;
And caching the target data by using the dynamic adaptation layer so as to respond by using the cached target data under the condition that the target proxy service inquires the target data next time.
8. A database middleware switching apparatus, the apparatus comprising:
the construction module is used for constructing a dynamic adaptation layer and executing the following steps by utilizing the dynamic adaptation layer:
The system comprises a request receiving module, a target agent service processing module and a storage module, wherein the request receiving module is used for acquiring a database operation request of the target agent service and configuration information of a plurality of database middleware, and the configuration information comprises attribute information of each database middleware and connection information used for connecting each database middleware;
The middleware determining module is used for determining target middleware in a plurality of database middleware according to the request type corresponding to the database operation request and the attribute information;
The information acquisition module is used for acquiring target connection information corresponding to the target middleware from a plurality of pieces of connection information;
the middleware switching module is used for switching to the target middleware according to the target connection information so as to realize the connection between the target proxy service and the target middleware;
The monitoring module is used for monitoring the performance data corresponding to the dynamic adaptation layer in real time in the process of executing the steps by the dynamic adaptation layer, wherein the performance data are used for representing the data affecting the performance of the dynamic adaptation layer;
The analysis module is used for analyzing the performance of the dynamic adaptation layer according to the performance data and determining the performance problem of the dynamic adaptation layer;
and the processing module is used for determining a corresponding processing strategy according to the performance problem and processing the performance problem according to the processing strategy.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411916125.9A CN119576662B (en) | 2024-12-24 | 2024-12-24 | Database middleware switching method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411916125.9A CN119576662B (en) | 2024-12-24 | 2024-12-24 | Database middleware switching method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN119576662A CN119576662A (en) | 2025-03-07 |
| CN119576662B true CN119576662B (en) | 2025-10-14 |
Family
ID=94807166
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411916125.9A Active CN119576662B (en) | 2024-12-24 | 2024-12-24 | Database middleware switching method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119576662B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112738237A (en) * | 2020-12-29 | 2021-04-30 | 武汉华盛东方科技有限公司 | Equipment intelligent perception internet of things gateway management platform based on multi-protocol self-adaption |
| CN118820262A (en) * | 2024-09-18 | 2024-10-22 | 戎行技术有限公司 | Database switching method and device based on data middle layer |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7738863B2 (en) * | 2005-08-25 | 2010-06-15 | Nokia Corporation | IP datacasting middleware |
-
2024
- 2024-12-24 CN CN202411916125.9A patent/CN119576662B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112738237A (en) * | 2020-12-29 | 2021-04-30 | 武汉华盛东方科技有限公司 | Equipment intelligent perception internet of things gateway management platform based on multi-protocol self-adaption |
| CN118820262A (en) * | 2024-09-18 | 2024-10-22 | 戎行技术有限公司 | Database switching method and device based on data middle layer |
Also Published As
| Publication number | Publication date |
|---|---|
| CN119576662A (en) | 2025-03-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7953860B2 (en) | Fast reorganization of connections in response to an event in a clustered computing system | |
| US7747717B2 (en) | Fast application notification in a clustered computing system | |
| US7386557B2 (en) | Persistent client-server database sessions | |
| US20220358106A1 (en) | Proxy-based database scaling | |
| US8484659B2 (en) | Distributed processing of binary objects via message queues including a failover safeguard | |
| US7809690B2 (en) | Performance metric-based selection of one or more database server instances to perform database recovery | |
| US20090292953A1 (en) | Modifying connection records | |
| WO2022063032A1 (en) | Distributed system-oriented fault information association reporting method, and related device | |
| US20070094343A1 (en) | System and method of implementing selective session replication utilizing request-based service level agreements | |
| US7275250B1 (en) | Method and apparatus for correlating events | |
| CN118885406A (en) | Database cluster abnormality testing method and device | |
| CN119576662B (en) | Database middleware switching method and device | |
| CN112711606A (en) | Database access method and device, computer equipment and storage medium | |
| EP3010194B1 (en) | Method of tracing a transaction in a network | |
| US20060085695A1 (en) | Use of incarnation number for resource state cycling | |
| CN117971310A (en) | Redis distributed current limiting method and system based on java | |
| CN115665098A (en) | Transaction recording method under distributed architecture | |
| US8533331B1 (en) | Method and apparatus for preventing concurrency violation among resources | |
| US7770054B2 (en) | Apparatus, system, and method to prevent queue stalling | |
| CN114996238B (en) | Database security management and control method and device, electronic equipment and storage medium | |
| CN120762870B (en) | Annotation-driven idempotent processing method and system | |
| CN117762673A (en) | Access request processing method and device, storage medium and electronic equipment | |
| CN114625488B (en) | Distributed transaction processing method, device, electronic device and computer storage medium | |
| CN120524183A (en) | Application attribute detection method and computing device | |
| CN119719087A (en) | Database security method, apparatus, device, medium, and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |