[go: up one dir, main page]

CN118819819B - A multi-database processing method based on load balancing - Google Patents

A multi-database processing method based on load balancing

Info

Publication number
CN118819819B
CN118819819B CN202410713724.4A CN202410713724A CN118819819B CN 118819819 B CN118819819 B CN 118819819B CN 202410713724 A CN202410713724 A CN 202410713724A CN 118819819 B CN118819819 B CN 118819819B
Authority
CN
China
Prior art keywords
data
database
load balancing
server
load
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202410713724.4A
Other languages
Chinese (zh)
Other versions
CN118819819A (en
Inventor
张扬
梁腾翔
吴家乐
李栋梁
李勇
林东煦
许铭芮
徐婷婷
崔超
祝家鑫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huaneng Information Technology Co Ltd
Original Assignee
Huaneng Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huaneng Information Technology Co Ltd filed Critical Huaneng Information Technology Co Ltd
Priority to CN202410713724.4A priority Critical patent/CN118819819B/en
Publication of CN118819819A publication Critical patent/CN118819819A/en
Application granted granted Critical
Publication of CN118819819B publication Critical patent/CN118819819B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083—Techniques for rebalancing the load in a distributed system
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3051—Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Quality & Reliability (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a multi-database processing method based on load balancing, which belongs to the technical field of load adjustment, classifies target data according to business requirements and data access characteristics, matches database servers with each classified data according to classified databases, selects a load balancing technology and determines configuration rules according to the characteristics of all database servers, sets a central load balancer as a middle layer to connect all database servers according to the configuration rules, routes requests to different database servers connected with the central load balancer according to distribution strategies after the central load balancer receives client requests, monitors execution performance indexes of each database server in real time, and adjusts the load balancing strategies and connection pool parameters of the database servers according to monitoring data. The system is kept to run in a high-performance state, and the stability of the system is improved.

Description

Multi-database processing method based on load balancing
Technical Field
The invention relates to the technical field of load adjustment, in particular to a multi-database processing method based on load balancing.
Background
At present, with the rapid development of services, the configuration of a single fixed database connection pool is difficult to meet the requirement of the development of the services, with the change of the concurrency quantity, the database connection configuration has to be adjusted, however, many systems run continuously for 7 x 24 hours, the configuration of the connection pool for modifying mysql each time must restart services, and each time stop affects the stable running of the service system to a certain extent, so that heavy operation and maintenance cost is brought, and the mode is relatively heavy and difficult to flexibly follow the rhythm of the service system.
Therefore, the invention provides a multi-database processing method based on load balancing.
Disclosure of Invention
The invention provides a multi-database processing method based on load balancing, which is used for adjusting load balancing strategies and database connection pool parameters by deploying a plurality of database servers and selecting a load balancing technology so as to keep the system running in a high-performance state.
In one aspect, the present invention provides a method for processing multiple databases based on load balancing, including:
Step 1, classifying target data according to service requirements and data access characteristics, and matching each classified data with a database server according to a classified database;
Step 2, selecting a load balancing technology and determining configuration rules according to the characteristics of all database servers, and setting a central load balancer as a middle layer according to the configuration rules to connect all database servers;
step 3, after the central load balancer receives the client request, the request is routed to different database servers connected with the central load balancer according to a distribution strategy;
And 4, monitoring the execution performance index of each database server in real time, and adjusting the load balancing strategy and the connection pool parameters of the database servers according to the monitoring data.
On the other hand, classifying the target data according to the service requirements and the data access characteristics includes:
Sending a query request to a service client to acquire service demand data matched with the query request;
Performing first classification on the target data according to the characteristics of the service demand data;
and carrying out priority classification on the data after the first classification according to the data access characteristics to obtain second classification data, wherein the second classification data is the result of classifying the target data.
In another aspect, matching a database server to each classification data includes:
Determining an initial adaptation degree of each second classification data with each server in the classification database;
wherein, the 1 Denotes an initial adaptation degree of the corresponding second classification data with the corresponding server,() Representing an adjustment function, in particularFor a pair ofIs used for the adjustment of the (a),Representing the association of the corresponding server with the ith data in the corresponding second classification data, which is obtained based on the server-data class association mapping table,Representing a preset weight coefficient corresponding to the ith data in the second classification data,Represents the mean function, ln represents the logarithmic function,The standard association degree set for the corresponding server is represented, and n represents the total number of all data in the corresponding second classification data;
According to the priority condition of the corresponding second classified data and combining each related initial adaptation degree, calculating to obtain the final adaptation degree of the corresponding second classified data and each server;
wherein, the Representing the final adaptation degree of the corresponding second classification data and the corresponding server; representing the highest priority among all priorities; representing the priority of the corresponding second classification data; representation pair Min represents a minimum value sign, max represents a maximum value sign;
And screening the server corresponding to the highest adaptation degree from all the final adaptation degrees under each second classification data as a database server corresponding to the second classification data.
On the other hand, according to the characteristics of all database servers, the process of selecting the load balancing technology and determining the configuration rules comprises the following steps:
Based on the characteristics of the database servers and preset standards, load information of each database server is obtained from a load information collection pool, and a first weight coefficient of all indexes under each database server is obtained through a pre-configured dynamic load balancing algorithm;
The specific calculation mode of the first weight coefficient is as follows:
wherein, the Representing an index set normalized for all indexes under a corresponding database serverIndex information degree of the j-th standardized index in (2), ln () represents a logarithmic function,Representing a normalized set of metricsA standard value of the j-th standardized index of the (b),Representing a normalized set of metricsThe first weight coefficient of the j-th standardized indicator,Representing that the corresponding database servers have m indexes in total, and m is more than 2; representing all Maximum value of (2); The representation is based on all Average value of (2).
On the other hand, according to the characteristics of all database servers, selecting a load balancing technology and determining configuration rules, including:
Based on all indexes and first weight coefficients of a database server, evaluating the load condition of the database server by combining an evaluation function to obtain load parameters of the database server;
Matching the load balancing technology of the database server according to the load parameter-load balancing technology mapping table, and customizing the configuration rule of the central load balancer according to the load balancing technology;
combining the configuration rule and the load parameter, dynamically adjusting a first weight coefficient of the database server index to obtain a second weight coefficient;
and determining a configuration rule based on the second weight coefficient and the load balancing technology, and setting a central load balancer as a middle layer to connect all database servers.
In another aspect, the distribution policy is determined jointly by the first policy and the second policy;
The first strategy comprises the IP address of a database server obtained by a central load balancer;
the second strategy comprises that the central load balancer obtains the carrying condition of the database server.
On the other hand, the process of monitoring the execution performance index of each database server in real time comprises the following steps:
Configuring a database performance monitoring tool into each database server, and monitoring log information generated under the execution performance index of the database server in real time according to the index preset standard of the database server;
if the analysis result of the log information contains information which is not consistent with the index preset standard, locking the inconsistent information and carrying out early warning based on the warning rule.
On the other hand, adjusting the load balancing strategy and the connection pool parameters of the database server according to the monitoring data, including:
When an early warning prompt is sent out, locating a database server with abnormality, and acquiring abnormality monitoring data in log information of the located server;
Acquiring the current abnormal state of the corresponding positioned server according to the abnormal monitoring data, and adjusting the load balancing strategy of the positioned server by combining a load balancing technology;
Meanwhile, analyzing the occupation condition of the connection pool resources of the positioned server in the current abnormal state to obtain a first load duty ratio;
monitoring the current occupation condition of the corresponding connection pool resources to obtain a second load duty ratio;
when the second load duty ratio is larger than the first load duty ratio, adjusting corresponding connection pool parameters;
and continuously monitoring the running condition of the database server after the load balancing strategy and the connection pool parameters are adjusted.
The invention provides a multi-database processing method based on load balancing, which is used for adjusting load balancing strategies and database connection pool parameters by deploying a plurality of database servers and selecting a load balancing technology so as to keep the system running in a high-performance state.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flow chart of a multi-database processing method based on load balancing according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Example 1:
As shown in fig. 1, the method for processing multiple databases based on load balancing provided by the embodiment of the invention includes:
Step 1, classifying target data according to service requirements and data access characteristics, and matching each classified data with a database server according to a classified database;
Step 2, selecting a load balancing technology and determining configuration rules according to the characteristics of all database servers, and setting a central load balancer as a middle layer according to the configuration rules to connect all database servers;
step 3, after the central load balancer receives the client request, the request is routed to different database servers connected with the central load balancer according to a distribution strategy;
And 4, monitoring the execution performance index of each database server in real time, and adjusting the load balancing strategy and the connection pool parameters of the database servers according to the monitoring data.
In this embodiment, the service requirements refer to specific requirements and requirements in terms of functions, performance, security and the like that the system needs to meet, such as data access efficiency, load balancing and disaster recovery, performance monitoring and tuning, service expansibility and the like.
In this embodiment, the data access features are features of the manner, frequency, and mode of demand for data, etc. of the target data being accessed and utilized in the system, including access frequency, access mode, data association, data size, etc.
In this embodiment, the classification database contains a plurality of servers, wherein the matching of the servers and the data is based on the classification data.
In the embodiment, the database server is a server specially used for storing and managing data, and mainly comprises a database management system, a storage system, an engine, a query processor and a connection pool, and mainly performs operations such as query, update, deletion and the like on the database.
In this embodiment, the load balancing technique is a technique for distributing workload among multiple servers to ensure that the servers can efficiently co-process requests, and mainly includes categories of DNS-based load balancing, hardware load balancing, software load balancing, four-tier load balancing, seven-tier load balancing, and the like.
In the embodiment, the configuration rules are rules and parameters which need to be set after the load balancing technology is determined, and the rules comprise a load balancing algorithm, a health checking mechanism, a session maintaining mode, a flow limiting strategy and the like.
In this embodiment, the central load balancer is a key component responsible for coordinating and managing the entire database server cluster.
In this embodiment, the middle layer refers to a layer of system components for connecting the client and the back-end database server, and mainly includes contents such as an application server, a cache server, a message queue, a security layer, and the like.
In this embodiment, the client request is a request from an application program to the system requesting the system to perform certain operations or to obtain specific data, including types of requests to insert, query, update, store new data, etc.
In this embodiment, the distribution policy is the way the central load balancer routes client requests to different database servers of the connection, including policies such as polling, minimum connection number, random, performance index based distribution, IP address hashing, etc.
In this embodiment, routing refers to the process of determining the path and manner in which data is transmitted in the network.
In the embodiment, the execution performance index is various data indexes obtained by real-time monitoring and evaluation of the performance of the database server, including indexes such as response time, throughput, load condition, connection pool utilization rate, concurrent connection number and the like.
In this embodiment, the monitoring data is data obtained after performance index monitoring is performed, and is mainly used to obtain data generated by different indexes in the running process, so that a load balancing strategy can be conveniently adjusted based on the condition of the data.
In this embodiment, the connection pool parameters refer to parameters used to configure and manage the database connection pool, including parameters such as maximum number of connections, minimum number of idle connections, connection timeout time, maximum latency, connection validity detection, etc.
The working principle and the beneficial effects of the technical scheme are that through data classification, load balancing technology, request routing and real-time monitoring, the stability, usability and performance of the system are improved, the data access efficiency is optimized, the load balancing of a database server is ensured, the system is kept to operate in a high-performance state, and the system stability is improved.
Example 2:
On the basis of the above embodiment 1, classifying the target data according to the service requirement and the data access characteristics includes:
Sending a query request to a service client to acquire service demand data matched with the query request;
Performing first classification on the target data according to the characteristics of the service demand data;
and carrying out priority classification on the data after the first classification according to the data access characteristics to obtain second classification data, wherein the second classification data is the result of classifying the target data.
In this embodiment, the service client is a service using a particular service system or application.
In this embodiment, the business requirement data is relevant data collected, collated or extracted according to the specific business requirement, and may be customer data, first party data, financial statement, etc.
In this embodiment, the first classification is to perform preliminary classification on the target data according to the characteristics of the service requirement data, for example, classification is performed according to the characteristics of the data type, the access frequency, the data size, and the like.
In this embodiment, prioritization refers to further sorting of the first categorized data according to data access characteristics and importance.
In this embodiment, the second classification is a result of prioritizing the target data based on the data access characteristics and importance after the first classification. The classification result comprises the sorting of the priority levels and the classification of the priority levels, and is divided into three levels of middle and low.
The technical scheme has the advantages that the system query response speed is improved, the data management is optimized, the system is kept to operate in a high-performance state, and the system stability is improved through classification and priority division.
Example 3:
on the basis of the above embodiment 1, matching a database server to each classification data includes:
Determining an initial adaptation degree of each second classification data with each server in the classification database;
wherein, the 1 Denotes an initial adaptation degree of the corresponding second classification data with the corresponding server,() Representing an adjustment function, in particularFor a pair ofIs used for the adjustment of the (a),Representing the association of the corresponding server with the ith data in the corresponding second classification data, which is obtained based on the server-data class association mapping table,Representing a preset weight coefficient corresponding to the ith data in the second classification data,Represents the mean function, ln represents the logarithmic function,The standard association degree set for the corresponding server is represented, and n represents the total number of all data in the corresponding second classification data;
According to the priority condition of the corresponding second classified data and combining each related initial adaptation degree, calculating to obtain the final adaptation degree of the corresponding second classified data and each server;
wherein, the Representing the final adaptation degree of the corresponding second classification data and the corresponding server; representing the highest priority among all priorities; representing the priority of the corresponding second classification data; representation pair Min represents a minimum value sign, max represents a maximum value sign;
And screening the server corresponding to the highest adaptation degree from all the final adaptation degrees under each second classification data as a database server corresponding to the second classification data.
In this embodiment, the initial fitness represents an initial degree of matching of each second classification data with each server in the classification database.
In this embodiment, the adjustment function is a function for adjusting the initial fitness according to the specific attribute of the data and the degree of association of the server.
In this embodiment, the degree of association indicates the degree of association between the server and specific data in the second classification data.
In this embodiment, the server-data class association mapping table is a mapping table for representing the association between servers and different data classes.
In this embodiment, the preset weight coefficient refers to a preset weight coefficient corresponding to data in the second classification data.
In this embodiment, the final fitness refers to a fitness value calculated under the condition of priority corresponding to the second classification data and each server.
The technical scheme has the advantages that the initial adaptation degree of each classified data and the server is calculated, the final adaptation degree is calculated by combining the priority, the system is kept to operate in a high-performance state, and the system stability is improved.
Example 4:
Based on the above embodiment 1, the process of selecting the load balancing technology and determining the configuration rule according to the characteristics of all database servers includes:
Based on the characteristics of the database servers and preset standards, load information of each database server is obtained from a load information collection pool, and a first weight coefficient of all indexes under each database server is obtained through a pre-configured dynamic load balancing algorithm;
The specific calculation mode of the first weight coefficient is as follows:
wherein, the Representing an index set normalized for all indexes under a corresponding database serverIndex information degree of the j-th standardized index in (2), ln () represents a logarithmic function,Representing a normalized set of metricsA standard value of the j-th standardized index of the (b),Representing a normalized set of metricsThe first weight coefficient of the j-th standardized indicator,Representing that the corresponding database servers have m indexes in total, and m is more than 2; representing all Maximum value of (2); The representation is based on all Average value of (2).
In this embodiment, the preset criteria refers to a set of criteria predetermined when designing the system, including the index considered by the load balancing algorithm and its weight allocation, the standard value range or threshold value of each index, and so on.
In this embodiment, the database server has the characteristics of data processing capability, reliability and stability, expansibility, high performance and the like.
In this embodiment, the load information collection pool is a software module for collecting, monitoring and storing load information of each node in the system in real time, periodically or on demand.
In this embodiment, the load information refers to the load condition of the database server, and includes various indexes such as CPU utilization, memory occupation condition, disk I/O operation, network traffic, and the like.
In this embodiment, the dynamic load balancing algorithm is an algorithm for adjusting system resource allocation in a real-time environment to achieve load balancing. The method comprises the steps of carrying out standardization processing on the obtained load information, mapping all indexes to the same dimension, and dynamically adjusting the load to be distributed to each server according to a preset index weight coefficient.
In this embodiment, the first weight coefficient is a coefficient for a dynamic load balancing algorithm obtained by calculating after normalizing all indexes under each database server.
In this embodiment, the index informativeness indicates that information uncertainty for all indexes under the database server is poor for all indexes.
The working principle and the beneficial effects of the technical scheme are that the dynamic load balancing algorithm effectively schedules the load of the database server through the standardized index set and the calculation of the first weight coefficient, so that the system is kept to operate in a high-performance state, and the system stability is improved.
Example 5:
Based on the above embodiment 4, selecting a load balancing technique and determining a configuration rule according to characteristics of all database servers includes:
Based on all indexes and first weight coefficients of a database server, evaluating the load condition of the database server by combining an evaluation function to obtain load parameters of the database server;
Matching the load balancing technology of the database server according to the load parameter-load balancing technology mapping table, and customizing the configuration rule of the central load balancer according to the load balancing technology;
combining the configuration rule and the load parameter, dynamically adjusting a first weight coefficient of the database server index to obtain a second weight coefficient;
and determining a configuration rule based on the second weight coefficient and the load balancing technology, and setting a central load balancer as a middle layer to connect all database servers.
In this embodiment, the evaluation function is a parameter for evaluating the load situation;
wherein, the The result of the evaluation is indicated,Representing that the database server has q metrics in total,The parameter value representing the kth index,A first weight coefficient representing a kth index.
In this embodiment, the load parameter is an index for evaluating the load condition of the database server. The method comprises the steps of CPU utilization rate, memory utilization rate, disk read-write speed, network bandwidth utilization rate, request response time and the like.
In this embodiment, the load conditions are the current workload and pressure conditions of the database server, including the number of requests currently being processed, resource utilization, response time, etc.
In this embodiment, the load parameter-load balancing technique mapping table is a mapping table indicating the correspondence between load parameters and load balancing techniques.
In the embodiment, the configuration rules refer to a series of operation rules determined according to information such as load parameters, load balancing technology, weight coefficients and the like, and the operation rules comprise rules such as load balancing algorithm selection, server health check setting, load scheduling strategies, weight coefficient adjustment rules and the like.
In this embodiment, the second weight coefficient is determined as follows:
wherein, the A second weight coefficient is represented and is used to represent,A first weight coefficient is represented and a second weight coefficient is represented,Representing a total of h configuration rules,The load parameter is indicated as such,The setting parameters representing the f-th configuration rule,The parameter weight representing the f-th configuration rule,The standard parameters are represented by the values of the standard parameters,Representing from allAcquiring setting parameters of a configuration rule corresponding to the maximum weight;1、 and the conversion coefficient is represented, so that the unification of the calculated values is convenient.
The working principle and the beneficial effects of the technical scheme are that the load balancing optimization of the database server is realized by dynamically adjusting the weight coefficient, selecting the proper load balancing technology and customizing the configuration rule, the system is kept to operate in a high-performance state, and the system stability is improved.
Example 6:
on the basis of the above embodiment 1, the distribution policy is determined by both the first policy and the second policy;
The first strategy comprises the IP address of a database server obtained by a central load balancer;
the second strategy comprises that the central load balancer obtains the carrying condition of the database server.
In this embodiment, the delivery situation refers to the current workload condition of the database server, including the resource utilization, network load, service availability, etc.
The technical scheme has the advantages that the central load balancer is used for acquiring the IP address and the carrying condition of the database server, intelligent load balancing scheduling is realized, the system is kept to operate in a high-performance state, and the system stability is improved.
Example 7:
on the basis of the above embodiment 1, the process of monitoring the performance index of each database server in real time includes:
Configuring a database performance monitoring tool into each database server, and monitoring log information generated under the execution performance index of the database server in real time according to the index preset standard of the database server;
if the analysis result of the log information contains information which is not consistent with the index preset standard, locking the inconsistent information and carrying out early warning based on the warning rule.
In this embodiment, the database performance monitoring tool is a software special for monitoring, analyzing and managing the execution performance of the database system, and generally comprises SQL Profiler, MYSQL ENTERPRISE Monitor, PGAdmin and other tools.
In this embodiment, the index preset criteria define the criteria of the performance index of the database server in advance.
In this embodiment, the log information refers to records of the running state of the database server and the performance index of execution, including performance statistics, query execution conditions, abnormal events, user activities, and the like.
In this embodiment, the alarm rules are a set of conditions and actions preset in the database performance monitoring tool for generating real-time alarm notifications based on the analysis results of the monitored data. The method comprises the steps of triggering conditions, duration, alarm level, alarm action, notification object and the like.
The technical scheme has the advantages that the database performance monitoring tool monitors the execution performance of the server in real time, analysis log information is compared with a preset standard, an abnormal condition triggers an alarm notice, the system is kept to operate in a high-performance state, and the system stability is improved.
Example 8:
Based on the above embodiment 7, adjusting the load balancing policy and the connection pool parameters of the database server according to the monitoring data includes:
When an early warning prompt is sent out, locating a database server with abnormality, and acquiring abnormality monitoring data in log information of the located server;
Acquiring the current abnormal state of the corresponding positioned server according to the abnormal monitoring data, and adjusting the load balancing strategy of the positioned server by combining a load balancing technology;
Meanwhile, analyzing the occupation condition of the connection pool resources of the positioned server in the current abnormal state to obtain a first load duty ratio;
monitoring the current occupation condition of the corresponding connection pool resources to obtain a second load duty ratio;
when the second load duty ratio is larger than the first load duty ratio, adjusting corresponding connection pool parameters;
and continuously monitoring the running condition of the database server after the load balancing strategy and the connection pool parameters are adjusted.
In the embodiment, the abnormal monitoring data refers to information and index data generated when the database server is abnormal in the operation process, and the information and index data comprise data such as abnormal events, performance index abnormality, connection pool resource occupation conditions, index comparison before and after load balancing strategy adjustment and the like.
In this embodiment, the current abnormal state is an abnormal situation that the database server has a performance degradation, an overload, a response delay, and the like at a certain time or within a certain period of time.
In this embodiment, the connection pool resource refers to a buffer in the database server for managing and allocating database connections, including connection count limitations, idle connections, active connections, latency, and the like.
In this embodiment, the occupation condition refers to the degree to which the connection pool resource or other system resources are used in a specific time period, and includes the occupation of the connection pool resource, the occupation of the CPU, the occupation of the memory, the occupation of the network bandwidth, and the like.
In this embodiment, the first load ratio refers to when analyzing occupancy of the located database server connection pool resources in an abnormal state.
In this embodiment, the second load duty ratio refers to monitoring and analyzing real-time occupancy of resources of the database server connection pool after adjustment of the server load balancing policy.
In this embodiment, the adjustments include operations such as request routing adjustments, weight adjustments, failover, connection pool management, and the like.
The technical scheme has the advantages that the system is kept to run in a high-performance state by monitoring the abnormal state and adjusting the load balance and the connection pool parameters, so that the system stability is improved.
It should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention, and not for limiting the same, and although the present invention has been described in detail with reference to the above-mentioned embodiments, it should be understood by those skilled in the art that the technical solution described in the above-mentioned embodiments may be modified or some technical features may be equivalently replaced, and these modifications or substitutions do not make the essence of the corresponding technical solution deviate from the spirit and scope of the technical solution of the embodiments of the present invention.

Claims (7)

1.一种基于负载均衡的多数据库处理方法，其特征在于，包括：1. A multi-database processing method based on load balancing, characterized by comprising: 步骤1：根据业务需求和数据访问特点将目标数据进行分类，并根据分类数据库，向每个分类数据匹配数据库服务器；Step 1: Classify the target data according to business requirements and data access characteristics, and match the database server to each classified data according to the classification database; 步骤2：根据所有数据库服务器的特点，选择负载均衡技术并确定配置规则，根据配置规则设置中央负载均衡器作为中间层连接所有数据库服务器；Step 2: Select load balancing technology and determine configuration rules based on the characteristics of all database servers. Set up a central load balancer as the middle layer to connect all database servers according to the configuration rules. 步骤3：当中央负载均衡器接收到客户端请求后，根据分发策略将请求路由给中央负载均衡器所连接的不同数据库服务器；Step 3: When the central load balancer receives the client request, it routes the request to different database servers connected to the central load balancer according to the distribution strategy; 步骤4：实时监控每个数据库服务器的执行性能指标，并根据监控数据调整负载均衡策略和数据库服务器的连接池参数；Step 4: Monitor the execution performance indicators of each database server in real time, and adjust the load balancing strategy and connection pool parameters of the database server based on the monitoring data; 其中，根据所有数据库服务器的特点，选择负载均衡技术并确定配置规则的过程中，包括：Among them, according to the characteristics of all database servers, the process of selecting load balancing technology and determining configuration rules includes: 基于数据库服务器的特点和预设标准，从负载信息收集池中获取每个数据库服务器的负载信息，通过预先配置的动态负载均衡算法，获取每个数据库服务器下所有指标的第一权重系数；Based on the characteristics of the database server and preset standards, the load information of each database server is obtained from the load information collection pool, and the first weight coefficient of all indicators under each database server is obtained through a pre-configured dynamic load balancing algorithm; 其中，第一权重系数的具体计算方式如下：The specific calculation method of the first weight coefficient is as follows: ；其中，表示对相应数据库服务器下所有指标进行标准化后的指标集合中第j个标准化指标的指标信息度，ln( )表示对数函数，表示标准化指标集合中第j个标准化指标的标准值，表示标准化指标集合中第j个标准化指标的第一权重系数，表示相应数据库服务器共有m个指标，且m＞2；表示所有中的最大值；表示基于所有的平均值。 ;in, Indicates the standardized indicator set of all indicators under the corresponding database server The index information degree of the jth standardized index in , ln( ) represents the logarithmic function, Represents a standardized set of indicators The standard value of the jth standardized indicator in , Represents a standardized set of indicators The first weight coefficient of the jth standardized index in , Indicates that the corresponding database server has a total of m indicators, and m>2; Indicates all The maximum value in ; Indicates that based on all The average value of . 2.根据权利要求1所述的一种基于负载均衡的多数据库处理方法，其特征在于，根据业务需求和数据访问特点将目标数据进行分类，包括：2. The multi-database processing method based on load balancing according to claim 1 is characterized in that the target data is classified according to business requirements and data access characteristics, including: 向业务客户端发送查询请求，获取与所述查询请求匹配的业务需求数据；Sending a query request to a business client to obtain business demand data matching the query request; 根据业务需求数据的特点将目标数据进行第一分类；The target data is first classified according to the characteristics of the business demand data; 按照数据访问特点对第一分类后的数据进行优先级划分，得到第二分类数据，其中，所述第二分类数据即为目标数据进行分类的结果。The first classified data is prioritized according to data access characteristics to obtain second classified data, wherein the second classified data is the result of classifying the target data. 3.根据权利要求1所述的一种基于负载均衡的多数据库处理方法，其特征在于，向每个分类数据匹配数据库服务器，包括：3. The multi-database processing method based on load balancing according to claim 1, characterized in that matching a database server to each classified data comprises: 确定每个第二分类数据与分类数据库中每个服务器的初始适配度；determining an initial fit of each second classification data to each server in the classification database; ；其中，1表示对应第二分类数据与相应服务器的初始适配度，( )表示调整函数，具体为对的调整，表示对应服务器与相应第二分类数据中第i个数据的关联度，是基于服务器-数据类别关联度映射表得到，表示对应第二分类数据中第i个数据的预设权重系数，表示均值函数，ln表示对数函数，表示向对应服务器设定的标准关联度，n表示对应第二分类数据中所有数据的总个数； ;in, 1 indicates the initial fit between the second classification data and the corresponding server. ( ) represents the adjustment function, which is right Adjustment, represents the correlation between the corresponding server and the i-th data in the corresponding second classification data, which is obtained based on the server-data category correlation mapping table. represents the preset weight coefficient corresponding to the i-th data in the second classification data, represents the mean function, ln represents the logarithmic function, represents the standard relevance set to the corresponding server, and n represents the total number of all data in the corresponding second classification data; 根据对应第二分类数据的优先级情况，且结合涉及到的每个初始适配度，计算得到对应第二分类数据与每个服务器的最终适配度；According to the priority of the corresponding second classification data and in combination with each initial fitness involved, the final fitness between the corresponding second classification data and each server is calculated; ；其中，表示对应的第二分类数据与相应服务器的最终适配度；表示所有优先级中的最高优先级；表示对应的第二分类数据的优先级；表示对的向下取整函数；min表示最小值符号；max表示最大值符号； ;in, Indicates the final degree of fit between the corresponding second classification data and the corresponding server; Indicates the highest priority among all priorities; Indicates the priority of the corresponding second classification data; Express The floor rounding function of ; min represents the minimum value symbol; max represents the maximum value symbol; 从每个第二分类数据下的所有最终适配度中筛选最高适配度对应的服务器作为对应第二分类数据的数据库服务器。A server corresponding to the highest fitness degree is selected from all final fitness degrees under each second classification data as a database server corresponding to the second classification data. 4.根据权利要求1所述的一种基于负载均衡的多数据库处理方法，其特征在于，根据所有数据库服务器的特点，选择负载均衡技术并确定配置规则，包括：4. A method for processing multiple databases based on load balancing according to claim 1, characterized in that, according to the characteristics of all database servers, load balancing technology is selected and configuration rules are determined, including: 基于数据库服务器的所有指标及第一权重系数，结合评估函数对所述数据库服务器的负载情况进行评估，得到所述数据库服务器的负载参数；Based on all indicators and the first weight coefficient of the database server, the load condition of the database server is evaluated in combination with the evaluation function to obtain the load parameter of the database server; 根据负载参数-负载均衡技术映射表匹配所述数据库服务器的负载均衡技术，根据负载均衡技术定制中央负载均衡器的配置规则；Matching the load balancing technology of the database server according to the load parameter-load balancing technology mapping table, and customizing the configuration rules of the central load balancer according to the load balancing technology; 结合配置规则和负载参数，动态调整数据库服务器指标的第一权重系数，得到第二权重系数；In combination with the configuration rules and the load parameters, the first weight coefficient of the database server indicator is dynamically adjusted to obtain a second weight coefficient; 基于第二权重系数和负载均衡技术确定配置规则，并设置中央负载均衡器作为中间层连接所有数据库服务器。A configuration rule is determined based on the second weight coefficient and the load balancing technology, and a central load balancer is set as an intermediate layer to connect all database servers. 5.根据权利要求1所述的一种基于负载均衡的多数据库处理方法，其特征在于，所述分发策略是由第一策略和第二策略共同确定的；5. A multi-database processing method based on load balancing according to claim 1, characterized in that the distribution strategy is determined by the first strategy and the second strategy; 所述第一策略包括：中央负载均衡器获得的数据库服务器的IP地址；The first strategy includes: an IP address of a database server obtained by a central load balancer; 所述第二策略包括：中央负载均衡器获取数据库服务器的运载情况。The second strategy includes: the central load balancer obtains the load status of the database server. 6.根据权利要求1所述的一种基于负载均衡的多数据库处理方法，其特征在于，实时监控每个数据库服务器的执行性能指标的过程中，包括：6. The method for processing multiple databases based on load balancing according to claim 1, characterized in that the process of real-time monitoring the execution performance index of each database server includes: 将数据库性能监控工具配置到每个数据库服务器中，并按照所述数据库服务器的指标预设标准实时监控所述数据库服务器的执行性能指标下所产生的日志信息；A database performance monitoring tool is configured in each database server, and log information generated under the execution performance index of the database server is monitored in real time according to the preset index standard of the database server; 若对日志信息的分析结果中存在不与指标预设标准一致的信息，则锁定不一致的信息并基于报警规则进行预警。If the analysis results of the log information contain information that is inconsistent with the preset standards of the indicator, the inconsistent information will be locked and an early warning will be issued based on the alarm rules. 7.根据权利要求6所述的一种基于负载均衡的多数据库处理方法，其特征在于，根据监控数据调整负载均衡策略和数据库服务器的连接池参数，包括：7. A multi-database processing method based on load balancing according to claim 6, characterized in that the load balancing strategy and the connection pool parameters of the database server are adjusted according to the monitoring data, comprising: 当发出预警提醒时，定位出现异常的数据库服务器，并获取所定位的服务器的日志信息中的异常监测数据；When an early warning is issued, locate the database server where the abnormality occurs, and obtain the abnormal monitoring data in the log information of the located server; 根据异常监测数据获取对应所定位的服务器的当前异常状态，且结合负载均衡技术，对所定位的服务器负载均衡策略进行调整；The current abnormal state of the corresponding located server is obtained according to the abnormal monitoring data, and the load balancing strategy of the located server is adjusted in combination with the load balancing technology; 同时，分析所定位的服务器的连接池资源在当前异常状态下的占用情况，得到第一负载占比；At the same time, the occupancy of the connection pool resources of the located server in the current abnormal state is analyzed to obtain a first load ratio; 监测对应连接池资源的当下占用情况得到第二负载占比；Monitor the current occupancy of the corresponding connection pool resources to obtain the second load ratio; 当第二负载占比大于第一负载占比时，对相应连接池参数进行调整；When the second load ratio is greater than the first load ratio, adjusting the corresponding connection pool parameters; 当负载均衡策略和连接池参数调整完成后，持续监控数据库服务器的运行情况。After the load balancing strategy and connection pool parameters are adjusted, continue to monitor the operation of the database server.
CN202410713724.4A 2024-06-04 2024-06-04 A multi-database processing method based on load balancing Active CN118819819B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410713724.4A CN118819819B (en) 2024-06-04 2024-06-04 A multi-database processing method based on load balancing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410713724.4A CN118819819B (en) 2024-06-04 2024-06-04 A multi-database processing method based on load balancing

Publications (2)

Publication Number Publication Date
CN118819819A CN118819819A (en) 2024-10-22
CN118819819B true CN118819819B (en) 2025-07-18

Family

ID=93080816

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410713724.4A Active CN118819819B (en) 2024-06-04 2024-06-04 A multi-database processing method based on load balancing

Country Status (1)

Country Link
CN (1) CN118819819B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120144413B (en) * 2025-05-14 2025-10-03 华能信息技术有限公司 A database load dynamic management system

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106326012A (en) * 2016-08-25 2017-01-11 中国农业银行股份有限公司 Web application cluster buffer utilization method and system

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108494868A (en) * 2018-03-30 2018-09-04 三盟科技股份有限公司 A kind of load-balancing method under the operation system based on cloud and system
CN110011928B (en) * 2019-04-19 2022-08-19 平安科技（深圳）有限公司 Flow balancing load method and device, computer equipment and storage medium
CN117768394A (en) * 2022-09-16 2024-03-26 中兴通讯股份有限公司 A load balancing processing method, device, storage medium and electronic device
CN116896503A (en) * 2023-06-28 2023-10-17 四川迅游网络科技股份有限公司 Acceleration method for dynamically judging data flow
CN117155936A (en) * 2023-07-31 2023-12-01 北京航天计量测试技术研究所 Method and device for realizing load balancing
DE202024100297U1 (en) * 2024-01-20 2024-02-27 Priti Dimri Load balancing system in a cloud environment for algorithmic efficiency

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106326012A (en) * 2016-08-25 2017-01-11 中国农业银行股份有限公司 Web application cluster buffer utilization method and system

Also Published As

Publication number Publication date
CN118819819A (en) 2024-10-22

Similar Documents

Publication Publication Date Title
US6912533B1 (en) Data mining agents for efficient hardware utilization
CN100568193C (en) Systems and methods for performance management in a multi-tier computing environment
US8190593B1 (en) Dynamic request throttling
US6643614B2 (en) Enterprise management system and method which indicates chaotic behavior in system resource usage for more accurate modeling and prediction
US7058622B1 (en) Method, apparatus and system for screening database queries prior to submission to a database
US7509671B1 (en) Systems and methods for assigning priority to jobs in a reporting system
US7607135B2 (en) Apparatus and method for enhancing performance of a computer system
US20020129127A1 (en) Apparatus and method for routing a transaction to a partitioned server
US20030204588A1 (en) System for monitoring process performance and generating diagnostic recommendations
US20080168446A1 (en) Method and Apparatus for Handling Service Requests in a Data Processing System
CN113315836B (en) File access request scheduling method and device, electronic equipment and storage medium
CN109271243B (en) Cluster task management system
CN114035962A (en) Resource allocation method and device, computer equipment and storage medium
US8418183B2 (en) Resource information collection and delivery method and system
CN113111083A (en) Method, device, equipment, storage medium and program product for data query
CN118819819B (en) A multi-database processing method based on load balancing
WO2008055129A2 (en) Method and apparatus for distribution of data among computing resources
CN112866132A (en) Dynamic load balancer and method for massive identification
CN118233469A (en) Computing task scheduling method and system based on delay perception and load balancing
CN119201399A (en) Task data configuration method, device, electronic device and storage medium
CN118400377B (en) Cloud storage platform resource allocation method
Tong et al. Service Reliability Oriented Modeling for the Failure of Cloud Data Center
US12028271B2 (en) Prioritizing messages for server processing based on monitoring and predicting server resource utilization
CN120263857A (en) Dynamic access control method and system for computer communication
US20250343833A1 (en) Independent load balancing to prevent node agent overload

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
EE01 Entry into force of recordation of patent licensing contract
EE01 Entry into force of recordation of patent licensing contract

Application publication date: 20241022

Assignee: HUANENG JINMEN THERMOELECTRICITY CO.,LTD.

Assignor: Huaneng Information Technology Co.,Ltd.

Contract record no.: X2025980034700

Denomination of invention: A multi-database processing method based on load balancing

Granted publication date: 20250718

License type: Common License

Record date: 20251114