CN108848039B - Server, message distribution method and storage medium - Google Patents
Server, message distribution method and storage medium Download PDFInfo
- Publication number
- CN108848039B CN108848039B CN201810370069.1A CN201810370069A CN108848039B CN 108848039 B CN108848039 B CN 108848039B CN 201810370069 A CN201810370069 A CN 201810370069A CN 108848039 B CN108848039 B CN 108848039B
- Authority
- CN
- China
- Prior art keywords
- client
- message
- message queue
- clients
- consuming
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 230000008569 process Effects 0.000 claims abstract description 27
- 238000009825 accumulation Methods 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/50—Queue scheduling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/53—Network services using third party service providers
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer And Data Communications (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention relates to a server, a message distribution method and a storage medium, wherein the method comprises the following steps: after the business system of each message consumer accesses the message system, starting a scanning process to scan the backlog value of the client consumption message queue of each business system and the message amount consumed per second; when the backlog value of a client consuming message queue of a service system is larger than a preset threshold value and the ratio of the backlog value of the consuming message queue to the message quantity consumed per second is larger than a preset ratio value, marking the client of the service system as a first client and sending marking information to the first client; and after receiving a balance request of the first client for reallocating the message queue, controlling the first client to be disconnected from the message queue within preset time, and allocating the message queue to clients of other business systems consuming the message queue except the first client. The invention can prevent the accumulation of messages and avoid the IO problem of the disk.
Description
Technical Field
The present invention relates to the field of communications technologies, and in particular, to a server, a message distribution method, and a storage medium.
Background
At present, the popularity of the message system is increasing, a message producer can apply for publishing a related message topic by using a message producer identity in the message system, and a message consumer can apply for subscribing the related message topic by using a message consumer identity in the message system. A consumer typically consumes a queue of consumption by a cluster of machines. Because the consumer cluster can be used by the user to access the thread to occupy the resource, the message consumption of the partial machine is very slow. When the message issuing speed is high and the machine consuming slowly occupies the message queue, the message is accumulated in the message system all the time, and the message system is easy to have disk IO problems, such as insufficient disk space or data loss, and the system performance is low.
Disclosure of Invention
The invention aims to provide a server, a message distribution method and a storage medium, aiming at preventing message accumulation, avoiding the disk IO problem and improving the system performance.
In order to achieve the above object, the present invention provides a server, which includes a memory and a processor connected to the memory, wherein the memory stores a processing system capable of running on the processor, and when executed by the processor, the processing system implements the following steps:
scanning, namely starting a scanning process to scan the backlog value of a client consumption message queue of each service system and the message quantity consumed per second after the service system of each message consumer accesses the message system;
marking, namely marking the client of the service system as a first client and sending marking information to the first client when the backlog value of a client consumption message queue of the service system is greater than a preset threshold value and the ratio of the backlog value of the consumption message queue to the message quantity consumed per second is greater than a preset ratio;
and a reallocation step, wherein after receiving a balance request of the first client for reallocating the message queue, the first client is controlled to be disconnected from the message queue within a preset time, and the message queue is allocated to the clients of other business systems consuming the message queue except the first client.
Preferably, the processing system, when executed by the processor, further implements the steps of:
in the clients of other service systems, when the backlog value of a client consumption message queue of the service system is larger than a preset threshold value and the ratio of the backlog value of the consumption message queue to the message quantity consumed per second is larger than a preset ratio value, marking the client of the service system as a second client, and sending the marking information to the second client;
after receiving a balance request of the second client for reallocating the message queue, analyzing whether the time for disconnecting the first client from the message queue is more than or equal to the preset time;
if so, controlling the second client to disconnect from the message queue within preset time, and distributing the message queue to clients of other service systems consuming the message queue except the second client;
if not, the second client is controlled to be disconnected from the message queue within preset time, and the message queue is distributed to the clients of other business systems consuming the message queue except the first client and the second client.
Preferably, the processing system, when executed by the processor, further implements the steps of:
counting the number of the clients of the currently marked service system, and calculating the proportion of the clients of the currently marked service system in all the clients of the service system consuming the message queue;
analyzing whether the proportion is larger than a preset proportion or not;
if so, sending the reminding message to the message management system so that the message management system sends the reminding message to a preset terminal.
Preferably, the processing system, when executed by the processor, further implements the steps of:
when receiving the message aiming at the restarting instance of the message queue, deleting the marks and the mark information of the clients of all the marked business systems, and redistributing the message queue to the clients of all the business systems consuming the message queue.
In order to achieve the above object, the present invention further provides a message distribution method, where the message distribution method includes:
s1, after the business system of each message consumer accesses the message system, starting a scanning process to scan the backlog value of the client consumption message queue of each business system and the message amount consumed per second;
s2, when the backlog value of the client consuming message queue of the service system is larger than the preset threshold value and the ratio of the backlog value of the consuming message queue to the message amount consumed per second is larger than the preset ratio value, marking the client of the service system as a first client and sending the marking information to the first client;
s3, after receiving the balancing request of the first client for reallocating the message queue, controlling the first client to disconnect from the message queue within a predetermined time, and allocating the message queue to the clients of other business systems consuming the message queue except the first client.
Preferably, after the step S3, the method further includes:
in the clients of other service systems, when the backlog value of a client consumption message queue of the service system is larger than a preset threshold value and the ratio of the backlog value of the consumption message queue to the message quantity consumed per second is larger than a preset ratio value, marking the client of the service system as a second client, and sending the marking information to the second client;
after receiving a balance request of the second client for reallocating the message queue, analyzing whether the time for disconnecting the first client from the message queue is more than or equal to the preset time;
if so, controlling the second client to disconnect from the message queue within preset time, and distributing the message queue to clients of other service systems consuming the message queue except the second client;
if not, the second client is controlled to be disconnected from the message queue within preset time, and the message queue is distributed to the clients of other business systems consuming the message queue except the first client and the second client.
Preferably, the method for message distribution further includes:
counting the number of the clients of the currently marked service system, and calculating the proportion of the clients of the currently marked service system in all the clients of the service system consuming the message queue;
analyzing whether the proportion is larger than a preset proportion or not;
if so, sending the reminding message to the message management system so that the message management system sends the reminding message to a preset terminal.
Preferably, the method for message distribution further includes:
when receiving the message aiming at the restarting instance of the message queue, deleting the marks and the mark information of the clients of all the marked business systems, and redistributing the message queue to the clients of all the business systems consuming the message queue.
Preferably, the predetermined threshold is 10 ten thousand, and the predetermined ratio is 10800.
The present invention also provides a computer readable storage medium having stored thereon a processing system, which when executed by a processor implements the steps of the above-described method of message distribution.
The invention has the beneficial effects that: when the client of each service system consumes the message queue, the invention scans the backlog value of the consumed message queue and the message amount consumed per second, if the client with low consumption is found, the client is marked with the mark of low consumption, then the redistribution operation is started, the client is controlled to be incapable of consuming the message queue within the preset time, and the message queue is distributed to other clients consuming the message queue except the client.
Drawings
FIG. 1 is a schematic diagram of an alternative application environment according to various embodiments of the present invention;
FIG. 2 is a diagram illustrating a first embodiment of a client side consumption message queue according to the present invention;
FIG. 3 is a diagram illustrating a second embodiment of a client consumption message queue according to the present invention;
FIG. 4 is a diagram illustrating a third embodiment of a client side consumption message queue according to the present invention;
FIG. 5 is a diagram illustrating a fourth embodiment of a client consumption message queue according to the present invention;
fig. 6 is a flowchart illustrating a message distribution method according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the description relating to "first", "second", etc. in the present invention is for descriptive purposes only and is not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present invention.
Fig. 1 is a schematic application environment diagram of a message distribution method according to a preferred embodiment of the present invention. The schematic diagram of the application environment includes a server 1, a client 2 of each service system, and a message management system 3. The server 1 may perform data interaction with the client 2 and the message management system 3 of each service system through a suitable technology such as a network or a near field communication technology.
The terminal where the client 2 of the service system is located includes, but is not limited to, any electronic product capable of performing human-computer interaction with a user through a keyboard, a mouse, a remote controller, a touch panel, or a voice control device, for example, a mobile device such as a Personal computer, a tablet computer, a smart phone, a Personal Digital Assistant (PDA), a game machine, an interactive web Television (IPTV), an intelligent wearable device, a navigation device, or the like, or a fixed terminal such as a Digital TV, a desktop computer, a notebook, a server, or the like.
The server 1 is a device capable of automatically performing numerical calculation and/or information processing in accordance with a command set or stored in advance. The server 1 may be a computer, or may be a single network server, a server group composed of a plurality of network servers, or a cloud composed of a large number of hosts or network servers based on cloud computing, where cloud computing is one of distributed computing and is a super virtual computer composed of a group of loosely coupled computers.
In the present embodiment, the server 1 may include, but is not limited to, a memory 11, a processor 12, and a network interface 13, which may be communicatively connected to each other through a system bus, wherein the memory 11 stores a processing system operable on the processor 12. It is noted that fig. 1 only shows the server 1 with components 11-13, but it is to be understood that not all of the shown components are required to be implemented, and that more or fewer components may be implemented instead.
The storage 11 includes a memory and at least one type of readable storage medium. The memory provides cache for the operation of the server 1; the readable storage medium may be a non-volatile storage medium such as flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the readable storage medium may be an internal storage unit of the server 1, such as a hard disk of the server 1; in other embodiments, the non-volatile storage medium may be an external storage device of the server 1, such as a plug-in hard disk provided on the server 1, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like. In this embodiment, the readable storage medium of the memory 11 is generally used for storing an operating system installed in the server 1 and various application software, such as a program code of a processing system in an embodiment of the present invention. Further, the memory 11 may also be used to temporarily store various types of data that have been output or are to be output.
The processor 12 may be a Central Processing Unit (CPU), controller, microcontroller, microprocessor, or other data Processing chip in some embodiments. The processor 12 is generally used for controlling the overall operation of the server 1, such as performing control and processing related to data interaction or communication with the client 2 of each service system. In this embodiment, the processor 12 is configured to run the program code stored in the memory 11 or process data, for example, run a processing system.
The network interface 13 may comprise a wireless network interface or a wired network interface, and the network interface 13 is generally used for establishing communication connection between the server 1 and other electronic devices. In this embodiment, the network interface 13 is mainly used to connect the server 1 with the clients 2 of multiple service systems, and establish data transmission channels and communication connections between the server 1 and the clients 2 of multiple service systems.
The processing system is stored in the memory 11 and includes at least one computer readable instruction or program stored in the memory 11, which is executable by the processor 12 to implement the method of the embodiments of the present application; and the at least one computer readable instruction may be divided into different logic blocks depending on the functions implemented by the respective portions.
In one embodiment, the processing system described above, when executed by the processor 12, performs the following steps:
scanning, namely starting a scanning process to scan the backlog value of a client consumption message queue of each service system and the message quantity consumed per second after the service system of each message consumer accesses the message system;
the business system may be a system used by each department in an enterprise, such as a financial system. The message system is installed in the server to realize the distribution of the message queue. After the client of each service system accesses the message system, the message queue can be distributed to one or more clients of the service system for consumption.
In the consumption process, a scanning process is started to perform real-time or timing scanning on the client of each accessed service system, wherein the scanning process comprises the steps of scanning the backlog value of the client consumption message queue of each service system and the message amount tps consumed per second so as to analyze whether the client consumption message queue of each service system is slow or not.
Marking, namely marking the client of the service system as a first client and sending marking information to the first client when the backlog value of a client consumption message queue of the service system is greater than a preset threshold value and the ratio of the backlog value of the consumption message queue to the message quantity consumed per second is greater than a preset ratio;
in this embodiment, when the backlog value of the message queue consumed by the client of the service system is larger than the predetermined threshold, and the ratio of the backlog value of the message queue consumed by the client of the service system to the message amount consumed per second is larger than the predetermined ratio, it indicates that the speed of consuming the message queue by the client of the service system is slower than the backlog speed of the message queue, that is, the speed of consuming the message queue is slower. Preferably, the predetermined threshold is 10 ten thousand, and the predetermined ratio is 10800.
Marking the client of the service system which consumes the message queue with lower speed, recording mark information, wherein the mark information comprises client identification, time of disconnection with the message queue, backlog value of the message queue, ratio of message quantity consumed per second and the like, and sending the mark information to the client of the service system.
And a reallocation step, wherein after receiving a balance request of the first client for reallocating the message queue, the first client is controlled to be disconnected from the message queue within a preset time, and the message queue is allocated to the clients of other business systems consuming the message queue except the first client.
After the first client receives the marking information, the rebalance process is started to send a balance request for reallocating the message queue to the message system, and after the message system receives the balance request, the connection between the first client and the message queue is controlled to be disconnected within a preset time (for example, the preset time is 10 minutes), and the message queue is only allocated to the clients of other business systems except the first client, which consume the message queue, so that the message queue is reallocated to the fast-consuming client.
In a specific example, as shown in fig. 2, the message queue is initially allocated to client 1 of the business system, client 2 of the business system, client 3 of the business system, and client 4 of the business system for consumption. And when the scanning process finds that the backlog value of the message consumption queue of the client 1 of the service system is greater than a preset threshold value and the ratio of the backlog value of the message consumption queue to the message quantity consumed per second is greater than a preset ratio value, marking a slow consumption mark on the client 1 of the service system and sending the mark information to the client 1 of the service system. When a balance request sent by the client 1 of the service system is received, a redistribution operation is performed, as shown in fig. 3, at this time, the client 1 of the service system cannot acquire a consumption queue for consumption, and the consumption queue is distributed to the client 2 of the service system, the client 3 of the service system, and the client 4 of the service system for consumption.
Compared with the prior art, when the client side of each business system consumes the message queue, the method has the advantages that the backlog value of the message queue consumed by the client side of each business system and the message amount consumed per second are scanned, if the client side with low consumption is found, the client side with low consumption is marked with a mark with low consumption, then the redistribution operation is started, the client side is controlled to be incapable of consuming the message queue within the preset time, the message queue is distributed to other client sides except the client side consuming the message queue, through the mode, the client side with low consumption is found in time, the client side with low consumption is controlled to be incapable of consuming the message queue temporarily, the message queue is consumed as soon as possible, accumulation is prevented, the problem of a disk is avoided, and the system performance is improved.
In a preferred embodiment, on the basis of the above embodiment, when the processing system is executed by the processor, the following steps are further implemented:
in the clients of other service systems, when the backlog value of a client consumption message queue of the service system is larger than a preset threshold value and the ratio of the backlog value of the consumption message queue to the message quantity consumed per second is larger than a preset ratio value, marking the client of the service system as a second client, and sending the marking information to the second client;
after receiving a balance request of the second client for reallocating the message queue, analyzing whether the time for disconnecting the first client from the message queue is more than or equal to the preset time;
if so, controlling the second client to disconnect from the message queue within preset time, and distributing the message queue to clients of other service systems consuming the message queue except the second client;
if not, the second client is controlled to be disconnected from the message queue within preset time, and the message queue is distributed to the clients of other business systems consuming the message queue except the first client and the second client.
After the first client is disconnected from the message queue, if the scanning process finds that the message queue consumed by the client of the service system is slow, the client of the service system is marked as a second client, and the message queue is redistributed again.
Before distribution, whether the time of disconnection between the first client and the message queue is greater than or equal to a preset time or not is analyzed, namely whether the slow consumption mark is invalid or not is analyzed, if the slow consumption mark is invalid, the first client can consume the message queue, and if the slow consumption mark is not invalid, neither the first client nor the second client can consume the message queue. By the method, the client side which consumes slowly is flexibly controlled, so that the first client side can consume the message queue again, the message queue is consumed as soon as possible, accumulation is prevented, the problem of disk IO is further avoided, and the system performance is improved.
In a specific example, as shown in fig. 4, when the scanning process finds that the client 2 of the business system consumes the message queue slowly, the client 2 of the business system is marked, if it is analyzed that the time for disconnecting the client 1 of the business system from the message queue is greater than or equal to the predetermined time, the client 1 of the business system is connected to the message queue, and the client 2 of the business system is marked and disconnected; if it is analyzed that the time for disconnecting the client 1 of the business system from the message queue is less than the preset time, as shown in fig. 5, both the client 1 of the business system and the client 2 of the business system are disconnected, and the message queue is consumed only by the client 3 of the business system and the client 4 of the business system.
In a preferred embodiment, on the basis of the above embodiment, when the processing system is executed by the processor, the following steps are further implemented:
counting the number of the clients of the currently marked service system, and calculating the proportion of the clients of the currently marked service system in all the clients of the service system consuming the message queue;
analyzing whether the proportion is larger than a preset proportion or not;
if so, sending the reminding message to the message management system so that the message management system sends the reminding message to a preset terminal.
In the process of scanning the client of the service system in the scanning process, the proportion of the client of the service system which is currently marked to the clients of all the service systems consuming the message queue is calculated, if the proportion is greater than a preset proportion, preferably the preset proportion is 2/3, a reminding message is sent to the message management system, and the message management system informs a client system responsible person according to the reminding message, for example, the reminding message is sent to a terminal of the client system responsible person in a mail manner, so that whether the performance of the client consuming message queue is normal or not is checked, the disk IO problem is further avoided, and the system performance is improved.
In a preferred embodiment, on the basis of the above embodiment, when the processing system is executed by the processor, the following steps are further implemented:
when receiving the message aiming at the restarting instance of the message queue, deleting the marks and the mark information of the clients of all the marked business systems, and redistributing the message queue to the clients of all the business systems consuming the message queue.
If the message system restarts the instance, the slow-consuming marks of the clients of all the marked service systems are deleted, the clients of all the service systems can consume the message queue again, and then the scanning process scans and finds the clients of the slow-consuming service systems, so that the clients of the service systems have an opportunity to consume the message queue again without waiting for receiving the balancing request, and the flexibility is improved.
As shown in fig. 6, fig. 6 is a flowchart illustrating an embodiment of a method for distributing messages according to the present invention, where the method for distributing messages includes the following steps:
step S1, after the business system of each message consumer accesses the message system, starting the scanning process to scan the backlog value of the client end consumption message queue of each business system and the message amount consumed per second;
the business system may be a system used by each department in an enterprise, such as a financial system. The message system is installed in the server to realize the distribution of the message queue. After the client of each service system accesses the message system, the message queue can be distributed to one or more clients of the service system for consumption.
In the consumption process, a scanning process is started to perform real-time or timing scanning on the client of each accessed service system, wherein the scanning process comprises the steps of scanning the backlog value of the client consumption message queue of each service system and the message amount tps consumed per second so as to analyze whether the client consumption message queue of each service system is slow or not.
Step S2, when the backlog value of the client consuming message queue of the service system is larger than the preset threshold value and the ratio of the backlog value of the consuming message queue to the message amount consumed per second is larger than the preset ratio value, the client of the service system is used as a first client to mark, and the mark information is sent to the first client;
in this embodiment, when the backlog value of the message queue consumed by the client of the service system is larger than the predetermined threshold, and the ratio of the backlog value of the message queue consumed by the client of the service system to the message amount consumed per second is larger than the predetermined ratio, it indicates that the speed of consuming the message queue by the client of the service system is slower than the backlog speed of the message queue, that is, the speed of consuming the message queue is slower. Preferably, the predetermined threshold is 10 ten thousand, and the predetermined ratio is 10800.
Marking the client of the service system which consumes the message queue with lower speed, recording mark information, wherein the mark information comprises client identification, time of disconnection with the message queue, backlog value of the message queue, ratio of message quantity consumed per second and the like, and sending the mark information to the client of the service system.
Step S3, after receiving the balancing request of the first client for reallocating the message queue, controlling the first client to disconnect from the message queue within a predetermined time, and allocating the message queue to the clients of other business systems consuming the message queue except the first client.
After the first client receives the marking information, the rebalance process is started to send a balance request for reallocating the message queue to the message system, and after the message system receives the balance request, the connection between the first client and the message queue is controlled to be disconnected within a preset time (for example, the preset time is 10 minutes), and the message queue is only allocated to the clients of other business systems except the first client, which consume the message queue, so that the message queue is reallocated to the fast-consuming client.
Compared with the prior art, when the client side of each business system consumes the message queue, the method has the advantages that the backlog value of the message queue consumed by the client side of each business system and the message amount consumed per second are scanned, if the client side with low consumption is found, the client side with low consumption is marked with a mark with low consumption, then the redistribution operation is started, the client side is controlled to be incapable of consuming the message queue within the preset time, the message queue is distributed to other client sides except the client side consuming the message queue, through the mode, the client side with low consumption is found in time, the client side with low consumption is controlled to be incapable of consuming the message queue temporarily, the message queue is consumed as soon as possible, accumulation is prevented, the problem of a disk is avoided, and the system performance is improved.
In a preferred embodiment, on the basis of the embodiment shown in fig. 6, after the step S3, the method further includes the following steps:
in the clients of other service systems, when the backlog value of a client consumption message queue of the service system is larger than a preset threshold value and the ratio of the backlog value of the consumption message queue to the message quantity consumed per second is larger than a preset ratio value, marking the client of the service system as a second client, and sending the marking information to the second client;
after receiving a balance request of the second client for reallocating the message queue, analyzing whether the time for disconnecting the first client from the message queue is more than or equal to the preset time;
if so, controlling the second client to disconnect from the message queue within preset time, and distributing the message queue to clients of other service systems consuming the message queue except the second client;
if not, the second client is controlled to be disconnected from the message queue within preset time, and the message queue is distributed to the clients of other business systems consuming the message queue except the first client and the second client.
After the first client is disconnected from the message queue, if the scanning process finds that the message queue consumed by the client of the service system is slow, the client of the service system is marked as a second client, and the message queue is redistributed again.
Before distribution, whether the time of disconnection between the first client and the message queue is greater than or equal to a preset time or not is analyzed, namely whether the slow consumption mark is invalid or not is analyzed, if the slow consumption mark is invalid, the first client can consume the message queue, and if the slow consumption mark is not invalid, neither the first client nor the second client can consume the message queue. By the method, the client side which consumes slowly is flexibly controlled, so that the first client side can consume the message queue again, the message queue is consumed as soon as possible, accumulation is prevented, the problem of disk IO is further avoided, and the system performance is improved.
In a preferred embodiment, on the basis of the above embodiment, the method for message distribution further includes:
counting the number of the clients of the currently marked service system, and calculating the proportion of the clients of the currently marked service system in all the clients of the service system consuming the message queue;
analyzing whether the proportion is larger than a preset proportion or not;
if so, sending the reminding message to the message management system so that the message management system sends the reminding message to a preset terminal.
In the process of scanning the client of the service system in the scanning process, the proportion of the client of the service system which is currently marked to the clients of all the service systems consuming the message queue is calculated, if the proportion is greater than a preset proportion, preferably the preset proportion is 2/3, a reminding message is sent to the message management system, and the message management system informs a client system responsible person according to the reminding message, for example, the reminding message is sent to a terminal of the client system responsible person in a mail manner, so that whether the performance of the client consuming message queue is normal or not is checked, the disk IO problem is further avoided, and the system performance is improved.
In a preferred embodiment, on the basis of the above embodiment, the method for message distribution further includes:
when receiving the message aiming at the restarting instance of the message queue, deleting the marks and the mark information of the clients of all the marked business systems, and redistributing the message queue to the clients of all the business systems consuming the message queue.
If the message system restarts the instance, the slow-consuming marks of the clients of all the marked service systems are deleted, the clients of all the service systems can consume the message queue again, and then the scanning process scans and finds the clients of the slow-consuming service systems, so that the clients of the service systems have an opportunity to consume the message queue again without waiting for receiving the balancing request, and the flexibility is improved.
The present invention also provides a computer readable storage medium having stored thereon a processing system, which when executed by a processor implements the steps of the above-described method of message distribution.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A server, comprising a memory and a processor coupled to the memory, the memory having stored therein a processing system operable on the processor, the processing system when executed by the processor performing the steps of:
scanning, namely starting a scanning process to scan the backlog value of a client consumption message queue of each service system and the message quantity consumed per second after the service system of each message consumer accesses the message system;
marking, namely marking a client of a business system as a first client when a backlog value of a client consumption message queue of the business system is larger than a preset threshold value and a ratio of the backlog value of the consumption message queue to a message quantity consumed per second is larger than a preset ratio value, and sending marking information to the first client, wherein the marking information comprises a client identifier, time for disconnecting the client from the consumption message queue, the backlog value of the consumption message queue and the ratio of the message quantity consumed per second;
and a reallocation step, wherein after receiving a balance request of the first client for reallocating the message queue, the first client is controlled to be disconnected from the message queue within a preset time, and the message queue is allocated to the clients of other business systems consuming the message queue except the first client.
2. The server of claim 1, wherein the processing system, when executed by the processor, further performs the steps of:
in the clients of other service systems, when the backlog value of a client consumption message queue of the service system is larger than a preset threshold value and the ratio of the backlog value of the consumption message queue to the message quantity consumed per second is larger than a preset ratio value, marking the client of the service system as a second client, and sending the marking information to the second client;
after receiving a balance request of the second client for reallocating the message queue, analyzing whether the time for disconnecting the first client from the message queue is more than or equal to the preset time;
if so, controlling the second client to disconnect from the message queue within preset time, and distributing the message queue to clients of other service systems consuming the message queue except the second client;
if not, the second client is controlled to be disconnected from the message queue within preset time, and the message queue is distributed to the clients of other business systems consuming the message queue except the first client and the second client.
3. The server according to claim 1 or 2, wherein the processing system, when executed by the processor, further performs the steps of:
counting the number of the clients of the currently marked service system, and calculating the proportion of the clients of the currently marked service system in all the clients of the service system consuming the message queue;
analyzing whether the proportion is larger than a preset proportion or not;
if so, sending the reminding message to the message management system so that the message management system sends the reminding message to a preset terminal.
4. The server according to claim 1 or 2, wherein the processing system, when executed by the processor, further performs the steps of:
when receiving the message aiming at the restarting instance of the message queue, deleting the marks and the mark information of the clients of all the marked business systems, and redistributing the message queue to the clients of all the business systems consuming the message queue.
5. A method of message distribution, the method comprising:
s1, after the business system of each message consumer accesses the message system, starting a scanning process to scan the backlog value of the client consumption message queue of each business system and the message amount consumed per second;
s2, when the backlog value of the client consuming message queue of the service system is larger than a preset threshold value and the ratio of the backlog value of the consuming message queue to the message quantity consumed per second is larger than a preset ratio value, marking the client of the service system as a first client and sending marking information to the first client, wherein the marking information comprises a client identifier, the time of disconnection from the consuming message queue, the backlog value of the consuming message queue and the ratio of the message quantity consumed per second;
s3, after receiving the balancing request of the first client for reallocating the message queue, controlling the first client to disconnect from the message queue within a predetermined time, and allocating the message queue to the clients of other business systems consuming the message queue except the first client.
6. The method for distributing messages according to claim 5, wherein after the step S3, the method further comprises:
in the clients of other service systems, when the backlog value of a client consumption message queue of the service system is larger than a preset threshold value and the ratio of the backlog value of the consumption message queue to the message quantity consumed per second is larger than a preset ratio value, marking the client of the service system as a second client, and sending the marking information to the second client;
after receiving a balance request of the second client for reallocating the message queue, analyzing whether the time for disconnecting the first client from the message queue is more than or equal to the preset time;
if so, controlling the second client to disconnect from the message queue within preset time, and distributing the message queue to clients of other service systems consuming the message queue except the second client;
if not, the second client is controlled to be disconnected from the message queue within preset time, and the message queue is distributed to the clients of other business systems consuming the message queue except the first client and the second client.
7. The method of message distribution according to claim 5 or 6, characterized in that the method of message distribution further comprises:
counting the number of the clients of the currently marked service system, and calculating the proportion of the clients of the currently marked service system in all the clients of the service system consuming the message queue;
analyzing whether the proportion is larger than a preset proportion or not;
if so, sending the reminding message to the message management system so that the message management system sends the reminding message to a preset terminal.
8. The method of message distribution according to claim 5 or 6, characterized in that the method of message distribution further comprises:
when receiving the message aiming at the restarting instance of the message queue, deleting the marks and the mark information of the clients of all the marked business systems, and redistributing the message queue to the clients of all the business systems consuming the message queue.
9. The method according to claim 5 or 6, wherein the predetermined threshold is 10 ten thousand, and the predetermined ratio is 10800.
10. A computer-readable storage medium, having stored thereon a processing system which, when executed by a processor, carries out the steps of the method of message distribution according to any of claims 5 to 9.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810370069.1A CN108848039B (en) | 2018-04-24 | 2018-04-24 | Server, message distribution method and storage medium |
| PCT/CN2018/102090 WO2019205371A1 (en) | 2018-04-24 | 2018-08-24 | Server, message allocation method, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810370069.1A CN108848039B (en) | 2018-04-24 | 2018-04-24 | Server, message distribution method and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108848039A CN108848039A (en) | 2018-11-20 |
| CN108848039B true CN108848039B (en) | 2021-11-02 |
Family
ID=64212225
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810370069.1A Active CN108848039B (en) | 2018-04-24 | 2018-04-24 | Server, message distribution method and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN108848039B (en) |
| WO (1) | WO2019205371A1 (en) |
Families Citing this family (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110377434A (en) * | 2019-07-04 | 2019-10-25 | 深圳前海微众银行股份有限公司 | A kind of service blowout method, device and message-oriented middleware |
| CN110515746A (en) * | 2019-08-22 | 2019-11-29 | 北京宝兰德软件股份有限公司 | A kind of method and device handling slow consumer |
| CN110502402B (en) * | 2019-08-28 | 2023-03-21 | 中国联合网络通信集团有限公司 | Message processing method and device and terminal device |
| CN112764892B (en) * | 2019-10-21 | 2025-01-17 | 伊姆西Ip控股有限责任公司 | Method, device and computer program product for managing processes |
| CN111045837B (en) * | 2019-11-26 | 2023-10-20 | 福建天泉教育科技有限公司 | Cross-service consumption method and storage medium |
| CN114787760B (en) * | 2019-12-20 | 2025-02-25 | 华为技术有限公司 | A method for improving SSD utilization, storage device and SSD controller |
| CN111008087B (en) * | 2019-12-25 | 2024-03-15 | 上海众源网络有限公司 | Message processing method and device |
| CN111475313B (en) * | 2020-03-04 | 2023-06-27 | 江苏理工学院 | Method and device for constructing message queue suitable for forward propagation of convolutional neural network |
| CN113806102B (en) * | 2020-06-15 | 2023-11-21 | 中国移动通信集团浙江有限公司 | Message queue processing method, device and computing device |
| CN111858099B (en) * | 2020-07-24 | 2024-05-28 | 海尔优家智能科技（北京）有限公司 | Message subscription method and device |
| CN111930533A (en) * | 2020-07-28 | 2020-11-13 | 银盛支付服务股份有限公司 | Method and system for orderly processing production and consumption same data based on kafka |
| CN112000476A (en) * | 2020-08-21 | 2020-11-27 | 南京星邺汇捷网络科技有限公司 | Queue-based task execution flow control method and system |
| CN112181737B (en) * | 2020-09-25 | 2024-08-13 | 中国建设银行股份有限公司 | Message processing method, device, electronic equipment and medium |
| CN112416614B (en) * | 2020-10-28 | 2025-05-06 | 网宿科技股份有限公司 | Data processing method, system and server based on message queue |
| CN112612607B (en) * | 2020-12-16 | 2024-07-09 | 平安消费金融有限公司 | Data processing method, device, computer equipment and medium based on message queue |
| CN112612631B (en) * | 2020-12-29 | 2024-04-02 | 杭州涂鸦信息技术有限公司 | Message queue accumulation detection method and related device |
| CN113419875B (en) * | 2021-06-29 | 2025-06-03 | 北京小米移动软件有限公司 | Data scanning method and device, electronic device, and storage medium |
| CN114237936B (en) * | 2021-12-16 | 2025-01-24 | 成都知道创宇信息技术有限公司 | Data consumption method, device, server and readable storage medium |
| CN114584625B (en) * | 2022-01-24 | 2024-03-26 | 北京达佳互联信息技术有限公司 | Message processing method and device, electronic equipment and storage medium |
| CN114827033B (en) * | 2022-04-15 | 2024-04-19 | 咪咕文化科技有限公司 | Data flow control method, device, equipment and computer readable storage medium |
| CN115174682B (en) * | 2022-06-17 | 2023-07-28 | 中国银联股份有限公司 | Data distribution method, system, center, device and computer readable storage medium |
| CN115412208A (en) * | 2022-08-09 | 2022-11-29 | 唯阜德(武汉)科技有限公司 | Message transmission method and system for message queue management system |
| CN115941548A (en) * | 2022-09-23 | 2023-04-07 | 深圳依时货拉拉科技有限公司 | Method and device for determining consumption delay time of consumer |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102457906A (en) * | 2010-10-26 | 2012-05-16 | 中国移动通信集团河南有限公司 | Load balancing control method and system of message queues |
| CN106385375A (en) * | 2015-07-31 | 2017-02-08 | 北京国双科技有限公司 | Load balancing method used for message queues and apparatus thereof |
| KR20180058880A (en) * | 2016-11-24 | 2018-06-04 | (주)케이아이엔엑스 | Load balancing and performance optimizing methods of message queues in the distributed openstack cloud data centers |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9807015B2 (en) * | 2014-03-19 | 2017-10-31 | Dell Products L.P. | Message processing using dynamic load balancing queues in a messaging system |
| CN105786909A (en) * | 2014-12-25 | 2016-07-20 | 北京东方通科技股份有限公司 | Message queue backlog load self-adaptive application triggering method and system |
| CN106656760B (en) * | 2016-12-28 | 2018-02-06 | 广州赛意信息科技股份有限公司 | A kind of message delivery method and system based on message queue |
-
2018
- 2018-04-24 CN CN201810370069.1A patent/CN108848039B/en active Active
- 2018-08-24 WO PCT/CN2018/102090 patent/WO2019205371A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102457906A (en) * | 2010-10-26 | 2012-05-16 | 中国移动通信集团河南有限公司 | Load balancing control method and system of message queues |
| CN106385375A (en) * | 2015-07-31 | 2017-02-08 | 北京国双科技有限公司 | Load balancing method used for message queues and apparatus thereof |
| KR20180058880A (en) * | 2016-11-24 | 2018-06-04 | (주)케이아이엔엑스 | Load balancing and performance optimizing methods of message queues in the distributed openstack cloud data centers |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108848039A (en) | 2018-11-20 |
| WO2019205371A1 (en) | 2019-10-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108848039B (en) | Server, message distribution method and storage medium | |
| CN108376118B (en) | Service distribution system, method, device and storage medium | |
| CN108776934B (en) | Distributed data calculation method and device, computer equipment and readable storage medium | |
| CN111813513A (en) | Real-time task scheduling method, device, equipment and medium based on distribution | |
| CN108924250B (en) | Service request processing method and device based on block chain and computer equipment | |
| CN109451051B (en) | Service request processing method, device, electronic device and storage medium | |
| CN108462760B (en) | Electronic device, automatic cluster access domain name generation method and storage medium | |
| CN109040227B (en) | Service request response method and device based on block chain and computer equipment | |
| CN107729137B (en) | Server, block chain signature verification decryption method and storage medium | |
| CN109766172B (en) | Asynchronous task scheduling method and device | |
| CN107451853B (en) | A method, device, system and storage medium for real-time distribution of red envelopes | |
| CN111813573B (en) | Communication method of management platform and robot software and related equipment thereof | |
| CN106657327A (en) | Message pushing method and message pushing device | |
| CN114124968B (en) | Load balancing method, device, equipment and medium based on market data | |
| CN114155026B (en) | Resource allocation method, device, server and storage medium | |
| CN114036195B (en) | A data request processing method, device, server and storage medium | |
| CN112600761A (en) | Resource allocation method, device and storage medium | |
| CN113791735A (en) | Video data storage method and device, computer equipment and storage medium | |
| CN115671716B (en) | Processing method, device, storage medium and electronic equipment for preloading instance application | |
| CN112035460A (en) | Identification distribution method, device, equipment and storage medium | |
| CN108520401A (en) | User list management method, device, platform and storage medium | |
| CN114070847A (en) | Current limiting method, device, equipment and storage medium of server | |
| CN116978031A (en) | Image data processing method and device and related products | |
| CN116382940A (en) | Shared memory communication method, device, equipment and storage medium | |
| CN108848136B (en) | Shared storage method of cloud service cluster |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |