CN119728626B

CN119728626B - A message notification method, device, computer equipment and storage medium

Info

Publication number: CN119728626B
Application number: CN202510230756.3A
Authority: CN
Inventors: 郭立民; 孔维亭; 郭涛
Original assignee: Inspur Jinan data Technology Co ltd
Current assignee: Inspur Jinan data Technology Co ltd
Priority date: 2025-02-28
Filing date: 2025-02-28
Publication date: 2025-07-08
Anticipated expiration: 2045-02-28
Also published as: CN119728626A

Abstract

The present application relates to a message notification method, device, computer equipment and storage medium, the method comprising: in response to the platform microservice receiving a database change task, creating a corresponding platform microservice message queue in the message queue; obtaining a database log, parsing to obtain database event information; pushing the database event information to the platform microservice message queue; judging whether the database change task is completed according to the database event information in the platform microservice message queue; in response to the completion of the database change task, feeding back the completion information of the database change task to the database. The use of this method can reduce the system load, improve the efficiency and stability of the system in high concurrency scenarios, ensure the real-time and reliability of messages, and realize cross-layer communication between microservices.

Description

Message notification method, device, computer equipment and storage medium

Technical Field

The present application relates to the field of message notification technologies, and in particular, to a message notification method, a device, a computer device, and a storage medium.

Background

With the rapid development of cloud computing technology, a micro-service architecture is widely applied to a cloud platform, and the micro-service architecture can independently deploy, expand and maintain each service by splitting system functions into a plurality of independent services, so that the usability and flexibility of the cloud platform system are improved. However, because of the independence of the microservices architecture, communication and data consistency problems between microservices are also increasingly apparent.

In the prior art, for communication between micro services, a conventional message notification mechanism is generally a polling mode, that is, the micro service continuously performs a status query until required resources or services are configured. However, the message notification method can cause serious degradation of system performance under a high concurrency scene, frequent state inquiry not only increases system load, but also can cause message delay or loss to influence the real-time performance and reliability of the system, and in addition, because the message transmission requirements of a cross-platform layer and a virtualization layer exist between micro services in a cloud platform system, the traditional message notification method also fails to provide a good solution for cross-layer communication between the micro services.

Disclosure of Invention

Based on the above, it is necessary to provide a message notification method, device, computer equipment and storage medium, which can reduce the system load, improve the efficiency and stability of the system in a high concurrency scenario, ensure the real-time performance and reliability of the message, and realize cross-layer communication between micro services.

On the one hand, a message notification method is provided, the method is applied to a cloud platform system, a database, a platform layer, a virtualization layer and a message general team are deployed in the cloud platform system, a plurality of platform micro services are deployed in the platform layer, and a plurality of management micro services are deployed in the virtualization layer, the method comprises the following steps:

Responding to the platform micro-service receiving the database change task, and creating a corresponding platform micro-service message queue in the message general queue;

Acquiring a database log, and analyzing to obtain database event information;

Pushing the database event information to a platform micro-service message queue;

judging whether the database change task is executed or not according to the database event information in the platform micro-service message queue;

And responding to the completion of the execution of the database change task, and feeding back the completion information of the database change task to the database.

In one embodiment, the database is a master-slave architecture, and the database comprises a master node and a slave node, and the method comprises the steps of obtaining a database log, analyzing to obtain database event information, and comprising:

setting disguised slave nodes in a database based on a master-slave protocol;

based on disguised slave nodes, acquiring database logs pushed by a master node through read-write threads;

Obtaining data change information according to the database log;

Analyzing according to the data change information to obtain database change task information;

And according to the database change task information, combining the database event type, and carrying out corresponding data structure adjustment to obtain the database event information.

In one embodiment, according to the database change task information, and in combination with the database event type, performing corresponding data structure adjustment to obtain database event information, including:

analyzing and generating serialized data according to the database change task information and the database event type;

responding to the database event type as an adding and deleting event, and analyzing the database change task information into a serialization array;

Responding to the database event type as an update event, and analyzing the database change task information into a serialized hash table;

And (3) resolving again based on the serialized data, converting the serialized data into a key value data structure, and obtaining the event information of the database.

In one embodiment, the masquerading slave node has a plurality of copies in the database, wherein the number of the copies is greater than or equal to three, and the pushing the database event information to the platform micro-service message queue includes:

Responding to the starting of a plurality of copies, and correspondingly generating a plurality of copy serial numbers;

Writing a plurality of copy serial numbers into the same ordered set to obtain a current copy sequence;

Selecting a master copy from a plurality of copies according to the current copy sequence, wherein the rest copies are slave copies;

based on the copy, acquiring a database log, and generating database event information according to the database log based on the master copy if the copy is the master copy;

in response to the replica being a slave replica, the database log received from the replica is discarded.

In one embodiment, according to the current copy sequence, a master copy is selected from a plurality of copies, and the remaining copies are slave copies, including:

Respectively carrying out heartbeat detection on a plurality of copies based on a preset heartbeat interval;

responding to the result of heartbeat detection as healthy and the current copy sequence has the copy sequence number corresponding to the copy, and updating the copy sequence number according to the current copy sequence;

responding to the result of heartbeat detection as a fault and the copy serial number corresponding to the copy exists in the current copy sequence, judging whether the copy can recover health in the expiration time, and responding to the judgment result as no, deleting the copy serial number in the current copy sequence;

Responding to the result of the heartbeat detection as healthy and that the copy serial number corresponding to the copy does not exist in the current copy sequence, and adding the copy serial number to the last of the current copy sequence;

And responding to the copy sequence corresponding to the copy as the first bit in the current copy sequence, wherein the copy is the master copy, and otherwise, the copy is the slave copy.

In one embodiment, a message re-pushing queue is further deployed in the cloud platform system, and pushes the database event information to a platform micro-service message queue, including:

obtaining a master copy of a disguised slave node;

Creating a main push single thread according to the main copy;

in response to generating database event information based on the master copy, pushing the database event information to a message team through a master push single thread;

in response to the failure of the master copy to successfully push the database event information to the message general queue, storing the database event information into the message re-push queue;

Creating a message re-pushing thread according to the message re-pushing queue;

Pushing the database event information in the message re-pushing queue to a message general team through a message re-pushing thread;

and in response to the message general team successfully receiving the database event information, pushing the database event information to a platform micro-service message queue.

In one embodiment, a message broadcast queue is further disposed in the message general team, and in response to the message general team successfully receiving the database event information, pushing the database event information to the platform micro service message queue, including:

In response to receiving the database event information by the message general team, storing the database event information into the message broadcast queue;

in response to the message broadcast queue receiving the database event information, distributing the database event information to all platform micro service message queues through the message broadcast queue, wherein the database event information from the message re-push queue is marked as a failed re-push message.

In one embodiment, determining whether the database change task is performed according to the database event information in the platform micro service message queue includes:

Based on the platform micro-service, performing front-end inquiry, and judging whether the platform micro-service has the front-end micro-service or not;

Responding to the platform micro-service without the front micro-service, executing a database change task by the platform micro-service, and feeding back the completion information of the database change task to the database;

and responding to the front-end micro service of the platform micro service, acquiring the task completion information of the front-end micro service of the platform micro service, and judging whether the database change task is executed and completed according to the task completion information of the front-end micro service.

In one embodiment, the front-end micro-service at least comprises one or more of a platform micro-service and a management micro-service, wherein the task completion information of the front-end micro-service is obtained by the following modes:

Responding to the existence of the front-end micro service of the platform micro service, acquiring the current front-end micro service of the platform micro service, setting the platform micro service as a delay start, and generating a delay start task corresponding to a database change task;

In response to the fact that the current front-end micro-service is not the management micro-service, the current front-end micro-service is used as a new platform micro-service, and front-end inquiry is recursively conducted until the obtained current front-end micro-service is the management micro-service;

Responding to the current prepositive micro-service as the management micro-service, calling the management micro-service to execute a resource configuration task corresponding to a database change task to obtain task completion information of the management micro-service;

And responding to the task completion information of the current front-end micro-service received by the platform micro-service, closing the delay starting task and executing the platform micro-service to obtain the task completion information of the platform micro-service.

In one embodiment, acquiring task completion information of a front-end service of a platform micro service, and determining whether a database change task is executed and completed according to the task completion information of the front-end micro service includes:

responding to the delayed start task generation, creating corresponding task record information and storing the task record information in a task record table, wherein the task record information at least comprises one of a task serial number, a required resource type and a request resource serial number;

distributing database event information to a platform micro-service message queue through a message general team, wherein the database event information at least comprises one of a service completion sequence number, a configuration resource type, a configuration resource sequence number, a configuration resource current state and a failure re-pushing mark;

Responding to the consistency of the required resource type and the configuration resource type, wherein the database event information is the association information of the platform micro-service;

and in response to the database event information being the related information of the platform micro service, comparing the database event information with a task record table, judging whether the database event information is the task completion information of the front micro service, otherwise, discarding the database event information in a platform micro service message queue.

In one embodiment, in response to the database event information being the related information of the platform micro service, the method compares the related information with the task record table to determine whether the database event information is the task completion information of the front micro service, and further includes:

Setting task completion information of the front-end micro-service, wherein the task completion information comprises failure completion information and effective completion information;

inquiring all task record information in a task record table, and responding to the fact that the task sequence number is consistent with the service completion sequence number, wherein the database event information is related information of the front micro-service;

Acquiring a failure re-pushing mark in response to the related information of the front-end micro-service of the database event information, and acquiring failure completion information in response to the failure re-pushing mark being yes;

Responding to the failure re-pushing mark as no, and acquiring the current state of the configuration resource;

and responding to the current state of the configuration resource to be steady state, wherein the database event information is the effective completion information of the front-end micro-service, otherwise, the database event information is the incomplete information of the front-end micro-service.

In one embodiment, executing the delayed start task corresponding to the platform micro service in response to the database event information being the task completion information of the front-end micro service includes:

Acquiring task completion information of the front-end micro-service;

responding to the task completion information of the front-end micro-service as effective completion information, and directly waking up and starting the task in a delay manner and executing the task;

responding to the task completion information of the front-end micro-service as failure completion information, waking up the task to be started in a delayed mode, and executing the task based on a polling inquiry mode;

And in response to the execution of the delay starting task, deleting the task record information corresponding to the delay starting task from the task record table.

On the other hand, a message notification device is provided, the device is applied to a cloud platform system, a database, a platform layer, a virtualization layer and a message general team are deployed in the cloud platform system, a plurality of platform micro services are deployed in the platform layer, a plurality of management micro services are deployed in the virtualization layer, and the device comprises:

The queue creation module is used for responding to the platform micro-service receiving the database change task and creating a corresponding platform micro-service message queue in the message general queue;

The log monitoring module is used for acquiring a database log and analyzing to obtain database event information;

the message pushing module is used for pushing the database event information to the platform micro-service message queue;

The message judging module is used for judging whether the database change task is executed or not according to the database event information in the platform micro-service message queue;

And the message feedback module is used for responding to the completion of the execution of the database change task and feeding the completion information of the database change task back to the database.

In yet another aspect, a computer device is provided comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program:

Acquiring a database log, and analyzing to obtain database event information;

In yet another aspect, a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:

Acquiring a database log, and analyzing to obtain database event information;

The method, the device, the computer equipment and the storage medium for notifying the information can realize timely capturing of database events such as data change and the like by monitoring and acquiring the database logs in real time and analyzing the database logs, avoid the delay problem of a traditional polling mechanism, are beneficial to improving the consistency and timeliness of data synchronization and guaranteeing the real-time performance and reliability of a system, simultaneously, can reduce frequent state inquiry of each micro service, reduce the system load and improve the stability of the system in a concurrent scene through an active notification mechanism of a message general team, and in addition, each micro service in a platform layer and a virtualization layer is communicated with the same database and the message general team, and information generated by the micro services of different layers is converted into the database events to be pushed to the message general team for distribution, so that cross-layer communication among the micro services can be realized, and a reliable and efficient cross-layer service communication mechanism is provided.

Drawings

FIG. 1 is an application environment diagram of a message notification method in one embodiment;

FIG. 2 is a flow diagram of a message notification method in one embodiment;

FIG. 3 is a detailed flow diagram of a method for implementing message notification by a database listening resolver in one embodiment;

FIG. 4 is a schematic diagram of a workflow of a database listening resolver in one embodiment for obtaining database event information from a node by masquerading;

FIG. 5 is a schematic workflow diagram of a master copy selector in one embodiment;

FIG. 6 is a schematic diagram of a workflow of a message pusher in one embodiment;

FIG. 7 is a block diagram of the structure of a message notification apparatus in one embodiment;

fig. 8 is an internal structural diagram of a computer device in one embodiment.

Detailed Description

The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

The message notification method provided by the application can be applied to an application environment shown in figure 1. Wherein the terminal 102 communicates with the server 104 via a network. The server 104 is provided with a cloud platform system, in which a database, a platform layer, a virtualization layer and a message team are disposed, a plurality of platform micro-services are disposed in the platform layer, and a plurality of management micro-services are disposed in the virtualization layer. The server 104 receives the service request sent by the terminal 102, the server 104 starts a plurality of platform micro services and a plurality of management micro services corresponding to a platform layer in the cloud platform system, the plurality of platform micro services and the plurality of management micro services communicate based on a message notification method, and finally the service request of the terminal 102 is completed. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smartphones, tablet computers, and portable wearable devices, and the server 104 may be implemented by a stand-alone server or a server cluster composed of a plurality of servers.

In one embodiment, as shown in fig. 2, a message notification method is provided, which is illustrated by using the method applied to the server 104 in fig. 1 as an example, and includes the following steps:

in response to the platform micro-service receiving the database change task, a corresponding platform micro-service message queue is created in the message general queue, step 201.

The micro-service is a micro-service unit for providing specific service functions in a cloud platform system, a plurality of micro-services are mutually independent and can be deployed and expanded independently, the platform micro-service is a micro-service deployed on a platform layer and generally comprises a computing service, a storage service and a network service, a message general team is a message middleware for transmitting messages in the micro-service system, and an information exchange unit is arranged in the message general team.

Specifically, in response to the starting of a plurality of platform micro-services, a plurality of platform micro-service message queues are created in the message general team, wherein the plurality of platform micro-service message queues are in one-to-one correspondence with the plurality of platform micro-services, the plurality of platform micro-service message queues are independent from each other, and the plurality of platform micro-service message queues are connected with an information exchange unit in the message general team.

Step 202, obtaining a database log, and analyzing to obtain database event information.

The database Log is a Log file for recording database operations and is used for tracking data changes in the database, such as inserting, updating or deleting data, the database event refers to event data obtained by analyzing the database Log, and generally comprises the operation type of the database and the specific data content of the operation, the database is preferably MySQL, the database Log is preferably Binlog, namely Binry Log, and is used for recording all changes of the database and storing the changes in a Binary form.

The database is in a master-slave architecture and comprises a master node and a slave node, wherein a disguised slave node is arranged in the database based on a master-slave protocol, the disguised slave node comprises a plurality of copies, one master copy is selected from the plurality of copies, a database log pushed by the master node is obtained through the master copy, analysis is carried out according to the database log to obtain database change task information, and corresponding data structure adjustment is carried out by combining with a database event type to obtain database event information.

Step 203, pushing the database event information to the platform micro service message queue.

The cloud platform system is further provided with a message re-pushing queue, a master copy of the slave node is disguised, database event information is pushed to a message general queue based on a master push single thread, the database event information is stored in the message re-pushing queue in response to pushing failure, re-pushing is performed based on the re-pushing message thread and marked as failure re-pushing information, and the database event information is broadcasted in the message general queue in response to receiving the database event information by the message general queue and distributed to all platform micro-service message queues.

Step 204, judging whether the database change task is executed to be completed or not according to the database event information in the platform micro service message queue.

The method comprises the steps of judging whether a front micro-service exists in a platform micro-service or not, responding to the fact that the front micro-service does not exist in the platform micro-service, executing a database changing task by the platform micro-service, feeding back completion information of the database changing task to a database, responding to the fact that the front micro-service exists in the platform micro-service, acquiring task completion information of the front micro-service of the platform micro-service, and judging whether the database changing task is executed or not according to the task completion information of the front micro-service, wherein the front micro-service comprises the platform micro-service and the management micro-service.

And step 205, responding to completion of the execution of the database change task, and feeding back the completion information of the database change task to the database.

Specifically, the platform micro-service executes and completes the database change task, records the database event corresponding to the database change task through the database log, and generates corresponding database change task information.

In the message notification method, database events such as data changes can be captured in time by monitoring and analyzing the database logs in real time, the delay problem of a traditional polling mechanism is avoided, the consistency and timeliness of data synchronization are improved, the real-time performance and reliability of a system are guaranteed, meanwhile, the frequent state inquiry of each micro service can be reduced through the active notification mechanism of a message general team, the system load is reduced, the stability of the system in a concurrent scene is improved, in addition, each micro service in a platform layer and a virtualization layer is communicated with the same database and the same message general team, information generated by the micro services in different layers is converted into database events to be pushed to the message general team for distribution, and therefore cross-layer communication among the micro services can be realized, and a reliable and efficient cross-layer service communication mechanism is provided.

In one embodiment, the database is a master-slave architecture, the database comprises a master node and a slave node, the database log is obtained, and the database event information is obtained by analysis, and the method comprises the following steps:

setting disguised slave nodes in a database based on a master-slave protocol;

Obtaining data change information according to the database log;

Specifically, in the embodiment, by setting the disguised slave node of the database, the database log is monitored and analyzed in real time, so that the instant capture and notification of the data change are realized, the delay problem caused by the traditional polling mechanism is avoided, and the instant notification mechanism is established, so that the data synchronization and the state update between the micro services are more timely and accurate.

In one embodiment, according to the database change task information, in combination with the database event type, corresponding data structure adjustment is performed to obtain database event information, including:

It should be noted that, in response to the database event type being an add-drop event, the database event information is parsed into a serialized array, and one possible implementation example is Serializable [ ] row;

in response to the database event type being an update event, parsing the database event information into a serialized hash table, one possible implementation example being map. Entry < Serializable [ ], serializable [ ] > row;

It should be noted that, taking MySQL and Binlog as examples, the typical database table includes field names and field values of each field, but the data recorded in Binlog does not include field names, but the position of each field value in the serialized array corresponds to the position of each field in the table structure one by one, so as to identify the corresponding field name;

Further, when the values of two data types of the database table tinyint (1) and the bit (1) are analyzed, 1 is converted into a boolean value true, 0 is converted into a boolean value false, and the boolean value false is the same as the default data processing specification of the database;

In the analysis process of database events, other events except for non-deserializing TABLE_MAP events, insertion, deletion and update events are supported to be configured, and non-deserializing date and time and character strings are supported to be configured so as to reduce performance loss;

In the analysis process, supporting analysis of partial table related events, discarding table events which are not concerned to greatly improve analysis efficiency, and storing the table and table fields which are concerned by a user in a database, wherein the configuration mode enables the messages pushed into a message queue to be controlled by updating the database under the premise of not stopping and delaying normal service operation in a production environment;

And by setting a thread pool, based on threads in the thread pool, taking out the database change task information from the cache queue and analyzing the database change task information, and pushing the result in sequence after the analysis is completed, the sequence of the pushed database event information after the analysis is ensured to be the same as the sequence of the time information of the original database, and the database event analysis efficiency is higher under the high concurrency scene.

Specifically, in this embodiment, the database change task information is converted into the corresponding serialized data according to the different database change task types, and then converted into the key value pair form, so that the complexity of analysis and conversion can be reduced, and the true reliability of the information can be ensured.

In one embodiment, the masquerading slave node has a plurality of copies in the database, wherein the number of the copies is greater than or equal to three, and pushing the database event information to the platform micro service message queue comprises:

Specifically, in the embodiment, multiple copies are deployed by disguising the slave nodes, and by combining an election mechanism, the new master copy can be selected rapidly to continue to bear tasks when the master copy fails, so that a failover mechanism is realized, and the continuity and reliability of message notification are ensured.

In one embodiment, according to the current copy sequence, one master copy is selected from a plurality of copies, and the remaining copies are slave copies, including:

In the embodiment, the heartbeat monitoring is introduced as a main copy election mechanism to perform health check and recovery judgment on the copy, so that the high availability of the copy is ensured, and the fault tolerance of the system is effectively improved by dynamically adjusting the copy sequence, so that the situation that event pushing is influenced by copy faults is avoided.

It is worth to say that the preset heartbeat interval is set to n, the expiration time is preferably set to 3n, the primary copy is disconnected after three heartbeats fail, problems of a heartbeat mechanism, such as network delay or short network interruption, which may cause misjudgment of the primary copy fault and trigger unnecessary failover, can be avoided, and in addition, the specific heartbeat time can be set according to the service scale of the system and tolerance of delay and is optimized by combining with full test and production practice.

It is worth to say that the ordered set is preferably a Redis ordered set, wherein the Redis ordered set is preferably deployed in a master-slave cluster mode, so that the situation that multiple copies cannot update and query the Redis ordered set when a single point of Redis fails can be avoided, and the main copy selection process cannot be normally executed.

obtaining a master copy of a disguised slave node;

Creating a main push single thread according to the main copy;

Creating a message re-pushing thread according to the message re-pushing queue;

In particular, the embodiment can ensure reliable transmission of the message by combining a message re-push queue and a main push single-thread mechanism, and if the message push fails, the system can automatically re-push, thereby ensuring final consistency of the message and accurate feedback of task completion information.

In particular, the embodiment enables the platform micro-service to receive all related database event information through a mechanism of a message broadcast queue, improves the expansibility and flexibility of a system, and can effectively identify and track the abnormality in the message processing process by carrying out marking processing on the failed re-push message.

Specifically, in this embodiment, in combination with the front-end query mechanism, the platform micro-service can determine whether the database change task is completed according to the task completion information of the front-end micro-service, so as to ensure that the dependency relationship of task execution is correctly processed, and avoid execution conflict between tasks.

In one embodiment, the front-end micro-service at least comprises one or more of a platform micro-service, a management micro-service, and task completion information of the front-end micro-service is obtained by:

Specifically, according to the embodiment, by a recursion query mechanism of the front-end micro service, the platform micro service can execute the database change task after the front-end micro service is completed, task execution which does not accord with the dependency relationship is avoided, and task scheduling precision of the system is improved.

In one embodiment, based on a dynamic calculation mode, setting delay starting time of a delay starting task corresponding to a database change task according to historical execution time of a resource configuration task corresponding to the management micro-service execution database change task;

Responding to the exhaustion of the delay starting time or the awakening of the delay starting task, closing the delay starting task and running the corresponding platform micro-service;

it should be noted that one possible calculation method of the delay start time is:

Acquiring all execution time lengths of a resource allocation task of a data change task executed by a front-end micro service of a platform micro service in a past detection period, wherein the front-end micro service is a management micro service, and recursively carrying out front-end query until a corresponding management micro service is found out when the front-end micro service is not the management micro service;

Selecting a plurality of execution time durations with the longest time from all the execution time durations, wherein the number of the plurality of execution time durations is preferably 3;

calculating an average execution duration based on the selected execution durations;

and multiplying the average execution time by an adjustment coefficient to obtain the delay starting time, wherein the adjustment coefficient is preferably 1.1-1.3.

Specifically, the embodiment ensures that the event information of the database is consistent with the task completion information of the front-end micro-service by comparing the task record table, further improves the accuracy and consistency of task execution, and simultaneously enhances the fault tolerance of the system by a processing mechanism of the failure re-push mark.

In one embodiment, in response to the database event information being the related information of the platform micro service, comparing with the task record table to determine whether the database event information is the task completion information of the front micro service, and further including:

Specifically, the embodiment ensures that the platform micro-service can accurately identify the task state of the front micro-service by setting the effective and invalid completion information and inquiring the task record table, thereby improving the stability and accuracy of task execution and reducing the execution failure caused by the task state judgment error.

In one embodiment, executing the delayed start task corresponding to the platform micro-service in response to the database event information being the task completion information of the front-end micro-service includes:

Acquiring task completion information of the front-end micro-service;

The platform micro-service can timely execute database change tasks after the tasks of the front-end micro-service are completed, and the tasks are ensured to be started and executed according to expectations through polling inquiry and health status checking, so that the overall response capability of the system and the reliability of task execution are improved.

It should be understood that, although the steps in the flowchart of fig. 2 are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in fig. 2 may include multiple sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, nor do the order in which the sub-steps or stages are performed necessarily performed in sequence, but may be performed alternately or alternately with at least a portion of the sub-steps or stages of other steps or other steps.

It should be noted that, in one embodiment, a database log monitoring parser and a message monitoring dispatcher are introduced to implement the message notification method shown in fig. 2, and the detailed workflow is shown in fig. 3, where the platform micro-service includes a computing platform micro-service, a network platform micro-service, and a storage platform micro-service, and the platform micro-service message queue includes a computing platform micro-service message queue, a network platform micro-service message queue, and a storage platform micro-service message queue.

In one embodiment, the database listening resolver obtains database event information by masquerading slave nodes, the specific workflow being shown in FIG. 4.

In one embodiment, a master copy selector is introduced to implement multi-copy deployment and copy selection, and the work flow of the master copy selector is shown in fig. 5, where the master copy selector is further connected with a monitoring device and an alarm device, so as to monitor whether the master copy selector operates normally at any time.

In one embodiment, the introduction of the message pusher enables pushing of database event information, the workflow of the message pusher is shown in FIG. 6.

In one embodiment, as shown in fig. 7, a message notification device is provided, in which a database, a platform layer, a virtualization layer and a message general team are deployed in a cloud platform system, a plurality of platform micro services are deployed in the platform layer, a plurality of management micro services are deployed in the virtualization layer, and the device is applied to the cloud platform system, and the device comprises a queue creation module, a log monitoring analysis module, a message pushing module, a message judging module and a message feedback module, wherein:

The log monitoring module is used for setting a disguised slave node in a database based on a master-slave protocol, acquiring a database log pushed by a master node through a read-write thread based on the disguised slave node, acquiring data change information according to the database log, analyzing according to the data change information to acquire database change task information, and carrying out corresponding data structure adjustment according to the database change task information and the database event type to acquire database event information.

The log monitoring module is also used for analyzing and generating serialized data according to the database change task information and combining the database event type, analyzing the database change task information into a serialized array in response to the database event type being an adding and deleting event, analyzing the database change task information into a serialized hash table in response to the database event type being an updating event, analyzing again based on the serialized data, and converting the serialized data into a key value data structure to obtain the database event information.

The log monitoring module is further used for responding to the starting of the plurality of copies to correspondingly generate a plurality of copy serial numbers, writing the plurality of copy serial numbers into the same ordered set to obtain a current copy sequence, selecting a master copy from the plurality of copies according to the current copy sequence, enabling the remaining copies to be slave copies, acquiring a database log based on the copies, responding to the copies being the master copies, generating database event information based on the master copies and the database log received from the slave copies, and responding to the copies being the slave copies, discarding the database log received from the slave copies.

The log monitoring module is further used for respectively carrying out heartbeat detection on a plurality of copies based on preset heartbeat intervals, updating the copy sequence according to the current copy sequence if the heartbeat detection result is healthy and the copy sequence corresponding to the copy exists, judging whether the copy can recover health in expiration time if the heartbeat detection result is failure and the copy sequence corresponding to the copy exists in the current copy sequence, deleting the copy sequence in the current copy sequence if the heartbeat detection result is negative, adding the copy sequence to the last of the current copy sequence if the heartbeat detection result is healthy and the copy sequence corresponding to the copy does not exist in the current copy sequence, and judging the copy as a master copy if the copy sequence corresponding to the copy is first and vice versa.

The message pushing module is used for acquiring a master copy of a disguised slave node, creating a master pushing single thread according to the master copy, pushing the database event information to a message general queue through the master pushing single thread in response to generation of the database event information based on the master copy, storing the database event information into a message re-pushing queue in response to failure of the master copy to successfully push the database event information to the message general queue, creating a message re-pushing thread according to the message re-pushing queue, pushing the database event information in the message re-pushing queue to the message general queue through the message re-pushing thread, and pushing the database event information to a platform micro-service message queue in response to failure of the message general queue to receive the database event information.

The message pushing module is further used for storing the database event information into a message broadcasting queue in response to the reception of the database event information by the message general team, and distributing the database event information to all platform micro-service message queues through the message broadcasting queue in response to the reception of the database event information by the message broadcasting queue, wherein the database event information from the message re-pushing queue is marked as a failed re-pushing message.

The message judging module is used for carrying out preposed inquiry based on the platform micro service to judge whether the preposed micro service exists in the platform micro service or not, responding to the condition that the preposed micro service does not exist in the platform micro service, executing a database changing task by the platform micro service, feeding back the completion information of the database changing task to the database, responding to the condition that the preposed micro service exists in the platform micro service, acquiring the task completion information of the preposed micro service of the platform micro service, and judging whether the database changing task is executed or not according to the task completion information of the preposed micro service.

The message judging module is further used for responding to the fact that the front-end micro service exists in the platform micro service, obtaining the current front-end micro service of the platform micro service, setting the platform micro service to be started in a delayed mode, generating a delayed starting task corresponding to a database changing task, responding to the fact that the current front-end micro service is not the management micro service, taking the current front-end micro service as a new platform micro service, recursively carrying out front-end query until the obtained current front-end micro service is the management micro service, responding to the fact that the current front-end micro service is the management micro service, calling a resource configuration task corresponding to the database changing task to be executed by the management micro service to obtain task completion information of the management micro service, responding to the fact that the platform micro service receives the task completion information of the current front-end micro service, closing the delayed starting task and executing the platform micro service to obtain the task completion information of the platform micro service.

The message judging module is also used for setting the delay starting time of the delay starting task corresponding to the database change task according to the historical execution time of the resource configuration task corresponding to the management micro-service execution database change task based on a dynamic calculation mode.

The message judging module is also used for responding to the delayed starting task generation, creating corresponding task record information and storing the task record information in a task record list, wherein the task record information at least comprises one item of task serial number, required resource type and request resource serial number, and the database event information is distributed to a platform micro-service message queue through a message general team, wherein the database event information at least comprises one item of service completion serial number, configuration resource type, configuration resource serial number, configuration resource current state and failure re-pushing mark, the database event information is the relevant information of the platform micro-service in response to the condition that the required resource type is consistent with the configuration resource type, the database event information is the relevant information of the platform micro-service in response to the condition that the database event information is the relevant information of the platform micro-service, and comparing is carried out in combination with the task record list, so as to judge whether the database event information is the task completion information of the front-end micro-service or not, and otherwise, the database event information is discarded in the platform micro-service message queue.

The message judging module is also used for setting task completion information of the front micro-service, wherein the task completion information comprises failure completion information and effective completion information, inquiring all task record information in a task record table, responding to the fact that the task serial numbers are consistent with the service completion serial numbers, enabling the database event information to be relevant information of the front micro-service, responding to the fact that the database event information is relevant information of the front micro-service, acquiring a failure push mark, responding to the fact that the failure push mark is yes, enabling the database event information to be failure completion information, responding to the fact that the failure push mark is no, acquiring the current state of configuration resources, responding to the fact that the current state of the configuration resources is steady, enabling the database event information to be effective completion information of the front micro-service, and otherwise enabling the database event information to be unfinished information of the front micro-service.

The message judging module is also used for acquiring task completion information of the front-end micro-service, directly waking up and executing the delay starting task in response to the task completion information of the front-end micro-service being effective completion information, waking up and starting the delay starting task in response to the task completion information of the front-end micro-service being invalid completion information and executing the delay starting task based on a polling inquiry mode, and deleting task record information corresponding to the delay starting task from a task record table in response to the delay starting task being executed.

The specific limitation of the message notification device can be referred to above as limitation of the message notification method, and will not be described herein. The respective modules in the above-described message notification apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In one embodiment, a computer device is provided, which may be a server, and the internal structure of which may be as shown in fig. 8. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is for storing message notification data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a message notification method.

It will be appreciated by persons skilled in the art that the architecture shown in fig. 4 is merely a block diagram of some of the architecture relevant to the present inventive arrangements and is not limiting as to the computer device to which the present inventive arrangements are applicable, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.

In one embodiment, a computer device is provided comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of when executing the computer program:

Acquiring a database log, and analyzing to obtain database event information;

In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:

Acquiring a database log, and analyzing to obtain database event information;

Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (SYNCHLINK) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The foregoing examples illustrate only a few embodiments of the application, which are described in detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application.

Claims

1. The method is applied to a cloud platform system, and is characterized in that a database, a platform layer, a virtualization layer and a message general team are deployed in the cloud platform system, a plurality of platform micro-services are deployed in the platform layer, and a plurality of management micro-services are deployed in the virtualization layer, and the method comprises the following steps:

Acquiring a database log, and analyzing to obtain database event information;

Judging whether the database change task is executed or not according to the database event information in the platform micro-service message queue, wherein the judging comprises the steps of responding to the existence of the front micro-service in the platform micro-service, acquiring the current front micro-service of the platform micro-service, setting the platform micro-service as time delay starting, and generating a time delay starting task corresponding to the database change task; creating a corresponding task record information in a task record list after the delay is started, wherein the task record information at least comprises a required resource type, distributing database event information to a platform micro-service message queue through a message general queue, wherein the database event information at least comprises a configuration resource type, responding to the condition that the required resource type is consistent with the configuration resource type, the database event information is the related information of the platform micro-service, responding to the database event information is the related information of the platform micro-service, comparing the task record list, judging whether the database event information is the task completion information of the front micro-service or not, otherwise, discarding the database event information in the platform micro-service message queue, wherein the task completion information of the front micro-service is obtained by at least comprising one or more of the platform micro-service and management micro-service, responding to the condition that the current front micro-service is not the management micro-service, taking the current front micro-service as a new platform micro-service, conducting front-end inquiry until the obtained current front micro-service is the management micro-service, responding to the current front micro-service and calling the management micro-service, and executing the corresponding management micro-service to the configuration micro-service, responding to the platform micro-service receiving the task completion information of the current front-end micro-service, closing the delay starting task and executing the platform micro-service to obtain the task completion information of the platform micro-service;

2. The method for notifying messages according to claim 1, wherein the database is a master-slave architecture, the database includes a master node and a slave node, the obtaining a database log, and analyzing to obtain database event information includes:

setting disguised slave nodes in the database based on a master-slave protocol;

Based on the disguised slave node, acquiring the database log pushed by the master node through a read-write thread;

Obtaining data change information according to the database log;

3. The method of claim 2, wherein the performing corresponding data structure adjustment according to the database change task information and the database event type to obtain database event information includes:

responding to the database event type as an adding and deleting event, analyzing the database change task information into a serialization array;

and resolving again based on the serialized data, and converting the serialized data into a key value data structure to obtain the database event information.

4. The method of claim 2, wherein the masquerading slave node has a number of copies in the database, wherein the number of copies is greater than or equal to three, and the pushing the database event information to the platform micro service message queue includes:

writing a plurality of the copy serial numbers into the same ordered set to obtain a current copy sequence;

Selecting a master copy from a plurality of copies according to the current copy sequence, wherein the remaining copies are slave copies;

Based on the copy, acquiring the database log, and generating the database event information according to the database log based on the master copy in response to the copy being the master copy;

And discarding the database log received from the slave copy in response to the slave copy.

5. The method of claim 4, wherein selecting a master copy from among the plurality of copies based on the current copy sequence, and the remaining copies being slave copies, comprises:

Responding to the result of the heartbeat detection as healthy and the current copy sequence has the copy sequence number corresponding to the copy, and updating the copy sequence number according to the current copy sequence;

Responding to the result of the heartbeat detection as a fault and the copy serial number corresponding to the copy exists in the current copy sequence, judging whether the copy can recover health in the expiration time, and responding to the judgment as no, deleting the copy serial number in the current copy sequence;

responding to the result of the heartbeat detection as healthy and the copy sequence number corresponding to the copy does not exist in the current copy sequence, and adding the copy sequence number to the last of the current copy sequence;

and responding to the copy sequence corresponding to the copy, wherein the current copy sequence is the first bit, the copy is the master copy, and otherwise, the copy is the slave copy.

6. The method for notifying messages according to claim 4, wherein a message re-pushing queue is further disposed in the cloud platform system, and the pushing the database event information to the platform micro service message queue includes:

Obtaining the master copy of the disguised slave node;

creating a main push single thread according to the main copy;

in response to generating the database event information based on the master copy, pushing the database event information to the message general team through the master push single thread;

Responding to the fact that the master copy fails to successfully push the database event information to the message general queue, and storing the database event information into the message re-push queue;

creating a message re-pushing thread according to the message re-pushing queue;

pushing the database event information in the message re-pushing queue to the message general team through the message re-pushing thread;

And in response to the message general team successfully receiving the database event information, pushing the database event information to the platform micro-service message queue.

7. The method of claim 6, wherein a message broadcast queue is further disposed in the message queue, and wherein pushing the database event information to the platform micro service message queue in response to the message queue successfully receiving the database event information comprises:

Storing the database event information into the message broadcast queue in response to the message general queue receiving the database event information;

and in response to the message broadcast queue receiving the database message, distributing the database message to all the platform micro service message queues through the message broadcast queue, wherein the database event information from the message re-push queue is marked as a failed re-push message.

8. The method of claim 1, wherein determining whether the database change task is performed according to the database event information in the platform micro service message queue comprises:

based on the platform micro-service, performing front-end inquiry to judge whether the platform micro-service has front-end micro-service;

Responding to the platform micro-service without the front-end micro-service, executing the database change task by the platform micro-service, and feeding back the completion information of the database change task to the database;

9. The message notification method of claim 8, wherein the task recording information further comprises at least one of a task number, a request resource number;

The database event information at least comprises one item of service completion sequence number, configuration resource current state and failure re-push mark.

10. The method of claim 9, wherein the determining whether the database event information is the task completion information of the front-end micro service by comparing the database event information with the task record table in response to the database event information being the related information of the platform micro service, further comprises:

inquiring all the task record information in the task record table, and responding to the fact that the task sequence number is consistent with the service completion sequence number, wherein the database event information is the related information of the front micro-service;

acquiring the failure re-pushing mark in response to the database event information being the related information of the front-end micro service, and acquiring the failure completion information in response to the failure re-pushing mark being yes;

And responding to the current state of the configuration resource to be steady state, wherein the database event information is effective completion information of the front-end micro-service, otherwise, the database event information is incomplete information of the front-end micro-service.

11. The method of claim 10, wherein executing the delayed initiation task corresponding to the platform micro service in response to the database event information being task completion information of the front-end micro service, comprises:

acquiring task completion information of the front-end microservice;

Responding to the task completion information of the front-end micro-service as the effective completion information, and directly waking up the delay starting task and executing the delay starting task;

The task completion information of the front-end micro-service is responded to be the failure completion information, the delay starting task is awakened, and the task is executed based on a polling inquiry mode;

and in response to the execution of the time delay starting task, deleting the task information corresponding to the time delay starting task from the task record table.

12. A message notification apparatus for implementing the message notification method of any of claims 1-11, the apparatus being applied to a cloud platform system, wherein a database, a platform layer, a virtualization layer, and a message team are deployed in the cloud platform system, a plurality of platform micro services are deployed in the platform layer, a plurality of management micro services are deployed in the virtualization layer, and the apparatus comprises:

13. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any one of claims 1 to 11 when the computer program is executed by the processor.

14. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 11.