CN114201501A - A message processing method, device, electronic device and storage medium - Google Patents
A message processing method, device, electronic device and storage medium Download PDFInfo
- Publication number
- CN114201501A CN114201501A CN202111341022.0A CN202111341022A CN114201501A CN 114201501 A CN114201501 A CN 114201501A CN 202111341022 A CN202111341022 A CN 202111341022A CN 114201501 A CN114201501 A CN 114201501A
- Authority
- CN
- China
- Prior art keywords
- target
- data change
- message
- change message
- target data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Information Transfer Between Computers (AREA)
Abstract
本公开关于一种消息处理方法、装置、电子设备及存储介质，其中，消息处理方法包括：获取目标数据变动消息；所述目标数据变动消息指示目标业务场景的数据变动；确定配置信息中与所述目标业务场景相匹配的目标时间信息；所述目标时间信息用于指示存储时长；根据所述目标时间信息确定所述目标数据变动消息在缓存中的存储时长，并将所述目标数据变动消息存储至所述缓存中；在确定满足任务执行条件时，对所述缓存中的所述目标数据变动消息执行对账任务。本公开减小了实时对账过程中对存储资源的占用，提高了存储资源利用率。
The present disclosure relates to a message processing method, device, electronic device, and storage medium, wherein the message processing method includes: acquiring a target data change message; the target data change message indicates a data change of a target business scenario; The target time information that matches the target business scenario; the target time information is used to indicate the storage duration; the storage duration of the target data change message in the cache is determined according to the target time information, and the target data change message is stored. Store in the cache; when it is determined that the task execution condition is satisfied, perform a reconciliation task on the target data change message in the cache. The present disclosure reduces the occupation of storage resources in the process of real-time account reconciliation, and improves the utilization rate of storage resources.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a message processing method and apparatus, an electronic device, and a storage medium.
Background
In a distributed system, many unsafe factors such as network failure, system error reporting, human errors and the like exist among cross-systems, and particularly in the e-commerce field and the payment field, the data consistency among the cross-systems needs to be ensured through account checking. The reconciliation is divided into real-time reconciliation and offline reconciliation, wherein the real-time reconciliation can find the problem of data inconsistency among systems more timely.
In the related art, when the real-time account checking is performed, the data change messages stored in the database are processed based on a timing task mechanism, so that all the data change messages in the database need to be stored for a predetermined time, such as at least one month, a large amount of storage resources are occupied, and the utilization rate of the storage resources is reduced.
Disclosure of Invention
The disclosure provides a message processing method, a message processing device, an electronic device and a storage medium, which are used for at least solving the problems that a large amount of storage resources are occupied and the utilization rate of the storage resources is reduced in a real-time account checking process in the related art. The technical scheme of the disclosure is as follows:
according to a first aspect of the embodiments of the present disclosure, there is provided a message processing method, including:
acquiring a target data change message; the target data change message indicates data change of a target service scene;
determining target time information matched with the target service scene in the configuration information; the target time information is used for indicating a storage duration;
determining the storage duration of the target data change message in a cache according to the target time information, and storing the target data change message in the cache;
and executing a reconciliation task on the target data change message in the cache when the task execution condition is determined to be met.
In an exemplary embodiment, the target time information includes a check time length and a storage time length adjustment factor;
the determining the storage duration of the target data change message in the cache according to the target time information includes:
determining the product of the check duration and the storage duration adjustment factor in the target time information;
and taking the product as the storage duration of the target data change message in a cache.
In an exemplary embodiment, the method further comprises:
determining a target delay time length corresponding to the target data change message according to the check time length;
generating a target delay message according to the target data change message and the target delay duration, and storing the target delay message to a delay queue;
and when the delay time of the target delay message in the delay queue reaches the target delay time, determining that a task execution condition is met.
In an exemplary embodiment, the method further comprises:
responding to the configuration instruction, and displaying a configuration page; the configuration page comprises a service scene area and a corresponding time information area;
and generating the configuration information according to the input content in the service scene area and the input content in the corresponding time information area.
In an exemplary embodiment, the performing a reconciliation task on the target data change message in the cache includes:
searching whether the cache has a to-be-reconciled data variation message corresponding to the target data variation message;
and if the target data change message and the to-be-reconciled data change message are found, carrying out data consistency check on the target data change message and the to-be-reconciled data change message.
In an exemplary embodiment, the obtaining the target data change message includes:
when monitoring a data change message of a target database, storing the data change message to a message queue;
and acquiring the data change message from the message queue, and preprocessing the data change message to obtain the target data change message.
According to a second aspect of the embodiments of the present disclosure, there is provided a message processing apparatus including:
a target data change message acquisition unit configured to perform acquisition of a target data change message; the target data change message indicates data change of a target service scene;
a target time information determining unit configured to perform determining target time information matching the target service scenario in configuration information; the target time information is used for indicating a storage duration;
the storage unit is configured to determine the storage duration of the target data change message in a cache according to the target time information, and store the target data change message in the cache;
and the account checking task execution unit is configured to execute an account checking task on the target data change message in the cache when the task execution condition is determined to be met.
In an exemplary embodiment, the target time information includes a check time length and a storage time length adjustment factor; the memory cell includes:
a calculation unit configured to perform determination of a product of a collation duration and a storage duration adjustment factor in the target time information;
a storage duration determination unit configured to perform storage of the product as the target data change message in a cache.
In an exemplary embodiment, the apparatus further comprises:
a target delay time length determining unit configured to determine a target delay time length corresponding to the target data change message according to the check time length;
a delay message generating unit configured to perform generating a target delay message according to the target data variation message and the target delay duration, and store the target delay message to a delay queue;
a task execution condition determination unit configured to perform determining that a task execution condition is satisfied when a delay time period of the target delay message in the delay queue reaches the target delay time period.
In an exemplary embodiment, the apparatus further comprises:
the configuration page display unit is configured to respond to the configuration instruction and display the configuration page; the configuration page comprises a service scene area and a corresponding time information area;
and the configuration information generating unit is configured to generate the configuration information according to the input content in the service scene area and the input content in the corresponding time information area.
In an exemplary embodiment, the reconciliation task execution unit includes:
the searching unit is configured to search whether the cache has the account data change message to be checked corresponding to the target data change message;
and the consistency checking unit is configured to perform data consistency checking on the target data change message and the to-be-checked account data change message if the searching unit finds the target data change message and the to-be-checked account data change message.
According to a third aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the message processing method of the first aspect.
According to a fourth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium, wherein instructions of the computer-readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the message processing method of the first aspect.
According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product comprising computer programs/instructions which, when executed by a processor, implement the message processing method of the first aspect described above.
The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects:
the target time information which is matched with the target service scene in the configuration information is determined by obtaining the target data change information which indicates the data change of the target service scene, the target time information is used for indicating the storage duration, the storage duration of the target data change information in the cache can be further determined according to the target time information, the target data change information is stored in the cache, and the reconciliation task is executed on the target data change information in the cache when the task execution condition is met, so that the data change information of different service scenes can correspond to different storage durations in the cache, the data change information of the corresponding service scene can be timely cleaned when the storage duration is up, the occupation of storage resources in the real-time reconciliation process is reduced, and the utilization rate of the storage resources is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.
FIG. 1 is a schematic diagram of an application environment illustrating a method of message processing in accordance with an exemplary embodiment;
FIG. 2 is a flow diagram illustrating a method of message processing in accordance with an exemplary embodiment;
FIG. 3 is a flow diagram illustrating another message processing method in accordance with an illustrative embodiment;
FIG. 4 is a block diagram illustrating a message processing apparatus in accordance with an exemplary embodiment;
FIG. 5 is a block diagram illustrating an electronic device in accordance with an example embodiment.
Detailed Description
In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
Referring to fig. 1, a schematic diagram of an application environment of a message processing method according to an exemplary embodiment is shown, where the application environment includes a first database 110, a second database 120, a real-time reconciliation server 130 and a terminal 140.
The first database 110 and the second database 120 respectively correspond to two different service systems in a distributed system, as shown in fig. 1, the first database 110 corresponds to a service system a in the distributed system, the second database 120 corresponds to a service system B in the distributed system, that is, the first database 110 is used for storing data in the service system a, and the second database 120 is used for storing data in the service system B, wherein data consistency between the service system a and the service system B needs to be ensured by real-time reconciliation, for example, the service system a may be a transaction system, and the service system B may be an accounting system.
The connection between the real-time reconciliation server 130 and the first database 110 and the connection between the real-time reconciliation server 130 and the second database 120 can be through a wired network or a wireless network. The real-time reconciliation server 130 is configured to provide a real-time reconciliation service to ensure data consistency between the business system a and the business system B.
The first database 110, the second database 120 and the real-time reconciliation server 130 may be one server or a server cluster composed of a plurality of servers.
The terminal 140 may be connected to the real-time reconciliation server 130 through a wired network or a wireless network, and the terminal 140 may be a smart phone, a tablet computer, a notebook computer, a desktop computer, or the like. Specifically, the terminal 140 may provide a reconciliation management service, and the reconciliation management service allows the business personnel to configure corresponding information.
It can be understood that the first database 110 corresponding to the business system a and the second database 120 corresponding to the business system B in fig. 1 are only examples, and in practical applications, more business systems and databases corresponding to the business systems may be included that need to perform data consistency verification, which is not specifically limited in this disclosure.
The message processing method of the embodiment of the present disclosure may be executed by a message processing apparatus, where the message processing apparatus may be configured in an electronic device, taking the electronic device as the real-time reconciliation server 130 as an example, the real-time reconciliation server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as cloud service, a cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and a big data and artificial intelligence platform.
Fig. 2 is a flow chart illustrating a message processing method according to an exemplary embodiment, as shown in fig. 2, including the steps of:
in step S201, a target data change message is acquired.
Wherein the target data change message indicates a data change of a target service scenario. Taking the application environment shown in fig. 1 as an example, the target data change message may be from the first database 110 or from the second database 120.
In an exemplary embodiment, in order to improve the message processing efficiency in the real-time reconciliation, the obtaining of the target data change message may include the following steps:
when monitoring a data change message of a target database, storing the data change message to a message queue;
and acquiring the data change message from the message queue, and preprocessing the data change message to obtain the target data change message.
In particular, the data change message may be embodied as a binlog message, which is a binary log that records all database TABLE structure changes (e.g., CREATE, ALTER TABLE, etc.) and TABLE data modifications (INSERT, UPDATE, DELETE, etc.). The real-time reconciliation server can monitor the binary log binlog of the target database, when the binlog message is monitored, the binlog message is stored in the message queue, and then the binlog message is acquired from the message queue and is preprocessed to obtain the target data change message.
The preprocessing of the data change message may include filtering and assembling, where the filtering is to determine whether the data change indicated by the data change message is a data change in a service scenario related to configuration information, if so, the data change message may be used as a target data change message, and if not, the data change message is not used as the target data change message; the assembling means that the format of the data change message is adjusted to meet the requirement of a preset format, so that the message processing efficiency is improved, wherein the preset format can be set according to actual needs.
The message queue in the embodiment of the present disclosure may be, but is not limited to, a Kafka message queue, where Kafka is a high throughput distributed message queue system, and the target database may write the data change message into the Kafka message queue after generating the data change message, and the Kafka message queue is responsible for forwarding, so that the processing efficiency of the real-time reconciliation server for the cross-system message is improved through the message queue.
In step S203, target time information matching the target service scenario in the configuration information is determined.
The configuration information represents a corresponding relation between a service scene and time information, the target time information is used for indicating storage duration, and the service scene comprises the target service scene.
The service scenario may be set according to a scenario that requires real-time account checking in practical application, and may include, for example, a payment scenario, a refund scenario, a ticket refund scenario, and the like. The time information may be a specific storage duration, for example, 3 minutes, and it can be understood that, in order to meet account checking requirements of different service scenarios, the time information corresponding to different service scenarios needs to be determined in combination with the checking duration in the corresponding service scenario, that is, the storage duration indicated by the time information corresponding to each service scenario needs to be greater than or equal to the checking duration in the service scenario, where the checking duration indicates a duration required for completing a service operation in the corresponding service scenario. Taking the payment scenario as an example, assuming that the payment operation needs to be completed within 2 minutes, the check time duration corresponding to the payment scenario is 2 minutes, and the storage time duration indicated by the time information corresponding to the payment scenario is at least 2 minutes, which may be 3 minutes, 4 minutes, and so on, for example.
Based on this, in an exemplary embodiment, the method may further include a generation process of the configuration information, and specifically, the generation of the configuration information may include:
responding to the configuration instruction, and displaying a configuration page; the configuration page comprises a service scene area and a corresponding time information area;
and generating the configuration information according to the input content in the service scene area and the input content in the corresponding time information area.
In practical application, the input content in the service scene area may be a regular expression corresponding to the service scene, that is, the service scene in the configuration information may be represented by a corresponding regular expression, and the regular expression may be obtained by performing the regular expression based on a data table of a corresponding service scene in the target database. Taking a payment scenario as an example, assuming that there are 100 payment tables in the target database, and each payment table has a corresponding table identifier, performing regular expression on the table identifiers of the 100 payment tables in the target database to obtain a regular expression for representing the payment scenario.
Based on this, in an optional implementation manner, the target data change message may carry a target service scene identifier indicating a target service scene, where the target service scene identifier may be a table identifier corresponding to a data table in which data change occurs in the target service scene in the target database, and when determining target time information in the configuration information that matches the target service scene, time information corresponding to a regular expression hit by the table identifier in the configuration information may be determined as the target time information that matches the target service scene.
It can be understood that the service scenario and the corresponding time information in the configuration information may be adjusted according to the actual situation. The configuration information is generated by the input content of the business scene area and the input content of the corresponding time area in the configuration page, so that the generation process of the configuration information is simplified, the generation flexibility of the configuration information is improved, and the processing efficiency of the real-time account checking message can be improved on the whole.
In step S205, a storage duration of the target data change message in the cache is determined according to the target time information, and the target data change message is stored in the cache.
When the time length of the target data change message existing in the cache reaches the determined corresponding storage time length, the target data change message is deleted from the cache to release the occupied cache space. In a specific implementation, the storage duration indicated by the target time information may be used as the storage duration of the target data change message in the cache.
In order to match the storage duration of the target data change message in the cache with the duration required for completing the target service operation in the target service scenario, and improve the flexibility of message processing while ensuring the completion of the reconciliation task, in an exemplary embodiment, the time information in the configuration information may include a check duration and a storage duration adjustment factor, wherein the check duration indicates the duration required for completing the service operation in the corresponding service scenario, and the storage duration adjustment factor is greater than or equal to 1. Based on this, the step S205, when determining the storage duration of the target data change message in the cache according to the target time information, may include:
determining the product of the check duration and the storage duration adjustment factor in the target time information;
and taking the product as the storage duration of the target data change message in a cache.
In a specific implementation, the stored duration adjustment factors in the time information corresponding to each service scenario in the configuration information may be the same or different, and when the stored duration adjustment factors in the time information corresponding to each service scenario are the same, in order to improve the generation efficiency of the configuration information and further improve the overall message processing efficiency, a global duration adjustment factor may be set, and the global duration adjustment factor is used as the stored duration adjustment factor in the time information corresponding to each service scenario.
Taking three service scenes as a payment scene, a refund scene and a ticket refund scene as examples, assuming that payment needs to be completed in 2 minutes, refund needs to be completed in 2 hours, ticket refund needs to be completed in 2 days, and a global duration adjustment factor is N (N ≧ 1), time information corresponding to the payment scene in configuration information can be represented as {2 minutes, N }, time information corresponding to the refund scene can be represented as {2 minutes, 60 minutes, N }, and time information corresponding to the ticket refund scene can be represented as {2 minutes, 24 minutes, 60 minutes, N }, wherein the checking duration corresponding to the payment scene is 2 minutes, the checking duration corresponding to the refund scene is 2 minutes, and the checking duration corresponding to the ticket refund scene is 2 minutes, 24 minutes, and 60 minutes. Then, based on the configuration information, it may be determined that, when the target service scenario is a payment scenario, the storage duration of the corresponding target data change message in the cache is 2 × N minutes; when the target service scene is a refund scene, the storage time of the corresponding target data change message in the cache is 2 × 60 × N minutes; when the target service scene is a ticket refunding scene, the storage time of the corresponding target data change message in the cache is 2 × 24 × 60 × N minutes.
The time information corresponding to the corresponding service scene is represented by the checking duration and the storage duration adjusting factor in the configuration information, so that the generation efficiency and the adjustment flexibility of the configuration information are improved, only the storage duration adjusting factor needs to be adjusted in subsequent adjustment, in addition, the storage duration in each service scene is matched with the checking duration of the corresponding service scene, the message processing efficiency is improved, and the completion rate of the reconciliation task is ensured.
In step S207, when it is determined that the task execution condition is satisfied, an account checking task is executed on the target data change message in the cache.
Still taking the three exemplary service scenarios as examples, the number of data changes related in the ticket refund scenario and the refund scenario is usually much smaller than the number of data changes in the payment scenario, that is, in this example, a large number of data change messages in the payment scenario are stored in the cache, and the check duration of the payment scenario is much shorter than that of the ticket refund scenario and the refund scenario, and if the storage duration of the data change messages in the cache in the payment scenario is reduced, the data change messages in the payment scenario can be cleared from the cache in time, so that the occupation of the storage space by the real-time account checking is greatly reduced.
In the embodiment of the disclosure, considering that real-time account checking generally supports a large number of service scenes, and the time length required by service operation in each service scene is usually different, the time length of the personalized indication storage time length is maintained for each service scene through the configuration information, so that the storage time length of the data change message in the cache in each service scene can be matched with the corresponding service scene, and thus the data change message processed by account checking can be flexibly and timely cleaned from the cache, the occupation of the storage space is reduced, the storage cost is saved, and the utilization rate of the storage resource is improved.
In a specific implementation, the task execution condition may be that a to-be-checked data change message corresponding to the target data change message appears in the cache, based on which the cache may be scanned according to a preset time interval, and when the to-be-checked data change message corresponding to the target data change message exists in the cache, it is indicated that the task execution condition is currently satisfied, and a check-out task may be performed on the target data change message in the cache, where the check-out task is to perform data consistency check on the target data change message and the to-be-checked data change message. It should be noted that, when scanning the cache according to the preset time interval, in order to minimize waste of the storage resource, the preset time interval does not exceed a unit time length of a storage time length of the data change message in the cache, for example, if the storage time length of the data change message in the cache is 2 minutes, the preset time interval does not exceed 1 minute.
In an exemplary embodiment, in order to improve the reconciliation efficiency of the real-time reconciliation while improving the utilization rate of the storage space, another message processing method as provided in fig. 3 includes:
in step S301, a target data change message is acquired.
Wherein the target data change message indicates a data change of a target service scenario.
In step S303, target time information matching the target service scenario in the configuration information is determined.
The configuration information represents a corresponding relation between a service scene and time information, the target time information is used for indicating storage duration, and the service scene comprises the target service scene. Specifically, the target time information may include a check time length and a storage time length adjustment factor.
In step S305, determining a storage duration of the target data change message in a cache according to the target time information, and storing the target data change message in the cache.
Specifically, the product of the check duration and the storage duration adjustment factor in the target time information may be determined, and the product may be used as the storage duration of the target data change message in the cache.
It should be noted that, for the specific implementation process of step S301 to step S305, reference may be made to the related description of the method embodiment shown in fig. 2 in the embodiment of the present disclosure, and details are not repeated here.
In step S307, a target delay duration corresponding to the target data change message is determined according to the check duration in the target time information.
Specifically, the check time length in the target time information may be used as the target delay time length corresponding to the target data change message.
In step S309, a target delay message is generated according to the target data change message and the target delay duration, and the target delay message is stored in a delay queue.
Specifically, a unique message identifier may be assigned to each data change message stored in the cache, and a data change message may be uniquely determined by the message identifier. The message identifier of the target data change message can be obtained when the target delay message is generated according to the target data change message and the target delay time length, and the target delay message is generated according to the message identifier and the target delay time length, wherein the target delay time length indicates the time length of the target delay message needing to be delayed in the delay queue.
Implementation manner of the delay queue is not specifically limited in this regard, and may be implemented, for example, by using a data structure of an ordered set of Redis (a memory-based key-value database), or by using message middleware RabbitMQ.
In step S311, when the delay time length of the target delay message in the delay queue reaches the target delay time length, it is determined that a task execution condition is satisfied.
Wherein the delay time period of the target delay message in the delay queue is a difference between the time when the target delay message is written into the delay queue and the current time, and when the difference reaches the target delay time period, indicating that the task execution condition has been met, step S313 may be executed.
In step S313, an account checking task is performed on the target data change message in the cache.
Specifically, the target delay message may be extracted from the delay queue, and the target data change message corresponding to the message identifier may be found from the cache according to the message identifier in the target delay message, so as to perform a reconciliation task on the target data change message.
The embodiment of the disclosure aims at the personalized configuration checking duration of different service scenes, and ensures that the reconciliation task can be executed on the data change message in the cache at the first time according to the checking duration and the delay queue, so that the real-time reconciliation efficiency is improved, the reconciliation result can be obtained in time, the problem of data inconsistency among systems can be found in time, and the processing efficiency of the data inconsistency among the systems can be improved.
In an exemplary embodiment, the step S313, when performing the reconciliation task on the target data change message in the cache, may include the following steps:
searching whether the cache has a to-be-reconciled data variation message corresponding to the target data variation message;
and if the target data change message and the to-be-reconciled data change message are found, carrying out data consistency check on the target data change message and the to-be-reconciled data change message.
And if the account is not found, generating an alarm message indicating account checking failure.
According to the method and the device, the data consistency is checked when the data change message to be checked corresponding to the target data change message in the cache is searched, and the alarm message indicating the checking failure is generated when the data change message is not searched, so that the real-time checking processing is efficiently realized, related business personnel can timely find the problem between systems, the real-time checking efficiency is improved, and the solution efficiency of the problem between systems is improved.
Fig. 4 is a block diagram illustrating a message processing apparatus according to an example embodiment. Referring to fig. 4, the message processing apparatus 400 includes a target data change message acquisition unit 410, a target time information determination unit 420, a storage unit 430, and a reconciliation task execution unit 440, in which:
a target data change message acquiring unit 410 configured to perform acquiring a target data change message; the target data change message indicates data change of a target service scene;
a target time information determination unit 420 configured to perform determining target time information matching the target service scenario in configuration information; the target time information is used for indicating a storage duration;
a storage unit 430 configured to determine a storage duration of the target data change message in a cache according to the target time information, and store the target data change message in the cache;
an account checking task execution unit 440 configured to execute an account checking task on the target data change message in the cache when it is determined that a task execution condition is satisfied.
In an exemplary embodiment, the target time information includes a check time length and a storage time length adjustment factor; the memory cell includes:
a calculation unit configured to perform determination of a product of a collation duration and a storage duration adjustment factor in the target time information;
a storage duration determination unit configured to perform storage of the product as the target data change message in a cache.
In an exemplary embodiment, the apparatus further comprises:
a target delay time length determining unit configured to determine a target delay time length corresponding to the target data change message according to the check time length;
a delay message generating unit configured to perform generating a target delay message according to the target data variation message and the target delay duration, and store the target delay message to a delay queue;
a task execution condition determination unit configured to perform determining that a task execution condition is satisfied when a delay time period of the target delay message in the delay queue reaches the target delay time period.
In an exemplary embodiment, the apparatus further comprises:
the configuration page display unit is configured to respond to the configuration instruction and display the configuration page; the configuration page comprises a service scene area and a corresponding time information area;
and the configuration information generating unit is configured to generate the configuration information according to the input content in the service scene area and the input content in the corresponding time information area.
In an exemplary embodiment, the reconciliation task execution unit includes:
the searching unit is configured to search whether the cache has the account data change message to be checked corresponding to the target data change message;
and the consistency checking unit is configured to perform data consistency checking on the target data change message and the to-be-checked account data change message if the searching unit finds the target data change message and the to-be-checked account data change message.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
In one exemplary embodiment, there is also provided an electronic device, comprising a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement any one of the message processing methods provided in the embodiments of the present disclosure when executing the instructions stored in the memory.
The memory may be used to store software programs and modules, and the processor may execute various functional applications and data processing by operating the software programs and modules stored in the memory. The memory can mainly comprise a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs needed by functions and the like; the storage data area may store data created according to use of the apparatus, and the like. Further, the memory may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory may also include a memory controller to provide the processor access to the memory.
The electronic device in the embodiments of the present disclosure may be a terminal, a server, or a similar computing device, taking the electronic device as a server as an example, fig. 5 is a block diagram of an electronic device for message Processing according to an exemplary embodiment, and as shown in fig. 5, the server 500 may generate a relatively large difference due to different configurations or performances, and may include one or more Central Processing Units (CPUs) 510 (the processors 510 may include but are not limited to a Processing device such as a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 530 for storing data, and one or more storage media 520 (e.g., one or more mass storage devices) for storing an application 523 or data 522. Memory 530 and storage medium 520 may be, among other things, transient storage or persistent storage. The program stored on the storage medium 520 may include one or more modules, each of which may include a series of instruction operations for the server. Still further, the central processor 510 may be configured to communicate with the storage medium 520 to execute a series of instruction operations in the storage medium 520 on the server 500. The server 500 may also include one or more power supplies 560, one or more wired or wireless network interfaces 550, one or more input-output interfaces 540, and/or one or more operating systems 521, such as Windows Server, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, and the like.
The input/output interface 540 may be used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the server 500. In one example, the input/output Interface 540 includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the input/output interface 540 may be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
It will be understood by those skilled in the art that the structure shown in fig. 5 is only an illustration and is not intended to limit the structure of the electronic device. For example, server 500 may also include more or fewer components than shown in FIG. 5, or have a different configuration than shown in FIG. 5.
In an exemplary embodiment, a computer-readable storage medium comprising instructions, such as the memory 530 comprising instructions, executable by the processor 510 of the apparatus 500 to perform the method described above is also provided. Alternatively, the computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
In an exemplary embodiment, there is also provided a computer program product comprising computer programs/instructions which, when executed by a processor, implement any of the message processing methods provided in the embodiments of the present disclosure.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (10)
1. A message processing method, comprising:
acquiring a target data change message; the target data change message indicates data change of a target service scene;
determining target time information matched with the target service scene in the configuration information; the target time information is used for indicating a storage duration;
determining the storage duration of the target data change message in a cache according to the target time information, and storing the target data change message in the cache;
and executing a reconciliation task on the target data change message in the cache when the task execution condition is determined to be met.
2. The message processing method according to claim 1, wherein the target time information includes a check time length and a storage time length adjustment factor;
the determining the storage duration of the target data change message in the cache according to the target time information includes:
determining the product of the check duration and the storage duration adjustment factor in the target time information;
and taking the product as the storage duration of the target data change message in a cache.
3. The message processing method of claim 2, wherein the method further comprises:
determining a target delay time length corresponding to the target data change message according to the check time length;
generating a target delay message according to the target data change message and the target delay duration, and storing the target delay message to a delay queue;
and when the delay time of the target delay message in the delay queue reaches the target delay time, determining that a task execution condition is met.
4. The message processing method according to any one of claims 1 to 3, characterized in that the method further comprises:
responding to the configuration instruction, and displaying a configuration page; the configuration page comprises a service scene area and a corresponding time information area;
and generating the configuration information according to the input content in the service scene area and the input content in the corresponding time information area.
5. The message processing method according to any one of claims 1 to 3, wherein the performing a reconciliation task on the target data change message in the cache comprises:
searching whether the cache has a to-be-reconciled data variation message corresponding to the target data variation message;
and if the target data change message and the to-be-reconciled data change message are found, carrying out data consistency check on the target data change message and the to-be-reconciled data change message.
6. The message processing method of claim 1, wherein the obtaining the target data change message comprises:
when monitoring a data change message of a target database, storing the data change message to a message queue;
and acquiring the data change message from the message queue, and preprocessing the data change message to obtain the target data change message.
7. A message processing apparatus, comprising:
a target data change message acquisition unit configured to perform acquisition of a target data change message; the target data change message indicates data change of a target service scene;
a target time information determining unit configured to perform determining target time information matching the target service scenario in configuration information; the target time information is used for indicating a storage duration;
the storage unit is configured to determine the storage duration of the target data change message in a cache according to the target time information, and store the target data change message in the cache;
and the account checking task execution unit is configured to execute an account checking task on the target data change message in the cache when the task execution condition is determined to be met.
8. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the message processing method of any of claims 1 to 6.
9. A computer-readable storage medium in which instructions, when executed by a processor of an electronic device, enable the electronic device to perform the message processing method of any of claims 1 to 6.
10. A computer program product comprising computer programs/instructions, characterized in that the computer programs/instructions, when executed by a processor, implement the message processing method of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111341022.0A CN114201501A (en) | 2021-11-12 | 2021-11-12 | A message processing method, device, electronic device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111341022.0A CN114201501A (en) | 2021-11-12 | 2021-11-12 | A message processing method, device, electronic device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114201501A true CN114201501A (en) | 2022-03-18 |
Family
ID=80647545
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111341022.0A Pending CN114201501A (en) | 2021-11-12 | 2021-11-12 | A message processing method, device, electronic device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114201501A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114860476A (en) * | 2022-04-27 | 2022-08-05 | 北京达佳互联信息技术有限公司 | Message processing method, device, server and storage medium |
| CN115292383A (en) * | 2022-07-25 | 2022-11-04 | 浪潮云信息技术股份公司 | Data consistency inspection method and system under distributed scene |
| CN116821188A (en) * | 2023-07-04 | 2023-09-29 | 康键信息技术（深圳）有限公司 | Medical message sending method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017000673A1 (en) * | 2015-06-29 | 2017-01-05 | 深圳市中兴微电子技术有限公司 | Shared cache allocation method and apparatus and computer storage medium |
| CN110719318A (en) * | 2019-09-06 | 2020-01-21 | 上海陆家嘴国际金融资产交易市场股份有限公司 | Message processing method and system |
| CN111694645A (en) * | 2020-05-26 | 2020-09-22 | 平安普惠企业管理有限公司 | Task processing method and related device in distributed task scheduling system |
| CN111861692A (en) * | 2020-07-27 | 2020-10-30 | 深圳乐信软件技术有限公司 | Rules engine-based account reversal method, system, server and storage medium |
| CN113094430A (en) * | 2021-03-25 | 2021-07-09 | 北京达佳互联信息技术有限公司 | Data processing method, device, equipment and storage medium |
| CN113220730A (en) * | 2021-05-28 | 2021-08-06 | 中国农业银行股份有限公司 | Service data processing system |
-
2021
- 2021-11-12 CN CN202111341022.0A patent/CN114201501A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017000673A1 (en) * | 2015-06-29 | 2017-01-05 | 深圳市中兴微电子技术有限公司 | Shared cache allocation method and apparatus and computer storage medium |
| CN110719318A (en) * | 2019-09-06 | 2020-01-21 | 上海陆家嘴国际金融资产交易市场股份有限公司 | Message processing method and system |
| CN111694645A (en) * | 2020-05-26 | 2020-09-22 | 平安普惠企业管理有限公司 | Task processing method and related device in distributed task scheduling system |
| CN111861692A (en) * | 2020-07-27 | 2020-10-30 | 深圳乐信软件技术有限公司 | Rules engine-based account reversal method, system, server and storage medium |
| CN113094430A (en) * | 2021-03-25 | 2021-07-09 | 北京达佳互联信息技术有限公司 | Data processing method, device, equipment and storage medium |
| CN113220730A (en) * | 2021-05-28 | 2021-08-06 | 中国农业银行股份有限公司 | Service data processing system |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114860476A (en) * | 2022-04-27 | 2022-08-05 | 北京达佳互联信息技术有限公司 | Message processing method, device, server and storage medium |
| CN115292383A (en) * | 2022-07-25 | 2022-11-04 | 浪潮云信息技术股份公司 | Data consistency inspection method and system under distributed scene |
| CN116821188A (en) * | 2023-07-04 | 2023-09-29 | 康键信息技术（深圳）有限公司 | Medical message sending method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110321387B (en) | Data synchronization method, equipment and terminal equipment | |
| CN111339073A (en) | Real-time data processing method and device, electronic equipment and readable storage medium | |
| CN114201501A (en) | A message processing method, device, electronic device and storage medium | |
| CN110363663B (en) | Block chain-based data batch processing method, device, equipment and storage medium | |
| CN111400356B (en) | Data query method, device and equipment | |
| CN111177222A (en) | Model testing method and device, computing equipment and storage medium | |
| CN112559525B (en) | Data checking system, method, device and server | |
| CN109740129B (en) | Report generation method, device and equipment based on blockchain and readable storage medium | |
| US12045125B2 (en) | Alert aggregation and health issues processing in a cloud environment | |
| CN113656194A (en) | Account checking result data notification method and device, electronic device and storage medium | |
| CN111090440A (en) | Information processing method, system, device and storage medium | |
| CN114756328A (en) | Container cloud platform inspection method and device | |
| CN112445874A (en) | Node information checking method, device, equipment and storage medium | |
| CN115454420A (en) | Artificial intelligence algorithm model deployment system, method, equipment and storage medium | |
| CN110737655B (en) | Method and device for reporting data | |
| CN110764838B (en) | Service model loading method and system, electronic equipment and storage medium | |
| CN112115016A (en) | Application performance monitoring system | |
| CN107193749B (en) | Test methods, devices and equipment | |
| CN111209333B (en) | Data updating method, device, terminal and storage medium | |
| CN115048413B (en) | Service domain data management method and device | |
| CN117851192A (en) | Application system high availability evaluation method and device | |
| CN110782380A (en) | Aeroderivative information management method, aeroderivative information management device and storage medium | |
| CN117112211A (en) | Task processing method and device, electronic equipment and storage medium | |
| CN115984022A (en) | Unified account checking method and device for distributed payment system | |
| CN112035358A (en) | Method, device and equipment for determining detection information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220318 |
|
| RJ01 | Rejection of invention patent application after publication |