CN120045358A - Method for realizing asynchronous call of interface based on message queue - Google Patents

Abstract

The present application provides a method for implementing asynchronous interface calls based on message queues, which relates to the field of cloud computing technology, including: S1, selecting a message queue product and installing and configuring it; S2, encapsulating the functions of the message queue to provide a unified event publishing and subscription mechanism; S3, defining and generating a synchronous interface through a front-end interface, wherein the synchronous interface is used to receive request parameters and execute corresponding business logic; S4, registering the synchronous interface as an asynchronous interface, and generating a corresponding publishing end and subscriber end of the message queue; wherein the calling process of the asynchronous interface includes: a client request triggers the publishing end of the message queue, the publishing end encapsulates the request parameters as a message body and sends it to the message queue, the subscriber end monitors the message queue and obtains the message body, and calls the synchronous interface to execute business logic.

Description

Method for realizing asynchronous call of interface based on message queue

Technical Field

The application belongs to the technical field of cloud computing, and particularly relates to a method for realizing asynchronous call of an interface based on a message queue.

Background

With the rapid development of cloud computing and distributed systems, modern software is increasingly demanding high-concurrency, low-latency interface calls. In particular, in the fields of electronic commerce, financial payment, real-time data processing, etc., the system needs to quickly respond to massive requests, so as to avoid the blocking problem caused by synchronous call. The asynchronous interface calling technology remarkably improves the throughput and user experience of the system by decoupling time-consuming operation from the main flow, and becomes an important direction of current technical evolution.

Currently, the mainstream asynchronous interface implementation mainly comprises three steps of processing time-consuming operation by creating independent threads and avoiding blocking the main threads. However, thread management is complex, resource consumption is large, and stability problems in high concurrency scenes are difficult to deal with. And delivering the request message to a queue, and asynchronously processing by a background program. Common products such as RabbitMQ, kafka can realize decoupling and peak clipping and valley filling, but a developer is required to manually write the logic of publishing, subscribing and exception handling of the queue, so that the development efficiency is low. Such as the @ Async annotation of the Spring framework, simplifies asynchronous method calls by declaration, but relies on a specific technology stack, lacks flexibility, and lacks uniform interfacing configuration capabilities.

The existing scheme needs deep coding of developers to realize integration of message queues, interface binding and exception handling, and has high technical requirements for the developers. From interface definition to asynchronous logic realization, multiple links are needed to be manually developed, the test and verification period is long, and the service requirement is difficult to quickly respond. The prior art lacks an automatic compensation mechanism for abnormal scenes such as message loss, processing timeout and the like, and needs to additionally develop retry or rollback logic. Synchronous and asynchronous mode switching depends on code modification, cannot be dynamically adjusted through configuration, and is difficult to adapt to changeable business scenes.

Disclosure of Invention

The application provides a method for realizing asynchronous call of an interface based on a message queue, which aims to solve the technical problems of system blocking, response delay and abnormal processing deficiency caused by low development efficiency, weak fault tolerance, high technical threshold and lack of flexible configuration of the traditional synchronous interface call.

The technical scheme adopted by the application is as follows:

The embodiment of the application provides a method for realizing asynchronous call of an interface based on a message queue, which comprises the following steps:

S1, selecting a message queue product and performing installation configuration;

S2, packaging the functions of the message queue and providing a unified event publishing and subscribing mechanism;

s3, defining and generating a synchronous interface through a front-end interface, wherein the synchronous interface is used for receiving request parameters and executing corresponding business logic;

S4, registering the synchronous interface as an asynchronous interface, and generating a corresponding publishing end and a subscribing end of the message queue;

The calling process of the asynchronous interface comprises the steps that a client side requests to trigger the issuing side of the message queue, the issuing side packages the request parameters into a message body and sends the message body to the message queue, and the subscribing side monitors the message queue and acquires the message body and calls the synchronous interface to execute service logic.

According to one embodiment of the application, the message queue product is selected from at least one of RabbitMQ, kafka, rocketMQ, activeMQ or a Redis-based implementation of the message queue.

According to an embodiment of the present application, in the step S2, the function package is implemented using JMS specification or SPRING AMQP framework.

According to an embodiment of the present application, in the step S3, the front end interface configuration includes:

the existing method or RPC service in the system is registered as a synchronous interface, and the request parameters of the interface are processed in a serialization or anti-serialization mode.

According to an embodiment of the present application, in the step S4, when the asynchronous interface is registered, the publishing end and the subscribing end of the message queue are automatically generated, and the publishing end and the subscribing end are both deployed in the same application instance.

According to one embodiment of the application, after the asynchronous interface call is completed, an execution result is returned to the client through a preset callback interface.

According to one embodiment of the application, the subscribing end of the message queue supports compensation or retry operations for the abnormal execution request when the synchronous interface is called.

According to one embodiment of the application, the method further comprises the step of configuring a call mode of the switching interface through the interface to realize dynamic switching between asynchronous call and synchronous call.

According to one embodiment of the application, when the publishing end of the message queue sends the message body, format conversion is performed on the request parameters so as to adapt to the synchronous interface parameter requirements of the subscribing end.

According to one embodiment of the application, the triggering condition for the compensation or retry operation includes a message processing timeout, an interface return error code, or a persisting mechanism of a message queue detecting an outstanding message.

By adopting the technical scheme, the application has the following beneficial effects:

The application supports flexible type selection of various main stream message queues (such as RabbitMQ, kafka and the like), adapts to performance requirements (such as high throughput and low delay) of different service scenes, and improves universality and expansibility of the scheme. The MQ difference of the bottom layer is shielded through standardized interfaces (such as JMS and SPRING AMQP), so that development complexity is reduced, and system compatibility and maintainability are enhanced. The interface configuration replaces manual coding, allows non-technicians to rapidly register the existing method or RPC service as a synchronous interface, greatly reduces the technical threshold and shortens the development period. And a queue component required by asynchronous call is automatically generated, so that one-key switching from a synchronous interface to an asynchronous mode is realized, invasive modification of codes is avoided, and deployment efficiency is improved. Decoupling of the request and the processing is realized through the message queue, the main flow quick response client is ensured, and meanwhile, the system fault tolerance is enhanced by utilizing mechanisms such as persistence, retry and the like of the queue, so that the request is prevented from being lost or repeatedly executed.

The application obviously reduces the development and maintenance cost of the asynchronous interface through interface configuration, standardized encapsulation and automatic generation mechanism, and simultaneously improves the reliability and expansibility of the system under a high concurrency scene by relying on the mature characteristic (such as peak clipping and valley filling and abnormal retry) of the message queue, thereby providing high-efficiency technical support for rapidly responding to the service demand.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:

Fig. 1 is a flow chart of a method for implementing asynchronous call of an interface based on a message queue according to an embodiment of the present application.

Detailed Description

In order to more clearly illustrate the general inventive concept, a detailed description is given below by way of example with reference to the accompanying drawings.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application, but the present application may be practiced in other ways than those described herein, and therefore the scope of the present application is not limited to the specific embodiments disclosed below. It should be noted that, without conflict, embodiments of the present application and features in each embodiment may be combined with each other.

In the present application, unless expressly stated or limited otherwise, a first feature "up" or "down" a second feature may be the first and second features in direct contact, or the first and second features in indirect contact via an intervening medium. In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Example 1

As shown in fig. 1, a method for implementing asynchronous call of an interface based on a message queue includes the following steps:

S1, selecting a message queue product and performing installation configuration.

Specifically, a message queue product is selected that first a selection needs to be made from a variety of message queue products, including RabbitMQ, kafka, rocketMQ, activeMQ or Redis-based implementations of message queues. The selection of an appropriate message queue product may take into account a number of factors such as performance requirements, reliability, extensibility, community support, etc.

Installation configuration, namely, after a message queue product is selected, the message queue product is installed and configured next. This process typically involves environmental preparation to ensure that the server environment meets the operating requirements of the selected message queue product, such as operating system version, java runtime environment (if applicable), network configuration, etc. Installation-the installation process is performed according to official documents or guidelines. Different message queue products have different installations, which may include downloading binary files, installation using a packet manager, or deployment through containerization techniques (e.g., docker). After the installation is completed, the message queue needs to be configured to adapt to the specific application scene. Configuration items may include, but are not limited to, network listening ports, persistent storage settings, cluster mode configuration (if used in a distributed system), security settings (such as user authentication and rights control), and the like. Testing and verification after installation and configuration is completed, a series of tests are typically performed to verify that the message queue is properly installed and that the configuration is valid, such as sending and receiving test messages, checking log output, etc.

For example, first, the most appropriate Message Queue (MQ) product needs to be chosen according to project requirements and application scenarios. Assuming our project is an application that handles large numbers of real-time transaction requests, high throughput, low latency, and hope for better scalability and active community support. Based on these requirements, we can compare several common MQ products:

RabbitMQ is written by Erlang, supports multiple protocols (such as AMQP, XMPP and the like), and has strong concurrency capability, excellent performance, active community and rich management interface.

RocketMQ developed by Aliba, is suitable for the big data field, has good expansibility and high single machine throughput (hundred thousand levels), but the C++ client support is still immature.

Kafka, which is designed specifically for large data, while supporting only the primary MQ functions, performs well in high throughput scenarios with low latency.

ActiveMQ is used as a tablet product, the maturity is high, the documents are more, but the single machine throughput is relatively lowest.

In this example, it may be preferable to choose RabbitMQ or RocketMQ in view of the high throughput and low latency requirements, while also requiring an active community to get better technical support. The final decision to select RabbitMQ may be a hurdle due to RocketMQ's lack of support on the C++ client.

Mounting and configuring RabbitMQ

Environment preparation, namely ensuring that the server operating system meets the requirement of RabbitMQ. For example, if a Linux system is used, it is confirmed that the necessary dependency items, such as the Erlang language runtime environment, have been installed.

Mounting RabbitMQ:

The latest version of the RabbitMQ installation package may be downloaded from an official website or installed using the package manager of the system (e.g., yum or apt-get). In the case of a production environment deployment, it is recommended to consult an official document for a more detailed installation guideline, including how to set up cluster modes to improve availability and fault tolerance.

Configuring RabbitMQ:

The configuration file is modified (usually under/etc/rabbitmq/directory), and the network monitoring port, user authentication, authority control and the like are adjusted according to actual requirements. Persistent storage is configured to ensure that important data is not lost in the event of a failure. Parameters of the message queue, such as message expiration time, maximum queue length, etc., are set according to the needs of the application. Testing and verification after initiation of the RabbitMQ service, it may be verified whether the installation was successful by sending a simple test message to the queue and attempting to receive this message from another terminal. Checking the log output ensures that no error information has occurred, indicating that the rubbi mq has been properly configured and is beginning to operate.

Further, whether the performance indexes such as throughput, delay and the like of the message queue meet the application requirements can be considered. For example, if a large number of real-time transaction requests need to be processed, a high throughput and low latency message queue product, such as RabbitMQ or RocketMQ, should be selected. The message queue is evaluated for data protection capability in the event of a system failure. For application scenarios where missing messages are not allowed, a message queue supporting persistent storage needs to be selected and the persistence mechanism thereof is known. Considering the growing demand of future services, a message queue product is selected that is easily scalable. For example, some products may be more adept at horizontal expansion (e.g., kafka), while others provide a rich set of functionality to accommodate different usage scenarios. Selecting a product that has active communities and technical support helps solve the problem and get best practice advice. At the same time, sufficient official documents and third party courses are also important factors. It is checked whether the selected message queue product supports the programming language and framework used in the project and how difficult it is to integrate with other systems.

S2, packaging the functions of the message queue and providing a unified event publishing and subscribing mechanism.

Specifically, in practical applications, different message queue products (such as RabbitMQ, rocketMQ, etc.) have respective unique APIs and usage patterns. In order to simplify the development process and improve the maintainability and expansibility of the system, the functions of the message queues at the bottom layer need to be abstracted and packaged, and a unified interface is provided for the upper layer business logic. This allows developers to publish and subscribe to events through the same programming model, regardless of the message queue product, without concern for the specific implementation details of the underlying layers.

Event publishing, which refers to sending certain operations or state changes in the form of messages to a message queue for subscription and processing by other system components or services. In this process, it is necessary to define the format of the messages (e.g., JSON or XML) and how to serialize/deserialize the messages. The encapsulated functionality should allow a developer to easily specify information about the Topic (Topic), exchange, etc. of a message without having to go deep into the relevant concepts of a particular message queue product. Event subscription refers to listening for messages in a particular topic or queue and triggering the corresponding processing logic upon receipt of a message. The encapsulated subscription mechanism should be able to automatically handle the receipt, parsing and distribution of messages to the appropriate processor while supporting error handling and retry strategies. For the developer, only the implementation of business logic needs to be concerned.

The mechanism can be realized by adopting the following two main technical means:

JMS (Java MESSAGE SERVICE) specification, a standard API that provides message services for Java applications, defines a generic set of messaging interfaces that allow applications to communicate without relying on specific message queue implementations. Many message middleware implements the JMS specification, so JMS can be utilized to create a cross-platform messaging solution.

SPRING AMQP framework if an ambp protocol compatible message queue (such as rabhitmq) is selected, SPRING AMQP framework can be used to simplify interactions with the message queue. SPRING AMQP provides a high level of abstraction, including components such as template classes (AMQPTEMPLATE), message listening containers (MessageListenerContainer), etc., greatly simplifying the message sending and receiving process.

For example, assume that RabbitMQ has been selected as the message queue product and the installation configuration is completed. It is now necessary to functionally encapsulate it in order to provide a unified event publish and subscribe interface for the business layer.

Packaging target

Abstracting, namely abstracting the characteristics and operation modes of different message queue products so that business logic codes can be independent of specific message queue products. And the message sending and receiving can be completed through simple API call, so that the development complexity is reduced. Maintainability, which is convenient for future expansion or replacement of message queue products, only the encapsulation layer code needs to be modified, and the upper layer application is not affected.

Encapsulation using JMS Specification or SPRING AMQP framework

Example packaging Using SPRING AMQP Frames

A. Configuration class

First, a configuration class is created to set up the basic components of the connection factory and template, which hides specific RabbitMQ connection information (e.g., host address, port number, etc.).

-Creating a Java configuration class for defining connection parameters of the rabitmq, such as host, port, username, password.

Definition ConnectionFactory, which is an entry connected to the rabitmq server.

Definition AMQPTEMPLATE, which is the core interface for sending and receiving messages.

B. Message sender

Next, a message sender service is defined that uses previously configured AMQPTEMPLATE to send messages. Here we reduce the operation of sending the message to a method call hiding specific interaction details of the rabitmq.

-Defining a MESSAGESENDER class containing a sendMessage method.

In the sendMessage method, a convertAndSend method of AMQPTEMPLATE is used to send messages to the designated switches and routing keys.

C. Message monitor

A message listener is then defined to subscribe to messages of a particular topic. This listener is responsible for receiving messages and performing the corresponding business logic processing.

Defining a MESSAGELISTENER class, implementing the MESSAGELISTENER interface.

-Logic for processing the received message is written in the onMessage method.

Configuration MessageListenerContainer, associate it with MESSAGELISTENER, and specify the queue name of the snoop.

Unified event publishing and subscribing mechanism

After the encapsulation, the service layer can use the message queue in the following way:

The event release is that only a message sender.sendmessage (String routingKey, object message) method is required to be called, and the proper switch name, routing key and message content are input. The method automatically processes the message format conversion and transmission process. Event subscription by implementing the onMessage (Message message) method in MESSAGELISTENER, when a message arrives in the listening queue, the system will automatically invoke the method and pass the message object to it. The developer only has to pay attention to how these messages are handled. The encapsulation method not only simplifies the use of the message queue, but also provides a good abstraction layer, so that the subsequent replacement of the message queue product or adjustment of the message processing logic can be easily completed without changing a large number of service codes. For example, if we decided to switch from RabbitMQ to Kafka, we only need to adjust the implementation of the encapsulation layer, without modifying any service code using the message queue.

Further, differences between different message queue products may also be hidden by defining a set of high-level APIs. These APIs should cover basic messaging (publishing), receiving (subscribing) and related management operations (e.g., create/delete queues). For example, sendMessage and subscribeMessage methods may be defined so that developers need only invoke these methods to accomplish message interactions without concern for specific message queue implementations. In addition to API-level abstractions, configuration parameters also need to be abstracted. This means that the developer can specify various settings of the message queues by a unified configuration (such as a property file or environment variable) rather than directly process configuration items specific to each message queue. This has the advantage that it is easier to switch between different message queues, requiring only configuration changes without having to modify the code.

Since different message queues may have different exception types, a unified exception handling mechanism should be provided in the encapsulation layer. This includes capturing exceptions thrown from the underlying message queue and converting it to a generic exception type for processing by the upper layer applications. In publishing messages, consideration needs to be given to how to convert the traffic data into a message format suitable for transmission. Typically, JSON or XML is a common choice. The encapsulation layer should support flexible message formatting policies, allowing the developer to select the appropriate serialization scheme as desired. To enhance flexibility, the encapsulation layer should support a topic or routing key based message distribution mechanism. This means that when a message is published, it can be decided which subscribers should receive the message according to their topic or routing key.

Subscribers should be able to process received messages in an asynchronous manner so as not to block the main thread. The encapsulation layer needs to ensure this and preferably provides a callback mechanism to allow the developer to perform certain operations after the message processing is complete. Many message queues support message acknowledgment mechanisms, i.e., a message is removed from the queue only after the consumer has successfully processed the message. The encapsulation layer should simplify this process, enabling the developer to easily enable or disable the message validation function. For messages that fail to process, the encapsulation layer should provide a retry mechanism. This may include policies such as a fixed number of retries, an exponential backoff algorithm, etc., ensuring that successful processing of the message is guaranteed as much as possible even if a temporary failure is encountered. Whichever message queues, the interfaces provided should be consistent. For example, all messaging operations follow the same pattern, as do all message receiving operations. This helps to reduce learning costs and improves portability of code. The design of the encapsulation layer should also take into account integration issues with other frameworks or libraries. For example, if the project uses a Spring framework, the encapsulation layer should be as ecologically compatible as possible with Spring to facilitate integration of functions such as dependent injection.

And S3, defining and generating a synchronous interface through a front-end interface, wherein the synchronous interface is used for receiving the request parameters and executing corresponding service logic.

In particular, the step involves developing a user-friendly front-end interface that allows a user to define the interface through a wizard-type configuration. This interface typically contains a series of forms, options, and input boxes through which the user can specify various properties of the interface, such as name, path, request mode (GET/POST, etc.), request parameter type, etc. Method selection and registration the user can select existing methods or RPC services within the system on the interface and register them as an accessible interface. This step may include selecting a particular method name, specifying the class or module in which the method is located, and determining the input-output parameter format of the method. Request parameter handling-for a selected method or service, it is necessary to define how to handle the request parameters from the client. This includes serialization of parameters (when a client sends a request) and de-serialization (when a server receives a request). For example, if the client sends data in JSON format, it needs to be ensured that these data can be correctly parsed and mapped onto the corresponding parameters of the method. Automatically generating codes, namely automatically generating corresponding interface codes by a system based on the configuration of a user. This is typically done by the back-end, which generates one or more files that implement the interface functions, including receiving request parameters, invoking corresponding business logic methods, and returning the results to the client, based on the information provided by the front-end. Interface deployment-the generated interface will be deployed into the application as a service endpoint for external invocation. This means that once the configuration is completed and the deployment is successful, any HTTP request conforming to the preset rules can trigger this interface, and thus execute the business logic behind it. Synchronous execution as mentioned in the description, a "synchronous interface is used to receive request parameters and execute corresponding business logic", which means that when a client initiates a request, the interface will immediately start processing the request and will not respond to other requests until processing is completed. Only after all business logic is executed, the result is returned to the client. The mechanism greatly simplifies the creation flow of the interface, reduces the technical threshold and enables non-professional programmers to participate in the design and implementation process of the interface. Meanwhile, the development efficiency is improved, and the time period from the requirement proposal to the functional online is shortened.

For example, assume an electronic commerce platform is being developed that requires an interface to be provided for an external system to query for inventory information. In this example, it will be shown how such a synchronization interface is defined and generated by the front-end interface.

The existing method or service is selected-in the e-commerce platform, there is a method named getProductStock which accepts a product ID as an input parameter and returns the stock quantity of the product. This method now needs to be registered as an HTTP interface.

Configuring interface attributes:

Interface name may be named as/api/v 1/getProductStock

HTTP method the GET method is selected because this is a query operation.

Request parameters specify the product ID parameters (e.g., productId) that need to be received from the client and set its type as a string. Response format-data format, such as JSON format, for determining interface response, contains field productId, stockQuantity, etc.

Serialization and deserialization, namely, due to the fact that JSON is selected as a data exchange format, configuration on a front-end interface is needed to convert JSON objects sent by a client into Java objects (productId) and convert Java objects of a server back into the JSON format to return to the client.

Automatically generating codes, namely automatically generating corresponding controller codes by the background according to the configuration made on the front-end interface. This piece of code will perform the following functions:

Snoop GET requests from/api/v 1/getProductStock paths.

ProductId parameters are extracted from the request.

The getProductStock method is invoked and passed in to the extracted productId.

And packaging the returned result of the getProductStock method into a JSON format and returning the JSON format to the client.

Example automatically generated pseudocode:

and (3) front-end verification, namely after configuration is completed, directly calling a newly created interface through a preview or test function provided by the front end, and checking whether an expected result can be correctly obtained. For example, a known product ID is entered to see if the returned inventory quantity is correct.

And (3) integrated testing, namely ensuring that the interface can work normally with other parts of the system, such as the parts interacting with the database, and ensuring the consistency and accuracy of data.

Furthermore, an intuitive and guided user interface can be provided, so that a user can easily complete interface definition. This includes selecting a method or service to expose, specifying an interface path, HTTP methods (GET, POST, etc.), request parameter formats and types, and so forth. The signature based on the selected method automatically populates the necessary entries such as parameter names, types (strings, integers, boolean values, etc.), and whether or not it is a necessary parameter. For parameters of complex object types, the definition of nested structures is supported.

Allowing users to preview the expected response formats, such as JSON structures, during configuration, helps them understand better the behavior of the interface. Built-in support is provided for converting request data sent by the client into the type of parameters required by the server-side method and vice versa. For example, converting the JSON formatted data into Java objects, or converting Java objects back into JSON format for return to the client. The user is allowed to set verification rules, such as length limits, numerical ranges, regular expression matching, etc., for each parameter, ensuring that the incoming data meets the expected requirements.

Template-driven code generation-standardized controller code templates are automatically generated according to user configuration, and the templates follow RESTful principle and are easy to understand and maintain. A default error handling mechanism is provided for the generated interface while allowing the user to customize specific exception handling logic, such as return custom error messages or status codes.

A simple tool or environment is provided that allows a developer to quickly test a newly created interface without leaving the current interface. This may include modeling different types of requests (success cases, failure cases) and viewing detailed response information. The log function is integrated, so that a developer can conveniently track the history record of interface call, analyze performance bottleneck or troubleshoot problems.

In support of interface version management, when an interface needs to be updated, a new version can be created without affecting existing applications that rely on the interface. And automatically generating an API document according to the interface definition, wherein the API document comprises contents such as request examples, parameter descriptions, response details and the like, and is convenient for other developers to review.

S4, registering the synchronous interface as an asynchronous interface, and generating a corresponding publishing end and a subscribing end of the message queue.

Specifically, a synchronous interface is selected, and first, an interface which needs to be converted into asynchronous processing is selected from the existing synchronous interfaces. For example, in the above-mentioned example, we have a synchronized interface/api/v 1/getProductStock that queries the inventory of goods.

And configuring asynchronous attribute, namely adding relevant configuration of asynchronous processing for the selected synchronous interface on the front-end interface. This may include, but is not limited to, specifying the type of message queue to use (e.g., rabbitMQ, kafka, etc.), setting the message body format, defining callback mechanisms, etc.

Creating a publishing end, namely automatically generating a message queue publishing end associated with the synchronous interface by the system after the synchronous interface is registered as an asynchronous interface. This publisher is responsible for receiving requests from clients and encapsulating the request parameters into message bodies for sending into message queues.

Message formatting, namely, carrying out proper serialization operation (such as conversion into JSON format) on request parameters according to configuration, so as to ensure that the message can be correctly processed by a message queue.

Creating a subscription end, and generating a subscription end of a message queue. The task of the subscriber is to monitor and acquire the published messages from the message queue, then decode the messages, and call the corresponding business logic method to execute the actual processing work.

Executing business logic once the subscriber receives the message and completes the de-serialization process, it invokes the actual business logic behind the original synchronization interface (e.g., invokes getProductStock method to query inventory information). It should be noted here that, due to asynchronous processing, the subscriber end does not directly return the result to the client after processing the request, but may notify the client of the processing result through a preset callback interface.

Compensation or retry mechanism in order to ensure reliability of service, if errors (such as network failure, data format mismatch, etc.) are encountered during processing at the subscriber end, a certain compensation or retry policy may be set. For example, if the message processing times out or the interface returns an error code, a retry mechanism is triggered to retry processing the message.

Persistent storage-for critical traffic scenarios, the persistence characteristics of the message queues may also be exploited to ensure that outstanding messages are not lost even in the event of a system crash.

In this way, the synchronous interface is registered as an asynchronous interface, and the publishing end and the subscribing end of the corresponding message queue are generated, so that the conversion of the interface processing mode is realized, the concurrency and the response speed of the system are enhanced, and the method is particularly suitable for operations with long time consumption or application scenes requiring high availability. In addition, the design is convenient for later maintenance and expansion, and the asynchronous processing flow can be flexibly adjusted according to the requirements.

For example, the previously defined synchronous interface/api/v 1/getProductStock is selected in the front-end interface and configured to become an asynchronous interface. This step may include:

Selecting a message queue product-selecting an appropriate message queue product from the provided options (e.g., rabbitMQ, kafka, etc.). In this example, we select RabbitMQ as the message queue.

A callback mechanism is provided defining how to notify the client when the asynchronous process is completed. May be an HTTP callback URL or other notification means.

The message format is determined, for example, by deciding to use JSON as the data format for message delivery.

Once the synchronous interface is successfully registered as an asynchronous interface, the system automatically generates a publisher responsible for receiving requests from clients and converting those requests into messages for transmission to a message queue.

When a client initiates a GET request to/api/v 1/getProductStock interface, instead of directly invoking business logic, the request parameters (e.g., productId) are encapsulated into a message body.

The message is sent to the designated message queue using the API of RabbitMQ or a corresponding framework (e.g., SPRING AMQP). At this point, the client immediately receives a response informing that the request has been accepted (but not yet processed).

At the same time, the system will also generate a subscriber that will continually monitor the designated message queue and trigger the corresponding processing logic when a new message is detected.

The subscriber monitors the specific queue and waits for the message containing the commodity ID to arrive.

When the message arrives, the subscriber decodes the message body, extracts the commodity ID, and invokes the original business logic method getProductStock (productId) to obtain inventory information.

After the processing is finished, if a callback mechanism is configured, notifying the client of the processing result in a preset mode (such as HTTP POST to callback URL).

The following is an example for illustrating the above procedure:

further, a user may specify which synchronous interfaces need to be converted to asynchronous interfaces through a wizard configuration tool provided by the front end interface. This includes, but is not limited to, interface name, request method type (GET, POST, etc.), input parameter definition, and output result format. The system automatically adjusts the behavior mode of the interface according to the configuration of the user, changes from direct processing and immediate response to receiving the request, immediately transfers the request to a message queue for processing, and immediately returns an acknowledgement response to the client.

At the publishing end, the requested data from the client needs to be properly encapsulated. This means that the request parameters are serialized according to a preset data format (e.g. JSON) to form the message body. At the same time, additional metadata, such as time stamps or unique identifiers, may also need to be added to facilitate subsequent tracking and management. It is determined how the message is to be sent to the message queue. Different switch types (direct, topic, headers, fanout) may be selected to accommodate different message routing requirements. In addition, the persistence attribute of the message can be set to ensure that the message is not lost even if the server is restarted.

The subscriber is responsible for listening for messages in a particular queue. Once a new message arrives, the subscriber will fetch and de-sequence it into the original request parameter format, and then call the corresponding business logic method to perform the actual operation. In the message processing process, if an abnormal condition (such as network failure, unreachable database, etc.) is encountered, the subscription terminal should have a certain error processing capability. This typically involves implementing a retry strategy (e.g., an exponential backoff algorithm), logging for subsequent analysis, and even in extreme cases triggering a compensating transaction to ensure data consistency.

A unified management interface is provided that allows an administrator to view all registered asynchronous interfaces and their associated configurations (e.g., message queues used, message processing status, etc.). This helps to simplify maintenance work and improve manageability of the system. The integrated monitoring function tracks the state of the message queue (such as queue length, message processing speed, etc.) in real time, and issues an alarm (such as message backlog exceeding a threshold, processing failure rate being too high, etc.) when an abnormal situation occurs.

The calling process of the asynchronous interface comprises the steps that a client side requests to trigger the issuing side of the message queue, the issuing side packages the request parameters into a message body and sends the message body to the message queue, and the subscribing side monitors the message queue and acquires the message body and calls the synchronous interface to execute service logic.

Specifically, the client initiates a request that when the client (e.g., a mobile application or Web front end) needs to query for certain information or submit data, it sends an HTTP request to the server to a particular API endpoint (e.g.,/API/v 1/getProductStock).

Trigger publishing side unlike conventional synchronous interfaces, the API endpoint here is configured in asynchronous mode. Therefore, when the server receives the request of the client, the server does not immediately start to process the service logic, but firstly packages the request parameters into a message body, and then sends the message body through the issuing end of the message queue.

The package request parameters are that at this stage, the system will sequence all necessary information (such as commodity ID, user ID, etc.) from the client, and generally convert it into JSON or other data format that is easy to transmit. This information constitutes the main content of the message body.

Send message to queue Next, the publisher sends the encapsulated message to the specified message queue using the API or related framework provided by the message queue (e.g., SPRING AMQP for RabbitMQ). At this point, the client may immediately receive an acknowledge response indicating that the request was received and being processed, but that the specific business logic has not yet been completed.

And the subscriber terminal continuously monitors, namely, at the same time, the preconfigured subscriber terminal (consumer) continuously monitors the appointed message queue and waits for the arrival of new messages. Once a new message is found to enter the queue, the subscriber will fetch it.

And (3) deserializing the message body, namely after the message is taken out, the subscriber terminal can perform deserializing operation on the message body and restore the original request parameters. This step ensures that the corresponding business logic method can be correctly invoked later.

Invoking the synchronization interface-based on the deserialized request parameters, the subscriber invokes business logic behind the originally defined synchronization interface (e.g., invokes getProductStock (productId) to query inventory information). This step is actually performed asynchronously in the background, and the interaction with the client has been completed.

And after the business logic processing is finished, if a callback mechanism is configured before, the processing result of the client can be notified through a preset mode (such as HTTP POST to callback URL provided by the client). If no callbacks are set, the client may be made aware of the final result in dependence on other means (e.g., polling).

For example, assume that there is an e-commerce platform that provides an API/API/v1/getProductStock to query inventory information. Initially, this was a synchronous interface, but in high concurrency cases, processing each request directly may lead to performance bottlenecks. Therefore, it is decided to convert it into an asynchronous interface based on a RabbitMQ implementation.

The client initiates a request that a user sends an HTTP GET request to the server through the mobile application or the Web front end to query the product inventory with the product ID of 12345 with/api/v 1/getProductStockproductId =12345.

Triggering the issuing end, namely after receiving the request, the system does not process the business logic immediately, but packages the request parameters (productId =12345 in this case) into a message body through the configured issuing end and sends the message body to a message queue of the RabbitMQ. The client will immediately receive a response informing that the request has been accepted and is in process.

Encapsulation request parameters at the publishing end, the system converts productId =12345 to a message body in JSON format, e.g., { "productId": "12345" }.

Send a message to a queue and then send the message to a designated message queue, such as the queue named productStockQueue, using an API provided by rubbi mq or SPRING AMQP framework.

{

"productId":"12345"

}

The subscriber end continuously listens to the productStockQueue queue, and the preconfigured subscriber end (consumer) continuously listens to the productStockQueue queue. Once a new message arrives, it will be fetched from the queue.

And (3) deserializing the message body, namely after the message is taken out, the subscriber terminal can perform deserializing operation on the message body, and the original request parameter productId =12345 is recovered.

And calling a synchronous interface, namely calling a business logic method getProductStock (productId) behind the initially defined synchronous interface by the subscriber terminal to inquire inventory information according to the deserialized request parameters. This step is performed asynchronously in the background, and the interaction with the client has been completed.

And processing the result, namely assuming that the query result shows that the current inventory of the commodity is 30. If the callback mechanism is configured before, the final processing result of the client can be notified through a preset mode (such as HTTP POST to callback URL provided by the client).

{

"productId":"12345",

"stockQuantity":30

}

If the callback mechanism is not set, the client may need to acquire the final processing result by polling or other means.

Further, the server may also first receive a request when the client initiates the request (e.g., via HTTP GET or POST). At this point, the system recognizes that this is a request that requires asynchronous processing. The system will extract the necessary parameters (e.g., merchandise ID, user information, etc.) from the request. These parameters are the basis for subsequent business logic processing. Upon receipt of the request, the server returns an acknowledge response to the client informing that the request has been accepted and is in process. This immediate feedback improves the user experience and avoids long waiting times.

Depending on the configuration, the request parameters are converted into a suitable message format (typically JSON). This step may involve serialization operations to ensure that data can be efficiently transferred between the different components. In addition to the request parameters, additional information may be added to the message, such as a timestamp, message ID, priority, etc., to facilitate subsequent tracking and management. Appropriate messaging policies (direct, topic, fan-out, etc.) are selected to accommodate different routing requirements. In addition, the persistence setting of the message needs to be considered, so that important information is not lost even if the system fails.

The subscriber (consumer) continues to listen to the designated message queue waiting for new messages to arrive. Such listening mechanisms are typically event driven and are capable of responding to new tasks in real time. Once a new message arrives, the subscriber will take it out and perform the deserialization operation to recover the original request parameters. This step is a precondition for accurate execution of the business logic. If problems (such as incorrect message formats) are encountered in the acquisition or reverse serialization process, the subscriber terminal should have basic error processing capability, log or take appropriate remedial action to prevent system crashes.

And calling a business logic method behind the original synchronous interface by using the recovered request parameters. This step is actually performed asynchronously in the background, independent of the interactive flow of the client. After the business logic is executed, the processing result can be further operated according to a preset rule. For example, if the operation is a query type operation, the result can be directly stored or notified to the client through a callback, and if the operation is an update type operation, the consistency and the integrity of the data need to be ensured. Considering the complexity of business logic, which may involve multiple steps or interactions between services, there is a need for an efficient transaction management mechanism to guarantee the atomicity, consistency, isolation, and durability (ACID characteristics) of the overall process.

In some embodiments of the application, the message queue product is selected from at least one of RabbitMQ, kafka, rocketMQ, activeMQ or a Redis-based implementation of the message queue.

Specifically, rabbitMQ

Background, written by the Erlang language, supports multiple protocols (AMQP, MQTT, etc.), and is widely used open source message broker software.

The method has the advantages of high concurrency capability, and high efficiency in processing a large number of concurrent connections due to the use of the concurrency model of Erlang. Easy to deploy and manage, providing rich plug-in systems and user friendly management interfaces. Flexibility-support of multiple message routing mechanisms (e.g., direct, topic, fan-out, etc.). The application scene is suitable for application scenes needing flexible routing information.

Kafka

Background-originally developed by LinkedIn, now an open source stream processing platform under the Apache item.

The method has the advantages of high throughput, and is particularly suitable for processing large-scale data streams, such as log aggregation, real-time analysis and the like. Durability and reliability, namely ensuring the reliability and the persistent storage of the message in a distributed commit log mode. The method has strong expansibility, supports horizontal expansion and is very suitable for big data environments. The method is suitable for scenes which are most suitable for scenes which need to process mass data streams, such as real-time data analysis, monitoring systems and the like.

RocketMQ

Background-developed by Aliba and contributing to the Apache foundation, is a low-latency, highly reliable distributed messaging middleware.

The method has the advantages of high throughput, optimization for a financial transaction system in design, and support for high concurrency message processing. And the system has strong expansibility, good horizontal expansibility and is convenient for coping with service growth. Transaction support, namely providing perfect distributed transaction support and ensuring the consistency and the integrity of the message. The method is suitable for application scenes with extremely high performance requirements and strong consistency, such as an e-commerce transaction system.

ActiveMQ

The message middleware of the old brand is maintained by the Apache foundation and supports a plurality of message protocols.

The method has the advantages of maturity, stability, long development history, complete documents and active communities. The multi-protocol supports a plurality of protocols such as STOMP, AMQP and the like in addition to the standard JMS. Easy to integrate, and can be easily integrated with frames such as Spring. The method is applicable to traditional enterprise-level applications which need to be compatible with multiple protocols.

Message queue based on Redis implementation

Background while Redis is primarily used as an in-memory database, it also provides simple message queuing functions (e.g., publish/subscribe mode).

The memory has the advantages of high performance, extremely high read-write speed when being used as a memory database. Is simple and easy to use and is very convenient for small applications that do not require complex message routing logic. Lightweight-no additional installation of special message queue services is required. The method is suitable for scenes with low requirements on message transmission and pursuing rapid deployment and simple use.

In some embodiments of the present application, in the step S2, the function package is implemented using JMS specification or SPRING AMQP framework.

Specifically, java MESSAGE SERVICE (JMS) is a standard API that provides messaging services for Java applications, and defines a set of generic messaging interfaces that allow applications to communicate without depending on the specific message queue implementation. Event publishing-through JMS APIs, the developer can easily send business data as messages to a specified Topic (Topic) or Queue (Queue). For example TopicPublisher or QueueSender may be used to publish messages. Event subscription-likewise TopicSubscriber or QueueReceiver are used to subscribe to and receive messages. JMS supports both point-to-point (Queue) and publish/subscribe (Topic) modes. Transaction management-JMS provides support for message passing transactions, allowing developers to send or receive multiple messages in one transaction context, ensuring message consistency and reliability. Standardization because JMS is a standard, different message queue products (such as ActiveMQ, rabbitMQ and the like) can be switched seamlessly without modifying service codes as long as JMS APIs are realized. The JMS has the advantages of mature stability, mature community support and abundant document resources for many years.

SPRING AMQP is a high-level abstract framework specifically designed for the AMQP Protocol (ADVANCED MESSAGE ringing Protocol) that is part of the Spring program. It simplifies the use of message queues based on the AMQP protocol (e.g., rabhitmq).

AMQPTEMPLATE this is one of the core classes provided by SPRING AMQP for simplifying the sending and receiving operations of messages. The developer can send a message through convertAndSend () method of AMQPTEMPLATE and receive a message using receiveAndConvert () method.

Message listening container SPRING AMQP provides MessageListenerContainer that automatically listens to messages in a specified queue and invokes a corresponding processor (MESSAGELISTENER) to process the message when it arrives.

The integrated Spring ecology SPRING AMQP is deeply integrated with a Spring frame, can work well in cooperation with other Spring components (such as Spring Boot and SPRING DATA) and provides more powerful functions and services. The usability is SPRING AMQP, the operation of the AMQP protocol is greatly simplified, and the workload of directly operating the low-level API by a developer is reduced. The flexibility is that a plurality of configuration options are supported, and the information transmission behaviors, such as an information confirmation mechanism, a retry strategy and the like, can be adjusted according to actual requirements. The strong ecological system benefits from the strong ecological system of Spring, SPRING AMQP can be conveniently integrated with other Spring technical stacks, and complex distributed application is constructed.

Whether JMS specification or SPRING AMQP framework is chosen, the purpose is to provide a unified interface layer, hiding the specific implementation details of the underlying message queues. Benefits of this include, but are not limited to, reduced coupling, business logic no longer depends directly on the particular message queue product, improving portability and maintainability of the system. Development efficiency is improved by the developer only needing to pay attention to how to send and receive messages using the high-level APIs, and not needing to go deep into the unique characteristics of each message queue. The system stability is enhanced, and the robustness and fault tolerance of the system can be effectively improved through a reasonable exception handling and retry mechanism.

In some embodiments of the present application, in the step S3, the front end interface configuration includes:

the existing method or RPC service in the system is registered as a synchronous interface, and the request parameters of the interface are processed in a serialization or anti-serialization mode.

In particular, the visual configuration provides a user-friendly front-end interface that allows a developer or system administrator to define and register new interfaces through a graphical wizard-type process. This interface typically contains input boxes, drop-down menus, etc. for specifying basic properties of the interface (e.g., name, path, HTTP method type, etc.). Selecting an existing method or RPC service the user can select a service or method from a list of existing methods within the system that needs to be disclosed as an interface. These methods may be either pre-existing business logic functions or Remote Procedure Call (RPC) services. Defining interface properties-in addition to selecting a specific method, some basic properties of the interface need to be defined, such as:

Interface path, e.g./api/v 1/getProductStock.

HTTP method, GET, POST, etc.

Request parameters-parameters defining which parameters are necessary, what their data type is (string, integer, boolean, etc.), and whether nested object types are supported.

Serializing or deserializing request parameters of an interface

When a client initiates a request, the data format it sends may not be the format directly applicable to the server-side method call. Therefore, it is necessary to convert the data of the client (typically JSON or XML format) into a format (such as Java object) suitable for the server side method. This step is called serialization. Reverse serialization-conversely, after the server performs the operation, the result returned to the client needs to be converted from the data structure (e.g., java object) at the server back to a format (e.g., JSON) that the client can understand. This step is called deserialization. Automated tools support modern frameworks and libraries typically provide automated serialization/deserialization tools. For example, the Jackson library in the Spring Boot may automatically handle the conversion of JSON to Java objects. Custom serializers and deserializers-for some special cases, custom serializers and deserializers may need to be written to meet specific needs. For example, if there is a complex nested object structure, or some fields require special coding, the conversion logic can be customized by implementing the corresponding interfaces. Error handling-various problems may be encountered during serialization and de-serialization, such as data format mismatch, lack of necessary fields, etc. A good design should contain appropriate error handling mechanisms to ensure that these problems can be captured in time and notified to the user or system administrator in a friendly manner.

In some embodiments of the present application, in the step S4, when the asynchronous interface is registered, a publishing end and a subscribing end of the message queue are automatically generated, and the publishing end and the subscribing end are both deployed in the same application instance.

Specifically, the automatic generation of a publishing end and a subscribing end

The publisher is responsible for receiving requests from clients and encapsulating the requests into messages for transmission into a message queue. When a synchronous interface is registered as an asynchronous interface, the system automatically generates post logic based on configuration information (e.g., selected message queue products, request parameter formats, etc.). Tasks that the publisher needs to handle include, but are not limited to, serializing request parameters (e.g., converting Java objects into JSON or XML format), constructing message bodies, selecting the appropriate message queue switch type, and eventually sending the message to the specified message queue.

The subscriber is responsible for listening to a particular message queue and once a new message arrives, it fetches the message from the queue and processes it. Similarly, the subscriber terminal is automatically generated based on preset rules. Its main responsibility is to deserialize the received message (i.e. convert the message body back to the original request parameter format) and then invoke the corresponding business logic method to perform the actual operation. In addition, the subscriber end needs to manage the retry mechanism, logging, etc. in the case of an exception to ensure that the message can be reliably processed.

The integration level is high, and complexity and potential delay caused by cross-service calling can be reduced by deploying the publishing terminal and the subscribing terminal in the same application instance. This approach is particularly suitable for those application scenarios where it is desirable to maintain a high degree of internal integration. Simplified management and maintenance, since the publishing and subscribing ends are located in the same application instance, it makes it easier for them to share resources (e.g., database connection pool, cache, etc.), while also simplifying monitoring and troubleshooting. The developer need only be concerned with the state of a single application. And the method is convenient to expand, and although the publishing end and the subscribing end both operate in the same application instance, the design should consider that the horizontal expansion is supported. This means that if the subsequent traffic increases, the load can be dispersed by adding more identical application instances without significant modification to the existing architecture.

Assuming that a commodity inventory query interface/api/v 1/getProductStock in an e-commerce platform is provided, we now turn it into an asynchronous interface:

Registering asynchronous interface, selecting the interface by front end interface tool, setting to asynchronous mode, selecting RabbitMQ as message queue product.

Generating a release end, namely automatically generating release end logic by a system, and transmitting a message which is responsible for receiving a client request and packaging parameters such as productId and the like into a JSON format to a queue named productStockQueue. Generating the subscription end, namely generating the logic of the subscription end at the same time, and continuously monitoring productStockQueue. When a new inventory inquiry request comes, the subscriber end reads the message content, deserializes the message content to obtain productId, and then calls the original business logic method to inquire the inventory information. The configuration, the dependency library and other resources can be conveniently shared, and the unified management and the optimization of the performance are also convenient.

In some embodiments of the present application, after the asynchronous interface call is completed, an execution result is returned to the client through a preset callback interface.

Specifically, callback mechanism after asynchronous interface call completion

First, review the basic workflow of an asynchronous interface:

The client initiates a request that the client sends to the server, the request is converted into a message and sent to the message queue. The issuing end processes that the issuing end of the server encapsulates the request parameters into a message body and sends the message body to a designated message queue. And the subscriber terminal processes that the subscriber terminal monitors the message queue, receives the message, performs deserialization processing, and calls a corresponding service logic method to execute actual operation. In this process, the client does not get the processing result immediately, but receives an acknowledgement that the request has been accepted and is in process.

Because of the asynchronous processing, the client cannot directly obtain the response result as a synchronous interface. In order for the client to learn the final result, a callback mechanism is typically employed. The "preset callback interface" referred to herein refers to a URL or other form of notification provided by the client to the server upon initial request. When asynchronous processing is complete, the server will use this callback interface to inform the client of the processing results.

Specific realization of callback interface

The client provides a callback address, namely, when an asynchronous request is initiated, the client needs to provide a callback address (such as HTTP URL) for receiving a processing result besides transmitting necessary service parameters.

The preparation of results is performed once the subscribing end has completed the business logic process and has obtained the results (e.g., the inventory of the queried commodity) which need to be organized into the appropriate data format (typically JSON or XML). And calling a callback interface, namely, the server can send the processing result back to the client through methods such as HTTP POST and the like by using the callback address provided before. This step may involve a serialization operation of the result data.

Error handling-if a problem is encountered (e.g., network failure, target is not reachable, etc.) when invoking the callback interface, a certain retry policy may be set. For example, an exponential backoff algorithm is used to attempt to retransmit the result until success or a maximum number of retries is reached. Logging-for each callback attempt and its result (success or failure), a detailed logging should be made for subsequent analysis and troubleshooting.

And receiving a result that the client needs to have a service to monitor callback requests from the server and analyze and process the received data. Updating the UI or triggering subsequent operations, namely, according to callback results, the client can update the user interface to display the latest information or trigger other related business logic operations.

In some embodiments of the present application, the subscription end of the message queue supports compensation or retry operations for executing abnormal requests when the synchronization interface is invoked.

In particular, in a distributed system, some requests may not be completed normally due to network fluctuations, temporary unavailability of services, etc. By means of the compensation or retry mechanism, the reliability and robustness of the system can be improved remarkably, and important service data cannot be lost even if temporary problems occur. For some application scenarios (e.g., financial transactions, order processing, etc.) where strong consistency is required, it is important to ensure that all requests are properly processed. The compensation and retry mechanism helps to maintain data consistency, preventing data inconsistency problems due to partial failures. Error capturing, namely, when the subscribing terminal calls the synchronous interface, the subscribing terminal should have a perfect exception capturing mechanism, and can identify various types of exception conditions. This includes, but is not limited to, network timeouts, HTTP error codes (e.g., 500 internal server errors), database connection failures, and the like. Logging-detailed error information should be recorded for each failed attempt, including a time stamp, request ID, specific error description, etc. Such log information is useful for subsequent problem investigation and analysis.

Compensation transactions in some cases a simple retry may not be sufficient to solve the problem, especially when multiple steps of operations are involved (e.g., transfer operations involving both deduction and deposit steps). At this point, the compensation transaction may be designed to cancel some of the previous operations, and then decide whether to retry the entire process according to circumstances. Manual intervention-for some complex anomalies, manual intervention may be required to compensate. For example, after checking the system status, repairing a potential problem, a retry is manually triggered or the data is manually corrected. Fixed number of retries, i.e. a fixed number of retries is set, such as a maximum of 3 retries. If this number of times is exceeded, the request is considered to be unable to continue processing, is typically marked as failed, and the relevant personnel is notified for further processing. Exponential backoff algorithm-in order to avoid frequently sending requests to services that have been overloaded, an exponential backoff algorithm may be employed to control the time interval of the retries. For example, the first retry interval is 1 second, the second retry interval is 2 seconds, the third retry interval is 4 seconds, and so on. Based on conditional retry, different retry strategies may be selected according to different error types. For example, a shorter retry interval may be set for network timeout class errors, while for service unavailability class errors may wait longer before retrying. Persisting incomplete tasks to prevent a system crash from causing a retry task in progress to be lost, the incomplete tasks may be persisted to a database or other persistent storage medium. Thus, even after the system is restarted, the processing can be continued from the interrupt. Idempotent design-considering that the same request may be retried multiple times, it must be ensured that the business logic is idempotent, i.e., the result is the same no matter how many times it is repeatedly performed. This is important to avoid the problem of data inconsistency caused by repeated processing of the same request.

In some embodiments of the present application, the method further includes configuring a call mode of the switching interface through the interface, so as to realize dynamic switching between asynchronous call and synchronous call.

Specifically, the graphical interface is that by providing an intuitive and graphical interface, a user can conveniently check the calling modes (synchronous or asynchronous) of all interfaces in the current system and can directly modify the interface. The operation flow is simplified, the switching of the interface calling mode can be completed without going deep into the code layer, and the operation difficulty and the technical threshold are reduced.

Instant effect-ideally, such a switch is effected in real time, i.e. once the call pattern of an interface is changed on the interface, the change will immediately affect all subsequent requests to that interface. Without restarting services, most modern frameworks and platforms support hot-loading or hot-update mechanisms, which means that the call mode of the interface can be changed dynamically without restarting the entire application.

Implementing a switch between asynchronous calls and synchronous calls

And the direct processing logic is used for executing corresponding business logic immediately after the server receives the request when the interface is set to synchronous, and directly returning the result to the client. The whole process is blocked until the processing is finished or an error occurs.

The method is suitable for operations with high response time requirements and short processing time, such as simple data query, state check and the like.

Message queue intervention-when the interface is configured as an asynchronous call, the server no longer processes the request directly, but rather encapsulates the request parameters into a message to be sent into the message queue. The client then immediately receives an acknowledge response indicating that the request has been accepted and is in process. And (3) background processing, namely taking out the message from the message queue by the subscriber terminal, and calling a corresponding service logic method to execute actual operation after reverse serialization. After the processing is finished, if a callback mechanism is configured, the processing result of the client can be notified in a preset mode.

Configuration options-a switch or drop-down menu is provided on the front-end interface for each interface for selecting the calling mode (synchronous/asynchronous) of that interface. In addition, additional configuration items may be included such as selection of a message queue, which message queue product may be required to be specified for use if an asynchronous call mode is selected (e.g., rabbitMQ, kafka, etc.). Callback address-for asynchronous call mode, a callback URL needs to be provided for receiving the processing result. Persistent storage to ensure that configuration information is not lost due to a system restart, the configurations should be persisted (e.g., saved to a database). In this way, even if the application is restarted, the previously set call mode is still valid.

In some embodiments of the present application, when the publishing end of the message queue sends the message body, format conversion is performed on the request parameter, so that the request parameter is adapted to the synchronous interface parameter requirement of the subscribing end.

Specifically, the function and function of the issuing side

The original request is received first, the publisher receives the original HTTP request from the client, which contains all necessary service parameters (e.g., commodity ID, user information, etc.).

Serialization in order to be able to transfer data efficiently between different components, the publisher needs to convert these request parameters into a format suitable for transmission, typically JSON or XML. This process is called serialization. For example, if the original request parameter is a Java object, it is converted into a JSON string.

{

"productId":"12345",

"userId":"user123"

}

In addition to simple serialization operations, the data format sometimes needs to be further adjusted according to the specific requirements of the subscriber end. For example, if a subscriber desires data of a particular structure (e.g., containing additional metadata), the publisher needs to reorganize the data in this structure before sending.

Sending messages to message queues

Selecting an appropriate message format, the publisher selects appropriate fields and data types to construct a message body based on the requirements of the subscriber. This may involve adding some additional information (e.g., time stamp, message ID, etc.) to facilitate subsequent tracking and management.

And finally, the issuing end uses the API or framework (such as SPRING AMQP) provided by the selected message queue product (such as RabbitMQ, kafka and the like) to send the formatted message body to the appointed message queue.

Ensuring synchronous interface parameter requirements of adaptation subscription end

Keeping the data consistent, it must be ensured that all necessary information is delivered to the subscriber without errors during the conversion process. This means that not only is the data properly serialized, but the integrity and accuracy of the data is also ensured. For example, if the subscriber relies on certain specific field names or data types, the publisher needs to translate in strict compliance with these requirements.

Input verification-an input verification step may be added before converting the request parameters into the message body, ensuring that all inputs are valid and conform to the intended format. This can prevent the occurrence of a failure of the subscriber-side process due to an invalid input. Exception handling-for any errors that may occur (e.g., serialization failure, field missing, etc.), the publisher should have appropriate exception handling mechanism, record detailed error logs, and give as friendly error hints as possible.

In some embodiments of the application, the triggering condition for the compensation or retry operation includes a message processing timeout, an interface return error code, or a persisting mechanism of a message queue detecting an outstanding message.

Specifically, the triggering condition of the compensation or retry operation

Message processing timeout

Defining that when a subscriber terminal fetches a message from a message queue and tries to call a corresponding synchronous interface to execute business logic, if a preset time limit (namely timeout) is exceeded, the processing is considered to be failed. Implementation details time monitoring a timer is set at the start of a processing task, and is marked as timeout once a set time threshold (e.g. 5 seconds, 30 seconds, etc.) is exceeded. Automatic retry-the system may be configured to automatically retry processing the message when a timeout is detected. The retry strategy may be customized according to the actual situation, such as a fixed number of retries, an exponential backoff algorithm, etc.

Interface return error code

After the subscriber terminal calls the synchronous interface to execute the business logic, if the interface returns an error code (such as HTTP status code 4xx or 5 xx) indicating failure, compensation or retry is needed. The implementation details are that the subscription terminal needs to have the capability of identifying the specific error code and take corresponding measures according to different error types. For example, for a temporary error (e.g., a connection failure due to network fluctuations), a retry may be selected, while for a permanent error (e.g., a request parameter is invalid), a log may be required and the administrator notified. Error handling policies-different handling policies may be defined for different types of error codes. For example, some error codes may indicate that an immediate retry is required, while other error codes suggest a later retry or a direct discard.

The persistence mechanism of a message queue detects outstanding messages

Definition many message queue products support a persistent storage function, which means that outstanding messages are not lost even if the server is restarted. When the system is restarted, the message queue checks whether there are messages that have not been processed yet and triggers a corresponding compensation or retry operation. Implementation details persistent storage-message queues save all unacknowledged received messages to disk or other persistent storage media, ensuring that they are recovered even if a failure occurs. And a recovery mechanism, wherein when the system resumes normal operation, the message queue automatically redistributes the unfinished messages to subscribers for processing. This typically involves cooperation with the subscriber end to ensure that each message is properly processed. Idempotent design to avoid the problem of data inconsistency caused by repeated processing of the same message, the subscriber should be designed to have idempotent, i.e. the result is the same no matter how many times it is repeatedly performed.

The application can be realized by adopting or referring to the prior art at the places which are not described in the application.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments.

The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and variations of the present application will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the application are to be included in the scope of the claims of the present application.

Claims (10)

1. A method for realizing asynchronous call of interfaces based on message queues is characterized by comprising the following steps:

S1, selecting a message queue product and performing installation configuration;

S2, packaging the functions of the message queue and providing a unified event publishing and subscribing mechanism;

s3, defining and generating a synchronous interface through a front-end interface, wherein the synchronous interface is used for receiving request parameters and executing corresponding business logic;

S4, registering the synchronous interface as an asynchronous interface, and generating a corresponding publishing end and a subscribing end of the message queue;

The calling process of the asynchronous interface comprises the steps that a client side requests to trigger the issuing side of the message queue, the issuing side packages the request parameters into a message body and sends the message body to the message queue, and the subscribing side monitors the message queue and acquires the message body and calls the synchronous interface to execute service logic.

2. The method of claim 1, wherein the message queue product is selected from at least one of RabbitMQ, kafka, rocketMQ, activeMQ or a Redis-based implementation of message queues.

3. The method according to claim 1, wherein in the step S2, the function package is implemented using JMS specification or SPRING AMQP framework.

4. The method according to claim 1, wherein in the step S3, the front end interface configuration includes:

the existing method or RPC service in the system is registered as a synchronous interface, and the request parameters of the interface are processed in a serialization or anti-serialization mode.

5. The method according to claim 1, wherein in the step S4, when registering the asynchronous interface, the publishing end and the subscribing end of the message queue are automatically generated, and the publishing end and the subscribing end are both deployed in the same application instance.

6. The method of claim 1, wherein after the asynchronous interface call is completed, an execution result is returned to the client through a preset callback interface.

7. The method of claim 1, wherein a subscriber to the message queue supports a compensation or retry operation for executing an abnormal request when invoking a synchronization interface.

8. The method of claim 1, further comprising configuring a call mode of the switching interface through the interface to effect dynamic switching between asynchronous calls and synchronous calls.

9. The method of claim 1, wherein the publishing end of the message queue format-converts the request parameters to adapt to the synchronous interface parameter requirements of the subscribing end when sending the message body.

10. The method of claim 7, wherein the triggering condition for the compensation or retry operation comprises a message processing timeout, an interface return error code, or a persistence mechanism of a message queue detecting an outstanding message.