CN114978813A - Gateway implementation method based on response type thread pool - Google Patents

Abstract

The invention provides a gateway implementation method based on a response type thread pool, which relates to the technical field of proxy service and comprises the following steps: establishing TCP connection between all the microservices and a gateway; the gateway receives a user request and carries out frame analysis to obtain request contents of different protocol types; the gateway distributes servers corresponding to the micro services for all the request contents according to a preset load strategy; the gateway respectively pushes the request contents of different protocol types to corresponding threads for processing, and the threads are connected with corresponding servers to obtain request results. The gateway supports various protocol types, realizes IO multiplexing based on the response type thread pool, and has higher performance threshold value compared with the existing gateway by using the response type thread pool, avoids deep object analysis of various protocols, reduces a large amount of memory creating and destroying processes, avoids the condition of lock waiting and the like to consume a large amount of server resources, accelerates user request content forwarding, and improves server calling.

Description

A Gateway Implementation Method Based on Responsive Thread Pool

technical field

The present invention relates to the field of communication technologies, in particular to a gateway implementation method based on a responsive thread pool.

Background technique

Under the system of modern enterprise software development, the architecture based on microservices is a more popular architecture design scheme. In the microservice architecture, the services are very scattered, which reduces the coupling degree and increases the difficulty of the unified management of the services.

Under the previous design idea based on SOA (ie service-oriented) architecture, common functions such as authentication, current limiting, logging, and monitoring need to be implemented separately in each service, which makes the system maintainer do not have a global view for unified management these functions.

In the microservice framework, each external service is deployed independently, and the external API or service addresses are different. For the internal, it is very simple, it can be automatically sensed through the registry; but in most cases, services are provided to external systems for calling, and it is impossible to share a registry. At the same time, the internal microservices are The internal network is not connected to the outside world. Moreover, even if each of our microservices is open to the outside world, for callers, the addresses or parameters for calling different services are not the same, which will cause the complexity of consumer clients, and microservices may be different technologies. The stack implementation, some of which are http, rpc or websocket, etc., will further increase the difficulty of calling the client.

Therefore, there is generally an api gateway, which routes to different services according to the requested url. After the entry is unified, it can also perform unified authentication, logging, and offloading operations, that is, the API gateway is a microservice collection By managing these general functions, the scalability of the system is improved, and the service itself is more focused on its own domain, and the service caller and service provider are well isolated.

At present, the mainstream API gateways are Netflix's zuul and KONG. Zuul is a synchronous blocking gateway design idea. Every time a request comes, zuul will allocate a thread to process it, and then forward it to the back-end service. The back-end will then enable the thread to process the request, and the thread of the gateway will be blocked during the back-end processing. When the number of requests is relatively large, it is easy to cause the thread pool to be full and unable to accept new requests. In addition, when users upload files, zuul does not perform special asynchronous IO processing internally, which makes the throughput of the entire system plummet.

KONG is a high-performance web platform based on Nginx and Lua, which integrates a large number of sophisticated Lua libraries, third-party modules and most dependencies. It is used to easily build dynamic web applications, web services and dynamic gateways that can handle ultra-high concurrency and high scalability. The performance of this gateway is high, but because the business logic and other related plug-ins depend on the Lua language, which is a scripting language, there is currently no good development and debugging tools. It is difficult for enterprises to customize development while using this technology.

SUMMARY OF THE INVENTION

In view of the above problems, the present invention provides a gateway implementation method based on a responsive thread pool, using the responsive thread pool method to provide a brand-new gateway implementation method, the gateway has no external dependence, and reduces the problem that the threads of the gateway cannot provide services. time, to realize the deployment of the distributed HA high-availability solution.

In order to achieve the above purpose, the gateway implementation method based on the responsive thread pool provided by the present invention includes:

Establish a TCP connection with the gateway for all microservices;

The gateway receives the user request and parses the frame to obtain the request content of different protocol types;

The gateway allocates the server corresponding to the microservice for all the requested content according to the preset load policy;

The gateway pushes the request contents of different protocol types to the corresponding thread for processing, and the thread establishes a connection with the corresponding server to obtain the request result.

As a further improvement of the present invention, all the microservices report the number of open interfaces, the URL address of the interface and the content of the request parameters of the interface to the gateway when starting;

The gateway allocates a server corresponding to the microservice to the request content according to the preset load policy, and obtains the URL address of the microservice interface and the request parameter content of the interface;

The thread establishes a connection with the server according to the URL address of the microservice interface and the content of the request parameters of the interface, and obtains the request result.

As a further improvement of the present invention,

The gateway directly pushes the URL address of the microservice interface and the request parameter content of the interface to the corresponding thread;

The thread directly forwards the request content to the corresponding server, and obtains the request result.

As a further improvement of the present invention, the gateway receives user requests and performs frame parsing, and serializes the request contents of different protocol types obtained by frame parsing and stores them in the memory object of the thread pool.

As a further improvement of the present invention, the request contents of different protocol types correspond to different microservice protocols, including http, tcp, udp, websocket, and RPC.

As a further improvement of the present invention, after all the microservices are started, they regularly report to the gateway the basic load of the server where they are located, including the CPU, memory, network and the current number of sessions of the application.

As a further improvement of the present invention, the load strategy includes: average load, random load and priority load;

The load average is to evenly distribute the user's request content to all the servers of the microservices that conform to the request content;

The random load is to randomly distribute the user's request content to all the servers of the microservices that conform to the request content;

The priority load is weighted and scored according to the basic load of the server where each microservice is regularly uploaded, and the user's request content is loaded to the server with the lowest score.

As a further improvement of the present invention, the priority load is a default load strategy, and the weighted scoring formula is:

S=CPU usage*40*CPU load+memory usage*40+(network card rate Mb/400Mb)*5+(current session number of application/2000)*15.

As a further improvement of the present invention,

The gateway supports session retention, and performs server allocation according to the load policy when the user initiates the user request for the first time;

The user's subsequent user request is directly forwarded to the previously allocated server for processing;

If the server is offline or abnormal, the server is reassigned for the user's subsequent user request.

As a further improvement of the present invention, the gateway receives user requests and performs frame analysis to obtain request contents of different protocol types; including:

The gateway monitors the TCP/UDP ports of each of the microservices, and performs frame-by-frame analysis of all the traffic to obtain request contents of different protocol types.

Compared with the prior art, the beneficial effects of the present invention are:

The gateway of the present invention supports a variety of protocol types, realizes the multiplexing of IO based on the responsive thread pool, and has wider versatility compared with the existing gateway, and the use of the responsive thread pool enables the gateway to have higher performance Threshold, avoids the in-depth object analysis of various protocols, reduces the process of creating and destroying a lot of memory, avoids the consumption of a lot of server resources such as lock waiting, and the application service is unavailable or the user waits timeout due to the consumption of a lot of server resources. In the event of an abnormal exit afterward, the forwarding of user requests and server calls are accelerated, and the deployment of distributed HA high-availability solutions can be supported.

The gateway of the present invention distributes the content requested by the user through a preset load strategy, and especially by using the priority load, the network resources can be fully utilized, and the processing efficiency of the server can be improved at the same time.

Description of drawings

1 is a flowchart of a method for implementing a gateway based on a responsive thread pool disclosed by an embodiment of the present invention;

FIG. 2 is a schematic diagram of an implementation process of a gateway based on a responsive thread pool disclosed by an embodiment of the present invention.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present invention.

Below in conjunction with accompanying drawing, the present invention is described in further detail:

As shown in Figure 1, a gateway implementation method based on a responsive thread pool provided by the present invention includes:

S1. Establish a TCP connection between all microservices and the gateway;

in,

This step realizes the registration localization of microservices.

further,

All microservices report the number of open interfaces, the URL address of the interface, and the content of the request parameters of the interface to the gateway when they are started;

After all microservices are started, periodically (for example, set the default 3s) to report to the gateway the basic load of the server where it is located, including CPU, memory, network, and the current number of sessions of the application.

S2. The gateway receives the user request and performs frame analysis to obtain the request content of different protocol types;

in,

The gateway monitors the TCP/UDP ports of each microservice, parses all the traffic frame by frame, and obtains the request content of different protocol types. The request content of different protocol types obtained by frame analysis is serialized and stored in the thread pool. In the memory object, multiple service protocols can be supported to realize the multiplexing of IO.

further,

The request content of different protocol types corresponds to different microservice protocols, including http, tcp, udp, websocket, and RPC.

S3. The gateway allocates the server corresponding to the microservice for all the requested content according to the preset load policy;

in,

Load policies include: average load, random load and priority load, and priority load is the default load policy;

specific,

Load average is to evenly distribute the user's request content to the servers of all microservices that match the request content;

Random load is to randomly distribute the user's request content to the servers of all microservices that match the request content;

The priority load is weighted and scored according to the basic load of the server where each microservice is regularly uploaded, and the user's request content is loaded to the server with the lowest score. The weighted scoring formula is:

S=CPU usage*40*CPU load+memory usage*40+(network card rate Mb/400Mb)*5+(current session number of application/2000)*15.

further,

The gateway allocates the server corresponding to the microservice for the request content according to the preset load policy, and obtains the URL address of the microservice interface and the request parameter content of the interface;

S4. The gateway pushes the request contents of different protocol types to the corresponding thread for processing, and the thread establishes a connection with the corresponding server to obtain the request result.

in,

The gateway directly pushes the URL address of the microservice interface and the request parameter content of the interface to the corresponding thread, reducing the invocation of memory copying;

The thread establishes a connection with the server according to the URL address of the microservice interface and the request parameter content of the interface, directly forwards the request content to the corresponding server, implements the call, and obtains the request result.

In this application, the gateway supports session retention, and performs server allocation according to the load policy when the user initiates a user request for the first time;

The subsequent user request of the user is directly forwarded to the server assigned last time for processing;

If the server is offline or abnormal, the server will be reassigned for the user's subsequent user request.

specific,

The implementation of http-based session retention is to perform hash processing according to the user's UA+ip address and other information, and store the string and the machine number of the load in the server cache. And try to write to the http only cookie on the user side.

The implementation of session retention based on tcp protocols such as TCP itself and RPC, socket, etc. is hashed by IP address + port number and stored in the server cache.

For network forwarding based on the UDP protocol, the configuration without session persistence does not take effect. That is, session retention is not performed for the forwarding of the UDP protocol suite.

Advantages of the present invention:

(1) The gateway of the present invention supports a variety of protocol types, and realizes the multiplexing of IO based on the responsive thread pool. Compared with the existing gateway, it has wider versatility. The use of the responsive thread pool enables the gateway to have more The high performance threshold avoids the in-depth object analysis of various protocols, reduces the process of creating and destroying a lot of memory, avoids the consumption of a lot of server resources such as lock waiting, and the unavailability of application services caused by the consumption of a lot of server resources. When the user waits for a timeout and then exits abnormally, it accelerates the forwarding of user requests and server calls, and supports the deployment of distributed HA high-availability solutions.

(2) The gateway of the present invention distributes the content requested by the user through a preset load strategy, and especially by using the priority load, the network resources can be fully utilized, and the processing efficiency of the server can be improved at the same time.

The above are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims (10)

1. a gateway implementation method based on responsive thread pool, is characterized in that, comprises: Establish a TCP connection with the gateway for all microservices; The gateway receives the user request and parses the frame to obtain the request content of different protocol types; The gateway allocates the server corresponding to the microservice for all the requested content according to the preset load policy; The gateway pushes the request contents of different protocol types to the corresponding thread for processing, and the thread establishes a connection with the corresponding server to obtain the request result. 2. gateway implementation method according to claim 1, is characterized in that: all report the request parameter content of open interface quantity, the URL address of interface and interface to gateway when all described microservices are started; The gateway allocates a server corresponding to the microservice to the request content according to the preset load policy, and obtains the URL address of the microservice interface and the request parameter content of the interface; The thread establishes a connection with the server according to the URL address of the microservice interface and the content of the request parameters of the interface, and obtains the request result. 3. gateway implementation method according to claim 2, is characterized in that: The gateway directly pushes the URL address of the microservice interface and the request parameter content of the interface to the corresponding thread; The thread directly forwards the request content to the corresponding server, and obtains the request result. 4. The gateway implementation method according to claim 1, characterized in that: the gateway receives a user request and performs frame parsing, and stores in the memory of the thread pool after serializing the request content of different protocol types obtained by the frame parsing in the object. 5 . The gateway implementation method according to claim 1 , wherein the request contents of different protocol types correspond to different microservice protocols, including http, tcp, udp, websocket, and RPC. 6 . 6. The gateway implementation method according to claim 1, characterized in that: after all the microservices are started, regularly report to the gateway the basic load of the server where they are located, including CPU, memory, network and application current session number . 7. The gateway implementation method according to claim 6, wherein the load policy comprises: average load, random load and priority load; The load average is to evenly distribute the user's request content to all the servers of the microservices that conform to the request content; The random load is to randomly distribute the user's request content to all the servers of the microservices that conform to the request content; The priority load is weighted and scored according to the basic load of the server where each microservice is regularly uploaded, and the user's request content is loaded to the server with the lowest score. 8. The gateway implementation method according to claim 7, wherein the priority load is a default load policy, and the weighted scoring formula is: S=CPU usage*40*CPU load+memory usage*40+(network card rate Mb/400Mb)*5+(current session number of application/2000)*15. 9. gateway implementation method according to claim 1, is characterized in that: The gateway supports session retention, and performs server allocation according to the load policy when the user initiates the user request for the first time; The user's subsequent user request is directly forwarded to the previously allocated server for processing; If the server is offline or abnormal, the server is reassigned for the user's subsequent user request. 10. The gateway implementation method according to claim 1, wherein: the gateway receives user requests and performs frame analysis to obtain request contents of different protocol types; comprising: The gateway monitors the TCP/UDP ports of each of the microservices, and performs frame-by-frame analysis of all the traffic to obtain request contents of different protocol types.

Images