CN111240832A - API management method and API gateway system - Google Patents

Abstract

The invention provides an API management method and an API gateway system. The method includes: when the API configuration data changes, notifying the server; after receiving the notification, the server pulls the changed API configuration data; The API configuration data is stored in a local file. It can reduce the probability of configuration errors, fully realize the requirements of API metadata management, avoid the risk of manually changing files, and realize unified management of all nodes, and the entire configuration change only needs to operate the configuration interface without modifying the lua script. , which is convenient for business development teams to use.

Description

An API management method and API gateway system

technical field

The invention relates to the technical field of Internet gateways, in particular to an API management method and an API gateway system.

Background technique

With the growth of business volume in the financial service industry, more and more interfaces are exposed to the outside world for internal services, there are many interface callers, and the environment is complex; higher requirements are placed on interface security, high performance, and unified management. Therefore, it is necessary to A unified access layer to manage all external interfaces; this access layer needs to be suitable for the characteristics of the financial industry to solve the following problems: unified API access point; protocol conversion; traffic routing; traffic control; permission and access control; Implement API metadata management; simplify back-end development; improve system operation and maintenance capabilities. There are existing Nginx (a high-performance HTTP and reverse proxy web server) + lua (a scripting language), Alibaba Cloud SLB and other solutions in the industry.

The Nginx+lua implementation method can achieve simple load balancing, routing management, and traffic control, but it cannot manage the interface metadata. The interface configuration changes need to modify each Nginx configuration file. The unified publishing process is complicated and prone to errors; the existence of lua scripts The technical threshold cannot be handed over to business developers for maintenance.

As Alibaba Cloud's traffic entry load balancing control software, SLB can perform functions such as routing, traffic distribution, and http service registration; however, the backend can only access the http protocol, and cannot implement current limiting according to merchants. The routing rules are simple and cannot meet the needs of the financial industry. , and the application must be on the cloud to use SLB.

SUMMARY OF THE INVENTION

In view of the problems existing in the related art, the purpose of the present invention is to provide an API management method and an API gateway system, which can reduce the probability of configuration errors, fully realize the requirements of API metadata management, and avoid the risks brought by manual file changes. The unified management of all nodes is realized, and the entire configuration change only needs to operate the configuration interface without modifying the lua script, which is convenient for the business development team to use.

According to an embodiment of the present invention, an API management method includes: when API configuration data changes, notifying a server; after receiving the notification, the server pulls the changed API configuration data; API configuration data is stored in local files.

According to an embodiment of the present invention, the API management method further includes: using a grayscale distribution process to distribute API configuration data, and the grayscale distribution process includes: when distributing the API configuration data, filtering according to the selected client ip is not within the selection range request; in the same domain, the grayscale delivery process does not end, and does not start a new process.

According to an embodiment of the present invention, the API management method further includes: when any one of the servers starts up, any one of the servers pulls the API configuration data from the gateway management end, and if the pulling is unsuccessful, obtains the API configuration data from a local file API configuration data.

According to an embodiment of the present invention, the API management method further includes: converting the API configuration information into an API mapping according to the API configuration data; matching with the API mapping according to the parsed http information; after successful matching, according to the client ip, merchant id, At least one of the access request parameters is subjected to current limiting processing.

According to an embodiment of the present invention, the API management method further includes: detecting an abnormal server through a heartbeat detection algorithm.

According to an embodiment of the present invention, in the API management method, pulling the changed API configuration data includes: the server monitoring a corresponding distributed application coordination service node, and coordinating changes in the service node according to the distributed application id pulls the changed API configuration data.

According to an embodiment of the present invention, an API gateway system includes a gateway management terminal and a server terminal, wherein the gateway management terminal is used to notify the server terminal when API configuration data changes; The gateway management end pulls the changed API configuration data and stores the pulled API configuration data in a local file.

According to an embodiment of the present invention, the gateway management end of the API gateway system is further configured to: use a grayscale distribution process to distribute API configuration data, and the grayscale distribution process includes: when distributing the API configuration data, according to the selected client ip Filter requests that are not within the selection range; under the same domain, the grayscale delivery process does not end, and does not start a new process.

According to the embodiment of the present invention, when the API gateway system starts up any one of the servers, any server pulls the API configuration data from the gateway management end, and if the pull is unsuccessful, the API configuration data is obtained from the local file .

According to the embodiment of the present invention, the server of the API gateway system is further configured to: convert the API configuration information into an API mapping according to the API configuration data, and, after the matching between the parsed http information and the API mapping is successful, according to the client At least one of ip, merchant id, and access request parameters performs current limiting processing.

According to the embodiment of the present invention, the API gateway system detects abnormal servers through a heartbeat detection algorithm.

According to the embodiment of the present invention, the server of the API gateway system is further configured to: the server monitors the corresponding distributed application coordination service node, and pulls changes according to the change id in the distributed application coordination service node of the API configuration data.

The beneficial technical effect of the present invention is that: the present invention controls all nodes through a unified gateway management terminal, and completes the API configuration. Configuration metadata is stored in a database that records all changes. There is a process control for configuration delivery, which can reduce the probability of configuration errors. Therefore, the requirements of API metadata management are fully realized, and unified batch push is carried out through the management terminal during delivery, which avoids the risk of manually changing files and realizes unified management of all nodes; the entire configuration change only needs to operate the configuration interface, not the The lua script needs to be modified to facilitate the use of the business development team.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings required in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some of the present invention. In the embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

1 is a flowchart of an API management method according to an embodiment of the present invention;

2 is a schematic diagram of a gateway system according to an embodiment of the present invention;

3 is a schematic diagram of a grayscale delivery process according to an embodiment of the present invention;

FIG. 4 is a flowchart of grayscale delivery according to an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art fall within the protection scope of the present invention.

As shown in Figure 1, the present invention provides an API (application program interface) management method, including:

S11, when the API configuration data changes, notify the server.

S12, after receiving the notification, the server pulls the changed API configuration data.

S13, the server stores the pulled API configuration data in a local file.

In the above technical solution of the present invention, when the API configuration data changes, the server is notified, and after receiving the notification, the server pulls the changed API configuration data, and stores the pulled API configuration data in a local file. API metadata can be configured to be stored in the database to record all change processes. Compared with the existing method of modifying configuration files, API configuration data no longer depends on the file system, but API configuration data is used as metadata for refined management. Persistent storage makes the configuration delivery process more secure and reliable.

As shown in FIG. 2, the present invention controls all nodes and completes API configuration through a unified gateway management terminal. Wherein, the gateway system of the present invention may include a management (admin) end, a service (server) end, and a service registration and discovery agent (Agent) end. The gateway system adopts a large cluster and multi-tenant mode to manage business access and deployment, and divides the access domain according to the two levels of cluster-business domain; the server side undertakes external traffic and is responsible for protocol conversion, current limiting, routing and other functions; registration and discovery Agent side Responsible for monitoring business application service instance information, and online or offline services according to the monitoring results. The reading process of the service registration information is as follows: directly read the service registration information corresponding to each instance from the ZK node. The service registration information may include APP Name, instance IP+port, instance status, and instance grouping label. The service registration information is real-time, so it does not need to be cached to a local file. For the server side in the gateway system, the abstract protocol adapter technology is used to connect to various back-end service types, and the back-end service can access the micro-service framework and message system to realize the protocol conversion of http-micro-service and http-message system. Therefore, the back-end service satisfies the access of the service-oriented framework.

Further, the admin side is used to manage the gateway running API configuration data. If the API configuration data changes, it will notify the server side through the gateway distributed application coordination service (zookeeper, ZK). After the server side receives the change notification, it calls the admin side. The interface pulls the changed API configuration data.

The admin side delivers API configuration data. The distribution configuration of API configuration data supports grayscale-full distribution, and rollback operation can be performed for the API configuration data that has been distributed. When the API configuration data is delivered, the server is notified of data changes through the gateway ZK. The delivery adopts the whole domain full delivery mode. The unified batch push through the admin side during delivery avoids the risk of manually changing files and realizes unified management of all nodes.

According to an embodiment of the present invention, as shown in FIG. 3 and FIG. 4 , after modifying the API configuration information, the delivery process is started, and then the grayscale delivery process is entered. When the API configuration data is delivered by the grayscale delivery process, requests that are not within the selection range are filtered according to the selected client IP. In addition, in the same domain, only one process is allowed to exist, the grayscale delivery process is not terminated, and a new process is not started (including not starting a new undo and rollback process). Only the following operations will lead to the end of the grayscale distribution process: the final full distribution and the process is confirmed or ignored or cancelled. The operation buttons for confirming and ignoring the process will appear only when the grayscale distribution process is in the full distribution stage.

The invention ensures that the impact of API configuration data changes on access requests is minimized through separate management of configuration and service, and API configuration data is distributed in grayscale. Every time configuration changes, the server side only needs to incrementally change the version through version comparison. Compared with the full update mode, the incremental changes have the least impact on the API interface (processed within 2 milliseconds), so that the access request is not aware of configuration changes. At the same time, the process control of the configuration delivery of the present invention can reduce the probability of configuration errors, make the configuration delivery process more secure and reliable, and should be simpler in API management. The entire configuration change only needs to operate the configuration interface, without modifying the lua script, so that business personnel can directly participate in configuration maintenance, which is convenient for business development teams to use.

In addition, when the server side of the gateway system starts: pull the API configuration data from the admin side according to the full mode. If the pull is unsuccessful, read the local configuration file to load the API configuration data. If the local loading is unsuccessful, the gateway starts fail. In this way, when the admin side or the gateway ZK cannot serve normally, the server side can rely on the local storage file to start and provide services, which can improve the robustness and reliability of the API gateway by reducing external dependencies on the server side.

According to the embodiment of the present invention, the server side pulling the changed API configuration data specifically includes: triggering the configuration information pulling service through the gateway ZK notification, each server side cluster corresponds to a ZK node, the server side can be based on the server side in the environment variable. The terminal cluster name monitors the ZK node, and calls the admin interface to pull the API configuration data according to the change id in the ZK node. In addition, the pulled API configuration data is first stored in a local file for alternative use when the gateway system is started, and then the API configuration information is parsed, and the API configuration information is converted into an API Map.

The traffic matching process can then proceed. In one embodiment, traffic may be processed in a netty (Java open source network application framework) worker thread. The admin side matches the API Map according to the http information parsed by Netty. The matching rule can be Host (client) + Path (route). If the match fails, the failure result will be returned; if the match is successful, it can be processed according to the following filter chain: current limiting configuration - service routing - calling back-end service - processing response (response) information - returning data, during this period, the back-end is called in a fully asynchronous manner. server side. The server side uses the filter chain technology to process requests. The requests include: API mapping, current limiting, authentication, exception handling, callback processing, and routing. According to the embodiment of the present invention, the multi-layer routing distribution technology can be used to realize the complex routing between the same url address to multiple different back-end applications and different instances within the application, and the routing can be set according to the parameters in the http request body and request header The rules meet the complex scenarios of customer routing rules in the financial industry.

The current limit configuration can be performed through the blocking capability of the API gateway. Specifically, access can be blocked according to the client ip, merchant id, or specified parameters in the access request, so the backend service can be protected from abnormal access attacks. Therefore, the current can be limited according to the dimensions of ip, session, merchant id, etc. At the same time, through the current limiting technology, the back-end service can be guaranteed to provide external services within the acceptable pressure range, and different current limiting strategies can be set for specific merchants to solve the problem. Flexible and complex current limiting scenarios in the financial industry. In one embodiment, a sliding window counter algorithm can be used to solve complex current limiting scenarios in the financial industry.

In addition, the heartbeat detection algorithm can also be used for the back-end service instances to detect abnormal servers, and the gateway system can also automatically remove the detected abnormal service nodes and issue alarm information. This enables quick responses to abnormal access, ensuring that normal requests and backend services are not affected. It can reduce the impact of abnormal back-end services, and improve the efficiency of abnormal recovery of back-end services.

Referring to Figure 2, the present invention also provides an API gateway system, including: a gateway admin end and a server end, wherein the gateway admin end is used to notify the server end when the API configuration data changes; the server end is used to receive After the notification arrives, pull the changed API configuration data from the gateway admin side and store the pulled API configuration data in a local file.

In one embodiment, the gateway admin end of the API gateway system is further configured to: use a grayscale distribution process to distribute API configuration data, and the grayscale distribution process includes: when distributing the API configuration data, filtering according to the selected client ip Requests that are not within the selection range; under the same domain, the grayscale delivery process does not end and a new process is not started.

In one embodiment, when the API gateway system starts up on any one of the servers, any server pulls the API configuration data from the gateway admin end, and if the pull is unsuccessful, the API configuration data is obtained from a local file.

In one embodiment, the server side of the API gateway system is further configured to: convert the API configuration information into an API mapping according to the API configuration data; , merchant id, and at least one of access request parameters to perform current limiting processing.

In one embodiment, the API gateway system detects an abnormal server side through a heartbeat detection algorithm.

In one embodiment, the server side of the API gateway system is further configured to: monitor the ZK nodes of each server-side cluster according to the server-side cluster name in the environment variable, and pull the changed API configuration data according to the change id in the ZK node .

The gateway system provided by the present invention controls all nodes through a unified gateway management terminal, and completes the API configuration. Configuration metadata is stored in a database that records all changes. There is a process control for configuration delivery, which can reduce the probability of configuration errors. Therefore, the requirements of API metadata management are fully realized, and unified batch push is carried out through the management terminal during delivery, which avoids the risk of manually changing files and realizes unified management of all nodes; the entire configuration change only needs to operate the configuration interface, not the The lua script needs to be modified to facilitate the use of the business development team.

The above are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims (12)

1. An API management method, comprising:

when the API configuration data changes, the server is notified;

after receiving the notification, the server pulls the changed API configuration data;

and the server stores the pulled API configuration data in a local file.

2. The API management method of claim 1, further comprising: issuing the API configuration data by adopting a gray level issuing process, wherein the gray level issuing process comprises the following steps:

when the API configuration data are issued, filtering the requests which are not in the selection range according to the selected client ip;

and under the same domain, the gray level issuing process is not ended, and a new process is not started.

3. The API management method of claim 1, further comprising: when any one of the service terminals is started, the any one service terminal pulls API configuration data from a gateway management terminal,

and if the pulling is unsuccessful, acquiring the API configuration data from the local file.

4. The API management method of claim 1, further comprising:

converting the API configuration information into API mapping according to the API configuration data;

matching with the API mapping according to the analyzed http information;

and after the matching is successful, performing the current limiting processing according to at least one of the client ip, the merchant id and the access request parameter.

5. The API management method of claim 1, further comprising: and detecting the abnormal server side by a heartbeat detection algorithm.

6. The API management method of claim 1, wherein pulling the changed API configuration data comprises: and the server monitors corresponding distributed application program coordination service nodes and pulls the changed API configuration data according to the change id in the distributed application program coordination service nodes.

7. An API gateway system is characterized by comprising a gateway management end and a service end, wherein,

the gateway management end is used for notifying the server end when the API configuration data changes;

and the server is used for pulling the changed API configuration data from the gateway management end and storing the pulled API configuration data in a local file after receiving the notification.

8. The API gateway system of claim 7 wherein the gateway management side is further configured to: issuing the API configuration data by adopting a gray level issuing process, wherein the gray level issuing process comprises the following steps:

when the API configuration data are issued, filtering the requests which are not in the selection range according to the selected client ip;

and under the same domain, the gray level issuing process is not ended, and a new process is not started.

9. The API gateway system of claim 7, wherein when any one of the servers starts up, the any one server pulls API configuration data from the gateway management side,

and if the pulling is unsuccessful, acquiring the API configuration data from the local file.

10. The API gateway system of claim 7 wherein the server is further configured to:

converting the API configuration information into an API map according to the API configuration data, an

And after the matching between the analyzed http information and the API mapping is successful, performing the current limiting processing according to at least one of the client ip, the merchant id and the access request parameter.

11. The API gateway system of claim 7, wherein the anomalous server is detected via a heartbeat detection algorithm.

12. The API gateway system of claim 7 wherein the server is further configured to: and the server monitors corresponding distributed application program coordination service nodes and pulls the changed API configuration data according to the change id in the distributed application program coordination service nodes.

Images