Detailed Description
In order to better understand the aspects of the present application, the present application will be described in further detail with reference to the accompanying drawings and detailed description. It will be apparent that the described embodiments are only some, but not all, embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The embodiment of the application provides a CDN resource scheduling method, a CDN resource scheduling device, electronic equipment and a storage medium.
For ease of understanding, some concepts involved in embodiments of the application are first described below.
Content delivery network (Content Delivery Network, CDN): a distributed network built on the existing network for accelerating the network access speed is characterized in that the server clusters of the edge nodes share the pressure of the source site server clusters by the functions of load balancing, content release, content management and content storage by depending on the server clusters deployed on the edge nodes in the whole world, so that users can acquire cached access resources nearby, network congestion is avoided, and the access speed is accelerated.
Basic principle of CDN: various cache servers are widely adopted, the cache servers are distributed in areas or networks where user accesses are relatively concentrated, when a user accesses a website, the user accesses are directed to the cache server which works normally and is closest to the website by using a global load technology, and the cache server directly responds to a user request.
The CDN network comprises the following components: mainly comprises a CDN center node and CDN edge nodes.
CDN center node: the system comprises a CDN network management center and a global load balancing DNS redirection analysis system, and is responsible for the distribution and management of the whole CDN network.
CDN edge node: the system mainly refers to a distributed node at different places, and consists of two parts, namely load balancing equipment and a high-speed edge cache server (also called a server for short).
Relationship between CDN edge nodes, edge cache servers and server groups: each CDN edge node typically includes a plurality of edge cache servers that form a server group.
CDN vendor: the party providing the CDN devices and CDN networks, also called the service party, is for example currently three major operators (telecommunications, mobile and corporate), some carriers (hua-si, zhong, network-hosting etc.), etc.
CDN clients: also referred to as "customer" for short, refers to a party using CDN devices and CDN networks provided by CDN vendors, and also referred to as served party, for example, various APPs (video APP, game APP, car APP, weather APP, health APP, shopping APP, social APP, news APP, etc.) internet companies, websites, traditional companies, enterprises and institutions, and so on. CDN clients provide services to users of APP under the client's flag through the devices and networks provided by CDN vendors, for example, video companies provide streaming media services to users of video APP under their company's flag (general users and member users) through the devices and networks provided by CDN vendors.
A visitor: the users of CDN clients, e.g., users of video APP, users of game APP, users of on-board APP, users of weather APP, users of health APP, users of shopping APP, users of social APP, users of news APP, etc.
Acceleration region of CDN customer: to facilitate management and data distribution, customers divide their served regions into multiple regions, i.e., acceleration regions, by geographic location. For example, weather APP is divided into a plurality of acceleration areas according to provinces, one province being one acceleration area; the health class APP of Beijing is divided into a plurality of acceleration areas according to administrative areas, and one administrative area is an acceleration area.
CDN resources: the embodiment of the application refers to an edge cache server of the CDN.
Directed acyclic graph (DIRECTED ACYCLIC GRAPH, DAG): if a directed graph cannot start from a vertex and go through several edges back to the vertex, the graph is a directed acyclic graph.
Query rate Per Second (query-Per-Second, QPS): is a measure of how much traffic a particular query server is handling in a given time. The QPS in the embodiment of the present application refers to the number of guest access requests per second.
The CDN resource scheduling method in the prior art comprises the following steps: when CDN resources are scheduled for clients, CDN manufacturers mainly consider to continuously increase the bandwidth capacity of CDN nodes, namely bandwidth resources are increased. The scheduling method depending on the increased bandwidth resources is suitable for clients consuming more bandwidth, such as video APP and game APP. However, for clients with large concurrency of visitor access requests such as a health APP, a weather APP and a vehicle-mounted APP, but small bandwidth occupation amount of data requested by the visitor access requests, the method is not applicable, and timely response to the visitor access requests cannot be realized, so that the service quality of the clients is low. For example, a user of the health APP cannot respond until the user looks for health information, and cannot display the health information.
The CDN resource scheduling method of the embodiment of the application comprises the following steps: when CDN resources are scheduled for clients, CDN manufacturers do not only consider bandwidth factors of CDN nodes, but also synthesize the characteristics of the clients, and aiming at clients with large concurrency of visitor access requests such as health APP, weather APP and vehicle-mounted APP, but less obvious bandwidth consumption, the number of visitor access requests per second of CDN edge nodes is monitored in real time, when the sudden increase of the number of visitor access requests is monitored, node resources are replaced or added for the clients in time (for example, a server with large QPS capacity is replaced or a server with large QPS capacity is added) so as to realize timely response to the visitor access requests and improve the service quality of the clients.
The following describes a CDN resource scheduling method provided by the embodiment of the present application.
Fig. 1 is a flowchart of a CDN resource scheduling method according to an embodiment of the present application, where, as shown in fig. 1, the method may include the following steps: step 101, step 102, step 103 and step 104;
in step 101, a real-time QPS of a server group of each edge node of the CDN is acquired.
In the embodiment of the application, the real-time QPS of the server group refers to the real-time visitor access request number per second of the server group.
In the embodiment of the present application, since a server group generally includes a plurality of servers (i.e., a plurality of edge cache servers), and each server has a real-time QPS, the real-time QPS of the server group may be the sum of the real-time QPS of all servers in the server group.
Specifically, the real-time QPS of all servers under each edge node of the CDN may be obtained first, then, according to the IP of each server, a server group to which each server corresponds is found, and the real-time QPS of all servers in each server group are summed with the server group as a calculation unit, to obtain the real-time QPS of each server group.
For example, the three server groups are a server group a, a server group B, and a server group C, where the server group a includes a server 11, a server 12, and a server 13, the server group B includes a server 21, a server 22, and a server 23, and the server group C includes a server 31, a server 32, and a server 33, and the real-time QPS of each of the above servers is shown in table 1 below, and the real-time QPS of all the servers in each of the server groups are summed up in a calculation unit of the server group, so that the real-time QPS of each of the server groups can be obtained.
TABLE 1
In the embodiment of the application, the real-time QPS of the server group of each edge node of the CDN is obtained to monitor the number of requests per second of the edge node in real time, so that when the number of requests suddenly increases to cause insufficient QPS resources of the edge node, resource scheduling is timely carried out, namely the QPS resources of the edge node are increased.
In step 102, a target server group with insufficient QPS is determined according to the real-time QPS and an upper limit QPS of each server group, where the upper limit QPS is the maximum QPS supported by the server group.
In the embodiment of the present application, the upper limit QPS of the server group may be the sum of the upper limits QPS of all servers in the server group.
In the embodiment of the application, the upper limit QPS of the server group refers to the maximum value of the number of visitor access requests that can be supported by the server group per second.
In the embodiment of the application, the target server group with insufficient QPS refers to the run-out server group, namely, the server group which cannot respond to all visitor access requests.
In the embodiment of the application, the real-time QPS of all the server groups can be traversed, and whether the running of the real-time QPS exceeds the upper limit QPS of the server groups or whether available QPS exists can be judged.
In some embodiments, for each server group, the real-time QPS of the server group may be directly compared with the upper limit QPS of the server group, and if the real-time QPS of the server group is greater than or equal to the upper limit QPS, the server group is determined to be a target server group with insufficient QPS; if the real-time QPS of the server farm is less than the upper QPS, then the QPS of the server farm is deemed sufficient.
In some embodiments, to prevent the number of visitor access requests on the server group from suddenly running, a redundancy parameter may be set, for example, the redundancy parameter is 0.9, where the step 102 includes the following steps: step 1021, step 1022, and step 1023;
in step 1021, a redundancy parameter is obtained, where the redundancy parameter has a value between 0 and 1, and the redundancy parameter is used to prevent the number of visitor access requests on the server set from running beyond.
In the embodiment of the present application, different redundancy parameters may be set for different server groups, or the same redundancy parameters may be set for different server groups, which is not limited in the embodiment of the present application.
In step 1022, a redundancy QPS for each server group is calculated according to the real-time QPS, the upper limit QPS, and the redundancy parameter of each server group, where redundancy qps=upper limit QPS.
In the embodiment of the present application, redundant QPS refers to the remaining QPS, i.e., the available QPS. For each server group, the redundancy QPS of the server group = the upper limit QPS of the server group-the real-time QPS of the server group.
In step 1023, the server group with redundancy QPS less than 0 is determined as a target server group with insufficient QPS.
In the embodiment of the application, for the target server group with insufficient QPS, the target server group is considered to have run beyond, and the resources of the server group are required to be scheduled.
In step 103, according to the correspondence between the server group and the clients, determining the domain name of the target client corresponding to the target server group and the target acceleration area, wherein the acceleration area is obtained based on the geographic location division.
In the embodiment of the present application, when a CDN manufacturer provides a service for a customer, a plurality of server groups are allocated to the customer, and a correspondence between the customer and the server groups allocated thereto is recorded, where the correspondence may be in the form of a correspondence between a customer domain name, an acceleration area, and a server group identifier, as shown in table 2 below, where beijin refers to Beijing, and Tianjin refers to Tianjin.
Customer domain name |
Acceleration region |
Server group |
www.aaa.com |
Beijing |
A,B,C |
www.bbb.com |
Beijing |
B,C,D |
www.aaa.com |
Tianjin |
A,D,E |
TABLE 2
In practical applications, multiple clients may select the same CDN vendor, where a CDN vendor may serve multiple clients, and when server sets are allocated to the multiple clients, the multiple clients may share one server set, or may use different server sets.
In the embodiment of the application, for all server groups with insufficient QSP, namely the target server group, all clients and acceleration areas affected by the insufficient QPS of the server group, namely the target acceleration areas of the target clients, can be found according to the corresponding relation between the server group and the clients.
In the embodiment of the application, CDN resource scheduling is performed for the target clients in the target acceleration area affected by insufficient QPS of the server group so as to increase the QPS resources of the corresponding server group.
In step 104, performing CDN resource scheduling for target clients in the target acceleration region; wherein, the CDN resource scheduling includes at least one of the following: replacing a first server in the target server group with a second server, and adding a third server in the target server; the upper limit QPS of the second server is greater than the upper limit QPS of the first server.
In the embodiment of the application, in order to facilitate CDN resource scheduling for clients, a resource pool can be allocated in advance for each client, and each resource pool comprises a plurality of standby servers. When CDN resource scheduling is performed subsequently, selecting a server with a large QPS upper limit value from the resource pool, replacing a server with a small QPS upper limit value in the server group, or selecting a server with a large QPS upper limit value from the resource pool, and adding the server into the server group.
In the embodiment of the application, the essence of CDN resource scheduling for the target clients in the target acceleration area is as follows: and increasing the QPS capacity of the server group corresponding to the target client in the target acceleration area so as to be capable of responding to more visitor access requests, and enabling the access requests of the users of the target client in the target acceleration area to be fed back.
As can be seen from the above embodiment, in this embodiment, a real-time QPS of a server group of each edge node of the CDN is obtained; determining a target server group with insufficient QPS according to the real-time QPS and the upper limit QPS of each server group, wherein the upper limit QPS is the maximum QPS supported by the server group; determining a domain name and a target acceleration region of a target client corresponding to the target server group according to the corresponding relation between the server group and the client, wherein the acceleration region is obtained based on geographic position division; performing CDN resource scheduling for target clients in a target acceleration area; wherein, the CDN resource scheduling includes at least one of the following: replacing a first server in the target server group with a second server, and adding a third server in the target server; the upper limit QPS of the second server is greater than the upper limit QPS of the first server.
Therefore, in the embodiment of the application, when CDN resources are scheduled for clients, CDN manufacturers do not only consider bandwidth factors of CDN nodes, but also synthesize the characteristics of the clients, the visitor access request number data of the CDN edge nodes are monitored in real time for the clients with large visitor access request concurrency but less obvious bandwidth consumption, and when the sudden increase of the visitor access request number is monitored, node resources are replaced or added for the clients in time for the edge nodes with insufficient resources, so that the timely response to the visitor access requests is realized, and the service quality of the clients is improved.
In another embodiment of the present application, when performing CDN resource scheduling for the target client in the target acceleration area, the CDN resource scheduling may be performed based on historical visitor access request data of the target client in the target acceleration area, and accordingly, as shown in fig. 2, the step 104 includes the following steps: step 201, step 202 and step 203;
in step 201, a historical peak QPS of guest access requests by the target guest in the target acceleration area is obtained.
In the embodiment of the application, the fact that the requirements of different clients on the number of requests in different time periods are generally different is considered, so that the time of day can be divided into a plurality of time periods, and the CDN node resources can be fully utilized by combining the factors of the peak time periods of different clients. For example, the time of day is divided into two time periods: 0-18 (early time period) and 18-24 (late time period).
Accordingly, in some embodiments, a historical peak QPS of guest access requests for different periods of time for the last N days of the target guest within the target acceleration area may be obtained, N being an integer greater than 1.
In step 202, resource planning information of a target client in a target acceleration area is acquired, wherein the resource planning information includes: the server group used and the priority of use of the server group.
In consideration of the possible situation that a plurality of clients share one server set, in the embodiment of the present application, the use priority of each client to the server set may be set, and different clients correspond to different use priorities.
In some embodiments, resource planning information for the last N days of target customers within the target acceleration zone may be obtained. For example, resource planning information for a last 2 days target customer within a target acceleration area is obtained.
Considering that a customer typically provides more than one task, e.g. a health class APP, for a user with multiple services, e.g. vaccination service, disease detection service, etc., each service may be considered as one task, a CDN vendor may assign a server group to a customer after assigning the server group to the customer, which may also assign a server group to different tasks according to their own task.
In one example, the resource planning information for the last 2 days of target customers within the target acceleration zone is shown in table 3 below.
Task ID |
Customer domain name |
Acceleration region |
Server group |
Priority level |
1 |
www.aaa.com |
Beijing |
A,B,C |
High height |
2 |
www.bbb.com |
Beijing |
B,C,D |
Low and low |
3 |
www.aaa.com |
Tianjin |
A,D,E |
High height |
TABLE 3 Table 3
In step 203, CDN resource scheduling is performed for the target clients in the target acceleration region according to the historical peak QPS and the resource planning information.
In the embodiment of the application, the maximum value of the number of the visitor access requests received by each target server group in the target acceleration area at the current time can be estimated according to the historical peak value QPS of the visitor access requests of the target clients in the target acceleration area, and the target server group with insufficient QPS resources, namely, which target server group needs to be added with QPS resources, can be determined by comparing the estimated maximum value with the upper limit QPS of the target server group.
Preferably, according to the historical peak value QPS of the guest access requests of the target clients in the target acceleration area in different time periods in the last N days, the maximum value of the number of guest access requests received by each target server group in the target acceleration area at the current time can be estimated more accurately.
In the embodiment of the application, the execution sequence of each task in the target acceleration area can be determined according to the resource planning information of the target client in the target acceleration area.
In one example, the resource planning information shown in table 3, as can be seen from table 3: the related task with the customer domain name www.aaa.com needs to be executed preferentially; task 1 and task 3 have a common server group A, and cannot be executed simultaneously; task 2 depends on task 1 and task 3 execution completion.
Therefore, in the embodiment of the application, the execution sequence of each task in the target acceleration area can be determined according to the resource planning information, the maximum value of the number of visitor access requests of each target server group in the target acceleration area is estimated according to the historical peak value QPS, each task is executed according to the determined execution sequence, in the task execution process, the target server group with insufficient QPS resources when executing each task is determined according to the estimated maximum value, the target server group with insufficient QPS resources can be accurately determined, and the QPS resources can be timely added for the target server group, so that the timely response to the visitor access requests can be realized, and the service quality of clients is improved.
In another embodiment provided by the present application, records of guest access client domain names may be collected from each server group of the CDN edge node, according to the area to which the guest IP belongs, the number of guest requests in the guest access records is summarized to each acceleration area of each client, and a time of day is differentiated into a plurality of time periods, and the peak request number of each time period is calculated, accordingly, as shown in fig. 3, the step 201 includes the following steps: step 301, step 302, step 303 and step 304;
In step 301, a visitor access record of a server group of each edge node of the CDN is obtained, where the visitor access record includes the number of visitor access requests of different visitor IPs for different customer domain names.
In one example, the guest access record is shown in table 4 below.
Visitor IP |
Customer domain name |
Number of access requests |
1.1.1.1 |
www.aaa.com |
500 |
1.1.1.1 |
www.bbb.com |
260 |
1.1.1.2 |
www.ccc.com |
400 |
TABLE 4 Table 4
In step 302, the number of visitor access requests in the visitor access record is summarized to the acceleration area of each client according to the area to which the visitor IP belongs.
In one example, the acceleration area to which the visitor IP belongs may be queried from the IP library every 5 minutes, and the QPS of different customers in the corresponding acceleration area within these 5 minutes may be calculated, for example as shown in table 5 below.
Customer domain name |
Acceleration region |
QPS |
www.aaa.com |
Beijing |
1200 |
www.bbb.com |
Beijing |
1500 |
www.aaa.com |
Tianjin |
500 |
TABLE 5
In step 303, the time of day of the last N days is divided into time periods, and the historical peak QPS of the visitor access requests of the clients in different time periods in each acceleration area of the last N days is calculated according to the visitor access request number of each client in each acceleration area.
In the embodiment of the application, the fact that the requirements of different clients on the number of requests in different time periods are generally different is considered, so that the time of day can be divided into a plurality of time periods, and the CDN node resources can be fully utilized by combining the factors of the peak time periods of different clients. For example, the time of day is divided into two time periods: 0-18 (early time period) and 18-24 (late time period).
For example, the QPS of each client in each acceleration region is first summarized to obtain the results shown in table 6 below.
Customer domain name |
Acceleration region |
QPS |
Time of |
www.aaa.com |
Beijing |
1200 |
2022-07-25 17:45:00 |
www.bbb.com |
Beijing |
1500 |
2022-07-25 17:45:00 |
www.aaa.com |
Tianjin |
500 |
2022-07-25 17:45:00 |
TABLE 6
Then, according to the reporting time, matching the collected QPS of each client in each acceleration area to a corresponding time period, for example, an early peak time period or a late peak time period, and updating the peak value QPS of each client in different time periods of each acceleration area, wherein when updating, if the current QPS is larger than the current peak value QPS, the current peak value QPS=the current QPS value; otherwise, the data is unchanged. For example, the results shown in the following table 7 can be obtained.
Customer domain name |
Acceleration region |
Today's early-to-early peak QPS |
Today's evening peak QPS |
www.aaa.com |
Beijing |
5000 |
8000 |
www.bbb.com |
Beijing |
3000 |
9500 |
www.aaa.com |
Tianjin |
1500 |
6000 |
TABLE 7
Finally, resource scheduling is performed according to the peak QPS usage of the last 2 days of clients (i.e., historical peak QPS) and the resource planning information of the last 2 days of clients, so that the peak of the peak QPS of today and yesterday needs to be calculated. At 0 point per day, the peak value of each time period today is updated to the peak value QPS data of the corresponding time period yesterday, and the peak value QPS today is recalculated based on the real-time QPS, so that the data shown in the following table 8 can be obtained.
TABLE 8
In step 304, from the historical peak QPS of the guest access requests of the N-day-last clients in different periods in each acceleration area, the historical peak QPS of the guest access requests of the N-day-last target clients in different periods in the target acceleration area is obtained.
According to the embodiment of the application, the historical peak value QPS of the visitor access request of the target client in the target acceleration area in the last N days in different time periods can be obtained from the historical peak value QPS of the visitor access request of the client in the different time periods in each acceleration area in the last N days according to the client domain name of the target client and the area identification of the target acceleration area.
Therefore, in the embodiment of the application, the time of each day can be distinguished into time periods (for example, in the morning and evening), the peak value data of the request number of the current time period is calculated according to the visitor access request number of the client in each acceleration area and is used as the request number demand in the time period, so that frequent switching of frequent service is avoided when the resource scheduling is performed for the client.
In another embodiment of the present application, for the clients that will be affected after the server group is monitored to run up, a directed acyclic graph may be constructed according to the correlation of the planned resources, and then the topology ordering of the directed acyclic graph is used for parallel scheduling, so as to improve the efficiency of resource scheduling, and correspondingly, as shown in fig. 4, step 203 includes the following steps: step 401 and step 402;
In step 401, a directed acyclic graph is constructed according to the resource planning information, wherein each node in the directed acyclic graph corresponds to a scheduled task of a target client in a target acceleration region.
In the embodiment of the application, a directed acyclic graph can be constructed according to the priority level of the target client, the client domain name and the acceleration region granularity.
In one example, a directed acyclic graph is constructed as shown in FIG. 5 from the priority level of the target client and the client domain name and acceleration region granularity, where each node corresponds to a scheduled task and the client domain name and acceleration region to which the scheduled task belongs. As can be seen from fig. 5: task 1 and task 2 may be performed in parallel, task 3 depends on task 1 and task 2, task 4 depends on task 3, task 5 and task 6 both depend on task 4, task 5 and task 6 may be performed in parallel, task 7 depends on task 5 and task 6, and task 8 depends on task 7. For ease of description, the directed acyclic graph shown in FIG. 5 can be converted to the representation shown in FIG. 6, wherein node i in FIG. 6 corresponds to the node of task i in FIG. 5.
In step 402, CDN resource scheduling is performed according to the historical peak QPS and the topological ordering of the nodes in the directed acyclic graph.
In the embodiment of the application, task execution can be performed according to the topological ordering of each node in the directed acyclic graph.
In one example, for the directed acyclic graph shown in fig. 6, tasks corresponding to the node 1 and the node 2 are executed at the same time according to the topological order, tasks corresponding to the node 3 are executed after the task corresponding to the node 1 and the node 2 are executed, tasks corresponding to the node 4 are executed after the task corresponding to the node 3 is executed, tasks corresponding to the node 5 and the node 6 are executed at the same time after the task corresponding to the node 4 is executed, tasks corresponding to the node 7 are executed after the task corresponding to the node 5 and the task corresponding to the node 6 are executed, and tasks corresponding to the node 8 are executed after the task corresponding to the node 7 is executed.
In the embodiment of the application, in the process of executing each task according to the topological order, the maximum value of the number of visitor access requests of each target server group in the target acceleration area is estimated according to the historical peak value QPS, the target server group with insufficient QPS resources when executing each task is determined according to the estimated maximum value, and the QPS resources are timely added for the target server group.
Therefore, in the embodiment of the application, the request number of the client in the acceleration area can be combined, for the client with high request number, a directed acyclic graph is built at the granularity of the client and the acceleration area, then the client is scheduled in parallel by using topological sequencing, the intersection of resources under the client is avoided, the scheduling result is influenced, the resource scheduling efficiency is improved, and the method is suitable for the scene of a plurality of visitor access request numbers but small bandwidth occupation.
In another embodiment provided by the application, after the directed acyclic graph is constructed and completed, tasks are scheduled in parallel by using topological sorting, the task with the degree of incidence of 0 can be selected for execution first, the degree of incidence of the corresponding task is reduced by 1, the steps are repeated until no task with the degree of incidence of 0 is executed, and the completion of the tasks is indicated, so that the tasks without correlation can be executed in parallel, and the scheduling efficiency is improved. Accordingly, as shown in fig. 7, the step 402 may include the following steps: step 701, step 702, step 703, step 704 and step 705;
In step 701, starting from a target node with a degree of entry 0 in the directed acyclic graph, calculating a historical average of visitor access request numbers of each target server group of target clients corresponding to the target node in a target acceleration area according to a historical peak value QPS, wherein the historical average=the historical peak value QPS/the number of target server groups.
In some embodiments, a target period to which the current time belongs may be determined from a target node with an entry degree of 0 in the directed acyclic graph, and a historical average value of visitor access request numbers of each target server group of target clients corresponding to the target node in the target acceleration area is calculated according to a historical peak value QPS corresponding to the target period.
For example, the request number history peak qps=max (today peak QPS, yesterday peak QPS) of the current task, and the visitor access request number history average value of each server group under the current task=the request number history peak QPS of the current task/the number of server groups in the current task home acceleration area.
In step 702, for a target server group having an upper limit QPS less than the historical average, a second server is selected from the resource pool of the target client, and a first server in the target server group is replaced with the second server.
In the embodiment of the application, for the server group with insufficient QPS, one server can be selected from the resource pool of the client to try to meet the current client requirement.
In step 703, for the target server group whose upper limit QPS is still smaller than the history average after the second server is replaced, selecting a third server from the resource pool of the target client, and adding the third server to the target server group until the upper limit QPS of the target server group is greater than the history average.
In the embodiment of the application, for the situation that the replacement resource cannot be met, a server is tried to be added until the requirement of a client in the acceleration area is met, and the resource addition is stopped.
In step 704, after the execution of the scheduling task corresponding to the target node is completed, the target node in the directed acyclic graph is deleted, and the degree of penetration of other nodes in the directed acyclic graph is reduced by 1.
In the embodiment of the application, after the execution of the current task is completed, the degree of invasiveness of the subsequent task is reduced by 1, and the current task is deleted from the directed acyclic graph.
In step 705, the above resource scheduling process is repeated for other nodes in the directed acyclic graph until the number of nodes in the directed acyclic graph is 0.
In the embodiment of the present application, the steps 701 to 704 may be repeated until there is no task with an input degree of 0, which indicates that all tasks scheduled at this time are executed.
In one example, as shown in fig. 8, for the directed acyclic graph shown in fig. 6, since the degree of entry of the node 1 and the node is 0 according to the topological order, tasks corresponding to the node 1 and the node 2 are executed simultaneously first, after the execution of the tasks corresponding to the node 1 and the node 2 is completed, the node 1 and the node 2 are deleted, and the degree of entry of each of the node 3 to the node 8 is reduced by 1. Since the degree of entry of the node 3 is 0 at this time, the task corresponding to the node 3 is continuously executed, and so on until the task corresponding to the node 8 is executed.
In the embodiment of the application, the tasks are scheduled in parallel by using the topological order of the directed acyclic graph, the task with the degree of 0 is selected for execution, the degree of the corresponding task is reduced by 1, and the steps are repeated until no task with the degree of 0 is left, which indicates that the task is executed, so that the tasks can be executed in parallel, and resources can be rapidly scheduled for clients with high running of the server group.
In another embodiment of the present application, a CDN resource scheduling system may also be provided, as shown in fig. 9, where the CDN resource scheduling system may include: the system comprises an IP section database, a basic data configuration module, a real-time data acquisition module, a data analysis module and a scheduling decision module; the IP section database is used for mainly storing relations among operators, IP sections and ASNs, and can be provided by a third party.
And the basic data configuration module is used for maintaining the edge server information, the server group information, the client domain name information and the resource planning information of the client in each acceleration area.
And the real-time data acquisition module is used for acquiring the real-time request number data of the edge server, summarizing the request number of each client on the corresponding acceleration area at fixed time, and transmitting the summarized data to the data analysis module.
The data analysis module is used for calculating the data reported by the real-time data acquisition module according to time intervals, finally calculating to obtain the peak value data of the request number of different clients in each acceleration area at different time intervals, and pushing the data result to the scheduling decision module.
And the scheduling decision module is used for combining the request number of the clients in the acceleration region, determining the clients with high request number, constructing a directed acyclic graph according to the granularity of the clients and the acceleration region, and scheduling the clients in parallel by using topological sequencing.
Therefore, in the embodiment of the application, when the scheduling system schedules resources for the acceleration area corresponding to the client, bandwidth is not considered any more, and the request number requirement is combined, specifically, visitor request number data of the edge server is collected in real time, the request number data summarize the data of the client in each acceleration area, scheduling is performed based on the request number of the visitor, and when the bandwidth requirement of the client is small but the request number is large, node resources are replaced or added for the client, so that the service quality of the client is ensured under the scene that the visitor access request number is large but the bandwidth is small.
Corresponding to the above method embodiment, the present application further provides a CDN resource scheduling apparatus, where the CDN resource scheduling apparatus described below and the CDN resource scheduling method described above may be referred to correspondingly.
Fig. 10 is a schematic structural diagram of a CDN resource scheduling device according to an embodiment of the present application, and as shown in fig. 10, the CDN resource scheduling device 1000 may include: an acquisition module 1001, a first determination module 1002, a second determination module 1003, and a scheduling module 1004;
An obtaining module 1001, configured to obtain a real-time QPS of a server group of each edge node of the CDN;
A first determining module 1002, configured to determine, according to a real-time QPS and an upper limit QPS of each server group, a target server group with insufficient QPS, where the upper limit QPS is a maximum QPS supported by the server group;
A second determining module 1003, configured to determine, according to a correspondence between a server group and a client, a domain name of a target client corresponding to the target server group and a target acceleration area, where the acceleration area is obtained based on geographic location division;
A scheduling module 1004, configured to schedule CDN resources for the target client in the target acceleration region; the CDN resource schedule comprises at least one of the following: replacing a first server in the target server group with a second server, and adding a third server in the target server; the upper limit QPS of the second server is greater than the upper limit QPS of the first server.
As can be seen from the above embodiment, in this embodiment, a real-time QPS of a server group of each edge node of the CDN is obtained; determining a target server group with insufficient QPS according to the real-time QPS and the upper limit QPS of each server group, wherein the upper limit QPS is the maximum QPS supported by the server group; determining a domain name and a target acceleration region of a target client corresponding to the target server group according to the corresponding relation between the server group and the client, wherein the acceleration region is obtained based on geographic position division; performing CDN resource scheduling for target clients in a target acceleration area; wherein, the CDN resource scheduling includes at least one of the following: replacing a first server in the target server group with a second server, and adding a third server in the target server; the upper limit QPS of the second server is greater than the upper limit QPS of the first server.
Therefore, in the embodiment of the application, when CDN resources are scheduled for clients, CDN manufacturers do not only consider bandwidth factors of CDN nodes, but also synthesize the characteristics of the clients, the visitor access request number data of the CDN edge nodes are monitored in real time for the clients with large visitor access request concurrency but less obvious bandwidth consumption, and when the sudden increase of the visitor access request number is monitored, node resources are replaced or added for the clients in time for the edge nodes with insufficient resources, so that the timely response to the visitor access requests is realized, and the service quality of the clients is improved.
Optionally, as an embodiment, the scheduling module 1004 may include:
A first obtaining sub-module, configured to obtain a historical peak value QPS of a visitor access request of the target client in the target acceleration area;
A second obtaining sub-module, configured to obtain resource planning information of the target client in the target acceleration area, where the resource planning information includes: a server group used and a priority of use of the server group;
And the scheduling sub-module is used for carrying out CDN resource scheduling for the target client in the target acceleration area according to the historical peak value QPS and the resource planning information.
Optionally, as an embodiment, the first obtaining sub-module may include:
A first obtaining unit, configured to obtain a historical peak value QPS of a visitor access request of the target client in different periods of time in the target acceleration area in the last N days, where N is an integer greater than 1;
The second obtaining sub-module may include: and the second acquisition unit is used for acquiring the resource planning information of the target client in the target acceleration area in the last N days.
Alternatively, as an embodiment, the first obtaining unit may include:
A first obtaining subunit, configured to obtain a visitor access record of a server group of each edge node of the CDN, where the visitor access record includes visitor access request numbers of different visitor IPs for different customer domain names;
the summarizing subunit is used for summarizing the visitor access request number in the visitor access record to the acceleration area of each client according to the area to which the visitor IP belongs;
the first computing subunit is used for distinguishing the time period of the day time of the last N days and computing the historical peak value QPS of the visitor access requests of the clients in different time periods in each acceleration area according to the visitor access request number of the clients in each acceleration area in the last N days;
and the second acquisition subunit is used for acquiring the historical peak value QPS of the guest access request of the target client in the target acceleration area in the last N days from the historical peak value QPS of the guest access request of the guest in the different periods in each acceleration area in each last N days.
Optionally, as an embodiment, the scheduling sub-module may include:
the construction unit is used for constructing a directed acyclic graph according to the resource planning information, wherein each node in the directed acyclic graph corresponds to one scheduling task of the target client in the target acceleration area;
and the scheduling unit is used for carrying out CDN resource scheduling according to the historical peak value QPS and the topological ordering of each node in the directed acyclic graph.
Optionally, as an embodiment, the scheduling unit may include:
A second calculating subunit, configured to calculate, from a target node with an input degree of 0 in the directed acyclic graph, a historical average value of visitor access request numbers of each target server group of the target clients corresponding to the target node in the target acceleration area according to the historical peak value QPS, where the historical average value = the historical peak value QPS/the number of target server groups;
a first scheduling subunit, configured to, for the target server group with an upper limit QPS smaller than the history average, select a second server from the resource pool of the target client, and replace a first server in the target server group with the second server;
A second scheduling subunit, configured to, for the target server group whose upper limit QPS is still smaller than the history average after the second server is replaced, select a third server from the resource pool of the target client, and add the third server to the target server group until the upper limit QPS of the target server group is greater than the history average;
The updating subunit is used for deleting the target node in the directed acyclic graph after the execution of the scheduling task corresponding to the target node is completed, and subtracting 1 from the degree of penetration of other nodes in the directed acyclic graph;
And the third scheduling subunit is used for repeating the resource scheduling process for other nodes in the directed acyclic graph until the number of the nodes in the directed acyclic graph is 0.
Optionally, as an embodiment, the calculating subunit is specifically configured to determine, from a target node with an input degree of 0 in the directed acyclic graph, a target period to which the current time belongs, and calculate, according to a history peak QPS corresponding to the target period, a history average value of a visitor access request number of each target client of the target client group in the target acceleration area corresponding to the target node.
Optionally, as an embodiment, the first determining module 1002 may include:
The third acquisition sub-module is used for acquiring redundancy parameters, the value of the redundancy parameters is between 0 and 1, and the redundancy parameters are used for preventing the number of visitor access requests on the server group from exceeding;
A calculation sub-module, configured to calculate a redundancy QPS for each server group according to the real-time QPS, the upper limit QPS, and the redundancy parameter of each server group, where redundancy qps=upper limit QPS, and redundancy parameter-real-time QPS;
And the determining submodule is used for determining the server group with the redundancy QPS smaller than 0 as a target server group with insufficient QPS.
The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.
Corresponding to the above method embodiment, the embodiment of the present application further provides an electronic device, including: a memory for storing a computer program; and the processor is used for realizing the steps of the CDN resource scheduling method when executing the computer program.
As shown in fig. 11, which is a schematic diagram of a composition structure of an electronic device, the electronic device may include: processor 1100, memory 1200, communication interface 1300, and communication bus 1400. The processor 1100, the memory 1200, and the communication interface 1300 all communicate with each other via the communication bus 1400.
In an embodiment of the present application, the processor 1100 may be a central processing unit (Central Processing Unit, CPU), an asic, a dsp, a field programmable gate array, or other programmable logic device, etc. The processor 1100 may call a program stored in the memory 1200, and in particular, the processor 1100 may perform operations in an embodiment of a CDN resource scheduling method. The memory 1200 is used for storing one or more programs, and the programs may include program codes, where the program codes include computer operation instructions, and in the embodiment of the present application, at least the programs for implementing the following functions are stored in the memory 1200:
Acquiring real-time QPS of a server group of each edge node of the CDN; determining a target server group with insufficient QPS according to the real-time QPS and the upper limit QPS of each server group; determining a domain name and a target acceleration region of a target client corresponding to the target server group according to the corresponding relation between the server group and the client; CDN resource scheduling is carried out for target clients in the target acceleration area; the CDN resource schedule includes at least one of: replacing a first server in a target server group with a second server, and adding a third server in the target server; the upper limit QPS of the second server is greater than the upper limit QPS of the first server.
In one possible implementation, the memory 1200 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, and at least one application program required for functions, etc.; the storage data area may store data created during use. In addition, memory 1200 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device or other volatile solid-state storage device. The communication interface 1300 may be an interface of a communication module for connecting with other devices or systems. Of course, it should be noted that the structure shown in fig. 11 is not limited to the electronic device in the embodiment of the present application, and the electronic device may include more or less components than those shown in fig. 11 or may be combined with some components in practical applications.
Corresponding to the above method embodiments, the present application further provides a computer readable storage medium, where a computer program is stored, where the computer program when executed by a processor implements the steps of the CDN resource scheduling method described above.
In addition, it should be noted that: embodiments of the present application also provide a computer program product or computer program that may include computer instructions that may be stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor may execute the computer instructions, so that the computer device performs the description of the CDN resource scheduling method in the foregoing corresponding embodiment, and therefore, a detailed description will not be given here. In addition, the description of the beneficial effects of the same method is omitted. For technical details not disclosed in the computer program product or the computer program embodiments according to the present application, reference is made to the description of the method embodiments according to the present application.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, so that the same or similar parts between the embodiments are referred to each other. Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The principles and embodiments of the present application have been described herein with reference to specific examples, but the description of the examples above is only for aiding in understanding the technical solution of the present application and its core ideas. It should be noted that it will be apparent to those skilled in the art that various modifications and adaptations of the application can be made without departing from the principles of the application and these modifications and adaptations are intended to be within the scope of the application as defined in the following claims.