CN110581873B - Cross-cluster redirection method and monitoring server - Google Patents
Cross-cluster redirection method and monitoring server Download PDFInfo
- Publication number
- CN110581873B CN110581873B CN201810593576.1A CN201810593576A CN110581873B CN 110581873 B CN110581873 B CN 110581873B CN 201810593576 A CN201810593576 A CN 201810593576A CN 110581873 B CN110581873 B CN 110581873B
- Authority
- CN
- China
- Prior art keywords
- cluster
- resource
- target
- monitoring server
- cache
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/50—Testing arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Transfer Between Computers (AREA)
Abstract
本发明实施例提供了一种跨集群重定向方法,包括:若检测到目标资源在第一集群中未被缓存,则向云缓存中除所述第一集群外的各个集群发送资源检索指令,以确定包含所述目标资源的目标集群;将所述目标集群对应缓存服务器的IP地址以及所述目标资源在所述目标集群对应缓存服务器磁盘上的存储路径,通过302重定向消息发送给终端。本发明实施例提供的一种跨集群重定向方法,通过将集群进行云缓存,并在云缓存中进行跨集群重定向,可以有效减少监控进程启动过程中的资源加载时间,降低监控服务器内存资源消耗,并且可以有效减少各集群管理服务器业务进程和数据库的压力。
An embodiment of the present invention provides a cross-cluster redirection method, comprising: if it is detected that a target resource is not cached in a first cluster, sending a resource retrieval instruction to each cluster in the cloud cache except the first cluster, to determine the target cluster including the target resource; send the IP address of the cache server corresponding to the target cluster and the storage path of the target resource on the disk of the cache server corresponding to the target cluster to the terminal through 302 redirection messages. The cross-cluster redirection method provided by the embodiment of the present invention can effectively reduce the resource loading time in the process of starting the monitoring process and reduce the memory resources of the monitoring server by performing cloud caching on the cluster and performing cross-cluster redirection in the cloud cache. consumption, and can effectively reduce the pressure on the business process and database of each cluster management server.
Description
技术领域technical field
本发明实施例涉及互联网通信技术领域,尤其涉及一种跨集群重定向方法及监控服务器。Embodiments of the present invention relate to the technical field of Internet communications, and in particular, to a cross-cluster redirection method and a monitoring server.
背景技术Background technique
缓存集群监控服务器通过旁路分光方式监控链路中的HTTP GET和HEAD请求,当用户请求的资源类型可缓存,并且请求次数达到预设阀值时,监控服务器将通知集群管理服务器,集群管理服务器通知本集群内的特定缓存服务器进行资源回源,然后把资源的url和文件名等信息记录在管理服务器的mysql数据库中。管理服务器在资源更新后,会把资源记录等信息同步到本集群内所有监控服务器,用户在访问这些在缓存中已有的资源时,监控服务器将发送302重定向消息引导中断用户到缓存来获取资源。The cache cluster monitoring server monitors HTTP GET and HEAD requests in the link through bypass splitting. When the resource type requested by the user can be cached and the number of requests reaches the preset threshold, the monitoring server will notify the cluster management server and the cluster management server. Notify the specific cache server in the cluster to return the resource to the source, and then record the information such as the URL and file name of the resource in the mysql database of the management server. After the resource is updated, the management server will synchronize the resource records and other information to all monitoring servers in the cluster. When users access these existing resources in the cache, the monitoring server will send a 302 redirect message to guide the interrupted user to the cache to obtain it. resource.
针对现网用户数量多,出口链路规模大,并且出口链路分布在多个机房的特点,目前采用不同出口链路由特定缓存集群服务的策略,目的是增加系统部署的灵活性和链路扩容的灵活性。在多缓存集群并存情况下,针对不同地域用户的同一资源请求可能走不同的省网和第三方出口链路,进而被不同集群的监控服务器所监控,从而造成多集群从公网多次重复回源同一资源,造成出口带宽的重复占用,从而影响到缓存增益比指标。In view of the large number of users on the existing network, the large scale of egress links, and the distribution of egress links in multiple computer rooms, the strategy of using different egress links to be served by a specific cache cluster is to increase the flexibility of system deployment and links Expansion flexibility. When multiple cache clusters coexist, the same resource requests for users in different regions may go through different provincial networks and third-party egress links, and then be monitored by monitoring servers in different clusters, resulting in multiple clusters returning from the public network multiple times. The source of the same resource will cause the repeated occupation of the egress bandwidth, thus affecting the cache gain ratio indicator.
因此现在亟需一种跨集群重定向方法来解决上述问题。Therefore, a cross-cluster redirection method is urgently needed to solve the above problems.
发明内容SUMMARY OF THE INVENTION
为了解决上述问题,本发明实施例提供一种克服上述问题或者至少部分地解决上述问题的跨集群重定向方法及监控服务器。In order to solve the above problem, the embodiments of the present invention provide a cross-cluster redirection method and a monitoring server that overcome the above problem or at least partially solve the above problem.
第一方面本发明实施例提供一种跨集群重定向方法,包括:若检测到目标资源在第一集群中未被缓存,则向云缓存中除所述第一集群外的各个集群发送资源检索指令,以确定包含所述目标资源的目标集群;Aspect 1 An embodiment of the present invention provides a cross-cluster redirection method, including: if it is detected that a target resource is not cached in a first cluster, sending resource retrieval to each cluster in the cloud cache except the first cluster instructions to determine a target cluster containing the target resource;
将所述目标集群对应缓存服务器的IP地址以及所述目标资源在所述目标集群对应缓存服务器磁盘上的存储路径,通过302重定向消息发送给终端。Send the IP address of the cache server corresponding to the target cluster and the storage path of the target resource on the disk of the cache server corresponding to the target cluster to the terminal through 302 redirection messages.
第二方面本发明实施例还提供了一种监控服务器,包括:In the second aspect, the embodiment of the present invention further provides a monitoring server, including:
确定模块,用于若检测到目标资源在第一集群中未被缓存,则向云缓存中除所述第一集群外的各个集群发送资源检索指令,以确定包含所述目标资源的目标集群;a determining module, configured to send a resource retrieval instruction to each cluster in the cloud cache except the first cluster if it is detected that the target resource is not cached in the first cluster, so as to determine the target cluster containing the target resource;
重定向模块,用于将所述目标集群对应缓存服务器的IP地址以及所述目标资源在所述目标集群对应缓存服务器磁盘上的存储路径,通过302重定向消息发送给终端。The redirection module is configured to send the IP address of the cache server corresponding to the target cluster and the storage path of the target resource on the disk of the cache server corresponding to the target cluster to the terminal through 302 redirection messages.
第三方面本发明实施例提供了一种监控服务器设备,包括:In a third aspect, an embodiment of the present invention provides a monitoring server device, including:
处理器、存储器、通信接口和总线;其中,所述处理器、存储器、通信接口通过所述总线完成相互间的通信;所述存储器存储有可被所述处理器执行的程序指令,所述处理器调用所述程序指令能够执行上述所述的一种跨集群重定向方法。a processor, a memory, a communication interface, and a bus; wherein, the processor, memory, and communication interface communicate with each other through the bus; the memory stores program instructions that can be executed by the processor, and the processing The above-mentioned method for cross-cluster redirection can be executed by invoking the program instruction by the server.
第四方面本发明实施例提供了一种非暂态计算机可读存储介质,所述非暂态计算机可读存储介质存储计算机指令,所述计算机指令使所述计算机执行上述方法。Fourth aspect An embodiment of the present invention provides a non-transitory computer-readable storage medium, where the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions cause the computer to execute the above method.
本发明实施例提供的一种跨集群重定向方法,通过将集群进行云缓存,并在云缓存中进行跨集群重定向,可以有效减少监控进程启动过程中的资源加载时间,降低监控服务器内存资源消耗,并且可以有效减少各集群管理服务器业务进程和数据库的压力。The cross-cluster redirection method provided by the embodiment of the present invention can effectively reduce the resource loading time in the process of starting the monitoring process and reduce the memory resources of the monitoring server by performing cloud caching on the cluster and performing cross-cluster redirection in the cloud cache. consumption, and can effectively reduce the pressure on the business process and database of each cluster management server.
附图说明Description of drawings
为了更清楚地说明本发明实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍,显而易见地,下面描述中的附图是本发明的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.
图1是本发明实施例提供的一种跨集群重定向方法流程示意图;1 is a schematic flowchart of a cross-cluster redirection method provided by an embodiment of the present invention;
图2是本发明实施例提供的一种监控服务器结构图;2 is a structural diagram of a monitoring server provided by an embodiment of the present invention;
图3是本发明实施例提供的监控服务器设备的结构框图。FIG. 3 is a structural block diagram of a monitoring server device provided by an embodiment of the present invention.
具体实施方式Detailed ways
为使本发明实施例的目的、技术方案和优点更加清楚,下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚地描述,显然,所描述的实施例是本发明一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本发明保护的范围。In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are the Some, but not all, embodiments are disclosed. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
目前,现有技术在进行重定向的过程中,主要存在以下两个问题:At present, in the process of redirection in the prior art, there are mainly the following two problems:
1、各集群均是周期性计算本集群中的热点资源,这种热点资源的计算方式增大了数据库的资源开销和管理进程的负荷。缓存集群管理服务器的数据库记录了已缓存资源文件的大小、创建时间、最近服务时间、服务连接次数、服务流量等信息,这些文件资源的记录数量通常达到几千万条。1. Each cluster periodically calculates the hot-spot resources in the cluster. This way of calculating the hot-spot resources increases the resource overhead of the database and the load of the management process. The database of the cache cluster management server records information such as the size, creation time, recent service time, number of service connections, and service traffic of cached resource files. The number of records of these file resources usually reaches tens of millions.
2、在进程重启后,监控服务器除去加载本集群内的资源外,还要加载大量的其它集群的热点资源,对监控服务器的内存容量提出了较高要求,并且延长了资源加载时间,并且加载到内存中的资源记录频繁更新,将影响到重定向的效率。2. After the process is restarted, the monitoring server not only loads the resources in this cluster, but also loads a large number of hot resources of other clusters, which puts forward higher requirements on the memory capacity of the monitoring server, and prolongs the resource loading time. Frequent updates of resource records to memory will affect the efficiency of redirection.
针对上述问题,图1是本发明实施例提供的一种跨集群重定向方法流程示意图,如图1所示,包括:In view of the above problems, FIG. 1 is a schematic flowchart of a cross-cluster redirection method provided by an embodiment of the present invention, as shown in FIG. 1 , including:
110、若检测到目标资源在第一集群中未被缓存,则向云缓存中除所述第一集群外的各个集群发送资源检索指令,以确定包含所述目标资源的目标集群;110. If it is detected that the target resource is not cached in the first cluster, send a resource retrieval instruction to each cluster in the cloud cache except the first cluster to determine the target cluster containing the target resource;
120、将所述目标集群对应缓存服务器的IP地址以及所述目标资源在所述目标集群对应缓存服务器磁盘上的存储路径,通过302重定向消息发送给终端。120. Send the IP address of the cache server corresponding to the target cluster and the storage path of the target resource on the disk of the cache server corresponding to the target cluster to the terminal through 302 a redirection message.
需要说明的是,本发明实施例的执行主体为第一集群中的一个监控服务器,该监控服务器接收到终端发送的目标资源检索请求后而第一集群中没有目标资源时,则需要使用本发明实施例提供的跨集群重定向方案在其他集群中获取目标资源。It should be noted that the execution body of the embodiment of the present invention is a monitoring server in the first cluster. After the monitoring server receives the target resource retrieval request sent by the terminal and there is no target resource in the first cluster, the present invention needs to be used. The cross-cluster redirection scheme provided by the embodiment acquires target resources in other clusters.
还需要说明的是,本发明实施例提供的跨集群重定向方案适用于部署多个集群的省份,由于各个集群部署在同一省内,机房之间设备通信经过的路由跳数少,网络出口机房之间或有直连链路相连,带宽和通信质量有保证。监控服务器都配置公网IP地址,不同集群之间的服务器可以互访,网路延迟在毫秒级,为不同集群之间监控服务器的内存资源检索提供了可能。It should also be noted that the cross-cluster redirection solution provided by the embodiment of the present invention is suitable for provinces where multiple clusters are deployed. Since each cluster is deployed in the same province, the number of routing hops through which equipment communication between computer rooms passes is small, and the network exits the computer room. There may be direct links between them, and the bandwidth and communication quality are guaranteed. The monitoring servers are all configured with public IP addresses, and servers between different clusters can access each other. The network delay is in the millisecond level, which provides the possibility of memory resource retrieval of monitoring servers between different clusters.
在步骤110中,所述云缓存为包括第一集群在内的所有集群所构成的云端缓存集合,云缓存可以很好解决同一资源在多个集群重复回源的问题。各个集群开启云缓存共享功能后,各个集群的管理服务器定期计算本集群内的热点资源,然后把这些热点资源定期同步到其它集群。监控服务器定期加载其它集群的热点资源,当用户访问的资源在本集群内没有缓存,但是在其它集群已经缓存的情况下,监控服务器将通知本集群内的一台缓存服务器,代理该用户请求到其它集群的缓存服务器获取资源,同时在本地保留文件副本。In
需要说明的是,当本集群内对该特定资源访问次数达到云缓存的触发阈值后,该资源将被记录的本集群的数据库中,变为本地缓存的资源。It should be noted that when the number of accesses to the specific resource in the cluster reaches the trigger threshold of the cloud cache, the resource will be recorded in the database of the cluster and become a locally cached resource.
在步骤110中,所述资源检索为当本集群中没有缓存目标资源,而其他集群中缓存了所述目标资源时,到其他集群中去检索哪个集群能够提供该目标资源的过程。通过资源检索能够从云缓存包括的多个集群中确定出能够提供目标资源的最优集群,并将该最优集群作为目标集群。In
进一步的,在步骤110中确定了目标集群之后,那么在步骤120中可以根据确定的目标集群中目标资源的存储确定缓存服务器的IP地址以及目标资源在所述目标集群对应缓存服务器磁盘上的存储路径,然后将这些信息以302重定向消息的方式发送给终端。从而终端可以直接从目标集群的缓存服务器上获取资源。Further, after the target cluster is determined in
本发明实施例提供的一种跨集群重定向方法,通过将集群进行云缓存,并在云缓存中进行跨集群重定向,可以有效减少监控进程启动过程中的资源加载时间,降低监控服务器内存资源消耗,并且可以有效减少各集群管理服务器业务进程和数据库的压力。The cross-cluster redirection method provided by the embodiment of the present invention can effectively reduce the resource loading time in the process of starting the monitoring process and reduce the memory resources of the monitoring server by performing cloud caching on the cluster and performing cross-cluster redirection in the cloud cache. consumption, and can effectively reduce the pressure on the business process and database of each cluster management server.
在上述实施例的基础上,所述若检测到目标资源在第一集群中未被缓存,则向云缓存中除所述第一集群外的各个集群发送资源检索指令,以确定包含所述目标资源的目标集群,包括:Based on the above embodiment, if it is detected that the target resource is not cached in the first cluster, a resource retrieval instruction is sent to each cluster in the cloud cache except the first cluster, so as to determine whether the target resource contains the target resource. The target cluster for the resource, including:
加载所述第一集群的资源目录,并在所述第一集群的资源目录中检测是否含有所述目标资源;Loading the resource directory of the first cluster, and detecting whether the target resource is contained in the resource directory of the first cluster;
若在所述第一集群的资源目录中未检测到所述目标资源,则向云缓存中除所述第一集群外的各个集群对应的监控服务器发送资源检索指令,以确定包含所述目标资源的目标监控服务器;If the target resource is not detected in the resource directory of the first cluster, send a resource retrieval instruction to the monitoring server corresponding to each cluster except the first cluster in the cloud cache, so as to determine whether the target resource contains the target resource target monitoring server;
根据确定的目标监控服务器确定所述目标集群。The target cluster is determined according to the determined target monitoring server.
由上述实施例的内容可知,本发明实施例提供的跨集群重定向方法能够在云缓存中对其他集群缓存的内容进行检索,并确定含有目标资源的目标集群,再通过302重定向消息引导终端直接从目标集群进行目标资源获取。It can be seen from the content of the above embodiment that the cross-cluster redirection method provided by the embodiment of the present invention can retrieve the content cached by other clusters in the cloud cache, determine the target cluster containing the target resource, and then guide the terminal through the 302 redirection message. Obtain target resources directly from the target cluster.
其中,对于目标集群的具体确定过程为:只加载本集群,即第一集群的资源记录,而不再加载云缓存中其他集群的资源记录,那么对于在本集群中还未缓存但是在其它集群已经缓存的资源,第一集群的监控服务器就可以实时到其它集群的监控服务器进行资源检索,从而确定含有目标资源的目标监控服务器。Among them, the specific determination process for the target cluster is as follows: only the resource records of this cluster, that is, the first cluster are loaded, and the resource records of other clusters in the cloud cache are no longer loaded. For the resources that have been cached, the monitoring server of the first cluster can perform resource retrieval on the monitoring servers of other clusters in real time, so as to determine the target monitoring server containing the target resource.
可以理解的是,监控服务器和集群之间存在有对应关系,从而在确定了目标监控服务器后,相应的也可以确定目标集群。It can be understood that there is a corresponding relationship between the monitoring server and the cluster, so that after the target monitoring server is determined, the target cluster can also be determined accordingly.
在上述实施例的基础上,所述向云缓存中除所述第一集群外的各个集群对应的监控服务器发送资源检索指令,以确定包含所述目标资源的目标监控服务器,包括:On the basis of the above embodiment, the sending of resource retrieval instructions to the monitoring servers corresponding to each cluster except the first cluster in the cloud cache to determine the target monitoring server containing the target resource includes:
同时向云缓存中除所述第一集群外的各个集群对应的监控服务器发送资源检索指令,所述资源检索指令用于指示所述监控服务器在监控服务器的内存中进行目标资源检索;At the same time, send a resource retrieval instruction to the monitoring server corresponding to each cluster except the first cluster in the cloud cache, where the resource retrieval instruction is used to instruct the monitoring server to perform target resource retrieval in the memory of the monitoring server;
将第一个根据所述资源检索指令进行检索响应回复的监控服务器作为所述目标监控服务器。The first monitoring server that returns a retrieval response according to the resource retrieval instruction is used as the target monitoring server.
由上述实施例的内容可知,本发明实施例是在集群对应的监控服务器中进行的资源检索。具体的,本发明实施例提供的资源检索方案为第一集群的监控服务器同时向其他集群的监控服务器发送资源检索指令。该资源检索指令可以指示其他集群的监控服务器在监控服务器的内存中进行目标资源检索,而不必像现有技术一样到集群的管理服务器数据库中进行检索。It can be known from the contents of the above embodiments that the embodiments of the present invention are resource retrieval performed in the monitoring server corresponding to the cluster. Specifically, the resource retrieval solution provided by the embodiment of the present invention is that the monitoring server of the first cluster simultaneously sends resource retrieval instructions to the monitoring servers of other clusters. The resource retrieval instruction can instruct the monitoring servers of other clusters to perform target resource retrieval in the memory of the monitoring server, instead of searching from the database of the cluster management server as in the prior art.
并且,本发明实施例提供的方法在发送资源检索指令时采用一次性批量发出,目的是缩短资源检索次数,提高单次检索数量,进而提升资源检索性能。In addition, the method provided by the embodiment of the present invention adopts one-time batch issuance when sending resource retrieval instructions, which aims to shorten the number of resource retrievals, increase the number of single retrievals, and further improve resource retrieval performance.
需要说明的是,因为集群中包含的监控服务器数量多,数据加载在内存中,资源检索速度快并且资源检索被均衡到所有监控服务器。本发明实施例提供的在监控服务器内存中进行资源检索的方案避免了不同集群监控服务器到集群管理服务器数据库检索压力的同时,还能够提升检索效率。It should be noted that because the cluster contains a large number of monitoring servers, data is loaded in memory, resource retrieval is fast, and resource retrieval is balanced to all monitoring servers. The solution for retrieving resources in the memory of the monitoring server provided by the embodiment of the present invention avoids the retrieval pressure from different cluster monitoring servers to the cluster management server database, and can also improve retrieval efficiency.
进一步的,各个监控服务器在接收到资源检索指令后会检测资源记录中是否含有目标资源,若含有目标资源,则需要做出相应的检索响应回复,从而告知第一集群的监控服务器。那么,由于发送过程中是向所有集群同时进行了发送,但不同的集群可能进行响应的速度会不同,那么考虑到检索速度,本发明实施例优先将第一个收到的检索响应回复对应的监控服务器作为目标监控服务器,而接下来收到的检索响应回复将被舍弃。Further, after receiving the resource retrieval instruction, each monitoring server will detect whether the resource record contains the target resource, and if it contains the target resource, it needs to make a corresponding retrieval response reply, so as to inform the monitoring server of the first cluster. Then, since the sending process is sent to all clusters at the same time, but different clusters may respond at different speeds, considering the retrieval speed, the embodiment of the present invention preferentially replies the corresponding retrieval response to the first received retrieval response. The monitoring server is used as the target monitoring server, and the next retrieval response reply received will be discarded.
在上述实施例的基础上,所述将第一个根据所述资源检索指令进行检索响应回复的监控服务器作为所述目标监控服务器,还包括:On the basis of the above embodiment, the first monitoring server that responds to the retrieval response according to the resource retrieval instruction as the target monitoring server, further includes:
若在预设时长内,没有接收到任意一个监控服务器根据所述资源检索指令进行检索响应回复的信息,则发送资源请求获取失败的信息给终端。If no information is received from any monitoring server that responds to the retrieval according to the resource retrieval instruction within the preset time period, the information that the resource request fails to be acquired is sent to the terminal.
由上述实施例的内容可知,本发明实施例提供的方案监控服务器需要根据资源检索指令进行检索响应回复。考虑到资源检索的时效性,本发明实施例在监控服务器发送资源检索指令时,会启动定时器进行计时,在规定时间内没有收到其它集群监控服务器的检索响应回复,则认为用户请求的资源在所有缓存集群中都没有缓存,监控服务器把用户对该资源的请求告知管理服务器,由管理服务器统一记录该资源的访问次数,同时监控服务器放弃引导终端重定向获取资源的动作。It can be seen from the contents of the above-mentioned embodiments that the solution monitoring server provided by the embodiments of the present invention needs to reply to a retrieval response according to a resource retrieval instruction. Considering the timeliness of resource retrieval, in this embodiment of the present invention, when the monitoring server sends a resource retrieval instruction, a timer is started to count the time, and if no retrieval response reply from other cluster monitoring servers is received within a specified time, it is considered that the resource requested by the user is considered. There is no cache in all cache clusters. The monitoring server informs the management server of the user's request for the resource, and the management server records the access times of the resource uniformly.
在上述实施例的基础上,在所述云缓存中每个集群对应一个或多个监控服务器,且每个集群对应的每个监控服务器加载的资源记录相同。On the basis of the above embodiment, each cluster in the cloud cache corresponds to one or more monitoring servers, and the resource records loaded by each monitoring server corresponding to each cluster are the same.
需要说明的是,本发明实施例预先对各个监控服务器进行了分组,分组标准按照集群进行,即同一集群内部的所有监控服务器分配在同一组中。It should be noted that, in the embodiment of the present invention, each monitoring server is grouped in advance, and the grouping standard is performed according to the cluster, that is, all the monitoring servers in the same cluster are allocated to the same group.
那么由于同一集群内部所有监控服务器加载的资源记录完全一致,从而进行资源检索时只需要到每个集群中的特定一台监控服务器的内存中进行检索,例如:在有N个集群组成的云缓存系统中,监控服务器只需要到N-1台监控服务器上进行资源检索,从而使不同集群之间的资源检索开销达到最低。Then, since the resource records loaded by all monitoring servers in the same cluster are completely consistent, the resource retrieval only needs to be performed in the memory of a specific monitoring server in each cluster. For example, in a cloud cache consisting of N clusters In the system, the monitoring server only needs to perform resource retrieval on N-1 monitoring servers, so as to minimize the resource retrieval overhead between different clusters.
由上述内容可知,在进行资源检索时,每个集群对应了特定的一台监控服务器,从而直接在特定的这台监控服务器中进行检索。但为避免部分监控服务器上获取的检索请求太多,而另外一部分监控服务器获取的检索请求太少问题,需要合理分配各个集群监控服务器上的请求负载。故而本发明实施例还提供一种一个集群对应多个监控服务器的方案。As can be seen from the above content, when retrieving resources, each cluster corresponds to a specific monitoring server, so that the retrieval can be performed directly in the specific monitoring server. However, in order to avoid the problem of too many retrieval requests obtained on some monitoring servers and too few retrieval requests obtained by other monitoring servers, it is necessary to reasonably distribute the request load on each cluster monitoring server. Therefore, the embodiment of the present invention also provides a solution in which one cluster corresponds to multiple monitoring servers.
为了便于描述,本发明实施例以一个集群对应两个监控服务器的方式进行说明,但本发明实施例不对集群与监控服务器的对应数量作具体限定。For ease of description, the embodiment of the present invention is described in a manner that one cluster corresponds to two monitoring servers, but the embodiment of the present invention does not specifically limit the corresponding number of clusters and monitoring servers.
本发明实施例将一个集群对应两个监控服务器,其中一个为主服务器,另一个为副服务器。例如:集群1的监控服务器1进行资源检索的目标服务器为集群2的监控服务器1和监控服务器2,集群2中的监控服务器1为主服务器,集群2中的监控服务器2为备服务器,在主服务器不能提供服务的请求下,由备用服务器提供服务。在集群1中的各监控服务器与集群2中对应的监控服务器之间提前建立TCP连接并进行连接复用,从而可以有效加快资源检索速度。In the embodiment of the present invention, one cluster corresponds to two monitoring servers, one of which is the primary server and the other is the secondary server. For example, the target servers for resource retrieval by monitoring server 1 of cluster 1 are monitoring server 1 and monitoring server 2 of cluster 2. Monitoring server 1 in cluster 2 is the primary server, and monitoring server 2 in cluster 2 is the standby server. If the server cannot provide the service, the service is provided by the alternate server. A TCP connection is established in advance between each monitoring server in the cluster 1 and the corresponding monitoring server in the cluster 2, and the connection is multiplexed, so that the resource retrieval speed can be effectively accelerated.
在上述实施例的基础上,所述方法还包括:On the basis of the above embodiment, the method further includes:
当一个集群对应多个监控服务器时,对于任意一个集群,若向集群对应的监控服务器发送资源检索指令失败,则通过预设的冗余路径将所述资源检索指令向所述集群对应的另一个监控服务器重新发送。When a cluster corresponds to multiple monitoring servers, for any cluster, if the sending of a resource retrieval command to the monitoring server corresponding to the cluster fails, the resource retrieval command will be sent to another cluster corresponding to the cluster through a preset redundant path. The monitoring server resends.
由上述实施例的内容可知,本发明实施例提供的集群可以对应一个或多个监控服务器。并且其中的一个作为主服务器,其他的监控服务器作为副服务器备选。那么当向主服务器发送资源检索指令失败时,需要从副服务器中选出一个新的主服务器重新发送资源检索指令。具体的,本发明实施例会通过建立服务器与发送端监控服务器之间的冗余路径进行实现。It can be known from the contents of the foregoing embodiments that the cluster provided by the embodiment of the present invention may correspond to one or more monitoring servers. And one of them is used as the primary server, and the other monitoring servers are used as secondary servers. Then, when sending the resource retrieval instruction to the primary server fails, a new primary server needs to be selected from the secondary servers to resend the resource retrieval instruction. Specifically, the embodiment of the present invention will be implemented by establishing a redundant path between the server and the sending end monitoring server.
在上述实施例的基础上,所述将所述目标集群对应缓存服务器的IP地址以及所述目标资源在所述目标集群对应缓存服务器磁盘上的存储路径,通过302重定向消息发送给终端,包括:On the basis of the above embodiment, the IP address of the cache server corresponding to the target cluster and the storage path of the target resource on the disk of the cache server corresponding to the target cluster are sent to the terminal through 302 redirection messages, including :
基于所述目标集群对应的目标监控服务器的资源监控记录,确定存储目标资源的缓存服务器;Determine the cache server that stores the target resource based on the resource monitoring record of the target monitoring server corresponding to the target cluster;
将所述缓存服务器的IP地址信息以及所述目标集群对应缓存服务器磁盘上的存储路径信息,通过302重定向消息发送给终端。Send the IP address information of the cache server and the storage path information on the disk of the cache server corresponding to the target cluster to the terminal through 302 redirection messages.
由上述实施例的内容可知,本发明实施例提供的第一集群的监控服务器不再加载其它集群的资源记录,而只加载第一集群的资源记录。针对在第一集群中还未缓存但是在其它集群已经缓存的资源,监控服务器实时到其它集群监控服务器进行资源检索,根据检索结果发送重定向消息,引导用户到其它集群拥有该资源的服务器上获取资源。对用户请求的引导不再局限于本集群内部,而是延伸到组成云缓存的各个集群。It can be known from the content of the above embodiments that the monitoring server of the first cluster provided by the embodiment of the present invention no longer loads the resource records of other clusters, but only loads the resource records of the first cluster. For resources that have not been cached in the first cluster but have been cached in other clusters, the monitoring server retrieves resources from other cluster monitoring servers in real time, sends a redirection message according to the retrieval results, and guides users to obtain the resources from the servers that own the resources in other clusters. resource. The guidance of user requests is no longer limited to the inside of the cluster, but extends to the various clusters that make up the cloud cache.
那么根据在目标监控服务器上的检索结果,就能够整合资源存储的服务器IP地址信息以及资源在磁盘上的存储路径,并将上述信息以302重定向消息发送给终端,其中,302重定向消息的Location字段的IP地址指向其它集群拥有该资源的缓存服务器IP地址,而不再是第一集群缓存服务器IP地址,从而实现跨集群重定向引导用户直接从其它集群的缓存服务器上获取资源。Then, according to the retrieval result on the target monitoring server, the server IP address information stored in the resource and the storage path of the resource on the disk can be integrated, and the above information is sent to the terminal in a 302 redirection message, wherein the 302 redirection message The IP address of the Location field points to the IP address of the cache server that owns the resource in other clusters, instead of the IP address of the first cluster cache server, so that cross-cluster redirection is implemented to guide users to obtain resources directly from the cache servers of other clusters.
图2是本发明实施例提供的一种监控服务器结构图,如图2所示,所述系统包括:确定模块210以及重定向模块220,其中:FIG. 2 is a structural diagram of a monitoring server provided by an embodiment of the present invention. As shown in FIG. 2, the system includes: a
确定模块210用于若检测到目标资源在第一集群中未被缓存,则向云缓存中除所述第一集群外的各个集群发送资源检索指令,以确定包含所述目标资源的目标集群;The determining
重定向模块220用于将所述目标集群对应缓存服务器的IP地址以及所述目标资源在所述目标集群对应缓存服务器磁盘上的存储路径,通过302重定向消息发送给终端。The
具体的如何通过确定模块210以及重定向模块220进行跨集群重定向可用于执行图1所示的跨集群重定向方法实施例的技术方案,其实现原理和技术效果类似,此处不再赘述。Specifically, how to perform cross-cluster redirection by the
本发明实施例提供的一种监控服务器,通过将集群进行云缓存,并在云缓存中进行跨集群重定向,可以有效减少监控进程启动过程中的资源加载时间,降低监控服务器内存资源消耗,并且可以有效减少各集群管理服务器业务进程和数据库的压力。The monitoring server provided by the embodiment of the present invention can effectively reduce the resource loading time in the process of starting the monitoring process and reduce the memory resource consumption of the monitoring server by performing cloud caching on the cluster and performing cross-cluster redirection in the cloud cache, and It can effectively reduce the pressure on the business process and database of each cluster management server.
本发明实施例提供一种监控服务器设备,包括:至少一个处理器;以及与所述处理器通信连接的至少一个存储器,其中:An embodiment of the present invention provides a monitoring server device, comprising: at least one processor; and at least one memory communicatively connected to the processor, wherein:
图3是本发明实施例提供的监控服务器设备的结构框图,参照图3,所述监控服务器设备,包括:处理器(processor)310、通信接口(Communications Interface)320、存储器(memory)330和总线340,其中,处理器310,通信接口320,存储器330通过总线340完成相互间的通信。处理器310可以调用存储器330中的逻辑指令,以执行如下方法:若检测到目标资源在第一集群中未被缓存,则向云缓存中除所述第一集群外的各个集群发送资源检索指令,以确定包含所述目标资源的目标集群;将所述目标集群对应缓存服务器的IP地址以及所述目标资源在所述目标集群对应缓存服务器磁盘上的存储路径,通过302重定向消息发送给终端。FIG. 3 is a structural block diagram of a monitoring server device provided by an embodiment of the present invention. Referring to FIG. 3 , the monitoring server device includes: a processor (processor) 310, a communications interface (Communications Interface) 320, a memory (memory) 330, and a
本发明实施例公开一种计算机程序产品,所述计算机程序产品包括存储在非暂态计算机可读存储介质上的计算机程序,所述计算机程序包括程序指令,当所述程序指令被计算机执行时,计算机能够执行上述各方法实施例所提供的方法,例如包括:若检测到目标资源在第一集群中未被缓存,则向云缓存中除所述第一集群外的各个集群发送资源检索指令,以确定包含所述目标资源的目标集群;将所述目标集群对应缓存服务器的IP地址以及所述目标资源在所述目标集群对应缓存服务器磁盘上的存储路径,通过302重定向消息发送给终端。An embodiment of the present invention discloses a computer program product, where the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by a computer, The computer can execute the methods provided by the above method embodiments, for example, including: if it is detected that the target resource is not cached in the first cluster, sending a resource retrieval instruction to each cluster in the cloud cache except the first cluster, to determine the target cluster including the target resource; send the IP address of the cache server corresponding to the target cluster and the storage path of the target resource on the disk of the cache server corresponding to the target cluster to the terminal through 302 redirection messages.
本发明实施例提供一种非暂态计算机可读存储介质,所述非暂态计算机可读存储介质存储计算机指令,所述计算机指令使所述计算机执行上述各方法实施例所提供的方法,例如包括:若检测到目标资源在第一集群中未被缓存,则向云缓存中除所述第一集群外的各个集群发送资源检索指令,以确定包含所述目标资源的目标集群;将所述目标集群对应缓存服务器的IP地址以及所述目标资源在所述目标集群对应缓存服务器磁盘上的存储路径,通过302重定向消息发送给终端。Embodiments of the present invention provide a non-transitory computer-readable storage medium, where the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions cause the computer to execute the methods provided by the foregoing method embodiments, for example Including: if it is detected that the target resource is not cached in the first cluster, sending a resource retrieval instruction to each cluster in the cloud cache except the first cluster to determine the target cluster containing the target resource; The IP address of the cache server corresponding to the target cluster and the storage path of the target resource on the disk of the cache server corresponding to the target cluster are sent to the terminal through a 302 redirect message.
本领域普通技术人员可以理解:实现上述方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成,前述的程序可以存储于一计算机可读取存储介质中,该程序在执行时,执行包括上述方法实施例的步骤;而前述的存储介质包括:ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by program instructions related to hardware, the aforementioned program can be stored in a computer-readable storage medium, and when the program is executed, execute It includes the steps of the above method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.
通过以上的实施方式的描述,本领域的技术人员可以清楚地了解到各实施方式可借助软件加必需的通用硬件平台的方式来实现,当然也可以通过硬件。基于这样的理解,上述技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品可以存储在计算机可读存储介质中,如ROM/RAM、磁碟、光盘等,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行各个实施例或者实施例的某些部分所述的方法。From the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and certainly can also be implemented by hardware. Based on this understanding, the above-mentioned technical solutions can be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products can be stored in computer-readable storage media, such as ROM/RAM, magnetic A disc, an optical disc, etc., includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform the methods described in various embodiments or some parts of the embodiments.
最后应说明的是:以上实施例仅用以说明本发明的技术方案,而非对其限制;尽管参照前述实施例对本发明进行了详细的说明,本领域的普通技术人员应当理解:其依然可以对前述各实施例所记载的技术方案进行修改,或者对其中部分技术特征进行等同替换;而这些修改或者替换,并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still be The technical solutions described in the foregoing embodiments are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (7)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810593576.1A CN110581873B (en) | 2018-06-11 | 2018-06-11 | Cross-cluster redirection method and monitoring server |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810593576.1A CN110581873B (en) | 2018-06-11 | 2018-06-11 | Cross-cluster redirection method and monitoring server |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110581873A CN110581873A (en) | 2019-12-17 |
CN110581873B true CN110581873B (en) | 2022-06-14 |
Family
ID=68809380
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201810593576.1A Active CN110581873B (en) | 2018-06-11 | 2018-06-11 | Cross-cluster redirection method and monitoring server |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110581873B (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115150475B (en) * | 2021-03-31 | 2024-07-26 | 贵州白山云科技股份有限公司 | Scheduling method, device, medium and equipment |
CN114938394B (en) * | 2022-04-13 | 2024-05-17 | 京东科技信息技术有限公司 | Cross-cluster network control method, device, equipment and storage medium |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102834814A (en) * | 2010-04-20 | 2012-12-19 | 日本电气株式会社 | Distribution system, distribution control device, and distribution control method |
CN102855284A (en) * | 2012-08-03 | 2013-01-02 | 北京联创信安科技有限公司 | Method and system for managing data of cluster storage system |
US8706865B1 (en) * | 2011-01-06 | 2014-04-22 | Israel L'Heureux | Enhanced network communications using diagnostic information |
CN105138661A (en) * | 2015-09-02 | 2015-12-09 | 西北大学 | Hadoop-based k-means clustering analysis system and method of network security log |
CN106507181A (en) * | 2016-11-30 | 2017-03-15 | 北京酷我科技有限公司 | A kind of method for being obtained and stored in line video data |
CN107925685A (en) * | 2015-07-31 | 2018-04-17 | 思杰系统有限公司 | The redirector browsed for secure web-page |
-
2018
- 2018-06-11 CN CN201810593576.1A patent/CN110581873B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102834814A (en) * | 2010-04-20 | 2012-12-19 | 日本电气株式会社 | Distribution system, distribution control device, and distribution control method |
US8706865B1 (en) * | 2011-01-06 | 2014-04-22 | Israel L'Heureux | Enhanced network communications using diagnostic information |
CN102855284A (en) * | 2012-08-03 | 2013-01-02 | 北京联创信安科技有限公司 | Method and system for managing data of cluster storage system |
CN107925685A (en) * | 2015-07-31 | 2018-04-17 | 思杰系统有限公司 | The redirector browsed for secure web-page |
CN105138661A (en) * | 2015-09-02 | 2015-12-09 | 西北大学 | Hadoop-based k-means clustering analysis system and method of network security log |
CN106507181A (en) * | 2016-11-30 | 2017-03-15 | 北京酷我科技有限公司 | A kind of method for being obtained and stored in line video data |
Also Published As
Publication number | Publication date |
---|---|
CN110581873A (en) | 2019-12-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11431791B2 (en) | Content delivery method, virtual server management method, cloud platform, and system | |
CN100484069C (en) | File data distributing method and relative device | |
CN107250999B (en) | Distributed content discovery with in-network caching | |
CN103024053B (en) | Cloud storage means, resource scheduling system, cloud memory node and system | |
WO2018000993A1 (en) | Distributed storage method and system | |
US10560543B2 (en) | Rule based cache processing in application delivery controller for load balancing | |
US10120872B2 (en) | Data caching based on requestor identity | |
CN111464661B (en) | Load balancing method and device, proxy equipment, cache equipment and service node | |
CN102143212A (en) | Cache sharing method and device for content delivery network | |
WO2019134648A1 (en) | Implementation method and device for control plane resource migration, and network function entity | |
WO2017114397A1 (en) | Method, device and system for realizing heartbeat mechanism | |
CN112104752B (en) | Hot spot balancing method and system for cache nodes of content distribution network | |
CN111371866B (en) | Method and device for processing service request | |
CN108881034B (en) | A request-response method, device and system applied to a BT system | |
CN109873855A (en) | A method and system for resource acquisition based on blockchain network | |
CN113301079A (en) | Data acquisition method, system, computing device and storage medium | |
CN113676514B (en) | File source returning method and device | |
CN103107944B (en) | A kind of content positioning method and routing device | |
US20140222988A1 (en) | Method for adaptive content discovery for distributed shared caching system | |
CN110581873B (en) | Cross-cluster redirection method and monitoring server | |
WO2023274267A1 (en) | Content resource distribution method, content delivery network, cluster, and medium | |
CN115706741A (en) | Method and device for returning slice file | |
CN107667513A (en) | System and method for remote topology discovery | |
US9003034B2 (en) | Method for operating a local area data network | |
JP5544521B2 (en) | State management method, processing device, and state management program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |