[go: up one dir, main page]

CN108270603A - A kind of dispatching method and management system - Google Patents

A kind of dispatching method and management system Download PDF

Info

Publication number
CN108270603A
CN108270603A CN201611264306.3A CN201611264306A CN108270603A CN 108270603 A CN108270603 A CN 108270603A CN 201611264306 A CN201611264306 A CN 201611264306A CN 108270603 A CN108270603 A CN 108270603A
Authority
CN
China
Prior art keywords
metric
resource
current data
data task
scheduling
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201611264306.3A
Other languages
Chinese (zh)
Inventor
王鹏
王创
李芳�
贺婷
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Mobile Communications Group Co Ltd
China Mobile Group Shanxi Co Ltd
Original Assignee
China Mobile Communications Group Co Ltd
China Mobile Group Shanxi Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Mobile Communications Group Co Ltd, China Mobile Group Shanxi Co Ltd filed Critical China Mobile Communications Group Co Ltd
Priority to CN201611264306.3A priority Critical patent/CN108270603A/en
Publication of CN108270603A publication Critical patent/CN108270603A/en
Pending legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/60Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08Configuration management of networks or network elements
    • H04L41/0803Configuration setting
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08Configuration management of networks or network elements
    • H04L41/0896Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer And Data Communications (AREA)

Abstract

The embodiment of the present invention provides a kind of dispatching method and management system, for solving the technical issues of scheduling of resource ability of resource management system in the prior art is poor.This method includes:It determines that there are what user carried out to operate with the first of the relevant operation plan of current data task for setting;The first operation of response, is shown and the relevant at least one metric of current data task by display interface;Wherein, at least one metric include user setting for the resource requirement and restrictive condition of current data task in the process of implementation, resource requirement is used to indicate the requirement to the hardware resource of server, and restrictive condition is used to indicate the service requirement to server;Based at least one metric, operation plan corresponding with current data task is determined;Scheduling of resource is carried out based on operation plan, determines at least one destination server corresponding with current data task, at least one destination server disclosure satisfy that resource requirement and restrictive condition.

Description

一种调度方法及管理系统A scheduling method and management system

技术领域technical field

本发明涉及通信技术领域,特别涉及一种调度方法及管理系统。The present invention relates to the field of communication technology, in particular to a scheduling method and a management system.

背景技术Background technique

当前的大数据生态系统中,需求变得越来越复杂,传统的Map-Reduce(一种编程模型)计算任务已经不再适合所有需求,因此在Hadoop 2.0(一种分布式系统基础架构)或YARN(一种新的Hadoop资源管理系统)中,计算任务的类型可以由用户自己定义,在Spark(一种与Hadoop相似的开源集群计算环境)中,计算任务的流程更有弹性。此外,随着集群规模的不断增长,异构资源被用来运行大数据工作负载,包括open power、x86、SolarisSparc等等。在目前的资源管理环境(如YARN,MESOS等)中,大数据计算任务在运行时资源、环境和任务限制条件上可能有多种要求。In the current big data ecosystem, the requirements are becoming more and more complex, and the traditional Map-Reduce (a programming model) computing task is no longer suitable for all requirements. Therefore, in Hadoop 2.0 (a distributed system infrastructure) or In YARN (a new Hadoop resource management system), the type of computing tasks can be defined by the user. In Spark (an open source cluster computing environment similar to Hadoop), the flow of computing tasks is more flexible. In addition, as the cluster size continues to grow, heterogeneous resources are used to run big data workloads, including open power, x86, SolarisSparc, and more. In the current resource management environment (such as YARN, MESOS, etc.), big data computing tasks may have various requirements in terms of runtime resources, environment and task constraints.

但目前,主要应用的大数据资源管理软件(如YARN和Mesos等)不能以更灵活的模式来对计算任务进行调度,虽然YARN可以支持多个维度(如CPU、内存、硬盘等),但不能支持更复杂的需求。比如,它不能支持更多的调度限制条件(如操作系统类型等)以及运行时的调度限制条件(如“在上一台主机上运行这个任务”)等,且它的调度器的可配置项目是固定不变的,用户不能在真实的生产环境中扩展或者定制化调度限制条件,调度配置过程不方便,从可能在进行资源调度过程中可能无法确定到最合适的服务器执行任务,甚至造成对资源的消耗。But at present, big data resource management software (such as YARN and Mesos, etc.) that are mainly used cannot schedule computing tasks in a more flexible mode. Although YARN can support multiple dimensions (such as CPU, memory, hard disk, etc.), it cannot Support for more complex requirements. For example, it cannot support more scheduling constraints (such as operating system type, etc.) and runtime scheduling constraints (such as "run this task on the last host"), etc., and its scheduler's configurable items It is fixed and unchanged. Users cannot expand or customize the scheduling constraints in the real production environment. The scheduling configuration process is inconvenient. It may not be possible to determine the most suitable server to perform tasks during the resource scheduling process, and even cause damage to the server. resource consumption.

综上可知,现有技术中资源管理系统存在对资源调度能力较差的技术问题。To sum up, it can be known that the resource management system in the prior art has a technical problem of poor resource scheduling capability.

发明内容Contents of the invention

本发明实施例提供一种调度方法及管理系统,用于解决现有技术中资源管理系统的资源调度能力较差的技术问题。Embodiments of the present invention provide a scheduling method and a management system, which are used to solve the technical problem of poor resource scheduling capability of the resource management system in the prior art.

第一方面,本发明实施提供一种调度方法,包括:In the first aspect, the implementation of the present invention provides a scheduling method, including:

确定存在用户进行的用于设置与当前数据任务相关的调度计划的第一操作;Determine that there is a first operation performed by the user for setting a scheduling plan related to the current data task;

响应所述第一操作,通过显示界面显示与所述当前数据任务相关的至少一个度量信息;其中,所述至少一个度量信息中包括用户设置的针对所述当前数据任务在执行过程中的资源要求和限制条件,所述资源要求用于指示对服务器的硬件资源的要求,所述限制条件用于指示对服务器的运行要求;In response to the first operation, at least one metric information related to the current data task is displayed through a display interface; wherein the at least one metric information includes resource requirements set by the user during execution of the current data task and a restriction condition, the resource requirement is used to indicate the requirement on the hardware resources of the server, and the restriction condition is used to indicate the operation requirement on the server;

基于所述至少一个度量信息,确定与所述当前数据任务对应的调度计划;Determine a scheduling plan corresponding to the current data task based on the at least one metric information;

基于所述调度计划进行资源调度,确定与所述当前数据任务对应的至少一个目标服务器,所述至少一个目标服务器能够满足所述资源要求及所述限制条件。Resource scheduling is performed based on the scheduling plan, and at least one target server corresponding to the current data task is determined, and the at least one target server can meet the resource requirement and the restriction condition.

可选的,在确定所述管理系统的显示界面中存在用户进行的用于设置与当前数据任务相关的调度计划的用户操作之前,所述方法还包括:Optionally, before determining that there is a user operation performed by a user in the display interface of the management system for setting a scheduling plan related to the current data task, the method further includes:

获取用户针对所述管理系统中资源度量收集器的新增资源度量的新增度量信息,所述资源度量收集器用于收集相应服务器的多项资源度量和每项资源度量对应的度量参数;Acquiring newly added metric information of the user for the newly added resource metric of the resource metric collector in the management system, the resource metric collector being used to collect multiple resource metrics of the corresponding server and the metric parameters corresponding to each resource metric;

及,获取所述用户通过所述管理系统中限制条件组件设置的针对执行数据任务的服务器运行的限制条件信息;And, obtaining the restrictive condition information set by the user through the restrictive condition component in the management system for the operation of the server that executes the data task;

所述响应所述第一操作,通过显示界面显示与所述当前数据任务相关的至少一个度量信息,包括:In response to the first operation, displaying at least one metric information related to the current data task through a display interface, including:

响应所述第一操作,基于所述新增度量信息及所述限制条件信息,在所述显示界面中显示所述至少一个度量信息。In response to the first operation, display the at least one metric information on the display interface based on the newly added metric information and the restriction condition information.

可选的,所述基于所述至少一个度量信息,确定与所述当前数据任务对应的调度计划,包括:Optionally, the determining the scheduling plan corresponding to the current data task based on the at least one metric information includes:

确定是否存在针对所述显示界面中所述至少一个度量信息进行的第二操作,所述第二操作为用于调整所述至少一个度量信息对应的度量参数的操作;Determine whether there is a second operation performed on the at least one metric information in the display interface, where the second operation is an operation for adjusting a metric parameter corresponding to the at least one metric information;

若存在所述第二操作,基于调整后的度量参数及所述至少一个度量信息生成与所述当前数据任务对应的调度计划,或,若不存在所述第二操作,基于所述至少一个度量信息及其相应的度量参数生成与所述当前数据任务对应的调度计划。If the second operation exists, generate a scheduling plan corresponding to the current data task based on the adjusted metric parameter and the at least one metric information, or, if the second operation does not exist, based on the at least one metric The information and its corresponding measurement parameters generate a scheduling plan corresponding to the current data task.

可选的,基于所述调度计划进行资源调度,确定与所述当前数据任务对应的至少一个目标服务器,包括:Optionally, performing resource scheduling based on the scheduling plan, and determining at least one target server corresponding to the current data task includes:

确定所述调度计划对应的至少一个调度指令,所述至少一个调度指令中每个调度指令指示了对服务器相应的资源要求及限制条件;Determine at least one scheduling instruction corresponding to the scheduling plan, and each scheduling instruction in the at least one scheduling instruction indicates corresponding resource requirements and constraints on the server;

基于所述至少一个调度指令进行资源调度,确定与所述当前数据任务对应的至少一个目标服务器。Resource scheduling is performed based on the at least one scheduling instruction, and at least one target server corresponding to the current data task is determined.

可选的,所述确定与所述当前数据任务对应的至少一个目标服务器,包括:Optionally, the determining at least one target server corresponding to the current data task includes:

确定所述管理系统连接的多个服务器中能够满足所述调度计划对应的资源要求及限制条件的至少一个目标服务器。Determine at least one target server among the multiple servers connected to the management system that can meet the resource requirements and constraints corresponding to the scheduling plan.

第二方面,本发明实施例提供一种管理系统,包括:In a second aspect, an embodiment of the present invention provides a management system, including:

第一确定模块,用于确定存在用户进行的用于设置与当前数据任务相关的调度计划的第一操作;A first determining module, configured to determine that there is a first operation performed by a user for setting a scheduling plan related to the current data task;

操作模块,用于响应所述第一操作,通过显示界面显示与所述当前数据任务相关的至少一个度量信息;其中,所述至少一个度量信息中包括用户设置的针对所述当前数据任务在执行过程中的资源要求和限制条件,所述资源要求用于指示对服务器的硬件资源的要求,所述限制条件用于指示对服务器的运行要求;An operation module, configured to display at least one metric information related to the current data task through a display interface in response to the first operation; wherein, the at least one metric information includes the execution time for the current data task set by the user Resource requirements and constraints in the process, the resource requirements are used to indicate the requirements for the hardware resources of the server, and the constraints are used to indicate the operating requirements for the server;

第二确定模块,用于基于所述至少一个度量信息,确定与所述当前数据任务对应的调度计划;A second determining module, configured to determine a scheduling plan corresponding to the current data task based on the at least one metric information;

调度模块,用于基于所述调度计划进行资源调度,确定与所述当前数据任务对应的至少一个目标服务器,所述至少一个目标服务器能够满足所述资源要求及所述限制条件。A scheduling module, configured to perform resource scheduling based on the scheduling plan, determine at least one target server corresponding to the current data task, and the at least one target server can meet the resource requirement and the restriction condition.

可选的,所述系统还包括:Optionally, the system also includes:

获取模块,用于在确定所述管理系统的显示界面中存在用户进行的用于设置与当前数据任务相关的调度计划的用户操作之前,获取用户针对所述管理系统中资源度量收集器的新增资源度量的新增度量信息,所述资源度量收集器用于收集相应服务器的多项资源度量和每项资源度量对应的度量参数;及,获取所述用户通过所述管理系统中限制条件组件设置的针对执行数据任务的服务器运行的限制条件信息;The obtaining module is used to obtain the user's newly added resource measurement collector in the management system before determining that there is a user operation for setting a scheduling plan related to the current data task in the display interface of the management system. newly added metric information of resource metric, the resource metric collector is used to collect multiple resource metrics of the corresponding server and the metric parameters corresponding to each resource metric; Information about restrictions on the operation of servers performing data tasks;

所述操作模块用于:响应所述第一操作,基于所述新增度量信息及所述限制条件信息,在所述显示界面中显示所述至少一个度量信息。The operation module is configured to: display the at least one metric information on the display interface based on the newly added metric information and the restriction condition information in response to the first operation.

可选的,所述第二确定模块用于:Optionally, the second determination module is used for:

确定是否存在针对所述显示界面中所述至少一个度量信息进行的第二操作,所述第二操作为用于调整所述至少一个度量信息对应的度量参数的操作;Determine whether there is a second operation performed on the at least one metric information in the display interface, where the second operation is an operation for adjusting a metric parameter corresponding to the at least one metric information;

若存在所述第二操作,基于调整后的度量参数及所述至少一个度量信息生成与所述当前数据任务对应的调度计划,或,若不存在所述第二操作,基于所述至少一个度量信息及其相应的度量参数生成与所述当前数据任务对应的调度计划。If the second operation exists, generate a scheduling plan corresponding to the current data task based on the adjusted metric parameter and the at least one metric information, or, if the second operation does not exist, based on the at least one metric The information and its corresponding measurement parameters generate a scheduling plan corresponding to the current data task.

可选的,所述调度模块用于:Optionally, the scheduling module is used for:

确定所述调度计划对应的至少一个调度指令,所述至少一个调度指令中每个调度指令指示了对服务器相应的资源要求及限制条件;Determine at least one scheduling instruction corresponding to the scheduling plan, and each scheduling instruction in the at least one scheduling instruction indicates corresponding resource requirements and constraints on the server;

基于所述至少一个调度指令进行资源调度,确定与所述当前数据任务对应的至少一个目标服务器。Resource scheduling is performed based on the at least one scheduling instruction, and at least one target server corresponding to the current data task is determined.

可选的,所述调度模块用于:Optionally, the scheduling module is used for:

确定所述管理系统连接的多个服务器中能够满足所述调度计划对应的资源要求及限制条件的至少一个目标服务器。Determine at least one target server among the multiple servers connected to the management system that can meet the resource requirements and constraints corresponding to the scheduling plan.

本发明实施例中,通过确定存在用户进行的用于设置与当前数据任务相关的调度计划的第一操作时,管理系统能够响应第一操作,并通过显示单元显示与调度计划相关的至少一个度量信息,至少一个度量信息中可以包括针对当前数据任务在执行过程中的资源要求和限制条件,该资源要求可以用于指示执行数据任务对服务器的硬件资源的限制,限制条件用于指示执行当前数据任务对服务器的运行要求,进而,基于少一个度量信息即可确定与当前数据任务对应的调度计划,而基于该调度计划可以进行资源调度,确定与当前数据任务对应的至少一个目标服务器,该至少一个目标服务器能够满足资源要求及限制条件,故调度过程中,通过显示至少一个度量信息能够便于用户进行查看调度计划的配置过程,提高配置过程中的准确度及用户体验,同时,显示的度量信息中包括用户设置的资源要求及限制条件,从而使得调度计划与用户处理数据过程中的实际需求更为符合,使得对服务器资源的调度较为准确,且通过用户的自定义,使得管理系统满足多样化的计算任务需求,实现细粒度的资源调度,提高对资源的管理效果。In the embodiment of the present invention, when it is determined that there is a first operation performed by the user for setting the scheduling plan related to the current data task, the management system can respond to the first operation and display at least one metric related to the scheduling plan through the display unit Information, at least one piece of metric information may include resource requirements and constraints during the execution of the current data task, the resource requirements may be used to indicate the limitations of the hardware resources of the server for executing the data task, and the constraints are used to indicate the execution of the current data task The operation requirements of the task on the server, and then, based on one less measurement information, the scheduling plan corresponding to the current data task can be determined, and resource scheduling can be performed based on the scheduling plan, and at least one target server corresponding to the current data task can be determined. A target server can meet the resource requirements and constraints. Therefore, during the scheduling process, displaying at least one measurement information can facilitate the user to view the configuration process of the scheduling plan, improve the accuracy and user experience during the configuration process, and at the same time, the displayed measurement information Including the resource requirements and constraints set by the user, so that the scheduling plan is more in line with the actual needs of the user in the process of processing data, making the scheduling of server resources more accurate, and through user customization, the management system can meet the needs of diversification Computing task requirements, realize fine-grained resource scheduling, and improve the management effect of resources.

附图说明Description of drawings

图1为本发明实施例中管理系统的架构示意图;Fig. 1 is a schematic diagram of the architecture of the management system in the embodiment of the present invention;

图2为本发明实施例中调度方法的主要流程图;Fig. 2 is the main flowchart of the scheduling method in the embodiment of the present invention;

图3为本发明实施例中设置调度计划的新增资源度量的示意图;FIG. 3 is a schematic diagram of newly added resource metrics for setting a scheduling plan in an embodiment of the present invention;

图4为本发明实施例中设置调度计划的限制条件的示意图;FIG. 4 is a schematic diagram of setting constraints of a scheduling plan in an embodiment of the present invention;

图5为本发明实施例中管理系统的可视化的图形界面的示意图;FIG. 5 is a schematic diagram of a visualized graphical interface of the management system in an embodiment of the present invention;

图6为本发明实施例中管理系统的可视化的图形界面的示意图;6 is a schematic diagram of a visualized graphical interface of the management system in an embodiment of the present invention;

图7为本发明实施例中管理系统的模块结构图。Fig. 7 is a block diagram of the management system in the embodiment of the present invention.

具体实施方式Detailed ways

为使本发明实施例的目的、技术方案和优点更加清楚,下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例是本发明一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例,都属于本发明保护的范围。In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

首先,对本发明实施例所应用的系统架构进行介绍。First, the system architecture applied in the embodiment of the present invention is introduced.

本发明实施例中,管理系统可以是用于对服务器集群进行资源管理的系统,其可以基于用户对数据任务的请求,分配相应的服务器进行任务处理。在实际应用中,该管理系统中可以是由一个主站(调度中心)和多个子站构成的分布式管理系统,各个子站可以均是相应的服务器。In the embodiment of the present invention, the management system may be a system for resource management of server clusters, which may assign corresponding servers to perform task processing based on user requests for data tasks. In practical applications, the management system may be a distributed management system composed of a master station (dispatch center) and multiple sub-stations, and each sub-station may be a corresponding server.

如图1所示,其为本发明实施例中管理系统的架构示意图及其相应的应用环境。该管理系统可以主要包括调度引擎和服务器集群。其中,调度引擎可以基于用户创建的调度计划,对用户请求处理的任务,例如对大量数据的批处理任务等,进行相应的计算,并基于调度计划分配相应的服务器资源执行任务,而服务器集群主要用于供调度引擎调度,对各项任务进行处理。As shown in FIG. 1 , it is a schematic diagram of the architecture of the management system and its corresponding application environment in the embodiment of the present invention. The management system may mainly include a scheduling engine and a server cluster. Among them, the scheduling engine can perform corresponding calculations on the tasks requested by the user based on the scheduling plan created by the user, such as batch processing tasks for a large amount of data, and allocate corresponding server resources to perform tasks based on the scheduling plan. The server cluster mainly It is used for scheduling by the scheduling engine to process various tasks.

具体来说,本发明实施例中管理系统可以包括四个组件,即:可插拔的资源度量收集器、可插拔的作业/任务限制条件组件、配置调度计划的图形界面及将调度计划翻译成资源需求的解析器。Specifically, the management system in the embodiment of the present invention may include four components, namely: a pluggable resource metric collector, a pluggable job/task constraint component, a graphical interface for configuring the scheduling plan, and translating the scheduling plan parser into resource requirements.

其中,可插拔的资源度量收集器可以是被部署在服务器集群的每个服务器中,负责收集服务器对应的资源度量,该资源度量可以用于指示相应服务器的硬件资源及其属性,例如CPU内核、存储器的大小等。Among them, the pluggable resource metric collector can be deployed in each server of the server cluster, and is responsible for collecting the resource metric corresponding to the server, and the resource metric can be used to indicate the hardware resource and its attributes of the corresponding server, such as CPU core , the size of the memory, etc.

可插拔的作业/任务限制条件组件可以用于为用户添加个性化的作业/任务限制条件提供相应的接口,实现运行时的调度限制。The pluggable job/task constraint component can be used to provide a corresponding interface for users to add personalized job/task constraints, and realize runtime scheduling constraints.

图形界面可以用于管理调度计划,通过图形界面可以显示调度计划相关的资源配置,例如资源度量以及限制条件等。用户可在可视的图形界面上创建/修改/查看调度计划。The graphical interface can be used to manage the scheduling plan, and the resource configuration related to the scheduling plan can be displayed through the graphical interface, such as resource metrics and constraints. Users can create/modify/view scheduling plans on a visual graphical interface.

调度计划解析器位于调度引擎中,用于对调度计划进行解析翻译成相应的调度指令,进而调度器将会选择最合适的资源容器分配给这个任务,实现细粒度的资源调度。The scheduling plan parser is located in the scheduling engine, and is used to parse and translate the scheduling plan into corresponding scheduling instructions, and then the scheduler will select the most appropriate resource container to allocate to this task to achieve fine-grained resource scheduling.

另外,本文中术语“和/或”,仅仅是一种描述关联对象的关联关系,表示可以存在三种关系,例如,A和/或B,可以表示:单独存在A,同时存在A和B,单独存在B这三种情况。另外,本文中字符“/”,一般表示前后关联对象是一种“或”的关系。In addition, the term "and/or" in this article is only an association relationship describing associated objects, which means that there may be three relationships, for example, A and/or B, which may mean: A exists alone, A and B exist at the same time, There are three cases of B alone. In addition, the character "/" in this article generally indicates that the contextual objects are an "or" relationship.

下面结合附图对本发明优选的实施方式进行详细说明。Preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings.

如图2所示,本发明实施例中提供一种调度方法,应用于管理系统中,该管理系统可以使用于对服务器资源进行调度和管理的系统,该方法包括以下步骤:As shown in FIG. 2, an embodiment of the present invention provides a scheduling method, which is applied to a management system. The management system can be used in a system for scheduling and managing server resources. The method includes the following steps:

S11:确定存在用户进行的用于设置与当前数据任务相关的调度计划的第一操作;S11: Determine that there is a first operation performed by the user for setting a scheduling plan related to the current data task;

S12:响应第一操作,通过显示界面显示与调度计划相关的至少一个度量信息;其中,至少一个度量信息中包括用户设置的针对当前数据任务在执行过程中的资源要求和限制条件,资源要求用于指示对服务器的硬件资源的要求,限制条件用于指示对服务器的运行要求;S12: In response to the first operation, display at least one measurement information related to the scheduling plan through the display interface; wherein, the at least one measurement information includes resource requirements and constraints set by the user during the execution of the current data task, and the resource requirements are used Used to indicate the requirements for the hardware resources of the server, and the constraints are used to indicate the operating requirements for the server;

S13:基于至少一个度量信息,确定与当前数据任务对应的调度计划;S13: Based on at least one metric information, determine a scheduling plan corresponding to the current data task;

S14:基于调度计划进行资源调度,确定与当前数据任务对应的至少一个目标服务器,至少一个目标服务器能够满足资源要求及限制条件。S14: Perform resource scheduling based on the scheduling plan, determine at least one target server corresponding to the current data task, and at least one target server can meet resource requirements and constraints.

本发明实施例中,当确定存在用户进行的用于设置当前数据任务的调度计划的第一操作时,通过响应第一操作可以在显示界面中显示与调度计划相关的至少一个度量信息,其包括针对当前数据任务在执行过程中的资源要求和限制条件等信息,故用户可以基于该显示界面较为清楚地查看或设置度量信息,进而基于度量信息即可以生成与当前数据任务对应的调度计划,则基于调度计划机箱内资源调度即可确定适用于执行当前数据任务的至少一个目标服务器使得调度过程的准确性较高。In the embodiment of the present invention, when it is determined that there is a first operation performed by the user for setting the scheduling plan of the current data task, at least one metric information related to the scheduling plan can be displayed on the display interface by responding to the first operation, which includes For information such as resource requirements and constraints during the execution of the current data task, the user can clearly view or set the measurement information based on the display interface, and then based on the measurement information, the scheduling plan corresponding to the current data task can be generated, then Based on the resource scheduling in the chassis of the scheduling plan, at least one target server suitable for executing the current data task can be determined, so that the accuracy of the scheduling process is high.

在实际应用中,第一操作可以是用户进行的用于调出能够设置与数据任务相关的调度计划的入口或页面的操作,例如第一操作可以是设计人员通过管理系统中的输入装置(如鼠标)或操作体(如手指)对显示界面中相应选项进行的点击操作,或者,也可以是通过其它方式进行的操作,例如通过输入相应的操作指令的输入操作等等,本发明实例例对此不作具体限制。In practical applications, the first operation may be an operation performed by the user to call out an entry or page capable of setting a schedule related to a data task. For example, the first operation may be performed by a designer through an input device in the management system (such as mouse) or an operating body (such as a finger) to click on the corresponding options in the display interface, or it can also be an operation performed in other ways, such as an input operation by inputting a corresponding operation instruction, etc. The examples of the present invention are This is not specifically limited.

当前数据任务可以是用户通过终端设备向服务器请求处理的大批量数据,其也可以是网络数据,例如通讯社交过程中使用的微信、音乐、视频,或是吃喝玩乐时的用到的团购、电商、移动支付等都会不断产生海量的信息数据,当然还可以是其它需要处理的大数据,本领域技术人员可以根据实际需求设置待处理的数据任务,本发明实施例对此不作具体限制。The current data task can be a large amount of data requested by the user to the server through the terminal device for processing, and it can also be network data, such as WeChat, music, and video used in the communication and social process, or group buying, e-mail, etc. Businesses, mobile payments, etc. will continuously generate massive amounts of information data, and of course it can also be other big data that needs to be processed. Those skilled in the art can set the data tasks to be processed according to actual needs, which is not specifically limited in the embodiment of the present invention.

可选的,在S11之前,用户可以通过管理系统所包含的相应的组件对数据任务的调度计划对应的度量进行设置,则管理系统可以获取用户设置的相应信息。该过程可以包括:确定用户针对管理系统中资源度量收集器的新增资源度量的新增度量信息,资源度量收集器用于收集相应服务器的多项资源度量和每项资源度量对应的度量参数,及,确定用户通过管理系统中限制条件组件设置的针对执行数据任务的服务器运行的限制条件信息。Optionally, before S11, the user can set the metrics corresponding to the scheduling plan of the data task through the corresponding components included in the management system, and the management system can obtain the corresponding information set by the user. The process may include: determining the newly added metric information of the user for the newly added resource metric of the resource metric collector in the management system, the resource metric collector is used to collect multiple resource metrics of the corresponding server and the metric parameters corresponding to each resource metric, and , to determine the restrictive condition information set by the user through the restrictive condition component in the management system for the operation of the server executing the data task.

其中,度量参数可以是能够表征相应资源度量对应的操参数,例如版本参数、速率参数、存储量参数、内核参数等等。运行的限制条件可以是指在处理数据任务中对主机的运行的限定,例如允许在多个或一个主机中执行。Wherein, the metric parameter may be an operational parameter capable of characterizing the corresponding resource metric, such as a version parameter, a rate parameter, a storage capacity parameter, a kernel parameter, and the like. The operation restriction may refer to the restriction on the operation of the host in processing the data task, for example, it is allowed to execute in multiple or one host.

具体来说,在设置过程中,可以包括但不仅限于以下设置。Specifically, during the setting process, it may include but not limited to the following settings.

设置1:用户通过管理系统包含的资源度量收集器组件可以对相应服务器的多项细粒度的资源度量进行收集,而现有技术中使用的开源组件YARN只能支持固定的几个维度(如CPU、存储器、硬盘等),而本发明实施中管理系统中的资源度量收集器可以收集的多项细粒度的资源度量可以包括GNU编译器集合(GNU Compiler Collection,GCC)、Java运行环境(Java Runtime Environment,JRE)、软件使用范围授权(License)等。Setting 1: The user can collect multiple fine-grained resource metrics of the corresponding server through the resource metric collector component contained in the management system, while the open source component YARN used in the prior art can only support several fixed dimensions (such as CPU , memory, hard disk, etc.), and multiple fine-grained resource metrics that can be collected by the resource metric collector in the management system in the implementation of the present invention can include GNU Compiler Collection (GNU Compiler Collection, GCC), Java Operating Environment (Java Runtime Environment, JRE), software usage authorization (License), etc.

例如,如图3所示,用户可以实现AgentInterface接口来定义不同的实现类,即添加新的度量收集插件或者在默认的Resource Metric Agents配置文件中增加新的度量,以定义更细粒度的资源度量收集。比如图3中,通过在Resource Metric Agents配置文件中增加新的<metric/>节点,定义默认的JRE版本度量收集程序会运行脚本“java-version”来获取Java版本并汇报给集群的调度引擎。可以增加更多的<metric/>节点,比如GCC Version(GCC版本)、License Name(许可项目)等,以实现对多个资源度量收集。For example, as shown in Figure 3, users can implement the AgentInterface interface to define different implementation classes, that is, add new metrics collection plug-ins or add new metrics to the default Resource Metric Agents configuration file to define more fine-grained resource metrics collect. For example, in Figure 3, by adding a new <metric/> node in the Resource Metric Agents configuration file, the default JRE version metric collection program will run the script "java-version" to obtain the Java version and report it to the cluster scheduling engine. More <metric/> nodes can be added, such as GCC Version (GCC version), License Name (license item), etc., to realize the collection of metrics for multiple resources.

设置2:用户通过可插拔的作业/任务限制组件(即限制条件组件)可以添加个性化的作业/任务限制条件,实现运行时的调度限制。Setting 2: Users can add personalized job/task restriction conditions through the pluggable job/task restriction component (ie, restriction component) to realize runtime scheduling restriction.

例如,如图4所示,用户可以实现Agent Interface定义新的实现类或者把在Default Contraint Agent的配置文件中增加新的<metric/>节点来增加依赖限制,以定义复杂的任务依赖语义,如“Taskof this job will be run on same hosts”,即要求“此作业的所有任务都要在相同的主机上运行”,等等。For example, as shown in Figure 4, the user can implement the Agent Interface to define a new implementation class or add a new <metric/> node in the configuration file of the Default Contraint Agent to increase the dependency restriction to define complex task dependency semantics, such as "Taskof this job will be run on same hosts", which requires "all tasks of this job will be run on the same hosts", and so on.

本发明实施例中,在检测到第一操作后,通过响应第一操作,通过管理系统包括的图形界面组件即可以在相应的可视化显示界面中显示与调度计划相关的至少一个度量信息,即调度相应的资源要求和限制条件均可显示在图形界面中,即用户界面(UserInterface,UI),本文中称其为显示界面。In the embodiment of the present invention, after the first operation is detected, by responding to the first operation, at least one metric information related to the scheduling plan can be displayed in the corresponding visual display interface through the graphical interface component included in the management system, that is, the scheduling Corresponding resource requirements and constraints can be displayed on a graphical interface, that is, a user interface (User Interface, UI), which is referred to as a display interface in this paper.

如图5所示,其为管理系统中可视化的图形界面的示意图。用户可以在图形界面上创建/修改/查看调度计划,而用户通过前述的设置1和2实现调度插件接口来添加资源要求和限制条件,这些新的要求和限制条件会被添加在UI中,如图5的虚线框1内所显示的即为新增的资源度量,虚线框2内所显示的即为相应的限制条件。As shown in FIG. 5 , it is a schematic diagram of a visualized graphical interface in the management system. Users can create/modify/view the scheduling plan on the graphical interface, and the user implements the scheduling plug-in interface through the aforementioned settings 1 and 2 to add resource requirements and constraints, and these new requirements and constraints will be added in the UI, such as What is shown in the dotted box 1 in FIG. 5 is the newly added resource metric, and what is shown in the dotted box 2 is the corresponding restriction condition.

在实际应用中,用户还可以对图形界面中的相应的信息进行设置,In practical applications, users can also set the corresponding information in the graphical interface,

因此,本发明实施例中,通过管理系统包括的多个组件能够实现可插拔和可扩展的资源维度和作业/任务限制,而通过可视化的图形界面显示相应的度量信息,以便用户进行相应的管理和查看,避免了现有技术中在调度过程中无法查看和/或修改相应的调度计划信息的弊端,可视化的显示界面提高用了户体验度。Therefore, in the embodiment of the present invention, pluggable and extensible resource dimensions and job/task restrictions can be realized through multiple components included in the management system, and the corresponding measurement information is displayed through a visual graphical interface, so that users can perform corresponding The management and viewing avoid the drawbacks in the prior art that the corresponding scheduling plan information cannot be viewed and/or modified during the scheduling process, and the visual display interface improves user experience.

本发明实施例中,在基于至少一个度量信息,确定与当前数据任务对应的调度计划时,可以是确定是否存在针对显示界面中至少一个度量信息进行的第二操作,第二操作为用于调整至少一个度量信息对应的度量参数的操作。In the embodiment of the present invention, when determining the scheduling plan corresponding to the current data task based on at least one metric information, it may be determined whether there is a second operation for at least one metric information in the display interface, and the second operation is for adjusting An operation on a metric parameter corresponding to at least one metric information.

即在显示界面中显示至少一个度量信息时,可以检测是否存在用户针对显示界面中的资源度量和限制条件进行设置操作。例如,用户可以通过如图4所示的显示界面设置资源度量中License Name的度量要求(即配置度量参数),如将允许软件使用范围授权由“matlab”修改为“Java”,或者,将“GCC Version”对应的度量要求由“>=2.6”修改为“=2.6”,等等。That is, when at least one metric information is displayed on the display interface, it may be detected whether there is a setting operation by the user on the resource metric and the restriction condition on the display interface. For example, the user can set the measurement requirements of the License Name in the resource measurement (that is, configure the measurement parameters) through the display interface shown in Figure 4, such as modifying the authorization of the scope of use of the software from "matlab" to "Java", or changing " The metric requirement corresponding to GCC Version" is changed from ">=2.6" to "=2.6", and so on.

或者,用户还可以对限制条件进行设置,例如,将图5中选项栏“runtimeconstraint conditions(即运行时的限制条件)”下的选项“Task running on same job’stask hosts”的约束的偏好由“Required”修改为“Preferred”,等等。Or, the user can also set the restriction conditions, for example, the option "Task running on same job'stask hosts" under the option column "runtime constraint conditions (ie, runtime constraints)" in Figure 5 will be selected by " "Required" is changed to "Preferred", etc.

通过在选好偏好的选项后,用户可以命名并保存该调度计划,具体来说,若存在第二操作,基于调整后的度量参数及至少一个度量信息生成与当前数据任务对应的调度计划,或,若不存在第二操作,基于至少一个度量信息及其相应的度量参数生成与当前数据任务对应的调度计划。After selecting the preferred option, the user can name and save the scheduling plan. Specifically, if there is a second operation, generate a scheduling plan corresponding to the current data task based on the adjusted measurement parameter and at least one measurement information, or , if there is no second operation, generate a scheduling plan corresponding to the current data task based on at least one metric information and corresponding metric parameters.

当然,在实际应用中,用户通过设置,可以保存多个版本的调度计划,不同调度计划可以具有不同的命名和不同的度量参数等,后期可以直接调用调度计划。Of course, in practical applications, users can save multiple versions of scheduling plans through settings, and different scheduling plans can have different names and different measurement parameters, etc., and the scheduling plans can be called directly later.

故本发明实施例中,由于管理系统中显示的相应度量信息的图形界面也为可操作界面,故通过用户对度量信息的相关度量参数的配置,也有助于提高调度计划的准确性,提高调度计划配置过程中的灵活性。Therefore, in the embodiment of the present invention, since the graphical interface of the corresponding measurement information displayed in the management system is also an operable interface, the configuration of the relevant measurement parameters of the measurement information by the user can also help to improve the accuracy of the scheduling plan and improve the scheduling efficiency. Flexibility during plan configuration.

可选的,调度引擎在获得生成的调度计划后,可通过调度引擎中的解析器组件把计划即设置1和2的资源度量要求和运行时作业/任务限制条件解析翻译成调度指令,进而调度器将会选择最合适的资源容器分配给这个任务,即调度器可以基于调度计划进行资源调度,确定与当前数据任务对应的至少一个目标服务器,即通过该至少一个目标服务器即可执行当前数据任务,实现细粒度的资源调度。Optionally, after the scheduling engine obtains the generated scheduling plan, it can use the parser component in the scheduling engine to translate the plan, that is, set 1 and 2 resource measurement requirements and run-time job/task constraints into scheduling instructions, and then schedule The scheduler will select the most suitable resource container to allocate to this task, that is, the scheduler can perform resource scheduling based on the scheduling plan, and determine at least one target server corresponding to the current data task, that is, the current data task can be executed through the at least one target server , to achieve fine-grained resource scheduling.

在实际应用中,该至少一个目标服务器可以是管理系统中服务器集群中的服务器,其能够满足调度计划对应的资源要求和限制条件,其中,至少一个服务器的数量可能与调度计划中的限制条件相关,例如当限制条件为“在相同的主机上运行任务”,则分配处理当前数据任务对应的目标服务器可能只有一个。In practical applications, the at least one target server may be a server in the server cluster in the management system, which can meet the resource requirements and constraints corresponding to the scheduling plan, wherein the number of at least one server may be related to the constraints in the scheduling plan , for example, when the restriction condition is "run the task on the same host", there may be only one target server corresponding to the task assigned to process the current data.

下面结合流程图6来说明本发明实施例中管理系统对资源的调度和管理过程。The process of scheduling and managing resources by the management system in the embodiment of the present invention will be described below in conjunction with Flowchart 6 .

首先,获取用户通过可插拔的资源度量收集器来定义从计算节点上收集的资源度量,及通过可插拔的作业/任务限制条件组件来定义作业/任务限制条件;First, get users to define resource metrics collected from computing nodes through pluggable resource metric collectors, and define job/task constraints through pluggable job/task constraint components;

然后,当检测用户通过图形界面创建作业调度计划的时候,前述定义的项目(即资源独立和限定条件)会作为候选条件显示在图形界面中;Then, when the detection user creates a job scheduling plan through the graphical interface, the aforementioned defined items (ie resource independence and limiting conditions) will be displayed in the graphical interface as candidate conditions;

进而,一旦调度计划被创建好,它就可以被绑定到某个作业上,用户可以通过作业配置参数来把一个调度计划绑定到某个作业,例如:对于map Reduce作业,指定conf.set(“SchedulingPlan”,<PLAN_NAME>);Furthermore, once a schedule is created, it can be bound to a job. Users can bind a schedule to a job through job configuration parameters. For example, for a map reduce job, specify conf.set ("SchedulingPlan", <PLAN_NAME>);

然后,调度引擎中的解析器组件将会把资源度量要求和运行时作业/任务限制条件解析成调度器可读的指令;Then, the parser component in the scheduling engine will parse the resource metric requirements and runtime job/task constraints into scheduler-readable instructions;

最后,通过管理系统中调度引擎根据调度指令来调度作业/任务。Finally, the scheduling engine in the management system schedules jobs/tasks according to scheduling instructions.

当然,执行完成后,管理系统还可以对资源进行回收,或者,在执行过程中可以通过可插拔的资源度量收集器收集服务器相应的资源度量,例如内存暂用情况等等。Of course, after the execution is completed, the management system can also reclaim the resources, or, during the execution process, the pluggable resource metric collector can be used to collect the corresponding resource metrics of the server, such as temporary memory usage and so on.

因此,本发明实施例中,通过为管理系统提供可插拔的接口来为运行时计算任务定义资源和限制规则的度量,以及设置图像界面来为作业创建调度计划,以及一个解析器来把调度计划翻译成资源调度引擎的资源需求,资源调度引擎将会根据每个应用的资源需求来分配资源,从而提高管理系统对资源的管理。Therefore, in the embodiment of the present invention, by providing a pluggable interface for the management system to define the metrics of resources and limit rules for runtime computing tasks, and setting a graphical interface to create a scheduling plan for jobs, and a parser to convert the scheduling The plan is translated into the resource requirements of the resource scheduling engine, and the resource scheduling engine will allocate resources according to the resource requirements of each application, thereby improving the management system's management of resources.

如图7所示,基于同一发明构思,本发明实施例还提供一种管理系统,该管理系统所处的系统架构如图1所示。该管理系统可以包括第一确定模块201、操作模块202、第二确定模块203和调度模块204。As shown in FIG. 7 , based on the same inventive concept, an embodiment of the present invention also provides a management system. The system architecture of the management system is shown in FIG. 1 . The management system may include a first determination module 201 , an operation module 202 , a second determination module 203 and a scheduling module 204 .

其中,第一确定模块201可以用于确定存在用户进行的用于设置与当前数据任务相关的调度计划的第一操作;Wherein, the first determining module 201 may be configured to determine that there is a first operation performed by the user for setting a scheduling plan related to the current data task;

操作模块202可以用于响应所述第一操作,通过显示界面显示与所述当前数据任务相关的至少一个度量信息;其中,所述至少一个度量信息中包括用户设置的针对所述当前数据任务在执行过程中的资源要求和限制条件,所述资源要求用于指示对服务器的硬件资源的要求,所述限制条件用于指示对服务器的运行要求;The operation module 202 may be configured to display at least one metric information related to the current data task through a display interface in response to the first operation; Resource requirements and constraints during the execution process, the resource requirements are used to indicate the requirements for the hardware resources of the server, and the constraints are used to indicate the operating requirements for the server;

第二确定模块203可以用于基于所述至少一个度量信息,确定与所述当前数据任务对应的调度计划;The second determining module 203 may be configured to determine a scheduling plan corresponding to the current data task based on the at least one metric information;

调度模块204可以用于基于所述调度计划进行资源调度,确定与所述当前数据任务对应的至少一个目标服务器,所述至少一个目标服务器能够满足所述资源要求及所述限制条件。The scheduling module 204 may be configured to perform resource scheduling based on the scheduling plan, determine at least one target server corresponding to the current data task, and the at least one target server can meet the resource requirement and the restriction condition.

可选的,系统还可以包括获取模块,其可以用于在确定所述管理系统的显示界面中存在用户进行的用于设置与当前数据任务相关的调度计划的用户操作之前,获取用户针对所述管理系统中资源度量收集器的新增资源度量的新增度量信息,所述资源度量收集器用于收集相应服务器的多项资源度量和每项资源度量对应的度量参数;及,获取所述用户通过所述管理系统中限制条件组件设置的针对执行数据任务的服务器运行的限制条件信息;Optionally, the system may further include an obtaining module, which may be used to obtain the user's information on the user's information about the current data task before determining that there is a user operation for setting a scheduling plan related to the current data task in the display interface of the management system. new metric information of newly added resource metrics of the resource metric collector in the management system, the resource metric collector is used to collect multiple resource metrics of the corresponding server and the metric parameters corresponding to each resource metric; Restriction information set by the restriction component in the management system for the operation of the server that executes the data task;

所述操作模块202可以用于响应所述第一操作,基于所述新增度量信息及所述限制条件信息,在所述显示界面中显示所述至少一个度量信息。The operation module 202 may be configured to display the at least one metric information on the display interface based on the newly added metric information and the restriction condition information in response to the first operation.

可选的,第二确定模块203可以用于:确定是否存在针对所述显示界面中所述至少一个度量信息进行的第二操作,所述第二操作为用于调整所述至少一个度量信息对应的度量参数的操作;若存在所述第二操作,基于调整后的度量参数及所述至少一个度量信息生成与所述当前数据任务对应的调度计划,或,若不存在所述第二操作,基于所述至少一个度量信息及其相应的度量参数生成与所述当前数据任务对应的调度计划。Optionally, the second determining module 203 may be configured to: determine whether there is a second operation performed on the at least one metric information in the display interface, and the second operation is to adjust the at least one metric information corresponding to The operation of the metric parameter; if the second operation exists, generate a scheduling plan corresponding to the current data task based on the adjusted metric parameter and the at least one metric information, or, if the second operation does not exist, A scheduling plan corresponding to the current data task is generated based on the at least one metric information and corresponding metric parameters.

可选的,调度模块204可以用于确定所述调度计划对应的至少一个调度指令,所述至少一个调度指令中每个调度指令指示了对服务器相应的资源要求及限制条件;进而,基于所述至少一个调度指令进行资源调度,确定与所述当前数据任务对应的至少一个目标服务器。Optionally, the scheduling module 204 may be configured to determine at least one scheduling instruction corresponding to the scheduling plan, and each scheduling instruction in the at least one scheduling instruction indicates corresponding resource requirements and constraints on the server; furthermore, based on the At least one scheduling instruction performs resource scheduling, and determines at least one target server corresponding to the current data task.

可选的,调度模块204还可以用于确定所述管理系统连接的多个服务器中能够满足所述调度计划对应的资源要求及限制条件的至少一个目标服务器。Optionally, the scheduling module 204 may also be configured to determine at least one target server among multiple servers connected to the management system that can meet the resource requirements and constraints corresponding to the scheduling plan.

本发明实施例中,通过确定存在用户进行的用于设置与当前数据任务相关的调度计划的第一操作时,管理系统能够响应第一操作,并通过显示单元显示与调度计划相关的至少一个度量信息,至少一个度量信息中可以包括针对当前数据任务在执行过程中的资源要求和限制条件,该资源要求可以用于指示执行数据任务对服务器的硬件资源的限制,限制条件用于指示执行当前数据任务对服务器的运行要求,进而,基于少一个度量信息即可确定与当前数据任务对应的调度计划,而基于该调度计划可以进行资源调度,确定与当前数据任务对应的至少一个目标服务器,该至少一个目标服务器能够满足资源要求及限制条件,故调度过程中,通过显示至少一个度量信息能够便于用户进行查看调度计划的配置过程,提高配置过程中的准确度及用户体验,同时,显示的度量信息中包括用户设置的资源要求及限制条件,从而使得调度计划与用户处理数据过程中的实际需求更为符合,使得对服务器资源的调度较为准确,且通过用户的自定义,使得管理系统满足多样化的计算任务需求,实现细粒度的资源调度,提高对资源的管理效果。In the embodiment of the present invention, when it is determined that there is a first operation performed by the user for setting the scheduling plan related to the current data task, the management system can respond to the first operation and display at least one metric related to the scheduling plan through the display unit Information, at least one piece of metric information may include resource requirements and constraints during the execution of the current data task, the resource requirements may be used to indicate the limitations of the hardware resources of the server for executing the data task, and the constraints are used to indicate the execution of the current data task The operation requirements of the task on the server, and then, based on one less measurement information, the scheduling plan corresponding to the current data task can be determined, and resource scheduling can be performed based on the scheduling plan, and at least one target server corresponding to the current data task can be determined. A target server can meet the resource requirements and constraints. Therefore, during the scheduling process, displaying at least one measurement information can facilitate the user to view the configuration process of the scheduling plan, improve the accuracy and user experience during the configuration process, and at the same time, the displayed measurement information Including the resource requirements and constraints set by the user, so that the scheduling plan is more in line with the actual needs of the user in the process of processing data, making the scheduling of server resources more accurate, and through user customization, the management system can meet the needs of diversification Computing task requirements, realize fine-grained resource scheduling, and improve the management effect of resources.

前述图2-图6实施例中调度方法的过程和实例同样适用于本实施例的管理系统,通过前述对调度方法的详细描述,本领域技术人员可以清楚的知道本实施例中管理系统的实施例,所以为了说明书的简洁,在此不再详述。The process and examples of the scheduling method in the aforementioned embodiments of Figures 2-6 are also applicable to the management system of this embodiment. Through the foregoing detailed description of the scheduling method, those skilled in the art can clearly understand the implementation of the management system in this embodiment For example, for the sake of brevity, it will not be described in detail here.

本领域内的技术人员应明白,本发明的实施例可提供为方法、系统、或计算机程序产品。因此,本发明可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且,本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。Those skilled in the art should understand that the embodiments of the present invention may be provided as methods, systems, or computer program products. Accordingly, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

本发明是参照根据本发明实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器,使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中,使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品,该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上,使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理,从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

显然,本领域的技术人员可以对本发明进行各种改动和变型而不脱离本发明的精神和范围。这样,倘若本发明的这些修改和变型属于本发明权利要求及其等同技术的范围之内,则本发明也意图包含这些改动和变型在内。Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these modifications and variations.

Claims (10)

1. a kind of dispatching method, applied to management system, which is characterized in that including:
It determines that there are what user carried out to operate with the first of the relevant operation plan of current data task for setting;
First operation is responded, is shown by display interface and believed with the relevant at least one measurement of the current data task Breath;Wherein, at least one metric include user setting for the current data task in the process of implementation Resource requirement and restrictive condition, the resource requirement is used to indicate the requirement to the hardware resource of server, the limitation item Part is used to indicate the service requirement to server;
Based at least one metric, operation plan corresponding with the current data task is determined;
Scheduling of resource is carried out based on the operation plan, determines at least one destination service corresponding with the current data task Device, at least one destination server disclosure satisfy that the resource requirement and the restrictive condition.
2. the method as described in claim 1, which is characterized in that there are users in the display interface for determining the management system Before what is carried out is used to set the user's operation with the relevant operation plan of current data task, the method further includes:
Newly-increased metric of the user for the newly-increased resource measurement of resource measurement collector in the management system is obtained, it is described Resource measurement collector is used to collect the multinomial resource measurement metric parameter corresponding with each resource measurement of respective server;
And obtain the service for execution data task that the user is set by restrictive condition component in the management system The restrictive condition information of device operation;
Response first operation, is shown and the relevant at least one measurement of the current data task by display interface Information, including:
First operation is responded, based on the newly-increased metric and the restrictive condition information, in the display interface Show at least one metric.
3. method as claimed in claim 2, which is characterized in that described based at least one metric, determining and institute The corresponding operation plan of current data task is stated, including:
It determines whether there is and is operated for the second of metric progress at least one described in the display interface, described second It operates as adjusting the operation of the corresponding metric parameter of at least one metric;
If there are the described second operation, worked as based on the metric parameter after adjustment and at least one metric generation with described If the corresponding operation plan of preceding data task or, there is no the described second operation, based at least one metric and its Corresponding metric parameter generates operation plan corresponding with the current data task.
4. the method as described in any claims of claim 1-3, which is characterized in that resource tune is carried out based on the operation plan Degree determines at least one destination server corresponding with the current data task, including:
Determine the corresponding at least one dispatch command of the operation plan, each dispatch command at least one dispatch command Indicate corresponding resource requirement and restrictive condition to server;
Scheduling of resource is carried out based at least one dispatch command, is determined corresponding at least one with the current data task Destination server.
5. method as claimed in claim 4, which is characterized in that described to determine corresponding with the current data task at least one A destination server, including:
It determines to disclosure satisfy that the corresponding resource requirement of the operation plan and limit in multiple servers of the management system connection At least one destination server of condition processed.
6. a kind of management system, which is characterized in that including:
First determining module to be used to set and the relevant operation plan of current data task for determining that there are what user carried out First operation;
Operation module for responding first operation, is shown relevant extremely with the current data task by display interface A few metric;Wherein, at least one metric include user setting for the current data task Resource requirement and restrictive condition in the process of implementation, the resource requirement, which is used to indicate, wants the hardware resource of server It asks, the restrictive condition is used to indicate the service requirement to server;
Second determining module for being based at least one metric, determines tune corresponding with the current data task Degree plan;
Scheduler module carries out scheduling of resource for being based on the operation plan, determines corresponding extremely with the current data task A few destination server, at least one destination server disclosure satisfy that the resource requirement and the restrictive condition.
7. system as claimed in claim 6, which is characterized in that the system also includes:
Acquisition module to be used for setting and current number for there are what user carried out in the display interface for determining the management system Before user's operation according to the relevant operation plan of task, user is obtained for resource measurement collector in the management system The newly-increased metric of newly-increased resource measurement, the resource measurement collector are used to collect the multinomial resource measurement of respective server Metric parameter corresponding with each resource measurement;And it obtains the user and is set by restrictive condition component in the management system The restrictive condition information for the server operation for performing data task put;
The operation module is used for:First operation is responded, based on the newly-increased metric and the restrictive condition information, At least one metric is shown in the display interface.
8. system as claimed in claim 7, which is characterized in that second determining module is used for:
It determines whether there is and is operated for the second of metric progress at least one described in the display interface, described second It operates as adjusting the operation of the corresponding metric parameter of at least one metric;
If there are the described second operation, worked as based on the metric parameter after adjustment and at least one metric generation with described If the corresponding operation plan of preceding data task or, there is no the described second operation, based at least one metric and its Corresponding metric parameter generates operation plan corresponding with the current data task.
9. the system as described in any claims of claim 6-8, which is characterized in that the scheduler module is used for:
Determine the corresponding at least one dispatch command of the operation plan, each dispatch command at least one dispatch command Indicate corresponding resource requirement and restrictive condition to server;
Scheduling of resource is carried out based at least one dispatch command, is determined corresponding at least one with the current data task Destination server.
10. system as claimed in claim 9, which is characterized in that the scheduler module is used for:
It determines to disclosure satisfy that the corresponding resource requirement of the operation plan and limit in multiple servers of the management system connection At least one destination server of condition processed.
CN201611264306.3A 2016-12-31 2016-12-31 A kind of dispatching method and management system Pending CN108270603A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611264306.3A CN108270603A (en) 2016-12-31 2016-12-31 A kind of dispatching method and management system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611264306.3A CN108270603A (en) 2016-12-31 2016-12-31 A kind of dispatching method and management system

Publications (1)

Publication Number Publication Date
CN108270603A true CN108270603A (en) 2018-07-10

Family

ID=62755398

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611264306.3A Pending CN108270603A (en) 2016-12-31 2016-12-31 A kind of dispatching method and management system

Country Status (1)

Country Link
CN (1) CN108270603A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110543352A (en) * 2019-08-16 2019-12-06 浙江大华技术股份有限公司 task allocation method of scheduling system and related device thereof
CN114153574A (en) * 2021-11-04 2022-03-08 浙江大华技术股份有限公司 Resource setting method and device applied to distributed stream data flow engine

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102393822A (en) * 2011-11-30 2012-03-28 中国工商银行股份有限公司 Batch scheduling system and method
CN103294484A (en) * 2013-06-27 2013-09-11 曙光信息产业(北京)有限公司 Processing method used for FLUENT computing program
CN103365713A (en) * 2012-04-01 2013-10-23 华为技术有限公司 Resource dispatch and management method and device
CN103593323A (en) * 2013-11-07 2014-02-19 浪潮电子信息产业股份有限公司 Machine learning method for Map Reduce task resource allocation parameters
CN103729246A (en) * 2013-12-31 2014-04-16 浪潮(北京)电子信息产业有限公司 Method and device for dispatching tasks
US20150113540A1 (en) * 2013-09-30 2015-04-23 Teradata Corporation Assigning resources among multiple task groups in a database system
CN104655082A (en) * 2013-11-22 2015-05-27 中国航空工业集团公司西安飞机设计研究所 Airplane control plane deflection angle measuring system

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102393822A (en) * 2011-11-30 2012-03-28 中国工商银行股份有限公司 Batch scheduling system and method
CN103365713A (en) * 2012-04-01 2013-10-23 华为技术有限公司 Resource dispatch and management method and device
CN103294484A (en) * 2013-06-27 2013-09-11 曙光信息产业(北京)有限公司 Processing method used for FLUENT computing program
US20150113540A1 (en) * 2013-09-30 2015-04-23 Teradata Corporation Assigning resources among multiple task groups in a database system
CN103593323A (en) * 2013-11-07 2014-02-19 浪潮电子信息产业股份有限公司 Machine learning method for Map Reduce task resource allocation parameters
CN104655082A (en) * 2013-11-22 2015-05-27 中国航空工业集团公司西安飞机设计研究所 Airplane control plane deflection angle measuring system
CN103729246A (en) * 2013-12-31 2014-04-16 浪潮(北京)电子信息产业有限公司 Method and device for dispatching tasks

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110543352A (en) * 2019-08-16 2019-12-06 浙江大华技术股份有限公司 task allocation method of scheduling system and related device thereof
CN110543352B (en) * 2019-08-16 2022-06-07 浙江大华技术股份有限公司 Task allocation method of scheduling system and related device thereof
CN114153574A (en) * 2021-11-04 2022-03-08 浙江大华技术股份有限公司 Resource setting method and device applied to distributed stream data flow engine
CN114153574B (en) * 2021-11-04 2025-10-31 浙江大华技术股份有限公司 Resource setting method and device applied to distributed stream data stream engine

Similar Documents

Publication Publication Date Title
US8918448B2 (en) Application component decomposition and deployment
US10324754B2 (en) Managing virtual machine patterns
US8667019B2 (en) Placement goal-based database instance consolidation
US9396028B2 (en) Scheduling workloads and making provision decisions of computer resources in a computing environment
US20150051942A1 (en) Smart cloud workload balancer
US20130275965A1 (en) Control of java resource runtime usage
CN105007323A (en) System and method for arranging cloud resources
WO2018019232A1 (en) Method, device and system for stream computation
JP2017524202A (en) Policy-based resource management and allocation system
CN117296042A (en) Application management platform for hyperconverged cloud infrastructure
US10411977B2 (en) Visualization of workload distribution on server resources
WO2017088794A1 (en) System and method for resource management
US11347735B2 (en) Scalable query processing
CN108279882B (en) Framework generation method, device, equipment and computer readable medium
CN111045786B (en) Container creation system and method based on mirror image layering technology in cloud environment
JP7639005B2 (en) Autonomous Terraforming on Cloud Infrastructure
CN111522651B (en) Managing metadata for a distributed processing system
CN106295220A (en) A kind of medical data management method, device and Medically Oriented Data System
US12488051B2 (en) Failure tolerant and explainable state machine driven hypergraph execution
CN115469887A (en) Release method and device, electronic device and storage medium of cloud-native application
CN104572275B (en) A kind of process loading method, apparatus and system
US20260010570A1 (en) Failure tolerant graph execution
US20150163111A1 (en) Managing resources in a distributed computing environment
US20250371118A1 (en) Collaborative user interfaces to configure parameters of api endpoints
CN108270603A (en) A kind of dispatching method and management system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20180710

RJ01 Rejection of invention patent application after publication