CN116560811A - A simulation system and method applied to dispatching system - Google Patents

Abstract

The application relates to a simulation system and a simulation method applied to a dispatching system, wherein the system comprises a data acquisition module for acquiring key data of the dispatching system; the data preprocessing module is used for cleaning and converting task information of each task to obtain a directed acyclic graph of all tasks of the scheduling system; the simulation module is used for performing simulation operation on tasks of the dispatching system by adopting a shortest path algorithm and a dispatching strategy according to the directed acyclic graph to obtain a simulation result; the result output module is used for outputting simulation results. The system simulates and runs tasks of the dispatching system by adopting a shortest path algorithm and a dispatching strategy according to the directed acyclic graph in a simulation module, so that different dispatching strategies are simulated to provide data for the optimized dispatching system, and the performance and efficiency of the dispatching system are improved; the dispatching system can be automatically tested and adjusted through the simulation system, maintainability and expandability of the dispatching system are improved, and testing cost is reduced.

Description

A simulation system and method applied to dispatching system

technical field

The present application relates to the field of computer technology, in particular to a simulation system and method applied to a scheduling system.

Background technique

With the continuous development of computer technology, simulation technology has been widely used in various fields. In the scheduling system, because the actual operation will affect the scheduling system, it is difficult to actually test the scheduling system, and the simulation technology becomes a feasible method.

Simulation technology can establish a virtual system environment on the computer to simulate the real operation process and results. This method can avoid the impact of actual operation on the system. At the same time, it can test the stability and stability of the system by adjusting parameters and simulating different scenarios. reliability.

The operation status of the existing dispatching system is a black box, which does not have the ability to visualize, and cannot quantify the response time and processing capacity of the system. The stability and reliability of the dispatching system depend entirely on the subjective judgment of the operation and maintenance personnel based on past experience, and lack of objective Data basis: The cost of optimizing the scheduling algorithm in the scheduling system is high and the efficiency is low. Optimizing the scheduling algorithm in the production environment has a high risk factor and the optimization effect cannot be guaranteed.

Contents of the invention

The embodiment of the present application provides a simulation system and method applied to a dispatching system, which is used to solve the technical problems of high test cost and low efficiency in the performance test of the existing dispatching system that cannot be quantified and the results are judged manually.

In order to achieve the above purpose, the embodiment of the present application provides the following technical solutions:

A simulation system applied to a scheduling system, comprising a data acquisition module, a data preprocessing module, a simulation module and a result output module;

The data collection module is used to collect key data of the scheduling system, and the key data includes several tasks and task information of each task;

The data preprocessing module is used to clean and convert the task information of each task to obtain a directed acyclic graph of all tasks in the scheduling system;

The simulation module is used to simulate the tasks of the scheduling system by using the shortest path algorithm and scheduling strategy according to the directed acyclic graph, and obtain the simulation results;

The result output module is used to output simulation results.

In one implementation, the data collection module includes an identification submodule, a collection submodule, a transmission submodule, and an exception handling submodule;

The identification sub-module is used to obtain different data sources of dispatching system data collection;

The collection sub-module is used to select different collection methods according to different data sources to collect the data of the scheduling system, and obtain the key data of the scheduling system;

The transmission sub-module is used to transmit the collected key data to the data preprocessing module;

The abnormality processing sub-module is used for processing the collection abnormality during the data collection process of the dispatching system, so that the collection sub-module normally collects data.

In an implementation manner, the data preprocessing module includes a cleaning submodule and a conversion submodule;

The cleaning submodule is used to clean abnormal data in the task information of each task to obtain first task information; perform normalization processing on the first task information to obtain second task information;

The conversion submodule is used to convert all tasks in the key data into a directed acyclic graph according to the dependencies between tasks;

Wherein, the dependency relationship is taking tasks as nodes and taking task dependencies as edges.

In an implementation manner, the data preprocessing module includes a data storage submodule, and the data storage submodule is configured to store the directed acyclic graph into a Nebula graph database.

In an implementation manner, the data preprocessing module includes an optimization submodule, and the optimization submodule is used for indexing and query optimization of the Nebula graph database.

In one implementation, the simulation module includes a calculation sorting submodule, an allocation submodule, and a simulation adjustment submodule;

The calculation sorting submodule is used to calculate each task based on the directed acyclic graph using the shortest path algorithm to obtain the shortest running time of each task under different computing resources; according to the shortest running time of each task from Go to the big sort to get the task set;

The assigning submodule is configured to assign each task of the task set to a corresponding computing resource according to the priority of the task to obtain an assignment plan;

The simulation adjustment sub-module is used to simulate and run the tasks assigned to the computing resources according to the allocation scheme to obtain simulation data; and adjust the allocation scheme according to the simulation data using a scheduling strategy to re-use the tasks of the scheduling system Run the simulation and get the simulation results.

In an implementation manner, the content of the scheduling policy includes resource allocation optimization, task reallocation, task priority adjustment, and topology optimization.

The present application also provides a simulation method applied to a dispatching system, comprising the following steps:

Collect key data of the scheduling system, the key data includes several tasks and task information of each task;

Cleaning and converting the task information of each task to obtain a directed acyclic graph of all tasks in the scheduling system;

According to the directed acyclic graph, the tasks of the scheduling system are simulated and run by using the shortest path algorithm and the scheduling strategy, and the simulation results are obtained and output.

In one implementation, the cleaning and converting of the task information of each task to obtain the directed acyclic graph of all tasks in the scheduling system includes:

cleaning the abnormal data in the task information of each task to obtain the first task information; performing normalization processing on the first task information to obtain the second task information;

Convert all tasks in the key data into a directed acyclic graph according to the dependencies between tasks;

Wherein, the dependency relationship is taking tasks as nodes and taking task dependencies as edges.

In an implementation manner, the tasks of the scheduling system are simulated and run by using the shortest path algorithm and scheduling strategy according to the directed acyclic graph, and the obtained simulation results include:

Based on the directed acyclic graph, the shortest path algorithm is used to calculate each task, and the shortest running time of each task under different computing resources is obtained; according to the shortest running time of each task, the task set is obtained by sorting from small to large;

Allocating each task of the task set to a corresponding computing resource according to the priority of the task to obtain an allocation scheme;

Perform simulation operation on tasks allocated to computing resources according to the allocation scheme to obtain simulation data; and adjust the allocation scheme according to the simulation data using a scheduling strategy to re-use the simulation operation of tasks in the scheduling system to obtain simulation results;

Wherein, the content of the scheduling policy includes resource allocation optimization, task redistribution, task priority adjustment and topology optimization.

It can be seen from the above technical solutions that the embodiments of the present application have the following advantages: the simulation system and method applied to the scheduling system, the system includes a data acquisition module for collecting key data of the scheduling system; a data preprocessing module for Clean and convert the task information of each task to obtain the directed acyclic graph of all tasks in the scheduling system; the simulation module is used to simulate the tasks of the scheduling system by using the shortest path algorithm and scheduling strategy according to the directed acyclic graph Run to get the simulation result; the result output module is used to output the simulation result. The simulation system applied to the scheduling system obtains the directed acyclic graph through the data preprocessing module. In the simulation module, the tasks of the scheduling system are simulated using the shortest path algorithm and scheduling strategy according to the directed acyclic graph, and different scheduling strategies are obtained. and simulation data under parameters to simulate different scheduling strategies to provide data for optimizing the scheduling system, thereby improving the performance and efficiency of the scheduling system; the simulation system can also be used to automatically test and adjust the scheduling system to improve the maintainability of the scheduling system and scalability, reducing the cost of testing. It solves the technical problems that the performance test of the existing dispatching system cannot be quantified and the results are judged manually, and the test cost is high and the efficiency is low.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application or in the prior art, the accompanying drawings that need to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present application. Those skilled in the art can also obtain other drawings based on these drawings without any creative effort.

Fig. 1 is the frame diagram of the simulation system applied to the scheduling system described in the embodiment of the present application;

Fig. 2 is the frame diagram of the acquisition process of the data acquisition module in the simulation system applied to the scheduling system described in the embodiment of the present application;

Fig. 3 is the process frame diagram of the data preprocessing module in the simulation system applied to the scheduling system described in the embodiment of the present application;

FIG. 4 is a flowchart of a simulation module in a simulation system applied to a scheduling system according to an embodiment of the present application.

Detailed ways

In order to make the purpose, features and advantages of the present application more obvious and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the following The described embodiments are only some of the embodiments of the present application, but not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

In the embodiment of this application, in the scheduling system, the simulation technology can be used in the following aspects:

It can be used to predict the response time and processing capacity of the system. By establishing a virtual system environment, different load situations can be simulated, so as to predict the response time and processing capacity of the system.

Applied to evaluate the stability and reliability of the system, the stability and reliability of the system can be evaluated by simulating different abnormal conditions and faults.

Applied to optimize scheduling algorithms, different scheduling algorithms and parameters can be simulated to compare their advantages and disadvantages, thereby optimizing the scheduling algorithm.

In a word, the application method of simulation technology in dispatching system can help us better understand and test the system, so as to continuously optimize and improve the performance and reliability of the system.

The embodiment of the present application provides a simulation system and method applied to a dispatching system, which is used to solve the technical problems that the performance test of the existing dispatching system cannot be quantified and the results are judged manually, and the test cost is high and the efficiency is low.

Embodiment one:

Fig. 1 is a frame diagram of a simulation system applied to a scheduling system according to an embodiment of the present application.

As shown in FIG. 1 , the embodiment of the present application provides a simulation system applied to a scheduling system, including a data acquisition module 10 , a data preprocessing module 20 , a simulation module 30 and a result output module 40 connected in sequence.

In the embodiment of the present application, the data collection module 10 may be used to collect key data of the scheduling system, and the key data includes several tasks and task information of each task.

It should be noted that the task information includes task basic information, dependency information, running information, and computing resource occupation information. In this embodiment, the basic information includes task name, task description, task type, task creation time, task start time, task end time, task status (such as running, successful, failed) and so on. Dependency information includes dependencies between tasks, for example, task A needs to be executed after task B is completed. The running information includes the task execution log, execution time, execution result, error message, etc. The computing resource occupancy information includes the occupancy of resources such as CPU, memory, and disk used during task execution, as well as the minimum, maximum, and average amount of resources required by the task.

In the embodiment of the present application, the data preprocessing module 20 may be used to clean and convert task information of each task to obtain a directed acyclic graph of all tasks in the scheduling system.

It should be noted that the data preprocessing module 20 processes the data acquired by the data acquisition module 10 to obtain a directed acyclic graph of task execution in the scheduling system, which facilitates the simulation of subsequent scheduling system tasks.

In the embodiment of the present application, the simulation module 30 can be used to simulate the tasks of the scheduling system by using the shortest path algorithm and the scheduling strategy according to the directed acyclic graph, and obtain the simulation results.

It should be noted that the simulation module 30 simulates the running conditions of tasks on different computing resources according to the data obtained by the data preprocessing module 20, and obtains simulation results of running scheduling system tasks on different computing resources. In this embodiment, the simulation results include performance index data, task completion data, resource occupation and fault data. Performance indicators include performance indicators such as operating efficiency, resource utilization, and throughput. Task completion data includes data such as the running time of each task simulation run, run failure or run interruption, and resource occupancy includes the simulation run time of each task. Resource footprint.

In the embodiment of the present application, the simulation system applied to the dispatching system can check whether the dispatching system meets the expected performance requirements through performance indicators. Use the task completion data to analyze whether each task is completed as expected, whether it times out, fails or is interrupted, and counts the average completion time and longest completion time of each task. The resource occupancy of the scheduling system can be analyzed through resource occupancy, and the average utilization rate and maximum utilization rate of each computing resource can be counted for optimization. The fault data can be used to check whether there is a fault in the dispatching system. If so, it is necessary to locate and solve the problem in time to avoid affecting the normal operation of the system. According to the analysis of the simulation results, the simulation system applied to the dispatching system can optimize the dispatching strategy and improve the overall performance of the dispatching system to the greatest extent.

In the embodiment of the present application, the result output module 40 may be used to output simulation results.

It should be noted that the result output module 40 is responsible for outputting the simulation results, which may be the execution order of the tasks, resource occupation, execution time of the tasks, and the like.

The application provides a simulation system applied to the scheduling system, including a data acquisition module for collecting key data of the scheduling system; a data preprocessing module for cleaning and converting task information of each task to obtain the scheduling The directed acyclic graph of all tasks in the system; the simulation module is used to simulate the tasks of the scheduling system by using the shortest path algorithm and scheduling strategy according to the directed acyclic graph to obtain the simulation results; the result output module is used to output the simulation results . The simulation system applied to the scheduling system obtains the directed acyclic graph through the data preprocessing module. In the simulation module, the tasks of the scheduling system are simulated using the shortest path algorithm and scheduling strategy according to the directed acyclic graph, and different scheduling strategies are obtained. and simulation data under parameters to simulate different scheduling strategies to provide data for optimizing the scheduling system, thereby improving the performance and efficiency of the scheduling system; the simulation system can also be used to automatically test and adjust the scheduling system to improve the maintainability of the scheduling system and scalability, reducing the cost of testing. It solves the technical problems that the performance test of the existing dispatching system cannot be quantified and the results are judged manually, and the test cost is high and the efficiency is low.

It should be noted that the simulation system applied to the dispatching system can simulate the dispatching system to perform simulation operation under different load conditions, and obtain the simulation results; through the simulation results, the response time and processing capacity of the dispatching system can be predicted, in order to optimize the dispatching system Provide data reference. In this embodiment, the simulation system applied to the scheduling system can simulate the scheduling system to perform simulation operation under different abnormal or fault data, and obtain the simulation results; check whether there is a fault in the scheduling system according to the simulation results, and if there is , it is necessary to locate and solve problems in time to avoid affecting the normal operation of the system; realize the evaluation of the stability and reliability of the dispatching system, and improve the fault tolerance and reliability of the dispatching system. The simulation system applied to the dispatching system can avoid the impact of actual operation on the dispatching system, improve the stability and reliability of the dispatching system, and at the same time test the performance and efficiency of the dispatching system by simulating different scenarios, providing reference for system optimization .

FIG. 2 is a frame diagram of the acquisition process of the data acquisition module in the simulation system applied to the scheduling system described in the embodiment of the present application.

In one embodiment of the present application, the data collection module 10 includes an identification submodule, a collection submodule, a transmission submodule and an exception handling submodule;

The identification sub-module is used to obtain different data sources for data collection of the dispatching system;

The collection sub-module is used to select different collection methods according to different data sources to collect the data of the dispatching system and obtain the key data of the dispatching system;

The transmission sub-module is used to transmit the collected key data to the data preprocessing module;

The exception processing sub-module is used for processing the collection abnormality during the data collection process of the dispatching system, so that the collection sub-module collects data normally.

As shown in FIG. 2 , in the embodiment of the present application, the identification sub-module is mainly used to identify the data source of the collected data, and collect different data according to different data sources.

It should be noted that the basic information of the task is collected through the data source of the task management module, the running information of the task is collected through the data source of the task scheduling module, and the computing resource occupation information of the task is collected through the data source of the computing resource management module Collected. The identification sub-module is used to identify the data source and establish a connection with the collection sub-module to collect data.

As shown in FIG. 2 , in the embodiment of the present application, the collection sub-module needs to select a suitable data collection method according to the characteristics of different data sources. For example, the basic information of the task is stored in the task management module, which can be obtained by querying with SQL statements. The running information of the task is obtained by collecting the log files in the task scheduling module and parsing the log files. The computing resource occupation information of tasks needs to be collected through the monitoring system.

It should be noted that the key data collected by the collection sub-module is stored in the MySQL database, which is convenient for data calling.

In the embodiment of the present application, the transmission sub-module mainly transmits key data to the data preprocessing sub-module.

In the embodiment of the present application, during the data collection process, some abnormal situations may occur, such as unavailable data sources, interruption of the collection process, and the like. The exception handling sub-module mainly handles these exceptions, including recording logs, reconnecting data sources, and terminating collection.

FIG. 3 is a flowchart of a data preprocessing module in a simulation system applied to a scheduling system according to an embodiment of the present application.

As shown in Figure 3, in one embodiment of the present application, the data preprocessing module 20 includes a cleaning submodule, a conversion submodule and a data storage submodule;

The cleaning sub-module is used to clean the abnormal data in the task information of each task to obtain the first task information; normalize the first task information to obtain the second task information;

The conversion sub-module is used to convert all tasks in the key data into a directed acyclic graph according to the dependencies between tasks;

The data storage sub-module is used to store the directed acyclic graph to the Nebula graph database;

Among them, the dependency relationship is taking tasks as nodes and taking task dependencies as edges.

It should be noted that there may be irregular or incomplete abnormal data in the collected task information. Through the cleaning sub-module, it is necessary to clean and normalize the abnormal data in the task information of the task to ensure the accuracy and consistency of the data. sex. After cleaning and normalizing the first task information, use the conversion sub-module according to the dependencies between tasks, take the task as a node (where the basic information of the task is converted into a node attribute), and take the task dependency as an edge (wherein the task's The running time and resource occupation are converted into edge attributes), and converted into a directed acyclic graph. In this embodiment, the constructed DAG is stored in the Nebula graph database of the data storage submodule. Among them, Nebula is an open source distributed graph database that supports efficient storage and query of large-scale graph data. In the process of storing data in the data storage sub-module, the persistent storage of data is realized by mapping the node and edge information of the directed acyclic graph to the nodes and edges of the Nebula graph database.

In one embodiment of the present application, the data preprocessing module 20 includes an optimization sub-module, which is used for indexing and query optimization of the Nebula graph database.

It should be noted that the simulation system applied to the scheduling system can perform indexing and query optimization on the Nebula graph database for common query operations in task scheduling, such as task priority and task dependencies, to improve data efficiency. Query efficiency.

FIG. 4 is a flowchart of a simulation module in a simulation system applied to a scheduling system according to an embodiment of the present application.

As shown in Figure 4, in one embodiment of the present application, the simulation module 40 includes a calculation sorting submodule, an allocation submodule and a simulation adjustment submodule;

The calculation and sorting sub-module is used to calculate each task based on the shortest path algorithm based on the directed acyclic graph, and obtain the shortest running time of each task under different computing resources; sort according to the shortest running time of each task from small to large, Get task set;

The allocation sub-module is used to allocate each task of the task set to the corresponding computing resource according to the priority of the task, and obtain the allocation scheme;

The simulation adjustment sub-module is used to simulate and run the tasks assigned to the computing resources according to the allocation scheme to obtain simulation data; and adjust the allocation scheme by adopting the scheduling strategy according to the simulation data to re-use the simulation operation of the tasks of the scheduling system to obtain the simulation results;

Among them, the content of the scheduling policy includes resource allocation optimization, task redistribution, task priority adjustment and topology optimization.

In the embodiment of this application, the calculation and sorting sub-module obtains the preprocessed directed acyclic graph DAG from the Nebula graph database, uses the shortest path algorithm to traverse the directed acyclic graph, and obtains the The shortest runtime to run under the resource, for easy ordering of tasks.

It should be noted that the shortest path algorithm is a type of algorithm that finds the shortest path from the start point to the end point in a weighted directed graph (or undirected graph). Among them, the shortest path algorithm includes Dijkstra algorithm, Bellman-Ford algorithm and A* algorithm. These algorithms can calculate the shortest path from the starting point to each vertex, and give the corresponding path length. Exemplarily, the calculation and sorting sub-module adopts the Dijkstra algorithm, and uses the Dijkstra algorithm based on the graph-based shortest path algorithm to calculate the shortest running time of each task under different computing resources. It is necessary to establish a relationship between computing resources and tasks. Weighted directed graph. Each node in the weighted directed graph represents the running status of a task on a specific computing resource, an edge represents the dependency conversion relationship between two nodes, and the weight of the edge represents the time required for the completion of the previous task.

In the embodiment of this application, the assignment sub-module assigns the tasks to different computing resources according to the runnability according to the priority of the tasks, and obtains the assignment plan, which includes the actual start time and completion of each task time.

It should be noted that the priority of tasks is mainly defined according to business requirements and dependencies between tasks. Generally speaking, the higher the task priority, the earlier it will be executed. Tasks can be prioritized in terms of urgency, business value, dependencies, and resource usage. In this embodiment, the urgency level refers to prioritizing tasks with higher urgency, such as important data backup, emergency repair, and the like. Business value refers to prioritizing tasks that contribute more to business value, such as advertising display and transaction processing. Dependency refers to prioritizing the tasks currently at the top of the dependency chain to ensure that subsequent tasks can be completed in a timely manner. Resource occupation refers to prioritizing tasks that require a large amount of resources to avoid resource waste and system congestion.

In this embodiment of the application, the simulation adjustment sub-module first simulates the execution of the task to obtain the running time of the task; second, it updates the resource occupation of the task in real time, and adjusts the scheduling strategy according to the operation of the task. In this embodiment, during the simulation process, the scheduling strategy needs to be adjusted in real time according to the running conditions of the tasks, so that the entire simulation system can run more efficiently. The content of the scheduling strategy includes resource allocation optimization, task redistribution, task priority adjustment and topology optimization. The resource allocation optimization scheduling strategy is to optimize the resource allocation scheme according to the current resource occupancy and task requirements to maximize resource utilization. The scheduling strategy of task reallocation is based on the running time of a certain task is longer than expected, it can be suspended and reassigned to other idle computing resources to reduce the overall running time. The scheduling policy of task priority adjustment is based on the change of the priority of certain tasks, and the priority can be adjusted to ensure the rationality of the task processing sequence. The scheduling strategy of topology optimization is based on the changes in the dependencies, which can optimize the topology between tasks to reduce the dependencies between tasks and improve the overall operating efficiency.

In the embodiment of the present application, the simulation system applied to the scheduling system also collects and summarizes the simulation results obtained from each simulation run into a report, which facilitates query and statistics of simulation data.

Embodiment two:

The embodiment of the present application provides a simulation method applied to a scheduling system, including the following steps:

Collect the key data of the scheduling system, the key data includes several tasks and the task information of each task;

Clean and transform the task information of each task to obtain a directed acyclic graph of all tasks in the scheduling system;

According to the directed acyclic graph, the shortest path algorithm and scheduling strategy are used to simulate the tasks of the scheduling system, and the simulation results are obtained and output.

In the embodiment of this application, the task information of each task is cleaned and converted, and the directed acyclic graph of all tasks in the scheduling system is obtained, including:

Clean the abnormal data in the task information of each task to obtain the first task information; normalize the first task information to obtain the second task information;

Convert all tasks in the key data into a directed acyclic graph according to the dependencies between tasks;

Among them, the dependency relationship is taking tasks as nodes and taking task dependencies as edges.

In the embodiment of this application, the tasks of the scheduling system are simulated and run according to the directed acyclic graph using the shortest path algorithm and scheduling strategy, and the simulation results obtained include:

Based on the directed acyclic graph, the shortest path algorithm is used to calculate each task, and the shortest running time of each task under different computing resources is obtained; according to the shortest running time of each task, the task set is obtained by sorting from small to large;

Assign each task of the task set to the corresponding computing resource according to the priority of the task, and obtain the allocation scheme;

According to the allocation plan, the tasks assigned to the computing resources are simulated and run to obtain the simulation data; and according to the simulation data, the dispatch strategy is used to adjust the allocation plan, and the tasks of the scheduling system are simulated again to obtain the simulation results;

Among them, the content of the scheduling policy includes resource allocation optimization, task redistribution, task priority adjustment and topology optimization.

It should be noted that the content of the steps in the method of the second embodiment corresponds to the modules in the system of the first embodiment, and the content of the simulation system applied to the scheduling system has been elaborated in the first embodiment, and will not be repeated in the second embodiment Describe the steps in the method in detail.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and other media that can store program codes.

As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still understand the foregoing The technical solutions recorded in each embodiment are modified, or some of the technical features are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the application.

Claims (10)

1. The simulation system applied to the dispatching system is characterized by comprising a data acquisition module, a data preprocessing module, a simulation module and a result output module;

the data acquisition module is used for acquiring key data of the scheduling system, wherein the key data comprises a plurality of tasks and task information of each task;

the data preprocessing module is used for cleaning and converting the task information of each task to obtain a directed acyclic graph of all tasks of the scheduling system;

the simulation module is used for performing simulation operation on tasks of the dispatching system by adopting a shortest path algorithm and a dispatching strategy according to the directed acyclic graph to obtain a simulation result;

and the result output module is used for outputting a simulation result.

2. The simulation system applied to a dispatching system of claim 1, wherein the data acquisition module comprises an identification sub-module, an acquisition sub-module, a transmission sub-module and an exception handling sub-module;

the identification sub-module is used for acquiring data sources of different data acquisition of the dispatching system;

the acquisition submodule is used for selecting different acquisition modes according to different data sources to acquire the data of the dispatching system and obtain key data of the dispatching system;

the transmission sub-module is used for transmitting the collected key data to the data preprocessing module;

the abnormal processing submodule is used for processing the abnormal acquisition condition in the data acquisition and scheduling system so that the data acquisition submodule can normally acquire data.

3. The simulation system applied to a dispatching system of claim 1, wherein the data preprocessing module comprises a cleaning sub-module and a conversion sub-module;

the cleaning submodule is used for cleaning abnormal data in the task information of each task to obtain first task information; normalizing the first task information to obtain second task information;

the conversion sub-module is used for converting all tasks in the key data into directed acyclic graphs according to the dependency relationship among the tasks;

the dependency relationship takes a task as a node and takes task dependency as an edge.

4. The simulation system for a dispatch system of claim 1, wherein the data preprocessing module comprises a data storage sub-module for storing the directed acyclic graph to a Nebula graph database.

5. The simulation system for a dispatch system of claim 4, wherein the data preprocessing module comprises an optimization sub-module for indexing and query optimization of the Nebula graph database.

6. The simulation system applied to the scheduling system according to claim 1, wherein the simulation module comprises a calculation ordering sub-module, an allocation sub-module and a simulation adjustment sub-module;

the calculation sequencing submodule is used for calculating each task by adopting a shortest path algorithm based on the directed acyclic graph to obtain the shortest running time of each task under different calculation resources; sequencing from small to large according to the shortest running time of each task to obtain a task set;

the allocation submodule is used for allocating each task of the task set to a corresponding computing resource according to the priority of the task to obtain an allocation scheme;

the simulation adjustment sub-module is used for performing simulation operation on the tasks distributed to the computing resources according to the distribution scheme to obtain simulation data; and adjusting the allocation scheme by adopting a scheduling strategy according to the simulation data, and re-adopting the task of the scheduling system to perform simulation operation to obtain a simulation result.

7. The simulation system applied to a scheduling system of claim 6, wherein the contents of the scheduling policy include resource allocation optimization, task reassignment, task priority adjustment, and topology optimization.

8. A simulation method applied to a dispatching system, comprising the following steps:

acquiring key data of a scheduling system, wherein the key data comprises a plurality of tasks and task information of each task;

cleaning and converting the task information of each task to obtain a directed acyclic graph of all tasks of the scheduling system;

and performing simulation operation on tasks of the dispatching system by adopting a shortest path algorithm and a dispatching strategy according to the directed acyclic graph, obtaining a simulation result and outputting the simulation result.

9. The simulation method applied to the dispatching system according to claim 8, wherein the step of performing cleaning and conversion processing on the task information of each task to obtain a directed acyclic graph of all tasks of the dispatching system comprises the steps of:

cleaning abnormal data in the task information of each task to obtain first task information; normalizing the first task information to obtain second task information;

converting all tasks in the key data into a directed acyclic graph according to the dependency relationship between the tasks;

the dependency relationship takes a task as a node and takes task dependency as an edge.

10. The simulation method applied to the dispatching system according to claim 8, wherein the performing simulation operation on the tasks of the dispatching system by adopting a shortest path algorithm and a dispatching strategy according to the directed acyclic graph to obtain a simulation result comprises:

calculating each task by adopting a shortest path algorithm based on the directed acyclic graph to obtain the shortest running time of each task under different calculation resources; sequencing from small to large according to the shortest running time of each task to obtain a task set;

distributing each task of the task set to a corresponding computing resource according to the priority of the task to obtain a distribution scheme;

performing simulation operation on the tasks distributed to the computing resources according to the distribution scheme to obtain simulation data; the allocation scheme is adjusted by adopting a scheduling strategy according to the simulation data, and the tasks of the scheduling system are adopted again to perform simulation operation, so that a simulation result is obtained;

the content of the scheduling strategy comprises resource allocation optimization, task redistribution, task priority adjustment and topology optimization.