CN114428710B - Queue status verification method, device and electronic equipment - Google Patents

Abstract

The embodiment of the present invention relates to a queue status verification method, device and electronic device, the method comprising: obtaining a task to be executed; extracting queue identification information corresponding to the queue for executing the task from the task; obtaining a configuration file and configuration items corresponding to the queue identification information from a pre-built database according to the queue identification information; obtaining the queue of the task to be executed according to the configuration file and the configuration items; verifying the status of the queue according to the configuration file, so that when the status of the queue is determined to be normal later, the queue is used to execute the task. Before executing the task, first verify whether the queue status is normal. The verification process itself takes a small amount of time, avoiding the situation where the queue is not known to be incorrect until the task is executed and the task fails. The execution of queue tasks usually takes a lot of time. In this way, the time taken to execute tasks under abnormal queue status can be greatly saved, thereby improving work efficiency.

Description

Queue status verification method, device and electronic equipment

Technical Field

The embodiments of the present invention relate to the field of computer technology, and in particular to a queue status verification method, device and electronic device.

Background Art

DolphinScheduler is a distributed and scalable visual DAG workflow task scheduling system. It is a top-level project of Apache. In the existing DolphinScheduler framework, Flink, MR, and Spark tasks can be scheduled in sequence, and these tasks require the use of Yarn queues. However, in the existing DolphinScheduler framework, only one Yarn queue name is saved in the management of Yarn queues. When executing tasks, when submitting tasks to the queue according to the yarn queue name, or after submitting tasks to the corresponding queue, it is very likely that the queue does not exist or Yarn executes abnormally. In these cases, you can only realize that there is a fault in the queue after the task execution fails, which wastes a lot of time.

Summary of the invention

The present application provides a queue status verification method, device and electronic device to solve the problem that only one Yarn queue name is saved in the management of the Yarn queue in the DolphinScheduler framework in the prior art, which leads to a series of technical problems introduced above.

In a first aspect, the present application provides a queue status verification method, the method comprising:

Get the tasks to be executed;

Extract the queue identification information corresponding to the queue that executes the task from the task;

According to the queue identification information, obtain the configuration file and configuration items corresponding to the queue identification information from the pre-built database;

Get the queue of tasks to be executed according to the configuration files and configuration items;

Verify the status of the queue according to the configuration file so that when the status of the queue is determined to be normal, the queue can be used to execute tasks.

In a second aspect, the present application provides a queue status verification device, the device comprising:

The acquisition module is used to obtain the tasks to be executed;

An extraction module, used to extract queue identification information corresponding to the queue that executes the task from the task;

The processing module is used to obtain the configuration file and configuration items corresponding to the queue identification information from a pre-built database according to the queue identification information; and obtain the queue of the task to be executed according to the configuration file and configuration items;

The verification module is used to verify the state of the queue according to the configuration file, so as to use the queue to execute tasks when the state of the queue is determined to be normal.

In a third aspect, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

Memory, used to store computer programs;

The processor is used to implement the steps of the queue status verification method of any embodiment of the first aspect when executing the program stored in the memory.

In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the queue status verification method as in any embodiment of the first aspect are implemented.

The above technical solution provided by the embodiment of the present application has the following advantages compared with the prior art:

The method provided in the embodiment of the present application obtains the task to be executed, and extracts the queue identification information corresponding to the queue executing the task from the task. According to the queue identification information, the configuration file and configuration items corresponding to the queue identification information are obtained from the pre-built database. According to the configuration file and the configuration item, the queue is extracted from the pre-built queue library, and then the state of the queue is verified according to the configuration file. Then, when it is determined that the queue state is normal, the queue is used to execute the task. That is, before executing the task, first verify whether the queue state is normal. The verification process itself takes a small amount of time, avoiding the situation where the queue is not correct after the task is executed and the task fails. The execution of queue tasks usually takes a lot of time. In this way, the time taken to execute tasks in the case of abnormal queue status can be greatly saved, thereby improving work efficiency.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic flow chart of a queue status verification method provided by an embodiment of the present invention;

FIG2 is a system architecture diagram of DolphinScheduler provided by the present invention;

FIG3 is a DAG diagram provided by the present invention;

FIG4 is a schematic flow chart of a specific execution method for checking the state of a queue provided by the present invention;

FIG5 is a flow chart of another specific execution method for checking the state of a queue provided by the present invention;

FIG6 is a flowchart of another specific execution method for checking the state of a queue provided by the present invention;

FIG7 is a flowchart of another specific execution method for checking the state of a queue provided by the present invention;

FIG8 is a schematic diagram of the overall flow of a queue status verification method provided by the present invention;

FIG9 is a flow chart of the entire process method provided by the present invention, from uploading the configuration file of YARN to completing the task and reporting the result;

FIG10 is a schematic structural diagram of a queue status verification device provided by an embodiment of the present invention;

FIG. 11 is a schematic structural diagram of an electronic device provided in an embodiment of the present invention.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

To facilitate understanding of the embodiments of the present invention, specific embodiments will be further explained below in conjunction with the accompanying drawings. The embodiments do not constitute a limitation on the embodiments of the present invention.

FIG. 1 is a flow chart of a queue status verification method provided by an embodiment of the present invention. Before introducing the steps of the method, an application environment applicable to the method and a corresponding system architecture diagram are first introduced. As shown in FIG. 2 , FIG. 2 is a system architecture diagram corresponding to an embodiment of the present invention. The application environment can be a scenario in which a workflow is scheduled in DolphinScheduler. Before executing a task in a workflow, the method in the embodiment of the present application can be executed to verify the status of the task queue in the execution workflow.

DolphinScheduler is a distributed and scalable visual DAG workflow task scheduling system. It is a top-level project of Apache. Figure 2 shows the system architecture of DolphinScheduler.

Among them, the functions of UI and restful API Server are mainly reflected in the interface, that is, in the display interface, users can configure the workflow and upload the resource files corresponding to the tasks in the workflow to the file system. The file system can be, for example, a distributed file system (Hadoop Distributed File System, referred to as HDFS) or a simple storage server (Simple Storage Service, referred to as S3). It also includes adding workflow metadata to the database and querying the log information of the task. Among them, the task log information is obtained by monitoring the working status of the worker in the worker cluster through the ZK cluster (ZK Cluster). As shown in Figure 2, the ZK cluster is used to complete the service registration of the worker, monitor the working status of the worker through heartbeat or other monitoring means, including querying the log information of the task. Similarly, the ZK cluster is also used to complete the service registration of the master in the master cluster, monitor the service status of the master through heartbeat or other monitoring means, etc. The master in the Master cluster monitors the data changes in the database through data monitoring, and when it monitors that there is a new task, it executes the task segmentation in the directed acyclic graph DAG workflow. Then the tasks are distributed to a worker in the worker cluster in sequence.

The worker assigned the task downloads the configuration file corresponding to the task and completes the corresponding task execution work. The configuration file is downloaded from the file system (the figure shows the download of resource files). Specifically, the types of tasks that can be executed by the worker include but are not limited to the task types shown in the figure, such as sqoop, Flume, Shell, Python, MR, Spark, SQL, Flink, etc.

FIG3 illustrates a DAG graph. When the master splits the tasks in the DAG workflow, it can perform the task splitting work according to the task execution configuration file pre-generated in the interface (the file includes the task execution order, such as executing A first, then executing B and C, and finally executing D; or, the file includes the dependency relationship between tasks, such as before executing task B, first executing task A, before executing task D, first executing tasks B and C, etc.). Then, first send task A to a randomly selected worker in the worker cluster. When the worker completes the work and generates a response message to the master, the master then randomly sends task B and task C to at least one (possibly two) workers. After completing the tasks, task D is finally sent to a random worker.

Worker downloads the configuration files required to execute tasks from the file system, executes specific tasks, monitors the status of tasks and reports to the Master.

DolphinScheduler supports executing various tasks such as Flink, MR, Spark, etc. in the cluster. Through the above system architecture, various types of tasks such as Flink, MR, Spark, etc. can be scheduled, monitored and managed in the order of DAG, reducing the operation and maintenance costs of big data development. Moreover, the DAG graph and the task information therein can be configured by visual drag and drop, which is convenient and easy to use.

The steps of this method are executed by the worker in the above system architecture. The specific steps of the method are as follows, including:

Step 110, obtaining the task to be executed.

That is, the task that receives the master configuration.

Step 120: extracting queue identification information corresponding to the queue that executes the task from the task.

Specifically, the configured task carries queue identification information corresponding to the queue that executes the task.

In an optional example, the queue identification information is, for example, ID information of the queue.

Step 130: According to the queue identification information, a configuration file and configuration items corresponding to the queue identification information are obtained from a pre-built database.

Specifically, in an optional example, as introduced above, the queue identification information may include, for example, ID information of the queue.

According to the ID information of the queue, the queue information corresponding to the queue identification information can be obtained from the pre-built queue library. In a specific example, the pre-built queue library can be understood as the database in FIG. 1 .

After obtaining the queue information, the configuration files and configuration items corresponding to the queue information are downloaded from the pre-built database according to the queue information.

In an optional example, the pre-built database can be understood as the file system in FIG. 1 .

In fact, before executing the task, the embodiment of the present invention further includes: adding the queue information to a pre-built queue library.

Before executing queue storage, the queue must first be verified, including whether the queue exists, whether the queue status is normal, etc. If the queue does not exist, the queue information cannot be added to the queue library. The specific verification process is the same as the verification process for the queue status to be introduced in the embodiment of the present invention, so this process will be described in detail below.

In a specific example, the queue may be a yarn queue, and the queue cluster may be a yarn queue cluster, referred to as a yarn cluster.

After the queue is successfully verified, the queue will be stored. Otherwise, a message indicating that the queue has failed to be stored can be generated and displayed on the interface. This allows the staff to change the configuration file information or upload a new configuration information and verify again until the verification is successful and the queue is stored.

The queue information included in the queue library may include a queue name corresponding to the queue ID and indication information corresponding to the configuration file and indication items corresponding to the queue name. The indication information here may be, for example, a storage path of the configuration file and indication items.

Then, through the queue information, you can find the storage path of the corresponding configuration file in the pre-built database, and download the corresponding configuration file and configuration items in the storage path.

Step 140: Obtain a queue of tasks to be executed according to the configuration file and configuration items.

In an optional example, the configuration file is used to indicate a queue cluster for executing a task, and the configuration item is used to indicate a queue in the queue cluster.

In a specific example, the configuration file includes the host IP and cluster URL of Yarn. Based on these contents, the Yarn cluster can be known. The indicator item indicates which queue in the Yarn cluster the task is to be executed.

Therefore, the queue of tasks to be executed can be obtained through the configuration file and the configuration item. In an optional example, the configuration file may include, but is not limited to, the following files: core-site.xml, hdfs-site.xml, and yarn-site.xml.

Step 150: Verify the state of the queue according to the configuration file.

After obtaining the queue corresponding to the task to be executed, the task cannot be submitted to the queue immediately for execution. Instead, the queue status must be verified in advance. Only when the queue status is confirmed to be normal can the queue be used to execute the task. Otherwise, the queue will be directly prompted to be abnormal so that the staff can handle it in time. Avoid the situation where the queue abnormality is notified after the task is completed, which wastes a lot of running time.

Optionally, in an executable example, when the queue status check process is specifically performed, it can be implemented in the following manner, as shown in FIG. 4, the method includes:

Step 410: parse the configuration file to obtain configuration information.

Step 420: extract a preset first configuration parameter from the configuration information.

Step 430: Execute the operation instruction corresponding to the first configuration parameter according to the first configuration parameter to obtain the operation result.

Step 440: Verify the state of the queue according to the operation result.

In an optional example, the first configuration parameter may include, but is not limited to, one or more of the following: a Yarn Resource Manager host or a Yarn URL address.

When the first configuration parameter is the Yarn Resource Manager host, the executed operation instruction is to connect to the Yarn Resource Manager host through a network command to obtain the connection result. If the connection fails, the queue status is updated to host abnormality. Otherwise, the queue status can be temporarily determined to be normal.

And/or, when the first configuration parameter is a Yarn URL address, the executed operation instruction may include: attempting to build a connection to the Yarn URL address and obtaining a connection result. If the connection fails, it means that the queue status is a service exception, otherwise, it can be temporarily determined that the queue status is normal.

Of course, if the first configuration parameter includes multiple types at the same time, as long as the execution result corresponding to any one of the parameters is abnormal, it means that the queue is abnormal.

In another optional example, verifying the state of the queue according to the configuration file may also include the following method steps, as shown in FIG5 , including:

Step 510: construct a queue client according to the configuration file.

Step 520: Using the queue client, obtain all queue information in the queue cluster to verify whether the queue exists in the queue cluster.

Step 530: When it is determined that the queue does not exist in the queue cluster, it is determined that the queue state is abnormal.

Alternatively, in step 540, when it is determined that the queue exists in the queue cluster, it is determined that the queue state is normal.

Specifically, a client is created according to the configuration file. Then, the client is used to obtain all queue information in the queue cluster pointed to by the configuration file, which can usually be reflected in the form of a list. Then, it is determined whether the queue indicated by the indicator item exists in the queue list. If it exists, it can be determined that the queue status is normal. Otherwise, it is determined that the queue status is abnormal, and the queue status can be updated to a queue error.

Further optionally, after determining that the queue exists in the queue cluster, the method may further include the following method steps to verify the state of the queue, as shown in FIG. 6 , including:

Step 610: Verify the task submission authority of the queue.

Step 620: When it is determined that the task submission authority of the queue does not include task execution, it is determined that the queue state is abnormal.

Step 630: When it is determined that the task submission authority of the queue includes executing tasks, it is determined that the queue state is normal.

That is, the task submission permission of the queue is further checked through the client. If there is no submission permission for the task, the queue status can be updated to insufficient permission to indicate that the queue status is abnormal. Otherwise, the queue status can be updated to sufficient permission to indicate that the queue status is temporarily normal.

In addition to performing the above operations, the method may also include the following operation modes to further verify the queue status, as shown in FIG. 7 , including:

Step 710: Submit a test task to the queue and obtain the execution result.

Step 720: further verify the state of the queue according to the execution result.

Specifically, submitting a test task to a queue and obtaining execution results may include but are not limited to the following situations, for example:

First, if the task fails to run, the queue status can be updated to queue exception.

The second is that the task runs successfully, but the time exceeds the preset time threshold, then the queue status is updated to resource shortage.

The third type is that the task runs successfully and the time is less than or equal to the preset time threshold. The queue status can be updated to sufficient resources and the queue status is temporarily normal.

FIG8 shows a schematic diagram of the overall flow of the above verification process. For details, please refer to FIG8 . The specific execution process has been introduced in detail above, so it will not be described in detail here.

In the case of abnormal queue status, it also includes prompts of the abnormal queue situation to guide users to solve the problem.

For each abnormal state, the prompt information and specific process are as follows:

Host abnormality:

The message "Yarn host cannot be connected, please check the network, Yarn host status, and configuration files" is displayed. The Yarn queue edit box pops up, and the user can modify the Yarn queue information file. After clicking Save, the queue will be verified again and the queue status will be updated.

Service exception:

The message "Yarn service is abnormal, please check the Yarn service status and configuration files" is displayed. The Yarn queue editing box pops up, and the user can modify the Yarn queue information file. After clicking Save, the queue will be verified again and the queue status will be updated.

Queue Error:

The message "Yarn queue does not exist, please check the configuration file in the Yarn service to see if the queue exists" is displayed. The Yarn queue edit box pops up, and the user can modify the Yarn queue information file. After clicking Save, the queue will be verified again and the queue status will be updated.

Insufficient permissions:

The message "Insufficient Yarn queue permissions, please check whether the queue permissions are sufficient and the configuration file in the Yarn service" is displayed. The Yarn queue edit box pops up, and the user can modify the Yarn queue information file. After clicking Save, the queue will be verified again and the queue status will be updated.

Queue exception:

The message "Yarn queue is abnormal, please check whether the Yarn service is running normally and the configuration file" is displayed. The Yarn queue editing box pops up, and the user can modify the Yarn queue information file. After clicking Save, the queue will be verified again and the queue status will be updated.

Insufficient resources:

The message "Insufficient Yarn resources, please confirm whether to continue running" pops up. Click Continue to run the Yarn task.

The queue is normal:

Run the YARN task directly.

It should also be noted that before executing the above-mentioned method steps of the present invention, it is necessary to verify the queue and then complete the operation of warehousing the queue information. For the specific queue verification operation, please refer to the above operation process. Because the configuration file is pre-configured by the staff and stored in the database, even if the configuration task is not executed, the configuration file can be called to verify whether the queue exists. And only when it is determined that the queue exists, the queue information will be added to the queue library, and the queue execution task can be called in the queue library according to the queue information later.

Therefore, the above-mentioned verification work on the queue performed before the embodiment of the present invention does not conflict with all operations subsequently performed by the present invention.

FIG9 illustrates the entire process from uploading the YARN configuration file to the resource center to completing the task and reporting the result. For details, see FIG9, including:

1. The staff uploads the yarn configuration file to the file system in advance.

The upload operation mentioned here is performed only once. That is, all configuration files are uploaded to the file system. Later, when selecting configuration files based on the yarn cluster, there is no need to manually upload them again, but you can flexibly select them through the interface in the DolphinScheduler system.

For example, suppose that the DolphinScheduler system includes 3 workers. The configuration files of the 3 workers all point to yarn cluster A, but tasks A and B in task six are submitted to cluster A, and task C is submitted to cluster B. Because the workers all point to cluster A, the task execution fails. The means in the prior art require that task A be paused after execution, and the staff must manually point the configuration files on all workers to cluster B, and then execute tasks B and C, which is cumbersome. Moreover, this operation is only feasible in theory and is difficult to implement in actual execution. In the embodiment of the present application, by directly adding configuration items in the DolphinScheduler system, queues can be selected through the configuration items. When executing tasks, the configuration files corresponding to different queues can be directly downloaded from the file system according to the selected queues. There is no need for staff to manually upload operation files repeatedly, thereby expanding the functions of DolphinScheduler and improving ease of use.

2. Configure the yarn queue in queue management.

That is, the yarn queue information is stored in the warehouse. Before storage, the verification work listed above needs to be performed. Only after the verification is successful can it be stored in the warehouse.

3. When staff configure tasks, they can only select queues that have been created and are in normal status.

4. The system runs the workflow. If there is an unavailable queue in the workflow (at this time, the above-mentioned queue verification operation can also be performed), the workflow cannot run; if it does not exist, the workflow will be sent to the Master.

5. The Master disassembles the tasks in the workflow, encapsulates queue information and corresponding file dependency information, and sends the tasks to the Worker.

6. Worker downloads the corresponding configuration file based on the queue information encapsulated by Master.

7. According to the configuration file, verify the running status of the Yarn queue. If the Yarn queue runs normally, execute 8; otherwise, execute 9.

8. Submit the task, monitor the task status, report to the Master, and use the API to display the task status.

9. Report the abnormal status of the Yarn queue to the Master, and the Master will mark the status of the queue as unavailable, and the status of the task operation as failed.

The corresponding operation details in the overall operation process have been introduced in detail above, so I will not go into details about them.

The queue status verification method provided by the embodiment of the present invention obtains the task to be executed, and extracts the queue identification information corresponding to the queue executing the task from the task. According to the queue identification information, the configuration file and configuration items corresponding to the queue identification information are obtained from the pre-built database. According to the configuration file and the configuration item, the queue is extracted from the pre-built queue library, and then the status of the queue is verified according to the configuration file. Then, when it is determined that the queue status is normal, the queue is used to execute the task. That is, before executing the task, first verify whether the queue status is normal. The verification process itself takes a small amount of time, avoiding the situation where the queue is incorrect only after the task is executed and the task fails. The execution of queue tasks usually takes a lot of time. In this way, the time taken to execute tasks under abnormal queue status can be greatly saved, thereby improving work efficiency.

Moreover, the configuration files and configuration items are pre-configured in the pre-built database, and users do not need to upload them manually. Even when the task needs to be adaptively switched to a queue in another queue cluster for execution, the configuration file in the worker does not need to be manually uploaded and replaced by the user again. This effect is particularly evident when there are a large number of workers. Furthermore, precisely because the configuration files have been pre-configured in the pre-built database, when a task temporarily needs to switch queue clusters, the workflow does not need to be paused, but the subsequent task execution work is completed directly by calling the configuration file, which makes the operation more flexible and convenient, greatly improving work efficiency.

FIG. 10 is a queue status verification device provided by an embodiment of the present invention, the device comprising: an acquisition module 101 , an extraction module 102 , a processing module 103 , and a verification module 104 .

An acquisition module 101 is used to acquire tasks to be executed;

An extraction module 102 is used to extract queue identification information corresponding to a queue for executing a task from a task;

The processing module 103 is used to obtain a configuration file and a configuration item corresponding to the queue identification information from a pre-built database according to the queue identification information; and obtain a queue of tasks to be executed according to the configuration file and the configuration item;

The verification module 104 is used to verify the state of the queue according to the configuration file, so as to use the queue to execute the task when it is subsequently determined that the state of the queue is normal.

Optionally, the processing module 103 is specifically configured to obtain queue information corresponding to the queue identification information from a pre-built queue library according to the queue identification information;

According to the queue information, download the configuration files and configuration items corresponding to the queue information from the pre-built database.

Optionally, the configuration file is used to indicate the queue cluster that executes the task, and the configuration item is used to indicate the queue in the queue cluster.

Optionally, the verification module 104 is specifically used to parse the configuration file and obtain configuration information;

Extracting a preset first configuration parameter from the configuration information;

According to the first configuration parameter, executing an operation instruction corresponding to the first configuration parameter to obtain an operation result;

Based on the operation results, verify the status of the queue.

Optionally, the verification module 104 is further used to construct a queue client according to the configuration file;

Use the queue client to obtain all queue information in the queue cluster to verify whether the queue exists in the queue cluster;

When it is determined that the queue does not exist in the queue cluster, it is determined that the queue state is abnormal;

Otherwise, the queue status is determined to be normal.

Optionally, the verification module 104 is further used to verify the task submission permission of the queue after determining that the queue exists in the queue cluster;

When it is determined that the task submission authority of the queue does not include the task execution, it is determined that the queue state is abnormal;

Otherwise, the queue status is determined to be normal.

Optionally, the verification module 104 is further configured to submit a test task to the queue and obtain an execution result before executing a task using the queue when it is determined that the state of the queue is normal;

Based on the execution results, further verify the status of the queue.

A queue status verification device provided by an embodiment of the present invention obtains a task to be executed, and extracts queue identification information corresponding to the queue that executes the task from the task. According to the queue identification information, a configuration file and a configuration item corresponding to the queue identification information are obtained from a pre-built database. According to the configuration file and the configuration item, a queue is extracted from a pre-built queue library, and then the status of the queue is verified according to the configuration file. Then, when it is determined that the queue status is normal, the queue is used to execute the task. That is, before executing the task, first verify whether the queue status is normal. The verification process itself takes a small amount of time, which avoids the situation where the queue is incorrect only after the task is executed and the execution of the task fails. The execution of queue tasks usually takes a lot of time. In this way, the time taken to execute tasks under abnormal queue status can be greatly saved, thereby improving work efficiency.

Moreover, the configuration files and configuration items are pre-configured in the pre-built database, and users do not need to upload them manually. Even when the task needs to be adaptively switched to a queue in another queue cluster for execution, the configuration file in the worker does not need to be manually uploaded and replaced by the user again. This effect is particularly evident when there are a large number of workers. Furthermore, precisely because the configuration files have been pre-configured in the pre-built database, when a task temporarily needs to switch queue clusters, the workflow does not need to be paused, but the subsequent task execution work is completed directly by calling the configuration file, which makes the operation more flexible and convenient, greatly improving work efficiency.

As shown in FIG11 , an embodiment of the present application provides an electronic device in which a DolphinScheduler system is running. The electronic device includes a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

Memory 113, used for storing computer programs;

In one embodiment of the present application, the processor 111 is used to implement the queue status verification method provided by any one of the aforementioned method embodiments when executing the program stored in the memory 113, including:

Get the tasks to be executed;

Extract the queue identification information corresponding to the queue that executes the task from the task;

According to the queue identification information, obtain the configuration file and configuration items corresponding to the queue identification information from the pre-built database;

Get the queue of tasks to be executed according to the configuration files and configuration items;

Verify the status of the queue according to the configuration file so that when the status of the queue is determined to be normal, the queue can be used to execute tasks.

Optionally, according to the queue identification information, obtain queue information corresponding to the queue identification information from a pre-built queue library;

According to the queue information, download the configuration files and configuration items corresponding to the queue information from the pre-built database.

Optionally, the configuration file is used to indicate the queue cluster that executes the task, and the configuration item is used to indicate the queue in the queue cluster.

Optionally, parse the configuration file to obtain configuration information;

Extracting a preset first configuration parameter from the configuration information;

According to the first configuration parameter, executing an operation instruction corresponding to the first configuration parameter to obtain an operation result;

Based on the operation results, verify the status of the queue.

Optionally, verify the queue status according to the configuration file, including:

Construct a queue client according to the configuration file;

Use the queue client to obtain all queue information in the queue cluster to verify whether the queue exists in the queue cluster;

When it is determined that the queue does not exist in the queue cluster, it is determined that the queue state is abnormal;

Otherwise, the queue status is determined to be normal.

Optionally, after confirming that the queue exists in the queue cluster, the process further includes: verifying the task submission permission of the queue;

When it is determined that the task submission authority of the queue does not include the task execution, it is determined that the queue state is abnormal;

Otherwise, the queue status is determined to be normal.

Optionally, when it is determined that the queue is in a normal state, before using the queue to execute the task, the following is further included:

Submit a test task to the queue and obtain the execution result;

Based on the execution results, further verify the status of the queue.

An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the queue status verification method provided in any of the aforementioned method embodiments are implemented.

It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

The above is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features applied herein.

Claims (8)

1. A queue status verification method, characterized in that the method comprises: Get the tasks to be executed; Extracting queue identification information corresponding to the queue that executes the task from the task; According to the queue identification information, obtaining a configuration file and a configuration item corresponding to the queue identification information from a pre-built database; According to the configuration file and the configuration item, obtaining a queue for executing the task; Verify the state of the queue according to the configuration file, so that when it is subsequently determined that the state of the queue is normal, the queue is used to execute the task; The step of verifying the state of the queue according to the configuration file specifically includes: Parsing the configuration file to obtain configuration information; Extracting a preset first configuration parameter from the configuration information; According to the first configuration parameter, executing an operation instruction corresponding to the first configuration parameter to obtain an operation result; Verifying the state of the queue according to the operation result; The verifying the state of the queue according to the configuration file further includes: According to the configuration file, construct a queue client; Using the queue client, obtaining all queue information in the queue cluster to verify whether the queue exists in the queue cluster; When it is determined that the queue does not exist in the queue cluster, determining that the queue state is abnormal; Otherwise, it is determined that the queue status is normal. 2. The method according to claim 1, characterized in that the step of obtaining, according to the queue identification information, a configuration file and a configuration item corresponding to the queue identification information from a pre-built file library specifically comprises: According to the queue identification information, obtaining queue information corresponding to the queue identification information from a pre-built queue library; According to the queue information, configuration files and configuration items corresponding to the queue information are downloaded from a pre-built database. 3. The method according to claim 1 or 2 is characterized in that the configuration file is used to indicate a queue cluster that executes the task, and the configuration item is used to indicate a queue in the queue cluster. 4. The method according to claim 3, characterized in that after determining that the queue exists in the queue cluster, the method further comprises: Verify the task submission permission of the queue; When it is determined that the task submission authority of the queue does not include executing the task, determining that the queue state is abnormal; Otherwise, it is determined that the queue status is normal. 5. The method according to any one of claims 1, 2, or 4, characterized in that when it is determined that the state of the queue is normal, before using the queue to execute the task, the method further comprises: Submit a test task to the queue and obtain the execution result; According to the execution result, the state of the queue is further verified. 6. A queue status verification device, characterized in that the device comprises: The acquisition module is used to obtain the tasks to be executed; An extraction module, used to extract queue identification information corresponding to the queue executing the task from the task; A processing module, configured to obtain, from a pre-built database, a configuration file and a configuration item corresponding to the queue identification information according to the queue identification information; and obtain, according to the configuration file and the configuration item, a queue for executing the task; A verification module, used to verify the state of the queue according to the configuration file, so as to use the queue to execute the task when it is subsequently determined that the state of the queue is normal; Wherein, the verification module is specifically used to extract a preset first configuration parameter from the configuration information; According to the first configuration parameter, executing an operation instruction corresponding to the first configuration parameter to obtain an operation result; Verifying the state of the queue according to the operation result; According to the configuration file, construct a queue client; Using the queue client, obtaining all queue information in the queue cluster to verify whether the queue exists in the queue cluster; When it is determined that the queue does not exist in the queue cluster, determining that the queue state is abnormal; Otherwise, it is determined that the queue status is normal. 7. An electronic device, characterized in that it comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; A processor, for implementing the steps of the queue status verification method described in any one of claims 1 to 5 when executing a program stored in a memory. 8. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the queue status verification method according to any one of claims 1 to 5 are implemented.