CN102999317B - Towards the elasticity multi-process service processing method of many tenants - Google Patents

Abstract

The invention discloses a kind of elasticity multi-process service processing method towards many tenants, comprise the steps: the internal process of computer system to be divided into system process and tasks carrying process; System process comprises host process, host's detection procedure, tasks carrying process detection process three processes, and wherein host process is responsible for the management to tasks carrying process, and whether host's detection procedure is responsible for timing detection host process and is survived; Whether tasks carrying process detection process is responsible for Detection task executive process and is survived.The present invention under multi-tenant environment, can effectively guarantee the isolation of data between each user.On the other hand, the present invention can the quantity of elasticity increase and decrease tasks carrying process, and the service condition to resource in the various service processing of conservative control, saves the execution time of task greatly.

Description

Multi-tenant-oriented elastic multi-process service processing method

technical field

The invention relates to a process management method, in particular to a multi-tenant-oriented flexible multi-process service processing method, which belongs to the technical field of computer management.

Background technique

Computer operations are performed in units of processes. A process is an execution activity of a program on a computer, and each process is executed step by step in accordance with the programmed sequence. In principle, one CPU can only be assigned to one process, that is, only one process can run on one CPU at a time. If the program logic is too complex, the calculation time will take a long time, and the server resources will not be released for a long time, and the program efficiency is very low. After the user sends a task request, it takes a long time to receive a response, so the user experience is poor.

With the rapid development of computer technology, the emergence of server clusters, and the improvement of CPU computing power, hardware resources such as multi-core CPU and server memory are no longer the bottleneck. Thus, the realization of multi-process concurrent processing becomes possible. If a business logic program can be divided into multiple processes to execute concurrently, the original program running time can be divided into multiple parallel processes, which can be executed faster and utilize server resources more fully.

Using multi-process concurrent processing technology to divide large-scale and complex businesses into multiple sub-services and hand them over to different processes for processing. In the past, it was necessary to wait for the entire service task to be completed before it can be completed. Now only the largest sub-task needs to be completed to complete the entire task. In this way, the user can quickly get a system response and optimize the user experience. For example, it needs to process 1 million lists. If it is single-process processing, it is assumed that it takes M seconds to execute. If multi-process processing is used, for example, 10 processes are processed on average, it only needs to execute M/10 seconds to complete the processing. .

In addition, if the performance is simply improved from the perspective of the server, the price of the advanced server is extremely high, and it is difficult to achieve the efficiency of several times or even dozens of times. However, using multi-process concurrent processing technology can use ordinary servers to use different concurrency schemes according to different businesses in terms of efficiency, and sometimes even improve performance by hundreds of times.

However, the existing multi-process concurrent processing technology still has deficiencies in process management. For example, in a multi-tenant environment, that is, sharing the same system or program components in a multi-user environment, it is difficult to ensure the isolation of data between users. sex.

Contents of the invention

The technical problem to be solved by the present invention is to provide a multi-tenant-oriented elastic multi-process service processing method.

For realizing above-mentioned purpose of the invention, the present invention adopts following technical scheme:

A multi-tenant-oriented flexible multi-process service processing method, comprising the following steps:

Divide the internal processes of the computer system into system processes and task execution processes;

The system process includes three processes: a host process, a host detection process, and a task execution process detection process, wherein the host process is responsible for the management of the task execution process, and the host detection process is responsible for regularly detecting whether the host process is alive , if it is found that the host process is dead, then restart a host process; the task execution process detection process is responsible for detecting whether the task execution process is alive, if it is found that a task execution process is dead, restart a task execution of this type process.

Preferably, the host detection process and the task execution detection process mutually execute a heartbeat detection mechanism.

Wherein preferably, according to the requirement of multi-tenancy, multiple task execution processes are run simultaneously.

Preferably, before the actual service task processing code is executed, the code file is registered in the XML configuration file of the computer system, the management page starts the process, the computer system transmits parameters according to the management page, reads the configuration information, and finds the corresponding code file , loading the code file into the memory and running the program according to the rules to create the task execution process.

Preferably, the service request is first submitted to the task cache queue classified according to the task execution process, and then the corresponding task execution process takes it out of the task cache queue for processing.

Preferably, the number of task execution processes is flexibly increased or decreased according to the number of service requests and the current ability to process service requests.

Among them, preferably, real-time monitoring of memory, CPU, network usage, and comprehensive analysis; if the analysis conclusion is that server resources are idle, start other task execution processes, and schedule this task execution process to memory; if the analysis conclusion is that server If the resource exceeds the load, some task execution processes will be migrated to another server.

Preferably, the foreground page monitors the status of the task execution process; when a user submits a service task, a unique task identification number is generated, and the task parameter submitted by the user is stored in the database using the task identification number as a key value , put the task identification number into the task execution process; the task execution process reads data from the database, and then executes the service task.

The flexible multi-process service processing method provided by the present invention can effectively ensure the isolation of data between users in a multi-tenant environment. Tenants do not see each other's data. Through data isolation, the security of multi-tenant data is guaranteed, and the multi-tenant function can be realized. On the other hand, the present invention can flexibly increase or decrease the number of task execution processes, reasonably control resource usage in various service processes, and greatly save task execution time.

Description of drawings

Fig. 1 is a schematic diagram of a multi-tenant environment applying the elastic multi-process service processing method provided by the present invention.

Detailed ways

As shown in FIG. 1 , the elastic multi-process service processing method provided by the present invention is applied in a multi-tenant environment. In order to share the same system or program components in a multi-user environment, and still ensure the isolation of data between users, the present invention separates the task execution process from the system process, and the same set of computers applying the flexible multi-process service processing method The system (referred to as the multi-process service processing system) can run multiple task execution processes at the same time, and make them perform their own duties without interfering with each other, ensuring data isolation.

In the present invention, the internal processes of the computer system are divided into system processes and task execution processes. The system process includes three processes: the host process, the host detection process, and the task execution process detection process. The host process is responsible for the management of the task execution process, such as adding, deleting, sleeping, and waking up. The host detection process is responsible for periodically checking whether the host process is alive. If the host process is found to be dead, a host process is restarted to ensure that the host process remains alive during the system execution. The task execution process detection process is responsible for detecting whether the task execution process is alive, and restarts a task execution process of this type if a task execution process is found to be dead. Among them, the host process, the host detection process and the task execution process detection process mutually execute the heartbeat detection mechanism, thereby ensuring that the three system processes supervise each other, and no process will die.

The above-mentioned heartbeat detection mechanism is as follows: the system process includes three processes: the host process, the host detection process, and the task execution process detection process. They detect each other. When a certain system process is found to be abnormal, the original system process is killed and a similar type of system process, and handed over to the computer system to take over, thereby ensuring that the three system processes in the computer system are alive. The implementation method of the heartbeat detection mechanism is regular ping, which uses regular ping to check whether the server is running normally. Ping is to send an ICMP packet to the remote host. If no response is received within a given period of time, it will be considered as a timeout, that is, the server crashes; if a response is received, the received ICMP packet will be analyzed to obtain the TTL, type and time, and then judge the server health status.

In order to ensure the independence between each task execution process, the present invention adopts the registered process management mode, that is, the actual service task processing code is separated from the multi-process service processing system, and its code file is registered into the XML configuration file of the computer system before code execution In the process, the management page starts the process. The computer system automatically reads the configuration information according to the parameters passed by the management page, finds the corresponding code file, loads it into the memory and runs the program according to the rules to create a task execution process. The task execution process can be used repeatedly in order to reduce the overhead of creating the process.

In the above registration process management method, a computer system can run various service processing programs according to the configuration, and then can make full use of server resources, so that different programs can share one server. In an embodiment of the present invention, the registered process management method is realized through an online management system. The online management system is a central server that can manage the resources of different servers, including resource allocation, process scheduling, etc., and allocate server tasks according to the server load, and assign more tasks to servers with low load, so that different servers can be used by multiple programs.

The following shows part of the program pseudo-code for implementing the registered process management method.

In the elastic multi-process service processing method provided by the present invention, a task cache queue is used to schedule service requests. The service request is first submitted to the task cache queue classified according to the task execution process, and then the corresponding task execution process takes it out of the task cache queue for processing. Using the task cache queue can firstly be used as a buffer for service requests to reduce the impact on the server, secondly sort the service request tasks according to time, and keep the sequence, and finally, the tasks can be automatically adjusted according to the unprocessed tasks in the task cache queue The number of execution processes, increase the processing process.

The elastic multi-process service processing method provided by the present invention judges whether the current task execution process is sufficient according to the number of unexecuted tasks in the task cache queue. If it is not enough, add the number of task execution processes of the same type to make it faster to process service tasks. If the number in the current task cache queue is empty, reduce the number of task execution processes, reduce server resource usage, save resources, and allocate them reasonably.

A notable feature of this elastic multi-process service processing method is the flexible use of multi-process mode, that is, according to the number of service requests and the current ability to process service requests, the number of task execution processes can be elastically increased or decreased, and the resource allocation in various service processing can be reasonably controlled. Usage. This flexible use of multiprocessing can be realized by a single program component. This program component monitors the usage of system memory, CPU, network, etc. in real time, and conducts a comprehensive analysis. If the analysis concludes that a large number of server resources are idle, then this component will start other task execution processes, schedule this task execution process to memory, and upgrade the server. If the analysis concludes that the server resources are overloaded, the program component will migrate some task execution processes to another server at an appropriate time to achieve efficient operation of the process.

Part of the program pseudocode for flexible use of multi-process is as follows:

The above flexible use of multi-process method greatly saves the execution time of tasks, but server resources are limited, so service requests need to be processed separately. If it is a large task and a high-complexity operation, the task is divided into multiple subtasks, and multi-process processing is used for fast execution. In general, when dealing with simple service tasks, the traditional single-process processing is still used. According to the business service request, customize the processing method related to the service request, so that the resources can be allocated reasonably.

Another notable feature of the elastic multi-process service processing method is lock-free multi-process processing. The specific instructions are as follows:

Multi-process processing can reduce task processing time, divide large tasks into multiple small tasks, and process them by multiple processes to reduce processing time. In the present invention, the data uses the NoSQL (non-relational database) service to store the parameters passed by the service request, so that the parameter sharing of multiple processes and multiple servers can be realized. The front page performs status monitoring and other services on the task execution process. When a user submits a service task, a unique task identification number in the multi-process service processing system will be generated, and the task identification number will be used as the key value to format and store the task parameters submitted by the user into a non-relational database, such as a NoSQL memory database , and put the task identification number into the task execution process. The task execution process reads data from the NoSQL memory database to execute service tasks. Using the NoSQL service can add host servers horizontally, so that multiple servers can share parameters, and infinitely add host processes.

In this flexible multi-process service processing method, a NoSQL memory database is used as a data storage tool, and a queue structure is used as an intermediate fusion agent, so that processes can independently process tasks obtained by themselves without using locks, which can greatly improve operating efficiency.

The multi-tenant-oriented elastic multi-process service processing method provided by the present invention has been described in detail above. For those skilled in the art, any obvious changes made to it without departing from the essence of the present invention will constitute an infringement of the patent right of the present invention and will bear corresponding legal responsibilities.

Claims (7)

1. A multi-tenant-oriented elastic multi-process service processing method, characterized in that: Divide the internal processes of the computer system into system processes and task execution processes; The system process includes three processes: a host process, a host detection process, and a task execution process detection process, wherein the host process is responsible for the management of the task execution process, and the host detection process is responsible for regularly detecting whether the host process is alive ; The task execution process detection process is responsible for detecting whether the task execution process survives, and restarts a task execution process of the same type if a task execution process is found to be dead; The host process, the host detection process and the task execution process detection process mutually execute the heartbeat detection mechanism, when a certain system process is found to be abnormal, then kill the original system process, restart a system process of the same type, and Hand over to the computer system to take over. 2. The elastic multi-process service processing method as claimed in claim 1, characterized in that: To meet the requirements of multi-tenancy, run multiple task execution processes at the same time. 3. The elastic multi-process service processing method as claimed in claim 2, wherein the task execution process adopts a registered process management mode, and the registered process management mode is: The actual service task processing code is separated from the multi-process service processing system. Before the actual service task processing code is executed, the code file is registered in the XML configuration file of the computer system, the management page starts the process, and the computer system transmits parameters according to the management page. Read the configuration information, find the corresponding code file, load the code file into the memory and run the program according to the rules, and create the task execution process. 4. The elastic multi-process service processing method as claimed in claim 2, characterized in that: The service request is first submitted to the task cache queue classified according to the task execution process, and then the corresponding task execution process takes it out of the task cache queue for processing. 5. The elastic multi-process service processing method as claimed in claim 2, characterized in that: According to the number of service requests and the current ability to process service requests, the number of task execution processes can be flexibly increased or decreased. 6. The elastic multi-process service processing method as claimed in claim 5, characterized in that: Real-time monitoring of memory, CPU, and network usage, and comprehensive analysis; if the analysis conclusion is that the server resources are idle, start other task execution processes, and schedule this task execution process to the memory; if the analysis conclusion is that the server resources exceed the load, then Migrate part of the task execution process to another server. 7. The elastic multi-process service processing method as claimed in claim 1, characterized in that: The foreground page monitors the status of the task execution process; when a user submits a service task, a unique task identification number is generated, and the task identification number is used as a key value, and the task parameters submitted by the user are stored in the database. Put the task identification number into the task execution process; the task execution process reads data from the database, and then executes the service task.