CN114546587A - A method for expanding and shrinking capacity of online image recognition service and related device - Google Patents

Abstract

The present application discloses a method for expanding and shrinking capacity of an online image recognition service and a related device. The method monitors the number of images to be processed in the online image recognition service, so as to determine the execution of a prediction task according to the number of detected images to be processed. The amount of device resources required. Based on the preset conversion relationship, determine the preset monitoring indicators corresponding to the number of images to be processed in the container management system, and control the container management system to perform expansion and contraction operations on the containers held by the online image recognition service according to the preset monitoring indicators, so as to Make the amount of resources held by the expanded and contracted containers meet the amount of device resources. Finally, the prediction task is performed according to the amount of resources held. The above process uses the number of images to be processed as an evaluation index for performing expansion and shrinkage operations to reduce the occurrence of expansion and shrinkage misoperations, and divides the physical device cards into virtual device cards for performing prediction tasks through container encapsulation. , which improves the resource utilization of the server.

Description

A method for expanding and shrinking capacity of online image recognition service and related device

technical field

The invention relates to the technical field of data processing, and in particular, to a method for expanding and shrinking capacity of an online image recognition service and a related device.

Background technique

The online image recognition service is to provide a stable high-throughput prediction service online with the help of the deep learning online service framework and the computing resources of the Central Processing Unit (CPU). This service is used to identify and predict the images to be processed uploaded by the client according to the preset identification model. In order to reasonably allocate the resources occupied by each prediction task, in the related art, the amount of resources used for executing the prediction task is dynamically allocated through automatic expansion and contraction operations.

The traditional way of expanding and shrinking capacity is based on the CPU resource usage of the online image recognition service as the evaluation index, and by expanding and shrinking the computing resources required to perform prediction tasks, so that the online image recognition service can dynamically adapt to traffic peaks and Different resource demands during trough periods. This method of directly using CPU resource occupancy as a judging indicator is often prone to misoperations of expansion and contraction. For example, when other systems unrelated to the online image recognition service occupy a lot of CPU resources, the CPU occupancy rate will increase. When the online image recognition service detects an increase in CPU usage, it will perform scaling operations to obtain more computing resources. However, since the reason for the increase in CPU resource utilization is not caused by the online image recognition service, there should be no need to perform a capacity expansion or reduction operation on the online image recognition service. This leads to the occurrence of expansion and contraction errors and reduces the resource utilization of the server.

SUMMARY OF THE INVENTION

The present invention provides a capacity expansion/reduction method and a related device for an online image recognition service, which are used to use the number of images to be processed as an evaluation index for performing capacity expansion/reduction operations to reduce the occurrence of capacity expansion/reduction errors. The virtual device card used for performing prediction tasks is divided into the card, so as to improve the resource utilization rate of the server.

In a first aspect, an embodiment of the present invention provides a method for expanding or shrinking an online image recognition service, the method comprising:

Monitoring the number of images to be processed in the online image recognition service, and determining the amount of equipment resources required to perform the prediction task according to the number of the monitored images to be processed;

A preset monitoring index corresponding to the number of images to be processed in the container management system is determined based on a preset conversion relationship; wherein the container management system is used to manage the containers held by the online image recognition service; each The container is encapsulated with graphics processor resources, the container is the allocation unit of the resource amount held by the online image recognition service, and the logical encapsulation of the container is used for intercepting the invocation of the application programming interface of the unified computing device architecture library;

The container management system is controlled to perform capacity expansion and contraction operations on the containers held by the online image recognition service according to the preset monitoring indicators, so that the amount of resources held by the expanded and contracted containers can meet the requirements of the device resources. amount; perform the forecasting task according to the amount of resources held;

Wherein, when the expansion operation is performed, resources that are the same as the target resource amount are divided from the physical device cards of the resource cluster, and the resources are packaged in containers to construct a virtual device card; the target resource amount is based on the device resource amount. and the amount of resources held;

When the shrinking operation is performed, a preset deletion operation is performed on the container currently held by the online image recognition service, so that the response time of the online image recognition service is within a preset time range.

This embodiment of the present application determines the amount of equipment resources required to perform the prediction task according to the detected number of images to be processed by monitoring the number of images to be processed in the online image recognition service. Determine the preset monitoring indicators corresponding to the number of images to be processed in the container management system based on the preset conversion relationship, and control the container management system to perform expansion and contraction operations on the containers held by the online image recognition service according to the preset monitoring indicators, so as to Make the amount of resources held by the expanded and contracted containers meet the amount of device resources. Finally, the prediction task is performed according to the amount of resources held. The above process implements the division of the virtual device card for performing the prediction task from the physical device card in the manner of logical encapsulation, which improves the resource utilization rate of the server. In addition, the number of images to be processed can more accurately reflect the peak and trough of business traffic. The above process uses the number of images to be processed as an evaluation index for performing expansion and shrinkage operations, which can reduce the occurrence of expansion and shrinkage misoperations.

In some possible embodiments, determining the amount of equipment resources required to perform the prediction task based on the monitored number of the images to be processed includes:

Obtain the delay duration of the prediction task and the processing rate of the image processor;

The amount of device resources is determined according to the number of images to be processed, the delay duration and the processing rate.

The embodiment of the present application determines the amount of equipment resources required to execute the prediction task according to the delay requirement for executing the prediction task, the processing rate of the image processor, and the number of images to be processed, so as to improve the prediction accuracy of the amount of equipment resources.

In some possible embodiments, determining the preset monitoring indicators corresponding to the number of images to be processed in the container management system based on the preset conversion relationship includes:

If the number of the to-be-processed images is greater than the first monitoring threshold, determining that the number of the to-be-processed images corresponds to a preset monitoring index representing the execution of the expansion operation;

If the number of the to-be-processed images is less than the second monitoring threshold, it is determined that the number of the to-be-processed images corresponds to a preset monitoring index representing the execution of a scaling operation; wherein the second monitoring threshold is less than the first monitoring threshold threshold;

If the number of images to be processed is between the first monitoring threshold and the second monitoring threshold, it is determined that the number of images to be processed corresponds to a preset monitoring indicator indicating that no expansion or reduction operation needs to be performed.

In this embodiment of the present application, when the number of images to be processed is greater than the first monitoring threshold, it is determined that the number of images to be processed corresponds to a preset monitoring indicator representing the execution of the expansion operation; when the number of images to be processed is less than the second monitoring threshold, it is determined that the number of images to be processed corresponds to The number corresponds to the preset monitoring metrics that characterize the scaling operation. In the above process, a corresponding monitoring threshold is set for the number of images to be processed to determine whether to perform a capacity expansion or reduction task, so as to reduce the occurrence of wrong operation of capacity expansion or reduction.

In some possible embodiments, before dividing the virtual resources with the same amount of target resources from the physical device cards of the resource cluster, the method further includes:

Detecting the resource cluster, and obtaining a physical device card that satisfies the device resource amount;

The step of dividing the graphics processor resources with the same amount of target resources from the physical device cards of the resource cluster, and performing container encapsulation on the graphics processor resources to construct a virtual device card, includes:

Selecting the physical device card with the least graphics processing resources as the target physical device card, and dividing the graphics processor resource with the same amount of the target resources from the target physical device card;

Perform container encapsulation on the graphics processor resource, and use the encapsulated container as the virtual device card.

In the embodiment of the present application, after selecting the physical device card with the smallest video memory as the target physical device card, the graphics processor resource with the same amount of target resources is divided from the target physical device card, and the resource is packaged in a container to construct a virtual device card. The container encapsulation is used to intercept the call of the unified computing device architecture library to the application programming interface of the container. Through the above method, the minimum allocation unit of resources held by the online service system can be changed from the original physical device card to the physical device card. part of the virtual resources, thereby improving the resource utilization of the server.

In some possible embodiments, the method further includes:

When detecting the resource cluster, if it is not detected that there is a physical device card that meets the device resource amount in the resource cluster, performing a preset number of resource rechecks on the resource cluster based on a preset time interval;

During the execution of the resource rechecking process, if an entity device card satisfying the amount of the device resources is detected, the entity device card with the least graphics processing resources is selected as the target entity device card;

If it is not detected that there is a physical device card in the resource cluster that meets the device resource amount, a prompt indicating that there is currently no available resource is output, and the current capacity expansion task ends.

When detecting a resource cluster in this embodiment of the present application, if it is not detected that there is a physical device card that meets the amount of device resources in the resource cluster, a preset number of resource rechecks are performed on the resource cluster based on a preset time interval. In the process of performing resource rechecking, if a physical device card satisfying the amount of device resources is detected, the physical device card with the least graphics processing resources is selected as the target device card. Through the above rechecking method, the situation of re-uploading the prediction task can be reduced, and the user satisfaction can be improved.

In some possible embodiments, performing a preset deletion operation on the container currently held by the online image recognition service, so that the response time of the online image recognition service is within a preset time range, includes:

Determine the resource amount of graphics processor resources of each container, and use the container with the smallest amount of resources as the container to be processed;

delete the container to be processed, and monitor the response time of the online image recognition service;

If the response time is not within the preset time range, the to-be-processed container is reselected from the remaining containers, and the preset deletion operation is stopped until the response time is within the preset time range.

When performing the scaling operation in the embodiment of the present application, the container with the smallest amount of graphics processor resources is used as the container to be processed. And according to the comparison result between the response time of the online image recognition service and the preset time range, it is determined whether to continue to delete the container to be processed, so as to improve the resource utilization rate of the server.

In some possible embodiments, the number of images to be processed is determined according to interceptors or filters pre-deployed in the online image recognition service.

This embodiment of the present application pre-deploys the online image recognition service, and obtains the response time of the online image recognition service by pre-deploying an interceptor or filter.

In a second aspect, an embodiment of the present application provides a capacity expansion/reduction device for an online image recognition service, the device comprising:

a flow monitoring module, configured to monitor the number of images to be processed in the online image recognition service, and determine the amount of equipment resources required to perform the prediction task according to the monitored number of the images to be processed;

The indicator determination module is configured to determine the preset monitoring indicators corresponding to the quantity of the images to be processed in the container management system based on the preset conversion relationship; wherein, the container management system is used for maintaining the online image recognition service Some containers are managed; each container is encapsulated with graphics processor resources, the container is the allocation unit of resources held by the online image recognition service, and the logical encapsulation of the container is used to intercept the application of the unified computing device architecture library Program programming interface invocation;

The capacity expansion and contraction module is configured to control the container management system to perform expansion and contraction operations on the containers held by the online image recognition service according to the preset monitoring indicators, so that the containers corresponding to the expanded and contracted containers can be expanded or contracted. The amount of resources held satisfies the amount of equipment resources; the prediction task is performed according to the amount of resources held;

Wherein, when the expansion and contraction module performs the expansion operation, it divides the resources with the same target resource amount from the physical device cards of the resource cluster, and performs container encapsulation on the resources to construct a virtual device card; the target resource amount is Determined according to the amount of equipment resources and the amount of held resources;

When the capacity expansion/reduction module performs a capacity reduction operation, a preset deletion operation is performed on the container currently held by the online image recognition service, so that the response time of the online image recognition service is within a preset time range.

In some possible embodiments, to perform the determining of the amount of equipment resources required to perform the prediction task based on the monitored number of the images to be processed, the flow monitoring module is configured to:

Obtain the delay duration of the prediction task and the processing rate of the image processor;

The amount of device resources is determined according to the number of images to be processed, the delay duration and the processing rate.

In some possible embodiments, the determining a preset monitoring index corresponding to the number of images to be processed in the container management system based on a preset conversion relationship is performed, and the index determining module is configured to:

If the number of the to-be-processed images is greater than the first monitoring threshold, determining that the number of the to-be-processed images corresponds to a preset monitoring index representing the execution of the expansion operation;

If the number of the to-be-processed images is less than the second monitoring threshold, it is determined that the number of the to-be-processed images corresponds to a preset monitoring index representing the execution of a scaling operation; wherein the second monitoring threshold is less than the first monitoring threshold threshold;

If the number of images to be processed is between the first monitoring threshold and the second monitoring threshold, it is determined that the number of images to be processed corresponds to a preset monitoring indicator indicating that no expansion or reduction operation needs to be performed.

In some possible embodiments, before performing the dividing of virtual resources with the same amount of target resources from the physical device cards of the resource cluster, the capacity expansion and contraction module is further configured to:

Detecting the resource cluster, and obtaining a physical device card that satisfies the amount of device resources;

The step of dividing the graphics processor resources with the same amount of target resources from the physical device cards of the resource cluster, and performing container encapsulation on the graphics processor resources to construct a virtual device card, includes:

Selecting the physical device card with the least graphics processing resources as the target physical device card, and dividing the graphics processor resource with the same amount of the target resources from the target physical device card;

Perform container encapsulation on the graphics processor resource, and use the encapsulated container as the virtual device card.

In some possible embodiments, the capacity expansion/reduction module is further configured to:

When detecting the resource cluster, if it is not detected that there is a physical device card that meets the device resource amount in the resource cluster, performing a preset number of resource rechecks on the resource cluster based on a preset time interval;

During the execution of the resource rechecking process, if an entity device card satisfying the amount of the device resources is detected, the entity device card with the least graphics processing resources is selected as the target entity device card;

If it is not detected that there is a physical device card in the resource cluster that meets the device resource amount, a prompt indicating that there is currently no available resource is output, and the current capacity expansion task ends.

In some possible embodiments, the performing the preset deletion operation on the container currently held by the online image recognition service is performed, so that the response time of the online image recognition service is within a preset time range, and the extension The shrink module is configured as:

Determine the resource amount of graphics processor resources of each container, and use the container with the smallest amount of resources as the container to be processed;

delete the container to be processed, and monitor the response time of the online image recognition service;

If the response time is not within the preset time range, the to-be-processed container is reselected from the remaining containers, and the preset deletion operation is stopped until the response time is within the preset time range.

In some possible embodiments, the number of images to be processed is determined according to interceptors or filters pre-deployed in the online image recognition service.

In a third aspect, an embodiment of the present application also provides an electronic device, including:

processor;

memory for storing instructions executable by the processor;

Wherein, the processor is configured to execute the instructions to implement any method as provided in the first aspect of the present application.

In a fourth aspect, an embodiment of the present application further provides a storage medium, when an instruction in the storage medium is executed by a processor of an electronic device, the electronic device can execute any method provided in the first aspect of the present application .

In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements any of the methods provided in the first aspect of the present application when the computer program is executed by a processor.

Other features and advantages of the present application will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the present application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description, claims, and drawings.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present application more clearly, the following briefly introduces the drawings that need to be used in the embodiments of the present application. Obviously, the drawings introduced below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

Fig. 1 is the K8s system architecture diagram shown in the embodiment of this application;

2a is a flowchart of a method for expanding and shrinking the online image recognition service according to an embodiment of the present application;

FIG. 2b is a schematic diagram of virtual device card allocation according to an embodiment of the application;

FIG. 2c is a schematic diagram of prompt information shown in an embodiment of the present application;

3 is a schematic diagram of a GPUshare component shown in an embodiment of the application;

FIG. 4 is a structural diagram of an apparatus 400 for expanding and shrinking capacity of an online image recognition service according to an embodiment of the present application;

FIG. 5 is a schematic diagram of an electronic device according to an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described clearly and in detail below with reference to the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, "face will mean or, for example, A/B may mean A or B; "and/or" in the text is only an association to describe the associated object Relationship, it means that there can be three kinds of relationships, for example, A and/or B, it can mean that A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of this application, " Plural" means two or more.

In the description of the embodiments of the present application, unless otherwise specified, the term "plurality" refers to two or more, and other quantifiers are similar to it. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention. The application is not used to limit the application, and the embodiments of the application and the features in the embodiments can be combined with each other under the condition of no conflict.

In order to further illustrate the technical solutions provided by the embodiments of the present application, a detailed description is given below in conjunction with the accompanying drawings and specific embodiments. Although the embodiments of the present application provide method operation steps as shown in the following embodiments or the accompanying drawings, more or less operation steps may be included in the method based on routine or without creative work. In steps that logically do not have a necessary causal relationship, the execution order of these steps is not limited to the execution order provided by the embodiments of the present application. In the actual processing process or when the method is executed by the control device, the method may be executed sequentially or in parallel according to the methods shown in the embodiments or the accompanying drawings.

The traditional way of expanding and shrinking capacity is based on the CPU resource usage of the online image recognition service as the evaluation index, and by expanding and shrinking the computing resources required to perform prediction tasks, so that the online image recognition service can dynamically adapt to traffic peaks and Different resource demands during trough periods. This method of directly using the occupancy of CPU resources as a judgment indicator often leads to the occurrence of expansion and shrinkage misoperations.

In addition, when an existing deep learning prediction task is running in a cluster, a graphics processor (Graphic Processing Unit, GPU) device card, ie a physical device card, that satisfies the resource amount required by the prediction task is allocated for each prediction task. And control the prediction model to use the resources held by the physical device card to perform the prediction task. However, usually the amount of resources required to perform a single prediction task is much smaller than the amount of resources held by a single physical device card. The above resource allocation method using the physical device card as the minimum allocation unit will cause waste of server resource utilization, thereby affecting the throughput of the cluster prediction request.

In order to solve the above technical problems, the inventive concept of the present application is to monitor the number of images to be processed in the online image recognition service to determine the amount of equipment resources required to perform prediction tasks according to the number of detected images to be processed. Determine the preset monitoring indicators corresponding to the number of images to be processed in the container management system based on the preset conversion relationship, and control the container management system to perform expansion and contraction operations on the containers held by the online image recognition service according to the preset monitoring indicators, so as to Make the amount of resources held by the expanded and contracted containers meet the amount of device resources. Finally, the prediction task is performed according to the amount of resources held. The above process implements the division of the virtual device card for performing the prediction task from the physical device card in the manner of logical encapsulation, which improves the resource utilization rate of the server. In addition, the number of images to be processed can more accurately reflect the peaks and valleys of business traffic. The above process uses the number of images to be processed as an evaluation index for performing expansion and shrinking operations, which can reduce the occurrence of incorrect expansion and shrinkage operations.

The above-mentioned resource cluster can be understood as a collection of a large number of electronic devices with computing processing capabilities, and these electronic devices can cooperate to complete specific tasks. The electronic device may be a server, a virtual machine, or the like.

In a resource cluster environment, applications are deployed and run in a containerized manner, and a container management system for managing containers is deployed in the resource cluster. At present, the mainstream container management systems on the market are mostly open source container orchestration systems (Kubernetes, K8s). The K8s system adopts a distributed architecture to provide a series of complete functions such as deployment and operation, resource scheduling, service discovery, and dynamic scaling for containerized applications, which improves the convenience of large-scale container cluster management. Figure 1 shows the system architecture diagram of K8s.

Referring to FIG. 1, the K8s cluster 100 mainly includes a management node 110 and a working node 120, and each node can communicate with each other. In the example shown in FIG. 1, there is one management node 110 and multiple working nodes 120. Here, the nodes Can refer to servers in a cluster, but is not limited to physical devices.

Among them, the management node 110 is responsible for the management and control of the cluster, such as application programming interface server (kube-apiserver), controller manager (controller manager), scheduler (kube-scheduler) and distributed persistence running on the management node 110 Storage (etcd) and other critical processes. The worker node 120 is a workload node of the cluster, and the management node 110 will distribute the workload to the worker node 120. The worker node 120 runs key processes such as service proxies (kubelet, kube-proxy), and container runtime (Docker engine). For the specific functions of the above processes constituting the K8s system, reference may be made to the prior art, which will not be described in detail here.

The smallest deployable computing unit that can be created and managed in a K8s cluster 100 is called a collocated container group (pod). A pod represents a logical host containing one or more containers (Containers) that are always co-located and run in a shared context. K8s Cluster 100 also provides a modular API core that enables users to describe specifications for the desired behavior of pods in YAML or JSON objects (also known as PodSpecs). In the following, for the convenience of describing the present application, it is preset that each pod corresponds to one container.

It should be understood that the above K8s system is only an example, and the method of this application does not actually limit the specific system used by the container management system. However, in order to simplify the description, when referring to the container management system later, K8s is still mainly used. example is introduced.

In this application, the application to be run in the container mainly refers to the deep learning prediction task that needs to use the GPU device resources in the cluster. In the K8s cluster 100 , the GPU device can be configured on the worker node 120 . A container that can use GPU device resources is referred to as a GPU container in this application. Since GPU devices are expensive, GPU device resources are usually a relatively scarce resource in a K8s cluster. The above prediction task uses GPU device resources, so it needs to run in a GPU container.

In the prior art, in order to enable the native Docker container to support the use of GPU device resources, a layer of encapsulation is performed on the basis of the Docker container, such as the Nvidia-Docker container. Nvidia-Docker container is a GPU container. The K8s system provides the ability to create and schedule Nvidia-Docker containers, but the minimum granularity of Nvidia-Docker utilization of GPU device resources is one GPU device card (and only one GPU card is used exclusively), and more fine-grained resources are not supported. allocation. This directly results in a waste of GPU device resource utilization, which in turn affects the throughput of cluster prediction requests.

In order to solve the above problems, the present application provides a method for expanding and shrinking the capacity of an online image recognition service, which can be applied to a container management system deployed in a resource cluster, including but not limited to the above-mentioned K8s system.

The technical solutions provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings. Specifically, as shown in Figure 2a, the following steps are included:

Step 201: monitor the number of images to be processed in the online image recognition service, and determine the amount of equipment resources required to perform the prediction task according to the monitored number of the images to be processed;

A resource cluster usually provides resources for multiple microservice systems. In related technologies, the CPU occupancy rate in the resource cluster is often used as a judgment indicator for performing expansion and contraction operations. When other systems unrelated to the online image recognition service occupy more CPU resources, the CPU usage rate will increase. When the online image recognition service detects an increase in CPU resource utilization, it will perform scaling operations to obtain more computing resources.

Since the reason for the increase in CPU resource utilization is not caused by the online image recognition service, there should be no need to perform expansion or reduction operations on the online image recognition service. This leads to the occurrence of expansion and contraction errors, which in turn reduces the resource utilization of the server. Considering that the visitor flow of the online image recognition service can intuitively reflect the business demand of the online image recognition service, the embodiment of the present application determines the amount of equipment resources required to perform the prediction task according to the visitor flow. During implementation, considering that the processing rates of different customer requirements are different, a weight representing the processing rate of customer requirements is set, and the product of the weight and the number of images to be processed can represent the amount of equipment resources required to perform the prediction task.

During implementation, the amount of equipment resources can be determined according to the following formula (1):

y=ω ₁ x ₁ +ω ₂ x ₂ +ω ₃ x ₃ +b Formula (1)

Among them, y is the amount of device resources; ω _n is the preset weight of x _n , n∈[1, 2, 3]; x ₁ is the number of images to be processed; x ₂ is the delay time, that is, the client executes the The delay requirement of the prediction task; x ₃ is the processing rate of the image processor, which means the computing power of the GPU device, which is related to the GPU card model. ) to represent the computing power; b is the constant term of the linear equation, which can be set according to user needs.

In some possible embodiments, the number of images to be processed is determined according to interceptors or filters pre-deployed in the online image recognition service.

Step 202: Determine a preset monitoring index corresponding to the number of images to be processed in a container management system based on a preset conversion relationship; wherein the container management system is used to manage the containers held by the online image recognition service Each container is encapsulated with graphics processor resources, the container is the allocation unit of the online image recognition service holding the amount of resources, and the logical encapsulation of the container is used to intercept calls to the application programming interface of the unified computing device architecture library ;

The container management system is used to manage the containers held by the online image recognition service. The above K8s is a common container management system in the market. The container is the bearer of the graphics processor resources that execute the prediction service. The Unified Computing Device Architecture Library (CUDA) is the K8s driver for the physical device card. Since K8s does not support the sharing of graphics processor resources in the physical device card, the embodiment of this application intercepts all memory-related memory-related data in the CUDA library. APIs and compute-related APIs to limit and isolate GPU usage. In this way, a part of resources can be divided from the physical device card, and the part of resources can be packaged in a container to construct a virtual device card. Therefore, the minimum allocation unit of resources is changed from the physical device card to the container carrying the virtual device card, thereby improving the resource utilization rate of the server.

Specifically, as shown in FIG. 2b, for example, the equipment resources required for the prediction task 1 are 700MB, and the resources held by the idle physical device cards in the resource cluster are 1024MB. At this time, 700MB of virtual resources can be separated from the physical device card for container encapsulation, so as to construct the virtual device card 1 . Then assign the virtual device card 1 to the prediction task 1 for use. At this time, if there is a prediction task 2, and the equipment resources required for the prediction task 2 are 500MB, then 500MB can be divided from the physical device card (currently holding resources are 1024MB-700MB=524MB) to construct a virtual device card 2, and the Virtual device card 2 is used by prediction task 2. In this way, there is no need to allocate two physical device cards to the prediction task 1 and the prediction task 2, which improves the resource utilization rate of the server.

Step 203: Control the container management system to perform an expansion/reduction operation on the container held by the online image recognition service according to the preset monitoring index, so that the amount of resources held corresponding to the expanded/reduced container satisfies all requirements. the amount of equipment resources; perform the prediction task according to the amount of held resources;

In this embodiment of the present application, by setting a corresponding monitoring threshold for the number of images to be processed to determine whether to perform a capacity expansion or reduction task, the occurrence of incorrect operation of capacity expansion or reduction is reduced.

During implementation, the first monitoring threshold and the second monitoring threshold with different sizes can be adaptively defined according to business requirements and hardware processing capabilities. Specifically, if the number of images to be processed is greater than the first monitoring threshold, it is determined that the number of images to be processed corresponds to a preset monitoring index representing the execution of the expansion operation. If the number of images to be processed is less than the second monitoring threshold, it is determined that the number of images to be processed corresponds to the preset monitoring index representing the execution of the shrinking operation; if the number of images to be processed is between the first monitoring threshold and the second monitoring threshold , then it is determined that the number of images to be processed corresponds to a preset monitoring indicator indicating that no expansion or reduction operation is required. Wherein, the above-mentioned second monitoring threshold is smaller than the first monitoring threshold.

When the expansion operation is performed, the resources that are the same as the target resource amount are divided from the physical device cards of the resource cluster, and the resources are containerized to construct a virtual device card; the target resource amount is based on the device resource amount and all the resources. The amount of resources held is determined according to the description;

When the shrinking operation is performed, a preset deletion operation is performed on the container currently held by the online image recognition service, so that the response time of the online image recognition service is within a preset time range.

In some possible embodiments, the above target resource amount is the difference between the device resource amount and the held resource amount. Assuming that the amount of equipment resources required to perform the prediction task is 1000MB, and the total amount of resources (that is, the amount of resources held) corresponding to all containers currently held by the online image recognition service is 700MB, it represents the current holding of the online image recognition service directly. The delay for the resource to perform the prediction task does not meet the business requirement. In this case, it is necessary to provide more resources for the prediction task through the expansion operation.

When performing a capacity expansion operation, firstly, the resource cluster is detected, and a physical device card satisfying the device resource amount is obtained. If there are multiple acquired physical device cards, the physical device card with the least graphics processing resources is selected as the target physical device card. If there is only one, take the physical device card as the target physical device card. Further, the graphics processor resource with the same amount of target resources is divided from the target physical device card, and the graphics processor resource is encapsulated in a container, and the encapsulated container is used as a virtual device card.

In some possible embodiments, when the resource cluster is detected, if it is not detected that there is a physical device card in the resource cluster that meets the device resource amount, the resource cluster is rechecked for a preset number of times based on a preset time interval.

During the resource rechecking process, if a physical device card satisfying the amount of device resources is detected, the physical device card with the least graphics processing resources is selected as the target physical device card. If it is not detected that there is a physical device card that meets the amount of device resources in the resource cluster, as shown in Figure 2c, a prompt indicating that there is currently no available resource is output, and the current capacity expansion task ends.

Assuming that the amount of equipment resources required to perform the prediction task is 500MB, and the total resources corresponding to all containers currently held by the online image recognition service are 1200MB, it represents the delay of directly using the resources currently held by the online image recognition service to execute the prediction task. sufficient to meet business needs.

At this time, in order to improve the resource utilization rate of the server, the capacity reduction operation can be used to release some of the containers currently held by the online image recognition service, so as to achieve the purpose of increasing the delay. However, in the release process, it is necessary to ensure that the response time after the delay is increased can still meet the business requirements.

During implementation, the resource amount of graphics processor resources of each container currently held by the online image recognition service is first determined, and the container with the smallest resource amount is used as the container to be processed. Then delete the to-be-processed container and monitor the response time of the online image recognition service. If the response time is not within the preset time range, the container to be processed is reselected from the remaining containers, and the preset deletion operation is stopped until the response time is within the preset time range.

Specifically, assuming that the response time of business requirements is 100-120ms, and the range of response time required to execute the prediction task is determined to be 80ms according to the current amount of resources held by the online image recognition service, the container with the smallest amount of resources can be used as the pending processing. After deleting the container to be processed, the new response time is determined according to the current amount of resources held. Assuming that the new response time is 110ms, the preset deletion operation can be stopped, that is, this expansion is ended.

Through the above process, the number of images to be processed is used as an evaluation index for performing the capacity expansion/reduction operation, so as to reduce the occurrence of wrong operation of capacity expansion/reduction. In addition, a virtual device card for performing prediction tasks is divided from the physical device card by means of container encapsulation, thereby improving the resource utilization rate of the server.

In order to facilitate understanding of how to implement the container carrying the virtual device card as the minimum allocation unit of resources in the above process, the implementation process is described below with reference to Figure 3:

This application develops a new framework extension to support the sharing of physical device cards in the K8s system. This framework is referred to below as GPUShare. The role of GPUShare is to create and manage custom resources (SharePod) created in K8s. SharePod represents a pod that can connect to a shared physical device card on a container.

When implementing, create a description file of SharePod (SharePodSpec). The content of the description file includes the amount of device resources required for the prediction task, the identifier of the physical device card, and the node name of the physical device card in the resource cluster, etc. The amount of device resources is the demand for device resources when performing the prediction task, which specifically includes the demand for video memory and the demand for computing. Among them, the video memory occupancy requirements are shared in the space mode, and the operation occupancy requirements are shared in the time slice mode. For example, a video memory requirement of 0.5 means that the container can allocate up to 50% of the total memory space of the GPU device, while an operation requirement of 0.5 means that the container should have at least 50% of the kernel execution time in the time slice.

In order to realize the sharing of resources in the physical device card (ie graphics processor resources), GPUShare allocates the physical device card from K8s, and then transfers some of the resources in the physical device card to SharePod. These resources managed by GPUShare are packaged as virtual device cards (virtual GPU, vGPU) through containers. When GPUShare acquires these vGPUs from K8s, the physical locations of these vGPUs can be distributed across multiple nodes in the resource cluster. Thus a vGPU pool can be used to represent the set of all vGPUs managed by GPUShare. When a GPU joins the vGPU pool, it will be assigned a unique identifier (GPUID). As shown in FIG. 3 , the GPUShare component includes a scheduler 301 , a virtual device card manager 302 and a virtual device card database 303 .

When implemented, the scheduler decides the mapping between containers and vGPUs based on the resource state of the resource cluster's current GPU devices and the amount of device resources required to perform the predicted task. Then, the scheduler 301 generates a SharePodSpec using the GPUID value determined by the scheduling policy, and requests the virtual device card manager 302 to create a corresponding SharePod instance. Any scheduling algorithm can be implemented in this component, for example, the binpack algorithm. It can preferentially select idle resources to meet the demand of device resources, but at the same time it is the physical device card with the least remaining resources.

The role of the virtual device card manager 302 is to create a SharePod object, and then initialize the device environment in the container after receiving the SharePodSpec from the scheduler 301 . Specifically, the (vGPU library) sets environment variables and installs the later-mentioned virtual device card database 303 in the container to isolate their GPU usage. That is, the above process can limit and isolate the use of the GPU by encapsulating the virtual resource objects divided from the physical device card to intercept all memory-related APIs and computing-related APIs in the CUDA library.

The life cycle of a vGPU consists of four phases: create, active, idle, and delete. A vGPU is created when the vGPU device manager receives a SharePod request for which GPUID does not exist. If the GPUID in the SharePod request already exists, the virtual device card manager 302 simply retrieves the corresponding UUID from the vGPU pool without creating a new vGPU. The vGPU becomes active when it is bound to the SharePod. When a user in K8s deletes the SharePod, the vGPU is separated from the SharePod. When the vGPU is not bound by any SharePod, it will go into idle state. When the virtual device card manager 302 assigns the idle vGPU to another SharePod, it can become active again. Finally, the virtual device card manager 302 can decide at its own discretion when to delete idle vGPUs and release GPUs to K8s.

Overcommitment of resources often leads to performance interference issues, and in extreme cases, it can cause applications to fail or crash, such as resource over-allocation of physical device cards. Therefore, in addition to enabling resource sharing between containers, it is also necessary to ensure that the resource usage hosted by the container can be limited under the amount of device resources required for the predicted task, i.e. physical device card isolation. To achieve this, the present application develops a virtual device card database 303 . It is a library installed in each SharePod container that limits and isolates GPU usage by intercepting all memory-related APIs and compute-related APIs in the CUDA library.

In some implementations, a token can be used to isolate GPU usage between containers in a time-sharing fashion. A container can only run program code on a GPU device if it has a valid token. The token is associated with a time quota, and when the time quota expires, the container must re-acquire the token to continue program execution. Thus, the container will in turn use the GPU device by passing tokens between them, and the container's usage can be controlled by the amount of time it is allowed to hold the token.

Specifically, it is implemented through the front-end module of each container and the back-end module of each cluster node. A front-end module is a dynamic link library inside a container. It intercepts all CUDA library APIs related to video memory and computation through the Linux LD_PRELOAD mechanism, so that the application loads the above-mentioned virtual device card database 303 before loading the standard GPU CUDA library. If the container does not have a valid token, the front-end module will block the intercepted CUDA call until it re-obtains a valid token from the back-end module. Backend modules are independent daemons running on resource cluster nodes that manage tokens between containers. Since each physical device card is associated with its own token, only one backend module is required on the nodes of the resource cluster to independently manage each device's token.

Based on the same inventive concept, an embodiment of the present application further provides an apparatus 400 for expanding and shrinking the capacity of an online image recognition service, as shown in FIG. 4 , including:

The flow monitoring module 401 is configured to perform monitoring on the number of images to be processed in the online image recognition service, and determine the amount of equipment resources required to perform the prediction task according to the detected number of the images to be processed;

The indicator determination module 402 is configured to perform determining a preset monitoring indicator corresponding to the quantity of the images to be processed in the container management system based on a preset conversion relationship; wherein the container management system is used for identifying the online image service The held containers are managed; each container is encapsulated with graphics processor resources, the container is the allocation unit of resources held by the online image recognition service, and the logical encapsulation of the container is used to intercept the unified computing device architecture library. Application programming interface invocation;

The capacity expansion and contraction module 403 is configured to control the container management system to perform expansion and contraction operations on the containers held by the online image recognition service according to the preset monitoring indicators, so that the expanded and contracted containers correspond to The amount of held resources meets the amount of equipment resources; the prediction task is performed according to the amount of held resources;

Wherein, when the capacity expansion/reduction module 403 performs the capacity expansion operation, it divides resources with the same amount of target resources from the physical device cards of the resource cluster, and performs container encapsulation on the resources to construct a virtual device card; the target resource amount is determined according to the amount of equipment resources and the amount of held resources;

When the capacity expansion/reduction module 403 performs a capacity reduction operation, a preset deletion operation is performed on the container currently held by the online image recognition service, so that the response time of the online image recognition service is within a preset time range.

In some possible embodiments, to perform the determining of the amount of equipment resources required to perform the prediction task based on the number of the monitored images to be processed, the flow monitoring module 401 is configured to:

Obtain the delay duration of the prediction task and the processing rate of the image processor;

The amount of device resources is determined according to the number of images to be processed, the delay duration and the processing rate.

In some possible embodiments, the determining a preset monitoring indicator corresponding to the number of images to be processed in the container management system based on a preset conversion relationship is performed, and the indicator determining module 402 is configured to:

If the number of the to-be-processed images is greater than the first monitoring threshold, determining that the number of the to-be-processed images corresponds to a preset monitoring index representing the execution of the expansion operation;

If the number of the to-be-processed images is less than the second monitoring threshold, it is determined that the number of the to-be-processed images corresponds to a preset monitoring index representing the execution of a scaling operation; wherein the second monitoring threshold is less than the first monitoring threshold threshold;

If the number of images to be processed is between the first monitoring threshold and the second monitoring threshold, it is determined that the number of images to be processed corresponds to a preset monitoring indicator indicating that no expansion or reduction operation needs to be performed.

In some possible embodiments, before executing the partitioning of virtual resources with the same amount of target resources from the physical device cards of the resource cluster, the capacity expansion and contraction module 403 is further configured to:

Detecting the resource cluster, and obtaining a physical device card that satisfies the device resource amount;

The step of dividing the graphics processor resources with the same amount of target resources from the physical device cards of the resource cluster, and performing container encapsulation on the graphics processor resources to construct a virtual device card, includes:

Selecting the physical device card with the least graphics processing resources as the target physical device card, and dividing the graphics processor resource with the same amount of the target resources from the target physical device card;

Perform container encapsulation on the graphics processor resource, and use the encapsulated container as the virtual device card.

In some possible embodiments, the capacity expansion/reduction module 403 is further configured to:

When detecting the resource cluster, if it is not detected that there is a physical device card that meets the device resource amount in the resource cluster, performing a preset number of resource rechecks on the resource cluster based on a preset time interval;

During the execution of the resource rechecking process, if an entity device card satisfying the amount of the device resources is detected, the entity device card with the least graphics processing resources is selected as the target entity device card;

If it is not detected that there is a physical device card in the resource cluster that meets the device resource amount, a prompt indicating that there is currently no available resource is output, and the current capacity expansion task ends.

In some possible embodiments, the performing the preset deletion operation on the container currently held by the online image recognition service is performed, so that the response time of the online image recognition service is within a preset time range, and the extension The shrinking module 403 is configured to:

Determine the resource amount of graphics processor resources of each container, and use the container with the smallest amount of resources as the container to be processed;

delete the container to be processed, and monitor the response time of the online image recognition service;

If the response time is not within the preset time range, the to-be-processed container is reselected from the remaining containers, and the preset deletion operation is stopped until the response time is within the preset time range.

In some possible embodiments, the number of images to be processed is determined according to interceptors or filters pre-deployed in the online image recognition service.

The electronic device 130 according to this embodiment of the present application is described below with reference to FIG. 5 . The electronic device 130 shown in FIG. 5 is only an example, and should not impose any limitations on the functions and scope of use of the embodiments of the present application.

As shown in FIG. 5, the electronic device 130 takes the form of a general electronic device. Components of the electronic device 130 may include, but are not limited to: the above-mentioned at least one processor 131 , the above-mentioned at least one memory 132 , and a bus 133 connecting different system components (including the memory 132 and the processor 131 ).

Bus 133 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor, or a local bus using any of a variety of bus structures.

Memory 132 may include readable media in the form of volatile memory, such as random access memory (RAM) 1321 and/or cache memory 1322 , and may further include read only memory (ROM) 1323 .

The memory 132 may also include a program/utility 1325 having a set (at least one) of program modules 1324 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, which An implementation of a network environment may be included in each or some combination of the examples.

Electronic device 130 may also communicate with one or more external devices 134 (eg, keyboards, pointing devices, etc.), may also communicate with one or more devices that enable a user to interact with electronic device 130, and/or communicate with the electronic device 130 communicates with any device (eg, router, modem, etc.) capable of communicating with one or more other electronic devices. Such communication may take place through input/output (I/O) interface 135 . Also, the electronic device 130 may communicate with one or more networks (eg, a local area network (LAN), a wide area network (WAN), and/or a public network such as the Internet) through a network adapter 136 . As shown, network adapter 136 communicates with other modules for electronic device 130 via bus 133 . It should be understood that, although not shown, other hardware and/or software modules may be used in conjunction with electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives and data backup storage systems.

In an exemplary embodiment, there is also provided a computer-readable storage medium including instructions, such as a memory 132 including instructions, which are executable by the processor 131 of the apparatus 400 to perform the above method. Alternatively, the computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.

In an exemplary embodiment, a computer program product is also provided, including computer programs/instructions, and when the computer programs/instructions are executed by the processor 131, an online image recognition service scaling method as provided in the present application is implemented. .

In an exemplary embodiment, various aspects of an online image recognition service scaling method provided by the present application can also be implemented in the form of a program product, which includes program code, and when the program product runs on a computer device , the program code is used to cause the computer device to execute the steps in the method for expanding or shrinking the capacity of an online image recognition service according to various exemplary embodiments of the present application described above in this specification.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples (non-exhaustive list) of readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

The program product for expansion and contraction of an online image recognition service according to the embodiment of the present application may adopt a portable compact disk read only memory (CD-ROM) and include program codes, and may be executed on an electronic device. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

A readable signal medium may include a propagated data signal in baseband or as part of a carrier wave, carrying readable program code therein. Such propagated data signals may take a variety of forms including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable signal medium can also be any readable medium, other than a readable storage medium, that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any suitable medium including, but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages—such as Java, C++, etc., as well as conventional procedural programming Language - such as "as" language or similar programming language. The program code may execute entirely on the user's electronic device, partly on the user's device, as a stand-alone software package, partly on the user's electronic device and partly on a remote electronic device, or entirely on the remote electronic device or service Execute on the end. In cases involving remote electronic devices, the remote electronic devices may be connected to the user electronic device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to external electronic devices (eg, using Internet services provider to connect via the Internet).

It should be noted that although several units or sub-units of the apparatus are mentioned in the above detailed description, this division is merely exemplary and not mandatory. Indeed, according to embodiments of the present application, the features and functions of two or more units described above may be embodied in one unit. Conversely, the features and functions of one unit described above may be further subdivided to be embodied by multiple units.

Furthermore, although the operations of the methods of the present application are depicted in the figures in a particular order, this does not require or imply that the operations must be performed in the particular order, or that all illustrated operations must be performed to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined to be performed as one step, and/or one step may be decomposed into multiple steps to be performed.

As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable image scaling device to produce a machine such that the instructions executed by the processor of the computer or other programmable image scaling device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

The computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable image scaling device to function in a particular manner, such that the instructions stored in the computer readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable image scaling device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that an image executed on the computer or other programmable device The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

While the preferred embodiments of the present application have been described, additional changes and modifications to these embodiments may occur to those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be construed to include the preferred embodiment and all changes and modifications that fall within the scope of this application.

Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims (10)

1. a method for expanding and shrinking capacity of an online image recognition service, wherein the method comprises: Monitoring the number of images to be processed in the online image recognition service, and determining the amount of equipment resources required to perform the prediction task based on the monitored number of images to be processed; A preset monitoring index corresponding to the number of images to be processed in the container management system is determined based on a preset conversion relationship; wherein the container management system is used to manage the containers held by the online image recognition service; each The container is encapsulated with graphics processor resources, the container is the allocation unit of the resource amount held by the online image recognition service, and the logical encapsulation of the container is used for intercepting the invocation of the application programming interface of the unified computing device architecture library; The container management system is controlled to perform capacity expansion and contraction operations on the containers held by the online image recognition service according to the preset monitoring indicators, so that the amount of resources held by the expanded and contracted containers can meet the requirements of the device resources. amount; perform the forecasting task according to the amount of resources held; Wherein, when the expansion operation is performed, resources that are the same as the target resource amount are divided from the physical device cards of the resource cluster, and the resources are packaged in containers to construct a virtual device card; the target resource amount is based on the device resource amount. and the amount of resources held; When the shrinking operation is performed, a preset deletion operation is performed on the container currently held by the online image recognition service, so that the response time of the online image recognition service is within a preset time range. 2. The method according to claim 1, wherein the determining the amount of equipment resources required to perform the prediction task based on the monitored number of the images to be processed comprises: Obtain the delay duration of the prediction task and the processing rate of the image processor; The amount of device resources is determined according to the number of images to be processed, the delay duration and the processing rate. 3 . The method according to claim 1 , wherein the determining a preset monitoring index corresponding to the number of the images to be processed in the container management system based on a preset conversion relationship comprises: 3 . If the number of the to-be-processed images is greater than the first monitoring threshold, determining that the number of the to-be-processed images corresponds to a preset monitoring index representing the execution of the expansion operation; If the number of the to-be-processed images is less than the second monitoring threshold, it is determined that the number of the to-be-processed images corresponds to a preset monitoring index representing the execution of a scaling operation; wherein the second monitoring threshold is less than the first monitoring threshold threshold; If the number of images to be processed is between the first monitoring threshold and the second monitoring threshold, it is determined that the number of images to be processed corresponds to a preset monitoring indicator indicating that no expansion or reduction operation needs to be performed. 4 . The method according to claim 1 , wherein, before dividing the virtual resources with the same amount of target resources from the physical device cards of the resource cluster, the method further comprises: 5 . Detecting the resource cluster, and obtaining a physical device card that satisfies the device resource amount; The step of dividing the graphics processor resources with the same amount of target resources from the physical device cards of the resource cluster, and performing container encapsulation on the graphics processor resources to construct a virtual device card, includes: Selecting the physical device card with the least graphics processing resources as the target physical device card, and dividing the graphics processor resource with the same amount of the target resources from the target physical device card; Perform container encapsulation on the graphics processor resource, and use the encapsulated container as the virtual device card. 5. The method according to claim 4, wherein the method further comprises: When detecting the resource cluster, if it is not detected that there is a physical device card that meets the device resource amount in the resource cluster, performing a preset number of resource rechecks on the resource cluster based on a preset time interval; During the execution of the resource rechecking process, if an entity device card satisfying the amount of the device resources is detected, the entity device card with the least graphics processing resources is selected as the target entity device card; If it is not detected that there is a physical device card in the resource cluster that meets the device resource amount, a prompt indicating that there is currently no available resource is output, and the current capacity expansion task ends. 6 . The method according to claim 1 , wherein the preset deletion operation is performed on the container currently held by the online image recognition service, so that the response time of the online image recognition service is within a preset time. 7 . range, including: Determine the resource amount of graphics processor resources of each container, and use the container with the smallest amount of resources as the container to be processed; delete the container to be processed, and monitor the response time of the online image recognition service; If the response time is not within the preset time range, the to-be-processed container is reselected from the remaining containers, and the preset deletion operation is stopped until the response time is within the preset time range. 7. The method according to any one of claims 1-6, wherein the number of images to be processed is determined according to an interceptor or filter pre-deployed in the online image recognition service. 8. A capacity expansion/reduction device for an online image recognition service, wherein the device comprises: a flow monitoring module, configured to monitor the number of images to be processed in the online image recognition service, and determine the amount of equipment resources required to perform the prediction task according to the monitored number of the images to be processed; The indicator determination module is configured to determine the preset monitoring indicators corresponding to the quantity of the images to be processed in the container management system based on the preset conversion relationship; wherein, the container management system is used for maintaining the online image recognition service Some containers are managed; each container is encapsulated with graphics processor resources, the container is the allocation unit of resources held by the online image recognition service, and the logical encapsulation of the container is used to intercept the application of the unified computing device architecture library Program programming interface invocation; The capacity expansion and contraction module is configured to control the container management system to perform expansion and contraction operations on the containers held by the online image recognition service according to the preset monitoring indicators, so that the containers corresponding to the expanded and contracted containers can be expanded or contracted. The amount of resources held satisfies the amount of equipment resources; the prediction task is performed according to the amount of resources held; Wherein, when the expansion and contraction module performs the expansion operation, it divides the resources with the same target resource amount from the physical device cards of the resource cluster, and performs container encapsulation on the resources to construct a virtual device card; the target resource amount is Determined according to the amount of equipment resources and the amount of held resources; When the capacity expansion/reduction module performs a capacity reduction operation, a preset deletion operation is performed on the container currently held by the online image recognition service, so that the response time of the online image recognition service is within a preset time range. 9. An electronic device, characterized in that, comprising: processor; memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the method of any of claims 1-7. 10 . A storage medium, characterized in that, when the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 7 .

Images