TWI865333B - Computing platform system and graphics processing unit resource management method thereof - Google Patents

Abstract

A computing platform system and a graphics processing unit (GPU) resource management method thereof are provided. The method includes the following steps. At least a first graphics processing unit among a plurality of graphics processing units is assigned to a user. An idle threshold corresponding to the at least one first graphics processing unit is determined. Idle time of the at least one first graphics processing unit is checked. When the idle time of the at least one first graphics processing unit is greater than the idle threshold, at least one first graphics processing unit is released.

Description

Computing platform system and its graphics processing unit resource management method

The present invention relates to a computing platform system, and more particularly to a computing platform system and a graphics processing unit resource management method thereof.

Graphics Processing Unit (GPU) plays an important role in computer science and engineering. It not only provides powerful support for graphics processing, but also provides high-performance solutions for a wide range of general computing tasks. In particular, GPUs were originally designed to handle graphics rendering. However, with the development of technology, GPUs are now gradually evolving into general-purpose parallel processing units that can be used to perform a variety of computationally intensive tasks, not just graphics processing. For example, GPUs are currently widely used in machine learning, deep learning, scientific computing, and other fields that require a large amount of parallel computing.

Currently, computing platforms with multiple graphics processing units provide users with huge computing resources. This hardware configuration performs well in processing large-scale and complex computing tasks. However, despite the powerful computing capabilities of the graphics processing units of these computing platforms, there is a common problem, that is, these graphics processing units allocated to users are not always in a high-load operation state, and sometimes they are even idle. This phenomenon highlights the problem of waste of graphics processing unit resources, which has an adverse impact on energy efficiency and computing performance. When the graphics processing unit is idle, it is actually a waste of energy and hardware resources, which not only increases the operating cost of the computing platform, but also creates an unnecessary burden on the environment. At the same time, this waste also directly affects the scalability of computing projects, because underutilized graphics processing units mean that computing resources cannot be maximized.

In view of this, the present invention proposes a computing platform system and a graphics processing unit resource management method thereof, which can solve the above technical problems.

The embodiment of the present invention provides a method for managing resources of a graphics processing unit, which is applicable to a computing platform system including a plurality of graphics processing units. The method comprises the following steps. Allocate at least one first graphics processing unit among the plurality of graphics processing units to a user. Determine an idle threshold corresponding to the at least one first graphics processing unit. Check the idle time of the at least one first graphics processing unit. When the idle time of the at least one first graphics processing unit is greater than the idle threshold, release the at least one first graphics processing unit.

An embodiment of the present invention provides a computing platform system, which includes multiple graphics processing units, a storage device and a processor. The processor is coupled to the multiple graphics processing units and the storage device, and is configured to perform the following operations. At least one first graphics processing unit among the multiple graphics processing units is configured to a user. An idle threshold corresponding to the at least one first graphics processing unit is determined. The idle time of the at least one first graphics processing unit is checked. When the idle time of the at least one first graphics processing unit is greater than the idle threshold, the at least one first graphics processing unit is released.

Based on the above, in the embodiment of the present invention, the idle threshold value used to determine whether to release the graphics processing unit resources is flexibly determined according to the graphics processing unit configured for the user. When the idle time of the graphics processing unit configured for the user is greater than the corresponding idle threshold value, the graphics processing unit resources can be released. Based on this, not only can the utilization rate of the graphics processing unit resources be effectively improved, but also the waste of the graphics processing unit resources can be reduced.

Some embodiments of the present invention will be described in detail below with reference to the accompanying drawings. When the same element symbols appear in different drawings, they will be regarded as the same or similar elements. These embodiments are only part of the present invention and do not disclose all possible implementations of the present invention. More precisely, these embodiments are only examples of methods and systems within the scope of the patent application of the present invention.

Fig. 1 is a block diagram of a computing platform system according to an embodiment of the present invention. Referring to Fig. 1, the computing platform system 10 includes a plurality of graphics processing units 110_1-110_N, a storage device 120, and a processor 130. In some embodiments, the computing platform system 10 can be implemented by one or more servers.

The plurality of graphics processing units 110_1 to 110_N are a kind of hardware resource with high efficiency and strong parallel processing capability, which is suitable for processing various computationally intensive tasks. The present invention does not limit the number of graphics processing units 110_1 to 110_N. In addition, the hardware specifications of these graphics processing units 110_1 to 110_N can be the same or different, and the present invention does not set any limitation on this. It should be noted that the computing performance and installation cost of graphics processing units with different hardware specifications are also different.

The storage device 120 is used to store data and data such as software modules (such as operating systems, applications, and drivers) accessed by the processor 130. It can be, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk, or a combination thereof.

The processor 130 is coupled to the plurality of graphics processing units 110_1-110_N and the storage device 120, and may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor, a plurality of microprocessors, one or more microprocessors combined with a digital signal processor core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), any other type of integrated circuit, a state machine, an advanced RISC machine (ARM) based processor, and the like. The processor 130 can access and execute the instructions or program codes recorded in the storage device 120 to implement the graphics processing unit resource management method in the embodiment of the present invention.

FIG2 is a flow chart of a method for managing graphics processing unit resources according to an embodiment of the present invention, and the method flow of FIG2 can be implemented by various components of the computing platform system 10 of FIG1. Please refer to FIG1 and FIG2 simultaneously, and the steps of the method for managing graphics processing unit resources of this embodiment will be described below in conjunction with various components of the computing platform system 10 of FIG1.

In step S210, the processor 130 configures at least one first graphics processing unit among the multiple graphics processing units 110_1~110_N for a user. In detail, the user can create a virtual machine (Virtual machine) or a container (Container) on the computing platform system 10. The above-mentioned container is, for example, a Docker container. The virtual environment provided by the virtual machine or container can provide an independent and isolated running space, so that the application can run in a relatively independent environment. When the user creates a virtual machine or a container on the computing platform system 10, the processor 130 can configure at least one first graphics processing unit among the multiple graphics processing units 110_1~110_N for the user to use. The present invention does not limit the number of the first graphics processing unit, which may be determined according to actual applications. In other words, herein, the first graphics processing unit may be any one or any number of the graphics processing units 110_1 to 110_N.

In some embodiments, the processor 130 may configure the first graphics processing unit to a user according to the type and quantity of the GPU specified by the user. Alternatively, in some embodiments, the processor 130 may determine the type and quantity of the first graphics processing unit to be configured to the user according to the content of the computing task that the user wants to complete. For example, the processor 130 may configure the graphics processing unit 110_1 among the graphics processing units 110_1 to a user. Alternatively, the processor 130 may configure the graphics processing units 110_1 and 110_2 among the graphics processing units 110_1 to 110_N to a user.

In step S220, the processor 130 determines an idle threshold corresponding to at least one first graphics processing unit. That is, in the embodiment of the present invention, the processor 130 can flexibly determine the corresponding idle threshold according to the first graphics processing unit configured for the user. In some embodiments, different graphics processing units 110_1~110_N can correspond to different idle thresholds. For example, the graphics processing unit 110_1 can correspond to the first idle threshold, and the graphics processing unit 110_2 can correspond to the second idle threshold. The first idle threshold is different from the second idle threshold.

In some embodiments, the processor 130 may determine the idle threshold value according to the type of at least one first graphics processing unit. The type of at least one first graphics processing unit may include a hardware model or a computing performance level. In other words, the processor 130 may determine the idle threshold value corresponding to each graphics processing unit 110_1~110_N according to the hardware model or computing performance level of the graphics processing units 110_1~110_N. When at least one first graphics processing unit belongs to the first type, the idle threshold value is a first time length. When at least one first graphics processing unit belongs to the second type, the idle threshold value is a second time length.

FIG3 is a flow chart of determining an idle threshold value according to an embodiment of the present invention. Referring to FIG3, in some embodiments, step S220 of FIG2 can be implemented as steps S301 to S303.

In step S301, the processor 130 determines whether the type of at least one first graphics processing unit is a first type or a second type. For example, the processor 130 may determine whether the first graphics processing unit configured for the user is a first hardware model or a second hardware model. Alternatively, the processor 130 may determine whether the first graphics processing unit configured for the user is a first computing performance level or a second computing performance level.

When the type of at least one first graphics processing unit is the first type, in step S302, the processor 130 sets the idle threshold value to the first time length. When the type of at least one first graphics processing unit is the second type, in step S303, the processor 130 sets the idle threshold value to the second time length. In some embodiments, the processor 130 may perform a table lookup based on the type of at least one first graphics processing unit to determine the time length of the idle threshold value. For example, when the type of the first graphics processing unit is the first type, the processor 130 may set the idle threshold value to 48 hours. When the type of the first graphics processing unit is the second type, the processor 130 may set the idle threshold value to 24 hours.

It is worth mentioning that in some embodiments, for a GPU type with stronger computing power, the processor 130 may set a shorter idle threshold. That is, when the first graphics processing unit has stronger computing power, the processor 130 may set a shorter idle threshold. On the contrary, when the first graphics processing unit has weaker computing power, the processor 130 may set a longer idle threshold.

In addition, in some embodiments, the processor 130 may determine the idle threshold value according to the type and quantity of at least one first graphics processing unit. The idle threshold value decreases as the quantity of at least one first graphics processing unit increases. In other words, when the quantity of the first graphics processing unit increases, the idle threshold value determined by the processor 130 is lower.

FIG4 is a flow chart of determining an idle threshold according to an embodiment of the present invention. Referring to FIG4, in some embodiments, step S220 of FIG2 can be implemented as steps S401 to S404.

In step S401, the processor 130 determines the number of at least one first graphics processing unit. Then, the processor 130 may determine the idle threshold value according to the type and number of at least one first graphics processing unit. Further, in step S402, the processor 130 determines whether the type of at least one first graphics processing unit is the first type or the second type.

When the type of at least one first graphics processing unit is the first type, in step S403, the processor 130 determines an idle threshold value according to the first time length and the number of at least one first graphics processing unit. When the type of at least one first graphics processing unit is the second type, in step S404, the processor 130 determines an idle threshold value according to the second time length and the number of at least one first graphics processing unit. The first time length corresponding to the first type is different from the second time length corresponding to the second type.

For example, assume that the processor 130 configures two graphics processing units 110_1 and 110_2 of a first hardware model for the user. The processor 130 may determine the initial time length of the idle threshold value to be 48 hours based on the first hardware model. Thereafter, since the number of graphics processing units 110_1 and 110_2 is 2, the processor 130 may divide 48 hours by 2 to obtain the idle threshold value with a final time length of 24 hours. Alternatively, assume that the processor 130 configures three graphics processing units 110_3, 110_4, and 110_5 of a second hardware model for the user. The processor 130 may determine the initial time length of the idle threshold value to be 24 hours based on the second hardware model. Afterwards, since the number of the graphics processing units 110_3, 110_4, and 110_5 is 3, the processor 130 may divide 48 hours by 3 to obtain the idle threshold value of a final time length of 8 hours.

Alternatively, assume that the processor 130 configures a graphics processing unit 110_1 belonging to the first hardware model for the user. The processor 130 may determine the initial time length of the idle threshold value to be 48 hours based on the first hardware model. Thereafter, since the number of graphics processing units 110_1 is 1, the processor 130 may divide 48 hours by 1 to obtain the idle threshold value with a final time length of 48 hours. Alternatively, assume that the processor 130 configures six graphics processing units 110_3 to 110_8 belonging to the second hardware model for the user. The processor 130 may determine the initial time length of the idle threshold value to be 48 hours based on the second hardware model. Afterwards, since the number of the graphics processing units 110_3 to 110_8 is 6, the processor 130 may divide 48 hours by 6 to obtain the idle threshold value with a final time length of 8 hours.

2 , in step S230 , the processor 130 checks the idle time of at least one first graphics processing unit. Then, in step S240 , the processor 130 determines whether the idle time of at least one first graphics processing unit is greater than an idle threshold.

In various embodiments, the processor 130 may periodically or irregularly check the idle time of at least one first graphics processing unit. For example, the processor 130 may check the idle time of the first graphics processing unit allocated to the user every hour (but not limited thereto). Alternatively, the processor 130 may check the idle time of the first graphics processing unit allocated to the user in response to determining that the GPU resources are insufficient to be allocated to a new user.

In some embodiments, the processor 130 may detect a graphics processing unit usage (GPU Usage) of at least one first graphics processing unit. The graphics processing unit usage may be a percentage value. When the graphics processing unit usage is less than a usage threshold value, the processor 130 may continue to accumulate the idle time of at least one first graphics processing unit. On the other hand, when the graphics processing unit usage is greater than or equal to the usage threshold value, the processor 130 may reduce the idle time of at least one first graphics processing unit to a preset value. For example, when the graphics processing unit usage is greater than or equal to the usage threshold value, the processor 130 may return the idle time of at least one first graphics processing unit to zero or subtract a specific value.

For example, FIG. 5A and FIG. 5B are schematic diagrams of checking idle time according to an embodiment of the present invention. Referring to FIG. 5A , the processor 130 may configure one or more first graphics processing units for a user at time point T1. Then, the processor 130 may check the idle time of each first graphics processing unit at intervals of ΔT. In the example of FIG. 5A , the processor 130 may detect that the graphics processing unit utilization rate of a certain first graphics processing unit is X1% at the checking time point T2. Since the graphics processing unit utilization rate X1% is less than the utilization rate threshold value THu, the processor 130 may set the idle time of this first graphics processing unit to ΔT. Furthermore, the processor 130 will determine whether the idle time ΔT of the first graphics processing unit is greater than the idle threshold value. Assuming that the idle time ΔT is not greater than the idle threshold value. The processor 130 can then detect that the graphics processing unit utilization rate of a certain first graphics processing unit is X2% at the check time point T3. Since the graphics processing unit utilization rate X2% is less than the utilization rate threshold value THu, the processor 130 can continue to accumulate the idle time of the first graphics processing unit as 2*ΔT.

Please refer to Figure 5B again. The processor 130 can configure one or more first graphics processing units for the user at time point T1. Then, the processor 130 can check the idle time of each first graphics processing unit at intervals of time ΔT. In the example of Figure 5B, the processor 130 can detect that the graphics processing unit utilization rate of a certain first graphics processing unit is X1% at the checking time point T2. Since the graphics processing unit utilization rate X1% is less than the utilization rate threshold value THu, the processor 130 can set the idle time of this first graphics processing unit to ΔT. In addition, the processor 130 will determine whether the idle time ΔT of the first graphics processing unit is greater than the idle threshold value. Assume that the idle time ΔT is not greater than the idle threshold value. The processor 130 may then detect that the GPU usage rate of a first GPU is X3% at the checking time point T3. Since the GPU usage rate X3% is greater than the usage rate threshold THu, the processor 130 may reset the idle time of the first GPU to zero.

Returning to FIG. 2 , when the idle time of at least one first graphics processing unit is greater than the idle threshold value (step S240 determines yes), in step S250, the processor 130 releases at least one first graphics processing unit. For example, assume that the processor 130 configures two graphics processing units 110_1 and 110_2 of a first hardware model for a user. The processor 130 can obtain the idle threshold value with a final time length of 24 hours according to the above description. The processor 130 can determine whether the idle time of the graphics processing units 110_1 and 110_2 is greater than 24 hours. When the idle time of the graphics processing units 110_1 and 110_2 is greater than 24 hours, the processor 130 may release the graphics processing units 110_1 and 110_2. In other words, when the idle time of at least one first graphics processing unit is greater than the idle threshold, the processor 130 may shut down the user's virtual machine or container to release the first graphics processing unit. Based on this, the embodiment of the present invention can recycle idle GPU resources to achieve higher performance and more economical resource usage.

FIG6 is a flow chart of a method for managing graphics processing unit resources according to an embodiment of the present invention, and the method flow of FIG6 can be implemented by various components of the computing platform system 10 of FIG1. Please refer to FIG1 and FIG6 at the same time, and the steps of the method for managing graphics processing unit resources of this embodiment will be described below in conjunction with various components of the computing platform system 10 of FIG1.

In step S610, the processor 130 allocates at least one first graphics processing unit among a plurality of graphics processing units to a user. In step S620, the processor 130 determines an idle threshold corresponding to the at least one first graphics processing unit. In step S630, the processor 130 checks the idle time of the at least one first graphics processing unit. In step S640, the processor 130 determines whether the idle time of each first graphics processing unit is greater than the idle threshold. The relevant details of the above steps can be referred to the description of the aforementioned embodiment, and are not repeated here.

In this embodiment, when the idle time of at least one of the first graphics processing units is not greater than the idle threshold, the processor 130 decides not to release the at least one first graphics processing unit and returns to step S630. In other words, when the number of first graphics processing units is greater than 1, as long as the idle time of at least one first graphics processing unit is not greater than the idle threshold, the processor 130 decides not to release the at least one first graphics processing unit.

On the other hand, when the idle time of each of the at least one first graphics processing unit is greater than the idle threshold value (step S640 determines that it is yes), in step S650, the processor 130 stores the working environment state of the user's virtual machine or container. Then, in step S660, the processor 130 releases the at least one first graphics processing unit.

Specifically, the processor 130 may record the working environment state of the user's virtual machine or container before shutting down the user's virtual machine or container. In some embodiments, the processor 130 may generate a virtual machine snapshot of the virtual machine or a container snapshot or container image of the container to store the working environment state of the user's virtual machine or container when deciding to release the first graphics processing unit. In this way, the user can quickly rebuild a similar working environment state on the computing platform system 10 later.

In summary, in the embodiment of the present invention, the idle threshold value used to determine whether to release the graphics processing unit resources is flexibly determined according to the graphics processing unit configured for the user. When the idle time of the graphics processing unit configured for the user is greater than the corresponding idle threshold value, the graphics processing unit resources can be released. Based on this, not only can the utilization rate of the graphics processing unit resources be effectively improved, but also the waste of the graphics processing unit resources can be reduced. In addition, before releasing the graphics processing unit, the embodiment of the present invention can retain the user's work environment status to save the user's time in rebuilding the work environment and improve the user experience.

Although the present invention has been disclosed as above by the embodiments, they are not intended to limit the present invention. Any person with ordinary knowledge in the relevant technical field can make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be defined by the scope of the attached patent application.

10: Computing platform system 110_1~110_N: Graphics processing unit 120: Storage device 130: Processor S210~S250, S301~S303, S401~S404, S710~S660: Steps

FIG. 1 is a block diagram of a computing platform system according to an embodiment of the present invention. FIG. 2 is a flow chart of a graphics processing unit resource management method according to an embodiment of the present invention. FIG. 3 is a flow chart of determining an idle threshold value according to an embodiment of the present invention. FIG. 4 is a flow chart of determining an idle threshold value according to an embodiment of the present invention. FIG. 5A and FIG. 5B are schematic diagrams of checking idle time according to an embodiment of the present invention. FIG. 6 is a flow chart of a graphics processing unit resource management method according to an embodiment of the present invention.

S210~S250: Steps

Claims (10)

A method for managing graphics processing unit resources is applicable to a computing platform system including multiple graphics processing units, the method comprising: Allocating at least one first graphics processing unit among the multiple graphics processing units to a user; Determining an idle threshold corresponding to the at least one first graphics processing unit; Checking the idle time of the at least one first graphics processing unit; and When the idle time of the at least one first graphics processing unit is greater than the idle threshold, releasing the at least one first graphics processing unit. The method for managing graphics processing unit resources as described in claim 1, wherein the step of determining the idle threshold corresponding to the at least one first graphics processing unit comprises: Determining the idle threshold according to the type of the at least one first graphics processing unit, wherein when the at least one first graphics processing unit belongs to the first type, the idle threshold is a first time length; when the at least one first graphics processing unit belongs to the second type, the idle threshold is a second time length. A graphics processing unit resource management method as described in claim 2, wherein the type of the at least one first graphics processing unit includes a hardware model or a computing performance level. The method for managing graphics processing unit resources as described in claim 2, wherein the step of determining the idle threshold value according to the type of the at least one first graphics processing unit comprises: Determining the number of the at least one first graphics processing unit; and Determining the idle threshold value according to the type and number of the at least one first graphics processing unit. A graphics processing unit resource management method as described in claim 4, wherein the idle threshold decreases as the number of the at least one first graphics processing unit increases. The method for managing graphics processing unit resources as described in claim 1, wherein before the step of releasing the at least one first graphics processing unit when the idle time of the at least one first graphics processing unit is greater than the idle threshold, the method further comprises: When the idle time of the at least one first graphics processing unit is greater than the idle threshold, storing the working environment state of the user's virtual machine or container. The graphics processing unit resource management method as described in claim 1, wherein the step of checking the idle time of the at least one first graphics processing unit comprises: Detecting the graphics processing unit utilization rate of the at least one first graphics processing unit; and When the graphics processing unit utilization rate is less than the utilization rate threshold value, continuously accumulating the idle time of the at least one first graphics processing unit. The graphics processing unit resource management method as described in claim 7, wherein the step of checking the idle time of the at least one first graphics processing unit further includes: When the usage rate of the graphics processing unit is greater than or equal to the usage rate threshold value, reducing the idle time of the at least one first graphics processing unit to a preset value. The method for managing graphics processing unit resources as described in claim 1, wherein the number of the at least one first graphics processing unit is greater than 1, and when the idle time of the at least one first graphics processing unit is greater than the idle threshold value, the step of releasing the at least one first graphics processing unit comprises: When the idle time of each of the at least one first graphics processing unit is greater than the idle threshold value, releasing the at least one first graphics processing unit; and When the idle time of at least one of the at least one first graphics processing unit is not greater than the idle threshold value, deciding not to release the at least one first graphics processing unit. A computing platform system includes: a plurality of graphics processing units, a storage device recording a plurality of instructions; and at least one processor, coupling the plurality of graphics processing units and the storage device, configured to: configure at least one first graphics processing unit among the plurality of graphics processing units to a user; determine an idle threshold corresponding to the at least one first graphics processing unit; check the idle time of the at least one first graphics processing unit; and release the at least one first graphics processing unit when the idle time of the at least one first graphics processing unit is greater than the idle threshold.

Images