JP2024545660A - In-line interruption of acceleration processing units - Google Patents

Abstract

A method and system for in-line suspend of an accelerated processing unit (APU) is disclosed. The technique includes receiving a packet containing an operational mode and a command to be executed by the APU, suspending execution of the command received in the previous packet if the operational mode is a suspend start mode, and the APU executing the command in the received packet. Execution of the suspended command is resumed if the operational mode in the next received packet is a suspend end mode.
[Selected figure] Figure 3

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
This application claims the benefit of U.S. Patent Application No. 17/564,049, filed December 28, 2021, the contents of which are incorporated herein by reference.

Processors that need to process large amounts of data within a limited time can utilize one or more accelerated processing units (APUs). When using an APU, traditionally the processor controls the operation of the APU, including sending commands to be executed by the APU and receiving command completion acknowledgments from the APU. Typically, the computing resources of the APU are shared by multiple applications running on one or more processors. When an application requires the execution of an intensive workload with high priority, the ability to allow the application to reserve the computing resources of the APU for its exclusive use is valuable. However, suspending the current workload of the APU in favor of another workload and then resuming it requires the involvement of the processor, which typically requires communication between the APU and the processor. Such back-and-forth communication between the processor and the APU reduces the predictability of the workload execution time by the APU.

A more detailed understanding can be had from the following description, given by way of example in conjunction with the accompanying drawings, in which:

FIG. 1 is a block diagram of an example device capable of implementing one or more features of the present disclosure. FIG. 1B is a block diagram of an example system illustrating an APU usable by the device of FIG. 1A, according to which one or more features of the present disclosure may be implemented. FIG. 1 is a functional block diagram of an example system illustrating in-line interruption of an APU capable of implementing one or more features of the present disclosure. 1 is a flowchart of an example method for in-line interruption of an APU that can implement one or more features of the present disclosure.

A system and method for inline suspension of an APU is disclosed. Techniques are disclosed for triggering the suspension and then resumption of a workload being processed by an APU by inlining the respective operating modes with commands sent in packets by the processor to the APU. The ability to suspend and resume an APU in this manner allows high priority and intensive workloads to exclusively utilize the computing resources of the APU without processor involvement, thus allowing predictable workload execution times.

Aspects disclosed herein describe a method for in-line suspension of an APU. The method includes receiving a packet including an operational mode and a command to be executed by the APU, suspending execution of the command received in the previous packet in response to the operational mode being a suspension initiation mode, and executing, by the APU, the command in the received packet. The method further includes resuming execution of the suspended command in response to the operational mode being a suspension termination mode.

Aspects disclosed herein also describe a system for in-line interruption of an APU. The system includes at least one processor and a memory that stores instructions. The instructions, when executed by the at least one processor, cause the system to receive a packet including an operating mode and a command to be executed by the APU, and, in response to the operating mode being an interrupt start mode, interrupt execution of the command received in the previous packet, and cause the APU to execute the command in the received packet. The instructions further cause the system to resume execution of the interrupted command in response to the operating mode being an interrupt end mode.

Additionally, aspects disclosed herein include a method for describing a non-transitory computer-readable storage medium including hardware description language instructions describing an APU adapted to perform an in-line suspend of the APU capable of receiving a packet including an operating mode and a command to be executed by the APU, suspending execution of the command received in the previous packet in response to the operating mode being a suspend start mode, and executing, by the APU, the command in the received packet, further including resuming execution of the suspended command in response to the operating mode being a suspend end mode.

1A is a block diagram of an example device 100A that can implement one or more features of the present disclosure. Device 100A can be, for example, a computer, a gaming device, a handheld device, a set-top box, a television, a mobile phone, or a tablet computer. Device 100A includes a processor 102, an APU 106, a memory 104, a storage 116, an input device 108, and an output device 110. Device 100A can also include an input driver 112 and an output driver 114. In one aspect, device 100A can include additional components not shown in FIG. 1A.

The processor 102 may include a central processing unit (CPU) or one or more cores of a CPU. The APU 106 may represent a highly parallel processing unit, a graphics processing unit (GPU), or a combination thereof. The processor 102 and the APU 106 may be located on the same die or on separate dies. The memory 104 may be located on the same die as the processor 102 or may be located separately from the processor 102. The memory 104 may include volatile or non-volatile memory (e.g., random access memory (RAM), dynamic RAM (DRAM), cache, or a combination thereof).

Storage 116 may include fixed or removable storage (e.g., hard disk drive, solid state drive, optical disk, flash drive). Input device 108 may represent one or more input devices such as a keyboard, keypad, touch screen, touch pad, detector, microphone, accelerometer, gyroscope, biometric scanner, or network connection (e.g., wireless local area network card for receiving wireless IEEE 802 signals). Output device 110 may represent one or more output devices such as a display, speaker, printer, haptic feedback device, one or more lights, antennas, or network connection (e.g., wireless local area network card for transmitting wireless IEEE 802 signals).

The input driver 112 communicates with the processor 102 and the input device 108 and facilitates receiving input from the input device 108 to the processor 102. The output driver 114 communicates with the processor 102 and the output device 110 and facilitates sending output from the processor 102 to the output device 110. In one aspect, the input driver 112 and the output driver 114 are optional components and the device 100A can operate in the same manner in the absence of the input driver 112 and the output driver 114.

The APU 106 may be configured to accept computational and graphics rendering commands from the processor 102, process those computational and graphics rendering commands, and/or provide output to a display (output device 110). As described in more detail below, the APU 106 may include one or more parallel processing units configured to perform computations, for example, according to a single instruction multiple data (SIMD) paradigm. Thus, although various functions are described herein as being performed by or in conjunction with the APU 106, in various alternative examples, the functions described as being performed by the APU 106 may additionally or alternatively be performed by other computing devices having similar capabilities that are not driven by a host processor (e.g., the processor 102) and that may be configured, for example, to provide graphical output to a display. Regardless of whether the processing system is capable of performing processing tasks according to the SIMD paradigm, the processing system may be configured to perform the functions described herein.

FIG. 1B is a block diagram of an exemplary system 100B illustrating an acceleration system usable by the device of FIG. 1A that can implement one or more features of the present disclosure. FIG. 1B illustrates the execution of processing tasks on the APU 106 in further detail. The processor 102 can maintain in the memory 104 one or more modules for execution by the processor 102. The modules include an operating system 120, a driver 122, and an application 126. These modules can control various aspects of the operation of the processor 102 and the APU 106. For example, the operating system 120 can provide system calls, i.e., application programming interfaces (APIs), that can be employed by the application 126 to interface directly with the hardware. The driver 122 can control the operation of the APU 106, for example, by providing the APIs to the application 126 running on the processor 102 to access various functions of the APU 106.

APU 106 can execute commands related to graphics and non-graphics operations, including either parallel or sequential processing, and either sequential or non-sequential processing. APU 106 can be used to perform graphics pipeline operations, such as operations that process pixels and/or geometry calculations (e.g., rendering an image to a display (output device 110)) based on commands received from processor 102. APU 106 can also execute processing operations not related to graphics operations, such as operations related to processing multi-dimensional data, physics simulations, computational fluid dynamics, or other computational tasks based on commands received from processor 102. In alternative embodiments, APU 106 may perform signal processing operations (e.g., APU 106 may be embodied in a digital signal processor or DSP), may perform accelerated operations through the use of a bitstream-configured field programmable gate array (FPGA), may perform neural processing operations through the use of a neural processing unit (or NPU), or may perform other operations that may be more efficiently performed through the use of an accelerated processing unit (APU) rather than the processor 102.

APU 106 may include workgroup processors (WGPs) 132.1-132.M, where each WGP, e.g., 132.1, may have one or more SIMD units, e.g., 138.1.1-138.1.N, that may execute operations in a parallel fashion according to the SIMD paradigm. The SIMD paradigm is one in which multiple processing elements share a single program control flow unit and program counter, thereby allowing the same program to be executed on different data. In one example, each SIMD unit, e.g., 138.1.1, may execute 64 lanes (i.e., threads), each lane executing the same instruction simultaneously with other lanes in the SIMD unit, but executing the instruction on different data. Lanes may be switched off with prediction, such as when not all lanes need to execute a given instruction. Prediction may also be used to execute programs with branching control flow. Specifically, for programs with conditional branches (or other instructions where control flow is based on calculations performed by individual lanes), prediction of lanes corresponding to currently not executed control flow paths and serial execution of different control flow paths allows arbitrary control flow. In one aspect, each of WGPs 132.1-132.M may have a local cache. In another aspect, multiple WGPs may share a cache.

The basic unit of execution within a WGP, e.g., 132.1, is the work item. Typically, each work item represents a single instantiation of a program that can be executed in parallel on a particular lane. Work items can be executed simultaneously as a "wavefront" (or "wave") on a single SIMD, e.g., 138.1.1. One or more waves may be executed in a workgroup, each wave including a collection of work items designated to execute the same program. A workgroup is executed by executing each of the waves that make up the workgroup. Also, waves can be executed sequentially on a single SIMD unit, or partially or fully in parallel on different SIMD units 138.1.1-138.1.N. Thus, a wave can be thought of as a collection of work items that can be executed simultaneously on a single SIMD unit, e.g., 138.1.1. Thus, if commands received from processor 102 indicate that a particular program should be parallelized to the extent that the program cannot be run simultaneously on a single SIMD unit, the program may be split into waves that may be parallelized on two or more SIMD units (e.g., 138.1.1-138.1.N), serialized on the same SIMD unit (e.g., 138.1.1), or both parallelized and serialized as desired. Scheduler 136 may be configured to perform operations associated with initiating various waves on different WGPs 132.1-132.M and their respective SIMD units.

The parallelism provided by WGPs 132.1-132.M is well suited for graphics-related operations such as, for example, operations on pixel values (e.g., filter operations), operations on geometric data (e.g., vertex transformations), and other graphics-related operations. For example, an application 126 executing on processor 102 may involve computations performed by APU 106. Application 126 may issue processing commands to APU 106 using an API provided by driver 122. The processing commands are then provided to scheduler 136. Scheduler 136 converts the processing commands into computational tasks that are assigned to WGPs 132.1-132.M for parallel execution. For example, scheduler 136 may receive processing commands that include instructions to be performed on data (e.g., 1024 pixels of an image). In response, scheduler 136 may divide data into groups (e.g., each group containing data necessary to process 64 pixels) and initiate waves in one or more WGPs, with each wave associated with a group of data and an instruction to perform on the data. For example, scheduler 136 may initiate 16 waves (e.g., each responsible for processing 64 pixels) to be executed in SIMD 138 of one or more WGPs 132.

2 is a functional block diagram of an exemplary system 200 illustrating an in-line interruption of an APU that can implement one or more features of the present disclosure. The system 200 includes a processor 210 (e.g., processor 102 of FIG. 1A), an APU 215 (e.g., APU 106 of FIG. 1B), and a memory 220 (e.g., memory 104 of FIG. 1A). The APU 215 includes a command processor 250, a shader scheduler 230, and a shader 240 having WGPs 240.1-240.N (e.g., WGP 132 of FIG. 1B). The memory 220 is accessible by the processor 210 and the command processor 250 via memory interfaces 222 and 224, respectively. The processor 210 is configured to execute software modules, such as a user application 212 and a driver 214, through which the application 212 can interface with the command processor 250. Thus, an application 212, such as a computer game or simulator, can use an application programming interface (API) provided by the driver 214 to send commands to the command processor 250 that specify computational tasks to be performed by the shaders 240. Such commands are delivered in packets according to a packet format, as described further below.

The command processor 250 is configured to provide an interface between software modules running on the processor 210 and the execution (or processing) engines of the APU 215, such as the shader 240. The command processor 250 may include functional components such as the fetcher 255, the doorbell 260, the graphics command processor 270, the compute command processor 280, and the queue manager 290. As described above, the user application 212, via the driver 214, may generate packets of commands that are delivered to the command processor 250. One mechanism of delivery may be by storing these packets in an application-related queue in the memory 220 (via the memory interface 222) and then signaling the command processor's doorbell 260 that one or more new packets are available in that queue in memory. In response to receiving such a signal, the doorbell is configured to trigger the reading of the one or more new packets by the fetcher 255. The fetcher 255 then reads packets from the queues in the memory 220 (via the memory interface 224) and pushes the packets into the packet queues 257, 258 in first-in, first-out (FIFO) order. Packets containing drawing (graphics) commands are stored in the queue 257, and packets containing dispatch computation commands are stored in the queue 258. The graphics command processor 270 and the computation command processor 280 are configured to pop packets out of the queues 257 and 258, respectively, as the packets arrive. When multiple applications 212 (e.g., running simultaneously on the host 210) generate respective packets, the packets associated with each application may be stored in a respective queue in the memory, and the fetcher 255 is configured to read packets from each queue in the memory 220 and push the packets into the respective packet queues 257, 258.

In one aspect, the packet format may include a packet header and one or more commands. As disclosed herein, the packet header encodes an operating mode, including a pass-through mode, a pause start mode, and a pause end mode. In the pass-through operating mode, the command processor 250 operates under normal operating conditions. That is, a newly arrived command is processed by currently available computing resources. For example, if all WGPs 240.1-240.N are involved in processing a wave associated with a previously received command, the newly arrived command must wait until one or more of the WGPs are available. In contrast, in the pause start operating mode, the command processor 250 is configured to pause execution of the currently processed wave and make all WGPs available to execute the wave associated with the newly arrived command. This operating mode is maintained until a pause end operating mode is enabled, at which point execution of the paused wave is resumed (restored) and the command processor 250 returns to operating under normal operating conditions again. The manner in which these three operating modes may be handled is disclosed further below.

When a packet encoding the pass-through mode of operation in its header is received, the command processor 250 decodes the command of the packet. The command in the packet may be a command used to set status or control registers associated with components of the APU 215. The command in the packet may also be a command used for synchronization operations. A significant number of the commands in the packet may relate to computational tasks directed to the shader 240, such as drawing (graphics) commands and compute dispatch commands. Thus, when the command processor 250 decodes a command, it may act on the command (e.g., set a status register according to the command) or send the command to the destination component to act on it. The drawing (graphics) command or compute dispatch command is processed by the graphics command processor 270 or compute command processor 280, respectively. These processors 270, 280 convert the respective commands into shader commands. The queue manager 290 stores these shader commands in respective queues and connects these queues to the execution pipes that feed the shader scheduler 230. The shader scheduler 230 assigns shader commands to the available WGPs 240.1 to 240.N.

Thus, in the pass-through mode of operation, commands are processed by the command processor 250 based on the currently available computing resources, i.e., the computational tasks defined by these commands are scheduled (230) on the currently available WGPs of the shader 240. However, if a packet is received that encodes a suspend start mode of operation in its header, the command processor 250 initiates the suspend operation of the currently processed wave on the WGPs 240.1-240.N of the shader 240 before the commands in the packet are processed. Similarly, if a packet is received that encodes a suspend end mode of operation in its header, the commands in the packet are processed, and then the command processor 250 ends the suspend by resuming (restoring) the suspended waves to continue their processing. Commands received during the suspend phase (i.e., the phase beginning with a packet encoding a suspend start mode and ending with a packet encoding a suspend end mode) have all the computing resources of the APU available, and therefore can be exclusively scheduled on all the WGPs 240.1-240.N of the shader 240. Then, during the suspend phase, the APU exclusively processes commands in packets fetched from queues in memory 220 associated with the application that generated the packets (the application that initiated the suspend mode, according to aspects disclosed herein). The APU does not provide (e.g., does not fetch) packets stored in queues in memory 220 associated with other applications until the suspend phase ends.

Thus, as described above, when the pause start mode is decoded from the header of the packet, a pause operation is triggered. That is, the command processor 250 signals the queue manager 290 to stop connecting any new queues to the execution pipes that feed the shader scheduler 230. Additionally, the queue manager 290 is signaled to switch off, pause, or stop any queues currently connected to the execution pipes. In one aspect, based on the information in the packet's header, the pause can be performed by pausing the currently running wave, by draining such wave, or by a combination thereof. Once the pause operation is completed, all shader resources are available and the shader's WGP is unused and therefore available to be scheduled with the computational tasks defined by the commands received during the pause phase. The pause phase continues until a pause end mode is decoded from the header of the subsequent packet. At that time, the pause wave is resumed (restored) as described above.

Aborting a currently executing wave can be used by a procedure called the Compute Wave Save and Restore (CWSR) procedure, through which a wave can be suspended and restored. In the CWSR procedure, the command processor 250 commands the shaders 240.1-240.N currently executing the wave to save their state to memory and remove themselves from execution. The command processor 250 then triggers the hardware machine to save the wave replay list to a stack in memory. To resume (restore) the wave, the command processor 250 pushes the stack back to the hardware execution unit, and the played waves then restore their state and resume operation where they previously stopped.

3 is a flow chart of an exemplary method 300 for in-line interruption of APU 215 in which one or more features of the present disclosure may be implemented. During normal operation, processor 210 transmits commands associated with computational tasks to APU 215 via one or more packets. Thus, method 300 begins with receiving a packet at step 310. At step 320, the header of the received packet is decoded to determine an operational mode. If the determined operational mode is not an interrupt-start mode, then at step 340, the command in the received packet is executed based on the currently available computing resources of APU 215. Thus, the wave associated with the command in the received packet may have to share WGP 240.1-240.N with other (currently executing) waves associated with commands received from previously received packets. However, if the operational mode is determined to be an interrupt-start mode, then shader processing is interrupted at step 330. That is, WGP 240.1-240.N are decoded to make all computing resources of shader 240 available. The currently executing wave in N is suspended. Once the suspension is complete, in step 340, the commands in the received packet are executed exclusively by shader 240. If in step 350, it is determined that the operating mode is a suspended-end mode, then in step 360, the suspended wave is resumed (restored). Once the suspended wave is resumed (restored) (360), the suspension phase (which continues between steps 330 and 360) ends; that is, commands in subsequent packets are executed while sharing computing resources with the resumed (restored) wave until a packet is received that encodes a suspended-enter operating mode that again triggers another suspension phase.

In one aspect, the operating mode may be changed by APU 215 based on an event, for example, associated with the processing of commands received in one or more packets. The mode may be changed from a pass-through operating mode (during normal operating conditions) to a suspend-enter operating mode to effect operation in the suspend phase. Alternatively, the mode may be changed from a pass-through operating mode (during operation in the suspend phase) to an suspend-end operating mode to cease operation in the suspend phase. For example, during the processing of commands by graphics command processor 270 or compute command processor 280 (or during the processing of a wave that executes shader commands associated with these processed commands), an event may occur that requires taking over all computing resources for the execution of all or a subset of these commands. In such a situation, command processor 250 may decide to change the operating mode from pass-through mode to suspend-enter mode in order to dedicate all computing resources of the APU to the execution of this subset of commands. At the end of the execution of this subset of commands, command processor 250 may change the operating mode to suspend-end mode and return to a normal operating state. Alternatively, while processing a command under the suspend phase, an event may occur that requires the suspend phase to be terminated, in which case the command processor 250 can change the operating mode to the suspend termination mode.

It should be understood that many variations are possible based on the disclosure herein. Although features and elements are described above in particular combinations, each feature or element can be used alone without other features and elements, or in various combinations with or without other features and elements.

The provided methods can be implemented in a general purpose computer, processor, or processor core. Suitable processors include, by way of example, general purpose processors, special purpose processors, conventional processors, digital signal processors (DSPs), multiple microprocessors, one or more microprocessors associated with a DSP core, controllers, microcontrollers, application specific integrated circuits (ASICs), field programmable gate array (FPGA) circuits, any other type of integrated circuit (IC), and/or state machines. Such processors can be manufactured by configuring a manufacturing process using the results of processed hardware description language (HDL) instructions and other intermediate data, including netlists (such as instructions that can be stored on a computer readable medium). The results of such processing can be a mask work, which is then used in a semiconductor manufacturing process to manufacture a processor implementing aspects of the embodiments.

The methods or flow charts provided herein may be implemented in a computer program, software, or firmware embodied in a non-transitory computer-readable storage medium for execution by a general purpose computer or processor. Examples of non-transitory computer-readable storage media include read only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVDs).

Claims (25)

1. A method for in-line interruption of an accelerated processing unit (APU), comprising:
receiving a packet containing an operating mode and a command for the APU to execute;
suspending execution of a command received in a previous packet in response to the operational mode being a suspend-start mode;
and the APU executing a command in the received packet.
method. resuming execution of the interrupted command in response to the operational mode being an interrupt-and-terminate mode.
2. The method of claim 1. Aborting execution of a command received in the previous packet comprises:
disconnecting a queue from each execution pipe, the queue including shader commands associated with commands received in the previous packet;
2. The method of claim 1. Aborting execution of a command received in the previous packet comprises:
and suspending a wave currently being executed by the APU based on information contained in a received packet, the wave executing a shader command associated with a command received in the previous packet.
2. The method of claim 1. Aborting execution of a command received in the previous packet comprises:
employing a Compute Wave Save Restore (CWSR) procedure to abort a wave associated with the command received in the previous packet;
2. The method of claim 1. Aborting execution of a command received in the previous packet comprises:
Draining a wave currently being executed by the APU based on information contained in a received packet, the wave executing a shader command associated with a command received in the previous packet.
2. The method of claim 1. The operational mode is changed by the APU based on an event associated with processing a command in a received packet or a previously received packet.
2. The method of claim 1. The APU is one of a graphics processing unit, a digital signal processor, a field programmable gate array processor, or a neural processing unit.
2. The method of claim 1. said executing being performed by an execution engine of said APU;
2. The method of claim 1. the execution engine is a shader;
9. The method of claim 8. a received packet is fetched by the APU from a queue in memory, the queue being associated with an application that generated the received packet;
2. The method of claim 1. 1. A system for in-line interruption of an APU, comprising:
At least one processor;
A memory for storing instructions,
The instructions, when executed by the at least one processor,
receiving a packet containing an operating mode and a command for the APU to execute;
suspending execution of a command received in a previous packet in response to the operational mode being a suspend-start mode;
the APU executing a command in the received packet;
causing the system to
system. The instruction:
in response to the operating mode being a suspend-end mode, causing the system to resume execution of the suspended command;
The system of claim 12. Aborting execution of a command received in the previous packet comprises:
disconnecting a queue from each execution pipe, the queue including shader commands associated with commands received in the previous packet;
The system of claim 12. Aborting execution of a command received in the previous packet comprises:
and suspending a wave currently being executed by the APU based on information contained in a received packet, the wave executing a shader command associated with a command received in the previous packet.
The system of claim 12. Aborting execution of a command received in the previous packet comprises:
employing a Compute Wave Save Restore (CWSR) procedure to abort a wave associated with the command received in the previous packet;
The system of claim 12. Aborting execution of a command received in the previous packet comprises:
Draining a wave currently being executed by the APU based on information contained in a received packet, the wave executing a shader command associated with a command received in the previous packet.
The system of claim 12. The operational mode is changed by the APU based on an event associated with processing a command in a received packet or a previously received packet.
The system of claim 12. 1. A computer-readable storage medium comprising hardware description language instructions describing an accelerated processing unit (APU) configured to perform an in-line interruption of the APU, the computer-readable storage medium comprising:
The APU is
receiving a packet containing an operating mode and a command for the APU to execute;
suspending execution of a command received in a previous packet in response to the operational mode being a suspend-start mode;
the APU executing a command in the received packet;
It is possible to
A computer-readable storage medium. The APU is
In response to the operation mode being a suspend-end mode, execution of the suspended command can be resumed.
20. The computer readable storage medium of claim 19. Aborting execution of a command received in the previous packet comprises:
disconnecting a queue from each execution pipe, the queue including shader commands associated with commands received in the previous packet;
20. The computer readable storage medium of claim 19. Aborting execution of a command received in the previous packet comprises:
and suspending a wave currently being executed by the APU based on information contained in a received packet, the wave executing a shader command associated with a command received in the previous packet.
20. The computer readable storage medium of claim 19. Aborting execution of a command received in the previous packet comprises:
employing a Compute Wave Save Restore (CWSR) procedure to abort a wave associated with the command received in the previous packet;
20. The computer readable storage medium of claim 19. Aborting execution of a command received in the previous packet comprises:
Draining a wave currently being executed by the APU based on information contained in a received packet, the wave executing a shader command associated with a command received in the previous packet.
20. The computer readable storage medium of claim 19. The operational mode is changed by the APU based on an event associated with processing a command in a received packet or a previously received packet.
20. The computer readable storage medium of claim 19.

Images