DE102022203284A1 - Processor for carrying out a predetermined arithmetic operation and arithmetic unit - Google Patents

Abstract

The invention relates to a processor (2) for carrying out a predetermined arithmetic operation, in which one or more data elements (31, 32) are used to determine a result (34), comprising one or more arithmetic cores (4) and at least one buffer ( 10, 20), connectable to a main memory (6) and, when the main memory is connected, adapted to access the main memory, each computing core (4) being adapted to execute instructions; wherein the at least one buffer (10, 20) comprises a calculation circuit (14, 24) adapted to execute the arithmetic operation in response to an execution signal when the one or more data elements are stored in the buffer, the result being in the buffer is saved; wherein the processor (2) is set up to carry out the arithmetic operation either using one of the arithmetic cores (4) using the instructions or in the at least one buffer (10, 20) using the respective calculation circuit (14, 24).

Description

The present invention relates to a processor for carrying out a predetermined arithmetic operation and a arithmetic unit.

Background of the invention

Processors with multiple computing cores can be used for computationally intensive data processing or calculation methods. In particular, so-called streaming multiprocessors, such as graphics processors, can be used for highly parallelizable calculation methods. These generally have a large number of computing cores (also known as shaders) in order to achieve high computing speed. A bottleneck can consist of a limited memory bandwidth, which can mean that data required by the computing cores for the calculation process or data specific to the calculation process can only be read from the memory or stored in the memory with a delay, which results in corresponding waiting times for the computing cores results.

Disclosure of the invention

According to the invention, a processor for carrying out a predetermined arithmetic operation and a computing unit with such a processor with the features of the independent claims are proposed. Advantageous refinements are the subject of the subclaims and the following description.

The invention makes use of the measure of providing a calculation circuit in at least one buffer, which is designed to execute the predetermined arithmetic operation in response to an execution signal when data elements (which are used in the arithmetic operation) are stored in the buffer, the result being in the buffer is stored (without using a computing core; saving the result can be viewed as part of the computing operation). Furthermore, the processor is set up to carry out the arithmetic operation either using a computing core using instructions or to carry it out in the at least one buffer memory using the respective calculation circuit.

An advantage of the invention is that data traffic between the computing cores and the buffer memory can be avoided, in particular latency times for transferring the data elements into a computing core and for transferring the result can be avoided. Another advantage is that other operations can be carried out in the computing cores in parallel to the execution of the computing operation in the buffer (as long as simultaneous access to the buffer is avoided). The optional execution of the arithmetic operation in a arithmetic core is advantageous if the data elements and/or the result are required immediately for a further operation in the arithmetic core, so that they can remain in registers of the arithmetic core without access to the buffer memory.

In detail, the processor comprises a plurality of computing cores and at least one buffer memory, is connectable to a main memory and is set up to access the main memory (reading and writing including the buffer memory) when the main memory is connected to the processor. Each computing core is designed to execute instructions so that operations, including the predetermined computing operation, can be implemented using the instructions. The predetermined arithmetic operation uses one or more data elements to determine a result.

The term “computational operation” refers to a general operation that can be performed by computing cores or corresponding specific circuits. In computing cores, such a computing operation can be implemented by one or more instructions. Instructions can be, for example, machine instructions (instructions in machine language) or instructions from a so-called “Instruction Set Architecture” (ISA). The latter are used in particular in streaming multiprocessors, such as graphics processors. Arithmetic operations can be elementary operations such as arithmetic operations (e.g. addition, multiplication, ...), bit shift operations (i.e. moving bits of a data element), copy operations (e.g. a data element is copied to another memory location), or similar. Combinations of such elementary operations are also considered arithmetic operations, e.g. the multiplication of two data elements and addition of the product to a continuous addend (accumulator) (English: Multiply-Accumulate, MAC), a ← α + (b · c).

• mov.u16 %rh1, %ctaid.x;
• mov.u16 %rh2, %ntid.x;
• mul.wide.u16 %r3, %rh1, %rh2;
• add.u32 %r4, %r4, %r3;
• st.global.f32 [%result.x], %r4;

The term “optional” here may refer to the processor being set up to interpret and implement corresponding instructions. An ISA (Instruction Set Architecture) can be expanded or changed accordingly. For example, in a normal ISA, a typical MAC arithmetic operation can be medium through Instructions to be executed by a computing core are implemented as follows:

• mov.u16 %rh1, %ctaid.x;
• mov.u16 %rh2, %ntid.x;
• mul.wide.u16 %r3, %rh1, %rh2;
• add.u32 %r4, %r4, %r3;
• st.global.f32 [%result.x], %r4;

• Pim.mac.u16 %result.x, %ctaid.x, %ntid.x;

In an extended ISA, the MAC arithmetic operation could be implemented by a single instruction that causes the computation in the buffer:

• Pim.mac.u16 %result.x, %ctaid.x, %ntid.x;

In this example, the operation is essentially performed over a memory area. To do this, two memory locations are multiplied together at least twice and these results of the multiplication of two memory cells are added to calculate the result of the MAC arithmetic operation. This is done in parallel in hardware. There is no continuous addend as in a sequential approach.

The prefix “Pim” (Processing-in-memory) denotes instructions to be executed in the buffer (Pim instructions). The processor could therefore carry out the arithmetic operation in a arithmetic core using the above instructions or in the buffer memory using the calculation circuit using the Pim instruction.

The calculation circuit is a circuit arranged in the buffer or together with a memory latch of the buffer. This is set up to read the data elements from the buffer, carry out the calculation and save the result in the buffer. This takes place in particular in response to an instruction (execution signal), for example from a corresponding control unit of the processor. In particular, the instruction also includes memory addresses that relate to the data elements and the result, i.e. memory addresses where the data elements are located and memory addresses to which the result is to be written. The result can include one or more data elements.

Alternatively or additionally, the calculation can be carried out by the calculation circuit automatically in response to a write access to one of the data elements and/or automatically in response to a read access to the result. This means that the calculation is triggered when there is a write access to at least one of the memory addresses where the data elements are stored, or a read access to memory addresses where the result is to be stored. Write/read accesses can therefore be viewed as execution signals.

When using the calculation circuit, the data elements remain in the buffer memory without being loaded into a computing core, such as an ALU (Arithmetic Logic Unit) of a computing core. For example, during a MAC operation carried out by a computing core, a new operand is repeatedly loaded from the memory into the ALU and processed. When using a calculation circuit in the buffer memory (i.e. “in-memory computing”), the data remains in memory. At the same time, the calculation circuit, i.e. hardware, calculates an arithmetic operation without the ALU having to do anything, for example like a MAC operation.

Preferably, the execution signal is an instruction and/or a write access to one or more of the data elements and/or a read access to the result. For example, an instruction can also contain memory addresses for the data elements and the result. If the execution signal is a write access to (at least) one of the data elements or a read access to the result, the execution of the arithmetic operation in the calculation circuit takes place automatically, without a separate instruction. In this case, the memory addresses for the data elements and the result can be predetermined in the calculation circuit (in the sense of predetermined in hardware), i.e. the calculation circuit is assigned predetermined memory addresses in the buffer memory, or registers can be provided in the calculation circuit that correspond to the Memory addresses can be initialized.

If at least one of the required data elements is not already stored in the cache (cache miss or cache miss), it is first loaded into the cache, e.g. from the main memory or, in the case of several hierarchically organized caches, from another cache (e.g. from Level 2 cache into the level 1 cache). This is done by a management module of the cache, which is known to those skilled in the art and is included in the processor and is typically implemented as hardware and which implements a suitable cache strategy.

Accordingly, the processor is preferably set up to determine, if the arithmetic operation is to be carried out in the at least one buffer memory, whether the one or the plurality of data elements are in the at least one buffer, and if the one or more data elements are not (yet) stored in the at least one buffer, to load or store the one or more data elements in the at least one buffer. Loading (or reading and saving) can take place from the main memory or from another buffer, for example from the level 2 cache to a level 1 cache.

If the arithmetic operation is to be carried out in the at least one buffer, the processor is preferably set up to write the one or more data elements to predetermined memory addresses or to memory addresses in the at least one buffer that are determined by initializable registers. This embodiment is particularly advantageous if the arithmetic operation is carried out automatically, i.e. if the execution signal is a write access to (at least) one of the data elements or a read access to the result.

Preferably, the choice of whether the arithmetic operation is carried out using one of the arithmetic cores or in the at least one buffer memory is based on an expected degree of reusability of the one or more data elements and/or the result. The term “degree of reusability” basically refers to a probability with which a data element and/or result will be required, i.e. reused, in subsequent operations. The degree of reusability can, for example, be given directly as a probability or as a time period (e.g. given in clock cycles of the processor) or the inverse of the time period until a data element and/or result is used again (generally in an operation other than the arithmetic operation). The degree of reusability can be determined, for example, by analyzing operations in a task that includes the arithmetic operation. It is also possible to determine the degree of reusability through statistical analysis during the execution of one or more tasks that include the computing operation.

In particular, the expected level of reusability refers to operations within a task. The term “task” is intended to describe a self-contained part of a computer program or an entire computer program. A task generally includes several operations, namely other operations or other arithmetic operations in addition to the arithmetic operation. Of course, the arithmetic operation itself can occur several times within a task.

More preferably, the at least one buffer is selected to carry out the arithmetic operation if the expected degree of reusability is within a predetermined range. In this way, tasks that include the arithmetic operation and use specific data elements/results can be specifically selected for execution in the buffer memory.

The at least one buffer preferably comprises a plurality of first buffers and a second buffer, each first buffer being assigned to one of the computing cores or a subset of the computing cores and the second buffer being assigned to all computing cores, and wherein the processor is further configured to do so when the computing operation is carried out in the at least one buffer is to be carried out, the arithmetic operation can be carried out either in one of the first buffers or in the second buffer. The first caches can be viewed as a (relatively small) level 1 cache, the second cache can be viewed as a level 2 cache. It is therefore a hierarchical buffer structure as used in streaming multiprocessors. Overall, the processor can carry out the arithmetic operation through the computing cores, a first buffer or the second buffer.

The choice of whether the arithmetic operation should be carried out in one of the first buffers or in the second buffer is preferably made based on one or the expected degree of reusability of the one or more data elements and/or the result. More preferably, one of the first buffers is selected to carry out the arithmetic operation when the expected degree of reusability is above a predetermined first threshold, and/or the second buffer is selected to carry out the arithmetic operation when the expected degree of reusability is below or equal to the predetermined first threshold the predetermined first threshold. The arithmetic operation is advantageously carried out in the first buffer memory if the data elements or the result are immediately reused, i.e. have a high degree of reusability, since they remain so close to the computing core (s) assigned to the first buffer memory and are therefore integrated into one with low latency this computing core can be loaded. If the degree of reuse is lower, the second cache is expediently used, which can lead to a reduction in data traffic and fewer cache misses.

A computing unit according to the invention comprises a processor according to the invention and a main memory connected thereto, the main memory comprising a calculation circuit which is designed to execute the arithmetic operation in response to an execution signal if the one or more data elements are stored in the main memory, the result being in Main memory is stored, and wherein the processor is further configured to selectively cause the main memory to perform the arithmetic operation using the calculation circuitry of the main memory. It is therefore possible to carry out the arithmetic operation in the main memory, so that data transfer between the main memory and the processor can be avoided. As with the at least one buffer, the execution signal (for the main memory) is preferably an instruction and/or a write access to one (or more) of the data elements and/or a read access to the result.

Preferably, the choice as to whether the main memory should be caused to perform the arithmetic operation is based on one or the expected degree of reusability of the one or more data elements and/or the result. More preferably, the main memory is caused to carry out the arithmetic operation when the degree of reusability is below a predetermined second threshold. The arithmetic operation is expediently carried out in the main memory if it is very unlikely or practically impossible that the data elements and / or the result will be used again in an operation or that they will be used again within an expected period of time, after which they will be used in accordance with the Cache strategy can be deleted from the cache.

The second threshold, the first threshold introduced above and the range for the degree of reusability also introduced above can be selected appropriately. The exact specification depends on the definition of the degree of reusability and can be chosen appropriately by the expert.

Further advantages and refinements of the invention result from the description and the accompanying drawing.

The invention is shown schematically in the drawing using exemplary embodiments and is described below with reference to the drawing.

Brief description of the drawings

• 1 shows a subassembly of a processor according to a preferred embodiment.
• 2 shows a processor connected to a main memory according to a preferred embodiment.
• 3 shows a computing unit according to a preferred embodiment.

Embodiment(s) of the invention

1 shows a subassembly 3 of a processor according to a preferred embodiment. In the subassembly 3, several computing cores 4 are arranged, for example, in two (computing core) groups. A first cache 10 (eg level 1 cache) is assigned to each group. In general, each group can include one or more computing cores. The first buffer 10 is used to temporarily store data that is between the computing cores and a main memory or another buffer (in 1 not shown).

A diagram of a first buffer 10 is on the left 1 shown enlarged. The first buffer 10 comprises a memory array 12, ie an arrangement of memory cells (for example SRAM cells; SRAM: Static Random Access Memory). By means of an address circuit 16, data elements 31, 32, 34 stored in the memory array 12 can be addressed so that they can be accessed (read and write). A memory latch 18 or memory catch register is used to briefly store the data elements or their bits during memory accesses to the memory array 12, so that they can be read out using the memory latch, or the contents of the memory latch in the Memory cells can be transferred (based on the data element addressed by the address circuit 16).

The first buffer 10 has a calculation circuit 14 (or buffer calculation circuit or first buffer calculation circuit). The calculation circuit 14 is set up to carry out a specific arithmetic operation, such as the already mentioned MAC operation, for data elements stored in the memory array 12 of the first buffer 10 and to store the result of the arithmetic operation as a data element in the memory array 12. In the example shown, the arithmetic operation or the calculation circuit 14 that implements the arithmetic operation determines a result from two operands that are stored as data elements 31, 32 in the memory array 16, which is as Data element 34 is stored in memory array 16. In the example of the MAC operation, the final result of the sum formation is saved as a result; there is no interim saving/loading of intermediate results of a running total or similar. The arithmetic operation is carried out by the calculation circuit 14 of the first buffer 10 without the participation of the computing cores 4. There is no need to transfer the data elements to one of the computing cores and transfer the result back to the first buffer, so that delays caused by the data transfer can be avoided. The arithmetic operation is carried out by the calculation circuit in response to a corresponding instruction or generally in response to an execution signal.

2 shows a processor 2 connected to a main memory 6 for data transmission, according to a preferred embodiment. The main memory 6 can be, for example, a DRAM memory (DRAM: Dynamic Random Access Memory). The processor 2 can read data elements from the main memory 6 and store them in the main memory 6. The main memory usually has a significantly higher storage capacity than the buffer memory contained in the processor, while the computing cores can access buffer memory much faster.

The processor 2 comprises several sub-assemblies 3 (for example two, although the number can also be different), which correspond to the in 1 shown embodiment can be designed, so that their structure is not explained again. Deviating from the in 1 In the embodiment shown, it can (but does not have to) be provided that the first buffer stores 10 do not include a calculation circuit 14.

The processor 2 additionally includes a second buffer 20 (left in 2 shown enlarged as a diagram), which is assigned to all subassemblies 3 or all computing cores 4. Data stored in main memory 6 is typically transferred to the computing cores 4 and from there back to main memory 6 using the first and second buffers, using appropriate buffering strategies.

The second buffer 20 (e.g. level 2 cache) is constructed analogously to the first buffer 10, i.e. it comprises a memory array 22 (arrangement of memory cells, e.g. SRAM cells, an address circuit 26, by means of which data elements 31, 32 stored in the memory array 12 , 34 can be addressed so that they can be accessed (read and write), and a memory latch 28 that is used to briefly store the data elements or their bits during memory accesses to the memory array 22, so that they can be stored by means of of the memory latch can be read out, or the contents of the memory latch can be transferred to the memory cells.

The second buffer 20 includes a calculation circuit 24 (second buffer calculation circuit). The calculation circuit 24 is set up to carry out the specific arithmetic operation, for example the MAC operation already mentioned, for data elements stored in the memory array 22 of the second buffer 20 and to store the result of the arithmetic operation as a data element in the memory array 22. In the example shown, the arithmetic operation or the calculation circuit 24 that implements the arithmetic operation determines a result from two operands that are stored as data elements 31, 32 in the memory array 26, which is stored as data element 34 in the memory array 26. The arithmetic operation is carried out by the calculation circuit 24 of the second buffer 10 without the involvement of the computing cores 4 (or the first buffer). A transfer of the data elements to one of the computing cores, which would also run via the first buffer 10, is therefore no longer necessary.

3 shows a computing unit 1 according to a preferred embodiment. The computing unit 1 has a processor 2 (approximately according to the embodiments of 1 or 2 ) and a main memory 6 connected to it. In addition, an optional interface 8, which can be provided for data communication between the computing unit 1 and other computing units, is provided by way of example. The interface 8 is, for example, connected to the processor 2 and preferably to the main memory 6 for data communication.

Similar to the first and/or the second buffer 10, 20, the main memory 6 preferably comprises a calculation circuit 44 (main memory calculation circuit) which is set up to carry out the specific arithmetic operation for data elements stored in the main memory 6 and the result of the arithmetic operation as a data element in the main memory 6 to save.

Claims (12)

Processor (2) for carrying out a predetermined arithmetic operation in which one or more data elements (31, 32) are used to determine a result (34), the processor having one or more computing cores (4) and has at least one buffer (10, 20) and is connectable to a main memory (6) and is adapted to access the main memory when the main memory is connected, each computing core (4) being adapted to execute instructions; wherein the at least one buffer (10, 20) has a calculation circuit (14, 24) which is adapted to execute the arithmetic operation in response to an execution signal if the one or more data elements are stored in the buffer, the result being in the buffer is saved; wherein the processor (2) is set up to carry out the arithmetic operation either using one of the arithmetic cores (4) using the instructions or in the at least one buffer (10, 20) using the respective calculation circuit (14, 24). Processor after Claim 1 , where the execution signal is an instruction and/or a write access to one of the data elements and/or a read access to the result. which is set up to make the choice as to whether the computing operation is carried out using one of the computing cores (4) or in the at least one buffer (10, 20), based on an expected degree of reusability of the one or more data elements (31, 32) and /or the result (34). Processor after Claim 3 , which is set up to select the at least one buffer (10, 20) for carrying out the arithmetic operation if the expected degree of reusability is within a predetermined range. Processor according to one of the preceding claims, wherein the at least one buffer comprises a plurality of first buffers (10) and a second buffer (20); wherein each first buffer (10) is assigned to one of the computing cores (4) or a subset of the computing cores and the second buffer (20) is assigned to all computing cores; wherein the processor (4) is further set up, if the arithmetic operation is to be carried out in the at least one buffer, to carry out the arithmetic operation either in one of the first buffers or in the second buffer. Processor after Claim 5 , which is set up to make the choice as to whether the arithmetic operation should be carried out using one of the first buffers (10) or in the second buffer (20), based on an expected degree of reusability or, to the extent dependent on one of the Claims 3 or 4 the expected degree of reusability of the one or more data elements (31, 32) and/or the result (34). Processor after Claim 6 which is set up to select one of the first buffers (31) for carrying out the arithmetic operation if the expected degree of reusability is above a predetermined first threshold; and/or to select the second buffer (32) for carrying out the arithmetic operation if the expected degree of reusability is below the predetermined first threshold or is equal to the predetermined first threshold. Processor according to one of the preceding claims, which is designed to determine, when the arithmetic operation is to be carried out in the at least one buffer (10, 20), whether the one or more data elements are in the at least one buffer and if the one or the plurality of data elements are not stored in the at least one buffer, to load or store the one or more data elements in the at least one buffer. Processor according to one of the preceding claims, which is set up if the arithmetic operation is to be carried out in the at least one buffer (10, 20), the one or more data elements (31, 32) are sent to predetermined memory addresses or to memory addresses determined by initializable registers to write at least one buffer to it. Computing unit (1) comprising a processor (2) according to one of the preceding claims and a main memory (6) connected thereto; wherein the main memory (6) comprises a calculation circuit (44) adapted to execute the arithmetic operation in response to an execution signal when the one or more data items are stored in the main memory, the result being stored in the main memory; wherein the processor is further configured to selectively cause the main memory to perform the arithmetic operation using the calculation circuitry of the main memory. Calculation unit according to Claim 9 , wherein the processor is set up to make the choice as to whether the main memory (44) should be caused to carry out the arithmetic operation based on an expected degree of reusability or as far as depends on one of the Claims 3 until 8th the expected level of reusability of the one or more data elements and/or the result. Calculation unit according to Claim 10 , wherein the processor is set up to cause the main memory (6) to carry out the arithmetic operation if the degree of reusability is below a predetermined second threshold.

Images