TWI855303B - Multiply-and-accumulate unit and multiply-and-accumulate methods - Google Patents

Abstract

Floating point Multiply-Add, Accumulate Unit, supporting BF16 format for Multiply-Accumulate operations, and FP32 Single-Precision Addition complying with the IEEE 754 Standard. The Multiply-Accumulate unit uses higher radix and longer internal 2’s complement significand representation to facilitate precision as well as comparison and operation with negative numbers. The addition can be performed using Carry-Save format to avoid long carry propagation and speed up the operation. The circuit uses early exponent comparison to shorten the accumulate pipeline stage. Operations including overflow detection, zero detection and sign extension are adopted for 2s complement and Carry-Save format

Description

Multiplication and accumulation unit and multiplication and accumulation method

The field of this disclosure is the implementation of arithmetic logic circuits, including floating-point multiply-add-accumulate circuits, sometimes referred to as multiply-and-accumulate circuits, for use in high-speed processors, including processors configured to efficiently perform training and inference.

References for priority applications

This application claims priority to U.S. Patent Application No. 17/534,376 filed on November 23, 2021, U.S. Patent Application No. 17/465,558 filed on September 2, 2021, U.S. Patent Application No. 17/397,241 filed on August 9, 2021, U.S. Provisional Patent Application No. 63/190,74 filed on May 19, 2021, 9, U.S. Provisional Patent Application No. 63/174,460 filed on April 13, 2021, U.S. Provisional Patent Application No. 63/166,221 filed on March 25, 2021, U.S. Provisional Patent Application No. 63/165,073 filed on March 23, 2021, and U.S. Provisional Patent Application No. 63/239,384 filed on August 31, 2021. All eight of the above applications are incorporated herein by reference.

Arithmetic logic circuits including floating point, multiplication and accumulation units implemented in high-performance processors are relatively complex logic circuits. Multiplication and accumulation circuits are used for matrix multiplication and other complex mathematical operations, which are used in machine learning and inference engines.

Essentially, the multiplication and accumulation circuit generates the sum S(i) of a sequence of terms A(i)*B(i), usually expressed as follows:

Here, the sum S(i) at cycle (i) is equal to adding the term A(i)*B(i) to the sum S(i-1), which is the accumulation of terms A(0)*B(0) to A(i-1)*B(i-1). The final sum S(N-1) is the sum output of the multiplication and accumulation operations for N cycles (from 0 to N-1).

In the floating-point implementation, each cycle multiplies two input floating-point operands A(i) and B(i), including the exponent value and the effective value, to generate a multiplier output term A(i)*B(i), and then calculates the accumulator output sum S(i) by adding the multiplier output term A(i)*B(i) of the current cycle to the accumulator output sum S(i-1) of the previous cycle.

In a floating-point encoding format used to encode floating-point numbers in computation, numbers can be normalized so that the significand includes a one-bit integer (always "1" in binary) to the left of the binary point, and a fraction represented by some bits to the right of the binary point, with the number encoded using only the fraction. The binary 1 integer is omitted from the encoding because it can be implied by the normalization form. Operations on floating-point encoding format numbers encoded in this way take into account the integer to the left of the binary point, which is called "implied 1".

Multiplication of floating point numbers can be performed by adding the exponents, multiplying by the significand, and then normalizing the result by shifting the output significand and adjusting the output exponent to accommodate this shift.

Addition of floating-point numbers can be implemented by first identifying the larger exponent and the difference between the exponents of the operands, then shifting the significand of the operand with the smallest exponent to align with the larger exponent. Finally, the result is normalized, which may involve additional shifting of the significand and adjustment of the exponent.

Calculations on numbers in unsupported formats, such as floating point encoding formats, can result in exceptions. In data streaming architectures and other architectures that execute complex algorithms (such as machine learning algorithms), these exceptions can cause the algorithms to stop or fail. Exceptions that cause algorithms to stop or fail in real-time systems can cause system failures or other performance issues.

There is a need to provide an exception handling system that can be applied to complex data processing settings.

110:Bfloat16

130: IEEE754 single-precision 32-bit floating point (FP32)

202:Multiplier circuit

210a: Multiplier and adder block

210b: Index Block

210:Multiplexer

211:Multiplexer

212: Multiplexer

213: Operator A

214: Operator B

215: Base 8 converter

216: Operator C; line

217: BF16 format or FP32 format

218: Line

219: Line

220: Block

221: Line

222:Double bus

223:Bus

224: Output bus

225: Output bus

226: Bus

227: Bus

228:Bus

229: Bus

230: Carry-save adder

240: Accumulator

240A: Index control unit

240B: Exponential comparator unit

240C: Significant number part

241: 42-bit decimal carry register

242: 42-bit decimal sum register

250: Carry-preserve to sign-value conversion block

251:Bus

252:Bus

260: Basis 8 to Basis 2 Transformation and Normalization Blocks

270: FP32 or BF16 block

300: Simplified block diagram

410:8X8 BF16 multiplier circuit

420: Register

421: Register

422: Line

423: Line

424: Line

425: Line

426: Line

427: Line

428: 7-bit LSB bus

429: 7-bit LSB bus

430: 7-bit ripple-carry adder

440: Line

450: Register

460: Register

461: Inverter

462: Line

464: Exponential adder circuit

465: Line

466: 10-bit value

467: Special index detection block

468: Line

469: Line

470: Register

471a: Anti-mutex or gate

471b: Anti-mutex or gate

471c: Anti-mutex or gate

473: Line

501: Line

502: Significant number final adder circuit

503: Line

504: Cache

506: Overflow selection circuit

507: Line

508:Sign bit selection circuit

509: Line

510: Index selection circuit

511: Line

512: Valid number selection circuit

513: Line

514: 8-bit left shifter circuit

515:Bus

516:2's complement inverting and adding 1 circuit

517:Bus

518:Multiplexer circuit

519: Overflow signal

520: Register

521: Line

522: Exponent overflow detection circuit

523: Input bus; line

524: Index abnormal processing circuit

525: Line

526: Index abnormality detection circuit

527: Line

528: Output abnormal control signal generating circuit

529:Bus

530: Register

531:Sign bit

532: Register

533: Line

537: output bus; line

539: Line

541: Line

543: Line

544: And the gate

545: Line

547:Bus

549: Line

551: Line

553:Bus

560: Register Fcin

592: Base 8 converter block

592a: Final addition significand selection and base 8 conversion sub-block

592b: Index exception handling sub-block

602: Product

604: Rounding block

605: Overflow detection block

606:LZA circuit

607:Bus

608: Shift register SHR8/16/24 circuit

609: Shifter circuit

610: Shifter circuit

611:Bus

612: Total rounded to square

613:Bus

614: Carry-save adder circuit

617: And the gate

619: Inverter

630: Shifter index control signal generation/bypass control circuit

633: Line

634: Line

636: Output pipeline register

644: Line

646: Line

647: Line

648: Line

650: 16-bit status register

652:16-bit multiplier exponential comparison circuit

654: Exponent accumulator (Eacc) register

660:Multiplier

661:Reducer

662:Symbol expansion detection unit

664: And the gate

665: And the gate

667: Carry bus

668: And the gate

669: Total bus

670: or gate

671: line; input

673: Input

675: Input

677: Input

679: Line

681: Line

682: Line

683: Line

684: Simple product rounding block

685: Output

686: And the gate

687: And the gate

688: And the gate

689: Output

692:Bus

693:Bus

700: Final conversion

270: Normalization to symbolic numeric format block

270a: Convert from carry-save to sign-value format block

270b: Convert from base 8 to base 2 floating point number block

702: Bus

704:Bus

706: Line

708:43-bit adder circuit

710: Second circuit LZA/LOA

711: Line

712: Line

714:LZA POS selection circuit

715:Bus

716:Bus

717:Bus

718: Inverting plus 1 circuit

719: Line

720: Valid number selection multiplexer circuit

726:Sign bit

728: Effective zero detection circuit

730: Register

735:SHL left shifter circuit

736: Line

738:Bus

739:Signal

740: Exponential adder circuit

742: Line

744: Line

745:Bus

746: Line

747: Line

748: Register

750: Abnormal control (8-bit) register

751: Abnormal control register

752:Over/under detection circuit

753: Line

755: Line

756: Decimal zero register

758:norm_en

760: Under-detected multiplexer

770: 39-bit significant digit

788: Line

801: Control line signal Out_FP32

802: First multiplexer

803: Line

805:Bus

807:Bus

809:Bus

810A: Adder sign positive state circuit

811B: Status circuit

812A: Circuit

817: Line

819: Line

820:Multiplexer

821: Line

823: Round to zero selection line

825: Line

827: Line

829: Line

830: Rounding circuit

831: Line

833:Bus

835: Line

837:39-bit valid number register 770 bus fpst_l[38:0]

840:Multiplexer

850: Second rounding increment circuit

860: First round-up increment circuit

907: Line

930: 23-bit significand (IEEE 754) register

953: Eight Situations

957: Line

959: Line

961: Line

962: Infinite Control

963: Line

964:EPST_L[9:0] bus

965:Bus

967:Signal line

968: 9-bit Enorm[8:0] bus

969: Line

970: Zero Control Logic

971: Line

972: Line

974: First multiplexer (zero)

975: Line

976:Multiplexer

979: Index signal bus

980: Index register

982:Multiplier

983:Signal line

986: Under/overflow detection and index abnormality detection circuit

987: or gate

988:Symbol generation and exception handling circuits

990: Register

991: Line

992: Infinite detection circuit

994: Non-conventional circuits

1000: floating point number range

1010:Block

1100: Example high-level architecture block diagram

1102: Multiplier exception processing block

1104: Multiplier exception flag

1106: Bus

1108: Bus multiplier abnormal condition signal

1110:Multiplier block

1113: Operator A

1114: Operator B

1115: Abnormal result of multiplier

1116: Operator C

1118: Operator C basic conversion block

1120: Operator C abnormal condition signal bus

1122: Bus

1124: Accumulator loop

1126: Abnormal output control signal generation block

1128: Abnormal Control

1129: Output block

1130: Carry-save adder (CSA) block

1131: Output block

1132: Bus

1134: Adder normalization exception handling block

1138:Bus

1139: Adder abnormal result

1140: Adder exception flag block

1300:Exception processing structure

1302: Multiplier exception flag generation block

1304: Abnormal flag spawning block

1306: Adder abnormal flag generation block

1308: Floating point multiplier accumulator exception block

1310: Multiplier abnormal result generation block

1312: Abnormal result generation block

1314: Adder abnormal result generation block

1320A: Multiplier symbol generation status block

1320B: Block

1320C: Block

1322A: Multiplier exponent generation status block

1322B: Block

1322C: Block

1322D: Block

1324A: Multiplication decimal generation status block

1324B: Block

1324C: Block

1326A: Adder symbol generation status block

1326B: Block

1326C: Block

1328A: Adder index generation status block

1328B: Block

1328C: Block

1328D: Block

1330A: Adder decimal generation status block

1330B: Block

1330C: Block

1381:Multiplier overflow flag status block

1382: Multiplier under-bit flag status block

1383: Multiplier invalid flag status block

1387: Adder overflow flag status block

1388: Adder under-bit flag status block

1389: Adder invalid flag condition block

1400: Realization

1402: Least Significant Bit (LSB)

1404: Most Significant Bit (MSB)

1406: Least Significant Bit (LSB)

1408: Most Significant Bit (MSB)

1410: Least Significant Bit (LSB)

1412: Most significant bit (MSB)

1414: Multiplication operation enabled

1418: Multiplication operation enabled

1420: And the gate

1430: And the gate

1440: Product exponent and gate

1442: Output

1444: And the gate

1445: Anti-OR Gate

1446: Multiplier overflow flag status

1450: Anti-OR Gate

1460: Anti-OR Gate

1470: Product exponent anti-OR gate

1472: Output

1475: Anti-OR Gate

1480: Multiplier lacks bits and gates

1482: Multiplier under-bit flag status

1500: Realization

1501: Multiplication operation enabled

1502: Least Significant Bit (LSB)

1504: Most significant bit (MSB) octet

1506: Least Significant Bit (LSB)

1508: Most Significant Bit (MSB)

1510: And the gate

1520: Anti-OR Gate

1530: Jimen

1540: Anti-OR Gate

1550: Ji Gate

1560: Ji Gate

1570: Or gate

1580: Multiplier invalid and gated

1552: First input

1562: Second input

1572: Output

1582: Multiplier invalid flag condition

1600: Implementation

1618:Symbol A

1620:EX-or-gate

1622:Symbol B

1630: Jimen

1632: Second input

1700: Realization

1710: Or gate

1720: Jimen

1730: Or gate

1740: Anti-OR Gate

1750:Multiplexer

1760: Or gate

1770:Multiplexer

1752:Multiplier index

1762: Output

1772:Multiplier fractional bus

1800: Schematic implementation

1810: Jimen

1820: Anti-or gate

1830: Jiguan

1822: Non-import index is infinitely large

1832: Adder overflow

1850: Jiguan

1860:EX-or-gate

1870: Jiguan

1880: Anti-or Gate

1890: Jiguan

1872: Output exactly zero

1882: Non-input exact zero

1892: Adder lacks position

1900: Realization

1910: Or gate

1920:EX-or-gate

1930: Ji-Gan

1912: Output

1932: Adder invalid flag

1940: Anti-or Gate

1942: Output

1950: Jimen

1960:EX-or-gate

1965: Or Gate

1967: Lack of position

1968: Inverting Gate

1970: Jimen

1972: Lack of position

1980: Or Gate

1990: Jimen

1982: Output

1992: Signal adder symbol positive bit

2000: Realization

2005: Inverting Gate

2010: Anti-gate

2020: Gate

2030: Gate

2040: Or gate

2050: or gate

2060: And the gate

2070:EX-or-gate

2080: or gate

2012: Output

2022: Output

2032: Output

2042: Adder sign negative status bit

2052: Judgment

2082: Adder index all "0" selection

2100: Schematic implementation

2110: or gate

2120: or gate

2112: Judgment

2122: Adder index all "1" selection

2130: or gate

2131:Selector control

2132: Adder decimal output

2150:Multiplexer

[Figure 1] illustrates the encoding format of BFloat16 and the floating-point IEEE-754 standard.

[Figure 2] illustrates a high-level block diagram of a floating-point multiply-add-accumulate unit with carry-save accumulator for BF16 and FP32 formats.

[Figure 3] illustrates a hierarchical block diagram of a multiplier circuit with two inputs (operand A and operator B).

[Figure 4a] illustrates an example multiplier and adder block containing a partial product reduction tree of 8x8 multipliers.

[Figure 4b] illustrates an example index unit with a special index detection block.

[FIG. 5A] illustrates a hierarchical block diagram showing a base-8 converter including example final addition, significand selection, and base-8 conversion blocks and example exponent exception handling blocks.

[FIG. 5B] illustrates an exemplary schematic representation of the final partial product addition, significand selection, and base 8 conversion blocks.

[Figure 5C] illustrates an exemplary schematic representation of an exception processing block.

[Figure 6] illustrates the high-level hierarchical block diagram of the carry-save accumulation unit.

[Figure 7A] illustrates a high-level hierarchical block diagram of an accumulator that includes two hierarchical blocks: an exponent control unit and a significand unit.

[Figure 7B] illustrates an exemplary hierarchical block diagram and schematic diagram of an index control unit.

[Figure 7C] illustrates an exemplary hierarchical block diagram and schematic diagram of significant figure units.

[Figure 8A] illustrates an exemplary hierarchical block showing a normalized conversion to signed numeric format block containing two sub-blocks, a first conversion from carry-preserving to signed numeric sub-block and a second conversion from base 8 to base 2 floating point sub-block.

[Figure 8B] illustrates an exemplary schematic diagram of the conversion from carry-save to signed-valued blocks.

[Figure 8C] illustrates an exemplary schematic diagram of the conversion of a floating point number block from base 8 to base 2.

[Figure 9A] illustrates an example hierarchical block showing rounding and conversion to BF16 or IEEE 754 32-bit single precision format subblocks as well as exponentiation and exception handling subblocks.

[Figure 9B] illustrates an example schematic diagram showing rounding and conversion to BF16 or IEEE 754 32-bit SP format blocks.

[Figure 9C] illustrates an exemplary schematic diagram showing the index and exception handling blocks.

[Figure 10] illustrates the range of floating-point numbers handled by exception handling in the carry-save-accumulate unit used for machine learning.

[Figure 11] shows a high-level block diagram depicting the exception processing elements in the carry-save-accumulate unit used for machine learning.

[Figure 12A] illustrates a high-level block diagram architecture of the first operation mode including input A in BF16 format, input B in BF16 format, and input C in FP32 format, where BF16 specifies a 16-bit machine learning floating point encoding format called "B-float", or Google-developed (Brain Floating Point), and FP32 specifies a 32-bit single-precision IEEE 754 standard representation.

[Figure 12B] illustrates the high-level block diagram architecture of the second operation mode including input A in BF16 format, input B in BF16 format, and performing accumulation.

[Figure 12C] illustrates the high-level block diagram architecture of the third operation mode including input A in FP32 format and input C in FP32 format.

[Figure 13] illustrates a high-level block diagram of the exception handling structure.

[Figure 14A] depicts the multiplier overflow flag condition circuit.

[Figure 14B] shows the multiplier under-bit flag condition circuit.

[Figure 15] illustrates the multiplier invalid flag condition circuit.

[Figure 16] depicts the circuit for generating the multiplication symbol.

[Figure 17A] shows the multiplication exponent generation circuit.

[Figure 17B] depicts the multiplication decimal generation circuit.

[Figure 18A] illustrates the adder overflow flag condition circuit.

[Figure 18B] shows the adder under-bit flag condition circuit.

[Figure 19A] shows the adder invalid flag condition circuit.

[Figure 19B] depicts the adder sign positive condition circuit.

[Figure 20A] depicts the adder sign-negative circuit.

[Figure 20B] illustrates the circuit for the adder exponent generating all "0" conditions.

[Figure 21A] illustrates the circuit for the adder exponent to generate all "1" conditions.

[Figure 21B] shows the circuit for generating addition decimals.

[Content of invention] and [implementation method]

A detailed description of techniques for implementing an arithmetic unit with a configurable and reconfigurable dataflow architecture with exception processing is provided. An example reconfigurable dataflow architecture is described in U.S. Patent No. 10,831,507, issued November 10, 2020, to Shah et al., which is incorporated by reference as if fully set forth herein. The arithmetic unit can perform a plurality of floating-point arithmetic operations using input operands and generate at least one output operand, wherein the source of the input operands, the destination of the output operands, and the operation are configurable and reconfigurable via configuration data that can remain static during the dataflow operation.

During the execution of at least one floating-point arithmetic operation, exceptions associated with illegal operations and with the generation of results that are not normally representable by the used floating-point encoding format are detected, and the result of the operation is set to a value that is available for further processing during the operation without requiring special interrupt handling by, for example, a runtime processor. As a result, the dataflow operation can be completed without being interrupted by at least some of the exceptions.

In some embodiments, arithmetic operations and arithmetic units used on a control flow architecture can implement the exception handling techniques described herein.

Floating Point Carry-Save MAC (FP-CS-MAC)

Describes an FP-CS-MAC that can operate in three modes, such as: Input A (BF16) x Input B (BF16) + Accumulation loop Input A (BF16) x Input B (BF16) + Input C (FP32) or a single 32-bit floating point addition, such as: Input A (FP32) + Input C (FP32) Operand A can be in any format, and in this implementation, it is one of the following two formats: BF16 or FP32, where BF16 is a format containing an 8-bit exponent, 1 sign bit, and 7 bits of significand, including 1 integer bit implicitly, for a total of 8 significand bits. FP32 is called single precision 32-bit, IEEE floating point 754 standard.

Other encoding formats may be used and the described implementation may be adapted appropriately.

A tri-mode floating point carry-save MAC (FP-CS-MAC) unit is described, including circuitry implemented as a pipeline, operating in response to a pipeline clock. In some implementations, the pipeline clock can be on the order of gigahertz (GHz) or faster. When the pipeline clock is operating, each cycle of the clock corresponds to one pipeline cycle. Thus, in some embodiments, the pipeline cycle can be less than one nanosecond. In a pipeline, a pipeline stage includes an input register or data storage that holds stage input data at a first pipeline clock pulse (e.g., the leading edge of the clock pulse), and an output register or data storage that outputs data at a register stage at a next pipeline clock pulse (e.g., the leading edge of the next clock pulse, defining a pipeline clock cycle). When the first pipeline clock pulse starts a pipeline cycle (i), the output register of the stage saves the stage output data of the previous pipeline cycle (i-1), and the stage output data of one stage in the pipeline is at least part of the stage input data of the next stage. The circuit of each stage must be reliably stabilized within the pipeline cycle, so the fast pipeline clock poses great difficulties to the timing-critical stage.

One implementation of a tri-mode floating-point carry-save MAC (FP-CS-MAC) unit contains six pipeline stages. Further speed improvements can be achieved by increasing the number of pipeline stages. Further power reductions can be achieved by reducing the number of pipeline stages. In general, the optimal number of pipeline stages depends on the specific technology and design requirements. The first major unit is the BF16 multiplier, which in this case is implemented in two pipeline stages and includes a conversion unit for converting the multiplier result to a 16-bit 2's complement significand and exponent. The third pipeline stage is the carry-save accumulate stage. The next two stages convert the result in carry-sum format back to a traditional normalized signed numeric format, such as BF16 or FP32, as required by the output encoding format.

|a|<2之間，因為它們在十進制點左側包含隱含的1，並且僅包括有效數的小數部分。所述單元不支援非正規化數字並將它們截斷為零。因此，使用BF16或FP32，輸入運算元的範圍為±2-126到(2-2-7)×2127。如果小於±2-126，則超出此範圍的數字截斷為零，或者如果大於±(2-2-7)×2127，則轉換為±無窮大。 The last pipeline stage performs normalization and rounding to produce the result. In this case, the final format is BF16 or FP32 format. The input operands have a valid value between 1

|a|<2, since they include an implicit 1 to the left of the decimal point and include only the fractional part of the significand. The unit does not support denormalized numbers and truncates them toward zero. Therefore, with BF16 or FP32, the range of input operands is ±2-126 to (2-2-7)×2127. Numbers outside this range are truncated to zero if less than ±2-126, or converted to ±infinity if greater than ±(2-2-7)×2127.

Floating point encoding format

FIG. 1 illustrates the bit patterns of the two encoding formats. The first example diagram of the first bit format illustrates Bfloat16 110. The Bfloat16 floating point encoding format (sometimes "BF16") is a 16-bit numeric format. BF16 preserves the approximate dynamic range of IEEE single-precision numbers. The illustrated BF16 format includes a 7-bit fraction, an "implicit bit" or "hidden bit" to complete the significand, an 8-bit exponent, and a sign bit.

The second figure illustrates the IEEE 754 single precision 32-bit floating point (FP32) 130 encoding format. The illustrated IEEE 754 single precision 32-bit floating point 130 includes a 23-bit fraction, "implicit bits" or "hidden bits" to complete the significand, an 8-bit exponent and a sign bit. The two encoding formats are characterized in that numbers in the FP32 format can be converted to the BF16 format by discarding the 16 less significant bits of the 23-bit portion, and in some embodiments rounding is performed to select the lower-order bits.

System Block Diagram

FIG. 2 is a high-level block diagram of a floating-point multiply-add-accumulate unit with carry-save accumulators in BF16 and FP32 formats. Operand A 213 is illustrated as being in BF16 format or FP32 format 217. Operand B 214 is in BF16 format and is the first input to multiplier circuit 202. The second input is BF16 operand A 213. When both operand A and operand B are in BF16 format, operand A and operand B can occupy a single 32-bit register, each using 16 bits, representing the multiplier and multiplicand inputs to the multiplier. The product (A*B) output of multiplier circuit 210 is generated in carry-sum form on line 218, which is the input to the final adder in block 220. Block 220 also converts the result to 2's complement form and includes base-8 converter circuitry to support base-8 operations.

When the pipeline operates on a single 32-bit addition, (one operand) operand A can bypass multiplier circuit 202, and the second operand C used for the addition comes from line 216.

In this example, operand C 216 is a 32-bit operand that is input to base-8 converter 215, which outputs the result on line 219 to the first input of one of multiplexers 210 and 211. The second inputs to multiplexers 210 and 211 are two buses for the carry and sum value C/S-ACC (and the exponent not shown) on lines 224 and 226 fed back from the output of accumulator 240. Multiplexers 211 and 212 output the exponent and significand as two values on bus 223.

Carry-save adder 230 receives the output of block 220 on line 221 and the outputs of multiplexers 211, 212 on bus 223. Carry-save adder 230 outputs the exponent of the sum and the C/S value on dual bus 222 which enter accumulator 240. Accumulator 240 provides the C/S-ACC index and significand in carry-save form on output buses 224 and 225, which are fed back to multiplexers 211, 212, and provide the C/S-ACC index and significand in carry-save form on bus 226 to carry-save to sign value conversion block 250, which performs the final addition of the carry and sum of the significand on bus 226 and converts the resulting significand to sign value format on bus 227. Buses 252 and 251 pass data from accumulator 240 to carry-save to sign value conversion block 250.

Base 8 to base 2 conversion and normalization block 260 has input on bus 227 and outputs the normalized result on bus 228 for post-normalization, rounding, and conversion to FP32 or BF16 block 270, which converts the output to FP32 or BF16 format on bus 229. The operation outputs the result "Z" on bus 229 in either 32-bit FP32 format or 16-bit BF16 format.

Thus, FIG. 2 illustrates an example circuit that may be implemented as a multi-stage pipeline configured to execute in three modes, including multiplication and accumulation operations for a sequence of input floating-point operands. In this example, the circuit may be configured as a pipeline including a first stage including floating point multipliers with sum and carry outputs, a second stage including a multiplier output adder for the sum and carry outputs of the multipliers and a circuit for converting the multiplier adder output to a base-8 format with a 2's complement significand, a third stage including a significand circuit and an exponent circuit for an accumulator adder, a fourth stage for converting the accumulator sign bit, the accumulator exponent, and the accumulator significand and carry values to a signed valued significand format, a fifth stage for converting the signed valued significand format from base-8 aligned to base-2 aligned and producing a normalized exponent and significand, and a sixth stage for performing rounding and conversion to a standard floating point representation.

The technology described herein provides a multiply-accumulate method to calculate a sum S(i) of terms A(i)*B(i), where (i) ranges from 0 to N-1 and N is the number of terms in the sum. The method may include receiving a sequence of operands A(i) and B(i) in a floating point encoding format, where (i) ranges from 0 to N-1; multiplying the operands A(i) and B(i) to produce a term A(i)*B(i) in a format including a multiplier output exponent and a multiplier output significand, and converting the multiplier output significand to a 2's complement format; using a carry-save adder to add the term A(i)*B(i) to the sum S(i). )*B(i) in two's complement format is added to the significand of the sum S(i-1), and a sum and carry value is generated for the sum S(i); an exponent of the sum S(i) is selected from the multiplier output exponent of A(i)*B(i) and the exponent of the sum S(i-1) to generate an exponent of the sum S(i); and the sum and carry value and the exponent of the sum S(i) are converted to a normalized floating point encoding format.

In addition, the method may include providing the multiplier output exponent and the multiplier output significand of the term A(i)*B(i) in base 8 format, and generating the sum and carry value and the exponent of the sum S(i) in base 8 format before converting to a normalized floating point encoding format that may be base 2.

The alignment required in the accumulation addition stage depends on many conditions, including the overflow of the significand of the sum S(i-1), the sign extension of the sum S(i-1), and the difference between the addends: the term A(i)*B(i) and the exponent of the sum S(i-1). These conditions can be determined and combined for alignment in the same pipeline cycle (e.g., the third stage in the six-stage example), thereby achieving fast execution and faster pipeline clock. In an embodiment provided herein, the unit executes a method for calculating a sum S(i) of terms A(i)*B(i), where (i) ranges from 0 to N-1, and N is the number of terms in the sum, the method comprising: receiving a sequence of operators A(i) and B(i) in floating point encoding format, where (i) ranges from 0 to N-1; multiplying the operators A(i) and B(i) to generate the term A(i)*B(i) during a first pipeline cycle, wherein the format includes a multiplier output exponent of the term A(i)*B(i) and a multiplier output significand of the term A(i)*B(i); and performing the multiplication of the terms A(i)*B(i) during the first pipeline cycle. ) and the accumulator output index of the sum S(i-1) to generate a comparison signal for the sum S(i); adding the term A(i)*B(i) to the sum S(i-1) to generate the sum S(i) during the next pipeline cycle, the format of which includes the accumulator output index of the sum S(i) and the accumulator output valid number of the sum S(i), wherein the addition includes determining the accumulator output index of the sum S(i), and shifting one or both of the accumulator output valid number of the sum S(i-1) and the multiplier output valid number of the term A(i)*B(i) as the result of the comparison signal of the sum S(i).

The step of comparing the multiplier output index of the term A(i)*B(i) with the accumulator output index of the sum S(i-1) during the first pipeline cycle is performed to generate a comparison signal for the sum S(i), while the adjustment of the operands is performed in the next pipeline cycle (early index comparison) to enable the use of a pipeline with an accumulator stage having a shorter critical timing path and operating at a higher clock speed.

Floating point multiplier

m<2之間，即大於或等於1且小於2。因此，兩個運算元有效數的乘積在1

p<4的範圍內，並且永遠不會等於或大於4。 A floating-point multiplier consists of an exponent circuit and a significand circuit. The exponent section performs addition of the operand exponents, while the significand section performs binary multiplication of the operand significands. The operands entering the multiplier are "normalized" floating-point numbers, where the first bit is 1. Therefore, the operand significand (m) is between 1 and 2.

m<2, that is, greater than or equal to 1 and less than 2. Therefore, the product of the effective numbers of the two operands is 1.

p<4 and will never be equal to or greater than 4.

p<4的範圍內，則指數將遞增，而有效數向右移位一個二進制位置以進行正規化。 If the product p of the result of the multiplication of the significands is in 2

In the range p < 4, the exponent is incremented and the significand is shifted right by one binary position to normalize.

The first pipeline stage performs addition of exponents and multiplication of operand significands using 8x8-bit integer multipliers, including carry-save adders for partial products. After summing all partial products using carry-save adders, the result of the multiplier array can include two parts: the 8-bit sum of the carry-save adders and the 9-bit carry of the partial products in the most significant part of the multiplier array, and the 8-bit product of the least significant part of the multiplier array. In this example, the 8-bit partial products in the least significant part are summed using ripple carry adders because these bits come from the partial product reduction tree. This summation can be done using a ripple-carry adder because the time arrival distribution of the least significant portion from the multiplier is that portion that arrives in time from the least significant bit (LSB) to the most significant bit (MSB) is sufficient for a ripple-carry adder. Applying a ripple-carry adder (RCA) significantly reduces the complexity of the multiplier (Figure 4a).

The stage includes a multiplier circuit that provides a multiplier significand and a multiplier exponent value prior to the pipeline clock in response to first and second input operands that are temporarily stored on the pipeline clock. The multiplier circuit includes a significand multiplier circuit and an exponent adder circuit, the significand multiplier circuit having a carry-save adder for generating partial products of carry and sum values to generate high-order bits of a multiplier output significand, and a ripple-carry adder for generating partial products of the significand carry and sum value output low-order bits. In addition, the multiplier circuit includes a base-8 conversion circuit for converting the multiplier significand and the multiplier exponent value into a base-8 format of the multiplier output exponent and significand; and a two's complement conversion circuit for converting the multiplier significand value into a two's complement representation of the multiplier output significand.

The exponents are added separately. Both exponents are positive numbers greater than zero. When the addition result is a number greater than 256, the indication is the carry-out signal from the exponent adder. If the result exponent is equal to 255, a positive infinite indication is determined. If the exponent is equal to 0, the significand is set to zero according to the IEEE 754 standard rules. In this implementation, if the exponent of the product is 0, the significand of the result is forced to 0, thus representing a +/- zero floating point number (Figure 4b). In other embodiments, subnormal numbers may be treated differently.

Adding the exponent requires subtracting 127 from the result because both operands contain a 127 bias in the BF16 and FP32 encoding formats. Adding 129 to the result speeds up the conversion process, which is accomplished by inverting the MSB of the exponent at one of the inputs and introducing a 1 into the carry input of the adder. This greatly simplifies the circuit and can reduce the time required for pipeline stages (Figure 4b).

We prove the correctness of this procedure in the following way: The addition results in the addition of two biases of 127, resulting in a bias of 254. However, since the carry-out of the adder (i.e., 256) is ignored, the resulting bias will be -2. We can get 127 by adding 129 to the result of the operation. This is done by inverting the MSB of the operand, which in the case of a negative operand is equivalent to adding 128, since the MSB position contains a zero. In the case of a positive operand with an MSB equal to 1, this is also equivalent to adding 128. The extra 1 at the carry-in biases the result: -2+129, which is equal to the desired bias of 127.

The same pipeline stage converts the result to a base 8 number consisting of a 5-bit exponent and the significand shifted right by 7 positions as appropriate. Conversion to a 5-bit exponent requires a left shift from the 7th position for the value of the remaining 3 exponent bits to represent the quantity. This requires the significand to pass through a left shifter which will shift the significand left from bit 0 to the 7th position as required by the 3-LSB bits of the 8-bit exponent. (Figure 5b)

The multiplier saves computation time by recognizing that the arrival distribution of signals originating from the partial product reduction tree (PPRT) is uneven. The LSB bit arrives first, followed by the next bit, and so on for the first 8 least significant bits (LSBs) of the PPRT. Due to the unequal arrival distribution, the addition of the LSB portion can be masked ("hidden") under the delay of the multiplier array, thereby providing savings (in time) to the pipeline stage (e.g., the second pipeline stage in the example outlined above). Summing the LSB portion uses an 8-bit ripple carry adder (RCA) to reduce the size of the carry-propagate adder (CPA) using the carry-save adder for the partial product from 17 bits to 9 bits. The MSB portion for the next pipeline stage, including the final adder, is only 9 bits long. The significand of the product is formed in the pipeline stage by adding the most significant 9 bits from the final adder and extending it with the least significant 8 bits previously formed using the ripple-carry adder from the previous pipeline stage (Figure 4a).

FIG. 3 is a simplified block diagram 300 of a multiplier circuit 202 having two inputs (operand A on line 213 and operator B on line 214). The multiplier circuit 202 includes two blocks (multiplier and adder block 210a and exponent block 210b).

FIG4a illustrates an example of a multiplier and adder block 210a showing an 8x8 multiplier partial product reduction tree with carry-save adders for partial products of higher-significant bits without a final 16-bit adder (provided in the next stage) with a 7-LSB ripple carry adder block for partial product addition of lower-significant bits. Operand A 213 is stored in register 420 which includes three fields: Sa, Ea, and Fa. Sa is the sign bit. Ea is the eight exponent bits, and Fa is the fractional portion of the significand. The Fa field is applied to the first input of the 8X8 BF16 multiplier circuit 410 on line 422. Operand B 214 is stored in register 421, which contains three fields: Sb, Eb, and Fb. Sb is the sign bit. Eb is the eight exponent bits, and Fb is the fractional portion of the significand. The Fb field is applied to the second input of the 8X8 BF16 multiplier circuit 410 on line 423. The input to the multiplier circuit 410 on line 440 is the force zero bit, which, when zero, forces the 8X8 BF16 multiplier circuit to produce a zero output.

The 8X8 BF16 multiplier circuit 410 outputs two 7-bit LSB busses 428 and 429, which are inputs to the 7-bit ripple-carry adder 430. In addition, the 8X8 BF16 multiplier circuit 410 outputs 8 sum bits S8 426 and 9 carry bits C9 427. On line 424, the 7-bit ripple-carry adder 430 outputs 7 bits, and on line 425, the carry output bit COUT is output to the register 450. The register 450 has the following mapping: line 424 maps to PL[6:0], COUT on line 425 maps to C7, S8 on line 426 maps to Sp[14:7], and C9 on line 427 maps to Cp[14:6].

FIG. 4b illustrates an example exponent unit (e.g., 210b of FIG. 3 ) with a special exponent detection block 467. Operand A 213 as in FIG. 4a is in register 420, and operand B as in FIG. 4a is in register 421. Ea on line 465 is an input to the special exponent detection block and exponent adder circuit 464. Eb on line 462 is a second input to the special exponent detection block. The 7 least significant digits of Eb on line 462 are input to the exponent adder circuit 464, and the 8th bit is inverted by inverter 461 before entering the exponent adder circuit 464 at the 8th bit position. The carry value of the exponent adder circuit 464 is set to "1".

Exponent adder circuit 464 operates on Ea 465 and Eb 462, adding them together and subtracting the bias value of 127. The output is a 10-bit value 466 to register 470. Two extra bits, beyond the 8 bits required to encode the exponent, are used to detect exponent overflow conditions. These 10 bits are further checked in exponent exception handling circuit 524, as shown in Figure 5C.

The input index signal is checked in special index detection block 467 to be zero, as indicated by the signal on line 468, or invalid, as indicated by the signal on line 469. The sign bits Sa and Sb from registers 420 and 421 are input to anti-mutex OR gate 471a, whose output is applied to anti-mutex OR gate 471b. In addition, the invalid signal on line 469 is input to anti-mutex OR gate 471c. If the invalid signal is zero, the result sign is the exclusive OR function of Sa and Sb. If Invalid is true (equal to "1"), the product sign Sp is set to "zero" as specified in the coding standard.

Base 8 conversion

FIG5A is a simplified diagram showing a base-8 converter block 592 (e.g., block 220 of FIG2 ). The base-8 converter block 592 includes two sub-blocks, in this example, a final addition significand selection and base-8 conversion sub-block 592a and an exponent exception handling sub-block 592b.

Convert to base 8, 2's complement significand

The external input operand A is converted to base 8 encoding in the second pipeline stage. The operand A significand is converted to a 2's complement significand. The significand is expanded to 34 bits, including two significand sign bits. As shown in FIG. 5b, the resulting pipeline register 520 contains a 5-bit exponent, a 34-bit significand, and two additional status bits, for a total of 41 bits.

The conversion to base 8 is performed using the last 3 bits of the exponent to align the 24-bit operand significand from register 450 of FIG. 4a to a 32-bit base 8 significand, where the LSB of the significand is aligned with the LSB of the 32-bit significand if the 3-LSB of the exponent is equal to 0 (shifted 8 positions right from the binary point). Any value represented by the 3-LSB of the exponent is the amount by which the significand is shifted left (starting at the 8th bit position) to compensate for those bits truncated from the exponent. The remaining bits up to the binary point, and any two bits beyond, are filled with sign-extension bits. In the case where all three exponent LSBs are b'1, that is, equal to decimal 7, the first significand bit of the 32-bit significand will be nonzero, that is, the normalized significand. Since the significand is represented as a 2's complement, the two extra bits to the left of the significand point are used to store the sign bit (including the extended sign bit). The extra second sign bit is used, rather than one, in order to preserve the sign because a possible overflow condition would cause the 2-bit integer to overwrite the lower sign bit (Figure 5b).

Depending on the sign of the product, the significand is either traversed or inverted to create a 2's complement negative representation of the significand. This implementation differs from IEEE 754, where the significand can be either positive or negative. The operation is performed by adding a sign bit to the 24-bit significand and inverting the bits if the sign equals 1 (negative).

The exponent is checked between -126 and 126. If it is greater than 126, it is considered infinite; if it is less than -126, it is considered a denormalized number (less than -126) and converted to zero (Figure 5c).

In some implementations, the final register at this stage of the pipeline contains the normalized floating-point product with a 5-bit exponent and a 34-bit 2's complement significand (with repeated signs of the product and no implicit 1s) and three exponent status bits.

FIG5B illustrates an exemplary schematic diagram of the final partial product addition, significand selection, and base 8 conversion subblock 592a. Register 450 (FIG4a) includes fields for PL[6:0], C7, Sp[14:7], and Cp[14:6]. Register 470 (FIG4b) includes fields for a 10-bit product exponent (Ep) value. Register 504 includes status bits.

In the case where the pipeline is running in FP32 addition mode, operand A is in FP32 format and bypasses the multiplier. In this case, operand A is sourced from register 460 and occupies two combined 16-bit registers 420 and 421. The add_op control signal on line 511 indicates when the pipeline mode is set to addition (single precision floating point in this case) or accumulation.

Significand final adder circuit 502 receives Sp[14:7] on line 503, Cp[14:6] on line 501, and carry bit C7 on line 507 as inputs, and outputs overflow signal 519 to overflow select circuit 506. Overflow select circuit 506 has input bus 523, which is a combination of PL[6:0] on line 509 and the significand final adder circuit 502 output on line 521. NOR gate 522 has input exponent overflow bit on line 525 and zero force bit on line 468 and outputs signal on line 527. Overflow select circuit 506 outputs signals on line 527 and bus 529 that are routed to AND gate 544, which sets the significand to all zeros in the event of exponent overflow and in the event that the significand is forced to zero. Additionally, significand select circuit 512 uses the add_op control signal on line 511 to select between the bypassed significand Fa[22:0] on bus 515 or the AND gate 544 output on bus 553.

The index select circuit 510 selects between the 8 exponent bits, Ep[7:0] on line 517, or the bypassed exponent bits Ea[30:23] on line 513, and outputs the selected exponent on line 533 to the E_mult field of register 520. The sign bit select circuit 508 receives the Sp sign bit on line 473 (FIG. 4b) and the bypassed sign bit Sa as inputs, and outputs the sign bit 531 to the S_mult field in register 520.

The add_op control signal on line 511 is routed to the significand select circuit 512, the exponent select circuit 510, and the sign bit select circuit 508 as their control inputs.

The output of the significand select circuit 512 enters the 8-bit left shifter circuit 514. The lower three bits [2:0] of line 533 from the exponent select circuit 510 are output on line 533 to control the 8-bit left shifter circuit 514. The output bus 537 of the 8-bit left shifter circuit 514 feeds the multiplexer circuit 518, which selects between the input on line 537 (if the significand is positive) and the input on line 539 (if the significand is negative). This is selected by the sign bit 531. The 2's complement invert and add 1 circuit 516 creates the 2's complement of the shifter output on line 537 and outputs the complement value on line 539. The output of multiplexer circuit 518 on line 541 enters pipeline register 520 as a 34-bit F_mult significand. The program converts the selected significand into a 2's complement significand, which is 32 bits long with 2 sign bits and is stored in pipeline register 520.

FIG5C illustrates a block diagram of the exponent exception handling sub-block 592 b. As described with reference to FIG5 b, the significand final adder circuit 502 receives inputs Sp[14:7] 503, Cp[14:6] 501, and carry bit C7 507 from register 450. The overflow output of the significand final adder circuit 502 is connected to the exponent exception handling circuit 524. When an overflow condition is detected, the significand final adder circuit 502 asserts the overflow signal 519 as the first input of the exponent exception handling circuit 524. The second input of the exponent exception handling circuit 524 is the exponent bit Ep[9:0] (the sum of the input operand exponents) from register 470 on bus 517. The third input is the output signal of the exponent overflow detection circuit 522 on line 523. The output of the exponent exception handling circuit 524 is then input to the exponent exception detection circuit 526 and the exponent selection circuit 510 on line 549 (see Figure 5b for description).

The exponent bits Ep[9:0] on bus 517 are input to the exponent overflow detection circuit 522, which detects the overflow condition: exp_ovf=Ec[8]: if bit 8 is 1, exponent overflow is detected, exp_povf=~Ec[9]&Ec[8]: if bit 9 is 0 and bit 8 is 1; positive overflow, exp_novf=Ec[9]&Ec[8]: if both bit 9 and bit 8 are 1; negative overflow.

The first output on line 523 of circuit 522 is routed to exponent exception handling circuit 524, the second output on line 543 is routed to output exception control signal generating circuit 528, and the third output includes the exponent overflow bit on line 525 to gate 522 in FIG. 5B.

The index abnormality detection circuit 526 outputs the following three-bit abnormality to the register 532 through the bus 547: of (overflow); uf (underflow); and nv (invalid).

This occurs when the following conditions are detected:

of(overflow) - means if Ec is 11111111 and infinity is not detected, it is interpreted as overflow.

uf (under-bit) - means if Ec is 00000000 and zero (significant number) is not signaled, it is an under-bit condition.

nv (invalid) "1" - means the result is invalid.

The output exception control signal generating circuit 528 has four inputs. The first input is the add_op control signal on line 511, which indicates the accumulation or bypass addition mode, the second input is the status bit on line 509 (infinite, zero or invalid), the third input on line 545 is routed from the index selection circuit 510 multiplexed between the Ea[30:23] bits of register 460 or the output of the exponent exception handling circuit 524, and the fourth input comes from the second output on line 543 of the exponent overflow detection circuit 522. The output exception control signal generating circuit 528 outputs five bits on line 551, representing exp_mul_zero, exp_mul_inf, exp_zero_en, exp_inf_en, and f_zero_en stored in register 530.

exp_mul_zero means: the multiplier product exponent is zero, exp_mul_inf means: the multiplier product exponent is infinite, exp_zero_en means: enabled when (one of the multiplier input exponents is zero, and both multiplier input exponents are not zero) or the multiplier product exponent has a negative overflow, exp_inf_en means: enabled when one of the multiplier input exponents is infinite or the multiplier product exponent has a positive overflow, f_zero_en means: enabled when the exp_zero_en signal is enabled or the multiplier product exponent overflows (positive or negative) or when the multiplier product exponent is zero.

Carry-save Accumulation Unit

FIG6 illustrates a block diagram 600 of a carry-save accumulator (e.g., 240 of FIG2) for a significand. The base-8 converter 215 receives operand C as input and outputs operand C in base-8 format on line 219 to multiplexers 210 and multiplexer 211. Two additional inputs to multiplexers 210 and 211 are buses 224 and 225 fed back from accumulator sum register 242 and accumulator carry register 241. The outputs of multiplexers 210 and multiplexer 211 are routed to shifter circuits 609 and 610, which perform right shifts of 8/16/24 bits or left shifts of 8 bits. The outputs of shifter circuits 609 and 610 are routed to carry-save adder circuit (CSA) 614. Carry-save adder circuit 614 has a third input from the right shift circuit of 8/16/24 circuit 608, whose input is the product 602 of the A*B (BF16) or A (FP32) operands alone. The output of carry-save adder circuit 614 on lines 667 and 669 is routed to LZA circuit 606 which provides an output to S-bit register 636, and to overflow detection block 605 which provides an output to O-bit register 634.

The carry-save accumulation unit includes a significand circuit that receives the multiplier output significand of item A(i)*B(i) and the feedback sum and carry value of the previous accumulator output representing the sum value S(i-1) at the first pipeline clock of cycle (i). The significand circuit includes a two's complement, carry-save adder to generate the sum and carry accumulator output significand value for the sum S(i) at the second pipeline clock. The carry-save accumulation unit includes an index circuit that receives the multiplier output index of item A(i)*B(i) and the feedback index value of the previous accumulator output representing the sum value S(i-1) at the first pipeline clock to generate the accumulator output index value for the sum value S(i) at the second pipeline clock. The significand circuit includes a significand shifter responsive to an exponent compare signal stored at the first pipeline clock to align the multiplier output significand with the feedback sum and carry value for addition. The index circuit is responsive to the exponent compare signal stored at the first pipeline clock to generate an accumulator output exponent value. The pipeline includes an index compare circuit to compare the multiplier output exponent of term A(i)*B(i) with the feedback exponent value of the sum S(i-1) before the first pipeline clock to generate an exponent compare signal stored at the first pipeline clock.

The carry-save accumulation unit in this embodiment includes an overflow detector circuit to generate a first condition signal indicating an overflow condition for at least one of the sum and carry values fed back at a first pipeline clock, and a leading sign bit detector circuit to generate a second condition signal indicating that at least one of the sum and carry values fed back has an extended sign bit greater than or equal to the number 8 at the first pipeline clock. The exponent circuit and the significand circuit are also responsive to the first condition signal and the second condition signal. Overflow and leading sign bit adjustments and exponent comparison adjustments are combined to be implemented by the shifter in the same pipeline cycle, as described in reference to Table 1: CSA Unit Control below.

In addition, this stage of the pipeline has an accumulator mode and a summing mode, and includes a selector to provide the accumulator output of the feedback in the accumulator mode and to provide the third floating point input operand to the significand circuit and the index circuit in the summing mode. The significand circuit may include a significand shifter, responsive to an exponent compare signal stored at the first pipeline clock, to align the multiplier output significand and the sum and carry values of the feedback for addition in the accumulator mode, and to align the multiplier output significand and the significand of the third input operand for addition in the summing mode. The index circuit is responsive to the exponent compare signal stored at the first pipeline clock to generate the accumulator output exponent value. The pipeline includes an index comparison circuit to compare the multiplier output index with the fed-back index value before the first pipeline clock in the accumulator mode, and to compare the multiplier output index with the index of the third input operand in the summing mode to generate an index comparison signal stored in the first pipeline clock.

Valid digital circuit:

The carry-save adder (CSA) significand stage has two paths: the accumulator path, where the operands from the accumulator can be shifted right by 8, 16, or 24 bits and can be shifted left by 8 bits, and the multiplier path, where the operands from the multiplier can be shifted right by 8, 16, or 24 bits. When using base-8 exponents, a right shift of 8, 16, or 24 bits corresponds to an exponent difference of 1, 2, or 3 between the operands. An 8-bit left shift is done when the carry-save adder outputs a number that sign-extends more than 8 bits.

If the difference between the operand indices is greater than 3, it means that one of the operands has been shifted right by more than 24 bits, which aligns the operand too far to the right to fit within the larger operand. This situation is equivalent to adding zeros to the larger operand, or using a bypass multiplexer to simply pass the larger operand to the accumulator unchanged (Figure 7c).

This implementation eliminates the bypass multiplexer by summing the CSA with zero when the exponent difference is greater than 3, effectively bypassing the operands. The inputs to the CSA come from the multiplier and accumulator and are gated by AND gates. The shifter and exponent control unit detect this condition and set the appropriate operand to zero. This implementation saves one multiplexer stage in each path.

Detection of sign extension occurs after the 3:2 carry-save adder stage. If this condition is detected, the sign-extend bit S and overflow bit O are set and processed in the subsequent pipeline clock. In order not to lose the sign bit due to overflow, the calculation is carried over with duplicate signs. Additional complexity is introduced to improve accuracy. This involves extending the accumulator to 36 or 40 bits. In another implementation, introducing detection logic improves timing and accuracy. The detection logic obtains the input of the carry-save adder circuit (CSA) 614 from three inputs 683, 685, 689, rather than two outputs of the carry-save adder circuit (CSA) 614, and is the subject of another related disclosure.

Exponential Circuit:

The "Exponent Control Unit" compares the exponent difference between the first exponent operand from the multiplier and the second exponent operand from the accumulator. The Exponent Control Unit checks the condition generated by comparing the multiplier and accumulator exponents and selects the operand path according to Table 1. At the same time, the new accumulator exponent is determined and stored in the exponent accumulator (Eacc) register 654 (Figure 7b).

The exponent section has two branches: the left branch and the right branch. The left branch, consisting of inputs 671 and 673 (entering the OR gate), selects the larger of the two exponents, which then becomes the resultant exponent. This condition is selected according to Table 1. The right branch, consisting of inputs 675 and 677 (entering the exponent output OR gate), will select Ea+1 or Ea-1 according to the condition described in Table 1. If the significand overflow is signaled, the accumulator significand shall be shifted right by 8 bits (SHR_8) and the exponent shall be incremented by 1.

Overflow (O) detection is performed during CS addition. If an overflow is detected, the O bit is latched into the output pipeline register. The overflow condition will be corrected in the next cycle according to Table 1.

CS-Accumulation Implementation

The functions of the exponent path and the significand path are interdependent, depending on the exponent and the states of the "sign extend" (SE) and "overflow" (O) signals generated by the significand. There are two accumulators, one for the carry and the other for the sum. They are added to the product using a 3:2 carry-save adder (CSA) and traverse two separate paths, one for the carry and the other for the sum.

The destination register for the pipeline stage is an accumulator that contains the carry and sum (two registers). The conversion to the legacy format is performed in the following pipeline stages (Pipeline 4 and Pipeline 5). The carry-save stage can be a timing critical stage. Therefore, special attention is paid to the timing and area that guide the design decisions described in this section. The critical paths of the pipeline stage include: exponent control, three 2:1 multiplexers, a 5-bit subtractor, a 5-bit subtractor and compare unit in the exponent section, and exponent control, a 5-bit incrementer, a 3:2 carry-save adder (CSA), and an AND gate in the significand section. The critical path can traverse both the exponent path and the significand path, as is the case in this design.

Accumulator Design

FIG. 7A illustrates a simplified block diagram 610 of the accumulator 240, which includes three circuit blocks: an exponent control unit 240A, an exponent comparator unit 240B, and a significand portion 240C.

*其中： emult：是乘積指數 eaccu：是累加器指數 emmp 含義：emult大3以上 eamp 含義：eaccu大4以上 7B illustrates an exemplary hierarchical block diagram and schematic diagram of the index control unit 240A and the index comparator unit 240B. The shifter index control signal generation/bypass control circuit 630 receives inputs from: accum_ld, exp_zero_en, f_zero_en, e_cin_zero, 551, csa_ovf bits 634 O and Signext, which is the S bit 636 and the output of the 16-bit multiplier index compare circuit 652, which is stored in the 16-bit status register 650, the sixteen exponent compare bits*:

*Among them: emult: is the multiplication index eaccu: is the accumulator index emmp means: emult is greater than 3 eamp means: eaccu is greater than 4

There are other control signals:

accum_ld - Meaning: The accumulator receives the input C value.

exp_zero_en - Meaning: Set the product exponent to zero.

f_zero_en - Meaning: If Exponent=0, set the product significand to 0 (because non-normal is not allowed)

e_cin_zero - Meaning: Input C index equals zero

The outputs of the shifter index control signal generation/bypass control circuit 630 are control signals: accumulator shifter control on line 638, accumulator bypass control on line 636, multiplier bypass control on line 634, and multiplier shifter control on line 632, Ea_sel on line 646, Ea1m_sel on line 642, Em_sel on line 648, and Ea1p_sel on line 645.

Comparison circuit 652 compares the indices from the following two operands: (1) the multiplier index E_mult on line 521 and the accumulator index on line 679; (2) either input A (in bypass mode from the index E_mult on line 521) and the accumulator index on line 679; (3) either input A (in bypass mode from the index E_mult on line 521) and input C (input Ec from line 460). Comparison circuit 652 generates the following status bits which are stored in 16-bit status register 650: emult: multiplier index; eaccu: accumulator index; z_diff-emult is the same as eaccu; mgrt-emult is greater than eaccu; agrt-eaccu is greater than emult; em1p-emult is greater than 1; em2p-emult is greater than 2; em3p-emult is greater than 3; emmp-emult is greater than 3; ea1p-eaccu is greater than 1; ea2p-eaccu is greater than 2; ea3p-eaccu is greater than 3; ea4p-eaccu is greater than 4; eamp-eaccu is greater than 4 or more; emz-emult is zero; eaz-eaccu is zero; eminf-emult is infinitely large; and eainf-eaccu is infinitely large. The 16-bit status register 650 interfaces with the shift index control signal generation/bypass control circuit 630 via bus 621. The 16-bit status register 650 stores the comparison result of Eacc from the sum S(i-1), and during the generation of Eacc for the sum S(i), the E_mult register 520 stores the term A(i)*B(i) in the accumulation mode.

Comparator circuit 652 has an input on line 647 from subtractor circuit 646. Subtractor circuit 646 receives E_mult from pipeline register 520 on line 521 and the output of multiplexer 642, which selects between Ec of register 460 and the new index output on line 679 of OR gate 670, controlled by the accum_en signal on line 665 of multiplexer 642 indicating the mode. (Figure 7b)

Exp_Zero_En on line 618 is applied to inverter 619, whose output is used as an input to AND gate 617. The E_mult exponent bit from pipeline register 520 is also input to AND gate 617, whose output on line 681 is fed to AND gate 668, where the Em_sal bit on line 648 comes from the shifter index control signal generation/bypass control circuit 630 to pass or block E_mult. The output of AND gate 668 on line 671 is connected to a four-input OR gate 670. OR gate 670 has three other inputs, including the output of AND gate 615, which is selected by signal Ea_sel to pass or block Eaccum, and the output of multiplier 660 on line 616 and the output of subtractor 661 on line 663, which are each controlled by outputs Ea1p_sel and Ea1m_sel at AND gates 664 and 665, respectively. Based on the select signals (only one of which can be 1), 648, 646, 645, 642, the appropriate index is selected as the output of OR gate 670. This output is the Eacc signal, also called the new index, which is the input to the exponent accumulator (Eacc) register 654 and also the input to multiplexer 642.

The output of multiplexer 665 ("the new index of sum S(i) or the index of operator C, depending on the mode") is also recorded in register 460 in this embodiment, which is connected on line 644 as input to incrementer 660 and decrementer 661.

Thus, when the new index on line 679 represents sum S(i) using the compare bit generated by sum S(i-1), the new index will be compared with the E-mult value of item A(i-1)*B(i-1) in register 520 to generate a compare signal that will be latched with sum S(i) and used for shifter control during the generation of sum S(i+1).

FIG. 7C is a schematic diagram of the significand portion 240C. The shifter index control signal generation/bypass control circuit 630 is illustrated, which shows four output control signals. The first control signal is the accumulator shifter control on line 638, which is the select signal for the shifter circuits SHR8/16/24/SHL8 609 and 610. The two shifter circuits: SHR8/16/24/SHL8 609 and 610 receive inputs from a set of multiplexers 210 and 211 on lines 682 and 683. The accum_en signal on line 665 controls multiplexers 210 and multiplexer 211 to select between the sum on line 224, the carry on line 225, or the value on line 219, originating from register Fcin 560, or a logical "0" as the other input to multiplexer 211. Multiplexers 210 and multiplexer 211 output the selected value on bus 682 and bus 683 to shifter circuits SHR8/16/24/SHL8 609 and 610. Shifter circuits 609 and 610 output the shifted value on buses 692 and 693. Bus 692 may interface directly to bus 613 or may traverse optional round-up block 604 which adds rounding bits to bus 613. Bus 693 may interface directly to bus 611 or may traverse optional round-up sum block 612 and add rounding bits to 693. Buses 611 and 613 are the inputs to AND gates 687 and 688, whose outputs are used as inputs to carry-save adder circuit 614 on lines 689 and 685 (the AND symbol indicates the diversity of AND gates for each signal line on the bus: 613, 611, and 607. The inputs to AND gates 688 and 686 are selected by control signals on lines 633 and 634, respectively.

The F_mult value in pipeline register 520 is input to simple product round block 684 which is input directly to shift register SHR 8/16/24 circuit 608 on line 603. The select signal for SHR 8/16/24 circuit 608 is the multi-shifter control on line 632 which selects between the F_mult input on line 601 and the rounded product on line 603. The output is the product containing 42 bits (34+8) on bus 607 which is applied to the input of AND gate 686 whose output is the input of carry-save adder circuit 614.

The 42-bit 3:2 carry-preserve adder circuit 614 has three inputs, including output 683 of AND gate 686, output 689 of AND gate 687, and output 685 of AND gate 688. The outputs of the 42-bit 3:2 carry-preserve adder circuit 614 are two buses: sum bus 669 and carry bus 667. The two outputs 669 and 667 enter the 42-bit fractional sum register 242 through bus 669 and enter the 42-bit fractional carry register 241 through bus 667, respectively. Bus 669 and bus 667 are also inputs of overflow detection block 605 and sign extension detection unit 662. The overflow detection block 605 and the sign extension detection unit 662 provide outputs to the O bit 634, namely the csa_ovf signal and the S-bit 636 which is the sign extension signal. The sign extension detection unit 662 has an enable bit accum_en signal on line 665, which is set to a logical "1" when the operation is accumulation. The sign extension detection module can only operate when the "accum_en" signal is enabled.

There are three operation modes to choose from: input A (BF16) x input B (BF16) + input C (FP32), input A (BF16) x input B (BF16) + accumulation loop (sum), input A (FP32) + input C (FP32).

The "accum_en" signal is enabled only during the second mode state (accumulate). In addition mode, sign-extend detection is not required. It is only required in accumulation mode because the gradual increase of the sign-extend bit can only occur during accumulation operation.

Symbol expansion detection unit 662:

According to some aspects, a sign extend detection unit 662 is attached to the accumulator outputs for both the sum and the carry. When a 10-bit sign is detected (including two sign bits, and an additional 8 bits in the first byte of the sum or carry), the output is left-shifted in the next cycle (SHL_8) to maintain accuracy of the operands. Without sign extend detection, during normal operation, the significand bits of the operands are progressively shifted to the right until replaced by the extended sign bit, resulting in a loss of accuracy. In the implementation, an adjustment is performed to shift the operand left by 8 bit positions each time one of the operands detects at least 10 leading sign bits. The exponent is adjusted accordingly by decrementing the exponent value by one, which is performed in the same cycle. When S is detected in the carry or sum portion of the accumulator, the S bit is latched in the output pipeline register 636 for correction in the next cycle. The correction operation shifts the accumulator left by 8 bit positions (SHL_8). Sometimes this may cancel itself on the next action (requiring SHR_8), usually it remains unchanged as shown in Table 1.

Normalization and conversion to symbolic numeric format

FIG8A illustrates the normalization conversion to signed value format block 270, which includes two sub-blocks, the first sub-block is the conversion from carry-save to signed value format block 270a, and the second sub-block is the conversion from base 8 to base 2 floating point block 270b.

8B illustrates an exemplary schematic diagram of the carry-save to sign-value format block 270a. Two registers, the 42-bit fractional sum register 242 and the 42-bit fractional carry register 241 output the shift-carry [42:0] bus 704 and the sign-extend-sum [42:0] bus 702 as inputs to the 43-bit adder circuit 708. The second circuit LZA/LOA 710 receives the input bus 702 and the bus 704. The second circuit LZA/LOA 710 outputs two buses POS_P[5:0] on line 711 and POS_N[5:0] on line 712 to the third LZA POS select circuit 714. The output of LZA POS select circuit 714, for example, via bus 715, is POS[5:0], which is mapped to register 730 as a 6-bit position specifying the amount of left shift required to normalize the significand.

43-bit adder circuit 708 outputs signal SIGN on line 719 to control LZA POS select circuit 714 to route bus 716 to significand select multiplexer circuit 720 on the "0" branch input and to route bus 716 to the negative input: inverting add-one circuit 718. The "1" branch of 2's complement significand select multiplexer circuit 720 receives bus 717, which represents a negative significand, converted to positive one. Sign 719 controls the significand select multiplexer so that output 738 always contains a positive significand. The output of 2's complement select multiplexer circuit 720 is bus 738, which is mapped to register 730 as a 41-bit positive significand. The 5-bit exponent on line 706 maps directly to register 730 and sign bit 726. This step completes the conversion of the accumulator significand represented in carry-save format to signed numeric base-8 format.

At this stage, the two values on the sum bus 702 and the carry bus 704 (representing the significand in carry-save format) are added in the 43-bit adder circuit 708 to produce the sign-value format of the significand. The leading zero/leading one predictor (second LZA/LOA circuit) 710 will calculate two numbers: the number with leading zeros 711 (in the case of the significand 716 being positive) and the number with leading ones 712 (in the case of the significand 716 being negative). Based on the significand sign bit 719, the multiplexer LZA POS select circuit 714 will select the correct position and store it in the register 730. LZ and LO positions POS_P and POS_N are both 6-bit numbers long and are expected to contain 32 leading zeros or ones.

If the significand of the output of the 43-bit adder circuit 708 is negative, then the negative number is converted to a positive number (because IEEE 754 uses signed numerical representation, that is, positive significands). For this purpose, a 2's complement converter 718 is used. The sign bit 719 controls the multiplexer circuit 720 so that if the number is positive, it is directly stored in the 41-bit significand register 730. If the output is negative, the output on line 717, i.e. the value on 716 is converted to a positive value and is passed to register 730 on line 738.

The predicted 6-bit position of the significand will be added to the 5-bit exponent to produce a new 8-bit exponent that conforms to standard floating-point representation, and the significand will be aligned with the floating-point significand, using the same 6-bit predicted position (Figure 8c).

FIG8C illustrates an exemplary schematic diagram of block 270 b for base 8 to base 2 floating point conversion. Register 730 is connected to SHL left shifter circuit 735 via 41-bit bus 731. Register 730 is interfaced to significand zero detection circuit 728 via 41-bit bus 734. The 6-bit position field of register 730 provides Pos[5:0] on line 723 to control SHL left shifter circuit 735. Pos[5:0] on line 723 is also an input to exponent adder circuit 740. The exponent 5-bit field of register 730 is the second input to exponent adder circuit 740 on line 721. Exponent adder circuit 740 adjusts (increases) the exponent by the number of positions indicated by POS[5:0] shifted left by the significand and provides an output on line 736 to register 748. However, since the predictor may be wrong in one position, the output of the shifter is passed to overflow/under detection circuit 752, which will signal an error by issuing signal 739, which is applied to the carry input of exponent adder circuit 740 and at the control input of under-detect multiplexer 760. Underdetect multiplexer 760 has an input on line 746 where the significand on line 742 from shifter circuit 735 is in the same position (no error detected), and an input on line 747 where the significand on line 742 is shifted one bit to the left (an error detected). If signal 739 indicates an underdetection, the correct output is latched into register 770 via bus 745. The error in exponent adjustment is corrected by inputting a 1 to the adder via the carry input. The sign value is copied from register 730 to register 747.

The sign bit in register 730 is transferred to register 747 via line 744.

Exception control (8-bit) register 750 passes its value to exception control register 751 on line 724. The meaning of the exception control register bits is as follows:

exp_inf_en: Operator A is infinite or operator B is infinite

z_diff accumulator index equals product index

s_mult: product sign

s_cin: Input C symbol

e_mul_zero: multiplication exponent = 0

e_cin_zero: Input C index = 0

e_mul_inf: The product exponent is equal to infinity.

e_cin_inf: Input C index equal to infinity.

Bit six 726 of the exception control (8-bit) register 750 is "z_diff", which indicates the result of the exponent comparison between the accumulator and the product exponent. When equal to 1, the accumulator exponent is equal to the product exponent. When "z_diff" = 0, it indicates that the accumulator exponent is less than or equal to the product exponent. Bit [6] "z_diff" is the first input to the significand zero detection circuit 728 on line 726. The significand zero detection circuit 728 outputs a 1-bit signal on line 753, which replaces bit [6] "z_diff", which now becomes "pos_zero" 753 in the exception control register 751, indicating that the result significand is zero. Once the accumulation operation is complete and the operation is normalized, the second output of the significand zero detection circuit 728 provides a signal to the fractional zero register 756 on line 755.

FIG. 9A illustrates block 270 that performs the final conversion to BF16 or IEEE 754 32-bit single precision format of 700, including block 270a that performs rounding and conversion to BF16 or IEEE 754 32-bit single precision format of signed value and sub-block 270b that performs exponent and exception handling.

Round and convert to FP32 format

According to some aspects, the last stage is pipeline 6, which performs rounding of the result to a standard floating point sign/value number with the following: sign bit, 8-bit exponent, 23-bit, normalized significand (+1 implicit integer bit). During the conversion of the 31-bit significand to a 24-bit normalized significand with one implicit bit, the result is rounded from 31 bits to 24 bits. In this implementation, two rounding modes are implemented: round towards zero (RTZ), truncate, and round to nearest even (RNE). However, any other rounding mode, for example, round to nearest odd (RNO) can easily be merged in.

According to some aspects, the rounding logic examines the last 15 LSB bits of the 39 significand bits from register 770 (not counting the GRS bits that make up the total 42 bits (39+3 GRS bits)) and determines whether the remaining 24 bits need to be rounded (according to the rule applied: RNE or RTZ). The incrementer required for RNE is included in the rounding box. The three bits of GRS carried over from the accumulator (CSA) operation are ignored in this implementation. They can be incorporated into the final rounding of other possible implementations.

Rounding is done in one of several ways: (a) during CS accumulation, with round to nearest odd (RNO) applied, - for the sum signal, only if the round bit is inserted with the carry LSB on, - separately for each sum and carry signal, (b) during CS accumulation, and in pipeline 6 stage (final rounding), and (c) only in pipeline 6 stage with CSA disabled.

Each rounding mode is applied according to the accuracy and specific requirements imposed by the specific application.

The output of pipeline 6 is either FP32 or BF16, as required. Therefore, the significand length is either 24 (23 + implicit bit) or 8 bits (7 + implicit bit). This is controlled by the "Out_FP32" signal applied to the first multiplexer. If rounding results in a 25-bit significand, the significand is shifted right one position and the exponent is incremented by 1.

The correctly normalized and rounded result is stored in the output register of pipeline 6 as a BF16 number consisting of a 1-bit sign, an 8-bit exponent, and a 7-bit fractional significand, or a FP32 number consisting of a 1-bit sign, an 8-bit exponent, and a 23-bit fractional significand.

FIG9B illustrates an exemplary schematic diagram showing rounding and conversion to BF16 or IEEE 754 32-bit SP format. 39-bit Significand Register 770 Bus fpst_l[38:0] 837 provides fpst_l[32:0] 819 or fpst_l[16:0] 819 to a rounding circuit 830 including protection bits, rounding bits, and sticky bits. The significand portion to be rounded by the rounding circuit 830 is controlled on Out_FP32 select line 819. In the case where the 32-bit SP format is selected, 3 rounding bits are added to the upper 24 bits [38:16]. Multiplexer 840 selects between a "0" input on line 835 or the rounding circuit 830 output on line 825, where multiplexer 840 is controlled by round to zero select line 823. This occurs when the exponent exceeds -126 and the significand becomes non-normal, causing rounding to zero in this implementation. The output on line 827 routes the 23 bits (one implicit) and 3 rounding bits to the first round increment circuit 860, which properly rounds the significand in IEEE 754 SP 32-bit format on line 819.

When the BF16 output format is selected, the second round increment circuit 850 is used to round to the BF-16 format. The conversion of the 39-bit significand on line 817 from fpst_l[38:0] 817 to the BF-16 output is completed in the round increment circuit 850, resulting in 7 bits (one implicit), an increase of 1 round decision bit, and an additional 16 zeros. This represents the significand at the output on line 821 as one of the inputs to the multiplexer 802.

The first multiplexer 802 uses the control line signal Out_FP32 801 to select FP-32 or BF16 output. When Out_FP32 801 is valid, it outputs a FP-32 formatted significand on line 805. When the Out_FP32 801 control signal is not valid, the output on line 805 is a BF-16 formatted significand. The output of the first multiplexer 802 is bus 805, which is divided into bus 807 and bus 809, and enters the second multiplexer 810. Multiplexer 810 is controlled by the 24th bit of the output bus on line 805, the frnd[23] signal on line 803. As described in [0093], in the case of rounding to produce a 25-bit significand, the 24th bit will be 1. In this case, the frnd[23] bit signal on line 803 selects input bus 807, which shifts bus 805 right by one bit position. (The frnd[23] signal also increments the index of the result by 1 to adjust for the right shift. If frnd[23] is equal to 0, no right shift is required and bus 805 passes directly through line 831 via the selected input bus 809.)

The third multiplexer (zero) 820 selects between all "0"s on line 829 input or fnorm[22:0] on line 831. If the output exponent is infinite or non-normal, the output significand is forced to zero, which is accomplished via the control signal Zero_Sel on line 788, which selects all "0" input 829. If there are no exceptions, the normalized significand bus 831 is routed to bus 833 and mapped to the 23-bit significand (IEEE 754) register 930.

FIG9C illustrates an exemplary schematic diagram showing the index and exception handling block 270 b. The 8-bit index 748 is provided to the EPST_L[9:0] 964 bus and is incremented by 1 if frnd[23]=1. This is accomplished by routing frnd[23] to the carry position of the incrementor 982. The output of the index incrementor 982 is the 9-bit Enorm[8:0] bus 968, which is the first input to the first multiplexer (zero) 974. The second input to the first multiplexer (zero) 974 is the all-zero bus 829. The purpose of the first multiplexer (zero) 974 is to set the index to all "0" in case an exception is required, indicated by the exception control (8-bit) register 750 through the zero control logic 970.

The exception control (8-bit) register 750 operates on the following eight conditions 953: exp_inf_en, pos_zero, s_mult, s_cin, e_mul_zero, e_cin_zero, e_mul_zero, and e_cin_inf.

Zero control logic 970 has three inputs: sign_diff from the exclusive OR gate on line 961 and e_cin_zero on line 959, e_mul_zero from the exception control (8-bit) register 750 on line 957, and output control first multiplexer (zero) 974 on line 972. The output of first multiplexer (zero) 974 on line 975 passes through multiplexer 976 to provide index signal bus 979 stored in index register 980. If infinity control 962 sends an infinity signal on line 963, the all "1" input on line 907 passes through multiplexer 976 to set all index bits to "1" as recommended by the IEEE 754 standard.

The meaning of the signal is: (explained in paragraph 089)

exp_inf_en Operand A is infinite or operand B is infinite

pos_zero The result is effectively zero

s_mult product sign

s_cin inputs C symbol

e_mul_zero multiplication exponent = 0

e_cin_zero Input C index = 0

e_mul_inf The product exponent is equal to infinity.

e_cin_inf Input C exponent equal to infinity.

In addition, the signal "sign_diff" indicates that the sign of the product "s_mult" is different from the sign of the input C "s_cin". The signal is obtained by applying a mutual exclusion or function to the s_mult and s_cin signals obtained from the exception control (8-bit) register 750.

The exception control (8-bit) register 750 provides the following signals on bus 965 to the sign generation and exception handling circuit 988 and the under/overflow detection and exponent exception detection circuit 986: s_cin, s_mult, exp_inf_en, e_cin_inf, e_mul_inf, pos_zero, sign_diff. The control signals for circuit 986 are norm_en 758 on line 969 and fractional zero register 756 on line 971. The outputs of circuit 986 are the three signals ov (overflow), uf (underflow), and nv (invalid). A fourth output on line 991 is sent from the infinity detection circuit 992 to the sign generation and exception handling circuit 988 to indicate overflow.

The signals have the following abbreviations as follows: s_cin (sign of input C), s_mult (sign of product), e_mul_zero (product exponent zero), e_cin_zero (input C exponent zero), e_mul_inf (product exponent infinitely large), and e_cin_inf (input C exponent infinitely large).

Two related events cause underscaling. One is the creation of a tiny non-zero result between ±2-126 [where -126 is the smallest exponent value] which, because it is so small, may later cause some other anomaly, such as overflow on division. The other event is an unusual loss of accuracy during the approximation of such a small number. The loss of accuracy may be detected when the result passed is different from the result calculated if the exponent range and precision were unbounded. IEEE Standard 754 does not track accuracy other than requiring single and double precision. In the disclosed implementation, "denormal" numbers are not used, and any value with an exponent value of -126 and a significand less than 1 is converted to zero. Zero is represented by setting all significands to zero and setting the exponent value to zero, which is handled by the exception handling circuitry in our disclosed implementation.

The symbol generation and exception handling circuit 988 receives input from the exception control (8-bit) register 750 via bus 965 and the infinity detection circuit 992. The output of the symbol generation and exception handling circuit 988 is a symbol bit, which is stored in the register 990 via signal line 983.

Infinity detection circuit 992 operates on the index signal bus 979 input, and if it detects that all exponent bits are 1, it provides a "1" to OR gate 987, which in turn sets its output 788 to "1". This sets ZERO-SEL signal 788, which sets the significand to all zeros (Mux 820, FIG. 9B).

When the index value on index signal bus 979 is out of range, denormal circuit 994 detects this condition and signals an under-bit condition on signal line 967. This condition is also signaled to OR gate 987, which generates signal ZERO-SEL on line 788. The ZERO-SEL signal on line 788 (FIG. 9B) will instruct multiplexer 820 to insert all "0"s into the significand, thereby creating the correct IEEE 754 "zero" representation (both the exponent and the significand contain all "0"s).

A floating-point multiply-add-accumulate unit using carry-save addition and accumulation with base 8 exponents is described. This balances the critical timing of the exponent unit with the critical timing of the significand unit. In addition, a 2's complement system is used to represent positive and negative significands that also have operand signs, as opposed to using signed numeric representation as proposed in the floating-point IEEE-754 standard. This avoids unnecessary significand subtraction when the exponents are equal to determine the greater of the two as specified by the IEEE-754 standard. The introduction of the 2's complement representation requires novel leading zero (leading one) detectors (predictors) that apply to both positive and negative numbers. The same applies to overflow (OV) detection. Additionally, it is necessary to determine when adding the carry and sum will result in a sign extension (SE), which requires the introduction of novel design features.

The floating-point multiply-add-accumulate unit is described, supporting the BF16 format for multiply-accumulate operations, and FP32 single-precision addition compliant with the IEEE 754 standard. The multiply-accumulate unit uses higher internal precision and longer accumulators by converting the operands to a higher radix and longer internal 2's complement significand representation to facilitate accuracy as well as comparisons and operations with negative numbers. Additions are performed using the carry-save format to avoid long carry propagation and speed up operations. Operations such as overflow detection, zero detection, and sign extension are employed in the 2's complement and carry-save formats. The handling of overflow and sign extension allows for fast operations that are relatively independent of the accumulator size. Rounding suitable for machine learning is introduced in the accumulation operation without affecting the time, greatly improving the accuracy of the calculation.

Abnormal handling

Figures 10 to 21 relate to exception handling in the above circuits and methods. Depending on the specific encoding format used to represent the significand and exponent, floating point numbers can take on values in special cases such as positive or negative infinity, zero, and non-normal numbers.

FIG. 10 illustrates the floating point number range 1000 shown in block 1010 as a horizontal number line divided into regions of interest by a number of items. Definitions of Terms are defined in Table 1.

Floating point special numbers

The following list shown in Table 1 contains three rows. The first row lists the definitions of special floating-point numbers. The second row lists the values in the BF16 floating-point encoding format. The third row shows the values in the floating-point encoding FP32 format.

In the Notes column of Table 1, the term (+)Nan includes the value 7F81 in BF16 and the value 7F800001 in FP32 as two conventions for representing NaN. The term (+)Norm is listed as (+)Pi, (3.14...) and the term (-)Norm is listed as (-)Pi for testing purposes. The terms (+)Norm and (-)Norm are the smallest representable values. (+)Zero and (-)Zero have two representations that differ in the most significant sign bit. The same is true for all other terms related to the sign bit.

The BFloat16 floating point format, also known as the brain floating point format, (sometimes referred to as "BF16") is a 16-bit number encoding format. BF16 preserves the approximate dynamic range of IEEE single-precision numbers. The BF16 format consists of a 7-bit fraction (also called the mantissa or significand), a "hidden bit" or "hidden bit", an 8-bit exponent, and a sign bit. Single-precision floating point values can be converted to BF16 to speed up machine learning. The dynamic range is the same as single-precision FP32 (8-bit exponent) using 8-bit precision instead of 24-bit fraction. BFloat16 can reduce memory requirements, can reduce storage requirements, and can increase the computation speed of machine learning algorithms. BF16 is a truncated 16-bit version of the 32-bit single-precision IEEE 754 format intended for accelerating machine learning.

The second numeric format is IEEE 754 single-precision 32-bit floating point (FP32). IEEE 754 single-precision 32-bit floating point consists of a 23-bit fraction, an "implicit" or "hidden bit", an 8-bit exponent, and a sign bit.

Table 2 below lists the BFloat16 terms and their value definitions.

Table 3 below lists additional BFloat16 terms and their binary numeric definitions. Positive and negative infinite are defined when all exponent bits are equal to one and when all fraction bits are equal to zero. Positive and negative NaN (Not a Number) are defined when all exponent bits are equal to one and when not all fraction bits are equal to zero. Positive and negative NON are defined when all exponent bits are equal to zero and when not all fraction bits are equal to zero. Whether it is positive or negative infinite, NaN, or NON depends on the sign bit.

In some embodiments, the exception processing unit used for machine learning operations does not support denormal or NaN operations. Denormal numbers are treated as zero, and NaN numbers are treated as infinite.

Abnormal

Chapter 7 of the IEEE Standard 754-2019 specification describes the five categories of floating-point exceptions listed below. According to one embodiment, three of the following categories are implemented: (1) invalid operation, (3) overflow, and (4) under. This embodiment does not support division by zero and inexact.

1) Invalid operation

2) Divide by zero

3) Overflow

4) Lack of position

5) Inaccurate

According to some other embodiments, four categories are implemented: (1) invalid operation, (2) division by zero, (3) overflow, and (4) under. According to other embodiments, all five categories are implemented.

Invalid operation

The IEEE Standard 754-2019 specification describes the following as invalid operations: a) any general computation on a signaling NaN; b) multiplication: multiply(0,∞) or multiply(∞,0); c) fusedMultiplyAdd: fusedMultiplyAdd(0,∞,c) or fusedMultiplyAdd(∞,0,c); d) addition or subtraction or fusedMultiplyAdd: Infinite magnitude subtraction, such as addition (+∞, -∞); e) division: division(0,0) or division(∞,∞); f) remainder: remainder(x,y) when y is zero or x is infinite, and neither is NaN; g) square root if the operand is negative; and h) quantization when the result does not fit in the target format or when one operand is finite and the other is infinite.

According to one embodiment, exception handling in the carry-save accumulation unit implements the invalid operations a/b/c/d listed above. According to another embodiment, any combination of categories (a) to (h) is possible.

Invalid operation exception

The following table 4 lists invalid operations that generate exceptions.

Overflow exception

The following in Table 5 lists the overflow exceptions for the case of two operand maximums (maximum norms) multiplied by one of the maximum norms. These overflow exceptions produce the result "Signed Infinity". This overflow exception occurs when the result is greater than the signed maximum norm (Signed Maximum Norm) and only when there are no infinities on the inputs of the operands.

Abnormal position

In one embodiment, there are several cases where the result is less than the signed minimum norm. This exception occurs when there is no exact zero value on the operand input. When adder norm/rounding: (+) norm + (-) norm occurs, the result is a non-normal (non-normal) value (but not exact zero), but the actual result will be "signed zero". See Table 6, which shows an example of two operand minimum norms (minimum norms) multiplied by minimum norms.

In some embodiments, exception handling can be divided into "exception flag generation" and "exception result generation". For example, exceptions are handled on the floating-point multiplier block 1110 and the floating-point carry-save adder block 1130 of FIG. 12A.

Abnormal flag generation

In some embodiments, floating-point multiplier exception flags are provided for: (1) overflow, (2) under, and (3) invalid.

In some embodiments, the following floating-point adder exception flags are provided: (1) overflow, (2) under, and (3) invalid.

Abnormal result generation

The operations for exception handling are explained in Chapter 6 of IEEE Standard 754-2019. The following two tables summarize an embodiment and implementation of the multiplication and addition operations.

Multiplier operation

According to one embodiment, the contents of Table 7 list multiplication operations with annotations for invalid, under, and overflow operations.

Addition operation

According to one embodiment, the contents of the adder operation list of Table 8 have annotations for invalid and overflow operations.

An exception processing circuit is described herein for detecting at least one invalid operation or result in a multiplier and at least one invalid operation or result in an adder based on a floating point encoding format and setting the output operand of the multiplier or adder to a value that can be used for further processing.

High-level architecture

FIG. 11 illustrates an example high-level architectural block diagram 1100 depicting components of exception handling in a carry-save-accumulate unit for machine learning.

In one embodiment, exception handling in a carry-save-accumulate unit design includes three different input signals. These are operator A 1113, operator B 1114, and operator C 1116. Operator A 1113 can be in BF16 and FP32 formats. Operator B 1114 is in BF16 format, and operator C 1116 is in FP32 format. Operators are also referred to as inputs.

The multiplier exception handling block 1102 imports operand A 1113 and operand B 1114. The output of the multiplier exception handling block 1102 is connected to the following: (1) via bus 1106 to the multiplier exception flag 1104, (2) via bus multiplier exception condition signal 1108 to the exception output control signal generation 1126, and (3) the multiplier exception result 1115.

Operand C 1116 in FP32 format enters operand C basic conversion block 1118 and is output to: (1) via operator C exception condition signal bus 1120 to exception output control signal generation 1126, and (2) via bus 1122 to carry save adder block 1130.

The carry-save adder (CSA) block 1130 processes two inputs via bus 1122: (1) the multiplier exception 1115, and (2) the output from the operator C basic conversion block 1118. In one embodiment, the CSA block 1130 has an accumulator loop 1124 that will output data only at the end of the loop. The CSA module 1130 is output via bus 1132.

The adder normalization exception handling block 1134 takes two inputs. The first input is the CSA block 1130 output through bus 1132. The second input is the exception control 1128 from the exception output control signal generation 1126 block.

The outputs of the adder normalization exception handling block 1134 are the adder exception result 1139 and bus 1138 which is routed to the adder exception flag block 1140.

Operand A is a 32-bit bus and provides BF16 inputs for multiplication and FP32 inputs for addition. Operand B is used for multiplication only and always has BF16 16-bit input format. Operand C is used for addition or accumulator initialization and always has FP32 32-bit input format. The multiplier section has separate output flags (overflow, underflow, and invalid), and the multiplier exception result becomes a direct input to the adder. The adder exception signal is generated by the multiplier and connected to the "Exception Control Signal Generation" block, and has the exception signals from the operator C and the accumulator to generate the "Exception Control" signal for the adder normalization block for adder exception handling.

Operation mode

According to one embodiment, exception handling in a carry-save-accumulate unit design supports three different operation modes.

FIG12A illustrates a first operation mode 1200 according to the high-level block diagram architecture of FIG11. The multiply-accumulate operation shows operand A 1113 in BF16 format first multiplied with operand B 1114 in BF16 format, and then the product is added to operand C 1116 in FP32 format. The multiply-accumulate operation is completed in a single operation. This operation generates multiplication and addition exception flags and results.

Multiplier block 1110 imports operand A 1113 and operand B 1114. The output of multiplier block 1110 is connected to: (1) multiplier exception flag 1104 block via bus 1106, and (2) multiplier exception result 1115 to carry save adder (CSA) block 1130. In some embodiments, the first operation mode uses multiplier exception flag 1104 for statistical purposes.

The carry-save adder (CSA) block 1130 processes the following two inputs: (1) the multiplier exception result 1115 and (2) the operator C 1116. The CSA block 1130 has two outputs. The first output is the adder exception result 1139, which is routed to the output block 1129 in BF16 or FP32 format. The second output is bus 1138 which is routed to the adder exception flag block 1140.

FIG12B illustrates a second operation mode 1202 according to the high-level block diagram architecture of FIG11. The multiply-accumulate operation is shown as operand A 1113 in BF16 format multiplied with operand B 1114 in BF16 in a single operation. It produces an output result at the end of the accumulation loop. During accumulation, adder outputs and adder exceptions are disabled. This operation produces multiplication and addition exception flags and results.

Multiplier block 1110 imports operand A 1113 and operand B 1114. The output of multiplier block 1110 is connected to the following blocks: (1) to multiplier exception flag 1104 via bus 1106, and (2) to carry-save adder (CSA) 1130 via multiplier exception result 1115. In some embodiments, the second operation mode uses multiplier exception flag 1104 for statistical purposes.

The carry-save adder (CSA) block 1130 processes the following two inputs: (1) the multiplier exception result 1115 and the accumulator loop 1124. The CSA block 1130 has two outputs. The first output is the adder exception result 1139, which is routed to the output block 1131 in BF16 or FP32 format only at the end of the accumulator loop 1124. The second output is the bus 1138, which is routed to the adder exception flag block 1140.

FIG12C illustrates a third operation mode 1203 according to the high-level block diagram architecture of FIG11. An addition operation is shown as an addition of an FP32 formatted operand A 1113 and an FP32 formatted operand C 1116. This operation generates only an addition exception flag and result. Multiplier exception handling is disabled.

The carry-save adder (CSA) block 1130 adds two inputs. Operand A 1113 in FP32 format is added to operand C 1116 in FP32 format. CSA block 1130 has two outputs. The first output is the adder exception result 1139 which is routed to output block 1129 in BF16 or FP32 format. The second output is bus 1138 which is routed to adder exception flag block 1140.

Exception handling structure

According to some embodiments, exception handling is divided into "exception flag generation" and "exception result generation", both of which can also be divided into multiplier exceptions and adder exceptions.

The multiplier and adder flag generation produces overflow, underrun, and invalid flags. These flags are shown as a set of circuit implementations in the application below.

Multiplier and adder abnormal result generation includes three conditions: (1) sign generation, (2) exponent generation, and (3) fractional generation. The sign has a positive output or a negative output. When an abnormal situation occurs, the exponent can have all "0" and all "1" conditions, and when no abnormal situation occurs, the exponent can have a normal output. The fractional value can have two conditions; for all abnormal situations, it is "0", and for non-abnormal situations, it is normal.

FIG. 13 illustrates an exception handling structure 1300 depicted in a high-level block diagram. The floating point multiply accumulator exception block 1308 outputs a flag to the exception flag generation block 1304 and outputs a result to the exception result generation block 1312. The flags contain status information or data for processing by a dedicated exception handling block, as described further below.

The abnormal flag generation block 1304 can output the flag to the multiplier abnormal flag generation block 1302 or the adder abnormal flag generation block 1306. The multiplier abnormal flag generation block 1302 drives the multiplier overflow flag status block 1381, the multiplier under-bit flag status block 1382, and the multiplier invalid flag status block 1383.

The adder exception flag generation block 1306 drives the adder overflow flag condition block 1387, the adder underrun flag condition block 1388, and the adder invalid flag condition block 1389.

The abnormal result generation block 1312 provides the result to the multiplier abnormal result generation block 1310 and the adder abnormal result generation block 1314.

The multiplier abnormal result generation block 1310 outputs the result to: (1) the multiplier sign generation status block 1320A, (2) the multiplier exponent generation status block 1322A, and (3) the multiplication fraction generation status block 1324A.

Adder exception result generation block 1314 outputs the result to: (1) adder sign generation status block 1326A, (2) adder exponent generation status block 1328A, and (3) adder fraction generation status block 1330A.

Multiplier sign generation condition block 1320A outputs condition to blocks 1320B and 1320C. Multiplier exponent generation condition block 1322A outputs condition to blocks 1322B, 1322C, and 1322D. Multiplication fraction generation condition block 1324A outputs condition to blocks 1324B and 1324C.

Adder sign generation condition block 1326A outputs condition to blocks 1326B and 1326C. Adder exponent generation condition block 1328A outputs condition to blocks 1328B, 1328C, and 1328D. Adder fraction generation condition block 1330A outputs condition to blocks 1330B and 1330C.

The following figures from FIG. 14A to FIG. 21B illustrate schematic implementations of the high-level blocks shown in FIG. 13. For example, block 1381 in FIG. 13 is implemented in FIG. 14A, and block 1382 is implemented in FIG. 14B. The following figure titles correspond to the block names in FIG. 13.

Condition circuit

FIG14A depicts one implementation 1400 of the multiplier overflow flag condition circuit 1381. The schematic diagram shows that the multiplier overflow flag condition 1446 is valid at the high-order output from the multiplier overflow AND gate 1444. AND gate 1444 has the following three inputs: (1) multiplication operation enable 1414, (2) the output of the NOR gate 1445, and (3) 1442, which is the output of the product exponent AND gate 1440, also known as Ep (exponential product) and the multiplier product exponent.

The NOR gate 1445 has two inputs, eainf and ebinf. If input A is infinite, the signal index A is infinite (eainf) appears and is detected when the index is equal to 0xFF (meaning all "1's"). If input B is infinite, the signal index B is infinite (ebinf) appears and is detected when the index is equal to 0xFF (meaning all "1's"). Table 2 shown above defines the BFloat16 terms and their numeric definitions. Eainf is the output of AND gate 1420. The input of AND gate 1420 is the exponent of operand A and is shown to have the least significant bit (LSB) 1402 and the most significant bit (MSB) 1404, comprising eight exponent bits.

Ebinf is the output of AND gate 1430. The input to AND gate 1430 is the exponent of operand B and is shown with the least significant bit (LSB) 1406 and the most significant bit (MSB) 1408, comprising eight exponent bits.

The inputs to the product exponent and gate 1440 are the least significant bit (LSB) 1410 and the most significant bit (MSB) 1412, which contain eight exponent bits.

FIG. 14B depicts an implementation 1400 of a multiplier under-bit flag condition circuit 1382. The multiplier under-bit flag condition 1482 is active at the high-order output from the multiplier under-bit AND gate 1480. AND gate 1480 has three inputs: (1) multiplication enable 1418, (2) the output of NOR gate 1475, and (3) 1472, which is the output of product exponent NOR gate 1470.

NOR gate 1475 has two inputs, eaz (input A exponent is zero) and ebz (input B exponent is zero). Eaz is the output of NOR gate 1450. The input to NOR gate 1450 is the exponent of operand A and is shown as having least significant bit (LSB) 1422 and most significant bit (MSB) 1424, comprising eight exponent bits.

Ebz is the output of NOR gate 1460. The input to NOR gate 1460 is the exponent of operand B and is shown as having least significant bit (LSB) 1426 and most significant bit (MSB) 1428, comprising eight exponent bits.

The inputs to the product exponent NOR gate 1470 are the least significant bit (LSB) 1432 and the most significant bit (MSB) 1434, which contain eight exponent bits.

The multiplier under-bit flag is asserted when multiplication is enabled, Ep (product exponent) is 0x00, and any multiplier exponent input is non-zero. A non-zero multiplier exponent input means that any exponent of operand A or operand B is not 0x00.

The invalid flag of the multiplier is determined based on the following two conditions: (1) the multiplication operation is enabled and (2) invalid is "1". Invalid "1" is defined as the condition when the exponent of operand A is infinite (0xFF) and the exponent of operand B is zero (0x00) or when the exponent of operand B is infinite (0xFF) and the exponent of operand A is zero (0x00).

FIG. 15 depicts one implementation 1500 of the multiplier invalid flag condition circuit in block 1383. The multiplier invalid flag condition 1582 is asserted at the high-order output from the multiplier invalid AND gate 1580. AND gate 1580 has the following two inputs: (1) multiplication operation enable 1501, and (2) 1572, the output of OR gate 1570.

OR gate 1570 has two inputs, the first input is 1552 derived from AND gate 1550, and the second input is 1562 derived from AND gate 1560.

AND gate 1550 has two inputs: eainf and ebz. Signal ebz appears when input B is zero, that is, (+) zero or (-) zero, the condition is the index 0x00 (indicating all "0"). Eainf is the output of AND gate 1510. The input of AND gate 1510 is the index of operand A and is displayed as having the least significant bit (LSB) 1502 and the most significant bit (MSB) eight bits 1504. Ebz is the output of NOR gate 1520. The input of NOR gate 1520 is the eight-bit index of operand B and is displayed as having the least significant bit (LSB) 1506 and the most significant bit (MSB) 1508.

AND gate 1560 has two inputs: ebinf and eaz. Ebinf is the output of AND gate 1530. The input to AND gate 1530 is the eight-bit exponent of operand B and is shown as having least significant bit (LSB) 1506 and most significant bit (MSB) 1508. Eaz is the output of NOR gate 1540. Signal eaz appears when input A is zero, i.e., (+) zero or (-) zero), and the exponent is 0x00 (representing all "0"). The input to NOR gate 1540 is the exponent of operand A and is illustrated as having least significant bit (LSB) 1502 and most significant bit (MSB) eight bits 1504, comprising eight exponent bits.

FIG. 16 depicts an implementation 1600 of the multiplier symbol generation state circuit 1320. The multiplier flag state 1632 is generated by the AND gate 1630. The multiplier invalid flag state 682 of FIG. 15 is the first input of the AND gate 1630. The second input of the AND gate 1630 is derived 1632 from the output of the EX-OR gate 1620. Symbol A 1618 and symbol B 1622 comprise the inputs of the EX-OR gate 1620.

FIG. 17A shows an implementation 1700 of the multiplier exponent generation circuit 1322. The multiplier exponent 1752 is generated based on three conditions. They are: (1) all "0" (0x00 = zero), (2) all "1" (0xFF = infinity), and (3) a normal exponent.

The first case is when the exponents are all zero. The first case can occur if any exponent of operand A or any exponent of operand B is zero. The first case can also occur if the multiplier exponent calculation results in a negative overflow, which means that the multiplier exponent calculation result is less than -126.

The second condition is all "1". The second condition may occur if any exponent of operand A or any exponent of operand B is infinitely large. The second condition may also occur if the multiplier exponent calculation result is a positive overflow, that is, the multiplier exponent calculation result is greater than Emax (+127).

The third condition is a normal output, defined as none of the first or second conditions occurring. This is defined as anything other than all "0" or all "1".

The eight-bit wide multiplexer 1750 has three 8-bit buses as inputs. These are the all-0, all-1, and other buses. The output of the multiplexer 1750 is controlled by control gates using OR gate 1710, AND gate 1720, OR gate 1730, and NOR gate 1740.

FIG17B shows one implementation 1700 of the multiplier fraction generation circuit 1324. The multiplier fraction bus 1772 is 16 bits wide and generates based on two conditions: (1) abnormal, (2) normal fraction. The two multiplexer inputs are: (1) all "0" condition (0x00 = zero), and (2) normal fraction condition.

Multiplexer 1770 has two 16-bit input buses. These are the all-zero and fractional buses. OR gate 1760 controls the gating of the 1770 multiplexed 16-bit wide multiplexed outputs.

Signals exp_overflow and ezero are two inputs to OR gate 1760. Signal exp_overflow is the logical OR of positive overflow or negative overflow, where overflow is described in FIG. 17A above. In some embodiments, a status bit is provided after the exponent calculation to detect exponent overflow. The term ezero is defined as eaz (input A exponent is zero) or ebz (input B exponent is zero). The output of OR gate 1760 is 1762, which is a control that selects between the two 16-bit wide multiplexer 1770 input buses. All "0" occurs when: (1) the product exponent is positive overflow, or (2) there is a negative overflow, or (3) the exponent of operand A is zero, or (4) the exponent of operand B is zero.

In some embodiments, exception handling in the carry-save accumulation unit does not support denormal or NaN operations. In this case, denormal numbers are treated as zero and NaN numbers are treated as infinite. When any of these exceptions occur, the fractional output will be all "0". Otherwise, the output of multiplexer 1770 is a normalized fractional number.

FIG18A illustrates a schematic implementation 1800 of the adder overflow flag condition circuit in block 1387. A signal named adder overflow 1832 demonstrates the overflow flag condition and is the output of AND gate 1830. The adder overflow flag condition occurs when AND gate 1830 has the following three inputs: (1) normalize_enable, (2) AND gate 1810 output named overflow, and (3) NOR gate 1820 output named non-input exponent infinity 1822. The inputs to AND gate 1810 are the eight normalized exponent bits. The inputs to NOR gate 1820 are product exponent infinity and input C exponent infinity. In summary, during normalization, if the normalized exponent is equal to 0xFF and there are no inputs with infinite exponent conditions, an adder overflow 1832 flag condition may occur.

FIG18B shows one implementation 1800 of the adder under-bit flag condition circuit in block 1388. Adder under-bit 1892 is valid at the high-order output of AND gate 1890 having three inputs: (1) normalize_enable, (2) AND gate 1850 output named under-bit, and (3) NOR gate 1880 output named not-input-exact-zero 1882. The inputs to AND gate 1850 are the eight normalized exponent bits and fractional output zero. In summary, the adder under-bit flag condition typically occurs during normalization when the normalized exponent is 0x00 (equal to 0) and the input is not exactly zero. The following paragraphs describe one implementation of a circuit that can check for the not-input-exact-zero 1882 condition.

Further describing the circuit of FIG. 18B , the generation of the non-input exact zero signal 1882 is through three inputs of the inverted OR gate 1880. The three inputs are: (1) the multiplier product exponent zero, (2) the input C exponent zero, and (3) the output exact zero 1872, which is derived from the output of the AND gate 1870. The AND gate 1870 has two inputs. The first input is the output of the EX-OR 1860 gate named sign_diff. The EX-OR 1860 gate operates on two inputs (the multiplier product sign and the input C sign). The second input of the AND gate 1870 is (+) zero, where the (+) zero is the logical AND of the input C exponent greater than or equal to the multiplier product exponent and the fractional output zero signal.

Adder undershoot is a combination of the following conditions, as shown in the circuit. The first condition; (1) is enabled during final normalization. The second condition; (2) is when the final addition index result, that is, the normalized index is equal to 0x00 and the final addition fractional result (fractional output zero signal) is not exactly zero. The third condition; (3) is when the multiplier product index zero, input C index zero, and output inexact zero are all not enabled, which means that one of them is zero. If any of the three conditions is zero, then adder undershoot does not occur.

FIG19A shows one implementation 1900 of the adder invalid flag condition circuit in block 1389. Referring to Table 8, an invalid condition occurs when adding (+) infinity and (-) infinity. Adding (+) infinity and (+) infinity or adding (-) infinity and (-) infinity does not cause an invalid flag condition 1389. The circuit checks for two operands with infinities of opposite signs. The adder invalid 1932 flag is the output of AND gate 1930. AND gate 1930 has the following three inputs: (1) normalization enable, (2) the output of OR gate 1910 named 1912, and (3) the sign_diff output from EX-OR gate 1920.

The two inputs of adder invalid OR gate 1910 are the multiplier product exponent infinitely large and input C exponent infinitely large. EX-OR gate 1920 operates on two inputs, the multiplier product sign and the input C sign.

To summarize the circuit functionality, the adder invalid flag condition occurs when (1) final normalization is enabled, (2) if the inputs from input A (or multiplier product) and input C are of different signs (positive and negative, or negative and positive), and (3) the exponents of both inputs are infinite (0xFF).

Adder symbol generation circuit

FIG. 19B shows an implementation 1900 of an adder sign positive state circuit 810A for generating an adder sign positive output. As previously described, exception result generation has three parts: sign generation; exponent generation; and fractional generation. The sign output generated can be positive or negative. One implementation of the sign output has two circuits that combine to provide the correct sign output for exception result generation. The first circuit is the adder sign positive state circuit 810A. When the adder sign positive 1992 bit is equal to "1", the adder sign output has a positive state (+).

The sign output function forces the positive condition to zero based on the following additions: (1) (+)Zero + (-)Zero, (2) (+)Normal + (-)Normal (or vice versa), (3) the signs are different and one of the exponential inputs is infinitely large ((+)Inf + (-)Inf or vice versa), (4) the signs are different and the multiplier product is positive and the exponent is infinitely large ((+)Zero x (+)Inf = (+)Inf), (5) the signs are different and the exponent of operator C is greater than the exponent of operator A (or the multiplier product), (6) the equality and fractional outputs are zero, and (7) when both the sign of the multiplier product and the sign of input C are positive.

One implementation of a first circuit for generating a sign bit forced to "0" for the positive condition is shown in FIG. 19B, where the positive sign is the signal adder sign positive 1992, which is the output of the AND gate 1990. The AND gate 1990 has two inputs. The first input is the output of the OR gate 1980 named 1982, and the second input is sign_diff, which comes from the EX-OR gate 1960. The EX-OR gate 1960 gate operates on two inputs, the multiplier product sign and the input C sign. The OR gate 1980 has four inputs. These are the lack bits 1967, 1972 and the add zero and fractional output zero.

The output of AND gate 1950 is called the lack bit, which is a combination of the output 1942 of NOR gate 1940 and the AND fractional output zero. The input of NOR gate 1940 is an 8-bit normalized exponent vector. The two inputs of OR gate 1965 are the multiplier product exponent infinite and the input C exponent infinite. AND gate 1970 has a first input multiplier exponent infinite capability and a second input is the output of NOR gate 1968, whose input is the multiplier product sign.

FIG. 20A depicts an implementation 2000 of an adder sign negative condition circuit block 1326C for generating an adder sign negative (-) condition output. This is a second circuit for generating a sign output as described above. The sign output includes two circuits that combine to generate a positive (+) condition or a negative (-) condition. When the adder sign negative condition 2042 bit is equal to "1", the adder sign negative condition circuit 810B generates a negative (-) condition. The second circuit is first described as a schematic implementation, and the functionality is summarized in the following paragraphs.

One implementation of a circuit for generating a signed output bit forced to 1' for a negative (-) condition is shown in FIG. 20A. OR gate 2040 outputs adder signed negative condition 2042. The two inputs of OR gate 2040 are AND gate 2020 output 2022 and AND gate 2030 output 2032. When AND gate 2020 is determined, the first adder signed negative condition may occur. AND gate 2020 operates on the input multiplier product sign and the input C exponent, and if both signs are negative, the output is determined.

The determination of AND gate 2030 may result in a second adder sign negative condition. This will occur when the following inputs to AND gate 2030 are determined: the multiplier index is infinitely large, the multiplier product sign, and 2012. The output of anti-AND gate 2010 is 2012 enabled by the multiplier product index infinitely large and the output of inverting gate 2005 with the operator C sign as input.

To summarize the adder signed negative condition, the signed output is forced to a logical "1" for the negative condition (-) according to the following conditions: (1) when both operand A (or multiplier product) and operand C are signed "1" (both are negative numbers); (2) or when operand A (or multiplier product) is negative infinity and operand C is not positive infinity.

Additive exponential circuit

The adder exponent output is generated using three conditions. The first condition is all "0"s (0x00 = zero). The second condition is all "1"s, where 0xFF would be equal to the infinite condition. The third condition is the normal exponent.

FIG. 20B illustrates an implementation 2000 of adder index generation in the all “0” state circuit block 1328B. Adder index all “0” select 2082 is the output of an OR gate 2080 having three inputs. Each input represents a separate state.

When the multiplier product index zero or the input C index zero enters the OR gate 2050, the first adder index all "0" condition is generated. The condition determination 2052 triggers the adder index all "0" selection 2082.

When AND gate 2060 has (+) zero and sign_diff input, a second adder index all "0" condition is generated, where (+) zero is the input C index greater than or equal to the multiplier product index and the fractional output zero signal logically ANDed. EX-OR gate 2070 operates on the multiplier product sign and the input C sign to generate sign_diff. When AND gate 2060 is determined, 2062 can be operated to trigger the adder index all "0" selection 2082.

The third adder index all "0" condition is generated by the fractional output zero input to the OR gate 2080 to trigger the adder index all "0" select 2082.

FIG. 21A illustrates a schematic implementation 2100 of adder index generation for all "1" state circuit block 1328B. An OR output function with three input conditions determines the all "1" output condition and is described in the following paragraphs. State circuit 811B displays adder index all "1" select 2122 as the output of an OR gate 2120 containing three inputs. Each input represents a separate condition.

When the multiplier product exponent is infinite or the input C exponent is infinite and is a valid input to the OR gate 2110, the first all-"1" condition is generated. The condition is determined in 2112.

The second all-one condition is generated when the 8th bit of the adder normalized exponent [8] overflows.

The third all-ones condition is generated by infinite multiplier exponents, where the multiplier output exponent is infinite or has positive overflow.

In summary, when any exponent of input A (or multiplier product) or input C is infinite, or the final adder normalized exponent overflows, or the multiplier output exponent is infinite or positive overflow ((+)zero x (+)Inf = (+)Inf), an all-ones condition is generated.

FIG21B shows a schematic implementation 2100 of the adder fraction generation circuit 1330A. The circuit routes the actual normalized fraction value or 23 bits of zero based on the three selector states that control the multiplexer. The three conditions that force the first 23 bits of the bus to be zero are: (1) any normalized exponent overflow; (2) normalized exponent underrun; or (3) or the multiplier output exponent is infinite or positive overflow. Otherwise, this condition is normal and a second bus containing the 23-bit normalized fraction bus is routed to the adder fraction 2132 output.

The schematic diagram of circuit 812A shows that the adder fraction 2132 outputs a 23-bit bus provided by multiplexer 2150. OR gate 2130 outputs selector control 2131 to select between the two 23-bit input buses of multiplexer 2150. Multiplexer 2150 has two 23-bit buses as inputs. The first input bus is a 23-bit all-zero bit bus. The second input bus is a 23-bit normalized fraction bus. OR gate 2130 outputs control 2131 to multiplexer 2150 based on the three input conditions described in the above paragraph.

The output of adder decimal 2132 has two conditions, all "0" (0x00 = zero) and normal decimal. In one embodiment, the exception handling in the carry-save accumulation unit does not support denormal or NaN operations, and denormal numbers are treated as zero, while NaN numbers are treated as infinite. When any of these exceptions occurs, the decimal output will be all "0".

When an overflow or underbit exception occurs, or the multiplier output exponent is infinite or positive overflow ((+) zero multiplied by (+) Inf = (+) Inf), all "0" appears.

Multiplier-accumulator operation case study

In the next paragraph, x is the multiplication operator, + is the sum operator, and = is the equality operator.

Although the present invention is disclosed by reference to the various embodiments and examples described in detail above, it should be understood that these examples are intended to be illustrative rather than restrictive. It is expected that those skilled in the art will readily conceive of modifications and combinations that will be within the spirit of the present invention and the scope of the appended claims.

210:Multiplexer

211:Multiplexer

212: Multiplexer

213: Operator A

214: Operator B

215: Base 8 converter

216: Operator C; line

217: BF16 format or FP32 format

218: Line

219: Line

220: Block

221: Line

222:Double bus

223:Bus

224: Output bus

225: Output bus

226: Bus

227: Bus

228:Bus

229: Bus

230: Carry-save adder

240: Accumulator

250: Carry-preserve to sign-value conversion block

251:Bus

252:Bus

260: Basis 8 to Basis 2 Transformation and Normalization Blocks

270: FP32 or BF16 block

Claims (18)

A multiplication-accumulation unit comprises: a pipeline configured to perform multiplication-accumulation operations on a series of input floating-point operands, the pipeline comprising: a significand circuit receiving a multiplier output significand in a 2's complement format and feeding back a sum and carry value outputted by a feedback accumulator in a pipeline cycle, the significand circuit comprising a 2's complement, carry-save adder to generate a sum of an accumulator output and a carry accumulator output significand value; and an exponent circuit receiving a multiplier output exponent in a base-8 format and a feedback exponent value outputted by the feedback accumulator in a base-8 format in the pipeline cycle to generate an accumulator output exponent value outputted by the accumulator. A multiplication-accumulation unit as claimed in claim 1, wherein: the significand circuit includes a significand shifter, which responds to an exponent comparison signal to align the multiplier output significand with the feedback sum and carry value for addition; the index circuit responds to the exponent comparison signal to generate the accumulator output exponent value; and the pipeline includes an exponent comparison circuit, which compares the multiplier output exponent with the feedback exponent value before the pipeline cycle to generate the exponent comparison signal. A multiplication-accumulation unit as claimed in claim 2, wherein the pipeline comprises: an overflow detector circuit for generating an overflow signal indicating an overflow condition for at least one of the sum and carry values of the feedback; and the exponent circuit and the significand circuit also respond to the overflow signal. A multiplication-accumulation unit as claimed in claim 2, wherein the pipeline comprises: an overflow detector circuit for generating a first condition signal indicating an overflow condition of at least one of the sum and carry value of the feedback; a leading sign bit detector circuit for generating a second condition signal indicating that at least one of the sum and carry value of the feedback has greater than or equal to 8 extended sign bits; and the exponent circuit and the significand circuit are also responsive to the first condition signal and the second condition signal. A multiply-accumulate unit as claimed in any one of claims 1 to 4, wherein the pipeline comprises: a multiplier circuit, responsive to a first input operand and a second input operand, providing a multiplier significand value and a multiplier exponent value prior to the pipeline cycle; a base-8 conversion circuit, for converting the multiplier significand value and the multiplier exponent value to a base-8 format for the multiplier output exponent and significand; and a two's complement conversion circuit, for converting the multiplier significand value to a two's complement representation of the multiplier output significand. A multiplication-accumulation unit as claimed in any one of claims 1 to 4, wherein the pipeline comprises: A multiplier circuit, which is used to respond to a first input operand and a second input operand, and provide a multiplier effective value and a multiplier exponent value before the pipeline cycle, wherein the multiplier circuit comprises an effective number multiplier circuit and an exponent adder circuit, the effective number multiplier circuit having a carry-save adder for generating partial products of carry values and sum values to generate high-order bits of the multiplier output effective number, and a ripple carry adder for generating partial products of the effective number carry output and the low-order bits of the sum output. A multiply-accumulate unit as claimed in any one of claims 1 to 4, wherein the pipeline comprises: a multiplier circuit comprising a significand multiplier circuit and an exponent adder circuit, responsive to a first input operand and a second input operand, providing a multiplier significand value and a multiplier exponent value prior to the pipeline cycle, and wherein the first input operand and the second input operand are in floating point format including a shift amount so that a negative exponent is represented by a positive binary number, and a circuit for correcting the shift amount by inverting the most significant digit of one of the first operand and the second operand and adding 1 to a carry input of the exponent adder circuit. A multiplication-accumulation unit as claimed in any one of claims 1 to 4, wherein the pipeline has an accumulator mode and a summing mode, and comprises: a selector for providing the accumulator output of the feedback in the accumulator mode, and for providing a third floating-point input operand to the significand circuit and the exponent circuit in the summing mode. A multiply-accumulate unit as claimed in claim 1, wherein the pipeline comprises: a first stage comprising a floating-point multiplier having sum and carry outputs; a second stage comprising a multiplier output adder for the sum output and the carry output of the multiplier, and a circuit for converting the multiplier adder output to a base-8 format having a 2's complement significand; a third stage comprising the significand circuit and the exponent circuit; a fourth stage for converting the accumulator sign bit, the accumulator exponent, and the accumulator significand sum and carry value to a signed value significand format; a fifth stage for converting the signed value significand format from base-8 alignment to base-2 alignment and generating a normalized exponent and significand; and a sixth stage for performing rounding and converting to a standard floating point representation. A multiplication-accumulation method for calculating the sum S(i) of terms A(i)*B(i), wherein (i) ranges from 0 to N-1, and N is the number of terms in the sum, the method comprising: receiving a series of operands A(i) and operands B(i) in floating point format, (i) ranging from 1 to N in a multiplication-accumulation unit; multiplying the operands A(i) and B(i) by the multiplier pipeline stage of the multiplication-accumulation unit to generate the terms A(i)*B(i) and the multiplier output significand in a format including a multiplier output exponent in base 8 format, and converting the multiplier output significand into a 2's complement format; A carry-save adder is used in the accumulator pipeline stage of the addition unit to add the multiplier output valid number of the item A(i)*B(i) in the 2's complement format to the valid number of the sum S(i-1), and to generate a sum and a carry value for the sum S(i); an exponent of the sum S(i) is selected from the multiplier output exponent of the item A(i)*B(i) and the exponent of the sum S(i-1) through the index circuit of the multiplication-accumulation unit to generate the exponent of the sum S(i) in the base 8 format; and the sum and the carry value of the sum S(i) in the base 8 format and the exponent are converted into a normalized floating point format. The multiplication-accumulation method of claim 10 includes comparing the exponent of the sum S(i-1) with the multiplier output exponent of the item A(i)*B(i) to generate an exponent comparison signal; and in response to the exponent comparison signal, aligning the sum and carry value of the sum S(i-1) with the effective number of the item A(i)*B(i) so as to perform addition in the carry-save adder. The multiplication-accumulation method of claim 11 includes generating an overflow signal indicating an overflow condition for at least one of the sum and carry value of the sum S(i-1); and in response to the overflow signal, aligning the sum and carry value of the sum S(i-1) and the exponent with the multiplier output exponent and the multiplier output valid number of the item A(i)*B(i) so as to perform addition in the carry-save adder. The multiplication-accumulation method of claim 11 includes: generating a first condition signal indicating an overflow condition of at least one of the sum and carry value of the sum S(i-1); generating a second condition signal indicating that at least one of the sum and carry value of the sum S(i-1) has greater than or equal to 8 extended sign bits; and in response to the first condition signal and the second condition signal, aligning the sum and carry value and the exponent of the sum S(i-1) with the multiplier output exponent and the multiplier output valid number of the item A(i)*B(i) so as to perform addition in the carry-save adder. A multiplication-accumulation method as claimed in any one of claims 10 to 13, wherein multiplying operand A(i) by operand B(i) to generate the term A(i)*B(i) includes using a carry-save adder for generating a partial product of a carry value and a sum value to generate high-order bits of the multiplier output significand, and using a ripple-carry adder for generating a partial product of a carry value and a low-order bit of the sum output. A multiplication-accumulation method as claimed in any one of claims 10 to 13, wherein the operands A(i) and B(i) are in floating point format including a shift amount such that a negative exponent is represented by a positive binary number, and correcting the shift amount includes inverting the most significant digit of one of the operands A(i) and B(i) and adding 1 to a carry input of an exponent adder circuit. A multiplication-accumulation unit, comprising: a pipeline configured to perform floating-point multiplication-accumulation operations on a sum S(i) of terms A(i)*B(i), where (i) ranges from 0 to N-1, and N is the number of terms in the sum, the pipeline comprising: a multiplier pipeline stage, comprising a multiplier circuit for providing a multiplier significand and a multiplier index value of the term A(i)*B(i) in response to a first input operand and a second input operand, a base-8 conversion circuit for converting the multiplier significand and the multiplier index value of the term A(i)*B(i) into a base-8 format, and a base-8 conversion circuit for converting the multiplier significand value into the base-8 format of the term A(i)*B(i). A 2's complement conversion circuit for a 2's complement representation of a multiplier output significand; an accumulator stage, comprising a significand circuit, the significand circuit adds the multiplier output significand of the term A(i)*B(i) to the feedback sum and carry value of the sum S(i-1), and generates the sum and carry value of the sum S(i), the significand circuit comprising a 2's complement carry-save adder to generate the sum value of the sum S(i) and the carry accumulator output significand value; and an exponent circuit, which receives the multiplier exponent value of the term A(i)*B(i) and the feedback exponent value of the sum S(i-1), to generate the accumulator output exponent value for the sum S(i). A multiplication-accumulation unit as claimed in claim 16, wherein: the significand circuit of the accumulator stage includes a significand shifter responsive to an exponent comparison signal; the exponent circuit is responsive to the exponent comparison signal; and the pipeline includes an exponent comparison circuit for comparing the multiplier exponent value of the term A(i)*B(i) with the feedback exponent of the sum S(i-1) to generate the exponent comparison signal for generating the sum S(i). The multiplication-accumulation unit of claim 17, wherein the accumulator stage comprises: an overflow detector circuit for generating a first condition signal indicating an overflow condition of at least one of the sum and carry values of the feedback of the sum S(i-1); a leading sign bit detector circuit for generating a second condition signal indicating that at least one of the sum and carry values of the feedback of the sum S(i-1) has greater than or equal to 8 extended sign bits; and the exponent circuit and the significand circuit are also responsive to the first condition signal and the second condition signal.

Images