TWI911805B - System with output circuit for a vector-by-matrix multiplication array and operating method for output circuit - Google Patents

Abstract

In one example, a system comprises a vector-by-matrix multiplication array comprising nonvolatile memory cells arranged into rows and columns, a first set of columns storing W+ weights and a second set of columns storing W- weights; and an output circuit to receive a first current from a respective column in the first set of columns and a second current from a respective column in the second set of columns and to generate a first voltage and a second voltage, the output circuit comprising a first current-to-voltage converter comprising a first integration capacitor to provide the first voltage equal to an initial voltage minus a first discharge value due to the first current, and a second current-to-voltage converter comprising a second integration capacitor to provide the second voltage equal to the initial voltage minus a second discharge value due to the second current.

Description

A system having an output circuit for a vector matrix multiplication array and a method for operating the output circuit.

This application claims priority to U.S. Provisional Patent Application No. 63/534,755, filed August 25, 2023, entitled "Output Circuit for Neural Network Array," and U.S. Patent Application No. 18/386,901, filed November 3, 2023, entitled "Output Circuit for a Vector-By-Matrix Multiplication Array."

Numerous examples of output circuits and connection methods used in vector matrix multiplication arrays are revealed.

Artificial neural networks simulate biological neural networks (the central nervous system of animals, especially the brain) and are used to estimate or evaluate functions that depend on large amounts of input and are often unknown. Artificial neural networks typically consist of layers of interconnected "neurons" that exchange information with each other.

Figure 1 illustrates an artificial neural network, where circles represent neuron inputs or layers. Connections (called synapses) are indicated by arrows and have numerical weights that can be tuned based on experience. This allows the neural network to adapt to inputs and learn. Typically, a neural network comprises one layer with multiple inputs. There are usually one or more intermediate neuron layers and output neuron layers that provide the network's outputs. Neurons at each layer make decisions individually or collectively based on data received from synapses.

One of the major challenges in the development of artificial neural networks for high-performance information processing is the lack of sufficient hardware technology. In reality, neural networks rely on a vast number of synapses to achieve high connectivity between neurons, i.e., extremely high computational parallelism. In principle, this complexity can be achieved using digital supercomputers or specialized clusters of graphics processing units. However, in addition to high cost, these methods are also hampered by moderate energy efficiency compared to biological networks, mainly because biological networks perform low-precision analog computations and therefore consume far less energy. CMOS analog circuits have been used in artificial neural networks, but given the large number of neurons and synapses, most CMOS implementations have excessively large synapses.

The applicant previously disclosed an artificial (analog) neural network utilizing one or more nonvolatile memory arrays as synapses in U.S. Patent Application Publication 2017/0337466A1, which is incorporated herein by reference. The nonvolatile memory arrays operate as analog neural memory and include nonvolatile memory cells configured in columns and rows. The neural network includes: a first plurality of synapses configured to receive a first plurality of inputs and generate a first plurality of outputs from the first plurality of inputs; and a first plurality of neurons configured to receive the first plurality of outputs. The first plurality of synapses include a plurality of memory cells, each of which includes: a source region and a drain region formed in a semiconductor substrate and spaced apart, wherein a channel region extends between the source region and the drain region; a floating gate disposed above and insulated from a first portion of the channel region; and a non-floating gate disposed above and insulated from a second portion of the channel region. Each of the plurality of memory cells stores a weight value corresponding to the number of electrons on the floating gate. The plurality of memory cells multiply the first plurality of inputs by the stored weight values to generate the first plurality of outputs. Non-volatile memory cells.

Non-volatile memory is well known. For example, U.S. Patent 5,029,130 (“130 Patent”), incorporated herein by reference, discloses a discrete gate non-volatile memory cell array, which is a type of flash memory cell. Such a memory cell 210 is shown in FIG. 2. Each memory cell 210 includes a source region 14 and a drain region 16 formed in a semiconductor substrate 12, and a channel region 18 between the source region and the drain region. A floating gate 20 is formed above and insulated from (and controls the conductivity of) a first portion of the channel region 18, and is formed above a portion of the source region 14. The character line terminal 22 (which is typically coupled to a character line) has: a first portion disposed above and insulated from the second portion of the channel region 18 (and controlling the conductivity of the second portion); and a second portion extending above and on the floating gate 20. The floating gate 20 and the character line terminal 22 are insulated from the substrate 12 by a gate oxide. The bit line 24 is coupled to the drain region 16.

The memory cell 210 is erased by placing a high positive voltage on the word line terminal 22 (where electrons are removed from the floating gate), which causes the electrons on the floating gate 20 to tunnel from the floating gate 20 through the intermediate insulation to the word line terminal 22 via the Fowler-Nordheim (FN) tunnel.

Memory cell 210 is programmed by source-side injection (SSI) of hot electrons (where electrons are placed on the floating gate) by applying a positive voltage to word line terminal 22 and a positive voltage to source region 14. Electron current flows from drain region 16 to source region 14. When electrons reach the gap between word line terminal 22 and floating gate 20, they are accelerated and heated. Some of the heated electrons are injected into floating gate 20 through the gate oxide due to the attractive electrostatic force from floating gate 20.

Memory cell 210 reads data by applying a positive read voltage to the drain region 16 and the word line terminal 22 (this connects the portion of channel region 18 below the word line terminal). If the floating gate 20 is positively charged (i.e., electrons are erased), the portion of channel region 18 below the floating gate 20 is also connected, and current flows across channel region 18; this is detected as erased or a "1" state. If the floating gate 20 is negatively charged (i.e., electronically programmed), the portion of channel region below the floating gate 20 is mostly or completely disconnected, and current does not flow across channel region 18 (or a very small amount of current flows across the channel region); this is detected as programmed or a "0" state.

Table 1 depicts the typical voltage and current ranges that can be applied to the terminals of memory cell 210 for performing read, erase, and programming operations: Table 1: Operation of flash memory cell 210 in Figure 2 WL BL SL Read 2-3V 0.6-2V 0V erase ~11-13V 0V 0V Programming 1-2V 10.5-3μA 9-10V

Other isolated gate memory cell architectures are known to us, such as those for other types of flash memory cells. For example, Figure 3 depicts a quad-gate memory cell 310 comprising a source region 14, a drain region 16, a floating gate 20 above a first portion of a channel region 18, a selection gate 22 above a second portion of the channel region 18 (typically coupled to a word line WL), a control gate 28 above the floating gate 20, and an erase gate 30 above the source region 14. This architecture is described in U.S. Patent 6,747,310, which is incorporated herein by reference for all purposes. Here, except for the floating gate 20, all gates are non-floating gates, meaning that these gates are electrically connected or can be electrically connected to a voltage source. Programming is performed by injecting heated electrons from channel region 18 into the floating gate 20. Erasure is performed by electrons tunneling from the floating gate 20 to the erase gate 30.

Table 2 depicts the typical voltage and current ranges that can be applied to the terminals of memory cell 310 for performing read, erase, and programming operations: Table 2: Operation of flash memory cell 310 in Figure 3 WL/SG BL CG EG SL Read 1.0-2V 0.6-2V 0-2.6V 0-2.6V 0V erase -0.5V/0V 0V 0V/-8V 8-12V 0V Programming 1V 0.1-1μA 8-11V 4.5-9V 4.5-5V

Figure 4 depicts a three-gate memory cell 410, which is another type of flash memory cell. Memory cell 410 is the same as memory cell 310 in Figure 3, except that memory cell 410 does not have a separate control gate. The erase operation (thereby erased by using the erase gate) and read operation are similar to the erase and read operations in Figure 3, except that no control gate bias is applied. Programming operation is also performed without control gate bias, and therefore, a higher voltage is applied to the source line during programming operation to compensate for the lack of control gate bias.

Table 3 depicts the typical voltage and current ranges that can be applied to the terminals of memory cell 410 for performing read, erase, and programming operations: Table 3: Operation of flash memory cell 410 in Figure 4 WL/SG BL EG SL Read 0.7-2.2V 0.6-2V 0-2.6V 0V erase -0.5V/0V 0V 11.5V 0V Programming 1V 0.2-3μA 4.5V 7-9V

Figure 5 depicts a stacked gate memory cell 510, which is another type of flash memory cell. Memory cell 510 is similar to memory cell 210 in Figure 2, except that the floating gate 20 extends over the entire channel area 18, and the control gate 22 (which will be coupled to the character line here) extends over the floating gate 20, separated by an insulating layer (not shown). The erasure is performed by electron tunneling from FG to FN on the substrate. The program is performed by channel hot electron (CHE) injection in the region between channel region 18 and drain region 16, by electron flow from source region 14 to drain region 16, and the read operation is similar to the read operation for memory cell 210 with a higher control gate voltage.

Table 4 depicts the typical voltage range that can be applied to the terminals and substrate 12 of the memory cell 510 for performing read, erase, and programming operations: Figure 4: Operation of flash memory cell 510 as shown in Figure 5 CG BL SL substrate Read 2-5V 0.6 - 2V 0V 0V erase -8 to -10V/0V FLT FLT 8-10V / 15-20V Programming 8-12V 3-5V 0V 0V

The methods and techniques described herein can be applied to other non-volatile memory technologies, such as FinFET gate flash or stacked gate flash memory, NAND flash, silicon-oxide-nitride-oxide-silicon (SONOS, charge trapping in nitride), metal-oxide-nitride-oxide-silicon (MONOS, metal charge trapping in nitride), resistive RAM (ReRAM), phase-change memory (PCM), magnetic RAM (MRAM), ferroelectric RAM (FeRAM), charge trapping (CT) memory, carbon nanotube (CN) memory, two-level or multi-level one-time programmable (OTP) and related electronic RAM (CeRAM).

To utilize a memory array comprising one of the nonvolatile memory cell types described above in Artificial Neural Networks, two modifications are made. First, the lines are configured such that each memory cell can be individually programmed, erased, and read without adversely affecting the memory state of other memory cells in the array, as explained further below. Second, continuous (analogous) programming of memory cells is provided.

Specifically, the memory state (i.e., the charge on the floating gate) of each memory cell in the array can independently and continuously change from a completely erased state to a fully programmed state, and vice versa, with minimal interference to other memory cells. This means that the cell memory can effectively analog to or at least store one of many discrete values (such as 16 or 64 different values), allowing for extremely precise and individual tuning of all memory cells in the memory array, and making the memory array ideal for storing and fine-tuning the synaptic weights of the neural network. Neural networks employing non-volatile memory cell arrays

Figure 6 conceptually illustrates a non-limiting example of a neural network utilizing the non-volatile memory array of the present invention. This example uses a non-volatile memory array neural network for facial recognition applications, but any other suitable application can be implemented using a neural network based on a non-volatile memory array.

S0 is the input layer. In this example, the input layer is a 32×32 pixel RGB image with 5-bit precision (i.e., three 32×32 pixel arrays, one array for each color (R, G, and B), and each pixel with 5-bit precision). In some cases, different weight sets are applied to the synapse CB1 from the input layer S0 to layer C1, while in other cases, the weights are shared, and the input image is scanned with a 3×3 pixel overlay filter (core), shifting the filter by 1 pixel (or more than 1 pixel, as specified by the model). Specifically, the values of nine pixels in a 3×3 portion of the image (i.e., the filter or core) are provided to synapse CB1. These nine input values are multiplied by appropriate weights, and after summing the output of the multiplication, a single output value is determined and provided by the first synapse of CB1 to generate a pixel in the feature map of layer C1. The 3×3 filter is then shifted one pixel to the right within the input layer S0 (i.e., a row of three pixels is added on the right and a row of three pixels is discarded on the left), thereby providing nine pixel values in this newly positioned filter to synapse CB1, where these pixel values are multiplied by the same weights, and a second single output value is determined by the connecting synapse. This procedure continues for all three colors and for all bits (accuracy values) until the 3×3 filter has scanned the entire 32×32 pixel image of the input layer S0. The procedure is then repeated using different weight sets to generate different feature maps for layer C1 until all feature maps for layer C1 have been calculated.

In this example, there are 16 feature maps in layer C1, each with 30×30 pixels. Each pixel is a new feature pixel extracted by multiplying the input by the kernel, and therefore each feature map is a two-dimensional array. Thus, in this example, layer C1 constitutes 16 layers of two-dimensional arrays (it should be noted that the layers and arrays mentioned in this article are logical relationships, not necessarily physical relationships—that is, the array is not necessarily oriented in a physical two-dimensional array). Each of the 16 feature maps in layer C1 is generated from one of sixteen different sets of synaptic weights applied to the filter scan. The C1 feature maps can all target different forms of the same image feature, such as boundary recognition. For example, the first image (generated using the first weight set, shared by all scans used to generate this first image) can identify circular edges, while the second image (generated using the second weight set, which is different from the first weight set) can identify rectangular edges, or the aspect ratio of certain features, etc.

The excitation function P1 (pooling) is applied before entering layer S1 from layer C1. Its pooling is derived from the values of consecutive, non-overlapping 2×2 regions in each feature map. The purpose of pooling function P1 is to average the values of nearby locations (or, alternatively, use a max function) to, for example, reduce the dependency of edge locations and decrease the data size before moving to the next stage. At layer S1, there are 16 15×15 feature maps (i.e., sixteen different arrays, each with 15×15 pixels). The synapse CB2, which allows entry from layer S1 into layer C2, scans the map in layer S1 using a 4×4 filter, with the filter shifted by one pixel. At layer C2, there are 22 12×12 feature maps. The activation function P2 (pooling) is applied before entering layer S2 from layer C2, and its pooling is derived from the values of continuous, non-overlapping 2×2 regions in each feature map. At layer S2, there are 22 6×6 feature maps. The activation function (pooling) is applied at the synapse CB3 where layer S2 enters layer C3, where each neuron in layer C3 is connected to each feature map in layer S2 via individual synapses in CB3. At layer C3, there are 64 neurons. The synapse CB4 where layer C3 enters the output layer S3 completely connects C3 to S3; that is, each neuron in layer C3 is connected to every neuron in layer S3. The output at S3 includes 10 neurons, with the highest output neuron determining the category. This output can, for example, indicate the identification or classification of the content of the original image.

Each synaptic layer is implemented using arrays or portions of nonvolatile memory cells.

Figure 7 is a block diagram of the array that can be used for that purpose. The Vector Matrix Multiplication (VMM) array 32 includes non-volatile memory cells and serves as synapses between layers (such as CB1, CB2, CB3, and CB4 in Figure 6). Specifically, the VMM array 32 includes a non-volatile memory cell array 33, an erase gate and character line gate decoder 34, a control gate decoder 35, a bit line decoder 36, and a source line decoder 37, which decode the respective inputs of the non-volatile memory cell array 33. The inputs to the VMM array 32 can come from the erase gate and character line gate decoder 34 or from the control gate decoder 35. In this example, the source line decoder 37 also decodes the output of the non-volatile memory cell array 33. Alternatively, the bit line decoder 36 can decode the output of the non-volatile memory cell array 33.

The non-volatile memory cell array 33 serves two purposes. First, it stores the weights that will be used by the VMM array 32. Second, the non-volatile memory cell array 33 efficiently multiplies the input by the weights stored in the non-volatile memory cell array 33 and adds the results along the output lines (source lines or bit lines) to produce an output that will be the input to the next layer or the input to the final layer. By performing multiplication and addition functions, the non-volatile memory cell array 33 eliminates the need for separate multiplication and addition logic circuits and is also power efficient due to its in-situ memory calculations.

The output of the nonvolatile memory cell array 33 is supplied to a differential summer (such as a summing operational amplifier or a summing current mirror) 38, which sums the output of the nonvolatile memory cell array 33 to produce a single value for its convolution. The differential summer 38 is configured to perform summation of positive and negative weights.

The total output of the difference summer 38 is then supplied to the excitation function block 39, which rectifies the output. The excitation function block 39 can provide a sigmoid, hyperbolic tangent, or ReLU function. The rectified output of the excitation function block 39 becomes an element of the feature map of the next layer (e.g., C1 in Figure 6), and is then applied to the next synapse to produce the next feature map layer or the final layer. Therefore, in this example, the nonvolatile memory cell array 33 constitutes a plurality of synapses (which receive inputs from the previous neuronal layer or from an input layer such as an image database), and the summing operational amplifier 38 and the excitation function block 39 constitute a plurality of neurons.

The inputs (WLx, EGx, CGx, and optional BLx and SLx) to the VMM array 32 in Figure 7 can be analog, binary, or digital (in which case, the DAC is set to convert digital to the appropriate input analog level), and the output can be analog, binary, or digital (in which case, the output ADC is set to convert the output analog level to digital).

Figure 8 is a block diagram depicting the use of the multiple layers of VMM array 32, denoted here as VMM arrays 32a, 32b, 32c, 32d, and 32e. As shown in Figure 8, the input of Inputx is converted from digital to analog by the digital-to-analog converter 31 and provided to the input VMM array 32a. The converted analog input can be voltage or current. The first-layer input D/A conversion can be performed using a function or lookup table (LUT) that maps the input Inputx to the analog level suitable for the matrix multiplier of the input VMM array 32a. Input conversion can also be performed using an analog-to-analog (A/A) converter to convert external analog inputs to mapped analog inputs to the input VMM array 32a.

The output generated by the input VMM array 32a is set as the input to the next VMM array (hidden level 1) 32b, which in turn generates an output, which is set as the input to the next VMM array (hidden level 2) 32c, and so on. The various layers of the VMM array 32 serve as different synaptic and neuron layers of a convolutional neural network (CNN). Each VMM array 32a, 32b, 32c, 32d, and 32e can be a single physical non-volatile memory array, or multiple VMM arrays can utilize different portions of the same physical non-volatile memory array, or multiple VMM arrays can utilize overlapping portions of the same physical non-volatile memory array. The example shown in Figure 8 contains five layers (32a, 32b, 32c, 32d, 32e): one input layer (32a), two hidden layers (32b, 32c), and two fully interconnected layers (32d, 32e). Those familiar with this technique should understand that this is merely an example, and the system may alternatively contain more than two hidden layers and more than two fully connected layers. Vector Matrix Multiplication (VMM) array

Figure 9 depicts a neuronal VMM array 900, which is particularly suitable for memory cells 310 as shown in Figure 3, and serves as a synaptic and neuronal portion between the input layer and the next layer. The VMM array 900 includes a memory array 901 of nonvolatile memory cells and a reference array 902 of nonvolatile reference memory cells (at the top of the array). Alternatively, another reference array may be placed at the bottom.

In the VMM array 900, control gate lines such as control gate line 903 extend vertically (therefore, the reference array 902 in the column direction is orthogonal to control gate line 903), and erase gate lines such as erase gate line 904 extend horizontally. Here, the inputs to the VMM array 900 are set on control gate lines (CG0, CG1, CG2, CG3), and the outputs of the VMM array 900 appear on source lines (SL0, SL1). In one example, only even-numbered columns are used, and in another example, only odd-numbered columns are used. The current placed on each source line (SL0, SL1) is summed against all currents from memory cells connected to that specific source line.

As described in this paper regarding neural networks, the nonvolatile memory cells of VMM array 900, namely memory cells 310 of VMM array 900, can be configured to operate in subcritical regions.

The nonvolatile reference memory cell and the nonvolatile memory cell described in this paper are biased in the weak inversion (subcritical region): Ids = Io * e ^{(Vg - Vth)/nVt} = w * Io * e ^(Vg)/nVt , where w = e ^{(- Vth)/nVt,} where Ids is the drain-to-source current; Vg is the gate voltage on the memory cell; Vth is the critical voltage of the memory cell; Vt is the thermoelectric pressure = k*T/q, where k is the Boltzmann constant, T is the temperature in Kelvin, and q is the electron charge; n is the slope factor = 1 + (Cdep/Cox), where Cdep = The capacitance of the depletion layer, and Cox is the capacitance of the gate oxide layer; Io is the memory cell current at the gate voltage equal to the threshold voltage, and Io is proportional to (Wt/L)*u*Cox* (n-1) * Vt ² , where u is the carrier mobility of the memory cell, and Wt and L are the width and length, respectively.

For an I-to-V logarithmic converter that uses memory cells (such as reference memory cells or peripheral memory cells) or transistors to convert input current into input voltage: Vg = n*Vt*log [Ids/wp*Io] where wp is the w of the reference or peripheral memory cell.

For a memory array used as a vector matrix multiplier (VMM) with current input, the output current is: Iout = wa * Io * e ^(Vg)/nVt , or Iout = (wa/wp) * Iin = W * Iin, W = e ^{(Vthp - Vtha)/nVt.} Here, wa = w of each memory cell in the memory array. Vthp is the effective threshold voltage of the peripheral memory cells, and Vtha is the effective threshold voltage of the main (data) memory cells. It should be noted that the threshold voltage of the transistor is a function of the substrate bias voltage, and the substrate bias voltage, expressed as Vsb, can be modulated to compensate for various conditions at this temperature. The critical voltage Vth can be expressed as: Vth = Vth0 + γ (SQRT |Vsb - 2*φF) - SQRT |2*φF |) where Vth0 is the critical voltage with zero substrate bias, φF is the surface potential, and γ is the volume effect parameter.

Character lines or control gates can be used to input memory cells that act on the input voltage.

Alternatively, the flash memory cells of the VMM array described herein can be configured to operate in a linear region: Ids = β* (Vgs-Vth)*Vds; β = u*Cox*Wt/L W = α (Vgs-Vth) This means that the weight W in the linear region is proportional to (Vgs-Vth).

Character lines, control gates, bit lines, or source lines can be used as inputs to memory cells operating in a linear region. Bit lines or source lines can be used as outputs to memory cells.

For I-to-V linear converters, memory cells (such as reference memory cells or peripheral memory cells) or transistors operating in the linear region can be used to linearly convert input/output currents into input/output voltages.

Alternatively, the memory cells of the VMM array described herein can be configured to operate in saturation regions: Ids = * β* (Vgs-Vth) ² ；β= u*Cox*Wt/L Wα (Vgs-Vth) ² , which means that the weight W is proportional to (Vgs-Vth) ² .

Character lines, control gates, or erase gates can be used as inputs to memory cells that operate in the saturation region. Bit lines or source lines can be used as outputs to output neurons.

Alternatively, the memory cells of the VMM array described herein can be used in all regions or combinations thereof (subcritical regions, linear regions or saturated regions) of any layer or multiple layers of a neural network.

Other examples of the VMM array 32 in Figure 7 are described in U.S. Patent No. 10,748,630, which is incorporated herein by reference. As described in that application, source lines or bit lines can be used as neural outputs (current summation outputs).

Figure 10 depicts a neuronal VMM array 1000, which is particularly suitable for memory cells 210 as shown in Figure 2 and serves as a synapse between the input layer and the next layer. The VMM array 1000 includes a memory array 1003 of nonvolatile memory cells, a reference array 1001 of first nonvolatile reference memory cells, and a reference array 1002 of second nonvolatile reference memory cells. The reference arrays 1001 and 1002, arranged in the row direction of the array, are used to convert the current input flowing into terminals BLR0, BLR1, BLR2, and BLR3 into voltage inputs WL0, WL1, WL2, and WL3. In practice, the first and second nonvolatile reference memory cells are diode-connected through-type multiplexers 1014 (partially depicted), into which current inputs flow. The reference cells are tuned (e.g., programmed) to a target reference level. The target reference level is provided by a reference miniature array matrix (not shown).

The memory array 1003 serves two purposes. First, it stores the weights used by the VMM array 1000 in its individual memory cells. Second, the memory array 1003 effectively multiplies the inputs (i.e., the current inputs provided in terminals BLR0, BLR1, BLR2, and BLR3, which are converted into input voltages by reference arrays 1001 and 1002 to supply word lines WL0, WL1, WL2, and WL3) by the weights stored in the memory array 1003, and then adds all the results (memory cell currents) to produce an output on the individual bit lines (BL0 to BLN), which will be the input to the next layer or the input to the final layer. By performing multiplication and addition functions, the memory array 1003 eliminates the need for separate multiplication and addition logic circuits and is also power-efficient. Here, voltage inputs are located on word lines WL0, WL1, WL2, and WL3, and outputs appear on individual bit lines BL0 to BLN during read (inference) operations. The current placed on each of the bit lines BL0 to BLN performs a summation function on the current from all non-volatile memory cells connected to the bit position lines.

Table 5 depicts the operating voltages and currents used in the VMM array 1000. The row indicators in the table are set to the following voltages: word lines for selected cells, word lines for unselected cells, bit lines for selected cells, bit lines for unselected cells, source lines for selected cells, and source lines for unselected cells. The table also shows the operations for reading, erasing, and programming the row indicators. Table 5: Operation of the VMM array 1000 in Figure 10 WL WL - Not selected BL BL - Not Selected SL SL - Not selected Read 1-3.5V -0.5V/0V 0.6-2V (Ineuron) 0.6V-2V/0V 0V 0V erase ~5-13V 0V 0V 0V 0V 0V Programming 1-2V -0.5V/0V 0.1-3 uA Vinh ~2.5V 4-10V 0-1V/FLT

Figure 11 depicts a neuronal VMM array 1100, which is particularly suitable for memory cells 210 as shown in Figure 2, and serves as a synaptic and neuronal portion between the input layer and the next layer. The VMM array 1100 includes a memory array 1103 of nonvolatile memory cells, a reference array 1101 of first nonvolatile reference memory cells, and a reference array 1102 of second nonvolatile reference memory cells. Reference arrays 1101 and 1102 extend in the column direction of the VMM array 1100. The VMM array is similar to VMM 1000, except that in the VMM array 1100, the character lines extend in the vertical direction. Here, the inputs are set on the character lines (WLA0, WLB0, WLA1, WLB1, WLA2, WLB2, WLA3, WLB3), and the outputs appear on the source lines (SL0, SL1) during read operations. The current placed on each source line performs a summation function on all currents from the memory cells connected to that particular source line.

Table 6 depicts the operating voltages and currents used in the VMM array 1100. The row indicators in the table are set to the following voltages: word lines for selected cells, word lines for unselected cells, bit lines for selected cells, bit lines for unselected cells, source lines for selected cells, and source lines for unselected cells. The table also shows the operations for reading, erasing, and programming the row indicators. Table 6: Operation of the VMM array 1100 in Figure 11 WL WL - Not selected BL BL - Not Selected SL SL - Not selected Read 1-3.5V -0.5V/0V 0.6-2V 0.6V-2V/0V ~0.3-1V (Ineuron) 0V erase ~5-13V 0V 0V 0V 0V SL - Prohibited (~4-8V) Programming 1-2V -0.5V/0V 0.1-3 uA Vinh ~2.5V 4-10V 0-1V/FLT

Figure 12 depicts a neuronal VMM array 1200, which is particularly suitable for memory cells 310 as shown in Figure 3, and serves as a synaptic and neuronal portion between the input layer and the next layer. The VMM array 1200 includes a memory array 1203 of nonvolatile memory cells, a reference array 1201 of first nonvolatile reference memory cells, and a reference array 1202 of second nonvolatile reference memory cells. Reference arrays 1201 and 1202 are used to convert current inputs flowing into terminals BLR0, BLR1, BLR2, and BLR3 into voltage inputs CG0, CG1, CG2, and CG3. In practice, the first and second nonvolatile reference memory cells are diode-connected through-multiplexers 1212 (partially shown), with current input flowing into these multiplexers via BLR0, BLR1, BLR2, and BLR3. Each multiplexer 1212 includes a separate multiplexer 1205 and a cascaded transistor 1204 to ensure a constant voltage on the bit lines (such as BLR0) of each of the first and second nonvolatile reference memory cells during read operations. The reference cell is tuned to a target reference level.

The memory array 1203 serves two purposes. First, it stores the weights to be used by the VMM array 1200. Second, the memory array 1203 effectively multiplies the inputs (current inputs provided to terminals BLR0, BLR1, BLR2, and BLR3, wherein reference arrays 1201 and 1202 convert these current inputs into input voltages to supply to the control gates (CG0, CG1, CG2, and CG3)) by the weights stored in the memory array, and then adds all the results (cell currents) to produce an output, which is displayed on BL0 to BLN and will be the input to the next layer or to the final layer. By performing multiplication and addition functions, the memory array eliminates the need for separate multiplication and addition logic circuits and is also power efficient. Here, the inputs are located on the control gate lines (CG0, CG1, CG2, and CG3), and the outputs appear on the bit lines (BL0 to BLN) during read operations. The current placed on each bit line performs a summation function on all currents from the memory cells connected to the bit positioner lines.

The VMM array 1200 performs unidirectional modulation on the non-volatile memory cells in the memory array 1203. That is, each non-volatile memory cell is erased and then partially programmed until the desired charge is reached on the floating gate. If too much charge is placed on the floating gate (causing an error value to be stored in the cell), the sequence of cell erasure and partial programming operations restarts. As shown, two columns sharing the same erase gate (e.g., EG0 or EG1) are erased together (this is called page erasure), and thereafter, each cell is partially programmed until the desired charge is reached on the floating gate.

Table 7 depicts the operating voltages and currents used in the VMM array 1200. The row indicators in this table are set to the following voltages: word lines for selected cells, word lines for unselected cells, bit lines for selected cells, bit lines for unselected cells, control gate for selected cells, control gate for unselected cells in the same sector as the selected cell, control gate for unselected cells in a different sector from the selected cell, erase gate for selected cells, erase gate for unselected cells, source lines for selected cells, and source lines for unselected cells. The column indicators handle read, erase, and programmable operations. Table 7: Operation of VMM Array 1200 in Figure 12 WL WL - Not selected BL BL - Not Selected CG CG - Not Selected for the Same Sector CG - Not Selected EG EG - Not selected SL SL - Not selected Read 1.0-2V -0.5V/ 0V 0.6-2V (Ineuron) 0V 0-2.6V 0-2.6V 0-2.6V 0-2.6V 0-2.6V 0V 0V erase 0V 0V 0V 0V 0V 0-2.6V 0-2.6V 5-12V 0-2.6V 0V 0V Programming 0.7-1V -0.5V/ 0V 0.1-1uA Vinh (1-2V) 4-11V 0-2.6V 0-2.6V 4.5-5V 0-2.6V 4.5-5V 0-1V

Figure 13 depicts a neuronal VMM array 1300, which is particularly suitable for memory cells 310 as shown in Figure 3, and serves as a synaptic and neuronal portion between the input layer and the next layer. The VMM array 1300 includes a memory array 1303 of nonvolatile memory cells, a reference array 1301 of first nonvolatile reference memory cells, and a reference array 1302 of second nonvolatile reference memory cells. EG lines EGR0, EG0, EG1, and EGR1 extend vertically, while CG lines CG0, CG1, CG2, and CG3 and WL lines WL0, WL1, WL2, and WL3 extend horizontally. VMM array 1300 is similar to VMM array 1200, except that VMM array 1300 implements bidirectional tuning. Because it uses a separate EG line, individual cells can be completely erased, partially programmed, or partially erased as needed to achieve the desired charge on the floating gate. As shown, reference arrays 1301 and 1302 convert the input current in terminals BLR0, BLR1, BLR2, and BLR3 into control gate voltages CG0, CG1, CG2, and CG3 to be applied to the memory cells in the column direction (operated via diode-connected reference cells through multiplexer 1314). The current output (neuron) is in bit lines BL0 to BLN, where each bit line sums all currents from nonvolatile memory cells connected to the bit line.

Table 8 depicts the operating voltages and currents used in the VMM array 1300. The row indicators in this table are set to the following voltages: word lines for selected cells, word lines for unselected cells, bit lines for selected cells, bit lines for unselected cells, control gate for selected cells, control gate for unselected cells in the same sector as the selected cell, control gate for unselected cells in a different sector from the selected cell, erase gate for selected cells, erase gate for unselected cells, source lines for selected cells, and source lines for unselected cells. The column indicators handle read, erase, and programmable operations. Table 8: Operation of VMM Array 1300 (Figure 13) WL WL - Not selected BL BL - Not Selected CG CG - Not Selected for the Same Sector CG - Not Selected EG EG - Not selected SL SL - Not selected Read 1.0-2V -0.5 V/0V 0.6-2V (Ineuron) 0V 0-2.6V 0 -2.6V 0-2.6V 0-2.6V 0-2.6V 0V 0V erase 0V 0V 0V 0V 0V 4-9V 0-2.6V 5-12V 0-2.6V 0V 0V Programming 0.7-1V -0.5 V/0V 0.1-1uA Vinh (1-2V) 4-11V 0-2.6V 0-2.6V 4.5-5V 0-2.6V 4.5-5V 0-1V

Figure 22 depicts a neuronal VMM array 2200, which is particularly suitable for memory cell 210 as shown in Figure 2, and serves as a synapse and neuronal portion between the input layer and the next layer. In the VMM array 2200, inputs INPUT ₀ , ..., INPUT _N are received on bit lines BL0, ..., BLN, respectively, and outputs OUTPUT ₁ , OUTPUT ₂ , OUTPUT ₃ , and OUTPUT ₄ are generated on source lines SL0, SL1, SL2, and SL3, respectively.

Figure 23 depicts a neuronal VMM array 2300, which is particularly suitable for memory cell 210 as shown in Figure 2, and serves as a synapse and neuronal portion between the input layer and the next layer. In this example, inputs INPUT ₀ , INPUT ₁ , INPUT ₂ , and INPUT ₃ are received on source lines SL0, SL1, SL2, and SL3, respectively, and outputs OUTPUT ₀ , ..., OUTPUT _N are generated on bit lines BL0, ..., BLN.

Figure 24 depicts a neuronal VMM array 2400, which is particularly suitable for memory cell 210 as shown in Figure 2, and serves as a synapse and neuronal portion between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _M are received on word lines WL0, ..., WL _M , respectively, and outputs OUTPUT ₀ , ..., OUTPUT _N are generated on bit lines BL0, ..., BLN.

Figure 25 depicts a neuronal VMM array 2500, which is particularly suitable for memory cell 310 as shown in Figure 3, and serves as a synapse and neuronal portion between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _M are received on word lines WL0, ..., WL _M , respectively, and outputs OUTPUT ₀ , ..., OUTPUT _N are generated on bit lines BL0, ..., BLN.

Figure 26 depicts a neuronal VMM array 2600, which is particularly suitable for memory cell 410 as shown in Figure 4, and serves as a synapse and neuronal portion between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _N are received on vertical control gate lines CG0, ..., CG _N , respectively, and outputs OUTPUT ₁ and OUTPUT ₂ are generated on source lines SL0 and SL1.

Figure 27 depicts a neuronal VMM array 2700, which is particularly suitable for memory cell 410 as shown in Figure 4, and serves as a synapse and neuronal portion between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _N are received at the gates of bit line control gates 2701-1, 2701-2, ..., 2701-(N-1) and 2701-N, respectively, which are coupled to bit lines BL0, ..., BLN. Example outputs OUTPUT ₁ and OUTPUT ₂ are generated on source lines SL0 and SL1.

Figure 28 depicts a neuronal VMM array 2800, which is particularly suitable for memory cells 310 as shown in Figure 3, memory cells 510 as shown in Figure 5, and memory cells 710 as shown in Figure 7, and serves as synapses and neuronal portions between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _M are received on word lines WL0, ..., WL _M , and outputs OUTPUT _{0 ,} ..., OUTPUT _N are generated on bit lines BL0, ..., BLN, respectively.

Figure 29 depicts a neuronal VMM array 2900, which is particularly suitable for memory cells 310 as shown in Figure 3, memory cells 510 as shown in Figure 5, and memory cells 710 as shown in Figure 7, and serves as the synaptic and neuronal portion between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _M are received on control gate lines CG0, ..., _CGM . Outputs OUTPUT ₀ , ..., OUTPUT _N are generated on vertical source lines SL0, ..., SLN, respectively, wherein each source line _SLi is coupled to the source line of all memory cells in row i.

Figure 30 depicts a neuronal VMM array 3000, which is particularly suitable for memory cells 310 as shown in Figure 3, memory cells 510 as shown in Figure 5, and memory cells 710 as shown in Figure 7, and serves as the synapse and neuronal portion between the input layer and the next layer. In this example, inputs INPUT ₀ , ..., INPUT _M are received on control gate lines CG0, ..., _CGM . Outputs OUTPUT ₀ , ..., OUTPUT _N are generated on vertical bit lines BL0, ..., BLN, respectively, where each bit line _BLi is coupled to the bit lines of all memory cells in row i. Long Short-Term Memory (LSTM)

Previous technologies included a concept known as Long Short-Term Memory (LSTM). LSTM cells are commonly used in neural networks. LSTM allows neural networks to remember information at predetermined time intervals and use that information in subsequent operations. A known LSTM cell consists of a cell, an input gate, an output gate, and a forget gate. These three gates regulate the flow of information into and out of the cell and the time intervals at which information is remembered in the LSTM. Virtual Memory Models (VMMs) are particularly well-suited for LSTM cells.

Figure 14 illustrates an example LSTM 1400. This example LSTM 1400 includes cells 1401, 1402, 1403, and 1404. Cell 1401 receives the input vector _x0 and produces the output vector _h0 and cell state vector _c0 . Cell 1402 receives the input vector _x1 , the output vector (hidden state) _h0 from cell 1401, and the cell state _c0 from cell 1401, and produces the output vector _h1 and cell state vector _c1 . Cell 1403 receives the input vector _x2 , the output vector (hidden state) _h1 from cell 1402, and the cell state _c1 from cell 1402, and produces the output vector _h2 and cell state vector _c2 . Cell 1404 receives input vector _x3 , output vector (hidden state) _h2 from cell 1403, and cell state _c2 from cell 1403, and produces output vector _h3 . Additional cells can be used, and an LSTM with four cells is just an example.

Figure 15 illustrates an example implementation of LSTM cell 1500, which can be used in cells 1401, 1402, 1403 and 1404 in Figure 14. LSTM cell 1500 receives input vector x(t), cell state vector c(t-1) from the previous cell and output vector h(t-1) from the previous cell, and produces cell state vector c(t) and output vector h(t).

LSTM cell 1500 includes sigmoid function components 1501, 1502, and 1503, each of which uses numbers between 0 and 1 to control the amount of each component of the input vector allowed to pass through the output vector. LSTM cell 1500 also includes hyperbolic tangent components 1504 and 1505 for applying a hyperbolic tangent function to the input vector, multiplier components 1506, 1507, and 1508 for multiplying two vectors together, and adder component 1509 for adding two vectors together. The output vector h(t) can be provided to the next LSTM cell in the system or accessed for other purposes.

Figure 16 depicts LSTM cell 1600, which is an example of an implementation of LSTM cell 1500. For ease of reading, the same designations from LSTM cell 1500 are used in LSTM cell 1600. S-shaped function components 1501, 1502, and 1503, and hyperbolic tangent component 1504 each contain multiple VMM arrays 1601 and excitation function blocks 1602. Therefore, it can be seen that VMM arrays are particularly suitable for LSTM cells used in certain neural network systems. Multiplier components 1506, 1507, and 1508, and adder component 1509 are implemented digitally or analogously. Excitation function block 1602 can be implemented digitally or analogously.

An alternative to LSTM cell 1600 (and another example of an implementation of LSTM cell 1500) is shown in Figure 17. In Figure 17, sigmoid function components 1501, 1502, and 1503 and hyperbolic tangent component 1504 share the same physical hardware (VMM array 1701 and excitation function block 1702) in a time-multiplexed manner. The LSTM cell 1700 also includes: a multiplier component 1703 for multiplying two vectors together; an adder component 1708 for adding two vectors together; a hyperbolic tangent component 1505 (containing the excitation function block 1702); a register 1707 for storing the value i(t) when i(t) is output from the sigmoid function block 1702; a register 1704 for storing the value f(t) * c(t-1) when it is output from the multiplier component 1703 via the multiplexer 1710; and a register 1705 for storing the value i(t) * u(t) stores its value when it is output by multiplexer 1710 and multiplier component 1703; and register 1706 is used to store the value o(t) * c~(t) when it is output by multiplexer 1710 and multiplexer 1709 and multiplier component 1703.

LSTM cell 1600 contains multiple sets of VMM array 1601 and individual excitation function blocks 1602, while LSTM cell 1700 contains only one set of VMM array 1701 and excitation function blocks 1702, which is used to represent multiple layers in an instance of LSTM cell 1700. LSTM cell 1700 will require less space than LSTM 1600 because LSTM cell 1700 will require 1/4 of the space for VMM and excitation function blocks compared to LSTM cell 1600.

As can be further understood, an LSTM cell will typically contain multiple VMM arrays, each utilizing functionality provided by certain circuit blocks outside the VMM arrays, such as summer and excitation function blocks, and high-voltage generation blocks. Providing separate circuit blocks for each VMM array would require a significant amount of space within the semiconductor device and would be somewhat inefficient. Therefore, the examples described below reduce the circuitry used outside the VMM arrays themselves. Gate Return Unit

Analogous VMM implementations can be used in gate recursive unit (GRU) systems. A GRU is a gate control mechanism in a recursive neural network. A GRU is similar to an LSTM, except that a GRU cell typically contains fewer components than an LSTM cell.

Figure 18 illustrates an example GRU 1800. This example GRU 1800 includes cells 1801, 1802, 1803, and 1804. Cell 1801 receives input vector _x0 and produces output vector _h0 . Cell 1802 receives input vector _x1 and output vector _h0 from cell 1801, and produces output vector _h1 . Cell 1803 receives input vector _x2 and output vector _h1 (in the hidden state) from cell 1802, and produces output vector _h2 . Cell 1804 receives input vector _x3 and output vector _h2 (in the hidden state) from cell 1803, and produces output vector _h3 . Additional cells can be used, and a GRU with four cells is just one example.

Figure 19 illustrates an example implementation of GRU cell 1900, which can be used in cells 1801, 1802, 1803, and 1804 of Figure 18. GRU cell 1900 receives an input vector x(t) and an output vector h(t-1) from the previous GRU cell, and generates an output vector h(t). GRU cell 1900 includes sigmoid function components 1901 and 1902, each of which applies numbers between 0 and 1 to the components of the output vector h(t-1) and the input vector x(t). The GRU cell 1900 also includes a hyperbolic tangent component 1903 for applying the hyperbolic tangent function to the input vector, multiple multiplier components 1904, 1905 and 1906 for multiplying two vectors together, an adder component 1907 for adding two vectors together, and a complementary component 1908 for subtracting the input from 1 to produce the output.

Figure 20 depicts GRU cell 2000, which is an example of an implementation of GRU cell 1900. For the convenience of the reader, the same designations from GRU cell 1900 are used in GRU cell 2000. As can be seen in Figure 20, the sigmoid function components 1901 and 1902 and the hyperbolic tangent component 1903 each contain multiple VMM arrays 2001 and excitation function blocks 2002. Therefore, it can be seen that VMM arrays are particularly used in GRU cells used in some neural network systems. Multiplier components 1904, 1905, and 1906, adder component 1907, and complementary component 1908 are implemented digitally or analogously. Excitation function block 2002 can be implemented digitally or analogously.

An alternative to GRU cell 2000 (and another example of the implementation of GRU cell 1900) is shown in Figure 21. In Figure 21, GRU cell 2100 utilizes a VMM array 2101 and an excitation function block 2102, which, when configured as a sigmoid function, uses numbers between 0 and 1 to control the amount by which each component of the input vector is allowed to pass through the output vector. In Figure 21, the sigmoid function components 1901 and 1902 and the hyperbolic tangent component 1903 share the same physical hardware (VMM array 2101 and excitation function block 2102) in a time-multiplexed manner. The GRU cell 2100 also includes: a multiplier component 2103 for multiplying two vectors together; an adder component 2105 for adding two vectors together; a complement component 2109 for subtracting the input from 1 to produce the output; a multiplexer 2104; a register 2106 for retaining the value h(t-1) * r(t) when it is output from the multiplier component 2103 via the multiplexer 2104; a register 2107 for retaining the value h(t-1) * z(t) when it is output from the multiplier component 2103 via the multiplexer 2104; and a register 2108 for retaining the value h^(t) * (1-z(t)) when it is output from the multiplier component 2103 via the multiplexer 2104.

GRU cell 2000 contains multiple sets of VMM array 2001 and excitation function blocks 2002, while GRU cell 2100 contains a set of VMM array 2101 and excitation function blocks 2102, which are used to represent multiple layers in an instance of GRU cell 2100. GRU cell 2100 will require less space than GRU cell 2000 because GRU cell 2100 will require 1/3 of the space for VMM and excitation function blocks.

As can be further understood, a GRU system will typically comprise multiple VMM arrays, each utilizing functionality provided by certain circuit blocks external to the VMM arrays, such as summer and excitation function blocks, and high-voltage generation blocks. Providing separate circuit blocks for each VMM array would require a significant amount of space within the semiconductor device and would be somewhat inefficient. Therefore, the examples described below reduce the circuitry used externally to the VMM arrays themselves.

The inputs to the VMM array can be analog levels, binary levels, pulses, time-modulated pulses, or digital bits (in which case, a DAC is used to convert the digital bits to the appropriate input analog level), and the outputs can be analog levels, binary levels, timed pulses, pulses, or digital bits (in which case, an output ADC is used to convert the output analog level to digital bits).

Generally, for each memory cell in a VMM array, each weight W can be implemented by a single memory cell, a differential cell, or two mixed memory cells (the average of the two cells). In the case of differential cells, two memory cells are used to implement the weight W as a differential weight (W = W+ - W-). In the case of two mixed memory cells, two memory cells are required to implement the weight W as the average of the two cells.

Figure 31 illustrates a VMM system 3100. In some embodiments, the weights W stored in the VMM array are stored as difference pairs W+ (positive weights) and W- (negative weights), where W = (W+) - (W-). In the VMM system 3100, half of the bit lines are designated as W+ lines, i.e., bit lines connected to memory cells that will store positive weights W+, and the other half of the bit lines are designated as W- lines, i.e., bit lines connected to memory cells that implement negative weights W-. The W- lines are interspersed among the W+ lines. Subtraction operations are performed by summing circuits that receive current from the W+ and W- lines, such as summing circuits 3101 and 3102. The outputs of the W+ line and the W- line are combined to effectively derive W = W+ - W- for each pair of (W+, W-) cells of all pairs of (W+, W-) lines. Although the alternating arrangement of the W- line among the W+ lines has been described above, in other instances, the W+ and W- lines can be positioned arbitrarily anywhere in the array.

Figure 32 illustrates another example. In the VMM system 3210, the positive weight W+ is implemented in the first array 3211 and the negative weight W- is implemented in the second array 3212, which is separate from the first array, and the resulting weights are appropriately combined by the summing circuit 3213.

Figure 33 depicts a VMM system 3300. The weights W stored in the VMM array are stored as difference pairs W+ (positive weights) and W- (negative weights), where W = (W+) - (W-). The VMM system 3300 includes arrays 3301 and 3302. Half of the bit lines in each of arrays 3301 and 3302 are designated as W+ lines, i.e., bit lines connected to memory cells where positive weights W+ will be stored, and the other half of the bit lines in each of arrays 3301 and 3302 are designated as W- lines, i.e., bit lines connected to memory cells where negative weights W- are implemented. W- lines are interspersed among the W+ lines in an alternating manner. The subtraction operation is performed by summing circuits that receive current from the W+ and W- lines, such as summing circuits 3303, 3304, 3305, and 3306. The outputs of the W+ and W- lines from each of arrays 3301 and 3302 are combined to effectively derive W = W+ - W- for each pair of (W+, W-) cells of all pairs of (W+, W-) lines. Furthermore, the W values from each array 3301 and 3302 can be further combined by summing circuits 3307 and 3308 so that each W value is the result of subtracting the W value from array 3302 from the W value from array 3301. This means that the final result from summing circuits 3307 and 3308 is the difference between two difference values.

This analogy is used to describe the non-volatile memory cells in an analogy to be erased and programmed to maintain a very specific and precise amount of charge, i.e., the number of electrons, in a floating gate. For example, each floating gate can maintain one of N different values, where N is the number of different weights that can be indicated by each cell. Examples of N include 16, 32, 64, 128, and 256.

Previous technology systems required a considerable area and involved a considerable latency at the output stage. For example, multiple clock cycles were used to convert the analog current received from the VMM array into digital output data.

It is necessary to reduce the latency at the output to increase the overall operating speed of the system, which represents some or all of an artificial neural network.

Numerous examples of output circuits and interconnection methods used in neural network arrays are revealed.

VMM System Architecture Figure 34 depicts a block diagram of a VMM system 3400. The VMM system 3400 includes a VMM array 3401, a column decoder 3402, a high-voltage decoder 3403, a row decoder 3404, a bit line driver 3405 (such as for a programmable bit line control circuit system), an input circuit 3406, an output circuit 3407, a control logic 3408, and a bias generator 3409. The VMM system 3400 further includes a high-voltage generation block 3410, which includes a charge pump 3411, a charge pump regulator 3412, and a high-voltage level generator 3413. The VMM system 3400 further includes a (programmed/erase or weighted tuning) algorithm controller 3414, an analog circuit system 3415, a control engine 3416 (which may include, but is not limited to, functions such as arithmetic functions, excitation functions, embedded microcontroller logic), a test control logic 3417, and a static random access memory (SRAM) block 3418 for storing data such as intermediate data for input circuits (e.g., excitation data) or intermediate data for output circuits (neuron output data, partial and output neuron data) or for programmed data inputs (e.g., for a whole column or for multiple columns of data inputs).

Input circuitry 3406 may include circuitry such as a DAC (digital-to-analog converter), DPC (digital-to-pulse converter, digital-to-time-modulated pulse converter), AAC (analog-to-analog converter, such as a current-to-voltage converter, logarithmic converter), PAC (pulse-to-analog level converter), or any other type of converter. Input circuitry 3406 may implement one or more of normalization, linear or nonlinear scaling/scaling functions, or arithmetic functions. Input circuitry 3406 may implement a temperature compensation function for the input level. Input circuitry 3406 may implement excitation functions such as ReLU or S-type excitation functions. Input circuit 3406 can store digital excitation data to be applied as an input signal or combined with an input signal during programming or read operations. The digital excitation data can be stored in a register. Input circuit 3406 may include circuitry for driving array terminals such as CG, WL, EG, and SL lines, and may include sample-and-hold circuitry and a buffer. A DAC can be used to convert the digital excitation data into an analog input voltage to be applied to the array.

Output circuit 3407 may include circuits such as ITV (current-to-voltage circuit), ADC (analog-to-digital converter, used to convert analog neuron outputs to digital bits), AAC (analog-to-analog converter, such as current-to-voltage converter, logarithmic converter), APC (analog-to-pulse converter, analog-to-time-modulated pulse converter), or any other type of converter. Output circuit 3407 can convert array outputs into excitation data. Output circuit 3407 can implement excitation functions such as rectified linear excitation function (ReLU) or sigmoid function. Output circuit 3407 may implement one or more of the following for neural outputs: statistical normalization, regularization, scaling/scaling/gain functions, statistical rounding, or arithmetic functions (e.g., addition, subtraction, division, multiplication, shifting, logarithmic). Output circuit 3407 may implement temperature compensation functions for neural outputs or array outputs (such as bit-line outputs) to keep the array's power consumption approximately constant over a temperature range or to improve the accuracy of the array (neuron) output, for example, by keeping the IV slope approximately the same over a temperature range. Output circuit 3407 may include registers for storing output data.

Figure 35 illustrates output circuit 3500. Output circuit 3500 is an example implementation of output circuit 3407 in Figure 34. Output circuit 3500 includes reference circuit 3501 and row circuit 3502. Multiple examples of row circuit 3502 are used for multiple rows in VMM array 3401, that is, for individual rows of row circuit 3502 in VMM array 3401. Eight examples of row circuit 3502 are shown in Figure 35, but it should be understood that there may be more examples and associated rows in a VMM system.

Reference circuit 3501 receives reference current, such as bit line current, from one or more reference rows of non-volatile memory cells. A reference row may be part of a VMM array 3401 (as shown) or may be located in a separate array. Reference circuit 3501 includes a row multiplexer 3503, a current-to-voltage converter 3504, and a reference generator 3505. If more than one reference row is connected to reference circuit 3501, the row multiplexer 3503 selects a row and supplies current from that row to the current-to-voltage converter 3504, which converts the received current into voltage. Reference generator 3505 generates a voltage reference VADCREF (such as VADCREFH and VADCREFL) used by the analog-to-digital converter (ADC) 3508, where the voltage reference VADCREF is based on the voltage output from the current-to-voltage converter 3504. For example, if ADC 3508 is a continuous approximation register (SAR) ADC, then reference generator 3505 generates voltages VREFP, VREFN, and VCIM (for example, for a full-scale input voltage of 0.3 V for the ADC, VREFP = 0.3 V, VREFN = 0 V, and VCIM = 0.15 V). Because the reference voltage of the ADC 3508 is generated from the voltage output of the current-to-voltage converter 3504, which is similar to that of the current-to-voltage converter 3507, the resolution of the ADC 3508 is maintained. This is because the ADC's reference voltage automatically tracks any changes caused by variations in the operating conditions (such as temperature changes) of the current-to-voltage converter 3504 and the current-to-voltage converter 3507.

Each row circuit 3502 receives current, such as bit line current, from one or more rows in the VMM array 3401. Each row circuit 3502 includes a row multiplexer 3506, a current-to-voltage converter 3507, and an ADC 3508. If more than one row is connected to the row circuit 3502, the row multiplexer 3506 selects a row and supplies the current from that row to the current-to-voltage converter 3507, which converts the received current into voltage, and the ADC 3508 converts the voltage into a digital output. For example, the row circuit 3502 outputs DOUT0x [n:0]. If only one row is connected to the row circuit 3502, the row multiplexer 3506 is selectable.

Figures 36A and 36B illustrate a clock generator for generating a fast clock signal for the circuit described herein, wherein the fast clock signal is a clock signal with a frequency faster than the system clock received from a source outside the VMM system.

Figure 36A depicts a clock generator 3600 that receives the system clock CLKIN (e.g., a system clock received from a source outside the VMM system), the enable signal EN, and the configuration bit CFGx, and uses a delay-locked loop (DLL) 3601 and a clock generator block 3602 to generate a faster clock CLKOUT with a different clock frequency depending on the configuration bit CFGx. Figure 36B depicts a clock generator 3650 that receives the system clock CLKIN and uses a phase-locked loop (PLL) 3651 and a clock generator block 3652 to generate a faster clock CLKOUT. DLL 3601 and PLL 3651 are used to generate precise clocks that are faster than the system clock CLKIN and synchronized with the input clock CLKIN, for example, clocks that are 2 to 100 times faster than the input system clock CLKIN. Clock generator blocks 3602 and 3652 are used to generate different clock frequencies using the CLK_INT response to the configuration bit CFGx.

Figure 37 illustrates an output circuit 3700 that can be used in the row circuit 3502 of Figure 35. The output circuit 3700 is a differential circuit, meaning that the circuit output is a function of two inputs. The output circuit 3700 includes a current-to-voltage converter (ITV) 3704 (first current-to-voltage converter), an ITV 3705 (second current-to-voltage converter), a differential input serial approximation register analog-to-digital converter (SAR ADC) 3701, transistors 3713, 3715, 3724 and 3726 (which form part of a row multiplexer, coupling one row of the first group of rows to ITV 3704 and one row of the second group of rows to ITV 3705), and current sources 3702 and 3703. Current source 3702 represents the current drawn from a row selected by the row multiplexer in the VMM array 3401, where the row stores the W+ value. Current source 3703 represents the current drawn from a row selected by the row multiplexer in the VMM array 3401, where the row stores the W- value. Output circuit 3700 receives currents IW+ (first current) and IW- (second current) from two differential rows W+ and W- respectively, and outputs a digital output DOUT indicating these currents, which is equal to W = W+ - W-.

In one alternative, the output circuit 3700 can be implemented as a single-ended circuit, meaning it uses one ITV (3704 or 3705) and a single-input ADC. In another alternative, the differential output can be achieved by using two ITVs, and the single-input ADC can be formed by combining the two results. In yet another alternative, the differential output can be achieved by time-multiplexing and by using the ITV and a single-input ADC by combining the two results in time.

The ITV 3704 includes switches 3706, 3707, and 3708; integrating capacitors 3710 and 3711; an NMOS series transistor 3712; and an operational amplifier 3714. Before a read operation, switches 3706, 3707, and 3708 are closed, causing the top and bottom plates of integrating capacitors 3710 and 3711 to charge to Vsup, and VIN+ equal to Vsup. During the integration cycle, switch 3706 is open. Current source 3702 draws current, causing the voltage of VIN+ (the first voltage) to be pulled down proportionally to the current drawn by current source 3702. That is, VIN+ will be equal to the initial value of VIN+ before the read operation minus the first discharge value caused by the first current IW+. After the integration cycle, the voltages on capacitors 3710 and 3711 are sampled into the SAR ADC 3701. After this sampling cycle, the ADC will begin the conversion of this sampled voltage into digital output bits. In one example, the voltage on one or more of capacitors 3710 and 3711 is buffered before entering the SAR ADC 3701. In another example, capacitor 3711 is a capacitor in the binary capacitor array of the SAR ADC 3701 (i.e., the ITV 3704 and the SAR ADC 3701 share a capacitor to save die space). In this case, after the integration cycle, switches 3707 and 3708 are turned off, and the SAR ADC 3701 begins the conversion of the voltage on capacitor 3711 into digital output bits.

Similarly, the ITV 3705 includes switches 3717, 3718, and 3719; capacitors 3721 and 3722; an NMOS series transistor 3723; and an operational amplifier 3725. Before a read operation, switches 3717, 3718, and 3719 are closed, causing the top and bottom plates of integrating capacitors 3721 and 3722 to charge to Vsup and VIN- equal to Vsup. During the integration cycle, switch 3717 is open. Current source 3703 draws current, causing the voltage at VIN- (the second voltage) to be pulled down proportionally to the current drawn by current source 3703. That is, VIN- will be equal to the initial value of VIN- before the read operation minus the first discharge value caused by the first current IW-. Capacitors 3721 and 3722 are similar to capacitors 3710 and 3711 in ITV 3704. Switches 3718 and 3719 are similar to switches 3707 and 3708 in ITV 3704. The operation of ITV 3705 with respect to current IW- is similar to that of ITV 3704.

The bit line adjustment circuit of the ITV 3704 includes an operational amplifier 3714, transistors 3712, 3713, and 3715, which are always turned on when a read operation is required for bit line IW+. This circuit applies a fixed bias to the bit line during the read operation. Specifically, it applies VREF to the bit line during the read operation, which is applied to the positive input terminal of the operational amplifier 3714, regardless of the amount of current drawn from bit line IW+.

Similarly, the bit line adjustment circuit of the ITV 3705 includes an operational amplifier 3725, a transistor 3723, and force and sensing transistors 3724 and 3726, all of which are turned on when a read operation is required for bit line IW+. During a read operation, this circuit applies VREF to the bit line, which is applied to the positive input terminal of the operational amplifier 3725, regardless of the amount of current drawn from bit line IW+.

The SAR ADC 3701 receives differential voltages VIN+ and VIN-, as well as reference voltages VADCREFH and VADCREFL, and generates a digital output DOUT[n:0] based on the difference between VIN+ and VIN-.

It is worth noting that integrating capacitors 3710 and 3721 will require a relatively large die space due to their relatively large size. Alternatively, integrating capacitors 3711 and 3722 are reused from the capacitor array of the SAR ADC 3701 to save on die area. When the SAR ADC 3701 begins switching, switches 3707, 3708, 3718, and 3719 are turned off.

Figure 38A illustrates two illustrative output circuits 3800 that can be used in the line circuit 3502 in Figure 35. The output circuit 3800 includes circuits 3801, 3802 and a common capacitor network 3813. Circuits 3801 and 3802 are the same as output circuit 3700 in Figure 37, except that the integrating capacitors 3710 and 3721 in output circuit 3700 have been replaced by common integrating capacitors 3803 (first integrating capacitor) and 3804 (second integrating capacitor) in common capacitor network 3813, respectively; and switches 3805, 3807, 3809 and 3811 are used in common capacitor network 3813 of circuit 3801; and switches 3806, 3808, 3810 and 3812 are used in capacitor network 3813 of circuit 3802. Circuit 3801 includes ITV 3823 (first current to voltage converter) coupled to IW1+ (first current) and ITV 3824 (second current to voltage converter) coupled to IW1- (second current). Circuit 3802 includes ITV 3825 (third current to voltage converter) coupled to IW2+ (third current) and ITV 3826 (fourth current to voltage converter) coupled to IW2- (fourth current). Compared to the two examples using output circuit 3700 in Figure 37, this design uses two fewer capacitors.

ITV 3823 generates a first voltage, and ITV 3824 generates a second voltage and is coupled to SAR ADC 3827 (first SAR ADC) which includes CDAC 3829. ITV 3825 generates a third voltage, and ITV 3826 generates a fourth voltage and is coupled to SAR ADC 3828 (second SAR ADC) which includes CDAC 3830. During the read operation, the first voltage will be equal to the initial voltage (the voltage at the same node before the read operation) minus the first discharge value caused by IW1+, the second voltage will be equal to the initial voltage (the voltage at the same node before the read operation) minus the second discharge value caused by IW1-, the third voltage will be equal to the initial voltage (the voltage at the same node before the read operation) minus the third discharge value caused by IW2+, and the fourth voltage will be equal to the initial voltage (the voltage at the same node before the read operation) minus the fourth discharge value caused by IW2-.

Therefore, the shared integrating capacitor 3803 (the first integrating capacitor) is shared by ITV 3823 and ITV 3826 in a time-multiplexed manner, and the shared integrating capacitor is shared by ITV 3824 and ITV 3826 in a time-multiplexed manner.

Figure 38B illustrates four exemplified output circuits 3850 that can be used in the output circuit 3502 in Figure 35. The output circuits 3850 have the same design as the output circuit 3800, but have 8 ITVs instead of 4 ITVs and 4 SAR ADCs instead of 2 SAR ADCs. They use a common capacitor network 3870 including common capacitors 3871 and 3872, and a series of switches for selectively coupling the common capacitors 3871 and 3872 to ITVs 3851, 3852, 3853, 3854, 3855, 3856, 3857 and 3858. ITV 3851 is coupled to bit line current IW1+ (first current), ITV 3852 is coupled to bit line current IW1- (second current), ITV 3853 is coupled to bit line current IW2+ (third current), ITV 3854 is coupled to bit line current IW2- (fourth current), ITV 3855 is coupled to bit line current IW3+ (fifth current), ITV 3856 is coupled to bit line current IW3- (sixth current), ITV 3857 is coupled to bit line current IW4+ (seventh current), and ITV 3858 is coupled to bit line current IW4- (eighth current). ITVs 3851 and 3852 are coupled to SAR ADC 3861, ITVs 3853 and 3854 are coupled to SAR ADC 3862, ITVs 3855 and 3856 are coupled to SAR ADC 3863, and ITVs 3857 and 3858 are coupled to SAR ADC 3864. SAR ADC 3861 includes a binary capacitor array (CDAC) 3865, SAR ADC 3862 includes a CDAC 3866, SAR ADC 3863 includes a CDAC 3867, and SAR ADC 3864 includes a CDAC 3868. The design of the CDACs is described in more detail below with reference to Figure 42.

Figure 38C depicts four exemplary output circuits 3880 that can be used in the linear circuit 3502 in Figure 35. Output circuit 3880 is the same as output circuit 3850 in Figure 38B, except that it uses two SAR ADCs (SAR ADCs 3883 and 3884 containing CDAC 3885 and 3886 respectively) instead of four SAR ADCs. This is achieved by adding multiplexer 3881 and multiplexer 3882. Multiplexer 3881 allows signals from ITV 3851 and 3852 to be time-multiplexed with signals from ITV 3853 and 3854 to share SAR ADC 3883. Multiplexer 3882 allows signals from ITV 3855 and 3856 to be time-multiplexed with signals from ITV 3857 and 3858 to share SAR ADC 3884.

Figure 38D depicts output circuit 3890, which is the same as output circuit 3800 in Figure 38A, except that each ITV can be selectively coupled to one of two different bit lines by means of a switch or multiplexer (not shown). ITV 3823 can be coupled to IW1+ (first current) or IW3+ (fifth current), and ITV 3824 can be coupled to IW1- (second current) or IW3- (sixth current), ITV 3825 can be coupled to IW2+ (third current) or IW4+ (seventh current), and ITV 3826 can be coupled to IW2- (fourth current) or IW4- (eighth current).

Figure 38E depicts output circuit 3895, which is similar to output circuit 3890, except that two CDACs 3829 and 3830 are used in a single SAR ADC 3896.

Figure 39A depicts the timing diagram 3900 used to operate the output circuit 3850, which contains eight ITVs and four SAR ADCs. There are two main cycles.

The first period is the integration or sampling period (including sub-periods t1, t2, t3 and t4), in which the bit line current is integrated by the capacitor.

During the sub-cycle t1 of the first cycle, common capacitors 3871 and 3872 are coupled to ITVs 3851 and 3852, and bit lines IW1+ and IW1- are sampled and held in CDAC 3865.

During the sub-cycle t2 of the first cycle, common capacitors 3871 and 3872 are coupled to ITV 3853 and 3854, and bit lines IW2+ and IW2- are sampled and held in CDAC 3866.

During the sub-cycle t3 of the first cycle, common capacitors 3871 and 3872 are coupled to ITV 3855 and 3856, and bit lines IW3+ and IW3- are sampled and held in CDAC 3867.

During the sub-cycle t4 of the first cycle, common capacitors 3871 and 3872 are coupled to ITV 3857 and 3858, and bit lines IW4+ and IW4- are sampled and held in CDAC 3868.

The second cycle (including sub-cycle t5) involves the SAR ADCs 3861, 3862, 3863 and 3864 performing voltage conversion operations using the voltages stored in CDACs 3865, 3866, 3867 and 3868, respectively.

Figure 39B depicts a timing diagram 3920 for operating the output circuit 3880 in Figure 38C, which contains eight ITVs and two SAR ADCs. As can be seen, there is a ping-pong action between the sampling of the ITVs and the conversion of the SAR ADCs, wherein sampling and holding occur on one CDAC while conversion is performed on the other CDAC.

During sub-cycle t1, common capacitors 3871 and 3872 are coupled to ITVs 3851 and 3852, and bit lines IW1+ and IW1- are sampled and held in CDAC 3885.

During sub-cycle t2, common capacitors 3871 and 3872 are coupled to ITVs 3853 and 3854, and bit lines IW2+ and IW2- are sampled and stored in CDAC 3886. SAR ADC 3883 performs a conversion operation on the value stored in CDAC 3885 during sub-cycle t1.

During sub-cycle t3, common capacitors 3871 and 3872 are coupled to ITVs 3855 and 3856, and bit lines IW3+ and IW3- are sampled and stored in CDAC 3885. SAR ADC 3884 performs a conversion operation on the value stored in CDAC 3886 during sub-cycle t2.

During sub-cycle t4, common capacitors 3871 and 3872 are coupled to ITVs 3857 and 3858, and bit lines IW4+ and IW4- are sampled and stored in CDAC 3886. SAR ADC 3883 performs a conversion operation on the value stored in CDAC 3885 during sub-cycle t3.

During sub-cycle t5, the SAR ADC 3884 performs a conversion operation on the value stored in the CDAC 3886 during sub-cycle t4.

Figure 39C depicts a timing diagram 3940 for operating the output circuit 3800 in Figure 38A, which contains four ITVs and two SAR ADCs, wherein sampling and holding occur on one CDAC and switching is performed on the other CDAC.

During sub-cycle t1, common capacitors 3803 and 3804 are coupled to ITVs 3823 and 3824, and bit lines IW1+ and IW1- are sampled and held in CDAC 3829.

During sub-cycle t2, common capacitors 3803 and 3804 are coupled to ITVs 3825 and 3826, and bit lines IW2+ and IW2- are sampled and held in CDAC 3830. SAR ADC 3827 performs a conversion operation on the value stored in CDAC 3829 during sub-cycle t1.

During sub-cycle t3, common capacitors 3803 and 3804 are coupled to ITVs 3827 and 3828, and bit lines IW3+ and IW3- are sampled and held in CDAC 3829. SAR ADC 3828 performs a conversion operation on the value stored in CDAC 3830 during sub-cycle t2.

During sub-cycle t4, common capacitors 3803 and 3804 are coupled to ITVs 3827 and 3828, and bit lines IW4+ and IW4- are sampled and held in CDAC 3830. SAR ADC 3827 performs a conversion operation on the value stored in CDAC 3829 during sub-cycle t3.

During sub-cycle t5, the SAR ADC 3828 performs a conversion operation on the value stored in the CDAC 3830 during sub-cycle t4.

Figure 39D depicts a timing diagram 3950 for operating the output circuit 3890 in Figure 38D, which contains four ITVs and two SAR ADCs selectively coupled to eight different bit lines. Timing diagram 3950 utilizes bit line current overlap technology to reduce settling time. For example, after the output conversion of bit lines IW1+ and IW1-, bit lines IW2+ and IW2- are enabled, while bit lines IW1+ and IW1- remain enabled before the conversion procedure of bit lines IW2+ and IW2- is deactivated. This ensures efficient current loading to the ITV between read operations across multiple bit lines. This is useful when the ITV is shared by multiple bit lines.

During sub-cycle t1, bit lines IW1+ and IW1- are enabled, coupled to ITVs 3823 and 3824 via shared capacitors 3803 and 3804. Bit lines IW1+ and IW1- are sampled and stored in CDAC 3829, and SAR ADC 3827 performs a conversion operation on the values stored in CDAC 3829. After the conversion operation, IW2+ and IW2- are enabled, while IW1+ and IW1- remain enabled.

During sub-cycle t2, bit lines IW2+ and IW2- are enabled, coupled to ITVs 3825 and 3826 via shared capacitors 3803 and 3804. Bit lines IW2+ and IW2- are sampled and stored in CDAC 3830, and SAR ADC 3828 performs a conversion operation on the values stored in CDAC 3830. After the conversion operation, IW3+ and IW3- are enabled, while IW2+ and IW2- remain enabled.

During sub-cycle t3, bit lines IW3+ and IW3- are enabled, coupled to ITVs 3823 and 3824 via shared capacitors 3803 and 3804. Bit lines IW3+ and IW3- are sampled and stored in CDAC 3829, and SAR ADC 3827 performs a conversion operation on the values stored in CDAC 3829. After the conversion operation, IW4+ and IW4- are enabled, while IW3+ and IW3- remain enabled.

During sub-cycle t4, bit lines IW4+ and IW4- are enabled and coupled to ITVs 3825 and 3826 via capacitors 3803 and 3804. Bit lines IW4+ and IW4- are sampled and stored in CDAC 3830, and SAR ADC 3828 performs a conversion operation on the value stored in CDAC 3829.

In another example, current-driven techniques are used where a high-efficiency current load (e.g., from a stationary power supply) is present for the ITV circuit during switching between ITV output operations (current-to-voltage conversion, such as across multiple bit lines).

Alternatively, a pre-charge current can be applied to each bit line prior to the sampling cycle in all the schemes described herein. This helps ensure that the bit lines ramp up from low voltage to high voltage during a steady cycle.

Figure 40 illustrates an illustrative output circuit 4000 that can be used in the row circuit 3502 in Figure 35. The output circuit 4000 includes a SAR ADC 3701 (or any type of ADC architecture) as previously described with respect to the output circuit 3700 in Figure 37, an ITV 3704 for row W+, and an ITV 3705 for row W-. Unlike output circuit 3700, output circuit 4000 can be selectively connected to one of i different W+ current sources (current sources 4002-1, ..., 4002-i) via portions 4001-1, ..., 4001-i of the line multiplexer, and to one of i different W- current sources (current sources 4004-1, ..., 4004-i) via portions 4003-1, ..., 4003-i of the line multiplexer. At any given time, a pair of lines (one W+ line and one W- line) is connected to ITV 3704 for line W+ and ITV 3705 for line W-, and all other lines are disconnected from ITV 3704 and ITV 3705 using the multiplexer. Therefore, i different W+/W- row pairs share ITV 3704 and 3705 and SAR ADC 3701. Bit line current switching can be performed using one or more of the current guiding techniques and bit line current overlap techniques described above with reference to Figures 39C and 39D.

Figure 41 illustrates a reference voltage generator 4100 for generating VADCREFL and VADCREFH, which are used to generate voltages VREFP, VCIM, and VREFN as a reference voltage used by the SAR ADC 3701 shown in the previous figures. The reference voltage generator 4100 is similar to the current-to-voltage converter shown in Figure 37, and converts the reference current IBLEF into voltages VADCREFH and VADCREFL. The reference voltage generator 4100 includes switches 4101, 4102, 4103, 4104 and 4120; capacitors 4105 and 4106; transistor 4107; operational amplifier 4110; transistors 4108 and 4109; and reference current source 4111.

First, switches 4120, 4101, 4102, 4103 and 4104 are closed, thereby causing the top and bottom plates of capacitors 4105 and 4106 to charge to VDDA.

Subsequently, switch 4120 is turned off. Reference current source 4111 draws reference current IBLREF, causing VADCREFL to drop proportionally to the current drawn from reference current source 4111.

During the read operation, operational amplifier 4110 and transistors 4107, 4108, and 4109 impose VREF on the bit line, which is applied to the positive input terminal of operational amplifier 3714, regardless of the amount of current drawn from bit line IW+. Figure 42 depicts SAR ADC 4200, which is an example of SAR ADCs that can be used with: SAR ADC 3701 in Figure 37; SAR ADCs 3827 and 3828 in Figure 38A; SAR ADCs 3861, 3862, 3863, and 3864 in Figure 38B; SAR ADCs 3883 and 3884 in Figure 38C; SAR ADCs 3827 and 3828 in Figure 38D; and SAR ADC 3701 in Figure 40. The SAR ADC 4200 includes a CDAC 4201 (which are CDACs 3829, 3830, 3865, 3866, 3867, 3868, 3885, 3886, 3829, and 3830 in Figures 37, 38A, 38B, 38C, 38D, and 40), a comparator 4202, a SAR logic 4203, a switch 4204, and a switch 4205. When not operating in the SAR ADC 3701, some or all of the CDAC 4201 can be selectively used as capacitors 3803 and 3804 in Figure 38A, capacitors 3871 and 3872 in Figures 38B and 38C, and capacitors 3803 and 3804 in Figure 38D.

The SAR ADC 4200 operates by first sampling the input voltages VIN+ and VIN- into capacitor arrays (CDACs) 4201P and 4201N, respectively. SAR logic 4203 sequentially converts the voltages into digital bits, starting with the most significant bit and ending with the least significant bit. For example, for an 8-bit ADC, B7 will be converted first, and B0 will be converted last. Therefore, for an 8-bit ADC, there are 8 conversion clocks. For each conversion, VIN+ is compared with VIN-, and the comparison decision is used to switch the capacitor associated with the bit for the next bit comparison.

Figure 43 illustrates a voltage generator 4300 that can be used to generate VREFP, VREFN, and VCIM for use by the SAR ADC 4200 in Figure 42. The voltage generator 4300 includes switches 4301, 4302, 4303, and 4304; capacitors 4305 and 4306; switch 4307; variable capacitor 4308; and comparator 4309. In this example, the capacitance of capacitor 4305 equals the capacitance of capacitor 4306. By appropriately switching switches 4301 to 4304, VOUT = (VIN2 - VIN1) * (capacitor 4305) / (capacitor 4306). For example, to generate VREFP (where VOUT will be the value used for VREFP), VADCREFH from Figure 41 is used for VIN1, and VADCREFL from Figure 41 is used for VIN2. The variable capacitor 4308 can be adjusted to generate VREFN and VCIM.

Figure 44 illustrates a voltage generator 4400 that can be used to generate VREFP, VREFN, and VCM for use by the SAR ADC 4200 in Figure 42. The voltage generator 4400 includes a clamp 4401 and a resistor ladder 4402. The resistor ladder 4402 includes n+1 series-coupled resistors. Each node will have a different voltage, ranging from VREF at the top node of the resistor ladder 4402 to ground at the bottom node of the resistor ladder 4402. Appropriate nodes are selected to provide voltages VREFP, VREFN, and VCM for use in the SAR ADC 4200.

As used herein, the terms "above" and "on" inclusively include "directly on" (without intermediate materials, components, or spaces) and "indirectly on" (with intermediate materials, components, or spaces). Similarly, the term "adjacent" includes "directly adjacent" (without intermediate materials, components, or spaces) and "indirectly adjacent" (with intermediate materials, components, or spaces), "installed to" includes "directly installed to" (without intermediate materials, components, or spaces) and "indirectly installed to" (with intermediate materials, components, or spaces), and "electrically coupled" includes "directly electrically coupled to" (without intermediate materials or components electrically connecting the components together) and "indirectly electrically coupled to" (with intermediate materials or components electrically connecting the components together). For example, forming an element "above a substrate" may include forming an element directly on the substrate without intermediate materials/elements, or forming an element indirectly on the substrate with one or more intermediate materials/elements.

12: Semiconductor substrate; 14: Source region; 16: Drain region; 18: Channel region; 20: Floating gate; 22: Character line terminal; 24: Bit line; 28: Control gate; 30: Erase gate; 31: Digital-to-analog converter; 32, 900, 1000, 1100, 1200, 1300, 1601, 1701, 2001, 2101, 2200, 2300, 2400, 2500, 2600, 2700, 2800, 2900, 3000, 3401: VMM array; 32a, 32b, 32c, 32d, 32e: VMM array (layers) 33: Non-volatile memory cell array 34: Erase gate and character line gate decoder 35: Control gate decoder 36: Bit line decoder 37: Source line decoder 38: Differential summer/summation operational amplifier 39, 1602, 1702, 2002, 2102: Excitation function blocks 210, 310, 410, 510, 710: Memory cells 901, 1003, 1103, 1203, 1303: Memory cells Memory arrays 902, 1001, 1002, 1101, 1102, 1201, 1202, 1301, 1302: Reference array; 903: Control gate line; 904: Erase gate line; 1014, 1212: Diode-connected through-cell multiplexer; 1204: Series transistor; 1205, 1709, 1710, 2104: Multiplexer; 1314: Diode-connected reference cell through-cell multiplexer; 1400: LSTM. 1401, 1402, 1403, 1404, 1801, 1802, 1803, 1804: Cells; 1500, 1600, 1700: LSTM cells; 1501, 1502, 1503, 1901, 1902: S-shaped function components; 1504, 1505, 1903: Hyperbolic tangent components; 1506, 1507, 1508, 1703, 1904, 1905, 1906, 2103: Multiplier components; 1509, 1708, 1907, 2105: Adder components; 1704, 1705, 1706, 1707, 2106, 2107, 2108: Registers; 1800: GRU 1900, 2000, 2100: GRU cells; 1908, 2109: Complementary components; 2701-1, 2701-2, ..., 2701-(N-1), 2701-N: Bit line control gates; 3100, 3210, 3300, 3400: VMM systems; 3101, 3102, 3213, 3303 3304, 3305, 3306, 3307, 3308: Summation circuit; 3211: First array; 3212: Second array; 3301, 3302: Array; 3402: Column decoder; 3403: High-voltage decoder; 3404: Row decoder; 3405: Bit line driver; 3406: Input circuit; 3407, 3500, 3 700, 3800, 3850, 3880, 3890, 3895, 4000: Output circuit; 3408: Control logic; 3409: Bias generator; 3410: High voltage generation block; 3411: Charge pump; 3412: Charge pump regulator; 3413: High voltage level generator; 3414: Algorithm controller; 3415: Analog circuit. Path System 3416: Control Engine; 3417: Test Control Logic; 3418: SRAM Block; 3501: Reference Circuit; 3502: Horizontal Circuit; 3503, 3506: Horizontal Multiplexer; 3504, 3507, 3704, 3705, 3823, 3824, 3825, 3826, 3851, 3852, 3853. 3854, 3855, 3856, 3857, 3858: ITV; 3505: Reference Generator; 3508: Analog-to-Digital Converter; 3600, 3650: Clock Generator; 3601: Delay Locked Loop; 3602, 3652: Clock Generator Block; 3651: Phase-Locked Loop; 3701, 3827, 3828, 3861, 3862, 3863, 3864, 3883, 3884, 3896, 4200: SAR ADC 3702, 3703, 4002-1, ..., 4002-I, 4004-1, ..., 4004-i: Current source; 3706, 3707, 3708, 3717, 3718, 3719, 3805, 3806, 3807, 3808, 3809, 3810, 3811, 3812, 4101, 4102, 4103, 4104, 4120, 4204, 4205, 4301, 4302, 4303, 4304, 4307: Switch; 3710, 3711, 3721, 3722: Integrating capacitor; Capacitor 37 12, 3723: NMOS series transistors; transistors 3713, 3715, 3724, 3726, 4107, 4108, 4109: transistors 3714, 3725, 4110: operational amplifiers 3801, 3802: circuits 3803, 3804, 3871, 3872: common integrating capacitors; common capacitors; capacitors 3813, 3870: common capacitor network; capacitor network 3829, 3830, 3865, 3866, 3867, 3868, 3885, 3886, 4201, 4201P, 4201N: CDAC 3881, 3882: Multiplexers; 3900, 3920, 3940, 3950: Timing Diagrams; 4001-1,…,4001-i,4003-1,…,4003-i: Sections; 4100: Reference Voltage Generator; 4105, 4106, 4305, 4306: Capacitors; 4111: Reference Current Source; 4202, 4309: Comparators; 4203: SAR Logic; 4300, 4400: Voltage Generators; 4308: Variable Capacitor; 4401: Clamp; 4402: Resistor Ladder; BL0, BL1, BL2, BL3,…, BLN: Bit Lines; BLR0, BLR1, BLR2, BLR3: Terminals _c0 , _c1 , _c2 , _c3 c(t-1), c(t): Cell state vector C1, C2, C3, S1, S2, S3: Layer CB1, CB2, CB3, CB4: Synapse CFGx: Structure bits CG0, CG1, CG2, CG3, ..., CG _M-1 , CG _M : Control gate line/Control gate voltage CLKIN: System clock CLKOUT: Faster clock DOUT, DOUT[n:0]: Digital output EG0, EG1: Erasure gate/EG line EGR0, EGR1: EG line EN: Enable signal _h0 , _h1 , _h2 , _h3 , h(t-1), h(t): Output vector IBREF: Reference current Inputx, INPUT ₀ , INPUT ₁ , INPUT ₂ , INPUT ₃ , INPUT ₄ , ..., INPUT _N-1 , INPUT _N , INPUT _M-1 , INPUT _M : Input IW+, IW-: Current IW1+, IW1-, IW2+, IW2-, IW3+, IW3-, IW4+, IW4-: Bit line current OUTPUT ₀ , OUTPUT ₁ , OUTPUT ₂ , OUTPUT ₃ , OUTPUT ₄ , OUTPUT _N-1 , OUTPUT _N Output P1: Excitation function/pooling function P2: Excitation function S0: Input layer SL0,SL1,SL2,SL3: Source lines t1,t2,t3,t4,t5: Sub-periods VADCREFH,VADCREFL: Reference voltage VIN+,VIN-: Differential voltage/input voltage VREFP,VREFN,VCIM: Voltage WL0,WL1,WL2,WL3,WL4,WL5,WL6,WL7,WL _M-1 ,WL _M ,WLA0,WLB0,WLA1,WLB1,WLA2,WLB2,WLA3,WLB3: Character lines _x0 , _x1 , _x2 , _x3 ,x(t): Input vector

Figure 1 is a diagram illustrating an artificial neural network.

Figure 2 depicts the isolated gate flash memory cell in the prior art.

Figure 3 depicts another prior art technique for isolating gated flash memory cells.

Figure 4 depicts another prior art technique for isolating gate-extreme flash memory cells.

Figure 5 depicts another prior art technique for isolating gated flash memory cells.

Figure 6 is a diagram illustrating different levels of an artificial neural network that utilizes one or more non-volatile memory arrays.

Figure 7 is a block diagram illustrating a VMM system.

Figure 8 is a block diagram illustrating an example artificial neural network utilizing one or more VMM systems.

Figure 9 illustrates another example of a VMM system.

Figure 10 illustrates another example of a VMM system.

Figure 11 illustrates another example of a VMM system.

Figure 12 illustrates another example of a VMM system.

Figure 13 illustrates another example of a VMM system.

Figure 14 illustrates a prior art long short-term memory system.

Figure 15 depicts an instance cell used in a long short-term memory system.

Figure 16 illustrates an example implementation of the cell element in Figure 15.

Figure 17 depicts another example implementation of the cell in Figure 15.

Figure 18 illustrates a prior art gate return unit system.

Figure 19 illustrates an instance cell used in a gate return unit system.

Figure 20 illustrates an example implementation of the cell from Figure 19.

Figure 21 depicts another example implementation of the cell in Figure 19.

Figure 22 illustrates another example of a VMM system.

Figure 23 illustrates another example of a VMM system.

Figure 24 illustrates another example of a VMM system.

Figure 25 illustrates another example of a VMM system.

Figure 26 illustrates another example of a VMM system.

Figure 27 illustrates another example of a VMM system.

Figure 28 illustrates another example of a VMM system.

Figure 29 illustrates another example of a VMM system.

Figure 30 illustrates another example of a VMM system.

Figure 31 illustrates another example of a VMM system.

Figure 32 illustrates another example of a VMM system.

Figure 33 illustrates another example of a VMM system.

Figure 34 illustrates another example of a VMM system.

Figure 35 illustrates the output circuit used in a VMM system.

Figure 36A depicts the pulse generator.

Figure 36B depicts the pulse generator.

Figure 37 illustrates the output circuit used in a VMM system.

Figure 38A illustrates the output circuit used in a VMM system.

Figure 38B illustrates another output circuit used in a VMM system.

Figure 38C illustrates another output circuit used in a VMM system.

Figure 38D depicts another output circuit used in a VMM system.

Figure 38E depicts another output circuit used in a VMM system.

Figure 39A shows the timing diagram of the output circuit.

Figure 39B depicts the timing diagram of another output circuit.

Figure 39C depicts the timing diagram of another output circuit.

Figure 39D depicts the timing diagram of another output circuit.

Figure 40 illustrates the output circuit used in a VMM system.

Figure 41 depicts the reference voltage generator.

Figure 42 illustrates a continuous approximation register analog-to-digital converter.

Figure 43 depicts the voltage generator.

Figure 44 depicts the voltage generator.

C1: Layer C2: Layer C3: Layer CB1: Synapse CB2: Synapse CB3: Synapse CB4: Synapse P1: Excitation Function/Pooling Function P2: Excitation Function S0: Input Layer S1: Layer S2: Layer S3: Output Layer

Claims (7)

A system having an output circuit for a vector matrix multiplication array, comprising: a vector matrix multiplication array including nonvolatile memory cells arranged in columns and rows, a first set of rows storing W+ weights, and a second set of rows storing W- weights; and an output circuit for receiving a first current from a first row in the first set of rows and a second current from a second row in the second set of rows, and generating a first voltage representing the first current and a second voltage representing the second current, and receiving a third current from a third row in the first set of rows and a fourth current from a fourth row in the second set of rows, and generating a third voltage representing the third current and a fourth voltage representing the fourth current, the output circuit comprising: A first current-to-voltage converter selectively coupled to a first integrated capacitor to provide the first voltage, the first voltage being equal to an initial voltage minus a first discharge value caused by the first current; and a second current-to-voltage converter selectively coupled to a second integrated capacitor to provide the second voltage, the second voltage being equal to the initial voltage minus a second discharge value caused by the second current. A third current-to-voltage converter selectively coupled to the first integrated capacitor to provide the third voltage, the third voltage being equal to the initial voltage minus a third discharge value caused by the third current, wherein the first current-to-voltage converter and the third current-to-voltage converter share the first integrated capacitor in a time-multiplexed manner; and a fourth current-to-voltage converter selectively coupled to the second integrated capacitor to provide the fourth voltage, the fourth voltage being equal to the initial voltage minus a fourth discharge value caused by the fourth current, wherein the second current-to-voltage converter and the fourth current-to-voltage converter share the second integrated capacitor in a time-multiplexed manner. The system of claim 1 includes: a first analog-to-digital converter for converting a difference between the first voltage and the second voltage into a first digital output. The system of claim 2, wherein the first analog-to-digital converter is a continuous approximation register analog-to-digital converter. The system of claim 2 includes: a second analog-to-digital converter for converting a difference between the third voltage and the fourth voltage into a second digital output. The system of claim 4, wherein the second analog-to-digital converter is a continuous approximation register analog-to-digital converter. An operating method of an output circuit includes: during a first time cycle: coupling a first common capacitor and a second common capacitor to a first current-to-voltage converter and a second current-to-voltage converter to generate a first voltage and a second voltage, the first current-to-voltage converter being coupled to a first bit line of a non-volatile memory cell array, and the second current-to-voltage converter being coupled to a second bit line of the array; and storing the first voltage and the second voltage in a first analog-to-digital converter; during a second time cycle following the first time cycle: The first shared capacitor and the second shared capacitor are coupled to a third current-to-voltage converter and a fourth current-to-voltage converter to generate a third voltage and a fourth voltage. The third current-to-voltage converter is coupled to a third bit line of the array, and the fourth current-to-voltage converter is coupled to a fourth bit line of the array. The third and fourth current-to-voltage converters are different from the first and second current-to-voltage converters. The third and fourth voltages are stored in a second analog-to-digital converter. During a third time period following the first and second time periods: The first analog-to-digital converter converts the first voltage and the second voltage into a first digital output; and the second analog-to-digital converter converts the third voltage and the fourth voltage into a second digital output. An operation method of an output circuit includes: receiving current from individual bit lines of a non-volatile memory cell array via a first current-to-voltage converter, a second current-to-voltage converter, a third current-to-voltage converter, and a fourth current-to-voltage converter; during a first time period: connecting a plurality of shared capacitors to the first current-to-voltage converter and the second current-to-voltage converter; sampling and holding a first set of voltages received from the first current-to-voltage converter and the second current-to-voltage converter via a first analog-to-digital converter; and converting the first set of voltages into a digital output via the first analog-to-digital converter; and during a second time period: Connect the shared capacitors to the third current-to-voltage converter and the fourth current-to-voltage converter; sample and hold a second set of voltages received from the third current-to-voltage converter and the fourth current-to-voltage converter via a second analog-to-digital converter; and convert the second set of voltages into a digital output via the second analog-to-digital converter.