JPH06274661A - Synapse circuit and neural network system using the same - Google Patents

Abstract

(57) [Summary] [Structure] By turning on the gate, the voltage of the analog weight representing the synapse weight inputted from the source or drain is accumulated, and the drain or source is connected to the internal connection point NODE A. Transistor M1
, One end of which is connected to the internal connection point NODE A, the pulse stream of the analog voltage having a predetermined amplitude is input from the other end, and the capacitor C1 which loads the analog weight voltage to the amplitude, and the drain and the source are respectively predetermined. And a second transistor M2 that outputs a pulse of an output current when the voltage of the gate connected to the internal connection point NODE A exceeds a predetermined threshold value. [Effect] Since a synapse circuit can be realized with two transistors and one capacitor, high integration can be achieved with a minimum size.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for constructing a two-transistor synapse circuit for use in an artificial neural network system, and more particularly to a method for analog storage of synapse weights and transmission of neuron states. As a system using a stream of pulses (pulse density) as.

[0002]

As a pulse stream neural network system (pulse density type neural network), a typical technique for providing a pulse stream to a neural network is Tomlin issued on January 9, 1990.
son. Jr., USP 4,893,255.

Two known prior art techniques for synaptic function in pulse stream neural systems are 1992.
IEEE Transaction Neural Networks V issued in May 2016
ol.3, No.3 paper `` Integrated Pulse Stream Nutral Ne
tworks: Results, Issues and Pointers ”by A. Hamilton
Published by et al. The first conventional technology is
It uses the principle of a mutual-conductance multiplier circuit. FIG. 10 illustrates a first prior art technique for pulse stream synapses using transconductance multipliers. The heart of this synaptic structure is the transistors M1, M2 and M3.
Is a multiplier formed by. The transistors M1 and M2 output a current proportional to the weight voltage T _ij ,
This output current is pulsed by the switch transistor M3 controlled by the input neuron state S _j . The resulting output current is integrated over a period of time and represents the product of T _ij and S _j . In this conventional technique, the analog weight stored in the gate of the transistor M2 is stored using a pass gate, and this pass gate is required for each synapse. The operation of this multiplier can be explained with reference to the MOSFET transistor characteristic equation. This equation describes the drain-source current I _DS of a MOSFET in the first or third order range as:

[0004]

[Equation 1]

In the above equation, C _OX is the oxide capacity / area,
μ is the carrier mobility, W is the transistor gate width, and L is the transistor gate length. V _GS , V _T and V _DS
Are the gate-source voltage, the limit voltage and the drain-source voltage of the transistor.

The expression for this I _DS is the product term (μCo × W)
/ L × V _GS × V _DS , but also two other unwanted terms V _DS × V _T and V ² _DS . To eliminate these unnecessary terms, a second identical MOSFET
Then, a transistor M1 is used as shown in FIG. If V _DS1 = V _DS2 , the output current is defined by the following equation.

[0007]

[Equation 2]

In the above equation, V _GS2 indicates a weight voltage and V _GS1 indicates a zero weight voltage. By fixing V _DS2 at a constant value, I ₃ depends primarily on V _GS2 , which exhibits synaptic weights. Therefore, the circuit M1 / M
The output of 2 / M3 is a current pulse stream having a magnitude proportional to T _ij and a frequency proportional to S _j .

Extra buffer transistors M4 and M5, which operate within their respective primary ranges, have been added to minimize the variation across the chip. In addition to the transistors shown, extra transistors must be added to the synapse circuit to control access to the weight voltage T _ij ,
As a result, a total of 6 transistors are provided for each synapse. The disadvantage of such synapse construction is that it is not possible to use the smallest size transistor, which results in a synapse size of 130 μm × 165 μm when implemented in the 2 μm technology.

The second prior art uses the technique of a gated current source circuit to achieve synaptic multiplication (the second prior art is also by A. Hamilton et al. 5 1992). Monthly issued IEEE Transaction Neura
l Networks Vol.3, No.3 paper `` Integrated Pulse Stre
am Nutral Networks: Results, Issues and Pointers ”
It is described in). Since this circuit is a self-depleting neural synapse circuit, it differs from the one described above. Standard neural synapse circuits output a pure pulse stream as a function of the frequency of synaptic weights and pre-synaptic neuron pulses (pulses of the preceding neuron). All output pulses to a given post-synaptic neuron (posterior neuron) are summed to form an activity value for a particular post-synaptic neuron. Therefore, the post-synaptic neuron outputs a pulse stream with a frequency that is a function of this activity. In a self-depleting neural synapse circuit, activity is impaired after each pulse in a manner similar to the biological paradigm. This self-depleting neural synapse circuit is shown in FIG. The synapse weight T _{ij, which} is dynamically stored as a voltage at each synapse, controls a voltage-controlled current source (VCCS).
The current source comprises a transistor in its primary region, which has a drain-source current set by T _ij , which current is subsequently reflected at the output of VCCS. Assuming that the post-synaptic neuron S _i is not outputting a pulse, the transistor M4 is "O".
N ”, and the transistor M3 is in the“ OFF ”state.
This allows current to flow to the internal capacitor C _INT , transistor M
1 or via the transistor M
It means that it can be drawn from the internal capacitor C _INT via 2. The pulse S _j arriving at the synaptic input gates the current from VCCS via transistor M1 onto the active capacitor, while at the same time removing the fixed charge via transistor M2 by the balancing current I _bal . As a result of this actuation, the internal charge of the active capacitor either increases (indicating excitement) or decreases (indicating inhibition). Pulse Si arriving at the synapse
Gates the current from the _brownout current source I _pd to an internal capacitor, while at the same time shuts off the current from the VCCS or I _bal current source. This results in C _INT , and thus self-representation, that represents a decrease or impairment of neuronal activity.
Current is removed from the impaired synapse.

The problem with this synapse circuit is also the number of transistors required for implementation, which is 1 if the area required for the synapse is 2 μm.
It is shown that it is 30 μm × 140 μm.

Another conventional technique for providing an analog synapse element in a pulse stream system is IEEE Transactions on Neural Networks Vol.
Paper published in No.3, No.3 "VLSI Implementation of Syn
aptic Weighting and Summingin Pulse Coded Neural-T
ype Cells ”by Moon et al. Have been proposed by.
This conventional technique uses the concept of a resistor whose voltage is controlled so as to perform weighting in a pulse duty cycle. This circuit is problematic in that it requires the provision of at least 11 transistors.

Another possible third prior art technique for analog synapses is Proc. Int. Conf. Nural Ne, published June 1989.
tworks-IJCNN89, pp 191-196 "An Electrically
Trainable Artificial Neural Network (ETANN) with 1
0240 “Floting Gate” Synapses, ”Holler et al. Have been proposed by. The synapse cell circuit shown in FIG. 12 is a Gilbert-Muiltiplier NMOS variant. A pair of EEPROM cells are provided, in which electrons are added to the floating gates in the EEPROM cells by the Fowler-Noraheim tunneling phenomenon and diffusion of electrons between the floating gates, which can store or adjust the differential voltage representing the weight. Alternatively, the electrons are removed from this floating gate. The required floating gate differential voltage can be achieved by monitoring the conductance of each of the EEPROM MOSFETs. In particular, each floating gate voltage can be expressed by the following equation.

[0014]

[Equation 3]

In the above equation, C _pp is the capacitance between the floating gate and the top gate (control gate), and C _pp
tot is the total floating cage capacity. V _cg is the static control gate voltage for bias and Q _fg is the charge stored in the floating gate. Assuming that all the transistors in the MOS multiplier are in the saturated state, the output current difference is expressed by the following equation.

[0016]

[Equation 4]

In the above equation, ΔV _in is the input voltage difference and ΔQ _fg is the difference charge stored between the floating gates of the two EEPROMs. The main drawback of this synapse cell is not the problem of requiring a large number of transistors, but the time required to update the synapse weight value.
In this prior art, the weight is updated using a stream of 20 μs pulses. It would take 180 pulses to change the limit at 4V, so the time to update the weight in this particular case would be 3.6ms. This write time is not practical for large neural systems that need to update millions of synapses per second.

The main problem with the prior art described above is that they are dynamic analog circuits and therefore consume power even when not in use. This means that for reasons of power consumption it is not practical to use in large scale neural networks, eg networks with more than 1 × 10 ¹² synapses.

The fourth prior art of analog synapse is 1
IEDM Technical Diges issued in December 991
“An intelligent MOS Transistor Fe”
aturing Gate-Level Weighted Sum and Threshold Oper
ations ”proposed by T. Shibata and T. Ohmi. This synapse is different from the one described above, firstly, a synapse structure is provided in a single MOS transistor, and secondly, the synapse weight is not an analog voltage but a fixed capacitor. However, this means that this synapse is not programmable. The basic structure of a functional MOS transistor is shown in FIG.
Is shown in the schematic diagram. The potential of the floating gate is calculated by the following equation.

[0020]

[Equation 5]

In the above equation, it is assumed for simplicity that the substrate potential and floating gate charge are zero. p
If F exceeds VTH, the threshold voltage turns on the transistor as seen from the floating gate. Thus, the transistor "on" and "off"
Is determined by whether the weighted sum of the input signals is greater than VTH or not. Since the weighted sum calculation is performed in voltage mode using the capacitive coupling effect, essentially no power is consumed in the calculation. This is one of the most salient features of this device compared to current-mode addition.

As one of the prior arts that have also proposed analog synapses, the fifth prior art is Metal Nitr.
ide Oxide Silicon (MNOS) Networks. This technique essentially uses the technique of the EEPROM in that analog weights are accumulated as charges in the Nitride layer between the gate and channel of an FET (Field Effect Transistor) to obviously modulate the gate voltage. This prior art is the Proceeding AIP Conference on Snowbird
JSage et at Neural Networks for Computing
The paper “An Artificial Neural N” published by al.
etwork Integrated Circuit Based on MNOS / CCD
Principles ”by JPSage et al. The structure of this device is shown in FIG. At gate voltages of about 35 V (higher than normal CMOS operating voltage), quantum mechanical tunneling occurs, causing electrons and holes to move between the underlying silicon and the long-lived trap area in the nitride. The potential reservoir of the gate of this MNOS device is used as a reservoir with adjustable depth, which is filled with synaptic elements when V _j = 1. Therefore,
This reservoir provides a metered packet of charge representing ΣT _ij V _j that is passed through the forward CCD-like structure for post-synaptic addition. The nitride charge layer changes the depth of the reservoir, which in turn changes the multiplier T _ij . This network runs dynamically at the CCD clock rate (10 MHz and above), controlling the rate at which packets are passed through synapses. This MNOS technology can be electronically programmed at the expense of non-standard processes.

As a sixth conventional technique, a transistor analog synapse circuit is an IEIC issued in May 1990.
Paper `` Analog Memory Devices for Neural Network Lsl '' published in E Technical Report, ICD 90-37, P39
"Fujit" of NTT LSI Research Laboratories in Japan
proposed by a et al. FIG. 15 shows the structure of this synapse. The basis of this circuit is a floating gate transistor, unlike a normal flash cell,
It has a high ohmic resistance between the floating gate and the source of the transistor. This ohmic resistance is extremely large and is higher than the silicon dioxide (SiO ₂ ) insulator resistance. The cell has a two-mode PROGAM, and the floating cell also erases, writes, or reads. The cell has a complex programming mechanism, which uses a feedback mechanism to perform good programming. The basic operation of the circuit is Fowles-Nordheim
The tunnel effect is used to store an analog voltage in the floating point cell and a charge packet in the floating gate. The floating gate potential is controlled by accumulating or removing small charge packets using pulse technology. When the required voltage is reached, the cell can be accessed by raising the main gate of the transistor. Since the floating gate is electrically isolated from the main cell, the rise in potential at the main gate is reflected in the floating gate,
This floating gate, like the main gate, is changed by a small amount of potential. First, by setting the floating gate below the limit of the transistor,
The floating gate can be raised above the limit voltage by increasing the potential of the main gate. As the floating gate rises above the threshold voltage, the gate-source potential V _gs and the source-drain potential V
_The transistor begins to conduct a drain-source current I _ds which is a function of _ds . This paper does not show how to use this circuit to produce synaptic function. This method has two main drawbacks. The first drawback is the need for a switching transistor for each snap to select between READ and PROGRAM modes. This would require at least two extra transistors. The second drawback is the time required to program the floating gate. Erase and write pulses are on the order of μs, and one or more pulses are required if a large change in weight value is required. Thus, although this method does not require an update mechanism, it is believed that problems will arise if real-time listening is required, especially if multiple synapses are required.
An example of synapse by this circuit is Nikkei Microdevice,
(1990) December issue P46, which shows a resistive element that is not suitable for producing a pulse stream.

[0024]

SUMMARY OF THE INVENTION The present invention combines the advantages of the prior art described above for performing analog synapse weighting functions on a pulse stream, particularly for use within a pulse stream neural system,
Moreover, it provides a new method and apparatus that eliminates some of the disadvantages of the prior art.

[0025]

As shown in FIG. 7, a typical feature of the present invention is to turn on a gate to store a voltage of an analog weight representing a synapse weight inputted from a source or a drain. A first transistor M1 whose drain or source is connected to the internal connection point NODE A, and one end of which is connected to the internal connection point NODE A, and the pulse stream of the analog voltage having a predetermined amplitude is input from the other end, The capacitor C1 that loads the voltage of the analog weight to its amplitude, and the drain and the source are fixed at predetermined voltages, and the internal connection point NODE
A second transistor M that outputs a pulse of output current when the voltage of the gate connected to A exceeds a predetermined threshold value.
2 and a synapse circuit with.

[0026]

The analog weight can be accumulated at the internal connection point NODE A by turning on the gate of the transistor M1. When the voltage of the pulse stream is applied in this state, since the capacitor C1 is electrically insulated, a value loaded with an analog voltage appears at the internal connection point NODE A due to the amplitude of the pulse stream. By fixing the source voltage and the drain voltage of the transistor M2 and applying the bias, a pulse of the output current flows when the gate voltage exceeds a predetermined threshold value. That is, the amplitude of the pulse stream can be modulated by the analog weight. Further, as shown in FIG. 4, by adding the transistor M3, it becomes possible to construct an excitatory type and a suppressive type neuron.

According to the present invention, a synapse circuit can be basically realized with two transistors and one capacitor, so that high integration can be achieved with a minimum size.

[0028]

The features, objects and advantages of the present invention will become apparent from the following detailed description of the accompanying drawings.
It should be noted that similar parts are denoted by the same reference numerals throughout the accompanying drawings.

FIG. 1 shows an apparatus for implementing the weighting function in a stream of pulses. This device can be used to represent excitatory or inhibitory synapses, but cannot be used to simultaneously represent both excitatory and inhibitory synapses. This device consists of two transistors M1 and M2 and a capacitor C1. FIG. 2 shows an example of waveforms to be added to the circuit for the correction operation.

The circuit has a capacitor C1 having two functions.
Can be divided into two parts. A transistor M1 and a capacitor C1 are used to store a voltage in the analog weight. In the following description, it is assumed that the internal voltage of all NMOS transistors is fixed to the GND potential. Minimum value of GND and V
_{ANALOGU is} the analog weight that can take the maximum value of MAXWGHT
Input to the circuit via the E WEIGHT VOLTAGE input. The PULSE STREM input is set to GND during the loading process to ensure that the weights are loaded correctly. To properly load the analog weights on NODE A, LOAD WEIGT should be at least V _MAXWGHT + V _T , where V _T is the limit of the NMOS transistor. After a lapse of a predetermined time determined by the resistance of the transistor M1 and the capacitance value of the capacitor C1, the capacitor C1 is charged or discharged to the analog weight voltage. Initially, as shown in FIG. 2, the capacitor C1 is set to the voltage V ₂ . After the capacitors have been charged to the correct potential, the LOAD WEIGHT input is reset to GND potential, completing the process of loading analog syspth weights into synapses.

The second part of the circuit is the charged capacitor C
1 and a transistor M2 to form a multiplication part of the synapse. A constant potential is always applied across transistor M2. As shown in FIG. 1, the potential on the source side of the transistor is V _BIAS , the potential on the drain side is VDD, and there is a relationship such that VDD> V _BIAS . Therefore, the potential across the transistor is VDD−V _BIAS.
Is. Current flows in the transistor when the voltage V _GS between the gate and the source side is greater than the threshold value V _T of the transistor. Therefore, the potential on the gate side is V _BIAS + V
_The current flows only when it becomes higher than _T. The binary pulse stream signal is set to oscillate between a high potential of V _BIAS + V _T and a low potential of GND. LOAD WEIGT
When the input is at GND potential, the gate to transistor M2, shown as NODE A in FIG. 1, is electrically isolated. NO
Since DE A is coupled to the pulse stream input by capacitor C1, increasing the potential at the PULSE STREAM input also changes the potential of NODE A by the same amount. Initially, the PULSE STREAM input potential is GND.
Therefore, the potential of NODE A is the accumulated analog weight. As shown in FIG. 2, initially, when the input to the PULSE STREAM input is GND, the potential at NODE A is V ₂
Therefore, the potential on the gate side of the transistor M2 is also V ₂ . As already shown, the gate-side potential V _G is V
Current can only flow in transistor M2 if it is higher than _BIAS + V _T. However, it is ensured that the maximum weight voltage V _MAXWGT becomes lower than V _BIAS + V _T , so that no current flows in the transistor M2. When a pulse is generated, the potential at the _{PULS STREM} input is V _BIAS + V
Rose to _T, therefore, the potential of NODE A is V _BIAS + V _T +
Rise to V ₂ . At this point, the transistor M2
_Has a gate voltage V _G of V _BIAS + V _T + V ₂ and a source voltage V _S of V _BIAS + V _T. Therefore, V _GS voltage is V
_BIAS + V _T + V ₂ − (V _BIAS + V _T ) = V ₂ . Therefore, the current that can flow in the transistor M2 can be expressed by the following equation.

[0032]

[Equation 6]

[0033] However, it is assumed that _{0 <(V GS -V T)} ≦ V DS. By choosing the voltage correctly, the transistor M2 can be put into saturation. In this case, the current I _D is independent of V _DS and I _D is a function of V _GS . The waveform of FIG. 2 shows that after simulation, the analog weight of potential V ₁ is stored in the synapse.
The waveform in FIG. 2 shows that the potential at NODE A has dropped and the voltage has changed between the potential V _BIAS + V _T + V ₁ and the potential V ₁ .

By holding the potential across transistor M2 constant and allowing transistor M2 to be in saturation, a current pulse I is applied to transistor M2.
You can observe that _PS flows. The magnitude of these pulses is directly proportional to the analog weight stored at the synapse.

To show an example in which the magnitude i _{D of the} output current is directly proportional to the analog weight accumulated from this circuit,
A preferred embodiment is shown below. However, this embodiment is only one example of many combinations. In this embodiment, the transistor M1 has a width of 2 μm and a length of 0.7 μm.
m and the width of the transistor M2 is 0.7 μm and the width is 0.7 μm. The voltage is VDD = 10V, V _BIAS = 5V, the pulse stream voltage is set to swing between 0V and 5V,
The load weight voltage was high 7V and low 0V, and the analog weight voltage was simulated to be 0V to 5V. The magnitude of the current pulse for the accumulated analog voltage as a result of this embodiment is shown in FIG.

In the pulse stream system, the frequency of the pulse stream indicates the state of the neuron. If the neuron state is 0, indicating an "off" state, this can be represented by zeroing out the pulses produced by the neuron circuit. If the neuron state is 1, indicating a completely "on" state, then
This can be represented by a neuron circuit which produces a pulse at the maximum possible frequency f _max . Intermediate pulse frequencies can be used to indicate a state between the fully "on" and "off" states. For example, suppose neuron state 0 is represented by a pulse stream of frequency 0 and neuron state 1 is represented by a pulse stream of frequency f _max . Therefore,
f _max / 2 can represent a neuron state of 0.5, or a frequency of 3 * f _max / 4 can represent a neuron state of 0.75. Synaptic function doubles the neuronal state by the weighting function. In this case, the magnitude of the current pulse is a function of the accumulated analog weight voltage. The number of current pulses in the scheduled time is a function of the frequency of the pulse stream input. Therefore, the amount of current output from the circuit in any given time is a function of both the frequency of the pulse stream and the stored analog voltage. More current can be output from the circuit in a given time by holding the pulse stream constant and increasing the stored analog voltage. By keeping the accumulated analog voltage constant and increasing the frequency of the pulse stream signal, more current can be output from the synapse circuit. Therefore, the circuit functions as a synapse multiplication unit.

The circuit shown in FIG. 1 can only be used to perform a suppressive or excitatory function, not both functions at the same time. FIG. 4 shows a circuit modified to allow simultaneous suppression and excitatory functions. In order to represent both suppression and excitement weights using the circuit described above, the zero weight reference current I _ZERO must also be output from the synapse. If the current pulse I _PS is smaller than I _ZERO, then it can be said that it is a suppression current,
If it is larger than I _ZERO, it can be said to be an exciting current. Subtracting I _ZERO from I _{PS for} each synapse produces I _{NET, which} is the sum of all currents in the case of excitement. A third transistor M3 is added to the circuit to generate a zero reference current. Since the zero reference is for a constant analog weight voltage, the zero reference current depends only on the frequency of the pulse stream. To achieve this, PULSE ST
Connect to the unmodulated pulse stream signal at the REM input. By selecting appropriate values for VDDZ and V _ZERO , transistor M3 can be biased to generate a zero reference current, which is any analog weight selected to represent a zero weight value. Equal to the voltage. The voltage across transistor M2 holds VDDZ and V _ZERO constant so that the output from transistor M3 is also a series of current pulses. As the pulse stream frequency changes, the amount of current output by transistor M3 also changes. The internal voltage of the transistor M3 is also fixed at the GND potential.

By using a floating gate cell and using a transistor having a floating gate instead of the transistor M2, both types of synapse described above can be produced. This has the advantage that the analog weights do not have to be renewed, but the problem arises of reprogramming the analog weights in ms. FIG. 5 illustrates this particular embodiment.
A large resistor having a resistance value larger than that of silicon oxide (SiO ₂ ) is inserted between the drain and the floating gate. This keeps leakage current to a minimum.

6 and 7 show an embodiment using a two-transistor synapse circuit. This circuit can be divided into three parts. The first circuit part is a neuron circuit for generating a stream of voltage pulses, the frequency of said voltage pulse being a function of the input to the neuron unit, which input can be a current or a voltage. In this example, the input to the neuron circuit is assumed to be a voltage signal, the magnitude of which controls the frequency of the output pulse stream. Also, in this embodiment, the system may be configured such that the pulse stream has pulses of constant magnitude and time such that only the time between individual pulses may change, but pulse widths of different times may be used. I shall. The second circuit portion comprises a synapse circuit as described above.
The third circuit portion comprises a circuit that maintains a constant voltage across the appropriate synaptic transistor and outputs a current or voltage that represents the sum of all currents to a particular post-synaptic neuron. The configuration of the third circuit portion differs depending on the type of synapse embodied in the system.

The two-transistor synapse circuit 100 has a plurality of inputs and bidirectional terminals. These input terminals consist of an ANALOG WEIGHT VOLTAGE input 102, a LOAD WEIGHT input 101 and a PULSE STREAM input 105. The bidirectional terminal includes a VDD terminal 103 and a V _BIAS terminal 104. For purposes of this description, it is assumed that a potential across transistor M2 causes current to flow from terminal 103 to terminal 104. However, since these terminals are bidirectional, current can flow in the opposite direction.

Digital-to-analog converter (DA
C) 300 has a plurality of input and output terminals. The input terminal comprises a data bus terminal 301 which allows the input of a binary number stored externally, which digitally represents the analog weight stored in the synapse. The DAC converts the binary value into an analog voltage display and outputs the result to the output terminal 30.
2 is used to output.

The neuron circuit 400 has a plurality of input and output terminals. The input terminal comprises terminals 401 and 403. The terminal 401 can input an analog voltage or an analog current indicating the activity of the post-synaptic neuron represented by the neuron circuit. The function of this neuron circuit is to convert an analog activity representation of current or voltage into a pulse stream whose pulse frequency and / or pulse width represents the activity of a particular neuron. There are many different transformation functions depending on the algorithm, but in the preferred embodiment the activity of the presynaptic neuron is represented by a voltage and a voltage controlled oscillator (VCO) is used to generate the synaptic neuron activation potential. To an output pulse stream. Proper LOAD W for the synapse circuit to function properly
When the EIGHT ROW signal is active, it is necessary for the output terminal to remain at the GND potential. Therefore, the input terminal 4
03 LOAD WEI to the neuron circuit which is reversed in the initial stage
GHT signal can be input. In the preferred embodiment, the reversal process can be performed by a digital inverter or an analog comparator circuit can be used. Next, the output of the inverter is the VCO
AND is performed by the input from the AND gate, and as a result, the signal is output from the neuron circuit via the terminal 402. A representative stream of pulses is output from a neuron unit using a single output terminal 402.

A voltage controlled voltage source (VCVS) 500 has multiple inputs, outputs and bidirectional terminals. VCVS
The function of is to provide a constant voltage at the intended connection point. If the voltage at the connection point increases, VC
VS removes excess current from the junction to reduce the voltage at the junction. If the voltage at the junction falls below a certain voltage, VCVS will supply more current to raise the voltage at the junction. In this way
The voltage at the connection point is kept constant. The VCVS has a bidirectional terminal 501, which inputs the sum of all currents to a particular post-synaptic neuron. VCVS supplies or removes current to terminal 501 to keep terminal 501 at a constant, pre-specified voltage. V
The CVS also outputs a current or voltage in an analog display at the output terminal and supplies a predetermined amount of current to the terminal 501 to keep the voltage constant.

In addition to the specified circuit, an additional signal is supplied to the synapse circuit 100 to make it function properly. LOAD WEIGH
These signals, named T, are used to load the analog weight signal into the synapse circuit 100.

The synapse array is the synapse circuit 2 of the synapse circuit 100.
It is a dimensional array. The size of the synapse array is selected appropriately. In the illustrated embodiment, the output from the pre-synaptic neuron 400 is input horizontally and the output from the synapse array to the post-synaptic neuron flows vertically to the post-synaptic VCVS500 circuit. The synapse array is divided into multiple columns, each example (eg, 100AA, 100AB ... 100AZ) having a common LOAD WEIGHT signal. The LOAD WEIGHT signals of two rows are never input at the same time. Each column of the synapse matrix (for example, 100AA, 100BA ... 100Z
In A), the common ANALOG WEIGHT VOLTAGE input 102 is input from the output terminal 302 of the DAC 300.

Before the analog weight is loaded, all the neuron circuits 400 are set so that the output terminal 402 is at the GND potential. Data is read from the DATA BUS into the DAC via multiple input terminals 301 to load the analog weights into the synapse array. When all the data is loaded, the analog display of the digital weight is output to the synapse array via the output terminal 302 on the DAC side. By activating the LOAD WEIGHT input line to the desired column of synapses 101, the analog weights can be loaded into the correct synaptic circuit. Then LOAD
WEIGHT Input line is made inactive to complete analog weight storage. This process is repeated until all synaptic weights have been read into the synaptic array.

The system is ready for simulation in this state. To start the simulation, input terminal 401 of neuron unit circuit 400
The presynaptic neuron is activated by inputting an appropriate electric potential or current into the. As a result, the neuron unit circuit converts these active values into an appropriate pulse stream, which is output from the output terminal 402 and input to the synapse array. The pulse stream emitted from the neuron unit is input to the synapse, and each synapse outputs an appropriate current from the bidirectional terminal 104. All synapses in each column have one common bidirectional terminal 104
Have. As mentioned above, since the voltage at terminal 104 is constant, all currents are summed together to produce one output current which attempts to raise or lower the potential at bidirectional terminal 104. To do. This change in potential is detected by VCVS circuit 500, which corrects for the current by supplying the same or opposite current to terminal 104 via bidirectional terminal 501. The amount of current indicates the total activity towards any particular post-synaptic neuron, which is output at the output terminal 502 in the form of a voltage or current. The outputs from these units can be input to other neuron units 400 in the case of a multilayer network.

FIGS. 8 and 9 show a similar system using a three transistor synapse 200. The three transistor synapse circuit 200 has a plurality of input terminals and bidirectional terminals. The input terminals include an ANALOG WEIGHT input 202, a LOAD WEIGHT input 201, and a PULSE STREAM input 205. The bidirectional terminal is the VDD terminal 203
, V _BIAS terminal 204, VDDZ terminal 206, V
_It consists of _ZERO terminal 207. For the purpose of explanation, the terminal 203
It is assumed that the potential across transistor M2 is set to allow current to flow from terminal 204 to the terminal 204, but since these terminals are bidirectional, current can flow in the opposite direction. Also, for the purpose of explanation, the current is at the terminal 206.
It is assumed that the potential is set across transistor M3 so that current flows from the terminal 207 to terminal 207, but since these terminals are bidirectional, current can flow in the opposite direction.

A new unit 600 has been introduced that has a different function than that used in the two-transistor system. The unit 600 includes a plurality of bidirectional terminals 601 and 602 and an output terminal 603. The function of unit 600 is to connect terminals 601 and 602.
To maintain a constant potential at these terminals, these terminals may be independent of each other and have different or the same potential.
In the preferred embodiment, each bidirectional terminal is connected to VCVS to maintain a constant voltage at the input terminal to VCVS, V
It is configured to output a voltage representing the current flowing at the input terminal of the CVS. The outputs of the VCVS circuit are shown as Vnorm and Vzero in the figure, and these outputs are subtracted from each other using a voltage subtraction circuit such as a differentiation circuit, and this voltage is output from the terminal 603. Therefore, the current flowing through terminal 602 is subtracted from the current flowing through terminal 601 and the result of the subtraction is output from output terminal 603 as a representative voltage or as a current as in the preferred embodiment.

The operation of a system using a 3-transistor synapse is similar to that of a 2-transistor synapse.

[0051]

The effects of the present invention are as follows.

The present invention uses two transistors including a transistor required to address analog synapse weights and a capacitor for accumulating the analog synapse weights and one capacitor to perform an excitatory or inhibitory weighting function. Can provide the technology to do so. Therefore, the transistor synapse circuit can be configured with the minimum size, and the hardware of 1 million neurons or more can be realized.

The present invention includes a transistor required to address the analog synapse weight and a capacitor for storing the analog synapse weight.
Techniques can be provided to perform the excitement and suppression weighting functions using transistors and capacitors.

According to the present invention, when the pulse is not input to the synapse circuit, the power consumption is zero except that caused by leakage.

The present invention outputs a weighted current that can be summed together with many outputs using minimal circuitry.

The present invention only requires an NMOS transistor that performs the usual subtraction. This allows an efficient silicon layout.

The present invention can be reprogrammed to the ns order, which is not possible by using a floating gate structure that can only be reprogrammed to the ms order.

[Brief description of drawings]

FIG. 1 is a circuit diagram showing a two-transistor pulse stream analog synapse.

FIG. 2 is a graph showing an example of a correction function waveform of a two-transistor synapse.

FIG. 3 is a graph showing the action of a 2-transistor synapse in the range of analog weight values.

FIG. 4 is a schematic diagram showing a three-transistor pulse stream analog synapse that can produce inhibitory and excitatory functions.

FIG. 5 is a circuit diagram showing a two transistor synapse using a structure similar to that used for floating gate transistors.

FIG. 6 is a diagram showing a two-transistor synapse circuit and a neuron circuit.

FIG. 7 is a diagram showing an artificial neural network system that performs neural calculation using a 2-transistor synapse circuit.

FIG. 8 is a diagram showing a three-transistor synapse circuit and a neuron circuit.

FIG. 9 is a diagram showing an artificial neural network system that performs neural calculation using a 3-transistor synapse circuit.

FIG. 10 is a diagram showing a conventional pulse stream synapse circuit.

FIG. 11 illustrates a conventional self-depleting pulse stream synapse circuit.

FIG. 12 is a circuit diagram showing a conventional analog synapse.

FIG. 13 is a diagram showing a conventional analog synapse.

FIG. 14 is a diagram showing a structure of a conventional non-volatile analog snap.

FIG. 15 is a diagram showing a conventional one-transistor synapse circuit.

Claims (6)

[Claims] 1. A first transistor M for accumulating a voltage of an analog weight representing a synapse weight inputted from a source or a drain by turning on a gate, the drain or the source being connected to an internal connection point NODE A.
1, one end of which is connected to the internal connection point NODE A, the other end of which receives a pulse stream of an analog voltage having a predetermined amplitude, a capacitor C1 which loads the analog weight voltage to the amplitude, and a drain and a source Each is fixed at a predetermined voltage,
A synapse circuit comprising: a second transistor M2 that outputs a pulse of an output current when the voltage of the gate connected to the internal connection point NODE A exceeds a predetermined threshold value. 2. A first transistor M for accumulating a voltage of an analog weight representing a synapse weight inputted from a source or a drain by turning on a gate, the drain or the source being connected to an internal connection point NODE A.
1, one end of which is connected to the internal connection point NODE A, the other end of which receives a pulse stream of an analog voltage having a predetermined amplitude, a capacitor C1 which loads the analog weight voltage to the amplitude, and a drain and a source Each is fixed at a predetermined voltage,
When the voltage of the gate connected to the internal connection point NODE A exceeds a predetermined threshold value, the second transistor M2 that outputs a pulse of the output current and the drain and the source are fixed at the predetermined voltages,
A synapse circuit comprising: a third transistor M3 having a gate connected to the other end of the capacitor and outputting a zero reference current. 3. A first transistor, which stores a voltage of an analog weight representing a synapse weight inputted from a source or a drain by turning on the gate, the drain or the source being connected to one end of a resistor, and a floating gate. A second transistor having a floating gate connected to the other end of the resistor, a drain and a source fixed at a predetermined voltage, and a pulse stream of an analog voltage having a predetermined amplitude input from the gate, Is connected to the first transistor and the other end is connected to the floating gate of the second transistor, has a larger resistance value than the isolation material of the floating gate structure, and minimizes leakage current from the floating gate. The synapse characterized by Circuit. 4. A voltage controlled oscillator for converting an analog voltage indicating the activity of a neuron from a preceding neuron circuit into a pulse stream, a means for inverting a control signal, an AND of the output of the voltage controlled oscillator and the inverting means. Take A
A neuron unit 400 having an ND circuit, a digital-analog converter 300 for converting an analog weight representing a digitally given synapse weight into an analog voltage, and the control signal given to the gate to turn on the gate. Accumulates the analog weight voltage representing the synaptic weight input from the source or drain,
A first transistor whose drain or source is connected to the internal connection point, and one end of which is connected to the internal connection point and the pulse stream of the analog voltage having a predetermined amplitude is input from the other end, and the amplitude of the analog weight A second capacitor that outputs a pulse of an output current when a voltage-loading capacitor and a drain and a source are fixed at predetermined voltages, respectively, and the voltage of the gate connected to the internal connection point exceeds a predetermined threshold And a plurality of synapse circuits 100 each including a transistor and a voltage source 500 that controls an analog voltage by summing output currents output from the plurality of synapse circuits. Characteristic neural network system. 5. A voltage-controlled oscillator for converting an analog voltage indicating the activity of a neuron from the preceding neuron circuit into a pulse stream, a means for inverting a control signal, an AND of the voltage-controlled oscillator and the output of the inverting means. Take A
A neuron unit 400 having an ND circuit, a digital-analog converter 300 for converting an analog weight representing a digitally given synapse weight into an analog voltage, and the control signal given to the gate to turn on the gate. Accumulates the analog weight voltage representing the synaptic weight input from the source or drain,
A first transistor whose drain or source is connected to the internal connection point, and one end of which is connected to the internal connection point and the pulse stream of the analog voltage having a predetermined amplitude is input from the other end, and the amplitude of the analog weight A second capacitor that outputs a pulse of an output current when the voltage-loading capacitor, the drain and the source are fixed at a predetermined voltage, respectively, and the voltage of the gate connected to the internal connection point exceeds a predetermined threshold value A plurality of synapse circuits 100 each having a drain and a source fixed at a predetermined voltage, a gate connected to the other end of the capacitor, and a third transistor outputting a zero reference current; From the first voltage source that controls the analog voltage by summing the output currents output from the plurality of synapse circuits, and the plurality of synapse circuits Sum the output zero reference currents,
A plurality of neuro circuits each including a second voltage source for controlling an analog voltage and a circuit 600 for outputting a difference between analog voltages of the first voltage source and the second voltage source are provided. Neural network system. 6. A pulse stream of an analog voltage having a predetermined amplitude is input, a voltage of an analog weight representing a synapse weight is loaded on the pulse stream, and the pulse stream loaded with the voltage of the analog weight is supplied as an independent voltage source. A synapse weighting method, characterized in that a current pulse having an amplitude proportional to the voltage of the modulated pulse stream is converted and output.