JP3191678B2 - Neuro element - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neural element for constructing a neural network imitating the nervous system of the human brain, and more particularly to a neural element which can be easily realized by a semiconductor integrated circuit.

[0002]

2. Description of the Related Art An information processing apparatus (neurocomputer) having a neural network imitating the nervous system of the human brain has been receiving attention. Such a neurocomputer can obtain an ambiguous solution in information processing that Neumann-type computers are not good at by changing the connection strength between many neurons according to a predetermined learning algorithm.

As a conventional example of a neural element necessary for realizing a neural network as a device instead of a program, Japanese Patent Application Laid-Open No. 3-144785 discloses a device using an EEPROM. In this neuro element, each input signal is multiplied by a coefficient, the sum of the coefficients is input to a threshold function such as a sigmoid function, and the calculated value of the function is used as the output of the neuro element.

On the other hand, unlike a threshold function, a neural network using a radial basis function has been devised. However, as a configuration for realizing this kind of neural network by hardware, a Gaussian function is used for the radial network. Only a neural network with a basis function is known (“A Gaussian Synapse Circuit For Analog VLSINe
ural Networks ").

[0005]

The circuit of a neural element for constructing a neural network using a Gaussian function as a radial basis function is complicated and its manufacture is not easy, so that high integration is difficult. There is a problem that a neural network having a large number of nodes cannot be constructed.

The present invention has been made in view of the above circumstances, and has as its object the difference that an input to a virtual node of a neural network is an input to at least one of two transistors. A neural network using a radial basis function can be simplified by providing a dynamic amplifier and an arithmetic unit for calculating the product of the outputs of the two transistors using the output of only one transistor obtained from the differential amplifier. It is an object of the present invention to provide a neuro element which can be realized by hardware having a simple configuration and can be highly integrated.

[0007]

According to a first aspect of the present invention, there is provided a neural element for constructing a neural network using a radial basis function. The neural element has two transistors and is a virtual element of the neural network. Using a differential amplifier in which an input to a node is an input to at least one of the two transistors and an output of only one transistor obtained from the differential amplifier, a product of outputs of the two transistors is calculated. And a calculation unit that performs the calculation.

According to a second aspect of the present invention, in the first aspect, the arithmetic unit includes a multiplier provided with two pairs of signal input terminals and a reference input terminal. An output voltage of one of the transistors is applied to the reference input terminal with a voltage of 1/2 of the power supply voltage applied to the differential amplifier, and the other pair is supplied with a voltage of 1/2 of the power supply voltage at the signal input terminal. An output voltage of one transistor is supplied to a reference input terminal.

A neuron according to a third aspect is the neuron according to the first or second aspect, wherein the radial basis function is f (b, x).
= A / {1 + cosh (bx)} (where A: constant,
b: a parameter determined by a learning rule).

[0010] Figure 4 is a circuit diagram showing a configuration of a differential amplifier provided in neuro device according to the present invention, reference numeral Q _1,
Q ₂ is a transistor such as a bipolar transistor or a MOS transistor. The collector of the transistor Q ₁ is, connected to a constant voltage source 21 through one resistor R _c. The base of transistor Q ₁ is the input signal source 24 and one end is connected to a series circuit of one of the power supply 23 is grounded. The emitter of the transistor Q ₁ is, one variable resistor R
One end is connected via _E to a constant current source 22 whose one end is grounded. The collector of the transistor Q ₂ are connected to a constant voltage source 21 via the other resistor R _c. The base of transistor Q ₂ is one end connected to the other power supply 23 is grounded. The emitter of the transistor Q ₂ are connected to the constant current source 22 via the other variable resistor R _E. The characteristics of these two transistors Q ₁ and Q ₂ are the same. Both variable resistors _RE are variable resistors whose resistance values can be adjusted according to an external control voltage (not shown).

The aforementioned power supplies 23, ₂₃ apply a constant bias voltage V _BIAS to set the operating voltage of the circuit.
The input signal source 24 applies an input voltage V _in as an input signal which varies with time.

[0012] Here, both the variable resistor R _E, if set to 0 the value of R _E, transistor Q _1, the collector current I ₁ flowing in the Q _2, I _2, respectively, "revised integrated circuits Engineering (2)" ( As shown on page 35 of Corona Publishing, written by Yanai and Nagata), it is expressed by the following equations (1) and (2). In the equations (1) and (2), α is the current amplification factor, I _EE is the current value of the constant current source 22, q is the charge amount of electrons,
k is Boltzmann's constant and T is temperature.

[0013]

(Equation 1)

Based on the above equations (1) and (2), I ₁ ,
When the product of I ₂ is calculated, the multiplied value I ₁ · I ₂ is given by the following equation (3).

[0015]

(Equation 2)

[0016] Figure 5 is a graph showing the relationship between I ₁ · I ₂ and V _in, the horizontal axis represents the V _in, the vertical axis represents the I ₁ · I _2. Graph is symmetrical with respect to the longitudinal axis, i.e. V _in = 0, maximum at V _in = 0 take (αI _EE) ^2/4.

[0017] Here, when the resistance value of the variable resistor R _E is increased from 0, the shape of the graph, as indicated by a broken line in FIG. 5, comes swell gradually. This has the same effect as the case where the parameter b is introduced into the above equation (3) and the following equation (4) is used.

[0018]

(Equation 3)

In equation (4), since both α and I _EE are constant values, the value of the numerator in equation (4) is a constant, and I ₁ · I _{2 is the} radial basis function f (b, x) corresponds to equation (5). f (b, x) = A / {1 + cosh (bx)} (5) where A: constant, b: parameter determined by learning rule cosh (bx) = {exp (bx) + exp (−b)
x)｝ / 2

FIG. 4 shows that both transistors Q ₁ ,
Collector voltage V _1, V ₂ generated by the collector current I _1, I ₂ Q ₂ 'may be represented by the following formula (6) and (7), V ₁ · V ₂ from both equations, the following ( 8)
_Vcc is the power supply voltage of the constant voltage source 21. V ₁ = V _CC -R _C · I ₁ (6) V ₂ = V _CC -R _C · I ₂ (7) V ₁ · V ₂ = V _CC ² -V _CC · R _C · (I ₁ + I ₂ ) + I ₁ · I ₂ · R _C ² = V _CC ² -V _CC · R _C · I _EE + I ₁ · I ₂ · R _C ² = constant + I ₁ · I ₂ · R _C ² ... (8)

[0021] Therefore, the on resistance of the transistors Q _1, Q ₂ are the same, if the resistance values of both the variable resistor R _E are the same, equation (8), the product of the collector voltage V _1, V ₂ is The product is a value obtained by multiplying the product of the collector currents I ₁ and I ₂ by a constant and adding a constant, and it can be seen that V ₁ · V ₂ is proportional to the value of the radial basis function f (b, x).

On the other hand, from the equations (6) and (7), the relationship between the collector voltages V ₁ and V ₂ is as follows.

FIG. 6 is a graph showing the relationship between the collector voltage V _1, V ₂ and the input voltage V _in, the horizontal axis represents V _in, and the vertical axis represents the V ₁ or V _2. The solid line indicates V ₁ and the broken line indicates V ₂ . As apparent from FIG. 6, V ₁ ,
Graph of V ₂ is in the axially symmetrical axis V _CC / 2. Therefore, the following equation (9) is established between V ₁ and V ₂ . V ₁ −V _CC / 2 = V _CC / 2−V ₂ (9)

Here, if ΔV shown in the following equation (10) is set, the product of V ₁ and V ₂ can be expressed as the following equation (11). From this equation (11), (ΔV) by obtaining the ^2, product of V _1, V ₂ is calculated. ΔV = V ₁ −V _CC / 2 = V _CC / 2−V ₂ (10) V ₁ · V ₂ = (V _CC / 2 + ΔV) · (V _CC / 2 −ΔV) = V _CC ² / 4− ( ΔV) ² = constant− (ΔV) ² (11)

This (ΔV)^TwoIs a four-terminal two-input
It is realized as hardware using a multiplier as follows.
Can be Now, the input V of the multiplier_x, V_yAnd out
Force V _outIs expressed by the following equations (12), (13) and (14).
You. V_x= V_a-V_b … (12) V_y= V_c-V_d … (13) V_out= V_e-V_f= V_x・ V_y …(14)

Here, V _a = V ₁ (V ₂ ), V _b = V _CC
When _{/ 2, V c = V CC} / 2, V d = V 1 (V 2) to be applied to the input terminal of the multiplier so that, (12), (13)
From equations (14) and (14), the inputs V _x and V _y of the multiplier are given by the following equations (15) and (16). Therefore, as shown in equation (17), the output V _{out of the} multiplier becomes ｛− (ΔV ² ) _{V x = V 1 (V 2} ) -V CC / 2 = ΔV ... (15) V y = V CC / 2-V 1 (V 2) = - ΔV ... (16) V out = - (ΔV) 2 ... (17)

Thus, a neural element using the radial basis function f (b, x) shown in the equation (5) can be realized using _one of the collector voltages V ₁ and V ₂ of the differential amplifier. Become. In this case, by changing the resistance value of the variable resistor R _E in the circuit shown in FIG. 4, it is possible to change the parameters b of radial basis function shown in equation (4).

[0028]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be specifically described below with reference to the drawings showing the embodiments.

FIG. 2 is a schematic diagram showing an example of the configuration of a general neural network using a radial basis function. In this example, a one-dimensional output vector is obtained from a two-dimensional input vector. ing. The neural network includes an input layer having a first input node 1a and a second input node 1b, a first intermediate node 2a and a second intermediate node 2b.
And an output layer having an output node 3.

The input value in [1] is input to the first input node 1a, and the value converted using the parameter t [1] [1] is input to the first intermediate node 2a as the parameter t [2] [1]. ]
Are output to the second intermediate node 2b. On the other hand, the input value in is input to the second input node 1b.
[2] is input, and the value converted using the parameter t [1] [2] is stored in the first intermediate node 2a.
[2] The value converted using [2] is the second intermediate node 2b
Respectively. Note that t [1] [1], t
[2] [1], t [1] [2], t [2] [2] are learning rules of radial basis functions (for example, stochastic gradient descent, reference “Characterization of complexities in Czochralski”).
crystal growth by nonlinear forecasting ", T. Miyano
et al., J. Appl. Phys. 76 (5), 1 September 1994).

The first intermediate node 2a has a radial basis function f
Using (b, x), h [1] is calculated as in the following equation (18) and output to the output node 3. Also, the second intermediate node
2b is similarly calculated using the radial basis function f (b, x) as h
[2] is calculated as in the following equation (19) and output to the output node 3. Note that b [1] and b [2] are parameters determined by the learning rule of the radial basis function.

[0032]

(Equation 4)

The output node 3 calculates and outputs an output value out [1] as in the following equation (20). Note that c
[1] and c [2] are parameters determined by the learning rule of the radial basis function. out [1] = c [1] · h [1] + c [2] · h [2] (20)

In such a neural network configuration, a Gaussian function is used as a radial basis function in the prior art, but in the present invention, a radial basis function f (b, x ). f (b, x) = A / {1 + cosh (bx)} (5) where A: constant, b: parameter determined by learning rule cosh (bx) = {exp (bx) + exp (−b)
x)｝ / 2

In the above example, the input vector is two-dimensional and the output vector is one-dimensional. However, the dimensions of the input and output are not limited to these values and may be arbitrary.

FIG. 1 is a circuit diagram showing a hardware configuration of the neuro element of the present invention. This circuit includes a differential amplifier circuit section 11, bias sections 12, 12, and an operation section 15. The differential amplifier circuit section 11 includes a constant voltage source 21, two resistors _Rc , and two bipolar transistors or MOs.
It has transistors Q ₁ and Q ₂ such as S transistors, two variable resistors _RE, and a constant current source 22. Further, the bias units 12, 12 are provided with a power source 23,
23 and an input signal source 24. Such a circuit configuration example is known.

The collector of the transistor Q ₁ is, connected to a constant voltage source 21 through one resistor R _c. The base of transistor Q ₁ is the input signal source 24 and one end is connected to a series circuit of one of the power supply 23 is grounded. The emitter of the transistor Q ₁ is at one end through one of the variable resistor R _E is connected to the constant current source 22 which is grounded. The collector of the transistor Q ₂ is the other of the resistor R via the _c constant voltage source 21
Connected to The base of transistor Q ₂ is one end connected to the other power supply 23 is grounded. The emitter of the transistor Q ₂ are connected to the constant current source 22 via the other variable resistor R _E. These two transistors Q ₁ ,
Characteristics of Q ₂ are the same. Both variable resistors _RE are variable resistors whose resistance values can be adjusted according to an external control voltage (not shown).

Each of the power supplies 23 applies a constant bias voltage V _BIAS to set the operating voltage of the circuit. Also,
Input signal source 24 applies an input voltage V _in as an input signal which varies with time.

[0039] FIG. 3 is a block diagram showing the configuration of a circuit for generating a V _in as an input signal. The configuration example shown in FIG. 3 has n parameters t [1] [1],..., T [1]
This shows a general generation circuit that generates [V _in ] by inputting [n].

The Vin generating circuit shown _in FIG. 3 has n subtracters.
It has 31, n squarers 32, a summer 33, and a square root calculator 34. The i-th (1 ≦ i ≦ n) -th subtractor 31 outputs the input value in
The difference between [1] and the parameter t [1] [i] is determined, and the difference is output to the corresponding ith squarer 32. i-th 2
The multiplier 32 calculates the square value of the input difference, and outputs the square value to the summer 33. The summer 33 adds the square values from all the squarers 32 and outputs the added value to the square root calculator 34. The square root calculator 34 finds the value of the square root of the total value from the summer 33, and calculates the input voltage Vin _in proportion to the square root value.
Is output.

[0041] The input voltage V _in, when the example of the first intermediate node 2a in the neural network shown in FIG. 1, a voltage proportional to the values shown in the following equation (21).

[0042]

(Equation 5)

[0043] In the differential amplifier circuit section 11 described above, the transistors Q _1, Q ₂ of the on-resistance is the same, the resistance values of both the variable resistor R _E is substantially the same. Transistor Q
When the input voltage V _in from input signal source 24 is applied to the _first base, collector voltage V _1, V ₂ of the product of the values (V ₁ ·
V ₂ ) is a value obtained by multiplying the product of the collector currents I ₁ and I ₂ by a constant and adding a constant. V ₁ · V ₂ and the radial basis function f (b, x ) Is proportional to the value.

Transistor Q₁, Q_TwoFigure 1
The transistor Q_TwoThe collector of the
The first signal provided in the provided four-terminal two-input multiplier 16 is provided.
No. terminal S₁And the second reference terminal C_TwoConnected to
And both terminals S₁, C_TwoIs the collector voltage V_TwoIs each
Given. The arithmetic unit 15 is provided in the differential amplifier circuit unit 11.
Power supply voltage V of constant voltage source 21_ccIs output, and one end is connected.
Power supply 25 is provided, and the other end of the power supply 25 is
A first reference terminal C provided in the multiplier 16₁And the second signal end
Child S_TwoAnd both terminals C₁, S_TwoHas V_cc/ 2
Are given respectively. The multiplier 16 has a first signal terminal S₁
And the first reference terminal C₁And the second signal terminal S
_TwoAnd the second reference terminal C_TwoWith the voltage between
Multiply and multiply the result by ｛-(ΔV)^TwoOutput terminal OU as｝
T₁, OUT_TwoOutput from The operation unit 15 is a multiplier
16 output terminals OUT₁, OUT_TwoUsing the voltage between₁・
V _TwoIs obtained, and the radial basis function f (b,
Find the value of x) and output it.

As described above, according to the present invention, the radial basis function represented by the expression (5) can be realized only by a circuit having a differential amplifier, a multiplier, and the like which can be easily manufactured by an integrated circuit. Wiring between the differential amplifier and the multiplier can be reduced as much as possible, and high integration can be achieved.

In FIG. 1, the variable resistors R _E and R _E are connected to the emitters of the transistors Q ₁ and Q ₂ , but the present invention is not limited to this, and the transistors Q ₁ and Q 2
_A variable resistor may be connected to each of the _two bases.

In a neuro element using a Gaussian function as a radial basis function, the above-mentioned document “A Gaussian Synapse Cir.
As shown in “Cuit For Analog VLSI Neural Networks”, the gate width and gate length of a transistor must be changed in order to change the half-width of the Gaussian function. On the other hand, the half-value width is fixed and cannot be changed later, whereas in the neuro element of the present invention, the control voltage applied from the outside to the variable resistor (the emitter resistor or the base resistor of the transistor) is changed. By adjusting the magnitude of the resistance value, an arbitrary half-value width can be obtained for the radial basis function shown in the equation (5).

[0048]

As described above in detail, in the neural element according to the present invention, a neural network using a radial basis function can be realized by hardware having a simple configuration. The device can be easily manufactured with, for example, a semiconductor integrated circuit, and the wiring between the differential amplifier and the multiplier can be reduced as much as possible. The invention has excellent effects.

[Brief description of the drawings]

FIG. 1 is a circuit diagram showing a hardware configuration of a neuro element of the present invention.

FIG. 2 is a schematic diagram illustrating an example of a configuration of a neural network using a radial basis function.

3 is a block diagram showing the configuration of a circuit for generating the input voltage V _in.

FIG. 4 is a circuit diagram showing a configuration of a differential amplifier provided in a neuro element according to the present invention.

5 is a graph showing the relationship between I ₁ · I ₂ and V _in.

6 is a graph showing the relationship between the collector voltage V _1, V ₂ and the input voltage V _in.

[Explanation of symbols]

1a first input node 1b first intermediate node 2b second input node 2a the second intermediate node 3 output node 11 the differential amplifier circuit 12 bias unit 15 computation unit 16 the multiplier Q _1, Q ₂ transistor R _E, R _B variable resistance

Continued on the front page (58) Fields surveyed (Int.Cl. ⁷ , DB name) G06G 7 / 26,7 / 60 G06F 7/544

Claims (3)

(57) [Claims] 1. A neural element for constructing a neural network using a radial basis function, comprising: two transistors, wherein an input to a virtual node of the neural network is connected to at least one of the two transistors. A neuro element, comprising: a differential amplifier as an input to a transistor; and a calculation unit that calculates a product of outputs of the two transistors using an output of only one transistor obtained from the differential amplifier. . 2. The arithmetic unit includes a multiplier provided with two pairs of signal input terminals and a reference input terminal, and the pair includes a signal input terminal and an output voltage of one transistor.
1 of the power supply voltage applied to the differential amplifier to the reference input terminal
2. A neuron according to claim 1, wherein a voltage of 1/2 of the power supply voltage is applied to the other pair, and a voltage of 1/2 of the power supply voltage is applied to the signal input terminal and an output voltage of one transistor is applied to the reference input terminal. element. 3. The radial basis function is f (b, x) = A
3. The neuro element according to claim 1, wherein / {1 + cosh (bx)} (where A is a constant, and b is a parameter determined by a learning rule).