JPH07104847B2 - Neurocomputer - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a neurocomputer, and more particularly to a neurocomputer realized by connecting analog neuron chips via analog time-division transmission lines.

BACKGROUND ART Conventional sequential processing computers (von Neumann computers) have difficulty adjusting their data processing functions in response to changes in usage and environment. Therefore, parallel distributed processing methods using hierarchical networks have been proposed as a new adaptive data processing method. In particular, a processing method called the backpropagation method (D.
E. Rumelhart, GE Hinton, and R. J. Williams, “Learning
Internal Representations by Error Propagation,"PA
RALLEL DISTRIBUTED PROCESSING,Vol.1,pp.318−364,Th
e MIT Press, 1986) has attracted attention for its high practicality.

In the back propagation method, a hierarchical network is constructed from a kind of node called a basic unit and internal connections with weights. Figure 1 shows the basic configuration of the basic unit 1. This basic unit 1 executes processing similar to the continuous neuron model. In other words, it is a multi-input, single-output system, and has a multiplication processing unit 2 that multiplies multiple inputs {Y _h } by the weights {W _ih } of the internal connections.
and an accumulation processing unit 3 that adds up all the multiplication results, and a final output X _i that performs non-linear threshold processing on the sum.
Fig. 2 is a conceptual diagram of the configuration of a hierarchical neural network. A large number of basic units {1-h, 1-i, 1-j} of the configuration are connected hierarchically as shown in Fig. 2, and an output signal pattern corresponding to the input signal pattern is output.

During learning, the weights of the connections between each layer {W _ih } are adjusted so that the difference between the output pattern and the desired training pattern is small.
This type of learning is performed for multiple input patterns and multiplexed. Furthermore, when the input patterns are continuous, even if they are incomplete information that is slightly different from the complete information input during learning, an output pattern close to the teacher pattern during learning can be obtained, making associative processing possible.

In order to realize a neurocomputer with such a configuration, it is necessary to realize data exchange between the basic units 1 that make up the hierarchical network with as few wires as possible.
This is one of the issues that must be resolved, given that in order to achieve complex data processing, it is necessary to make the hierarchical network structure more multi-layered and increase the number of basic units.

However, in the data transfer method described above,
As is clear from the hierarchical network configuration shown in the figure, the number of wires between the two layers is extremely large.
When a hierarchical network is integrated into a chip, it becomes difficult to make it smaller and it becomes difficult to improve reliability. For example, if the number of basic units in two adjacent layers is the same and if we assume perfect coupling in which all basic units 1 are connected to each other, the number of wires increases in proportion to the square of the number of basic units.
In this way, the number of wirings increases rapidly.

DISCLOSURE OF THE INVENTION The present invention has been made in view of the above circumstances, and aims to realize data transmission between basic units that constitute a hierarchical network using a small number of wires, thereby:
The object is to make it possible to realize a network configuration data processing device.

FIG. 3 is a system block diagram of a neurocomputer according to the present invention.

The neural network 18 is a network with at least one layer, each layer being a collection of analog neuroprocessors (hereinafter referred to as ANPs), which input analog signals in a time-division manner from a common first analog bus on the input side of each layer, perform product-sum operations using digital weight data, and output the analog signals to a common second analog bus on the output side of that layer.

The control pattern memory 12 stores the control signal patterns of the neural network. The weight memory 14 stores the weight data. The sequencer 13 reads the control pattern memory.
The digital control means 15 is a general-purpose processing unit having an MPU and a main memory, and is connected to a network 18 via D/A and A/D converters 16, 17, and controls the neural network, control pattern memory, sequencer, and weight memory as a whole.
In this way, the present invention configures a neurocomputer system.

An analog neurochip is constructed by inputting an analog input signal to the analog neurochip in a time-division manner, multiplying this signal by weight data, and outputting a product-sum signal obtained by adding these product signals through a nonlinear function circuit. A hierarchical or feedback type neural network 18 is constructed using a plurality of these analog neurochips, and this neural network 18 is connected to a sequencer 13.
The neural network 18 receives a control signal output from the control pattern memory 12, the address to be accessed being given by the signal from the control pattern memory 12. Weight data obtained by learning or the like is supplied to the neural network 18 from the weight memory 14. The neural network 18, the control pattern memory 12, the sequencer 13, and the weight memory 14 are controlled and managed by digital signals from the digital control means 15. The MPU in the digital control means 15 executes the learning algorithm and checks the output signals, among other things. In this way, an analog neurocomputer system characterized by the use of time-division analog input signals and time-division analog output signals is realized.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram showing the basic configuration of a basic unit of a neuron model; FIG. 2 is a conceptual diagram of the configuration of a hierarchical neural network; FIG. 3 is a block diagram showing the basic configuration of a neurocomputer of the present invention; FIG. 4A is a schematic diagram of a package consisting of chips of an analog neuroprocessor ANP of the present invention; FIG. 4B is a diagram showing the internal configuration of an ANP of the present invention; FIG. 5 is a diagram showing the basic configuration of an analog neuroprocessor of the present invention; FIG. 6 is a block diagram of one embodiment of a basic unit of the present invention; FIG. 7 is a specific circuit diagram of an embodiment of a basic unit of the present invention; FIG. 8 is a specific circuit diagram of another embodiment of a basic unit of the present invention; FIG. 9 is a diagram explaining the operation timing of an integrator used in a basic unit of the present invention; FIG. 10A is a conceptual diagram of a hierarchical neural network; FIG. 10B is a conceptual diagram of a hierarchical neural network according to the present invention; FIG. 11 is a block diagram of an embodiment in which a hierarchical network is constructed using a neurocomputer of the present invention; 14A and 14B are timing diagrams of signal processing in the embodiment shown in FIGS. 11 and 12; FIG. 15 is a timing diagram of signal processing in the embodiment shown in FIGS. 11 and 12; FIG. 16 is a specific circuit diagram of an embodiment in which the neurocomputer of the present invention is realized as a hierarchical network; FIGS. 17A and 17B are timing diagrams of signal processing shown in FIG. 16; FIG. 18 is a diagram showing the timing of reading in digital weight data; FIG. 19A is a specific circuit diagram of a master control block; FIG. 19B is a diagram showing the structures of a control pattern memory and a microcode memory; FIG. 24 is a specific circuit diagram of a sequence generator; FIG. 25 is a specific circuit diagram of a phase control circuit; FIG. 26 is a specific circuit diagram of a shift register; FIG. 27A is a conceptual diagram explaining a feedback network; FIG. 27B is an explanatory diagram of a feedback network configured using a neurocomputer of the present invention; FIG. 28 is a block diagram of an embodiment of a feedback network; FIGS. 29A and 29B are timing diagrams showing signal processing in the embodiment shown in FIG. 28; and FIG. 30 is a diagram showing a configuration of a feedback network configured using a neurocomputer of the present invention.
31A and 31B are timing diagrams showing signal processing in the embodiment shown in FIG. 30; FIG. 32 is a diagram showing the second embodiment of the neural network according to the present invention;
33A and 33B are timing diagrams showing signal processing in the embodiment shown in FIG. 32; FIG. 34 is a block diagram of an embodiment in which a system combining hierarchical and feedback networks is configured using a neurocomputer of the present invention; FIG. 35 is a block diagram of another embodiment in which a hierarchical and feedback network is combined using a neurocomputer of the present invention; FIGS. 36A and 36B are timing diagrams showing signal processing in the embodiment shown in FIG. 35; FIG. 37 is a block diagram of yet another embodiment of the present invention; FIG. 38 is a block diagram showing an example of use of the present invention; and FIG. 39 is a block diagram showing another example of use of the present invention.

Basic Configuration of the Invention Figure 4A is a schematic diagram of a dual in-line package of an analog neuroprocessor (ANP) 11 composed of a neurochip of the present invention. This is called MB4442 and executes neuron model processing. The internal threshold processing section is a model replaced with a sigmoid function. The analog neurochip is called an ANP, and is a device that inputs and outputs analog data. Figure 4B is a diagram of the internal configuration of the ANP of the present invention. As shown in Figure 4B, ANP 11 is connected between analog bus B1 and analog bus B2. ANP 11 consists of an analog multiplier 22 that multiplies the input analog signal by a weight, an analog adder 23 that calculates the sum of the products, a sample/hold section 24 that holds the sum, and a nonlinear function section 25 that outputs the value of a sigmoid function. Each terminal of the ANP 11 in Figure 4A is explained below. The ANP 11 is internally composed of an analog circuit section and a digital circuit section. The +-6 volt terminal is the power supply terminal supplied to the operational amplifier in the analog circuit section. D _in
and D _out are terminals for analog input and output signals. AGND is the ground terminal for the analog circuitry. Rt+ and
The Rt- terminal is the external resistor R of the integrator circuit in the analog circuit section.
Ct+ and Ct- terminals are the terminals of the external capacitor C of the integrator circuit. DGND is the ground terminal of the digital circuit. +5 volts is the power supply terminal of the digital circuit. RST is the reset signal terminal that resets the charge of the capacitor of the integrator circuit. CSI and CSO
is the input/output terminal for the daisy chain control signal, and OC
is the offset cancel control signal terminal, and the S/H terminal is
SYNC is a control signal terminal for sample/hold, SYNC is a synchronization signal terminal for processing each layer, DCLK is a basic clock signal terminal for processing analog input signals, WCLK is a clock terminal for taking in digital weight data, and WD is a terminal for digital weight data input in bit serial format.

FIG. 5 shows the analog neuroprocessor (AN) of the present invention.
P) principle configuration diagram.

Analog input signals transmitted in a time-division manner from separate ANPs (not shown) are input to an analog multiplier 22 within the ANP 11 via an analog bus B1. The analog multiplier 22 multiplies the analog input signals by digital weight data WD, which are bit-serialized via a shift register 27 and then converted to serial form, to obtain a product signal representing the product of the analog input signals and the digital weight data. The next analog adder 23 is a Miller integrator circuit consisting of an external resistor R and capacitor C. It calculates the sum of the product signals obtained from the analog input signals transmitted in a time-division manner from multiple ANPs (the locations where ANPs exist are called nodes) connected to the analog bus B1 and the analog threshold input signal transmitted from a dummy node. The sample/hold unit 24 then holds the product signal for a desired period of time, after which the sampled and held output is further transformed via a nonlinear function unit 25. The output control unit 26 delays the signal for a predetermined period of time under the control of a sequence generator 28, and then outputs an analog output signal D _out to the analog bus B2. The sequence generator 28 also generates control signals to be supplied internally. The phase control section 29 mainly controls the phase of the control signal so that each switch connecting the analog circuit section and the digital circuit section within the ANP is turned on or off reliably. In particular, when the first switch is on and the second switch is turned off, the phase of the control signal is controlled so that the two switches are not turned on at the same time.

The sequence generator 28 generates a reset signal RS
It inputs T, DCLK, WCLK, SYNC, S/H, OC, and CSI from the master control block (described later) and outputs CSO, generating the internal control signals for the ANP.

In neural networks, high-speed calculations are required through simultaneous processing. In this invention, time-shared data is used, and in a steady state, each ANP performs simultaneous processing in a pipelined manner. In an ideal neural network, neurons require interconnections with each other, but if we were to try to realize a system in this state, the number of wires would increase. Therefore, in this invention, time-shared data is used, so the processing time for product-sum calculations within each ANP increases, but
By arranging the chips in parallel vertically, i.e. in the same layer direction, the processing time is improved by simultaneous processing of the ANPs that make up the neurochips in the layer. In addition, pipeline processing is possible in each layer, which also reduces the processing time. For example, when an input comes into each of three neurochips connected to an analog bus, all three inputs come in at the same time, and each ANP generates a product of the analog voltage and a weight in parallel with all three, and stores it as a charge in the integrator capacitor. Then, in the next time section, each ANP generates a product of the analog input on the same analog bus.
The product of the weight and the analog input signal is added to the capacitor of the integrator in the previous time domain. After the sum of the products of the weights and the analog input signals from all the ANPs in the previous stage is generated, the sum is sampled and held. Then, it is output through a sigmoid function,
This is output when the CSI control signal is input. Then, when the output is completed, CSI falls, and after a certain time delay, C
The SO is activated to give the right to use the output bus to the ANP consisting of the adjacent neurochip in the same layer.

The present invention will be described in detail below with reference to the preferred embodiment. Figure 6 shows the configuration of a basic unit, which is a neurochip, in a first embodiment. The multiplication unit 32, the addition unit 33, the threshold processing unit 34 in the figure are
is the execution unit of the continuous neuron model, but in this embodiment, there is an output holding unit 35. Specifically, if the multiple inputs connected to the basic unit 31 are Yi and the weight set corresponding to each connection is Wi, the multiplication unit 32 calculates Yi·Wi, and the addition unit 33 calculates X=ΣYi·Wi-θ, where θ is a threshold value. If the final output is Y, the threshold unit 34 calculates Y=1/(1+exp(-X)) (1).

The value of "+1" input from the dummy node is "-θ"
The result "X-θ" is output by the adder 33. Therefore, the threshold unit 34 performs only the conversion using the S-curve.

The multiplication unit 32 is composed of a multiplying D/A converter 32a, and receives an input of an analog signal (input via an input switch unit 37) from the basic unit 31 of the previous layer or from a dummy node circuit (described later), and weight information of a digital signal to be multiplied by the input (weight holding unit 38 described later).
The multiplying D/A converter 32a inputs an analog input signal to the reference voltage terminal of the D/A converter, and inputs each bit of the weight as a digital input signal to each digital input terminal, thereby generating the product of the analog input signal and the weight. The analog adder 33
a adds the output of the multiplying D/A converter 32a and the sum calculated last time and held in the holding circuit 33b to calculate a new sum. The holding circuit 33b holds the sum calculated by the analog adder 33a and uses the held value as the previous sum.
These addition processes are performed in synchronization with an addition control signal output from the control circuit 39. The threshold unit 34 is composed of a nonlinear function generating circuit 34a, which is an analog function generating circuit, and outputs a nonlinear signal such as a sigmoid function in response to the input. When the accumulation of the multiplication results, including the addition of the threshold value (-θ), is completed, the threshold value (-θ) is added to the sum value X held in the holding circuit 33b, and the sigmoid function of equation (1) is calculated to obtain an analog output value Y.
Reference numeral 35 is composed of a sample-and-hold circuit, which holds the output value Y of the analog signal of the nonlinear function generating circuit 34a, which is output to the basic unit 31 in the subsequent layer.

Also, 36 is an output switch section, which receives an output signal control from a control circuit 39 and turns on for a certain period of time, thereby holding the output.
35 processes the final output held by the analog bus B2, 37 is an input switch unit,
When an analog output from the final output of the basic unit 31 in the previous layer is received in response to an input control signal from the control circuit 39, the input is accepted by being turned on. 38 is a weight holding unit, which is composed of a parallel-out shift register or the like, and the gate of the buffer 38a is opened (control circuit 39
When the weight input control signal by is turned on, this weight signal is held as the bit-parallel weight required by the multiplication unit 32. The bit-parallel weight is given to the multiplication unit in parallel when the multiplication control signal is given. 39 is a control circuit for the digital circuit unit, which generates an internal synchronous signal from an external synchronous signal, and is configured in this way to control the internal analog processing functions. With this configuration, the input and output of the basic unit 31, which employs the signal processing configuration of Figure 6, are realized by analog signals.

The multiplying D/A converter 32a may receive the weight information of a digital signal in parallel, or may receive the weight information serially and then convert it into parallel. Alternatively, if the weight information is configured as an analog signal, the multiplying D/A converter 32a
Instead, an analog multiplier can be used.

FIG. 7 is a specific circuit diagram of one embodiment of the neurochip (ANP) of the present invention.

This unit is composed of an input section 42, a multiplication section 43, an addition section 44, a sample/hold section 45, a nonlinear function section 46, and an output section 47. Here, there is no output holding circuit, and the sample/hold section 45 has the function of holding the output.

The input section 42 is composed of an offset cancellation section 51 and a 1x buffer 49. The 1x buffer 49 is a voltage follower that feeds back the output of the operational amplifier to the - terminal and inputs the input voltage to the + terminal. The data input is an analog time-division pulse signal. OC is an offset control signal, and when this is 1, the analog switch 66 is turned on and the 1x buffer
On the other hand, when the offset control signal OC is 0, the analog switch 66
is turned off, the other analog switch 65 is turned on, and the data input is input to the 1x buffer 49. In other words, when the offset control signal OC is 1, 0 volts is forcibly input to the neuron unit, and the offset voltage generated in the operational amplifier output of the circuit up to the multiplier output is cancelled. In the figure, analog switches 65 and 66 are controlled to switch with the inverted and normal phases of the OC signal, but a phase control circuit prevents them from being turned on simultaneously. Hereafter, this will be referred to as OC being "phase controlled."

The positive/negative switching circuit 52 is constructed by cascading two multipliers. In the multiplier, an input resistor (10 kΩ) and a feedback resistor (10 kΩ) form a 10/10, i.e., an inverted 1x voltage, and the sign of the analog voltage is determined by whether it is passed through one stage or two stages. The control signal is the sign bit (SIGN) of the digital weight data, and this SIGN bit is connected to a MOS switch 70.
The control signal of this SIGN is also phase-controlled. When the sign bit is 1, the input terminal 42
The input voltage from input 42 is inverted by the first multiplier, and because switch 67 is also on, it also passes through the subsequent multiplier, resulting in a positive phase. If the sign bit is 0, switch 69 is turned on via inversion circuit 68. At this time, switches 67 and 70 are off, so the input voltage from input 42 is input to the negative terminal of subsequent operational amplifier 71 via switch 69. Therefore, a multiplier is formed by resistor 72 in the previous stage and feedback resistor 73 of the subsequent operational amplifier, and the voltage is inverted and multiplied by 1. In other words, depending on whether the sign bit is positive or negative, the input to input 42 is generated as a positive or negative voltage, which corresponds to the excitatory and inhibitory synaptic connections. The output from positive/negative switching circuit 52 is input to the R-2R resistor network 74 of D/A converter 53 in multiplier 43, i.e., the reference voltage terminal.

First, let's explain the R-2R type D/A converter.
The internal switches are turned on or off depending on the digital weights up to B. When the digital value is 1, current flows through the right switch 75 and into the virtual ground point 78 of the operational amplifier 76. The virtual ground point 78 of the operational amplifier 76 is controlled to have the same voltage as the + terminal, which is ground, so it is a virtual 0 volts. In the D/A converter 53, R is 10 kΩ and 2R is 10 kΩ. Regardless of the switch state, current flows through the 2R resistor, and the digital value determines whether the weight current flowing through that 2R flows to the virtual ground point 78. Let i be the current flowing through the rightmost 2R. The current through the 2R second from the right, i.e., corresponding to the LSB, is the voltage applied to the rightmost 2R divided by 2R, so i is 2R × i ÷ 2R. Therefore, a current 2i flows through the rightmost horizontal R. The third 2R from the right is 2R × i + R ×
A voltage of 2i is applied, and when this is divided by 2R, a current of 2i flows. Similarly, as we move to the left, the current increases by a power of 2, becoming 4i, 8i, etc. It is the MSB to LSB that determines whether or not this weight current, which has become a power of 2, flows to the operational amplifier. Therefore, the current corresponding to the digital weight flows into the virtual ground 78 in the form of a power of 2, and since the input impedance of the operational amplifier 76 is infinite,
This current flows through the feedback resistor 78 of the operational amplifier 36. Therefore, if the input voltage is E, the output voltage _Vout of the D/A converter is Here, D ₀ is the LSB and D _n-1 is the MSB.
That is, the output of the multiplication unit 3-3 is equivalent to the input voltage E multiplied by the weight.

The weighting coefficients are controlled by digital values input from MSB to LSB. Meanwhile, adder 44 performs cumulative addition by using a Miller integrator in a time-division multiplexed manner for each product of each voltage of the time-division multiplexed analog signal and the digital weighting data. Sample/hold circuit 45 then samples and holds the addition result.

Next, the adder 44 will be described. The adder 44 is an integrator consisting of a resistor R and a feedback capacitor C. The input of the adder 44 has a time-division addition control unit 55, and when the phase-controlled sample/hold signal S/H signal is 1, the output voltage of the multiplier 43 is input to the virtual ground point 79 of the operational amplifier. When the S/H signal is 0, an inverting circuit 80 turns on a switch 81, and the multiplier
Since the output of 43 is connected to ground via a resistor R, it is not added to the feedback capacitor C of the adder 44 .
Now, when the S/H signal is 1, the output voltage of the multiplier 43 is
The input voltage is input to the negative terminal of the operational amplifier 102 via the resistor R, and the current obtained by dividing the input voltage by the resistor R is input to the feedback capacitor C via the virtual ground. After this, the S/H signal becomes 0 again, and the multiplication unit 43 and the addition unit 44 are disconnected, so the multiplication unit 43
The feedback circuit 82 of the integrator circuit including the capacitor C has four
An offset cancel function is added using two switches. Now, if the offset control signal OC becomes 1, switches 83 and 84 are on, and 85 and 86 are off. When the offset control signal OC becomes 0,
An input voltage is applied to the data input terminal DATA-INPUT of the data input section 42, and the output of the multiplication section 43 corresponding to the input voltage is applied to the resistor R
The input to the capacitor C is
Switches 85 and 86 are on, and the polarity of capacitor C is negative on the side connected to the negative terminal of the operational amplifier and positive on the side connected to the output of operational amplifier 102. Next, when the offset control signal OC is 1, the data input is forced to 0. In this case, if there is no offset via the positive/negative switching circuit 42 and the D/A converter 53 of the multiplier 43, the output of the D/A converter 44 will be 0 volts. However, the presence of operational amplifiers 49, 103, 71, and 102 generates an offset voltage, which is stored in capacitor C of the adder 44. In this case, unlike the previous case where the offset control signal OC was 0, switches 83 and 84 are on, reversing the polarity of capacitor C. Therefore, when the offset control signal OC is set to 0, the polarity of capacitor C changes, and the offset voltage generated when an input signal is input is canceled. In this way, the present invention is configured to provide an equivalent offset cancellation function by reversing the polarity of capacitor C. The switch 87 is controlled by a reset signal, and when a reset signal is given at the start of processing, the voltage of the capacitor C _h is set to zero, forcing the output of the adder to zero.
This OC signal is also phase-controlled.

The output of the adder 44 is input to the sample/hold circuit 45. In the sample/hold circuit 45, when the phase-controlled sample/hold control signal S/H _out is 1, the output of the adder 44 is stored in the capacitor Ch via the switch 88. When the S/H _out control signal is 1, the output of the adder 44 is stored in the capacitor _Ch via the inverting circuit
The control signal of the switch 90 becomes 0 by 94, one terminal of the capacitor C _h is not grounded, and the switch 91
When the switch 91 is turned on, the final output signal of the unit is input to the capacitor C _h via the switch 91. That is, the final output signal at that time is fed back from the output terminal of the operational amplifier 96 and applied to the lower side of the capacitor C _h . Therefore, the voltage obtained by subtracting the value of the final output signal from the output of the adder 44 is held _in the capacitor C h.
When the control signal is 0, the switches 89 and 90 are turned on, the lower side of the capacitor _C is grounded, and as a result, the voltage stored in the capacitor C, that is, the voltage obtained by subtracting the final output value from the output of the adder 44, is input to the positive side of the 1x operational amplifier 93 via the switch 89.
The operational amplifier 93 acts as a buffer, and the output of the operational amplifier 93 becomes the input of the sigmoid function. When the S/H _out control signal is 1, the switch 88 is turned on, and when the voltage of the difference between the output value of the adder and the final output value is stored in the capacitor C _h , the switch 92 is turned on. Therefore, the operational amplifier 93
At this time, the offset voltage ΔV is input to the lower side of _Ch via the sigmoid function 46, the operational amplifier 96, and the analog switch 100 and the switch 91. Therefore, when the S/H _out control signal is 0, that is, when the switch 89 is on and the switch 92 is off, the voltage stored in _Ch , that is, (the output of the adder -
Offset voltage ΔV) is the op-amp 93 and the sigmoid function
The final output is obtained via 46, but when the S/H _out signal becomes 1, the offset voltage generated at this time is also Δv, so the offset voltage is cancelled out as a result.

The nonlinear function part that generates the sigmoid function has a nonlinear circuit selection control part, and when the phase-controlled Se1Sig signal is set to 1, the switch 95 is turned on and the output of the sigmoid function is input to the next stage. However, when the Se1Sig signal is 0, the inverting circuit
The control signal of the switch 98 becomes 1 via the switch 97, turning it on, and the output of the sigmoid function is cut off.
When the Se1Sig signal is 0, the output voltage of the sample/hold section is input directly to the operational amplifier 96 without passing through a sigmoid function. The operational amplifier 96 is essentially a 1x operational amplifier that feeds the output directly back to the - terminal, acting as a buffer.
That is, it acts as a buffer that makes the output impedance zero.

The output section 47 is connected to a time-division analog output section 64 and an output control section 63. When CSI is 1, the switch 99 is on and the switch 101 is also on, so the final output value of the operational amplifier 96 is output to DATA-OUTPUT and is fed back to its - terminal, so that the operational amplifier 96 functions as a 1x operational amplifier. At the same time, the final output value is sampled/
The feedback is sent to the hold unit 45. On the other hand, when the CSI is 0,
At this time, the switch 100 is turned on through the inverter 104, and the switches 101 and 109 are turned off.
The output of the operational amplifier 96 is not output to the DATA-OUTPUT line. However, since the switch 100 is turned on to form a 1x buffer, the output of the operational amplifier 96
The voltage follower operation is carried out without any disruption.
The output unit 47 is a circuit that determines whether to transmit an output pulse voltage by the output control input terminal CSI. This CSI is output as CSO via a delay circuit 105, and determines the time timing of the output analog signal to the adjacent neurochip in the layer. Therefore, in the present invention, the analog signal from the output unit 47 is transmitted in a time-division manner, so it does not compete with analog signals from other neurochips on the bus.

Figure 8 shows the offset cancellation OC in Figure 7.
0, OC1, sign SIGN to PN, -PN, sample/hold SH
to SH11, SH10, sample/hold S/H _out to SH21, SH2
Phase control is achieved using two signals: 0, the sigmoid selection signal Se1Sig is -SIGM, SIGM, and the daisy chain signal CSI is CS, -CS. In other words, this is an embodiment in which one control signal is composed of two signals, one positive and one negative, and the phases are shifted to prevent different switches controlled by the positive and negative phases of these control signals from being turned on simultaneously. The capacitor Cf and resistor _Rf connected to the output end of the D/A converter 53 are used to match the feedback signal of the operational amplifier 76 to the calculation speed of the D/A converter, and the digital input of the D/A converter is applied to the DT terminal.

In FIG. 8, the same parts as those in FIG. 7 are given the same numbers and their explanations are omitted.

Figure 9 is a timing diagram for the integrator. The data clock DCLK and the weighted clock WCLK are basic operating clocks, and a high-speed weighted clock WCLK is output during the half period of the high state of the data clock DCLK. The weighted clock WCLK signal is a synchronous clock for capturing weighted serial data. The data clock DCLK signal is a basic clock for processing the analog input signal. Synchronous signal
SYNC is a synchronization signal that synchronizes each analog neuron processor ANP in each layer. The change in the integrator's output voltage is shown by the waveform indicated by the triangle below. The integral waveform is controlled by the sample/hold control signal SH pulse, and integration is performed while this pulse is high. That is, charging of the integrator's capacitor C begins, and while this sample/hold control signal SH pulse is high, charge gradually accumulates in this capacitor and the voltage rises. When the sample/hold control signal SH pulse goes low and is cut off, the charging operation stops. Therefore, only the charge within this integration time range is significant. By controlling the pulse width of this sample/hold control signal to shorten or extend the integration time range, the input voltage remains the same, but the resulting integration voltage will be _Va ' when the sample/hold control signal S/H pulse width is W.

The sample/hold control signal SH falls, the polarity of the integrator capacitor changes due to switching control, and the integral output to which the offset is added is inverted. Then, when the offset control signal OC is high and the sample/hold control signal SH rises again, the offset voltage
_Vb ( _Vb ') is added to the capacitor, and when the SH signal falls, the offset-cancelled integral output value _Va - _Vb (Va _' - _Vb ') is sampled and held with the polarity reversed.

Next, we will explain the hierarchical neural network.
Figure A is a conceptual diagram of a hierarchical network. In a hierarchical network, input data coming in from input nodes 110 in the left-hand input layer is processed sequentially in one direction, toward the right. Each neuron 112 in the middle layer receives the output of the previous layer, including dummy nodes 111, through complete connections within the layer. For example, if the input layer has four input nodes 110, one dummy node 111 is added to them, and the input layer appears to each neuron 112 in the middle layer as five neurons. Here, the dummy node 111 controls the threshold, and is a sigmoid function of the sum-of-products result X. By adding a constant value -θ to the value X of f(x - θ), it is shifted in the positive direction of the X axis to obtain a value f(x - θ). This is equivalent to changing the weight corresponding to the dummy node 111 within the neuron, but the constant value θ is generated using a max value node circuit, which will be described later. By preparing weights for the dummy nodes in this way, it is possible to realize the threshold value using the weights. From the output layer neuron 112 to the hidden layer, it appears that there are four neurons. The input data added to the input layer is subjected to a product-sum operation using the weight data in the hidden layer neuron 112 and the output layer neuron 112, which results in the generation of output data.

When the hierarchical structure shown in Figure 10A is realized using the ANP of the present invention, as shown in Figure 10B, independent analog buses B1, B2, and B3 are provided between each layer, i.e., between the input and hidden layer, between the hidden layer and output layer, and at the output of the output layer. This results in a structure in which all vertical ANPs can be executed in parallel. A sample-and-hold circuit SH is attached to the output of the output layer.

FIG. 11 is a diagram showing the configuration of one embodiment of the present invention.

In the figure, the input side circuit 120 corresponds to the input layer, 121 is an analog neuron processor ANP, i.e., a basic unit, which is a basic unit of the hierarchical network, 121-h is a plurality of basic units that constitute the intermediate layer, and in the case of three or more layers, 121-h is a plurality of basic units that constitute one or more intermediate layers, and 121-i is a single basic unit that constitutes the output layer.
One or more basic units 121-j are output layer circuits.
Electrical connections are made between the basic units 121-i and 121-j, and a hierarchical network indicated by 130 is constructed by weights set corresponding to each of these connections.

The basic unit 121 includes at least a multiplication unit 122 and an addition unit 123.
3, a threshold unit 124, and optionally an output holding unit 125. The multiplication unit 122 receives a plurality of inputs and weights for these inputs, performs multiplication, and outputs the result to an addition unit 123.
The multiplication unit 122 adds up the multiplication results for all basic units 121 in the previous layer, and the threshold unit 124 converts the summation result obtained by the addition unit 123 using a non-linear threshold function to calculate the final output. If there is an output holding unit 125, it holds the final output obtained by the threshold unit 124.
The input and output to the device are configured to be realized using analog signals.

Reference numeral 140 denotes an analog bus, which is a common line provided for electrical connection between the input layer and the first hidden layer (140a), between the hidden layers (140b), and between the last hidden layer and the output layer (140c). Reference numeral 150 denotes a main control circuit, which controls data transfer within the hierarchical network 130. This main control circuit 150 includes a drive unit selection means 151, a weight setting means 152, a threshold processing activation means 153, and an output value transmission means 154.

In this embodiment, the drive unit selection means 151 processes the basic units 121 of the previous layer to be selected in time series order. Then, the output value transmission means 154 synchronizes with this selection process and transmits the final output of the analog signal held by the selected basic unit 121 via the analog bus 140.
It processes the data so that it is output to the basic unit 121 in the subsequent layer in accordance with the time division transmission format.
The multiplication unit 122 of the basic unit 121 in the subsequent layer is a weight setting means
The basic unit 121 of the previous layer set by 152
The weight corresponding to the connection with the input is selected in order, and the weight is multiplied by the input, and the adder 123 sequentially adds the multiplication results obtained by the multiplier 122. When it is confirmed that all product-sum processing for the basic unit 121 of the previous layer has been completed, the threshold processing start means 153 performs processing to convert the basic unit 121 with a non-linear threshold function f.
1 outputs f(X-θ). This latter layer becomes the new former layer, and the same process is repeated for the next latter layer. With this data transfer method, the output pattern corresponding to the input pattern is
It will be output from the output layer of the hierarchical network.

FIG. 12 shows an embodiment of a network configuration data processing device configured by connecting a plurality of basic units 121 in a hierarchical manner. In this embodiment, each basic unit 12
The weight given to 1 is temporarily stored outside the unit, and the CSI
This embodiment is an embodiment in which the control of the hierarchical network shown in FIG. 2 is provided by a single common analog bus 140.
(Sometimes, an identifier a to c is attached.) Therefore, the final output value output from the output switch section 36 of the basic unit 121 (which becomes the input to the basic unit 121 located in the subsequent layer) is configured to be output in analog signal output mode. In this embodiment, the h and i layers are intermediate layers, and the output layer is shown as the j layer.

In the figure, 161 is a weight output circuit provided for each basic unit 121, which outputs a weight for the weight holding unit 38 in the basic unit 121, 162 is a weight signal line which connects the output of the weight output circuit 161 to the weight holding circuit 38,
Reference numeral 20 (which may be given an identifier a to n) denotes an input circuit provided according to the number of input signals, which outputs an initial signal that becomes an input pattern corresponding to the input layer of the hierarchical network. Reference numeral 164 (which may be given an identifier a to d) denotes a synchronization control signal line including CSI, which transmits a synchronization control signal from the main control circuit 150 that controls data transfer to the weight output circuit 161, the input circuit 120, and the basic unit.
This is a signal line for transmitting the signal to the control circuit 39 of 121.
Although this synchronous control signal line 164 is shown as a common line in the figure, in detail, each circuit is connected to the main control circuit 150 by an individual signal line.

13 shows a detailed system configuration of the main control circuit 150. In the figure, the main control circuit 150 includes at least an external bus interface 150a, a microcode memory 150b, a program sequencer 150c, a control pattern memory 150d, and a weight data memory 150e. The external bus interface circuit 150a is connected to a host computer 150y and an external storage device 150z via a main bus 150x, and receives operation instructions from the host computer 150y. The microcode memory 150b stores microcode that defines the operation of the program sequencer 150c. The program sequencer 150c controls addresses of a control pattern memory 150d and a weight data memory 150e in accordance with the microcode in the microcode memory 150b. The control pattern memory 150d has output signal lines that are individually connected to the input side circuits 120, the intermediate layer, and the basic units 121 in the output layer, and in accordance with instructions from the program sequencer 150c, it controls one circuit of each group, or one basic unit 121 in each group, i.e., for each group of the input side circuits 120, the intermediate layer group, and the output layer group.
The weight data memory 150e outputs weights (digital data) to each weight output circuit 161 in accordance with instructions from the program sequencer 150c so that a weight is assigned to each basic unit 121 in synchronization with the time-division input signal. The host computer 150y includes an MPU and a main memory, and performs control to determine weights using a learning algorithm such as backpropagation, and to assign input pattern _Yi .
The external storage device stores data for constructing a neurocomputer.

Next, the operation process of the embodiment of FIG. 12 configured as above will be described with reference to the timing chart shown in FIG.

When a request for conversion to an output pattern is given from the host computer 150y via the main bus 150x, the main control circuit 150
By cyclically sending an output control signal to the input side circuits 120 in a time series manner, the
In other words, the main control circuit 150 turns on the synchronous control signal line 164a for each input circuit 120 in order to select the input circuits 120 in order from the control pattern memory 150d in response to an instruction from the program sequencer 150c.
In order to output the input pattern _Y1 to the analog bus 140a, only one of the n synchronous control signal lines 164, which is designated as 164a- ₁ in the figure, is turned on to open the gate of the input side circuit 120a, and the other synchronous control signal lines 164a are turned off.
(indicated as 164a-2 in the figure) is turned on, and the other synchronous control signal lines 164a are turned off.
The synchronous control signal line 164a is turned on and off until the input pattern _Yn of the input side circuit 120n is output to the analog bus 140a.
In order to provide a weight to each weight output circuit 161, the output of the weight data memory 150e is set for each weight output circuit 161 via the synchronous control signal line 164b in synchronization with the ON operation of each synchronous control signal line 164a.

In FIG. 14A, the synchronous control signal of this synchronous control signal line 164a is represented by Yi output control signal (i=1 to n), and the input side circuit
14A shows the process of cyclically selecting the input circuits 120 in a time series manner. Here, n is the number of input circuits 120. The input circuit 120 selected in this manner processes the analog signal Yi provided as an input pattern onto the analog bus 140 (referred to as input layer analog bus 140a in the figure) provided between the input circuit 120 and the h layer. This input pattern is provided via a host computer 150y. Therefore, as shown in FIG. 14A, the analog signal _Yi is transmitted to the input circuit 120 via the input layer analog bus 140a.
The number of 0s is sent out in order, and the first input pattern Yi, then the next input pattern Yi, and then the next input pattern _Yi are sent out repeatedly in this order.

When the multiplier 122 of each basic unit 121 in the h layer receives this analog signal Yi, it executes the above-mentioned multiplication process (Yi·Wi) using the weight Wi stored in the weight storage unit 38, which is set by the main control circuit 150. It is assumed that this weight Wi has already been determined by the MPU according to the back propagation method.

Therefore, as shown in FIG. 14B, the main control circuit 150 outputs the weight Wi according to the selected input side circuit 120 via the weight output circuit 161 in synchronization with the selection process of the input side circuit 120.
are set in the weight storage unit ₃₈ of each basic unit 121 in the h-layer.
and W _i are multiplied and summed with the previous sum of products.

The process of setting the weights to the basic units 121 can be realized in either analog or digital signal mode.

Since the weight is specified for each connection, it should be expressed as Wij (j is the basic unit number of the h layer) to be precise, but for ease of explanation it is expressed as Wi.

Here, the processing operation of the basic unit 121 will be explained according to the timing chart of signal processing of the basic unit 121 shown in Fig. 15. Note that here, the basic unit 121 (121a in the figure) in the hidden layer will be explained.

First, when the control circuit 39 receives a synchronous control signal from the control pattern memory 150d of the main control circuit 150 via the synchronous control signal line 164 (164b-1 in the drawing), it turns on the input control signal (c) corresponding to OC, and the input switch unit 37
At the same time, the main control circuit 150 turns on the weighted input control signal (d) that opens the gate of the buffer 38a and the output control signal ( _h1 ) that corresponds to CSI that turns on the output switch unit 36. At this time, the main control circuit 150 sequentially turns on the CSIs on the synchronous control signal line 164a in synchronization with the clock (a), so that the input side circuit 12
Input pattern signal held at 0a, 120b, ... 120n
_Yi is given to the multiplying D/A converter 32 a via the analog bus 140 and the input switch section 37 .

On the other hand, the main control circuit 150 also stores the weight data memory 150
The weight of W i is given to the weight output circuit 11 via the synchronous control signal line 164 b (164 b-2 in the figure). This weight (digital data) W _i is then input to the weight holding unit 38 through the buffer 38 a.
At this time, the CSI output control signal (h
₁ ) is turned on for one cycle of clock (a),
During this time, the analog gate of the sample-and-hold circuit of the basic unit 121 is in an open state, and the held analog value is output onto the analog bus 140b of the subsequent i-th layer via the output switch unit 36. Now, when the weight _W1 of the digital value is stored in the weight holding unit 38, the multiplication control signal (e)
is turned on, the multiplying D/A converter 32a receives the analog signal _Y1 and the weight _W1
Then, the addition control signal (f) is turned on, which activates the analog adder 33a, which is made up of an integrator, and adds the analog value held just before (which is initially cleared to zero) and the multiplication result of the multiplying D/A converter 32a, and the addition result is stored again in the sample-and-hold circuit 33b.

With the above operation, one bus cycle is completed, and in synchronization with the next clock (a), the input switch unit 37 provides the input pattern _Y2 of the input side circuit 120b, and the weight output circuit 16
Since the weight _W2 corresponding to this input pattern _Y2 is given from 1, the input pattern _Y2 is multiplied by the weight _W2 ,
The result of this multiplication is then added to the hold value of the sample-and-hold circuit 33b. This operation is repeated until the processing of the input pattern _Yn of the input side circuit 120n is completed. When the multiplication of the input patterns _Yn and _Wn is completed, the conversion control signal (g) is turned on, and the accumulated value of this multiplication result is input to the nonlinear function generating circuit 34a of the thresholding unit 124, and the corresponding Y value is held. That is, the thresholding unit 124 performs the above-mentioned calculation Y=1/(1+exp(-X+θ)), and as a result, the basic unit 121
The final output value Y, which is the final calculation output of the above, is calculated and stored, and this result is output to the subsequent analog bus (140b) at the rising edge of the next output control signal ( _h1 ). When this value Y is calculated, the accumulated value of the adder 33 is
It is cleared by the input clear signal in synchronization with the 120 selection cycle.

By performing the operations described above, each basic unit 121 obtains a final output value Y from the input pattern Y _i and the weight W _i .

Hereafter, the explanation will return to the configuration of the embodiment shown in Fig. 12. As described in detail with reference to Fig. 15, when the processing for the input patterns from all the input side circuits 120 is completed,
The main control circuit 150 again provides a synchronization control signal to each basic unit 121 (which may have an identifier a through n), which then executes a synchronization operation in accordance with the input pattern Y _i and new weight W _i newly provided to the input side circuit 120.

On the other hand, the basic unit 12 of the h layer obtained in this way
The final output value Y of the input signal is held and is transmitted to the basic unit 12 of the i-th layer located in the next stage via the analog bus 140b by the same processing as that performed on the input side circuit 120.
That is, the main control circuit 150 transmits the synchronous control signal via the synchronous control signal line 16 to the control circuit 39 of each of the basic units 121a to 121n in the h layer in a time-division transmission format.
4b (164b-1 in the figure), the output control signals _h1 to _h2 (the
15) in a time-series cyclic manner, the output switch section 36 of each basic unit 121a, 121b, ... is sequentially and cyclically turned on in a time-series manner. As a result, the analog signal of the final output value held in each basic unit 121a to 121n is sent to the multiplier section 122 of each basic unit in the i-th layer in a time-division transmission format. Each basic unit in the i-th layer executes the same processing operation as described above, and executes a similar time-division transmission process to obtain the final output value Y of the basic unit 121 in the i-th layer, thereby obtaining the final output value Y of the basic unit 121 in the output layer. In other words, the main control circuit 150 controls each basic unit in the same way via synchronization control signal lines 164c, 164d connected individually to each basic unit 121 in the hidden layer and output layer.

Fig. 16 is a block diagram of a neurocomputer of the present invention that realizes a hierarchical neural network. Analog neuron processors ANP1 to ANP5 are arranged in parallel in each layer from a neurochip, and analog buses (B1, B
In the figure, ANP1, 2, and 3 form the hidden layer, and ANP4 and 5 form the output layer.
On the input side, there are daisy-chain circuits 171 and 172 for inputting analog input signals in a timely manner. The circuits indicated by S/H are sample/hold circuits 173 and 174. ANP
Each of 1 to 5 requires a control logic signal, so a master control block (MCB) 181
The data clock DCLK is fed to the daisy chain circuits 171 and 172 on the input side of all ANPs.
The weighted clock WCLK is also given to all ANPs and the input daisy-chain circuit 17.
1,172 is a high-speed clock for weight data.
Weight data is input to 2 and 3 in synchronization with the weight clock WCLK. Also, the synchronization signal SYNC1 is a synchronization clock given to the ANP of the hidden layer, and the synchronization signal SYNC2 is a synchronization clock given to the ANP of the output layer.
SH1 and OC1 are sample/hold control signals and offset control signals for the ANP in the hidden layer, and SH2 and OC2 are sample/hold control signals and offset control signals for the ANP in the output layer.

The daisy chain circuits 171 and 172 on the left side are input side circuits that correspond to the input layer. To realize the input nodes, that is, the neurons in the input layer, analog input signals given from analog input ports 0 and 1 must be input into the circuit at the same timing as the ANP outputs the analog signals in a time-division manner. In other words, from the perspective of the output layer,
The basic operation of the output layer is to receive analog signals from ANPs 1, 2, and 3 in the hidden layer, preceding ANPs 4 and 5 in the output layer, via analog bus B2 in a time-sharing manner. The same relationship must exist between the hidden layer and the input layer. From the perspective of the ANPs in the hidden layer, the ANP in the input layer must appear to precede it. This imposes a constraint that the ANPs in the hidden layer must output analog signals to analog bus B2 in the same manner as they output analog input signals from analog input ports 0 and 1 to analog bus B1 according to a set rule. That is, input signals from analog input ports 0 and 1 are transmitted to analog bus B1 in a time-sharing manner. The analog signal from analog input port 0 is transmitted to analog bus B1 at an appropriate timing, and the next analog input signal from analog input port 1 is transmitted to the same analog bus B1 at the same time. To synchronize this, a daisy-chain circuit 171 receives an input control signal CSI, which is output at a fixed timing, and after a fixed time, the circuit outputs an output control signal CSO.
This CSI is output from CSO1 of the master control circuit 181. The daisy chain circuits 171 and 172 are a kind of delay circuit.
When each daisy chain circuit 171 receives an input control signal CSI from the master control 181, it passes the CSO signal to the next vertically adjacent daisy chain circuit 172 so that it outputs the analog output signal of analog input port 1. This operation is called daisy chain control.

When CSO1 of the master control circuit 181 rises, the switch 175 turns on, and the analog input signal of analog input port 0 held in the sample/hold circuit 173 is placed on the analog bus B1. Since CSO1 is the CSI of the daisy chain circuit 171, CSO rises a fixed time after it falls. This is the CSI of the daisy chain circuit 172, and at the same time, controls the switch 176 to turn it on, so the analog input signal of analog input port 1 held in the sample/hold circuit 174 is placed on the bus B1. This daisy chain control is necessary in this hierarchical system. In other words, if the analog input signal is output from analog input port 0 to the analog bus B1 via the sample/hold circuit 173, the analog input signal is then output from analog input port 1 to the sample/hold circuit 174.
4 to the same analog bus B1. Looking at each neuron in the hidden layer, the analog input signal from analog input port 0 and the next analog input signal from analog input port 1 come in sequentially in a time-division manner.

Each of the daisy chain circuits 171 and 172 delays the input control signal CSI by a specific time to prevent bus contention on the analog bus B1, and then outputs the output control signal CSO.

In the middle layer, ANP1, which receives the output control signal CSO2 from the master control block 181 as CSI, outputs an analog signal, then passes CSO to ANP2 as CSI, and ANP2 then outputs. ANP3, which receives CSO from ANP2 as CSI, then outputs an analog signal. In short, ANP1, 2, and 3 output in this order, and the daisy-chain operation of the middle layer is completed. In parallel with this, the master control block 181, which manages all operations, gives CSO3 to ANP4 in the output layer, which then outputs, and after the output is complete, ANP4 passes CSO to ANP5.
ANP5 will output when given.

The outputs from ANP4 and ANP5 of the output layer are sampled and held in sample/hold circuits 177 and 178, respectively, by a CSO3 signal from a master control block 181 and a daisy-chain output control signal CSO from ANP4.
This output voltage is output as an analog output signal from analog output ports 0 and 1, and is also used as an analog multiplexer.
After being selected by 179, it is A/D converted by A/D converter 180,
The data is input to a digital control means that is composed of an MPU 182, a memory 183, and a communication interface 184.
For example, the output signal is compared with the teacher signal stored in the MPU during learning to check whether it is the desired output signal, and the weight data in the weight memory (described later) is changed based on the result. The max value node circuit 187 receives the dummy node control signals DCS1, DCS2 from the master control block 181.
S2 is applied to output enables 1 and 2, and the output terminals are connected to analog buses B1 and B2.

Figure 17 is a timing diagram of the hierarchical neurocomputer according to the embodiment shown in Figure 16. The control signal lines for each layer are shown. First, the data clock DCLK and weighted clock WCLK, which are the basic operating clocks, are input simultaneously to all ANPs in the same layer and to daisy-chain circuits 171 and 172 on the input side.

The weight clock WCLK is a serial synchronous pulse for sending the weight digital data serially and is a synchronous clock for reading the weight from the weight memory block. The timing at which the input data is taken in is determined by the respective control signals. First, in the timing chart of Figure 17, CSO1 is the master control block 18.
1, that is, the daisy chain control signal CSO1 output from the daisy chain circuit 171, that is, the daisy chain control signal CSI to the daisy chain circuit 171. In the daisy chain circuit 171, CSI causes the first analog input signal to be output from the analog input port 0 to the analog bus B1 via the sample/hold circuit SH173. That is, in the timing chart, the analog signal is output to the analog bus B1. At this moment,
A voltage is applied to the analog bus B1, and ANP1, ANP2, and ANP3 perform multiplication and accumulation operations on this analog signal in parallel.
passes through the daisy chain circuit 171, and the next CSI rises as shown in a predetermined time after CSO falls, and that CSI enters the daisy chain circuit 172. The next CSI is a control signal that enters the second daisy chain circuit 172 in the input layer. Then, CSI
While this is high, the analog input signal from analog input port 1 is sent to ANP1, ANP2, and ANP3 via the sample/hold circuit SH174.
The input is input to NP3, where a product-sum operation is performed. DCS1 from the master control block 181 is a control signal to the dummy node. In addition to the input, each layer also has a signal from the dummy node, so it has a form of (number of neuron nodes + 1) nodes, and there are two inputs in the input layer, but the AN of each hidden layer
From the perspective of P, it appears to be three inputs. Explaining this in terms of time, two CSIs and one DCS1 are control signals that make up one block. The input cycle is
It starts from I and ends at the input to the dummy of DCS1. The dummy node is a max value node circuit 187, which outputs a fixed threshold voltage to the analog bus while DCS1 is input. That is, as shown by DC
While this voltage is being output after S1 rises, each ANP in the middle layer performs a multiplication and accumulation operation in the same way as a normal input,
That fixed voltage is added to the result of the multiplication and accumulation of the previous two analog input signals, i.e.
After the multiplication, an addition is performed. SYNC1 goes high on the falling edge of DCLK before CSO1 rises, and goes low on the next falling edge of DCLK after DCS1 rises. This signal synchronizes the input phase. The analog input and weight data are multiplied while WCLK is being input. The sample/hold signal SH1 input to the ANP in the hidden layer has two peaks, M1 and M2. The product is taken just before the first peak, M1, and the sum is generated and held at the peak. Then, at the next peak, M2, the offset voltage _Vb (see Figure 9) is subtracted and sampled/held. This process is repeated sequentially for all input analog signals until the product-sum calculation is complete. In this case, each ANP in the hidden layer, including the dummy, performs the product-sum operation three times. This completes one process for each ANP in the hidden layer, completing the sum of the products for the three inputs.

Also, in the timing chart, when DCLK is high immediately after DCS1 falls, the results of product-sum calculations on the three signals from analog 2 output ports 0 and 1 and the dummy node are held in the capacitors (C _h in sample/hold unit 45 in Figure 7) of ANP1, 2, and 3. This type of operation is basically repeated, but the rising edge of the CSO2 signal output from master control block 181 determines when the output signal of ANP1 is output to analog bus B2, which is located between the hidden layer and the output layer.

The offset control signal OC1 shown below SH1 is
Offset cancellation is performed inside the ANP. That is, each ANP is an analog circuit that includes an operational amplifier internally, and the circuit itself has an offset, so the control signal for canceling this offset is the OC signal. As shown in OC1, one pulse is output each time a multiply-and-accumulate operation is performed, and offset cancellation is performed internally. As shown in the timing chart, when CSO2 rises, the signal that was held in ANP1 is output from ANP1 to analog bus B2,
While CSO2 is high, ANP4 in the output layer performs a multiply-and-accumulate operation. The rising edge of CSO2 indicated by is the timing to output the result of the multiply-and-accumulate of the previous input result.

Next, the timing between the hidden layer and the output layer will be explained using FIG.

In the figure, the daisy-chain control signal output from the intermediate layer, , , and the output from the output layer,
The analog signal that appears on the analog bus in synchronization with the output of the daisy chain control signal from the input layer described above, and the result of one processing cycle before appears for the analog signal that is input on the analog bus in synchronization with the output of the daisy chain control signal from the input layer described above. The execution of pipeline processing will be explained later, but
At the rising edge of CSO2 shown in the timing chart, the output of ANP1 is output. Looking at the signal of SH2 in the timing chart at the rising edge of CSO2 shown in, two pulses are output. The SH2 signal is input to the first ANP4 in the output layer in the block diagram of Figure 16. That is, at the two peak pulses of the SH2 signal, ANP
One sum operation is performed within 4. As shown in the figure, the hidden layer has three hidden layer neurons, ANP1, 2, and 3, but one dummy node is added by the max value node circuit 187, so it is assumed that there are four neurons in total. Therefore, the two peak pulses of the SH2 signal are output four times as seen from the part , and the analog signal of the hidden layer is input to ANP4 with these four sets of peak pulses of the SH2 signal, and the product sum is calculated. Naturally, this operation is performed at the same time as the ANP in the hidden layer is performing the product sum operation on the input signal, and this is pipeline processing. The signal below CSO2 is the CSO signal of ANP1 in the hidden layer, and this is the CSI for ANP2 in the same hidden layer. This is the part indicated by . Below that is the CSO of ANP2, and below that is the CSI of ANP3, which is . Below that is the C of ANP3
SO is shown, and below that is the CSI of the dummy node, which is a signal output from DCS2, the master control block. Looking at the CSI, it is input in the order of , , , to ANP1, ANP2, ANP3 in the hidden layer, and then to the max value node circuit 187 of the dummy node. During this time, the SH2 signal outputs four pulse signals with two peaks. In other words, the neuron in the output layer of ANP4 adds four products of the input analog signal and the weight. When CSI is input to ANP1 at part , an analog signal is output from ANP1 to the analog bus between the hidden layer and the output layer, and this is input to ANP4. At this time, the corresponding weight data is input to ANP4, and the product is executed, added at the first peak of the SH2 signal, and sampled and held at the second peak. When this calculation is complete, ANP1
The CSO signal rises from this point, and this becomes the CSI for ANP2.
This is the state where the weight data and the data on the analog bus are multiplied and the sum is calculated. After a predetermined time has passed since the falling edge of ANP3, CSI to ANP3 goes high and a sum-of-products operation is performed in ANP4 as shown in . This sum-of-products operation is performed within ANP4, and at the point where the fixed voltage output from the matrix value node circuit 187 is A
This is input into NP4 and added to the accumulated product sum stored inside.

The above operations are also performed in parallel for ANP5 in the output layer. This is simultaneous processing. The timing at which the result of the product-sum operation calculated by ANP4 is output to the analog bus B3 connected to the output layer is the rising edge of CSO3 output from the master control block 181. The control signal that the max value node circuit 187 uses to output to the analog bus B2 is DCS2, which corresponds to this. Up to this DCS2, the operation is up to the output of the calculation result in the intermediate layer.
The same operation is performed for the signals written below this on the timing chart, and these are signal pulses that define the operation of the output layer connected to the hidden layer and the cascade.
When CSO3 rises, the result of the sum-of-products operation calculated by ANP4 is output. In the output layer, two outputs, ANP4 and ANP5, are output. For example, when CSO2 rises,
This is the signal that goes into ANP1, and its rising edge is delayed from DCLK. This is because when performing a multiplication operation between the analog input signal and the digital weight data, the digital data is read in serially with WCLK, and this is converted internally into parallel data. Taking into account the time it takes to read the digital data and for the analog input signal to reach the D/A converter, i.e. the multiplication processing unit, the rising edge of CSO2 is delayed. In other words, the reason there is a delay at the beginning is that
This includes the time to read the data, that is, the time to read the serial data. The data is set after a while from the rising edge of DCLK, that is, 16 WCLK.
The analog multiplication starts 8 WCLK cycles after CSO2 rises.

Figure 18 is a timing chart showing the timing of reading in the digital weight data. In this figure, the master clock MCLK, the synchronization signal SYNC, the weight clock WCLK, the data clock DCLK, and the actual weight data WDATA are shown. The weight data WDATA is read out bit-serial from the weight memory, and 16 bits are input serially. S
is the sign bit, and B14 to B0 are the numeric bits.
In the figure, the B8, B7, and B6 portions of the weight data WDATA are shown enlarged at the bottom of the figure as they correspond to the weight clock WCLK. The weight clock WCLK has a period of 250 nsec and a duty ratio of 50%. An address is given to the weight memory after the propagation delay time of the address update counter inside the sequencer from the falling edge of WCLK. In other words, the address of bit n of the weight memory (RAM) is the weight data W
This is the address of the weight memory where bit 7 of DATA is stored. After this address is determined, bit 7 is read out after time tAA. The transition from bit 7 to bit 6 is determined by the transition to the next cycle of the weight clock.
Bit 6 is read out in the next cycle.
The 16 bits are input to the ANP, and the product of this with the analog voltage input to the ANP is calculated by the internal D/A converter, so the start of the analog voltage input is input long after the rising edge of the data clock DCLK. In other words, there is a time period between when the analog input voltage is input and when it reaches the D/A converter, so it is necessary to control this time and the time when the digital weight data is set internally, and input the analog voltage so that the arrival time of the weight data and the analog arrival time exactly match.

For example, the analog input voltage rises from around weight data B7, and weight data B0 is input.
After that, it is necessary to control the time so that multiplication with the analog value starts when all the weight data has been determined internally. The addition is then carried out during the period when DCLK next goes low.

The operation time of the ANP is determined by the SYNC signal, WCLK, and data DCLK. The analog input voltage has a significant time error in the voltage arrival time from the input terminal of the ANP to the D/A converter that performs the multiplication with the digital weight data.
To allow for a margin, the rising edge of CSI begins later than the rising edge of DCLK.

19A is a block diagram of the master control block 181. The master control block 181 is a part that controls all control signals. Its main components are an external bus interface circuit 200, a control pattern memory 201, a microprogram sequencer 202, and a microcode memory 203.
03, and an address creation unit 204. The external bus interface circuit 200 is an interface for connecting to an MPU or the like, and is connected to address lines 205, data lines 206, and control signal lines 207. An upper address comparison circuit 208 and a D-FF 209, which is a register, of the external bus interface circuit 200 decode an upper address given from an MPU or the like, and when the upper address is a predetermined address, set the lower address and data in the D-FFs 209 and 211, respectively, using a latch signal from a timing circuit 214 as a trigger.
The address and data are transmitted to bus drivers 210 and 212, respectively.
The address is used when referring to the microcode memory 203 and writing the microcode from the MPU side via the data bus. The lower address is also passed to the microprogram sequencer 202 via the bus driver 210 as a microcode address, and is then input to the internal memory via the internal address bus and internal data bus.
The control pattern memory 201 can be referenced from the PU side using a specific address.

Data from the MPU or main memory is latched into the D-FF 211 via the data line 206, and then sent to the separate I/O RAM 213 in the microcode memory or the separate I/O RAM 215, 2 in the control pattern memory 201 via the bus driver 212.
16. When a data strobe signal from the MPU or memory is applied to the timing circuit 214 via the control signal line 207, an acknowledge signal is returned.
A timing circuit 214 controls the latch timing of the D-FFs 211 and 209 and the write timing of the microcode memory 203 and the control pattern memory 201 via the WR signal.

The "1" and "0" patterns of the complex control signals given to the neurochip as shown in the timing chart of FIG. 17 are stored for one cycle in the control pattern memory 201, and the patterns for one cycle are input to the microprogram sequencer 20.
2. The control pattern is generated by reading out the reset signal Reset, the data clock DCLK, the weight clock WCLK, CSO1, CSO2, CSO3, and
Control signals such as SYNC1, SYNC2, SH1, SH2, OC1, and OC2 are read from the separate I/ORAM 215, and control information associated with the patterns, i.e., sequence control flags, are read from the second separate I/ORAM 216.
For example, if the control pattern memory 20 stores a pattern of 1000110001, this is a "1,0" bit pattern.
Control pattern memory to repeat bit patterns
By controlling the address of 201, the repetition of this pattern is read out from the control pattern memory 201. In other words, since the control signal patterns are very complicated, these patterns are stored in advance in this separate memory.
The structure is such that the bit patterns are stored in the I/ORAM 215, and the addresses of the separate I/ORAM 215 are specified under the control of the microprogram sequencer 202, thereby outputting the bit patterns in sequence. Therefore, several identical patterns are repeated, and how this repetition is realized is determined by address control. This pattern for one cycle is called the original pattern. In order to repeat the original pattern, it is necessary to feed back specific information from the control pattern memory 201 to the microprogram sequencer 202. That is, the sequencer control flag in the second separate I/ORAM 216 is used as a condition input to the microprogram sequencer 202.
By inputting into the microprogram sequencer
The microprogram sequencer 202 controls the first separate I/ORAM 215 to return to the first address where the original pattern is stored. This causes the original pattern to be repeated. That is, the microprogram sequencer 202 sequentially generates address signals to the separate I/ORAM 215 via the general-purpose port output line 202-1 until the condition is met. Normally, this address is incremented, but when the condition that the address has reached the end of the original pattern is met, the address is returned to the first address where the original pattern is stored. As a result, the specific pattern is repeated and a control pattern is output from the separate I/ORAM 215.

19B shows the relationship between the information stored in the memories 201 and 203 that control the master control block 181. In the figure, the control pattern memory 1 is a first separate I/ORAM.
The control pattern memory 2 corresponds to the second separate I/O RAM 215, and the control pattern memory 2 corresponds to the second separate I/O RAM 216. The microcode memory 203 stores the control code of the sequencer 202, and mainly stores the Jump
The p instruction and the Repeat instruction are stored in the control pattern memory 2. Looking in the direction of address increment, there is a Repeat instruction at a specific address, and the number of repetitions of pattern 1 in the control pattern memory according to this repeat instruction is stored in the corresponding address in the control pattern memory 2. For example, if it is "10", 10 repetitions will be executed. In this way, when the address increments and reaches the Jump instruction in the microcode memory, the second Jump in the microcode memory 203 jumps to 500H, and P
When Pattern2 is repeated five times, the third Jump in the microcode memory 203 again outputs "100H".
and outputs Pattern 1. In this way, the original pattern is repeated and read out from the control pattern memory 1.

WCLK is generated in synchronization with the read clock of the address referencing this control pattern memory 201, and information is read from the weight memories 185 and 186 in synchronization with WCLK. Addresses for the weight memories 185 and 186 are accessed by address signals output from address 1 and address 2 of the address generation unit 204. Address 1 and address 2 are separated to correspond to the hidden layer and the output layer, respectively. The weight data to be provided to the ANP in the hidden layer is read from weight memory 185 specified by address 1, and the weight data for the ANP in the output layer is the content read from weight memory 186 specified by address 2. Since the contents of weight memories 185 and 186 are stored in each address, one bit at a time in the direction in which the address increases, a count control signal to address counters 217 and 218 must be provided from the microprogram sequencer 202. These addresses are incremented by the address counters 217 and 218 and provided as address signals to the weight memories 185 and 186 one after another via bus drivers 219 and 220. Then, a plurality of weight data are read out from the weight memories 185 and 186 .

The WCLK from the first separate I/ORAM 215 and the counter control signal from the microprogram sequence 202 are applied to AND circuits 221 and 222 in the address creation unit 204 .
When the counter control signal is high, the address counter is updated by WCLK, and bits 1 to 16 of WCLK increment the address counters 217 and 218. For the remaining bits 17 to 26 of WCLK, the counter control signal is set low to inhibit WCLK, stopping the increment of the address counters 217 and 218. Then, in synchronization with SYNC1 and SYNC2, counter reset signals are sent from the microprogram sequence 202 to AND circuits 221 and 222, respectively, to reset the address counters 217 and 218. This returns the addresses of the weight memories 185 and 186 to the starting addresses. The mode signal output from the master control block 181 is used to select between normal use of the weight memories, i.e., a mode in which the weight memories are disconnected from the MPU data bus and weight data is provided to the ANP, and a mode in which the weight memories are connected to the MPU data bus and the weight memories are referenced by the MPU.

The mode signal is generated by resetting the flip-flop 224 using the lower bits of the data from the MPU as a trigger, which is an AND signal generated by an AND circuit 223 from one bit of the lower address and the write signal from the timing circuit 214 to WR. When this mode signal is 0, the weight memory is in normal use.

The write signal WT and one bit of the internal address bus are input to the clock terminal of a flip-flop 224 via an AND circuit 223, and the LSB of the internal data bus is input to the data terminal of the flip-flop 224.
The master control block 181 is checked to see if it is selected, and if it is, the lower address and data are taken into DFFs 209 and 211. This kind of interface operation is also performed for other devices connected to the MPU, but since the weight memory normally supplies weight data to the ANP, if it is connected directly to the MPU data bus, bus contention will occur. To prevent this,
When the LSB is captured by the flip-flop 224, the mode is set to 1, the weight memory is not chip selected as described below, and data is not generated on the data bus from the weight memory. At a predetermined timing, the internal address bus specifies an address of either the microcode memory 203 or the control pattern memory 201, and the desired data is written from the internal data bus to the accessed address. As a result,
The programs stored in the microprogram sequencer 202, the microcode memory 203, and the separate I/O RAM 216 are changed, or the control pattern stored in the separate I/O RAM 215 is changed.

Figure 20A shows the data storage configuration of the weight data memory 230. In this diagram, eight bits in the column direction represent 8-bit data information stored at the same address, and each bit is provided to ANP1, ANP2, ..., ANP8 from the bottom up. The addresses in the row direction are different, and as shown in the diagram, the addresses increase as you move to the left. Weight data consists of 16 bits, including the sign bit, and is stored from lowest to highest addresses. The MSB is the sign bit, and the other 15 bits are numeric bits. When the address from the microprogram sequencer 202 is incremented in synchronization with WCLK, one word of weight data, i.e., 16 bits, is read out sequentially from the MSB to the LSB. These weight data are simultaneously passed to eight ANPs. Because data is stored in this manner in the order of increasing addresses, an address counter for this weight data is required. That is, a counting control is required so that when one word of weight data from the MSB to the LSB is counted, it becomes one weight data. This control is also performed by the microprogram sequencer 202.

Figure 20B shows a concrete circuit of the weight memory blocks 185 and 186. The memory 230 is a RAM called MB8464A-70. The output is A
It is 8 bits corresponding to NP1 to ANP8. Basically, it is the bus signal line seen from the MPU bus and the master control block 18
1 uses either address 1 or 2 visible from 1. Addresses 1 and 2 are addresses 1 and 2 in Figure 19A mentioned above.
Addresses 1 and 2 are input in an incremented manner in synchronization with WCLK. Eight bits of data are read out simultaneously, and each bit is simultaneously applied to ANP1 to ANP8.

When the mode signal is 0, the weight memory 230 is chip selected via the AND gate 233, and at this time, addresses 1 and 2 from the microprogram sequencer 202 become valid in the multiplexer 234. Then, weight data is sent from the weight memory 230 to ANP1 to ANP8. Meanwhile, since the output of the inverter circuit 231 is high, the tri-state bus transceiver 232 is disabled and the output of the weight memory 230 is not output to the MPU.

When outputting to the MPU, the mode signal is set to 1, and the address decoder 23 receives appropriate address information from the MPU.
5 to chip select the memory 230, and
Address is given from PU. When mode signal is 1, MPU
The read/write direction is controlled by the read signal Read on the data line coming from the MPU via an AND gate 236.
Determined by Signal.

Next, the learning algorithm will be explained.

Fig. 20C is a flowchart of the learning algorithm called backpropagation used in the present invention. Learning proceeds as follows: Complete information to be learned is input from the MPU via an input control circuit (not shown) to the input of the neural network of the present invention, i.e., a hierarchical network composed of a set of ANPs. Then, the input signal passes through the input side circuit, intermediate layer, and output layer, and is sent to the network output via an A/D converter and then to the MPU. The learning algorithm exists in the main memory of the MPU.
The PU takes in the teacher signal from the main memory and checks the error between the network output and the teacher signal. If the error is large, the MPU changes the weight data, which represents the strength of the network connections, so that the network will produce the correct output. This weight data is added to the ANP of each layer via the weight memory 230.

When the weight data is updated by the learning algorithm,
The backpropagation learning algorithm shown in Figure 20C is followed. When the learning algorithm starts,
The MPU is the current output Y _L of the Lth neuron ANP _L in the output layer.
The error between Y and the teacher signal _L is found and substituted into Z _L. The output Y _L is the output of the neuron ANP _L , so if a sigmoid function is used as the nonlinear element, for example, it is the output value of this nonlinear function. Therefore,
In the neuron ANP _L , the error Z _L must be propagated to the input side of the nonlinear function. When the error propagation is performed, the nonlinear function of the energy function, that is, the energy obtained by multiplying the square of the error signal by 1/2, that is, _EL = 1/2( _L - _YL ) ² , is used as the deviation with respect to the input _XL , that is, can be transformed as follows:

Here, if the nonlinear function f(X _L ) is a sigmoid function, It is expressed as:

Transforming the differential f'(X _L ) of this sigmoid function gives f'(X _L ) = Y _L (1 - Y _L ). This is V _L shown in S2 of the flowchart. Therefore, δ, that is, the nonlinear function input of energy X _L
The deviation with respect to is V _L × Z _L , that is, U _L shown in S2
The error δ for this energy nonlinear function input must be further back-propagated to the intermediate layer.

The K-th neuron in the hidden layer is A _k . The output of A _k is
Let Y _k be the nonlinear function input XL of the neuron ANP _L in the output layer.
is the output of all hidden layer neurons {Y ₁ ...Y
_kMAX } is expressed as a sum of products multiplied by the weight W _LK . Therefore, the deviation of X _L with respect to the weight W _LK is On the other hand, the variation of the weight W _LK with respect to the energy E _L is given by the following equation:

That is, the T _LK of S3 is This represents the deviation of the energy weight. Therefore, this T _LK can be used as the weight change ΔW, but in order to speed up the convergence, the second
Add a term to make the following recurrence formula and modify the weights:

ΔW _LK = αT _LK + β ΔW _LK W _LK = W _LK + ΔW _LK Here, n and α are constants. Now, we are focusing on a specific neuron ANP _L in the output layer, but if we assume that this ANP _L is connected to all neurons in the hidden layer, then each ANP _L
It is necessary to repeat K from 1 to K _max for this. This is the repetition shown in R1 of the flowchart, and it is repeated the number of times equal to K _max , the number of neurons in the hidden layer. When this repetition is completed, the backpropagation for a specific neuron ANP _L in the output layer is completed. Therefore, this needs to be done for all neurons in the output layer {ANP ₁ , ANP ₂ , ..., ANP _{L max} }, so as shown in flowchart R2, L
is repeated from 1 to L _max . In other words, it is repeated as many times as the number of neurons in the final output layer, L _max .

Next, learning proceeds from the intermediate layer to the input layer. The algorithm is almost the same, but the error signal cannot be expressed as the difference between the teacher signal and the output voltage, resulting in equation S5.
That is, Z _k is the term corresponding to the output error signal of A _k , the Kth neuron in the hidden layer. This is clear from the following equation.

Therefore, the error signal _Zk of the intermediate layer is calculated by repeating (R3) the index L of _Zk in S5 from 1 to _Lmax , that is, the number of outputs. After that, it is the same as the algorithm between the intermediate layer and the output layer. That is, first, the differential value _Vk of the sigmoid function is calculated, and this is used to find _Uk , that is, the change in energy relative to the nonlinear function input, in S6. In S7, this _Uk is used to find the product _Tkj with the output of the input layer, _Yj . Using this as the main part of the weight change, a second term is added to speed up convergence as shown in S8 to find _ΔWkj , and this _ΔWkj is added to the previous value _Wkj to get the new _Wkj . This is the weight update. This weight update is repeated the number of inputs _Jmax (R4). That is, J =
By repeating the process from 1 to j _max , the weights between the input layer and the hidden layer are updated. Note that Z _k in S5 corresponds to the error signal of the hidden layer output, and is expressed as the backward propagation of the variation _UL with respect to the function input value of the output layer's energy, while W _LK can only be determined once the weights between the hidden layer and the output layer have been determined. In other words, the calculation for updating the weights starts from neuron ANP _L in the output layer and moves to neuron ANP _K in the hidden layer, and the weight change ΔW in neuron ANP _K in the hidden layer cannot be calculated until the ΔW in the previous step has been determined. Therefore, this learning process is called backpropagation because it is only possible to calculate it by going back to the final input layer.

Backpropagation learning involves inputting the training data as complete information, performing a forward operation to output the results, and then changing the strengths of all connections backward to minimize the resulting error. Therefore, this forward operation is necessary. The analog neural network portion of the present invention is effectively utilized in this forward operation. The algorithm for backpropagating the output values is executed by the MPU. Note that if the nonlinearity is not a sigmoid function, the nonlinear differential value will be different. For example, if the learning algorithm is tanh(x), the nonlinear differential result will be _VL = 1 - | _YL | in the output layer (S2') and _Vk = 1 - | _Yk | in the hidden layer (S6'), as shown in Figure 20D.

Other components are given the same reference numerals as in FIG. 20C and will not be described further.

Figure 21 is a diagram of the input-side daisy chain circuits 173 and 174. In the figure, 240, 241, and 242 are D-type flip-flops. At the rising edge of the DCLK signal, data input to the D terminal is set, and output Q is set to 1. The first flip-flop 240 sets the CSI signal at the falling edge of DCLK. Then, at the next rising edge, the output signal is set to the second flip-flop 241.

The output is input to the D terminal of the third flip-flop 242. The clock signal that sets the input is the output of a 4-bit counter 243. The counter 243 is WCLK
It is triggered by the falling edge of DCLK. It is cleared by the falling edge of DCLK. Therefore, the counter 243 becomes all 0 at the falling edge of DCLK, and after eight falling edges of WCLK are input, the upper bit QD signal becomes high, which triggers the flip-flop 242 to output a high signal to CSO. When the output of the flip-flop 241 becomes 0, C
By this operation, CSI falls, and after a predetermined time equivalent to eight WCLK pulses has passed, CSO is output, resulting in a daisy-chain operation.

22 is a specific circuit diagram of the max value node circuit 187 which forms the neuron of the dummy node. In the figure, the resistor 250 is connected to Zener diodes 251 and 252, resistor 2
53, voltage followers 254, 255 are circuits that form a constant voltage. When a current flows from +12 volts to -12 volts through resistors 250, 253 and Zener diodes 251, 252, the inputs of voltage followers 254, 255 become +7 volts and -7 volts, respectively.
These voltages are fed to the voltage follower 254,
The two constant voltages are output through an output resistor 256 of 255. The analog switches 257-258 are connected to each other so as to extract these two constant voltages in a time-division manner.
64. When the T mode signal is 0, the constant voltage is given to the next voltage follower 265 via the analog switch 257. When T mode is 1, that is, in the test mode, the output is held to the analog ground by the analog switch 258, so that 0 volts is input to the voltage follower 265. In the test mode, the offset on the bus is notified to the MPU. The voltage follower 265 is enabled by the switch control of the output section. When the output enable is 1, the analog switch 26
When 0 is on, it acts as a voltage follower and its output is given, but at this time, no output is given to the dummy node output.
Conversely, when Output Enable is 0, a signal is output to the dummy node output. The analog switch 260 and its output are controlled by Output Enable 1 or 2, which is 0 enabled. That is, when Output Enable 1 or 2 is 0, a constant voltage is output to the dummy node output. The upper dummy node output is for the dummy node in the input layer, and the second is for the dummy node in the middle layer. The output voltage of this dummy node is fixed to an appropriate value, so it can be used as a threshold voltage. Zener diodes 251 and 252 output a constant voltage in a reverse bias state, and this fixed voltage can be varied within a range from +7 volts to -7 volts. The enable states of Output Enables 1 and 2 are determined by the output voltages from other ANPs connected to the analog bus and the dummy node control signal DCS from the master control block 181 to avoid conflicts on the analog bus.

FIG. 23 shows a nonlinear function generator circuit, and FIGS. 24 and 25 show
Figure 26 shows the hardware of the digital logic inside the ANP.

Figure 23 shows a transistor circuit network that realizes a sigmoid function. The term "sigmoid function" used here refers to a continuous, monotonically non-decreasing function, and does not specifically exclude linear functions. In the figure, transistors 343, 356, 378, 390, 298, and 314 and their paired transistors form a differential amplifier, and the transistors connected to the collector side form current mirror circuits. The collector current flowing through the collector of the transistor on the left side of the differential ANP is the output current. The current direction is changed by the current mirror and output. The current enters resistor 336, which is connected to output V0.
Resistor 336 converts the voltage into a current. Because it has no drive capacity, the output is received by a high-impedance operational amplifier buffer. The circuit on the input side of transistors 337 and 339 is a bias circuit. The piecewise linear method is used to realize a sigmoid function. The slope of each section of the sigmoid function is determined by the ratio of emitter resistor 344 connected to the emitter and output resistor 336. The emitter resistors of transistors 343 and others are also included at this time. The gain of each differential ANP is different. The breakpoint for each piecewise linear transition utilizes the saturation characteristics. The saturation characteristics are all different. The saturation characteristics of each ANP are changed so that the total value of the current output from each operational amplifier at the output point V0 becomes a sigmoid function. Transistor 345 and resistor
R1 is a current source. Transistor 346 and resistor R2, transistor 353 and resistor R3, etc. are all current sources that supply the same current. In other words, the resistors are determined so that the current values are the same. They are all the same current source. The collectors of transistors 345 and 346 are connected, so the sum of the currents flows to the intersection of resistors 344 and 347. Transistors 343 and 348
The collector currents of the two transistors will be the same when they are balanced.
The transistor 351 is used to improve the characteristics of the current mirror. The transistor 350 is diode-connected. Changing the direction of the current means that the output
It can either draw in current or expel current.
As shown in the figure, transistor 351 of the current mirror
Current flows from the collector of the transistor to the output. There are many transistor arrangements below, but the transistors whose emitters and collectors are connected to the same point are the same transistor. For example, transistors 358 and 360 are the same transistor, which is the same as transistor 345.
59 and 361 are also the same transistor, which corresponds to 346.
The transistors 8 and 369 are the same, and correspond to 353. The same applies below. Therefore, this is a circuit with a total of six operational amplifiers with constant current power supplies driven by the same current, which operate in different current directions depending on whether the output voltage is positive or negative. Transistors 337 and 338 are level shifters, and 330 and 327 are also level shifters.
The level shift circuit is used to make the operating range of the sigmoid function approximately the same for both positive and negative sides. Transistor 352 is used to correct the collector current of transistor 351 so that it is equal to the collector current of transistor 353. The same applies to transistors 367, 385, 287, and 307.

Figure 24 shows a specific circuit of the sequence generator 28 (Figure 5) for generating pulse signals to be supplied to the neurochip. 401 and 402, and 404 and 405 are inverters.
Each inverter is a clock inverter. Clocks are created separately for the rising and falling edges of the latch signal of the flip-flop FF. The flip-flop in the figure latches with a rising clock, and the inverters are used as FFs to form the rising latch FF. For example, in the case of DCLK, the signal that passes through one inverter 401 becomes the clock signal for the rising latch. Then, the signal that passes through inverter 402 becomes the clock signal for the rising latch DCL
Similarly, the output of inverter 404 is
The output of inverter 405 is the rising clock WCL
At F.F410, the falling edge of DCLK latches the SYNC signal. F.F410 and 415 delay the SYNC signal by one cycle of DCLK to create the SNC2 signal, and synchronize the SYNC and its 1 cycle.
A 1τ pulse is created with the clock delayed signal.
At the 1τ (1 cycle of DCLK) pulse after NC rises
Discharges the integrating capacitor in the ANP, i.e. CRST
The signal DSH2 is the reset signal for the capacitor. The other signal DSH2 is a pulse that lasts for 1τ of DCLK from the falling edge of SYNC. This is the sample/
This is the sample/hold signal for the hold capacitor. In 411 FF, the clock is WCLK and the data is DCLK.
Therefore, the DCLK signal is latched by WCLK. After that, the SYNC signal is high at the NAND gate 414, and the D
When CLK is high, the first WCLK is triggered and FF is generated.
The clock is 443. The NAND gate 414 and inverter 440 make an AND. In the F.F. 443, when the SYNC signal is high, the first signal WCLK that arrives captures the digital weight data, i.e., the sign bit of WD. This signal is the MSB, i.e., the sign bit, of the weight digital data that comes in serially. In other words, FF
The sign bit is latched by the F.F. 443 at the timing of 411 and the AND gate (414, 440). 4-bit binary counter 416
counts the number of WCLK pulses. Since 16-bit digital weight data is input, it counts 16. When the count is complete, the output goes high and is input to inverter 423. This signal indicates that 16 counts have been completed. This signal is used to control the input of weight data that is input serially to ANP into shift register 27 (Fig. 5). The least significant bit of counter 416 is also input to inverter 422.
The output of this inverter 422 generates the CSO signal, which is the control signal for the daisy chain.
In order to prevent conflict between the signals sent from the two ANPs in the previous stage on the analog bus B1, it is necessary to form a delay circuit to execute a daisy-chain operation so that the next CS is sent after the previous CS has dropped. The delay time of this delay is determined by counting WCLK and using the counter value.
Counter 416 finishes counting, and flip-flop 433
The signal indicating that the operation is finished is latched via inverter 423, which is clocked by WCLK, i.e., the 17th WCLK. Inverters 437 and 438
The latched signal is returned to the counter 416 through the inverter 416, and the counter 416 is disabled to prevent further incrementing.
When the output of 438 goes low, the counter 416 stops counting. The output of F.F.433 is input to the flip-flop 442. This becomes the gate signal for the output of the shift register 408. In other words, the 16 digital weight data coming in are shifted in order by the shift register 408, and when the 15 bits of data (numerical bits excluding the sign bit) are arranged in parallel, they are output. The gate signal WR prevents output while the data is being shifted, and outputs the data when all the data has been input. The shift register 408
The contents of are given to the ANP multiplier. The signal output from F.F433 is branched and used as the enable signal for the shift register. F.F442 latches on the rising edge of the output of F.F433. The shift is completed by 16 falling edge latches of WCLK, and then the gate is opened on the falling edge, although a rising edge latch could also be used. F.F412 creates a pulse signal for selecting the sigmoid function.
When a reset signal is received using F.F412, the WD, or weighted digital input signal, is set to 0 or 1 to select whether or not to use the sigmoid. This method may not be used in this system. In reality, the sigmoid selection signal is generated directly from outside. The circuit below is a daisy chain circuit. The output of counter 416 is delayed by F.F434, and this delay triggers the last F.F445. This shifts the DCLK by 1τ, but rather than simply shifting it, the beginning is dropped. In other words, the CSI signal itself may not be one DCLK cycle long, so to make the CSI into a CSO, the beginning of the waveform is delayed by, for example, 2 microseconds, and the rest of the signal is generated as is. Gate 425
and 427 are CSI buffer gates, a positive buffer and an inverter buffer.

Fig. 25 shows the phase control circuit 29 (Fig. 5) which generates the sample/hold S/H signal and the OC signal. The S/H signal is divided into two parts, one that goes to inverter 515 and one that goes to gate 524. The same is true for the OC signal. The S/H signal is divided into gate 524 and inverter 515, and after entering gate 525 via inverter 515 there are eight inverters. Two signals are generated for the S/H signal: one that is in phase with the signal and one that is in opposite phase. This is done by cascading several inverters and cross-linking them to prevent the two outputs from becoming 1 at the same time. In other words, the two outputs of the sample/hold S/H signal are
The inverter chain is a delay circuit that prevents both S/H signals from turning on at the same time. The delay time is determined by the length of the inverter chain, and one is delayed by several stages after it turns on, before the other turns on. The same is true for S/HD0 and S/HD1. The circuit for the OC signal is basically the same, but because the CRST signal is input to gates 528 and 529, when CRST is 1, both outputs are forcibly set to 1.
Both OC0 and OC1 are prevented from becoming 1 at the same time, but in the case of OC, they are set to become 1 only when CRST is 1. This realizes the reset function of discharging the charge of the integrator capacitor through the control of the analog switch.

Figure 26 shows the 15-bit shift register 27 (Figure 5). Gates 602, 603, 6014, and F.F. 627 correspond to one bit, and we will use this to explain. The output from the previous time is input to gate 603, which is the output of F.F. 628. Since it is input from the previous bit, it becomes the data signal for shifting. The other signal input to gate 603 is SHFT, which is an inverter of the shift signal.
This is a shift control signal, and when it is valid, it commands a shift. Gate 602 also receives the output of F.F. 627 itself. This means that it is feeding back its own output. The other input of gate 602 also receives an inverted SHFT signal, but its phase is different from that of gate 603.
This means that when shifting is disabled, the current output is maintained as is. Since the clock signal is always input regardless of shifting, no shifting occurs unless the clock is input and shifting is enabled. The previous bit is shifted only when the shift signal SHFT is enabled, and the shift operation is achieved by inputting it through gate 603. The WR signal is input into the AND of gates 632, 633, etc. This is a selection signal that determines whether or not to output each bit, and also serves as a control signal that determines whether or not to pass the data stored in the shift register to the multiplier. In addition, to achieve fanout, for example, inverter 620 is responsible for the reset signals of five of the 15 FFs, and gate 626 is responsible for the reset signals of 10 FFs. The fanout shift register 608 has the functions of shift enable SHFT and output enable WR.

Next, a case where the neurocomputer according to the present invention is configured as a feedback network will be described.

FIG. 27A is a conceptual diagram of a feedback network.

In the case of a feedback network, there is basically an input, but the network also has a feedback path in which the signal it outputs also returns. This feedback method is sometimes used as a type that uses one layer of a hierarchical neural network in time division multiplexing, and sometimes as a so-called Hopfield type neural network.

In the former case, the input and output signals of the ANP are time-division, so
At the output point of each ANP, the output data of the same ANP is output sequentially for each fixed sequence cycle, and each sequence cycle functions as the input layer, hidden layer, and output layer of the hierarchical neural network. In the latter case, the output voltage is fed back until the ANP's output reaches a specific value, that is, until it stabilizes. When the fed-back result is output, the state is switched back and forth until the result matches the previous data, that is, the data it output before, and convergence is achieved when a stable solution is reached.

According to an embodiment of the present invention, as shown in FIG. 27B:
The feedback path is realized by the common analog bus CB, and there is a U-shaped feedback section. Then, the result of each calculation is output, and the output from each ANP is fed back through the feedback path. This feedback path operation is repeated.

Fig. 28 shows the configuration of a first embodiment of a feedback network. In the figure, 121 denotes a plurality of basic units 1 constituting a processing layer. These basic units 12
Similarly, the basic unit 121 has a multiplication unit 122, an accumulation unit 123, a threshold unit 124, and may also have an output holding unit 125. Similarly, the input and output of this basic unit 121 are configured to be realized by analog signals.
Electrical connections are made between the output section of the basic unit 121 and the input section of all basic units 121 via analog buses 141, 142, and 143, and a hierarchical network or a Hopfield network is equivalently constructed by weights set corresponding to each of these connections.

Reference numeral 142 denotes an analog bus, which is a common line provided for electrical connection between the output section and the input section that constitute the feedback line.
51, weight setting means 152, threshold processing starting means 153 and output value sending means 154, and controls data transfer.

The drive unit selection means 151 processes to sequentially select the basic units 121 in time series. Then, in synchronization with this selection process, the output value transmission means 154 transmits the final output of the analog signal held in the output holding unit of the selected basic unit 121 to the multiplication processing units 122 of all the basic units 121 via the analog bus 142 in accordance with a time-division transmission format.
Upon receiving this input, the multiplier 122 of each basic unit 121 processes the weight setting means 15
2, the weights corresponding to the respective connections are sequentially selected, and the input is multiplied by the weights.
3 sequentially accumulates the multiplication results obtained by the multiplication unit 122. When it is confirmed that all accumulation processes for the basic units 121 of the processing layer have been completed, the threshold processing activation means 153 activates the threshold processing unit 124 of the basic unit 121 and controls it to apply a constant threshold voltage and perform calculation of the sigmoid function.

This process is then repeated cyclically a predetermined number of times. By this data transfer method, for example, an output pattern corresponding to an input pattern is obtained as the output of a hierarchical network by time-division multiplexing the layers.

That is, this embodiment discloses a network configuration data processing device of the present invention configured as a single layer. In this embodiment, in order to make the multi-layered network configuration data processing device into a single layer structure, a common line analog bus 142 is provided, and this analog bus 142 is configured as follows:
The output of each basic unit 121 in a single layer is fed back to the input, so that the output of each basic unit 121 is connected to the input of all the basic units 121 .

Next, the operation process of the embodiment of FIG. 28 configured as above will be described with reference to the timing chart shown in FIG. 29A.

When a request for conversion to an output pattern is received, the main control circuit 150
The input side circuit 120 is selected in order in time series by sending CSI control signals to the input side circuit 120. This selection process is illustrated in FIG. 29A.
The input circuits 120 process the analog signals Yi given as an input pattern so as to transmit them in time series onto the analog bus 143. Therefore, as shown in Fig. 29A, the analog signals _Yi are transmitted in order onto the analog bus 143, the number of which is equal to the number of input circuits 120.

As shown in the timing chart of Fig. 29B, when the multiplier 122 of each basic unit 121 receives the analog signal _Yi , it executes multiplication using the weight _Wi of the weight holding unit 8 that is set by the main control circuit 150. Then, the accumulator 122 of each basic unit 121
23 sequentially accumulates the multiplication results that are obtained for each selection of the input side circuit 120, and when all the accumulation processes are completed by completing the selection of all the input side circuits 120,
The threshold section 124 of each basic unit 121 performs processing to obtain the final output value Y, which is the final calculation output of the basic unit 121.

The final output value Y thus obtained is held internally. Subsequently, when the output switch section 36 of each basic unit 121 receives an output control signal as CSI from the multiplexing control circuit 150, it is turned on in order and time series, and the analog signal of this held final output value is fed back to the multiplier section 122 of the basic unit 121 via the analog bus 142. By this process, the final output value of the basic unit 121 of the input layer is equivalently transmitted to the basic unit 121 of the hidden layer in a time-division transmission format. Thereafter, by repeating the same process for the basic unit 121, the final output value Y corresponding to the output layer is obtained. Figure 29A shows the signal to the input side circuit 120.
Corresponding to the _Yi output control signal, a timing diagram of the output control signal to the basic unit 121 is shown, as well as a timing diagram of the analog signal of the final output value Y on the analog bus 142.

Although the embodiment of FIG. 28 has the drawback that it is not possible to input continuous input patterns to the hierarchical network, it has the advantage that the hierarchical network can be simplified and therefore can be made extremely small when fabricated into a chip.

Figure 30 shows an embodiment in which the neurocomputer of the present invention is realized by a feedback network operating as a hierarchical network. ANPs 1, 2, and 3 perform product-sum operations on time-division analog input signals from analog input ports 1 and 2. ANPs 1, 2, and 3 then operate as an intermediate layer, outputting the time-division analog signals to analog bus B2. This output signal is fed back to analog bus B1 via analog common bus CB, which serves as a feedback path. ANPs 1, 2, and 3 then perform product-sum operations on this feedback signal again, operating ANPs 1, 2, and 3 as an output layer, thereby realizing a hierarchical network using one layer of ANPs 1, 2, and 3. Max Value Node Circuit 187
receives the DCS output of the master control block and generates a dummy signal on the analog bus B2. DCLK and WCLK are input from the master control block to a daisy-chain circuit 171, which determines the timing of the rise and fall of the CSI signal.

FIG. 31A is a timing chart of the feedback hierarchical network shown in FIG.

WCLK is generated only while DCLK is rising, and after DCLK rises, the analog signal becomes stable, and the weight data is input serially, but before it is aligned in parallel, the CSO from the master control block 181 is generated.
1 is input to the daisy chain circuit 171 and rises as shown. At this time, the analog signal held in the sample/hold S/H from the analog input port 1 appears on the analog bus B1 via the analog switch 175, and a multiplication and accumulation operation is performed by ANP1, 2, and 3. At the next input of DCLK, the daisy chain circuit 172
When CSI to the analog input port rises as shown, the signal from the sample/hold circuit S/H holding the input signal from the analog input port appears on the analog bus B1 via the analog switch, and the second multiplication and accumulation operation is performed by ANP1, 2, and 3. After DCLK is input at the next timing, the master control block outputs a dummy signal DC
S occurs, and in ANP1, 2, and 3, the
The sum of the previous calculation is executed. While the next SYNC signal is rising, the sum of products calculations of the output layers of ANP1, 2, and 3 are performed. Only while the address count inhibit signal of the weight memory address 1 signal is rising, WCLK that counts the address counter is enabled, and at other times the count is inhibited. Next, when CSO2 is given to ANP1 from the master control block, ANP1 outputs the result of the previous sum of products to analog bus B2, and the analog common bus
The signal is fed back to analog bus B1 through CB, and as shown, multiply-and-accumulate is performed again in ANP1, 2, and 3. After a predetermined delay is added in the daisy-chain circuit inside ANP1, CSO2 adds the input signal CSI to ANP2 as shown, and at this time, the output signal of ANP is again added to analog bus B2 via common bus CB and analog buses A1 and B1 to ANP1, where the multiply-and-accumulate is performed. Similarly, CSO from ANP2 becomes the CSI signal of ANP3 after a predetermined delay, and when this CSI signal rises as shown, the output signal of ANP3 is again sent to ANP via analog bus B2, common bus CB, and analog bus B1.
Similarly, when the signal DCS from the dummy node rises, ANP1, 2, and 3 perform summation on the fixed voltage. Then, when the signal CSO2 rises again, ANP
The output from 1, 2 through S/N is as shown in
There is no output from analog input port 2.

Here, ANP1, 2, and 3 act as intermediate layers,
, , ANP1, 2, 3 operate as the output layer. Therefore, according to this embodiment, a hierarchical network can be configured with only one layer of ANP1, 2, 3.

Fig. 32 shows an embodiment of an analog neurocomputer according to the present invention, which is configured with a Hopfield type feedback network, and Fig. 33 shows its timing chart. The outputs of the memory address terminal and mode terminal of the master control block 181 are input to the weight memory block 18.
The data output B10 of this weight memory block 185 is connected to ANP1, B11 to ANP2, and B12 to ANP3. The output signal from the CSO1 terminal of the master control block 181 is applied to the daisy chain circuit 171 and the switch 175, and at the rising edge of this signal, the output of the sample/hold circuit 173 from the analog input port 1 is connected to the analog bus B
After being delayed for a predetermined time by the daisy-chain circuit 171, the output of CSO is generated, which is then sent to the daisy-chain circuit 172.
and the signal of the sample/hold circuit 174 connected to the analog input port 2 is placed on the analog bus B1 via the switch 176. Similarly, the output signal CSO of the daisy chain circuit 172' opens the output switch 176' of the sample/hold circuit 174' connected to the analog input port 3, and the signal is placed on the analog bus B1. In ANP1, as shown in Figure 33, one product-sum operation is performed in one period of the DCLK signal, and when the DCLK signal is high, the weight clock is driven and the digital weight data input in synchronization with the weight clock and
It is multiplied by the analog input signal, and when the latter half of DCLK is low, the sample/hold signal SH goes high.
The sum operation is performed on the capacitor of the integrator. That is, while CSO1, i.e., CSI of daisy chain circuit 1, is high, ANP1, 2, and 3 perform a multiply-and-accumulate operation on the analog signal on bus B1. Also, master control block 18
When the OC signal from the analog input port 1 goes high, ANP1, 2, and 3 perform offset cancellation and sample/hold to complete one product-sum operation cycle. Next, the input signal CSI to the second daisy-chain circuit 172 goes high, so ANP1, 2, and 3 perform product-sum operation on the input signal from the next analog input port. After the product-sum operation cycle is completed, the daisy-chain circuit 17
The CSI signal is input at 2', an output signal is generated from the sample/hold circuit 174', and the third multiply-accumulate operation cycle begins as shown by .

Next, the master control block 181 generates a CSO2 signal, and the signal generated during the previous multiplication and accumulation cycle is fed back from ANP1 via the analog bus CB, and ANP1, ANP2, and ANP3 simultaneously perform multiplication and accumulation operations on the fed-back signal. Next, after a predetermined time delay, the CSO output signal from ANP1 is applied to ANP2, and ANP2 then outputs the signal stored during the previous multiplication and accumulation cycle in a daisy chain fashion. This signal is fed back via the analog bus CB and ANP
ANP1, ANP2, and ANP3 drive the product-sum operation. After a similar delay of a predetermined time, the CSO of ANP2 is added to ANP3, and the output from ANP3 is fed back via the analog bus CB, and product-sum operations are performed in ANP1, ANP2, and ANP3.
In the feedback network, as shown in Figs. 33A and 33B, the outputs of the three ANPs go through six product-sum operation cycles and are sent to the sample/hold circuits 17, respectively.
Analog output ports 0, 1, and 2 via 7, 178, and 178'
The signal is output to the sample/hold circuits 177 and 17
The output signals of 178' are selected by an analog multiplexer 179 and sent to an MPU 182 via an A/D converter 180.
The output is given to a digital control circuit including a memory 182 and a communication interface 184. The MPU 182 checks whether the neuron output state at the current time is the same as the neuron output state at the previous time. If they are the same, it is determined that convergence has occurred. In this way, the process is carried out via a single common analog bus CB. If a stable solution is reached by repeating the feedback operation, this is the final output.

The embodiment shown in FIG. 34 is an embodiment in which the feedback network disclosed in FIG. 28 is used as part of the hierarchical network of the embodiment disclosed in FIG.

Figure 35 shows an optimal embodiment of a combination of a feedback network and a hierarchical network. A daisy chain is provided as the input layer, and ANP1, 2, and 3 are provided in the intermediate layer. ANP4 and 5 are provided in the output layer.
The outputs of P1, P2, and P3 are fed back to the analog bus B1 via the analog bus B2 and the common analog bus CD. A max value node circuit 187, which functions as a dummy node, is connected to the analog buses B1 and B2. The outputs of ANP4 and ANP5, which form the output layer, are fed back to the sample/hold circuits 177 and 178.
are output to analog output ports 0 and 1 via the respective buses. B3 is an output layer analog bus.

The operation of the neural network shown in FIG. 35 will be explained with reference to FIG.

First, DCLK and WCLK are input from the master control block to the daisy chain circuit 171 and ANP1, 2, 3, 4, and 5. When CSO1 is input as CSI to the first daisy chain circuit 171 from the master control block 181, as shown, the signal from analog input port 0 appears on the analog bus B1 via the sample/hold circuit 173 and switch 175, and product-sum operations are performed in ANP1, 2, and 3 under the control of SH1 and CS1.

Next, when the CSI signal input to the second daisy chain circuit 172 rises as shown after a predetermined time has elapsed since CSO1 falls, a product-sum operation is performed in the hidden layer ANP1, 2, 3 as shown in SH1 by the analog bus B2 via the sample/hold circuit 174 and switch 176 from the analog input port 1. Similarly, when the CSI signal to the third daisy chain circuit rises as shown after a predetermined time has elapsed since CSO1 falls, a product-sum operation is performed in the hidden layer ANP1, 2, 3. Then, the output of the hidden layer ANP1, 2, 3 is
When CSO2 rises as shown and is added to ANP1, it is output to analog bus B2 and the output is fed back to analog bus B1 via common analog bus CB, so that multiplication and accumulation operations are performed again in ANP1, ANP2, and ANP3 in the middle layer, and SH
Since the output of ANP1 is generated on analog bus B2, ANP4 and ANP5 also perform product-sum operations under the control of SH2 and OC2. That is, in this embodiment, product-sum operations are performed simultaneously in the hidden layer ANP1, ANP2, and ANP3 and the output layer ANP4 and ANP5.

Next, after a certain time has passed since CSO2 dropped,
When a CSI signal is input as shown in ANP2, the ANP2 output signal is fed back to the analog bus B1 via ANP2 and the common bus CB, so that product-sum operations are performed again in ANP1, 2, and 3, and product-sum operations are also performed in ANP4 and 5 at the same time.

Furthermore, when the CSI signal is input to ANP3, as shown in
NP3 generates an output signal on the address bus B1, so ANP1, 2, 3
The product-sum operation is performed simultaneously in ANP4 and ANP5 of the output layer.

Next, when a dummy signal DCSI is given to the max value node circuit 187, a constant voltage is output to the analog bus B, and this voltage is fed back via the common bus CB and the analog bus B1, and a product-sum operation is performed on this voltage by ANP1, 2, and 3. At the same time, a product-sum operation is also performed on the output layer ANP4 and 5.

SYNC1 is high during the period when the multiplication and accumulation is performed in the hidden layer and during the period when the multiplication and accumulation is performed in the hidden layer and the output layer.
2 is high while the multiplication and accumulation operations are performed in the hidden layer and the output layer. When CSO3 is output, ANP4 produces an output at , and after a predetermined time after the CSO3 signal falls, ANP5 also produces an output at .

While the address 1 and enable signals are low,
WCLK is inhibited.

The embodiment shown in FIG. 37 is an embodiment for simplifying the hierarchical network of the embodiment shown in FIG. 12.
In this embodiment, the number of basic units 121 is one. Since one basic unit 121 cannot separately hold data relating to the plurality of basic units 121 shown in FIG. 12, when using this embodiment, it is necessary to provide an external memory means for this purpose, or
A plurality of accumulators 123 and a plurality of output holders 125 are prepared internally, and a mechanism for selecting them is provided.

The neurocomputer of the present invention, which performs non-linear data processing, has a wide range of applications. For example, Figure 38 shows an example of application of the neurocomputer to an adaptive control device for a multi-joint robot. In this adaptive control device, when a target joint angle is given as a command value for a multi-joint robot arm in which the required torque changes dynamically depending on the joint angle and angular velocity, the following is executed:
The required torque is calculated to realize feed-forward control. That is, the given angle command value is
The inputs X and Y are supplied to the first-stage transmission line in a time-division manner together with the angle command value and angular acceleration command value, and are output as a control command torque from the final-stage output layer through the data transfer process described above. Figure 39 shows an example of an application used in exclusive OR. That is, inputs X and Y are supplied to the first-stage transmission line in a time-division manner, and the operation result is output by processing them in parallel. Given these diverse applications, we believe that this invention, which can realize a neurocomputer as hardware, is of great value.

When we look at recall processing in a hierarchical network,
When a network is trained in advance to learn the correspondence between the "name of an object" and its features such as "shape, color, and taste," the network can recall the "name of an object" based on the information it has accumulated through learning, such as "round, red, sweet and sour" indicating an "apple," or "round, yellow, sour" indicating a "lemon."

When we look at associative processing in feedback networks,
If the network is trained in advance to learn the three-character font "A, B, C," even if a corrupted character font is input to the network, the network will be able to restore and interpolate the image information and associate the correct character.

As can be seen from the above, in a hierarchical network, a name stored in the network as weights is selected based on multiple conditions, whereas in a Hopfield network, the whole can be associated with partial information.

Furthermore, when we look at the function of a three-layer network, if the information input and output to the network is two-valued (logical value), any logic circuit can be realized by connecting logic elements AND, OR, and NOT in at least two stages, and it can be said from McCullough and Pitts' formal neuron model that these logic elements can be realized using neuron elements.

When the input and output information to the network is continuous, changing the weights on the neurons results in scaling the nonlinear function (the dichmoid function) vertically and horizontally, and by providing a threshold value, performing a parallel translation, various basic functions required to approximate the desired function can be expressed. Therefore, by appropriately setting the weights between the first and second layers, various basic functions can be prepared, and then the second and third layers can be used.
Networks with three or more layers can create various functions by superimposing basic functions using the weights between layers.

According to the present invention, when considering a front layer consisting of n neurochips and a back layer consisting of m neurochips, the number of wirings would be nm in the past, but according to the embodiment of the present invention, it is possible to reduce the number of wirings to one analog bus, so that the number of wirings can be significantly reduced.Furthermore, when inputting analog signals to a layer consisting of n neurochips, they can be input simultaneously via the analog bus as in the broadcasting system, so that the n neurochips in one layer can perform parallel calculations.Furthermore, pipeline processing is performed for each layer, so that the calculation speed can be increased.

In addition, since the neurochip is constructed with analog circuits, the circuit size can be small, and therefore the power consumption is small, so a neurocomputer can be constructed using many neurochips. Furthermore, the number of neurochips can be easily increased by changing the control pattern stored in the control pattern memory in the master control block.

──────────────────────────────────────────────────── Continued from the front page (72) Inventor: Kazuo Asakawa 2-10-1 Miyamaedaira, Miyamae-ku, Kawasaki, Kanagawa Prefecture 3-403 (72) Inventor: Hiroyuki Tsuzuki 3-22 Yurigaoka, Asao-ku, Kawasaki, Kanagawa Prefecture 102-108 (72) Inventor: Shuichi Endo 1775-2 Hisasue, Takatsu-ku, Kawasaki, Kanagawa Prefecture (72) Inventor: Takashi Kawasaki 409-1 Miyauchi, Nakahara-ku, Kawasaki, Kanagawa Prefecture (72) Inventor: Toshiharu Matsuda 3-5-9 Sugasengaya, Tama-ku, Kawasaki, Kanagawa Prefecture (72) Inventor: Masataka Tsuchiya 6-3-35 Tamagawa Gakuen, Machida City, Tokyo (72) Inventor: Katsuya Ishikawa 123-4 Miyazaki, Miyamae Ward, Kawasaki City, Kanagawa Prefecture (72) Inventor: Hiroshi Iwamoto 31-2-709 Hiragata-cho, Kanazawa Ward, Yokohama City, Kanagawa Prefecture

Claims (40)

[Claims] [Claim 1] A network configuration data processing device, the basic unit of which is a basic unit (11) that receives a plurality of inputs from a previous layer and weights to be multiplied by these inputs to obtain a sum of products, and that converts the obtained sum of products by a nonlinear function to obtain a final output, and that forms internal connections between the input layer and the previous hidden layer, between the hidden layers, and between the last hidden layer and the output layer, and that sets the weights corresponding to these internal connections, thereby forming a hierarchical network, wherein input and output of the basic unit (11) are realized by analog signals, and to realize the transfer of these analog signals, electrical connections between the input layer and the previous hidden layer, between the hidden layers, and between the last hidden layer and the output layer are respectively formed by analog buses (B1, B2), and the basic units (11) of the previous layer are configured to be selected sequentially in time series, and the basic units (11) selected in this way transfer data to the basic units (11) of the subsequent layer via the analog buses (B1, B2).
2. A network configuration data processing device according to claim 1, wherein said network configuration data processing device processes said network configuration data so that its final output value is output in accordance with a time-division transmission format. [Claim 2] A network-configuration data processing device having a basic unit (11) as its basic unit, which receives a plurality of inputs and weights to be multiplied by these inputs to obtain a product sum, and transforms the obtained product sum value using a nonlinear function to obtain a final output, a plurality of said basic units (11) forming a processing layer, and by feeding back the final output of each basic unit (11) of this processing layer to the input of all basic units (11) of the processing layer, equivalently forming a hierarchical network consisting of a plurality of processing layers, wherein the input and output of said basic units (11) are realized by analog signals, and to realize the transfer of these analog signals, the feedback lines of the basic units (11) of said processing layer are constituted by analog buses (CB), and the basic units (11) of said processing layer are configured to be selected sequentially in time series, and the basic units (11) selected in this way output their own final output values to the basic units (11) of said processing layer via said analog buses (CB) in accordance with a time-division transmission format. [Claim 3] A network configuration data processing device according to claim 1, characterized in that the network configuration data processing device is configured using the network configuration data processing device according to claim 2 in the form of an input layer, some or all of the intermediate layers or output layer, or a combination thereof. [Claim 4] A neural network (18) consisting of a set of analog neurochips that input analog signals from a first analog bus in a time-division manner, execute product-sum operations using digital weight data, and output analog signals to a second analog bus; a control pattern memory (12) that stores patterns of control signals for the neural network (18); a sequencer (13) that generates addresses for the control pattern memory (12); and a weight memory (14) that stores weight data.
4), and the neural network (18) and control pattern memory (1
2), a sequencer (13), and a digital control means (15) for controlling the weight memory (14) as a whole. [Claim 5] A multiplication unit (22) that multiplies an analog signal by serial weight data, an addition unit (23) that adds the output signal of the multiplication unit (22) in response to an analog signal that is input in a time-division manner, an addition time control means that varies the time for charging a signal to a capacitor of the addition unit (23), and a sample/hold means (2) connected to the addition unit (23).
4), a nonlinear circuit (25) to which an output signal of the sample/hold means (24) is input, and an output section (26) to which an output signal of the nonlinear circuit (25) is output as an analog time-division signal. [Claim 6] A first system comprising: an input side circuit (120) for inputting an analog input signal; an input layer analog bus (140a) connected to the output of said input side circuit (120); and an analog neuron processor (121-h) connected in parallel to said input layer analog bus (140a).
a hidden layer of the first hidden layer, an analog bus (140b) for the hidden layer to which the output of the first hidden layer is connected, a second hidden layer in which analog neuron processors (121-i) are connected in parallel to the analog bus (140b) for the hidden layer, and the analog neuron processors (121-i) of the hidden layer.
a next hidden layer analog bus (140c) connected to said next hidden layer analog bus (140c); a control means (150) for controlling said hidden layer analog neuron processor (121-h) to input the time-division analog signals generated on said input layer analog bus and make it perform sequential product-sum operations; a control means (150) for controlling said next hidden layer analog neuron processor (121-i) to input the time-division analog signals generated on said hidden layer analog bus and perform sequential product-sum operations; and a means (121-j) for outputting the time-division analog signals generated on said next hidden layer analog bus. 7. A neuron unit comprising: a first analog bus to which an analog signal is input in a time-division manner; a multiplication and accumulation operation is performed using digital weight data; and an analog signal is output to a second analog bus. 8. The neuron unit according to claim 7, wherein said product-sum operation inputs an analog input signal to a reference voltage terminal of a D/A converter and inputs digital weight data to a digital input terminal. 9. A neuron unit characterized in that an analog signal is input from a first analog bus in a time-division manner, a product-sum operation is performed using digital weight data, and the result of the product-sum operation is output as an analog signal to a second analog bus through a nonlinear function circuit. [Claim 10] The nonlinear function generating means of the neurocomputer of claim 9 is characterized in that a current mirror circuit is connected to the collector of a current switching circuit, and multiple amplification stages are provided, each of which has an emitter resistor and a constant current circuit connected to the emitter side, the collector output of each amplification stage is connected to one end of an output resistor, and a nonlinear function is formed by limiting each piecewise linearity of the nonlinear function in each stage by the ratio of the emitter resistor to the output resistor. 11. A method for synthesizing a signal from a first analog bus to a second analog neuron processor, comprising: a first analog bus for receiving an analog signal; an analog neuron processor connected to said first analog bus for performing a multiply-and-accumulate operation on an input signal from said first analog bus and weight data; a second analog bus connected to said analog neuron processor; a weight memory for storing said weight data; and an analog signal from said first analog bus in a specific time section is input to said analog neuron processor.
and control means for calculating the product of an analog input signal and said weight data, adding this product to the sum of products obtained in the previous time segment, and outputting the resulting analog signal in a specific time segment on said second analog bus. [Claim 12] The control means comprises a master control block that controls the analog neuron processor, and the master control block comprises a control pattern memory that generates a control pattern for the control signals of the analog neuron processor, a sequencer that controls the sequence of addresses in the control pattern memory, a microcode memory that stores instructions that control the sequence of the sequencer, address control means connected to the sequencer that controls the addresses of the weight memory, and interface means that connects to an external digital control means, and is characterized in that it allocates the right to use the analog bus to one analog neuron processor in a specific time zone. 13. The control pattern memory includes first and second control pattern memories.
The neurocomputer according to claim 12, characterized in that it comprises a control pattern memory, wherein the first control pattern memory stores a control pattern corresponding to a control signal that controls the ANP, and the second control pattern memory has at least a starting address to the first control pattern memory. 14. The control pattern memory includes first and second control pattern memories.
The neurocomputer according to claim 12, characterized in that it comprises a control pattern memory, wherein the first control pattern memory stores a control pattern corresponding to a control signal that controls the ANP, and the second control pattern memory has a top address to the pattern memory and a number of repetitions of the original pattern. 15. The original pattern of said first control pattern memory is a data clock that provides a read timing of said memory for analog signal processing of ANP.
The neurocomputer of claim 14, characterized in that it stores at least one of DCLK, a weighted clock WCLK, an S/H signal for sampling/holding analog signals in each layer, an offset control signal within the ANP, CSI and CSO signals, a synchronization signal SYNC for determining the operating time within each layer, a reset signal, and an enable signal for a dummy node. 16. The neurocomputer according to claim 12, wherein said microcode memory has a repeat command for repeating said original pattern and an command for specifying the start address of each original pattern. 17. The method of claim 12, wherein data in the bit width direction of the original pattern is simultaneously read out from said control pattern memory by updating the address.
A neurocomputer as described in paragraph . [Claim 18] The neurocomputer of claim 12, characterized in that the master control block has an address counter that counts addresses in the weight memory, the address counter is counted by the weight clock WCLK, the count operation is enabled by a count control signal from the sequencer, and the counter is reset by the sequencer to return to the starting address where the weight data in the weight memory is stored. [Claim 19] A neurocomputer as described in claim 12, characterized in that when the contents of the control pattern memory and the contents of the microcode memory are given as data from the MPU side via an external interface circuit, the master control block decodes the upper address from the MPU and inputs a latch signal of appropriate timing sequence data formed in the timing path, latches the lower address and data according to the latch signal, and stores the control pattern at the addresses of the microcode memory and control pattern memory specified by the lower address. 20. The neurocomputer according to claim 11, wherein said weight memory supplies weight data from one memory device to a plurality of ANPs in parallel in the bit width direction at the same time. 21. The weight memory includes a weight data storage means and a bidirectional buffer connected to an MPU, a selection means for selecting an address from the master control block and an address from the MPU in a first mode to transmit the output of the weight memory to an ANP, and a selection means for selecting an address from the master control block and an address from the MPU in a second mode to transmit the weight memory to an ANP.
12. The neurocomputer according to claim 11, wherein data is transmitted and received between the MPU and the weight memory by referring to an address from the MPU. 22. The present invention is characterized in that at least first and second ANPs are connected between the first and second analog buses, and a control circuit is provided which controls the first ANP to generate an output signal on the analog bus when a CSI signal is applied to the first ANP, and to apply a CSI signal to the second ANP after a predetermined time has elapsed, thereby generating an output signal on the analog bus.
12. The neurocomputer according to claim 11. 23. At least first and second ANPs are connected between the first and second analog buses, and a CSO signal is generated by delaying the CSI signal of the first ANP by a specific time, and this C
12. The neurocomputer according to claim 11, further comprising a daisy-chain circuit for generating an output from the second ANP on an analog bus using the SO signal as a CSI signal for the second ANP. [Claim 24] A neurocomputer as described in claim 23, characterized in that the daisy-chain circuit comprises an FF that inputs a CSI signal and outputs a CSO, and a counter means that determines the output timing of the CSO based on the count number of WCLK pulses, a predetermined delay amount from the end edge of the CSI. 25. A neurocomputer as claimed in claim 11, characterized in that a CSO signal is generated by delaying the CSI signal from the first daisy-chain circuit by a specific time, and this CSO signal is used as the CSI signal for the second daisy-chain circuit, and an output is generated on the analog bus under the control of the second daisy-chain circuit. 26. A neurocomputer according to claim 15, wherein an input layer is formed from said daisy chain circuits. 27. ANP performing multiple daisy-chain operations.
For one ANP on the first layer consisting of
An SI signal is added, and a CSO signal is generated after a predetermined time. This CSO signal is used as the CSI signal for the adjacent ANP. This daisy-chain operation is performed sequentially for multiple ANPs in one layer, and the ANP in the first layer
After the daisy chain operation is completed, the next layer ANP
12. The neurocomputer according to claim 11, wherein the daisy-chain operation is started for: 28. A neurocomputer as claimed in claim 11, characterized in that a means for generating a predetermined voltage is provided on the analog bus, and this means is a max value node circuit which constitutes a dummy node that generates the predetermined voltage for a specific time by enabling its enable signal from the main control circuit at a specific time. 29. A neurocomputer according to claim 28, wherein said predetermined voltage generating means is a voltage corresponding to a threshold value of a nonlinear function circuit. 30. A neurocomputer according to claim 28, wherein said predetermined voltage generating means is set to 0 voltage during testing to test the sequence. [Claim 31] A neurocomputer as described in claim 28, characterized in that the predetermined voltage generating means is a dummy node connected to each analog bus and generating a dummy signal to the target ANP, and daisy chain operation is performed including this dummy node. [Claim 32] A neurocomputer as described in claim 28, characterized in that the max value node circuit has means for generating a fixed voltage and memory control means for transmitting the fixed voltage to a dummy node output in response to an enable signal from a master control block. [Claim 33] A neurocomputer comprising a neuron unit consisting of an analog circuit section which performs a product-sum operation on an analog input signal to generate an analog output signal as a signal, and a digital circuit section which controls said analog circuit section, said digital circuit section comprising a sequence generator which converts a control signal from a master control block into a switching control signal within an ANP, a phase control circuit which controls the phase of a control signal which controls the switching of a switch element included in said analog circuit, and a shift register means which converts serial weight daisy chains given from a weight memory into parallel data. 34. The sequence generator receives a reset signal and outputs a CR signal to discharge the charge on the capacitor of the integrator.
34. The neurocomputer of claim 33, which receives as input an ST signal, a synchronization signal SYNC, and DCLK, WCLK, and WD, and generates at least a sample/hold circuit, an enable signal for the parallel output of the shift register, an enable signal for the sign bit, an internal clock CLK, weight data, a shift clock, a selection control signal for selecting whether to pass through a nonlinear function, and a CSO signal for daisy chain. [Claim 35] A neurocomputer as described in claim 33, characterized in that the phase control means comprises first and second delay means for delaying the opposite phase signal and the positive phase signal, and means for feeding back the output of one delay means to the input of the other delay means, and performs phase control so that the two output signals are positive phase and opposite phase and the switches controlled by them are not turned on at the same time. [Claim 36] A neurocomputer as described in claim 33, characterized in that the shift register is enabled by a control signal, serializes weight data bit-serial by a clock formed from WCLK, and then enables it by a WR signal to output it in parallel. 37. An input layer, at least one hidden layer ANP and at least one
A neurocomputer characterized by a hierarchical network consisting of two output layers. 38. A neurocomputer that forms a hierarchical network by time-division multiplexing each layer by feeding back the output signals of a layer consisting of a plurality of ANPs to the input side via an analog bus. 39. A neurocomputer comprising a feedback network that repeats feedback operations until the output signal of a layer consisting of a plurality of ANPs reaches a steady state. 40. A neurocomputer comprising a neural network constructed by combining a hierarchical neural network and a feedback neural network using ANP.