CN1464477A - A process for constructing multiple weighing value synapse nerve cell - Google Patents

Abstract

A method for constructing neurons of multi-weight synapses, comprising the steps of: (1) determining the number of synapses of neurons; (2) constructing multi-weight synapses; (3) selecting synapses according to synaptic functions and system characteristics Activation threshold; (4) forming multi-weight synaptic neurons; (5) determining the threshold according to synaptic function, activation function, input data and synaptic number.

Description

A neuron construction method for multi-weight synapses

technical field

The invention belongs to the field of computers and provides a neuron construction method of multi-weight synapse.

Background technique

Neuron is the basic unit of neural network and an important factor affecting the performance of neural network. The general neuron structure is shown in Figure 1. It can be seen from the figure that each input (synapse) of a neuron has only one weight, which is designed to simulate the structure of biological neurons. The mathematical expression of the neuron is as follows, where Y is the output of the neuron, Xi _is the neuron $Y=f\left[\underset{i=0}{\overset{\mathrm{no}}{\mathrm{\Σ}}}{W}_i{x}_i-\mathrm{\θ}\right]$ The input of the neuron, W _i is the input weight of the neuron, θ is the activation threshold of the neuron, and f is the activation function of the neuron. It can be seen from the formula that each input (synapse) of a neuron corresponds to a weight, and the neuron constitutes a hyperplane in a high-dimensional space. By adjusting each weight W, the hyperplane can be adjusted direction.

The later development of neurons is representative of the neural network adopted by the radial basis function (RBF) network. $Y=f[\underset{i=0}{\overset{\mathrm{no}}{\mathrm{\Σ}}}{(W_i-x_i)}^2-{\mathrm{\θ}}^2]$ Metamodel, as shown in the following formula. It can be seen that each input (synapse) of a neuron still has only one adjustable weight. Different from the normal neuron model, this neuron constitutes a hypersphere in high-dimensional space instead of an open hyperplane. By adjusting the weight W, the center point of the hypersphere can be adjusted. Various experiments have proved that the performance of this type of neural network is better than that of a network composed of ordinary neurons.

From the above introduction, it can be seen that the hypersurface formed by neurons with only one weight synapse is relatively simple and cannot meet the needs of complex surfaces. Moreover, when ordinary neurons implement neuron networks with software or hardware, due to the different neuron structures, they need to be designed and implemented for different neuron networks, which wastes resources and time, especially in hardware implementation. The waste is even greater. On the basis of fully studying the existing neuron structure, this patent application proposes a multi-weight synaptic neuron structure, which makes the neuron more versatile, more powerful, and more flexible.

Contents of the invention

The purpose of the present invention is to provide a neuron construction method of multi-weight synapse, which effectively solves the problem of commonality between different original neurons, and significantly enhances the function and flexibility of neurons.

A kind of neuron construction method of multi-weight value synapse of the present invention is characterized in that, comprises the following steps:

(1) Determine the number of synapses of neurons;

(2) Construct multi-weight synapses;

(3) Select the activation threshold according to the synaptic function and system characteristics;

(4) Select the excitation function according to the system characteristics and the activation threshold;

(5) Form multi-weight synaptic neurons.

The construction of a multi-weight synapse refers to that each input of a neuron contains two or more adjustable weights.

The synaptic function mentioned here means that the synaptic function is arbitrary, and any form of function can be adopted according to the actual situation of the specific application where the neuron is located.

The neuron described herein is a function of any form in the actual situation of a specific application.

The synaptic function mentioned therein refers to the relationship formed by the input Xi of each neuron, each weight Wij on it, and the value Fi finally input to the neuron, that is, Fi=F(Xi, Wi1, Wi2,. .., Win); F is the synaptic function.

Description of drawings

Fig. 1 is a general neuron model diagram;

Figure 2 is a neuron structure diagram of a multi-weight synapse;

Fig. 3 is a flow chart of the present invention;

Fig. 4 is a partial body diagram of neurons constructed in the embodiment of the present invention.

Detailed ways

The neuron structure schematic diagram proposed by the present invention is shown in FIG. 2 , that is, each synapse (input) has multiple weights, not just one weight. Moreover, the combination of these weights can be changed according to actual needs, for example, multiplication, addition, division, subtraction or other various combinations and operations can be performed.

As shown in Figure 2, the neuron model of the present invention is shown in the following formula:

Y=f(∑F _i -θ)

in,

F _i ＝F(X _i ,W _i1 ,W _i2 ,...,W _ij ...,W _im ) Y is the output of neuron, X _i is the input of neuron, F is the synaptic function of neuron , W _ij is the jth weight of the i-th synapse of the neuron, m is the number of weights of each synapse (in this scheme, m>1), θ is the activation threshold of the neuron, f is the activation function of the neuron. In this scheme, the form of F can be arbitrary, it can be a continuous function or a discontinuous function. The feature of the F function in the patent application is that the weight value is not only one, that is, m>1.

It can be seen from the model of this scheme that the existing various neuron models are just a simple special case of this scheme. The multi-weight synapse neuron constructed according to the scheme obviously has wider versatility, stronger function and greater flexibility than the existing one-weight synapse neuron.

Please refer to Fig. 3, the neuron construction method of a kind of multi-weight value synapse of the present invention, comprises the following steps:

1. Determine the number of synapses in neurons;

2. Constructing a multi-weight synapse means that each input of a neuron contains two or more adjustable weights;

3. Select the activation threshold according to the synaptic function and system characteristics, wherein the synaptic function means that the synaptic function is arbitrary, and any form of function can be used according to the actual situation of the specific application of the neuron; wherein the synaptic function The touch function refers to the relationship formed by the input Xi of each neuron, each weight Wij on it, and the value Fi finally input to the neuron, that is, Fi=F(Xi, Wi1, Wi2, . . . , Win). F is the synaptic function;

4. Select the excitation function according to the system characteristics and activation threshold; it is a function of any form in the actual situation of the specific application;

5. Form multi-weight synaptic neurons, and determine the threshold θ according to the synaptic function, activation function, input data and the number of synapses.

Example

The technology patented in this application has been applied to the CASSANDRA-II neural computer, realizing a general neural network compatible with common neuron models and RBF neuron models. the neuron $Y=f\{\mathrm{\Σ}{\left[\frac{W_{i1}(x_i-W_{i2})}{\left|W_{i1}(x_i-W_{i2})\right\}}\right]}^{\mathrm{the\; s}}{\left|W_{i1}(x_i-W_{i2})\right|}^m-\mathrm{\θ}\}$ The model adopts a structure with two input weight synapses, as shown in the following formula. $f_i=f(x_i,W_{i1},W_{i2})$ $=\mathrm{\Σ}{\left[\frac{W_{i1}(x_i-W_{i2})}{\left|W_{i1}(x_i-W_{i2})\right|}\right]}^{\mathrm{the\; s}}{\left|W_{i1}(x_i-W_{i2})\right|}^m$ Among them, the synaptic function is: Y is the output of the neuron, Xi is the synapse (input) of the neuron, Wi1 and Wi2 are the two weights of the synapse with the neuron label i, and θ is the activation threshold of the neuron , f is the activation function, s is the parameter for determining the sign method of the single item, when s=0, the sign of the single item is always positive, when s=1, the sign of the single item is the same as that of Wi1(Xi-Wi2), m is power parameter. It is not difficult to see that if all Wi2 are 0, s=1, m=1, then the model is an ordinary single-weight neuron structure; if all Wi1 are 1, s=0, m=2, then the model The model is the RBF neuron structure, so the neuron model can be flexibly adapted to these two existing neuron structures, and the CASSANDRA-II constructed from it also has good versatility. In fact, by modifying the parameters s and m, the model can construct a more complex neuron structure, not just the above-mentioned hyperplane and hyperspherical structure. Figure 4 is s=0, and the distribution of m is 1/3 and 1 /2 is the shape in the three-dimensional space constructed by the model neurons. It can be seen that the multi-weight synaptic neuron method proposed in the patent application is a neuron construction method with good versatility, strong functions and high flexibility, which significantly improves the function and practicability of the neural network.

Claims (5)

1, a kind of neuron construction method of multi-weight value synapse, is characterized in that, comprises the steps: (1) Determine the number of synapses of neurons; (2) Construct multi-weight synapses; (3) Select the activation threshold according to the synaptic function and system characteristics; (4) Select the excitation function according to the system characteristics and the activation threshold; (5) Form multi-weight synaptic neurons. 2. The neuron construction method of multi-weight synapse according to patent claim 1, characterized in that, the construction of multi-weight synapse refers to that each input of neuron includes two or two or more adjustable weights. 3. The neuron construction method of multi-weight synapse according to patent claim 1, characterized in that, the synaptic function refers to that the synaptic function is arbitrary, and can be based on the actual situation of the specific application where the neuron is located. A function that takes an arbitrary form. 4. The neuron construction method for multi-weight synapse according to patent claim 1, wherein the neuron is a function of any form in the actual situation of the specific application. 5. According to the multi-weight neuron construction method described in patent claim 1 or 3, it is characterized in that the synaptic function refers to the input Xi of each neuron, each weight Wij on it, and the final The relationship formed by the value Fi input to the neuron is Fi=F(Xi, Wil, Wi2, . . . , Win); F is the synaptic function.

Images