JP2002008032A - Pattern detection device and method, image input device and method, neural network circuit - Google Patents

Abstract

(57) [Problem] To improve the reliability of pattern detection and reduce the wiring problem between elements, so that the circuit scale and power consumption can be suppressed. The A pattern detection apparatus, means for inputting the pattern, a plurality of neuron element, and means for detecting a predetermined pattern contained in the input pattern, from the plurality of neuron elements n ₁ ~n ₄ the output pulse signal of, is identifiably transmitted to neuron element n _'j even via a common bus line 301 time axis by receiving inherent delay by each synapse circuit S _1j to S _4j, neuron element n ' _j outputs a pulse signal at an output level according to the arrival time pattern of a plurality of pulse signals P ₄ , P ₃ , P ₂ , P ₁ inputted from the neuron elements n _{1 to} n ₄ within a predetermined time window .

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a pattern detecting apparatus and method for performing pattern recognition, detection of a specific subject, and the like using a parallel operation device such as a neural network, and a neural network circuit.

[0002]

2. Description of the Related Art Conventionally, in the field of image recognition and voice recognition, a type in which a recognition processing algorithm specialized for a specific recognition target is sequentially calculated and executed as computer software, or a dedicated parallel image processor (SIM) is used.
D, MIMD machine, etc.).

A typical example of the image recognition algorithm will be described below. First, as a feature to calculate and perform a feature amount related to the similarity with the recognition target model, model data of the recognition target is expressed as a template model, and similarity to the input image (or its feature vector) is determined by template matching or the like. Calculation of degree, calculation of higher-order correlation coefficient, etc., mapping of input pattern to eigenimage function space obtained by principal component analysis of target model image, and calculation of distance between model and feature space Method (Sirovic
h, et al., 1987, Low-dimensional procedure for the
characterization of human faces, J. Opt. Soc. Am.
[A], vol. 3, pp.519-524), graphically represent multiple feature extraction results (feature vectors) and their spatial arrangement,
A method of calculating similarity by elastic graph matching (Lades et al. 1993, DistortionInvariant Object Re
cognition in the Dynamic Link Architecture, IEEE T
rans. on Computers, vol.42, pp.300-311), a method of performing a predetermined transformation on an input image to obtain a position, rotation, and scale-invariant representation and then collating it with a model (Seibert, et a).
l. 1992, Learning and recognizing 3D objects from
multiple views in aneural system, in Neural Networ
ks for Perception, vol. 1 Human and Machine Percep
tion (H. Wechsler Ed.) Academic Press, pp.427-444
Etc.

As a pattern recognition method using a neural network model inspired by the information processing mechanism of a living body, a method of performing hierarchical template matching (Japanese Patent Publication No. 60-712, Fukushima & Miyake, 1982 Neocognitron: A new a)
lgorithm for pattern recognition tolerant of defor
mation and shifts in position, Pattern Recognitio
n, vol.15, pp.455-469), a method of obtaining a scale and position-invariant representation of the center of an object using a dynamic routing neural network (Anderson, et al. 1995, RoutingNetwo)
rks in Visual Cortex, in Handbook of Brain Theory
and Neural Networks (M. Arbib, Ed.), MIT Press, p
p.823-826), other multilayer perceptrons, radial basis functions
(Radial Basis Function) There is a method using a network or the like.

On the other hand, as an attempt to more faithfully incorporate an information processing mechanism based on a biological neural network, a neural network model circuit has been proposed in which information is transmitted and represented by a pulse train corresponding to an action potential (Mur).
ray et al., 1991 Pulse-Stream VLSI Neural Networks
Mixing Analog and Digital Techniques, IEEE Trans.
on Neural Networks, vol.2, pp.193-204.
No. 62157, JP-A-7-334478, JP-A-8-153148
And Japanese Patent No. 2879767).

As a method of recognizing and detecting a specific object by a neural network composed of pulse train generating neurons, a higher order (second or higher order) by Eckhorn et al. On the assumption of a linking input and a feeding input is used. ) Model (Eckhorn, et al. 1990, Feature linking via synchr
onization among distributed assemblies: Simulation
of results from cat cortex, Neural Computation, V
ol.2, pp.293-307), that is, a method using a pulse-coupled neural network (hereinafter abbreviated as PCNN) (US Pat. No. 5,664,0).
65 and Broussard, et al. 1999, Physiologically
Motivated Image Fusion for Object Detection using
a Pulse Coupled Neural Network, IEEE Trans. on Neu
ral Networks, vol. 10, pp. 554-563, etc.).

As a method of reducing the wiring problem in the neural network, there is a method of encoding the address of a so-called pulse output neuron in an event-driven manner (Address Event Representation: AER) (Lazzaro, et al. 1993). , Silicon Auditory
Processors as Computer Peripherals, In Tourestzky,
D. (ed), Advances in Neural Information Processin
g Systems 5. San Mateo, CA: Morgan Kaufmann Publishe
rs). In this method, the neuron I of the pulse train output side is
Even if output signals from different neurons are temporally arranged on a common bus by binary coding using D as an address, the input neuron automatically decodes the address of the original neuron. can do.

[0008]

When the above-mentioned conventional method is realized by a circuit or the like (particularly with respect to application to image recognition), a function constituting each processing unit (for example, feature extraction, etc.)
, Especially at the arithmetic element level, module level,
Alternatively, at any system level, there has been no method for expressing information relating to a two-dimensional pattern by utilizing the dynamic characteristics on the time axis and utilizing the information for recognition or the like.

That is, in many cases, spatial pattern information is converted from a finite number of states at a certain time of a spatially arranged arithmetic element (module) (typically, a binary value of firing and non-firing). ) On the premise that processing proceeds according to a pattern, processing has been limited to processing in the domain of digital information expression.

For this reason, the information processing capability is limited, and when it is realized as a circuit, its scale tends to be excessively large and expensive. In particular, the proportion of the area occupied by an enormous number of wirings associated with connections between neurons is not small but large, which has been a problem.

In order to alleviate the above-mentioned problems, the above-mentioned AER has been proposed. However, this method requires a mechanism for sequentially encoding and decoding the addresses of neurons, and the circuit configuration is complicated. There was a point.

On the other hand, in a neural network model for performing predetermined processing by inputting / outputting / transmitting a spike train, analog information (such as an interval between spikes) in the time axis domain of the spike train is used for coding image information. As for the method of realizing the recognition function by using a neural network, no specific configuration including a document related to the PCNN described above is disclosed.

Further, in the configuration using analog circuit elements,
It is generally known that simplification of the circuit configuration (small number of elements), high speed, and low power consumption are brought about as compared with the digital method. Reliability and resistance to noise have been considerable problems.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problem, and has dramatically improved the information processing ability for pattern detection, and has been able to detect a predetermined pattern with a smaller circuit scale and a higher accuracy. It is an object of the present invention to provide a pattern detection device which can be realized in the above.

It is another object of the present invention to provide a low-cost neural network circuit with a simple configuration by reducing the number of wirings required for connection between neurons.

[0016]

According to the present invention, there is provided, in accordance with the present invention, a pattern detecting apparatus comprising: an input means for inputting a pattern; and a plurality of signal processing elements.
Pattern detection means for detecting a plurality of predetermined features of the pattern input from the input means to detect a predetermined pattern included in the pattern, wherein each of the plurality of signal processing elements In response to an input from the input means or another signal processing element, a pulse signal is output to another signal processing element or to the outside, and a predetermined part of the plurality of signal processing elements is within a predetermined time range. A pulse signal is output at an output level according to an arrival time pattern of a plurality of input pulse signals.

According to another aspect of the present invention, an input means for inputting a pattern and a pattern detection means for detecting a predetermined pattern included in the pattern input from the input means are provided to the pattern detection device. A feature detection layer that extracts a predetermined feature from the input pattern; a feature integration layer that integrates and outputs an output from the feature detection layer by a predetermined method; And a feature position detection layer that receives the output from the layer and outputs position information where a predetermined feature or pattern exists.

According to another aspect of the present invention, a neural network circuit includes a plurality of processing layers each including a plurality of neuron elements for inputting a plurality of signals and outputting a pulse signal in parallel. A pulse signal output from a plurality of neuron elements in another layer to at least one of the neuron elements in the predetermined processing layer; a synapse coupling unit provided for each of the plurality of neuron elements; The synapse coupling means is input to a plurality of neuron elements via a common bus line, and applies a pulse phase shift amount unique to each of the plurality of neuron elements to a pulse signal output from the plurality of neuron elements. It is characterized by the following.

According to another aspect of the present invention, a pattern is input from an input unit, and a plurality of signal processing elements are used.
In a pattern detection method of detecting a predetermined plurality of features with respect to the input pattern and detecting a predetermined pattern included in the pattern, each of the plurality of signal processing elements may include the input unit or another In response to an input from the signal processing element, a pulse signal is output to another signal processing element or to the outside, and a predetermined part of the plurality of signal processing elements includes a plurality of pulses input within a predetermined time range. A pulse signal is output at an output level according to a signal arrival time pattern.

According to another aspect of the present invention, in a pattern detection method for inputting a pattern and detecting a predetermined pattern included in the input pattern, a pattern detection method detects a predetermined pattern from the input pattern by a feature detection layer. The output from the feature detection layer is integrated and output in a predetermined manner by the feature integration layer, and the output of the feature integration layer is received. It outputs the position information to be performed.

[0021]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS <First Embodiment> An embodiment of the present invention will be described below in detail with reference to the drawings.

In the present embodiment, when performing pattern recognition using a neural network, the spatial pattern information is represented by
By utilizing the dynamic characteristics of the constituent neurons in the time domain (for example, spatio-temporal firing characteristics of spike train, input characteristics by spatio-temporal integration of neurons, spike train spike interval, etc.), Provided is a pattern detection device that dramatically improves the information processing capability related to spatial pattern recognition, and that can be realized with high accuracy with a smaller circuit scale than before.

FIG. 1 is a diagram showing the overall configuration of the pattern detection / recognition apparatus of the present embodiment. Here, the pattern information is processed by a What route and a Where route. Wh
The at path mainly deals with information related to recognition (detection) of an object or a geometric feature, and the Where path mainly deals with information on the position (arrangement) of the object or feature.

What path is a so-called Convolutional network structure (LeCun, Y. and Bengio, Y., 1995, “C
onvolutional Networks for Images Speech, and Time
Series ”in Handbook of Brain Theory and Neural Ne
tworks (M. Arbib, Ed.), MIT Press, pp. 255-258). However, it differs from the conventional one in that the interlayer connection in the same path can form mutual connection (described later). The final output of the What path corresponds to a recognition result, that is, a recognized target category. The final output of the Where path indicates a location corresponding to the recognition result.

The data input layer 101 is a photoelectric conversion element such as a CMOS sensor or a CCD element for detecting and recognizing an image, and is a voice input sensor for detecting and recognizing voice. Also, the analysis result of the predetermined data analysis unit (for example, principal component analysis, vector quantization, etc.)
May be inputted. The data input layer 101 performs data input common to the two paths.

Hereinafter, the case of inputting an image will be described. The What path includes the feature detection layer 102 ((1, 0),
(1, 1), ..., (1, N)) and the feature integration layer 103 ((2, 0),
(2, 1), ..., (2, N)).

The first feature detection layer (1, 0) is Gabor wavel
The local low-order features (which may include the color component features in addition to the geometric features) of the image pattern input from the data input layer 101 by the et conversion or the like are transferred to each position on the full screen (or the full screen). (Each of the predetermined sampling points), a plurality of (types) are detected at the same location. For this purpose, each has a receptive field 105 and a plurality of detection modules 104 for detecting different characteristics. Each detection module 104 has a receptive field structure corresponding to the type of the feature amount (for example, in the case of extracting a line segment in a predetermined direction as a geometric feature, the gradient of the line segment which is the geometric structure). And a neuron element that generates a pulse train according to the degree.

The feature integration layer 103 (2,0) has a plurality of feature integration modules 106. Each feature integration module 106
It has a predetermined receptive field structure, is composed of neuron elements that generate pulse trains, and integrates a plurality of neuron element outputs in the same receptive field from the feature detection layer 102 (1,0) (calculation such as local averaging). . Each receptive field has a common structure among neurons in the same layer.

The subsequent feature detection layers 102 ((1, 1,
1), (1, 2),..., (1, N)) and each feature integration layer 103
((2,1), (2,2),..., (2, N)) each have a predetermined receptive field structure obtained by learning, and the former ((1,1,), …) Indicates each feature detection module 1
In 04, several different features were detected and the latter ((2,
1),...) Are for integrating detection results relating to a plurality of features from the preceding feature detection layer 102, and include a feature integration layer 103 (2,
It has the same function and structure as 0).

On the other hand, in the Where path, the characteristic position detecting layer 107 is provided.
((3, 0),..., (3, k)), receives the output of the predetermined (not necessarily all) feature integration layer 103 on the What path, and outputs low-order, middle-order, and high-order features. Involved in the output of the position. Hereinafter, a more detailed description of each layer of the Where path will be omitted.

As shown in FIG. 2A, the structure for connecting the neuron elements 201 between the layers includes a signal transmission unit 203 (wiring or delay line) corresponding to an axon or a dendrite of a nerve cell. And the synapse circuit S202. (A) of FIG.
Here, the configuration of the connection related to the output (input from the viewpoint of the cell N) of the neuron group (ni) of the feature integration (detection) cell that forms the receptive field for a certain feature detection (integration) cell (N) is shown. ing. Signal transmission unit 203 indicated by bold line
Constitutes a common bus line, and pulse signals from a plurality of neurons are transmitted in time series on this signal transmission line. The same configuration is adopted when receiving an input from a cell (N) of an output destination. In this case, the input signal and the output signal may be divided and processed on the time axis with exactly the same configuration, or two systems for input (dendritic side) and output (axon side) may be used. The processing may be performed by providing the same configuration as that of FIG.

The synapse circuit S202 has an interlayer connection (a connection between a neuron on the feature detection layer 102 and a neuron on the feature integration layer 103, and each layer has a connection to a subsequent layer and a preceding layer. May be involved)
Some are involved in connections between neurons in the same layer. The latter is mainly used, if necessary, mainly for coupling a pacemaker neuron described later with a feature detection or feature integration neuron.

In the synapse circuit S202, the so-called excitatory coupling amplifies the pulse signal, and the inhibitory coupling conversely attenuates. When transmitting information using a pulse signal, amplification and attenuation can be realized by any of amplitude modulation, pulse width modulation, phase modulation, and frequency modulation of the pulse signal. In the present embodiment, the synapse coupling circuit S202 is mainly used as a pulse phase modulation element, and amplification of a signal is substantially advanced in a pulse arrival time.
The decay is translated as a substantial delay.

In FIG. 2A, each neuron element
n _j outputs a pulse signal (spike train) and uses a so-called integral-and-fire type neuron element as described later. As shown in FIG. 2 (C),
The synapse circuit and the neuron element may be combined to form a circuit block.

Each feature position detection layer 107 in the Where path
Receives the output of the What path feature integration layer 103, retains the positional relationship on the data input layer 101, and uses each point on the coarsely sampled grid points for recognition of the feature extraction results on the What path. Only neurons corresponding to useful components (pre-registered from recognition category patterns) respond by filtering or the like. For example, Wh
In the uppermost layer in the ere path, neurons corresponding to the category of the recognition target are arranged on a lattice, and represent at which position the corresponding target exists. Also, the neurons in the middle layer in the Where path receive sensitivity from the top down from the upper layer and respond only when a feature that can be arranged around the location of the corresponding recognition target is detected. Adjustment or the like can be performed.

The hierarchical feature detection in which the positional relationship is maintained is represented by W
When performing the here route, the receptive field structure is local (for example, elliptical) and the size gradually increases in the upper layer (or, from the middle layer to the upper layer, is larger than one pixel on the sensor surface). (The size is constant), the characteristic elements (graphic elements, graphic patterns)
The positional relationship between them can be such that each feature element (graphic element) is detected in each layer while preserving the positional relationship on the sensor surface to some extent.

As another output form of the Where path, a gaze area of a predetermined size on the data input layer is obtained from the saliency map of the feature obtained based on the output result of the feature detection layer (1, 0). It may be set and output the position and size of the area and the presence or absence of the recognition target category in the area. In still another embodiment, the size of the receptive field increases in the upper layer hierarchically, and only the neuron that outputs the maximum value among the neurons corresponding to the detected target category is fired in the uppermost layer. A neural network may be used. In such a system, information on the arrangement relationship (spatial phase) in the data input layer is stored to some extent in the uppermost layer (and each intermediate layer).

Next, the neurons constituting each layer will be described. Each neuron element is a so-called integrate-and-
This is an extended model based on fire neurons.
The point where the result of linearly adding the input signal (pulse train corresponding to the action potential) spatiotemporally exceeds the threshold value is fired, and a pulse-like signal is output.
Same as ate-and-fire neurons.

FIG. 2B shows an example of a basic configuration representing the operation principle of a pulse generation circuit (CMOS circuit) as a neuron element. This is a known circuit (IEEE Tran
s. on Neural Networks Vol. 10, pp. 540). Here, it is configured to receive excitatory and inhibitory inputs as inputs.

Hereinafter, the principle of operation of this pulse generation circuit will be described. Excitatory input side capacitor C1 and resistor R
The time constant of one circuit is smaller than the time constant of the capacitor C2 and the resistor R2.
T2 and T3 are shut off. Note that the resistance is actually
It is composed of transistors coupled in diode mode.

The potential of the capacitor C1 increases,
Above that of C2 by the threshold of transistor T1, transistor T1 becomes active, further activating transistors T2 and T3. Transistors T2, T
Reference numeral 3 denotes a current mirror circuit, and the output of the circuit shown in FIG. 2B is output from the capacitor C1 by an output circuit (not shown). When the amount of charge stored in the capacitor C2 becomes maximum, the transistor T1 is turned off, and as a result, the transistors T2 and T3 are also turned off, so that the positive feedback becomes zero.

In the so-called refractory period, the capacitor C2 discharges, and the neuron does not respond unless the potential of the capacitor C1 is higher than the potential of the capacitor C2 and the difference does not exceed the threshold value of the transistor T1. Capacitor C1, C
A periodic pulse is output by repeating the alternate charging and discharging of No. 2 and its frequency is generally determined according to the level of the excitatory input. However, depending on the presence of the refractory period, it is possible to limit the maximum value or to output a constant frequency.

The potential of the capacitor, and thus the amount of charge stored, is
It is temporally controlled by a reference voltage control circuit (time window weight function generation circuit) 204. Reflecting this control characteristic,
This is weighted addition within a time window to be described later for the input pulse (see FIG. 7). This reference voltage control circuit 204 is based on an input timing from a pacemaker neuron to be described later (or a mutual coupling input with a neuron in a subsequent layer).
A reference voltage signal (corresponding to the weight function of FIG. 7B) is generated.

In some embodiments, the input of suppressiveness is not always necessary. However, the divergence (saturation) of the output is prevented by making the input from the pacemaker neuron to the feature detection layer neuron to be suppressive as described later. be able to.

In general, the relationship between the sum of the input signals and the output level (pulse phase, pulse frequency, pulse width, etc.) changes depending on the sensitivity characteristics of the neuron. Can be changed by the top-down input of. In the following, for convenience of explanation, it is assumed that the circuit parameters are set such that the frequency of the pulse output corresponding to the total value of the input signal rises sharply (thus, almost binary in the frequency domain), and the output is performed by pulse phase modulation. It is assumed that the level (eg, timing at which phase modulation is applied) fluctuates.

As a pulse phase modulating section, a circuit as shown in FIG. 5 to be described later may be added and used. As a result, the weight function within the time window is controlled by the reference voltage. As a result, the phase of the pulse output from this neuron changes, and this phase can be used as the output level of the neuron.

A known circuit configuration may be used in which, when the sum of inputs obtained by using a window function or the like exceeds a threshold value, an oscillation output is output with a predetermined timing delay.

The configuration of the neuron element is a neuron belonging to the feature detection layer 102 or the feature integration layer 103. When the firing pattern is controlled based on the output timing of a pacemaker neuron described later, a pulse from the pacemaker neuron is used. After receiving the output, the circuit configuration may be such that the neuron outputs a pulse with a phase delay according to the input level (simple or weighted sum of the above input) received from the receptive field of the preceding layer. . In this case, before the pulse signal is input from the pacemaker neuron, there is a transitional transition state in which the neurons output pulses at random phases with respect to each other according to the input level.

When a pacemaker neuron is not used, as described later, between the neurons (the feature detection layer 10
Based on the synchronizing firing signal caused by the mutual coupling between (2 and the feature integration layer 103) and the network dynamics, the firing timing of the output pulse of the feature detecting neuron according to the input level as described above is controlled. The circuit configuration may be simple.

The neurons of the feature detection layer 102 have a receptive field structure according to the feature category as described above, and input pulse signals (current values) from the neurons of the preceding layer (the input layer 101 or the feature integration layer 103). Or a potential), a non-decreasing and non-linear function that asymptotically saturates to a certain level, such as a sigmoid function, according to the sum when the weighted sum (described later) of the time window function exceeds a threshold value. That is, pulse output is performed at an output level that takes a so-called squashing function value (in this case, the output is given by a phase change, but may be a change based on frequency, amplitude, and pulse width).

The feature detection layer (1, 0) detects a pattern structure (lower-order feature) having a predetermined spatial frequency in a local area of a certain size and having a vertical directional component. Assuming that there is a neuron N1, if a corresponding structure exists in the receptive field of the neuron N1 on the data input layer 101, a pulse is output with a phase corresponding to the saliency (contrast). Such a function can be realized by a Gabor filter. Hereinafter, a feature detection filter function performed by each neuron of the feature detection layer (1, 0) will be described.

In the feature detection layer (1, 0), multi-scale,
It is assumed that Gabor wavelet transform represented by a filter set of multi-directional components is performed, and each neuron (or each group including a plurality of neurons) in a layer has a predetermined G value.
It has an abor filter function.

The Gabor wavelet is given by the following equation (1)
Has a shape obtained by modulating a sine wave having a constant directional component and a spatial frequency with a Gaussian function, as given by
It is specified by the index m of the scaling level and the index n of the direction component. As a wavelet, this set of filters has similar functional shapes and different main directions and sizes. This wavelet is that the function form is localized in the spatial frequency domain and the real space domain, the simultaneous uncertainty regarding the position and the spatial frequency is minimized, and it is the function that is most localized in the real space and the frequency space Known (J, G. Daugman (1985), Unce
rtainty relation for resolution in space, spatial
frequency, and orientation optimized by two-dimens
ional visual cortical filters, Journal of Optical
Society of America A, vol.2, pp. 1160-1169).

[0054]

[Outside 1]

Here, (x, y) is a position in the image, a is a scaling factor, θ _n is a directional component of the filter, and W
Is the fundamental spatial frequency, σ _x and σ _y are the x directions of the filter function,
It is a parameter that gives the size of the spread in the y direction. In the present embodiment, θ _n is 0 degree, 30 degrees, 60 degrees, 9 in six directions.
Taking values of 0 degrees, 120 degrees, and 150 degrees, a is given as 2 and m is given as an integer taking a value from 1 to 4.

The parameters σ _x , which determine the characteristics of the filter,
σ _y and a are desirably set such that there is no bias (sensitivity) to a specific spatial frequency and direction by appropriately and homogeneously overlapping each other in the Fourier domain. Therefore, for example, if the half-value levels for the amplitude maximum value after the Fourier transform are designed to be in contact with each other in the Fourier domain,

[0057]

[Outside 2] Becomes Here, U _H and U _L are the maximum and minimum values of the spatial frequency band covered by the wavelet transform, and M gives the number of scaling levels in that range.

Each of the filters g _mn (x, y) and the input grayscale image
The Gabor wavelet transform is performed by performing the dimensional convolution operation. That is,

[0059]

[Outside 3]

Here, I is an input image, and _Wmn is a Gabor wavelet transform coefficient. A set of W _mn (m = 1, .., 4; n = 1, ..., 6) is obtained at each point as a feature vector. ' ^* ' ^Indicates complex conjugate. Thus, the feature detection layer (1,
Each neuron of (0) has a receptive field structure corresponding to gmn, and has an output level (here, a non-linear squashing function of a wavelet transform coefficient value obtained by performing a product-sum input of a distribution weight coefficient and image data). In the embodiment, the pulse is output based on the phase, but may be based on the frequency, amplitude, and pulse width. As a result, the Gabor wavelet transform of Expression (4) is performed as the output of the entire layer (1, 0).

On the other hand, the subsequent feature detection layers ((1, 1), (1,
Each of the neurons 2),..., (1, n)) is different from the above-described detection layer and forms a receptive field structure for detecting a characteristic unique to a pattern to be recognized by a so-called Hebb learning rule or the like. The size of the local region where the feature detection is performed gradually becomes closer to the size of the entire recognition target in a later layer, and a medium-order or higher-order feature is geometrically detected. For example, in the case of performing face detection and recognition, the middle-order (or higher-order) features represent features at the level of graphic elements such as eyes, nose, and mouth that constitute the face. The feature detection layer neuron may have a mechanism for receiving a coupling of shunting inhibition (shunting inhibition) based on the output of the preceding layer in order to stabilize the output.

Next, the feature integration layer 103 ((2,0), (2,
The neurons 1),... Will be described. As shown in FIG. 1, the feature detection layer 102 (for example, (1, 0)) to the feature integration layer 10
The connection to 3 (for example, (2,0)) is configured to receive an excitatory connection input from a neuron of the same feature element (type) in the preceding feature detection layer in the receptive field of the feature integration neuron, Each neuron in the integrated layer performs sub-sampling by local averaging for each feature category (calculation of an average value of inputs from neurons forming a receptive field of a feature detection neuron, etc.).

In other words, in the feature integration, a plurality of pulses of the same type of feature are input, integrated in a local area (receptive field) and averaged, so that the position of the feature is fluctuated and deformed. Can be reliably detected. For this reason, the receptive field structure of the feature integration layer neuron is uniform regardless of the feature category (for example, each is a rectangular region of a predetermined size, and the sensitivity or the weight coefficient is uniformly distributed therein). You may comprise so that it may become.

Next, a method of pulse encoding and detecting a two-dimensional graphic pattern will be described. FIG. 3 shows the feature integration layer 103.
FIG. 3 is a diagram schematically showing a state of propagation of a pulse signal from a to a feature detection layer 102 (for example, from a layer (2,0) to a layer (1,1) in FIG. 1).

Each neuron n _i (n _{1 to} n) on the feature integration layer 103 side
₄ ) correspond to different feature amounts (or feature elements), respectively, and the neuron n ′ _j on the feature detection layer 102 side uses higher-order features (graphics) obtained by combining the features in the same receptive field. Element).

The connection between the neurons is performed when the pulse is propagated.
Between and neuron n_iFrom neuron n ' _jSynaptic connection to
(S_ij) Delay due to time delays etc.
Delay, and as a result, via the common bus line 301
The neuron n '_jPulse train P arriving at_iIs the feature integration layer 1
Learning as long as pulse output is made from each neuron in 03
The amount of delay in synaptic connections determined by
(In FIG. 3A,
P_Four, P_Three, P_Two, P₁In order of arrival).

FIG. 3B shows the case where the time window synchronization control is performed using the timing signal from the pacemaker neuron, and the feature integrated cell n ₁ ,
Pulse propagation from n ₂ , n ₃ (representing different types of features) to a certain feature detection cell (n ' _j ) (which performs higher-order feature detection) on layer number (1, k + 1) The timing etc. are shown.

FIG. 6 is a diagram showing a network configuration when the feature detection layer neuron has an input from a pacemaker neuron. In FIG. 6, pacemaker neurons 603 (n _p ) form feature receptive neurons 602 (n _j , _nk etc.) that form the same receptive field and detect different types of features.
, And form the same receptive field as those, and receive excitatory connections from the neurons 601 on the feature integration layer (or input layer). Then, at a predetermined timing (or frequency) determined by the sum of the inputs (or to control the state of the receptive field as a whole, such as an average activity level of the entire receptive field). Pulse output is performed to the feature detection neuron 602 and the feature integration neuron.

Each feature detection neuron 602 is configured such that its time windows are phase-locked with each other using the input as a trigger signal. However, as described above, before the pacemaker neuron input, the phase is not locked. Each neuron outputs a pulse with a random phase. In the feature detection neuron 602, a pacemaker neuron 603 is used.
Before the input from, the time window integration described later is not performed.
The same integration is performed using a pulse input from the pacemaker neuron 603 as a trigger.

Here, the time window is determined for each feature detection cell (n ′ _i ), and is common to each neuron in the feature integration layer that forms the same receptive field for the cell and the pacemaker neuron 603, Gives the time range for time window integration.

The pacemaker neuron 603 at the layer number (1, k) (k is an integer of 0 or more) outputs the pulse output to the layer number (1, k).
By outputting to each feature-integrated cell of (2, k-1) and the feature detection cell (layer number (1, k)) to which the pacemaker neuron 603 belongs, the feature detection cell adds the input temporally. Is given. The start time of this time window is a reference time for determining the arrival time of the pulse output from each integrated cell. That is, the pacemaker neuron 603 outputs the pulse output time from the feature integrated cell,
And a reference pulse for time window integration in the feature detection cells.

Each feature integrated cell (n ₁ , n ₂ , n ₃ ) of layer number (2, k)
Receives the pulse input from the pacemaker neuron of the feature detection layer of layer number (1, k + 1), and the input from the preceding feature detection layer or sensor input layer (layer number (1, k)) is sufficient. If the action potential is rising to a level (for example,
In a certain time range or time window, the average number of input pulses is larger than a threshold value), the output is made at the time of the falling edge of the pulse from the pacemaker.

In the present embodiment, the feature-integrated cell is not configured to receive timing control from the pacemaker neuron on the feature detection layer of the preceding layer number (1, k). This is because, in a feature-integrated cell, not the arrival time pattern of the input pulse, but rather the input level within a certain time range (such as the total time of the input pulse).
This is because the timing at which the time window is generated is not so important because the pulse is output at a phase determined by the above (which may depend on any of the frequency, the pulse width, and the amplitude, but the phase is used in the present embodiment). Note that this is not intended to exclude a configuration in which the feature-integrated cells receive timing control from the pacemaker neuron in the preceding feature detection layer, and it goes without saying that such a configuration is also possible.

Each pulse is given a predetermined amount of phase delay when passing through the synapse circuit, and reaches the feature detecting cell through a signal transmission line such as a common bus line. The arrangement of the pulses on the time axis at this time is indicated by the pulses (P ₁ , P ₂ , P ₃ ) indicated by the dotted lines on the time axis of the feature detection cells.

Each pulse (P ₁ ,
P ₂ , P ₃ ) time window integration (usually, one integration; however, charge accumulation by multiple time window integrations or averaging of multiple time window integrations may be performed. If the result is larger than the threshold value, a pulse output (P _d ) is made based on the end time of the time window. The learning time window shown in FIG. 3B is referred to when a learning rule described later is executed.

[0076] Figure 4 is a diagram showing the structure of the synapse circuit S _i. Figure (A) in 4, the synapse circuit 202 (S _i), neuron n each subcircuit give a binding target synaptic to each neuron n _'j (phase delay) of the _i 401
Indicates that they are arranged in a matrix.
In this way, wiring from the synapse circuit to the connection destination neuron can be performed on the same line (local common bus 301) corresponding to each receptive field (wiring between neurons can be performed virtually). ), It is possible to reduce (eliminate) the wiring problem which has conventionally been a problem.

Further, when a plurality of pulse inputs from the same receptive field are received, the connection destination neuron determines which neuron has emitted each pulse from the arrival time of the pulse with respect to the time window (the characteristic detection cell is Corresponding to the feature to be detected,
It can be identified on the time axis by the phase delay inherent to the low-order features that constitute it.

As shown in FIG. 4B, each synapse coupling small circuit 401 includes a learning circuit 402 and a phase delay circuit 403. The learning circuit 402 adjusts the delay amount by changing the characteristics of the phase delay circuit 403, and also stores the characteristic value (or its control value) on the floating gate element or on a capacitor coupled to the floating gate element. It is something to memorize.

FIG. 5 is a diagram showing a detailed configuration of a synapse connection small circuit. The phase delay circuit 403 is a pulse phase modulation circuit. For example, as shown in FIG. 5A, monostable multivibrators 506 and 507, resistors 501 and 504, capacitors
503, 505 and a transistor 502 can be used. FIG. 5B shows a square wave P1 ([1] in FIG. 5B) input to the monostable multivibrator 506, and a square wave P2 (the same [2]) output from the monostable multivibrator 506. ), The respective timings of the square wave P3 (the same [3]) output from the monostable multivibrator 507.

The details of the operation mechanism of the phase delay circuit 403 are omitted, but the pulse width of P1 is determined by the time until the voltage of the capacitor 503 by the charging current reaches a predetermined threshold, and the width of P2 Is determined by the time constant of the resistor 504 and the capacitor 505. When the pulse width of P2 is
If the falling point shifts later, the rising point of P3 also shifts by the same amount, but the pulse width of P3 does not change. As a result, only the phase of the input pulse is changed. Is modulated and output.

A refresh circuit that uses the control voltage Ec as a reference voltage
The pulse phase (delay amount) can be controlled by changing the learning circuit 402 that controls the amount of charge stored in the capacitor 508 that applies the connection weight to the capacitor 509. In order to maintain the connection weight for a long period of time, after the learning operation, as a charge of a floating gate element (not shown) added to the outside of the circuit of FIG. The coupling load may be stored. Other configurations designed to reduce the circuit scale (for example, see Japanese Patent Application Laid-Open No. 5-37317)
A publicly known circuit configuration such as that described in Japanese Patent Application Laid-Open No. H10-327054 can be used.

In the case where the network takes a configuration of a covalent connection form of connection weight (one weight coefficient represents a synapse connection), the delay amount at each synapse (P _ij of the following equation (5)) ) Can be uniform within the same receptive field. In particular, the connection from the feature detection layer 102 to the feature integration layer 103 is performed when the feature integration layer 103 is the preceding layer.
Since it is involved in sub-sampling by local averaging of the output and the like, it can be configured in this way regardless of the detection target (that is, regardless of the problem).

In this case, as shown in FIG. 4B, each small circuit in FIG. 4A is a single circuit S _{k, i} connected by the local common bus line 401. It becomes an economical circuit configuration. On the other hand, the feature integration layer 103 (or the sensor input layer 1
When the connection from (01) to the feature detection layer 102 is made like this, the feature detection neuron detects an event called simultaneous arrival (or almost simultaneous arrival) of pulses representing a plurality of different feature elements. is there.

When the connections have symmetry, the connections giving the same load (phase delay) are represented by the same small synapse coupling circuit, so that a considerable number of synapse connections are represented by a small number of circuits. It can be configured to be. Especially in the detection of geometric features, since the distribution of the connection load in the receptive field often has symmetry,
The number of synapse connection circuits can be reduced, and the circuit scale can be significantly reduced.

FIG. 5 shows an example of a learning circuit at a synapse that achieves simultaneous arrival of a pulse or a predetermined amount of phase modulation.
What has a circuit element as shown in FIG. That is, the learning circuit 402 is connected to the pulse propagation time measurement circuit 510.
(Here, the propagation time refers to a time difference between a pulse output time at a pre-synapse of a neuron of a certain layer and an arrival time of the pulse at an output destination neuron on the next layer), a time window generating circuit 511. , And a pulse phase modulation amount adjustment circuit 512 that adjusts the pulse phase modulation amount at the synapse so that the propagation time becomes a constant value.

The propagation time measuring circuit 510 receives a clock pulse from a pacemaker neuron that forms the same local receptive field as described later, and sets a predetermined time width (time window: see FIG. 3B). A configuration in which the propagation time is obtained based on the output of the clock pulse from the counter circuit is used. By setting the time window based on the firing time of the output destination neuron, an extended Hebb's learning rule as described below is applied.

The learning circuit 402 may make the time window narrower as the frequency with which objects of the same category are presented increases. In this way, the more familiar the pattern (ie, the number of times of presentation and the number of times of learning) are, the closer to the detection mode of simultaneous arrival of a plurality of pulses (coincidence detection). By doing so, the time required for feature detection can be reduced (the operation of instantaneous detection becomes possible) .However, it is possible to perform detailed comparative analysis of the spatial arrangement of feature elements, identification between similar patterns, etc. Becomes unsuitable.

The learning process of the delay amount is extended, for example, to the complex number domain, so that the neuron n _i of the feature detection layer is obtained.
Connection weight C _ij between the feature integration layer neuron n _j
Is given as C _ij = S _ij exp (iP _ij ) (5). Here, S _ij is the coupling strength, P _ij is the phase, and i before it is a pure imaginary number, and is a phase corresponding to the time delay of the pulse signal output from the neuron j to the neuron i at a predetermined frequency. S _ij reflects the receptive field structure of neuron i and generally has a different structure depending on the target to be recognized and detected. This is formed separately by learning (supervised learning or self-organization) or is formed as a predetermined structure.

On the other hand, a learning rule for self-organization regarding the amount of delay is as follows:

[0090]

[Outside 4] Given by However,

[0091]

[Outside 5] Is the time derivative of C, τ _ij is the above time delay (preset amount), and β (β1) is a constant.

[0092] Solving the above equation, C _ij converges to βexp (-2πiτ _ij), hence, P _ij converges to-tau _ij. An example of the application of the learning rule will be described with reference to the time window at the time of learning shown in FIG. 3 (B). The anterior neuron (n1, n2, n
The connection weight is updated according to equation (6) only when both 3) and the posterior neuron (feature detection cell) are firing within the time range of the learning time window. In FIG. 3B, the feature detection cells are fired after the elapse of the time window, but may be fired before the elapse of the time window in FIG.

As a learning rule, another method may be used. Also, by introducing the principle of competitive learning, the pulses may arrive at a predetermined interval or more apart from each other (the difference in time delay becomes a predetermined value or more).

As described above, each feature detection layer 102 receives a pulse signal relating to a plurality of different features from the same receptive field and performs spatio-temporal weighted summation (sum of weights) and threshold processing. Pulses corresponding to each feature amount arrive at predetermined time intervals according to a delay amount (phase) determined in advance by learning.

The learning control of the pulse arrival time pattern is not the focus of the present application and will not be described in detail. For example, as the characteristic element constituting a certain figure pattern is the most remarkable feature that contributes most to the detection of the figure. Introducing competitive learning is such that feature elements arriving first and having such similar saliency levels arrive at a certain amount of time apart from each other. Alternatively, it arrives at different time intervals between predetermined feature elements (feature elements constituting a recognition target and considered to be particularly important: for example, a feature having a large average curvature, a feature having a high linearity, and the like). It may be designed as follows.

The saliency corresponds to the reaction intensity (here, the amount of pulse delay) of the detected cell of the characteristic element when the figure is detected. In this case, the neurons corresponding to each lower-order feature element in the same receptive field on the feature integration layer, which is the preceding layer, fire synchronously (pulse output) at a predetermined phase.
Will do. In the case where the same neuron of the feature integration layer 103 has a connection to a feature detection neuron that detects the same higher-order feature at a different position (in this case, the same higher-order feature is formed although the receptive field is different) Has a bond)
Needless to say, as long as the same neuron fires synchronously.

However, the output level (here, the phase is used as a reference, but the frequency, amplitude, and pulse width may be used as a reference) may be the sum (or average, etc.) of contributions from a plurality of pacemaker neurons given for each receptive field. ). In each neuron on the feature detection layer 102, the calculation of the spatiotemporally weighted sum of the input pulses (the sum of the weights) is performed only in a time window of a predetermined width for the pulse train arriving at the neuron. The means for realizing the weighted addition within the time window is not limited to the neuron element circuit shown in FIG. 2, and it goes without saying that it may be realized by other methods.

This time window corresponds to the refractory period of the actual neuron.
(refractory period) It corresponds to the time zone to some extent. In other words, there is no output from the neuron regardless of any input during the refractory period (time range other than the time window), but the firing according to the input level occurs in the time window other than that time range. Is similar to the neuron.

The refractory period shown in FIG. 3B is a time period from immediately after the firing of the feature detection cells to the start of the next time window. Needless to say, the length of the refractory period and the width of the time window can be arbitrarily set, and the refractory period need not be shorter than the time window as shown in FIG. Even without using a pacemaker neuron, the start time of the time window is determined by the mechanism of synchronous firing between the neurons in the feature detection layer and the feature integration layer due to weak mutual coupling between neurons and predetermined coupling conditions (EMIzhikevich, 1999 'Weakly Pulse-Coupled
Oscillation, FM Interactions, Synchronization, an
d OscillatoryAssociative Memory 'IEEE Trans.on Ne
ural Networks, vol.10. pp.508-526.), these neurons are identical. It is known that the synchronous firing is generally caused by a mutual coupling and a pull-in phenomenon between neurons.

Therefore, also in the present embodiment, such an effect can be obtained without a pacemaker neuron, by configuring so as to satisfy the weak mutual connection between neurons and a predetermined synaptic connection condition.

In the present embodiment, as schematically shown in FIG. 6, as a mechanism already described, for example, a pacemaker neuron (for a fixed frequency) which receives an input from the same receptive field for each feature detection layer neuron. By inputting timing information (clock pulse) based on (pulse output), the above-described common start timing may be provided.

In the case of such a configuration, it is not necessary to perform time window synchronization control over the entire network (if it is necessary), and there is fluctuation or fluctuation of clock pulses as described above. Is also uniformly affected by the output from the local receptive field (the fluctuation of the position of the window function on the time axis is the same between neurons forming the same receptive field). The reliability of detection does not degrade. Since a highly reliable synchronous operation is enabled by such local circuit control, the tolerance of variation regarding circuit element parameters is also increased.

Hereinafter, for the sake of simplicity, a feature detection neuron that detects a triangle as a feature will be described. The feature integration layer 103 at the preceding stage has L-shaped patterns (f ₁₁ , f ₁₂ ,...) Having various orientations as shown in FIG. 7C, and continuity (connectivity) with the L-shaped pattern. Those that respond to graphical features (feature elements) such as a combination pattern (f ₂₁ , f ₂₂ ,...) Of the line segments, and a combination (f ₃₁ ,. I do.

Also, f in FIG.₄₁, f₄₂, f₄₃Has a different orientation
Feature that constitutes a triangle ₁₁, f₁₂, f₁₃Compatible with
The following shows the features to be performed. New layer connection by learning
The characteristic delay amount is set between the
In the sign detection neuron, the time window is divided
Sub time window (time slot) (w₁, w_Two,…)smell
Corresponding to the main and different features that make up the triangle.
The setting is made in advance so that Luz arrives.

For example, w ₁ , w ₂ ,
..., the w _n, as shown in FIG. 7 (A), pulses corresponding to the combination of a set of features as constituting a triangle as a whole arrive first. Here, the L-shaped pattern (f ₁₁ ,
f ₁₂ , f ₁₃ ) arrive within w ₁ , w ₂ , w ₃ respectively, and the feature element
pulse corresponding to _{(f 21, f 22, f} 23) , respectively w _1, w _2, w ₃
The amount of delay is set by learning so as to arrive within.

The pulses corresponding to the characteristic elements (f ₃₁ , f ₃₂ , f ₃₃ ) arrive in the same order. In the case of FIG. 7A, a pulse corresponding to one characteristic element arrives in one sub time window (time slot). The meaning of dividing into sub-time windows is that, by performing pulse detection (detection of characteristic elements) corresponding to different characteristic elements developed and expressed on the time axis in each sub-time window individually and reliably, those characteristics are obtained. The purpose of the present invention is to increase the changeability and adaptability of the processing mode, such as whether the conditions of all feature elements are to be detected or a certain percentage of features are to be detected. . For example, in a situation where the recognition (detection) target is a face, and the search (detection) of the eyes, which is a part of the recognition (detection), is important (when the priority of the eye pattern detection is to be set high in the visual search) By introducing a feedback coupling from a higher-order feature detection layer, the reaction selectivity (detection sensitivity of a specific feature) corresponding to a feature element pattern constituting an eye can be selectively enhanced. By doing so, it is possible to give higher importance to lower-order feature elements constituting higher-order feature elements (patterns) and detect them.

Further, if it is set in advance so that a pulse arrives in a sub-time window earlier as important features,
By making the weighting function value in the sub-time window larger than the value in the other sub-time windows, it is possible to make a feature with higher importance easier to be detected. The importance (detection priority between features) can be obtained by learning or can be defined in advance.

Therefore, if it is only necessary to detect a certain percentage of characteristic elements, the division into sub-time windows is almost meaningless, and may be performed in one time window. Note that pulses corresponding to a plurality (three) of different characteristic elements may arrive and be added.
(See FIG. 7D). That is, it may be assumed that a pulse corresponding to a plurality of characteristic elements ((D) in FIG. 7) or an arbitrary number of characteristic elements is input to one sub-time window (time slot). In this case, in FIG. 7 (D), in the first sub-time window, pulses corresponding to the other feature elements f ₂₁ and f ₂₃ that support the detection of the apex portion f ₁₁ of the triangle arrive, and similarly, 2 In the second sub-time window, pulses of other feature elements f ₂₂ and f ₃₁ arrive to support the detection of the apex angle portion f ₁₂ .

The number of divisions into sub-time windows (time slots), the width of each sub-time window (time slot), the class of the feature, and the assignment of the pulse time interval corresponding to the feature are not limited to those described above. Needless to say, it can be changed. For example, a sub time window corresponding to a feature element such as 'X' or '+' may be set in addition to the above-described feature element. Such feature elements can be said to be redundant (or unnecessary) in the detection of a figure of a triangle, but conversely, the detection accuracy of a figure pattern of a triangle can be increased by detecting that they do not exist.

Further, even when a deformation not represented by the combination of these characteristic elements is applied (for example, when a rotation within a certain range is given), the output of the neuron of the characteristic integration layer representing the above-mentioned characteristic element is obtained. The pulse responds with a continuous phase delay (delay amount: a range in which the pulse arrives in a predetermined sub-time window (time slot)) according to the degree of deviation from the ideal pattern (so-called gr).
For aceful degradation, the output is stabilized so that the allowable range for the deformation of the detected graphic feature becomes a certain level or more. For example, a feature f shown in FIG.
_11, f _12, than f ₁₃ and triangle (Q1) which is formed by the features corresponding to f _41, f _42, triangular and (Q2) which is formed by the features corresponding to f _43, at least the direction are different from each other It should be.

In this case, detection (integration) corresponding to each feature
When cells are present, for a triangle (Q3) corresponding to an intermediate orientation of the two _{triangles, f 11, f 12, f} 13 corresponding to the detection (integration) cells and f _41, f _42, f ₄₃ The detection (cells) corresponding to the above are all lower than the maximum response output, and are directly at the output level according to the convolution operation value with the filter kernel as the receptive field structure determined according to the type of feature. If the vector quantity as an output from the cell is integrated as a value unique to the intermediate figure, it is possible to detect an intermediate figure (when rotation is applied) in the state of two triangles.

For example, qualitatively, the rotation angle is small,
F closer to Q1₁₁, F₁₂, F₁₃From cells corresponding to
The output is relatively large, and conversely, the closer it is to Q2, the more f₄₁,
f₄₂, F ₄₃The output from the cell corresponding to is increased.

Next, the calculation of the spatiotemporally weighted sum of input pulses (sum of weights) will be described. As shown in FIG. 7B, in each neuron, the weighted sum of the input pulses is calculated by a predetermined weighting function (for example, Gaussian) for each of the sub time windows (time slots), and the sum of the weighted sums is a threshold value. Be compared. τ _j represents the center position of the weighting function of the sub time window j, and is based on the start time of the time window (elapsed time from the start time)
Expressed by In general, the weighting function may be a function of a distance (deviation on a time axis) from a predetermined center position (representing a pulse arrival time when a feature to be detected is detected). It may be a function.

When the weight function is the above-described distance function, assuming that the center position τ of the weight function of each sub-time window (time slot) of the neuron is a time delay after learning between neurons, A neural network that performs weighted summation (sum of weights) can be considered as a kind of time-axis domain Radial Basis Function Network (hereinafter abbreviated as RBF). The time window F _Ti of the neuron n _i using the Gaussian function as the weight function is represented by σ for the spread of each sub time window and b _ij for the coefficient factor.

[0115]

[Outside 6] Becomes

The weight function may take a negative value. For example, when a neuron of a certain feature detection layer is to finally detect a triangle, a feature (F _faulse ) that is clearly not a component of the figure pattern (for example, “X”, '+' Etc.)
Is detected, the detection output of the triangle is not finally made even if the contribution from other feature elements is large,
In the input sum calculation processing, the characteristic (F _faulse )
Can be given a weighting function that gives a negative contribution and a coupling from the feature detection (integrated) cells.

[0117] space sum when the input signal to the neuron n _i of the feature detection layer X _i (t) is

[0118]

[Outside 7] Can be expressed as Here, ε _j is the initial phase of the output pulse from the neuron n _j , and converges to 0 by synchronous firing with the neuron n _i , or changes the phase of the time window by the timing pulse input from the pacemaker neuron. When forcibly synchronizing to 0, ε _j may always be 0. When the pulse input of FIG. 7A and the weighted sum by the weight function shown in FIG. 7B are executed, a temporal transition of the weighted sum value as shown in FIG. 7E is obtained. The feature detection neuron outputs a pulse when the weighted sum reaches a threshold value (Vt).

[0119] The output pulse signal from the neuron n _i, as described above, with the spatio-temporal sum squashing nonlinear function to become an output level and time delay given by learning (the so-called total input sum) of the input signal (phase) Output to neurons in the upper layer (pulse output is fixed frequency (binary), and phase modulation amount which becomes a squashing nonlinear function for spatio-temporal sum of input signal is added to phase corresponding to fixed delay amount determined by learning. Output).

FIG. 8 is a flowchart showing the processing procedure of each layer described above. The flow of processing from low-order feature detection to high-order feature detection is summarized as shown in FIG.
First, in step S801, low-order feature detection (for example, calculation of a Gabor wavelet transform coefficient at each position) is performed. Next, in step S802, low-order feature integration processing for performing local averaging of those features and the like is performed. Further, steps S803 to 80
In step S4, detection and integration of secondary features are performed. In steps S805 to S806, detection and integration of high-order features are performed. Then, in step S807, the presence or absence of a recognition (detection) target or the detection position output is performed as the output of the final layer. Steps S803-804 and S805-806
Can be arbitrarily set or changed according to the task (such as a recognition target).

FIG. 9 is a flowchart showing the processing procedure of each feature detection neuron 602. First, step S9
In 01, a pulse corresponding to a plurality of feature categories is applied to the same receptive field 10 in the input layer 101 or the feature integration layer 103 as the preceding layer.
Step S9: receiving input from the neuron 601 forming 5
In step 02, a time window and a weighting function are generated based on a local synchronization signal input from the pacemaker neuron 603 (or obtained by interaction with a preceding layer neuron). The sum of the weights by the function is obtained, and it is determined in step S904 whether or not the threshold has been reached.
At 05, pulse output is performed. Although steps S902 and S903 are shown in time series, they are actually performed almost simultaneously.

The processing procedure of each feature integration neuron is as shown in the flowchart of FIG. That is, in step S1001, a pulse is input from the feature detection processing module 104 in the same category, which is a feature detection neuron that forms a local receptive field specific to the neuron. Input pulses are added in a time range other than the initial period.
In step S1003, a determination is made as to whether the total value of the input pulses (measured on the basis of potential, for example) has reached a threshold value. If the threshold value has been reached, in step S1004, a pulse is output at a phase corresponding to the total value. do.

Since the input pulse corresponds to the feature at each position in the spatial domain (or the spatial arrangement relationship of the feature elements), it is possible to form a spatiotemporal RBF.

More specifically, by weighting each neuron output value and performing addition, a sufficient number of a predetermined set of feature elements (feature detection cells) and a sufficient number of sub-time values are obtained. A spatio-temporal function of a pulse pattern corresponding to an arbitrary figure pattern can be expressed by calculating a weighted sum (load sum) in a window (time slot). If the category of recognition symmetry and its shape change are limited to some extent, necessary feature detection cells and sub-time windows
(Time slots) can be reduced.

In the present embodiment, the common bus is a local bus line which is assigned to the same receptive field. However, the present invention is not limited to this, and the interlayer connection from one layer to the next layer is the same bus line. , The pulse phase delay amount may be divided and set on the time axis. Further, a common bus line may be used between adjacent receptive fields having a relatively large overlapping ratio.

It is to be noted that the processing (or the threshold processing) is performed so that the result of the weighted product-sum operation within each sub-time window (time slot) becomes a nonlinear squashing function value without using the above-mentioned spatiotemporal RBF. ) And take the product of them. For example, with a circuit configuration (not shown), a threshold processing result (binary) is obtained for each sub time window, stored in the temporary storage unit, and a logical product of the threshold processing results sequentially obtained is obtained in a time-series manner. do it.

When the product is obtained by performing the threshold processing, it goes without saying that the tolerance of feature detection under a pattern loss or low contrast condition is reduced.

The above-described processing (detection of a graphic pattern by spatiotemporal RBF) can be realized as an operation similar to the associative memory recall process. That is, even if a defect of a low-order (or medium-order) feature element to be detected in a certain local area (or entire area) occurs, some other feature elements are detected, and the sum value (8) is calculated. If the threshold is exceeded,
For the entire spatiotemporal RBF network, detection of medium-order (or higher-order) feature elements (firing of the corresponding neurons) can be performed.

Note that the configuration of the network is as shown in FIG.
It is needless to say that the present invention is not limited to the above, and may be MLP or the like as long as the configuration includes a layer for detecting a predetermined geometric feature element.

Next, FIG. 11 shows a case in which focusing on a specific subject, color correction of the specific subject, and exposure control are performed by mounting the pattern detection (recognition) device according to the configuration of the present embodiment on the imaging device. It will be described with reference to FIG. FIG. 11 is a diagram illustrating a configuration of an example in which the pattern detection (recognition) device according to the embodiment is used for an imaging device.

An imaging apparatus 1101 shown in FIG. 11 includes an imaging optical system 1102 including a photographing lens and a driving control mechanism for zoom photographing,
D or CMOS image sensor 1103, imaging parameter measurement unit 1104, video signal processing circuit 1105, storage unit 1106, control signal generation unit 1107 that generates control signals such as control of imaging operation, control of imaging conditions, and viewfinders such as EVF And a recording medium 1110, and the above-described pattern detection device is provided as a subject detection (recognition) device 1111.

The image pickup apparatus 1101 detects, for example, a face image of a person registered in advance (detection of the position and size of a person) from a captured image by using a subject detection (recognition) apparatus 1111. When the position and size information of the person is input from the subject detection (recognition) device 1111 to the control signal generation unit 1107, the control signal generation unit 1107
Focus control for that person based on the output from 04,
A control signal for optimally controlling exposure condition control, white balance control, and the like is generated.

As a result of using the above-described pattern detection (recognition) apparatus in an image pickup apparatus, it is possible to perform low-power consumption, high-speed (real-time) optimal control of person detection and photographing based on the person detection. Become.

(Second Embodiment) In the first embodiment, the synapse is configured to perform phase modulation. However, in the present embodiment, the pulse width (analog value) is controlled under the network configuration shown in FIG. ) Recognition of a graphic pattern or the like is performed by a network composed of a synapse element performing a modulation operation and an integrate-and-fire neuron as described in the first embodiment.

[0135] modulation by synaptic each pulse width and the pulse width of the postsynaptic the presynaptic signal, W _b, is given by W _a = S _ij W _b when the W _a. Here, S _ij has the same meaning as the coupling strength (Equation (5)) in the first embodiment. In order to increase the dynamic range of the modulation, it is necessary to make the basic pulse width of the pulse signal sufficiently smaller than the period (basic pulse interval).

The firing (pulse output) of the neuron occurs when the potential exceeds a predetermined threshold value due to accumulation of electric charges accompanying the inflow of a plurality of pulse currents representing predetermined characteristic elements.
In the present embodiment, weighted addition of arrival pulses for each sub time window (see Embodiment 1) is not particularly required, but integration is performed in a time window having a predetermined width. In this case, the feature element (figure pattern) to be detected depends only on the temporal sum of the signals input to the feature detection layer neurons (sum of the pulse current values). Further, the width of the input pulse corresponds to the value of the weight function in the first embodiment. In addition, when performing the addition with weighting of the arrival pulse for each sub time window, information such as the time interval of the arrival pulse, the presence / absence of a feature detected in the arrival time pattern itself, deformation, etc. is displayed as in the first embodiment. I have.

Here, the output characteristic of the feature detection layer neuron may be such that the greater the saliency of the corresponding feature (see Embodiment 1), the greater the frequency, or the pulse width or amplitude. It may have great properties.

FIG. 12A shows a structural unit of a synapse element 401 used in this embodiment. A learning circuit 402 similar to that of the first embodiment and a pulse width modulation circuit 1201 are provided. The latter is a well-known circuit (for example, see Patent No. 27176 by the present applicant).
62 publication) can be used.

In addition, instead of the above-described pulse width modulation, the modulation of the pulse frequency may be performed by each synapse element. In this case, the configuration of the synapse element 401 corresponding to the above configuration includes a learning circuit 402 and a pulse frequency modulation circuit 1202 as shown in FIG. As the pulse frequency modulation circuit 1202, a known configuration can be used. In addition, the modulation by the synapse is assuming that the pulse frequency before the synapse and the pulse frequency after the synapse are f _b and f _a , respectively.
f _a = S _ij f _b .

When the neuron output is modulated by frequency modulation or pulse width modulation, it goes without saying that the pulse modulation circuit or the like is provided with a control mechanism for maximizing the pulse width and saturating it to the upper limit (pulse period). No.

(Third Embodiment) The feature detecting neuron used in the present embodiment detects a graphic feature and its deformation amount based on a pulse arrival time interval (analog value) and an order, or a degree of importance (described later). ) Is detected.

The network configuration is the same as that shown in FIG. However, a phase offset (modulation amount) according to the position in each receptive field is given to each interlayer synaptic connection. FIG. 13 is a diagram showing a sampling structure in the receptive field of the feature detection neuron used in the present embodiment. For example, if the shape of the receptive field is an ellipse as shown in FIG.
The phase modulation amount at a lattice point (S ₁ , S ₂ ,...) Sampled spirally from the center position is configured to gradually increase (the phase modulation amount at any point is determined by the nearest lattice point ).

FIG. 14 is a diagram showing an example of the amount of phase modulation according to the position of the characteristic element in the receptive field. As described above, the phase modulation amount (p ₁ , p ₂ , ...) according to the position in the receptive field is given in the modulation range of the sub-time window to which the corresponding feature belongs, so that the arrangement of the feature is Information can be extracted from the position within each sub-time window of the spike train. Needless to say, the sampling may be performed by a method other than the spiral shape.

Further, weighted addition using a time window function in a neuron as in the first embodiment, or temporal integration of a pulse width modulation signal within a predetermined time window as in the second embodiment is performed. It is assumed that there is a corresponding sub-time window for each. By doing this,
The position of the spike in each sub-time window indicates the spatial position of the feature element in the same receptive field, and as a result, the pattern of the time intervals of the plurality of spikes determines the arrangement of the feature element (or higher-order graphic pattern). Can be represented as follows.

Further, a window function or phase modulation according to the importance of the feature can be given. Here, the importance is a “remarkable” characteristic element (or graphic pattern) that facilitates discrimination between a pattern to be detected (recognized) and another pattern, and differs for each pattern category. This is obtained empirically by learning, for example, when a perturbation of a parameter that greatly evaluates the contribution of the feature detection is given (by increasing the sub-window function value of the corresponding feature, or reducing the amount of phase modulation). Change, etc.), the parameter can be updated in a direction in which the reliability of detection (recognition) increases.

In the embodiment described above, a method has been described in which spike trains corresponding to a plurality of different features from the same receptive field are used for the purpose of expressing, detecting, and recognizing two-dimensional pattern information. Here, low-order or high-order feature elements (or graphic pattern elements) are detected by performing threshold processing of a weighted weighted sum within a time window of a pulse train, so that detection (recognition) can be performed in a complicated and diverse background. Even if deformation (including position fluctuation, rotation, etc.) of the target to be performed, loss of feature detection due to illumination, noise, or the like occurs, a desired pattern can be reliably detected. This effect can be realized regardless of the specific network structure.

Further, a process corresponding to a pattern existing around the detection (recognition) target, a process of giving a priority to a specific portion of the detection / recognition target, and a process to be performed when a pattern that should not be detected is detected, etc. Processing depending on the context becomes possible. On the other hand, it may be configured such that a familiar pattern is detected (identified) instantaneously.

In addition, the presence / absence of detection of a feature can be detected by the arrival of a pulse (digital information) within a predetermined time range from a predetermined neuron, and the deviation (deformation) of the feature from an ideal pattern. Information (pulse delay amount, pulse width, frequency, etc.) according to the degree of delay
As a result, the accuracy of discrimination between patterns having high similarity and the processing time can be dramatically improved.

Further, in the above-described configuration, each position in the time window of a pulse train temporally arranged on a single input line of a predetermined neuron corresponds to a characteristic of a predetermined pattern. The wiring problem between the neuron elements can be reduced, and the reliability can be maintained at a high level.
There is an effect that the scale and power consumption of a circuit for performing recognition and detection of a predetermined object by the dimensional pattern processing can be significantly reduced compared to the related art.

[0150]

As described above, according to the pattern detecting apparatus of the present invention, a desired pattern can be reliably detected even if a feature to be detected or the like is lost.

In addition, the accuracy of discrimination between patterns having a high degree of similarity and the processing time can be drastically improved.

Further, according to the neural network circuit of the present invention, the wiring between neuron elements can be reduced, and the circuit size and power consumption can be significantly reduced while maintaining high reliability. There is.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a network configuration according to an embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of a synapse section and a neuron element section.

FIG. 3 is a diagram showing how multiple pulses propagate from a feature integration layer or an input layer to a feature detection layer neuron in the first embodiment.

FIG. 4 is a diagram showing a configuration diagram of a synapse circuit.

FIG. 5 is a configuration diagram of a synapse connection small circuit, and Embodiment 1
FIG. 3 is a diagram showing a configuration diagram of a pulse phase delay circuit used in FIG.

FIG. 6 is a diagram showing a network configuration in a case where an input from a pacemaker neuron is provided to a feature detection layer neuron.

FIG. 7 shows a configuration of a time window when processing a plurality of pulses corresponding to different feature elements input to the feature detection neuron,
It is a figure which shows the example of a weight function distribution, and the example of a characteristic element.

FIG. 8 is a flowchart showing a processing procedure of each layer.

FIG. 9 is a flowchart illustrating a processing procedure of each feature detection neuron.

FIG. 10 is a flowchart illustrating a processing procedure of each feature integration neuron.

FIG. 11 is a diagram illustrating a configuration of an example in which the pattern detection (recognition) device according to the embodiment is used for an imaging device.

FIG. 12 is a diagram illustrating a configuration of a synapse small circuit used in the second embodiment.

FIG. 13 is a diagram showing a sampling structure in a receptive field of a feature detection neuron used in the third embodiment.

FIG. 14 is a diagram illustrating an example of a phase modulation amount according to a position of a feature element in a receptive field.

Claims (34)

[Claims] An input means for inputting a pattern, and a plurality of signal processing elements, wherein detection is performed on a plurality of predetermined features of the pattern input from the input means, and a predetermined number of features included in the pattern are detected. Pattern detection means for detecting a pattern, wherein each of the plurality of signal processing elements, in response to an input from the input means or another signal processing element, further pulse signal to another signal processing element or external Pattern detection, wherein predetermined portions of the plurality of signal processing elements output pulse signals at output levels according to arrival time patterns of the plurality of pulse signals input within a predetermined time range. apparatus. 2. The method according to claim 1, wherein the plurality of signal processing elements include: a feature detection element belonging to a feature detection layer for extracting a predetermined feature; and a feature integration layer for integrating and outputting an output from the feature detection layer by a predetermined method. 2. The pattern detection device according to claim 1, wherein the predetermined part of the signal processing devices is the feature detection device that receives inputs from a plurality of the feature integration devices. 3. apparatus. 3. The pattern detection apparatus according to claim 2, wherein the feature detection element outputs a pulse signal at an output level according to the saliency of the feature or the arrangement of the feature. 4. The feature integration device outputs a pulse signal at an output level according to a sum of inputs from a plurality of feature detection devices corresponding to the same feature within a predetermined time range. Item 3. The pattern detection device according to Item 2. 5. The pattern detection apparatus according to claim 2, wherein a plurality of the feature detection layers and the feature integration layers are alternately connected in a cascade. 6. A plurality of feature detection elements each of which detects the same feature in a local range in a preceding feature detection layer corresponding to a position in the feature integration layer. The pattern detecting apparatus according to claim 5, further comprising a local receptive field structure for inputting a signal from an element. 7. Each of the feature detection elements of the feature detection layer receives a signal from a feature integration element corresponding to a different feature in the preceding feature integration layer and detects a higher-order feature. The pattern detection device according to claim 5, wherein 8. The pattern detection apparatus according to claim 2, wherein at least a part of the feature detection layers includes a plurality of filters that perform a local spatial frequency analysis on a predetermined directional component. 9. The method according to claim 1, wherein at least a part of said feature detection layer is Ga
The pattern detection device according to claim 2, wherein bor wavelet conversion is performed. 10. The pattern detection method according to claim 2, wherein the plurality of feature detection elements in the same receptive field of the feature integration element output pulses in phase synchronization with a predetermined pattern. apparatus. 11. The feature detecting element determines whether or not a predetermined feature corresponding to the pulse exists within a predetermined time window range unique to each pulse in an arrival time pattern of the plurality of pulse signals. 3. The pattern detection device according to claim 2, wherein the pattern is detected. 12. The feature detection layer according to claim 1, further comprising: a feature integration element on a preceding layer, which is associated with each feature detection element in the layer, and which is based on a total output from elements on the same receptive field. 3. The pattern detecting apparatus according to claim 2, further comprising a timing element that outputs a pulse at a pulse interval to provide a phase synchronization signal of the feature detection operation element. 13. The feature detection element in the feature detection layer provided with the timing element, and a feature integration element in the preceding feature integration layer in the same receptive field of the feature detection element. 13. The pattern detection device according to claim 12, wherein a predetermined timing pulse is output to the device. 14. The pattern detection apparatus according to claim 2, wherein each of the feature detection elements in the feature detection layer outputs a pulse-like signal with a phase corresponding to the feature to be detected. 15. The pattern according to claim 2, further comprising a plurality of the feature detection layers and the feature integration layer, wherein a signal representing a detection result of the predetermined pattern is output from a signal processing element of a last layer. Detection device. 16. The plurality of signal processing elements are coupled via coupling means, and the coupling means performs a predetermined modulation on an output pulse signal of one of the signal processing elements, and the other of the signal processing elements. The pattern detection device according to claim 1, wherein the pattern is transmitted. 17. The pattern detection device according to claim 16, wherein the modulation is a delay of a pulse phase. 18. The pattern detecting apparatus according to claim 17, wherein said coupling means reduces the amount of delay of said pulse phase as the number of times of inputting pulses representing patterns of the same category increases. 19. The pattern detection apparatus according to claim 17, wherein the delay amount of the pulse phase is substantially constant irrespective of the type of the feature. 20. The pattern detection device according to claim 16, wherein the modulation is pulse width modulation. 21. The pattern detection device according to claim 16, wherein the modulation is frequency modulation. 22. An output level according to a weighted sum obtained by multiplying a plurality of pulse signals input within the predetermined time range by a predetermined weighting coefficient value that changes over time and adding the plurality of pulse signals input within the predetermined time range. The pattern detection device according to claim 1, wherein the pattern detection device outputs a pulse signal. 23. The pattern detection apparatus according to claim 1, wherein the output level is a phase, frequency, amplitude, or pulse width of the pulse signal. 24. An image processing apparatus for controlling a processing operation of a processing target image based on a result of detecting a predetermined pattern from a pattern of the processing target image by the pattern detection apparatus according to claim 1. 25. An input unit for inputting a pattern, and a pattern detecting unit for detecting a predetermined pattern included in the pattern input from the input unit, wherein the pattern detecting unit detects a predetermined pattern from the input pattern. A feature detection layer that extracts a predetermined feature; a feature integration layer that integrates and outputs an output from the feature detection layer by a predetermined method; and the presence of a predetermined feature or pattern in response to the output from the feature integration layer And a feature position detection layer that outputs position information to be performed. 26. The pattern detection apparatus according to claim 25, wherein a plurality of the feature detection layers and the feature integration layers are alternately connected in a cascade. 27. The method according to claim 26, wherein the number of the feature position detection layers is smaller than that of the feature integration layer, and the feature position detection layers are respectively connected to predetermined portions of the feature integration layer. 3. The pattern detection device according to 1. 28. An image processing apparatus, wherein a processing operation of an image to be processed is controlled based on a result of detecting a predetermined pattern and its position from a pattern of the image to be processed by the pattern detection device according to claim 25. Processing equipment. 29. A plurality of processing layers configured by arranging a plurality of neuron elements for inputting a plurality of signals and outputting pulse signals in parallel, wherein at least one of the predetermined processing layers has the neuron element. A pulse signal output from a plurality of neuron elements in another layer is input via a synapse coupling means provided for each of the plurality of neuron elements and a bus line common to the plurality of neuron elements. The neural network circuit according to claim 1, wherein the synapse coupling means applies a pulse phase shift amount unique to each of the plurality of neuron elements to a pulse signal output from the plurality of neuron elements. 30. A signal representing an image to be processed.
An image processing apparatus, wherein the image processing apparatus controls a processing operation of an image to be processed based on an output signal as a result of processing by the neural network circuit according to claim 9. 31. A pattern is input from an input unit, and a plurality of signal processing elements are used to detect a plurality of predetermined features of the input pattern, and a predetermined pattern included in the pattern is detected. A pattern detection method for detecting, wherein each of the plurality of signal processing elements, in response to an input from the input unit or another signal processing element, further outputs a pulse signal to another signal processing element or external, A pattern detection method, wherein a predetermined part of the plurality of signal processing elements outputs a pulse signal at an output level according to an arrival time pattern of a plurality of pulse signals input within a predetermined time range. 32. An image processing method, comprising: controlling a processing operation of an image to be processed based on a result of detecting a predetermined pattern from a pattern of the image to be processed by the pattern detection method according to claim 31. 33. A pattern detection method for inputting a pattern and detecting a predetermined pattern included in the input pattern, wherein a predetermined characteristic is extracted from the input pattern by a characteristic detection layer, The output from the detection layer is integrated by a predetermined method by the feature integration layer and output, and the output from the feature integration layer is output by the characteristic position detection layer to output the position information where the predetermined feature or pattern exists. Characteristic pattern detection method. 34. An image processing method for controlling a processing operation of a processing target image based on a result of detecting a predetermined pattern and its position from a pattern of the processing target image by the pattern detection method according to claim 33. Processing method.