JPH05108601A - New network learning device - Google Patents

Abstract

(57) [Summary] [Object] The present invention provides a method for making the evaluation result of time series data from a neural network more accurate. [Structure] The present invention is a neural network (NW).
When learning time-series data with, compare output values with teacher signals for all time-series and patterns, and do not look at the learning progress based on all data, but rely on evaluating the learning progress. Data that cannot be evaluated are excluded from the targets of the learning progress evaluation.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neural network learning device for learning the relationship between inputs and outputs.

[0002]

2. Description of the Related Art FIG. 10 shows a conventional neural network learning apparatus that handles time series data. This device includes an input / output pattern holding unit 10 and an input / output pattern holding unit 1 that hold an input and a desired output (teaching signal) for the input.
A learning execution unit 11 that inputs an input pattern of 0 to a neural network (NW) and calculates an output value, a learning parameter holding unit 12 that holds a predetermined learning rule and learning parameters, learning based on the learning rule and learning parameters. The execution unit 11 includes a weight updating unit 13 that compares the output of the network with the teacher signal and updates the weight of the network.

FIG. 11 shows teacher signals and output values used in this conventional device. Here, in the conventional device, the learning rule compares the output value with the teacher signal for all time series and all patterns. The difference is defined as learning progress here. As the learning progress, an individual learning progress indicating an individual learning progress of each output data and an overall learning progress indicating a learning progress of the entire device are defined.
Usually, for example, when "learning progress = 0.1 has been performed", this indicates the overall learning progress, and all the differences between the output value and the teacher signal are learned for all time series and all patterns. Indicates that the progress was less than 0.1. The lower the learning progress, the more the learning progresses. When the learning progresses to a certain reliable range, the learning is stopped,
This time, the weight of the weight update unit 13 is used to evaluate the unlearned data.

Here, the conventional learning algorithm will be described in detail. A neural network is usually a multi-layer network, each layer is composed of many units, and a connection weight w is defined between each unit. Each unit calculates the output value of the network as shown.

Here, when a unit receives inputs from a plurality of units, the sum of the input and the threshold value θ of each unit becomes the input value net (see FIG. 3).

[0006]

[Equation 1] The output value of the unit is calculated by applying the activation function to the sum net of this input. A sigmoid function that is a differentiable nonlinear function is used as the activation function. Then, the output value O _i of the unit U _i

[0007]

[Equation 2] Becomes As a network used in the back propagation method, an Elman type network will be described here.

As shown in FIG. 9, the Elman-type network has an input layer, a hidden layer, and an output layer. The output data of the hidden layer is copied and held, and the held output data is hidden again. It is a neural network (NW) with a context layer that inputs into layers. Input layer,
Each of the hidden layer and the output layer has a plurality of nodes.

Then, input data is input to each unit of the input layer. The output value s of the hidden layer at time t is copied to the context layer, and the values of this context layer and the input layer become the input of the hidden layer at time t + 1. Therefore, this context layer holds the history of past input data. The weight updating unit changes the weight so that the output of the network becomes a desired output. The mean square error of the output values O of all patterns p, all time series t, and all nodes and the desired output value w is taken.

[0010]

[Equation 3] This is called the overall learning progress. In learning, all weights of the network are updated so that this error is reduced.

[0011]

However, in the learning of time series data by the conventional algorithm, the output value and the teacher signal are compared with respect to all the time series and all the patterns, and the learning desired by all of them is performed. It is not always the case that the progress is lower.

For example, in the example of FIG. 11, time series data of 10 columns are output from 10 output nodes.
At the output from the output node (the leftmost column in the figure), the individual learning progress is 1-0.336 = 0.664 at t = 0 and 1,
At 0.303, neither exceeded 0.8, but t
= 3 or later, the individual learning progress is 0.1 or less, and it can be seen that the learning is progressing.

However, according to the conventional method, when the overall learning progress is viewed, the result is that the learning has progressed only up to 0.6. This state does not change even if the number of learnings is increased considerably. In particular, in recognition of time-series data, in many cases, as a teacher signal, a teacher signal is commonly given to all time series in a certain pattern. In such an example, the pattern changes and the teacher signal also changes.
It seems natural that learning at the 3rd time does not proceed as expected. In the case of humans, it is the same as the beginning and when it is shown what is totally different from what has been seen for a long time.

When this time series is applied to recognition, it suffices to see all the outputs of the time series and then recognize only one thing. Therefore, even if the unreliable part such as the first time series is ignored, it may be sufficient if the other part shows a reliable recognition result.

Further, according to the conventional learning algorithm, the progress of learning becomes ambiguous, and it is impossible to know where to end the learning, so that the result must be decided by human beings. For example, when evaluating the results based on the difference in the number of nodes in the hidden layer, it is necessary to make the learning progress the same, which is ambiguous by the conventional means.

The present invention has been made in view of the above problems, and it is an object of the present invention to appropriately determine the progress of learning and improve the reliability of the apparatus.

[0017]

The present invention adopts the following means in order to solve the above problems. According to the present invention, when learning time-series data, the learning progress based on all the output data is not seen by comparing the output value with the teacher signal for all time-series and patterns. For data that is not reliable for evaluation, we decided to exclude that part from the targets of learning progress evaluation.

That is, as shown in the principle diagram of FIG.
The neural network learning device of the present invention outputs a time series data to an input, and has a plurality of output nodes (N) each showing a specific judgment result, and a neural network (NW), and an input to the neural network. And an input / output pattern holding unit (10) for holding a teacher signal that is a desired output for the
A learning execution unit (11) that inputs the input pattern of the input / output pattern holding unit 10 to a neural network and calculates an output value, a learning parameter holding unit (12) that holds predetermined learning rules and learning parameters, and the learning. Based on the rule and the learning parameter, the output of the network in the learning execution unit (11) and the input / output pattern holding unit (10)
And a weight update unit (13) for updating the weight of the network.

Then, the degree of learning determined by the error between the output of the network and the teacher signal is obtained, and when the degree of learning becomes less than a certain value, the learning is completed, but in the present invention, An output monitor unit (21) for monitoring the output of the neural network is provided, and the output monitor unit (21) monitors the output of the neural network, and if the output is not necessary for advancing the learning progress, A learning object selection means (22) for ignoring the output and excluding it from the objects of the learning progress evaluation is provided.

[0020]

When learning the time-series data, the output values of all the time-series and patterns are compared with the teacher signal to obtain the learning progress of the entire apparatus.

[0021]

[Equation 4] And is 0.6 in FIG. 11, for example. On the other hand, in FIG. 11, among the outputs of the first output node, t =
Assuming that 0 to 2 are ignored, equation (1) becomes

[0022]

[Equation 5] And the result of the overall learning progress is 0.1. Note that, in FIG. 11, rows indicate outputs for each time, and columns indicate time series data for each output node. Each output node shows a pattern such as a finger digit. Hereinafter, the present invention will be described more specifically. <Neural Network> The present invention is applied to a neural network (NW) that performs time series data processing, and a hierarchical neural network (NW) can be used as such a neural network (NW).

The hierarchical neural network (NW) has at least two layers, namely, an input layer and an output layer which are clearly distinguished. Originally, a certain degree of treatment is possible with only these two layers, but it is more effective to have one or more intermediate layers called hidden layers between these layers.

In the neural network (NW), the network needs to be adjusted, that is, learned so that a desired output can be obtained with respect to an input pattern, and a typical method is the back provacuation method.

In the present invention, the neural network (N
As W), a hierarchical network having an input layer, a hidden layer, and an output layer can be used, and among them, it is suitable for recognition using an Elman type recurrent network.

The Elman type recurrent network is as shown in FIG. 9, and the description thereof has been given above, so that it is omitted here. The present invention is applied to the case where the time-series data is output for each node (N), and the error between the individual data and the teacher signal is obtained as a whole and is regarded as the progress of learning. Any neural network (NW) can be used as long as it can obtain various outputs, and since the output data can be time-series data, the input to the neural network (NW) is time. It may or may not be series data. <Evaluation of Learning Progress> In the present invention, the evaluation of the overall learning progress is performed by the equation (2).

That is, in the present invention, the output monitor section (21) monitors the output of the neural network, and when the output is not necessary for advancing the learning progress, the output is ignored and the learning progress is evaluated. Remove from the target. This operation is performed by the learning target selection means (22).

Here, whether the output is necessary for advancing the progress of learning is determined by the operator operating the neural network learning device, or automatically by the computer provided in the network. There is.
The standard is determined by the characteristics of the data and other experiences.

In the former case, a CRT display is provided as the output monitor section (21), and the operator sees the output displayed on the CRT display, and unnecessary data is learned by a keyboard as a learning target selection means (22). Remove from the learning progress evaluation target.

In the latter case, as shown in FIG. 8, the monitor section (21) collects all the output data, and the learning target selection means (22) sets a reference value for excluding the learning progress evaluation target. The structure has a reference value setting unit (24) to be set. Then, in advance, the operator sets the reference value setting unit (2
A predetermined reference value is set in 4), and the monitor unit (21)
When the individual learning progress determined by the error between the output data and the corresponding teacher signal is larger than the reference value, the output data is ignored.

In both the former and latter cases,
As shown in FIG. 2, the learning target selection means (22) is provided with an evaluation constant setting unit (23) for presetting the number of output data to be excluded from the targets of learning progress evaluation, and the evaluation constant setting unit (23). The output data can be ignored by the number of evaluation constants set in.

Here, the learning is repeated with the evaluation constant variable, and the relation between the evaluation constant, the progress of learning and the recognition rate (certainty of recognition described in the recognition evaluation below) is obtained, and the best recognition result is obtained. It is advisable to obtain a pair of the evaluation constant and the progress of learning and then use the pair for recognition.

Here, when the evaluation constant is changed, the learning progress is made the same for each evaluation constant so that the evaluation of the result is performed correctly. <Recognition Evaluation> In the present invention, the learning progress can be made appropriate by the above operation, but it can be used when the history judgment unit (C) for evaluating the output data as described below is provided after the learning is completed.

That is, this is a case where a history judging unit (C) is added to the neural network learning device to form a time-series data recognizing device. As the history judging unit (C),
As shown in FIG. 7, adding means 1 for sequentially adding the time series data output from the output node (N) to obtain the total.
A maximum value selecting means 2 for selecting the maximum value among the output total values of the output nodes (N), and a threshold value setting means 3 for presetting a threshold value to be used as a reference for output evaluation. And the maximum value selected by the maximum value selecting means 2 and the threshold value set by the threshold value setting means 3 are compared, and when the maximum value is larger than a predetermined threshold value, the maximum value The evaluation means 4 outputs the node (N) corresponding to the above as a recognition result, and outputs the evaluation result that the recognition is impossible when the maximum value is smaller than a predetermined threshold value.

In this case, the time-series data output from the output node (N) is sequentially added to each output node (N) to obtain the total, and then the output total of each output node (N) is calculated. Of the values, the maximum value is selected, and when the maximum value is larger than a predetermined threshold value, the node (N) corresponding to the maximum value is set as the recognition result, and the maximum value is larger than the predetermined threshold value. When it is small, it is unrecognizable.

In this case, since the data that can be ignored according to the present invention has a large difference from the teacher signal, its value is small and influences the posture even if it is not added in the summation. Don't give Therefore, it is useful to utilize the present invention for such recognition. In the "Problems to be solved by the invention" section, "Ignoring unreliable parts such as the first time series may be sufficient if other parts show a reliable recognition result." That is the case in such a case.

To facilitate understanding, let us consider a case in which a neural network (NW) recognizes the gestures of a person such as goo, choki, and par using the bending angle of the finger and the twisting angle of the wrist as input data.

The meaning of the output node (N) of the neural network (NW) is defined for each corresponding node (N). For example, as shown in FIG. 6, when there are ten output nodes (N), the first node (N) is “goo”, the fourth node (N) is “choki”, 8
Node (N) is defined to mean "par", and other nodes are defined to mean other patterns.

When the output is "1", the input data is recognized as "meaning" defined in the corresponding node (N). Also here, t =
Time series data of 0 to 29, 30 in total, is for each node (N)
Shall be output respectively.

Then, the time-series output values output from each output node (N) are sequentially added to each output node (N) to obtain the sum, and the total output value of each output node (N) is calculated. , Select the maximum value.

Here, until t = 14, that is,
It is assumed that 15 pieces of time series data are obtained from each output node (N), and the total value of each node (N) at that time is 1st node = 9.375, 2nd node = 0.000, 3rd node =
0.000, 4th node = 0.174, 5th node =
0.632, 6th node = 0.096, 7th node =
0.000, 8th node = 4.317, 9th node =
It is assumed that 0.501 and the tenth node = 0.000. Since the maximum value is 9.375, the first node (N)
Seems most certain.

Then, the maximum value is selected, and it is compared whether or not the selected maximum value is larger than a predetermined threshold value. This threshold value is a reference value when the output time-series data can be evaluated as probable if it is a certain value or more when evaluating the time-series data, and is determined by an empirical rule. Is. Therefore,
What is used as the threshold value can be determined by statistical processing or the like. Here, the threshold value is set to "8.
0 ”.

When the maximum value is larger than the threshold value, the node (N) corresponding to the maximum value is set as the recognition result. Here, the maximum value is 9.375, which is larger than the threshold value, so that it is recognized that the input represents “goo”.

On the other hand, when the maximum value is smaller than a predetermined threshold value, it is concluded that the input is unrecognizable, that is, it is unknown what the input data represents. It should be noted that when comparing with a value greater than or less than the threshold value, which of the same values as the threshold value is determined may be arbitrary by the setter.

The threshold does not have to be common during a certain recognition mode and can be changed dynamically. That is, the threshold value is set to 6.0, the processing is performed once, and when the processing result is unrecognizable, the threshold value is changed to 7.0 again and the processing is repeated.
The threshold setting can be repeated until satisfactory results are obtained.

The threshold value is set at each output node (N).
It can be set for all or individually. For example, when there are five output nodes (N), a threshold value is separately set for each output node (N), and the output node (N) corresponding to the maximum value is, for example, the third node (N). For example, the threshold value corresponding to the node (N) is compared with the maximum value to evaluate the output.

Furthermore, the comparison judgment between the threshold value and the maximum value is
It can be done after all outputs are completed or before. That is, after the input to the neural network (NW) has finished and all the output values have been output, the output values of each output node (N) are summed, and the maximum value is obtained to make the determination by the comparison processing as described above. And when the input to the neural network (NW) is finished and all the output values are output, the output values of the output nodes (N) are sequentially summed to obtain the maximum value, and the maximum value is calculated. When the value exceeds the threshold value, the node (N) corresponding to the maximum value may be the recognition result. <Relationship Between Number of Nodes in Hidden Layer, Progress of Learning, and Recognition Rate> In learning, the number of nodes in the hidden layer affects the final recognition rate. In learning, the number of nodes in the hidden layer is changed and the learning is repeated.Finally, the recognition rate is calculated using the saved weight of the progress of each learning of each node number, and the hidden result that gives the best recognition result. The best recognition rate can be obtained by finding the set of the number of layers and the progress of learning and using this for subsequent recognition.

That is, the learning is repeated with the number of hidden layer nodes being variable, the relationship between the number of nodes, the progress of learning, and the recognition rate is obtained, and the number of hidden layer nodes and the learning rate corresponding to the optimum recognition rate are obtained. , And then use that set for recognition.

When the number of nodes in the hidden layer is changed, the learning progress is made the same for each number of nodes so that the results can be correctly evaluated.

[0050]

Embodiments of the present invention will be described below with reference to the drawings. <Embodiment 1> This embodiment is a time-series data recognition device using a hierarchical neural network (NW) as shown in FIG. 2, and the neural network (NW) has a learning section (A). After the learning in the learning unit (A) is completed, the unlearned data is evaluated using the weight of the weight updating unit 13.

First, a neural network (NW)
Has an input layer, a hidden layer, and an output layer, and each layer has a plurality of nodes (units). The plurality of units forming each layer are connected to each other with a certain weight. The weight of the network can be learned by giving a set of an input pattern and a desired output pattern (teacher signal) to this network.

Learning proceeds as follows. First, an input pattern in the network is given and an output is obtained. If the learning is not correct, teach the network the correct (desired) output value. Then, the network adjusts the internal structure of the network (coupling strength = weight) so that the correct output value and the actual output value decrease. By repeating this many times, the network automatically learns the weights that satisfy a certain input / output relationship. This learning algorithm is the back pro vacation method.

When the network learned in this way is used, the correct output displayed is returned for the learned input pattern, but the output pattern obtained by interpolating the learned input / output pattern is returned for the input pattern not further learned. This is a major feature of neural networks (NW).

Thus, the neural network (N
W) has a learning unit (A), and evaluates unlearned data using the weight of the weight updating unit 13 after the learning in this learning unit (A) is completed.

Here, the learning section (A) is an input / output pattern holding section 1 for holding an input to the neural network (NW) and a teacher signal which is a desired output for the input.
0, a learning execution unit 11 that inputs an input pattern of the input / output pattern holding unit 10 to a neural network (NW) and calculates an output value, a learning parameter holding unit 12 that holds predetermined learning rules and learning parameters, Based on the learning rule and the learning parameter, the learning execution unit 11
The output of the network and the teacher signal of the input / output pattern holding unit 10 are compared, and the weight updating unit 13 that updates the network weight, and the connection weight storage that stores the updated network weight at an arbitrary learning stage The unit 14 is provided.

The learning execution unit 11 will be described in more detail.
As described above, the neural network (NW) has a multi-layered network structure, each layer is composed of many units, and the coupling weight W is defined between each unit.

Each unit calculates the output value of the network as shown below. When a unit receives inputs from a plurality of units, the sum of the sums and the threshold value θ of each unit (distinguished from the threshold value to be compared with the above-mentioned maximum value in the present invention) are input values. It will be net (see Figure 3).

[0058]

[Equation 6] The output value of the unit is calculated by applying the activation function to the sum net of this input. A sigmoid function that is a differentiable nonlinear function is used as the activation function. Then, the output value O _i of the unit U _i

[0059]

[Equation 7] Becomes The network used in the back-propagation method is a multilayer network as described above, but here, a case will be described in which it has three layers of an input layer, a hidden layer, and an output layer as shown in FIG.

Each unit in the hidden layer is associated with every unit in the input layer. Each unit in the output layer is connected to each unit in the input layer and hidden layer. And there is no bond between units within each layer.

Input data to the network is given to each unit in the input layer. Therefore, the output value h of each unit in the hidden layer is

[0062]

[Equation 8] Becomes The output value o of each unit in the output layer is

[0063]

[Equation 9] Becomes Next, the weight updating unit 13 will be described in more detail.

The weight updating unit 13 changes the weight of the network so that the output of the network becomes a desired output. Mean square error E between the actual output value (o _pi ) when a certain pattern p is given and the desired output value (t _pi ).
take _p .

[0065]

[Equation 10] In order to train, all weights in the network are changed to reduce this error.

The learning rule for the output layer is

[0067]

[Equation 11] The learning rule for hidden layers is

[0068]

[Equation 12] Is. Next, the connection weight storage unit 14 stores the network weight at any learning stage.

The recognition device for determining the history of the time-series output data loads the network weight stored in the weight storage unit 14 and gives the input pattern to calculate the output of the network. When the network learned as described above is used, a correct output is returned for a learned input pattern, but an output pattern based on the learned input / output pattern is returned for an input pattern that has not been learned.

With the above system, a plurality of patterns of postures and states of human hands are learned. For example, the state in which a number is expressed with 10 fingers is learned. 1 when only the index finger of one hand is standing, 2 when the middle finger is standing, 3 when the ring finger is standing, 4 further the little finger stands, 5 the thumb stands, and so on.
It is assumed that a plurality of patterns of hand shape data that have changed little by little for all patterns are measured and learning is completed when the overall learning progress reaches 0.1.

Initially, by comparing the output value with the teacher signal for all time series and patterns to obtain the learning progress of the entire apparatus, the overall learning progress is

[0072]

[Equation 13] For example, this is similar to the conventional case, and
The overall learning progress was 0.6.

Therefore, in FIG. 2, the evaluation constant is set to 3 and all the output nodes are targeted and the outputs from t = 0 to 2 are ignored. Then, equation (1) becomes

[0074]

[Equation 14] And the result of the overall learning progress was 0.1 as shown in FIG. Note that FIG. 5 shows a teacher signal representing the numeral 1 by the finger. Each node (N) is a number from 1 to 10
In the output indicating the numeral 1, all the output values from the first output node (N) are “1.
000 ", the output values from the other nodes (N) are all preferably" 0.000 ".

As a result of the above, since it was judged that the learning progress was 0.1 and the learning was sufficiently performed, the present apparatus was used for the recognition of unlearned data thereafter. When unlearned data was input to this device, an output result as shown in FIG. 6 was obtained.

When this output result is evaluated, the output at t = 2 is 0.90 at the eighth output node (N).
The output at 0, t = 3,4,5 is 0.955 respectively.
Therefore, at t = 5, it seems reasonable to judge that the output when the eighth node (N) is the most weighted and the recognition result is “8”.

However, when the output is further continued, the first
The output of the nth node (N) is 0.90 when t = 7.
It is 0.925 when 0, t = 8, and 0.950 when t = 9 and 10, respectively. Here, the weight of the recognition result of the first node (N) also becomes large, and therefore,
At the time point of t = 10, it should be judged that the eighth node (N) is “8” as the recognition result, or the first node (N) is “1” as the recognition result. It is not always clear whether or not to judge.

Therefore, in this embodiment, the history judgment section (c)
As shown in FIG. 7, the maximum of the addition means 1 for sequentially adding the time-series data output from the output node (N) to obtain the total and the output total value of each output node (N) A maximum value selecting means 2 for selecting a value, a threshold value setting means 3 for presetting a threshold value to be a reference for evaluation of output, a maximum value selected by the maximum value selecting means 2 and the above The threshold value set by the threshold value setting means 3 is compared, and when the maximum value is larger than a predetermined threshold value, the node (N) corresponding to the maximum value is output as a recognition result, and the maximum value is output. The evaluation means 4 is provided for outputting the evaluation result that the value cannot be recognized when the value is smaller than a predetermined threshold value.

A threshold value of 8.0 was set in the threshold value setting means 3. Then, as shown in FIG. 6, t = 1
The sum of each output at the stage of 5 was obtained by the adding means 1, and the maximum value was selected by the maximum value selecting means 2.

Since the maximum value is 9.375 of the first node (N), this 9.375 and the threshold value are the evaluation means 4.
It was judged that the maximum value was larger than the threshold value, since 9.375> 8.0. Therefore, as the final result, the input unlearned data was recognized as the result corresponding to the first node (N).

As described above, even when the output result is unstable, it is easy to judge the output result, and the most probable result can be output. The present invention is suitable for a neural network learning device used for such recognition. Second Embodiment Next, an example in which the optimum number of hidden layer nodes, the learning progress, and the recognition rate are derived by varying the number of hidden layer nodes will be described.

The present embodiment is an example in which an Elman type recurrent neural network (NW) is used for recognizing a human hand motion, for example, sign language. The recurrent neural network (NW) used here is
As shown in FIG. 9, a neural network having an input layer, a hidden layer, and an output layer, having output data of the hidden layer copied and held, and further having a context layer for inputting the output data to the hidden layer again ( NW).

The operation examples of the input layer, hidden layer, and output layer are as follows.
The principle is the same as that described in the first embodiment, and the learning is performed by the back-propagation method. In addition to this, the recurrent neural network (NW) has high recognition efficiency because the context layer holds past input information.

The context layer is a copy of the hidden layer value one hour before, and this will be added as the next input data. By this feedback function, the context layer can hold the pattern of past input data.

In such a network, the input is time-series data regarding the hand such as the bending angle and the position of the hand, and the output uses a teacher signal as shown in FIG. Each node (N) (vertical column) corresponds to each sign language type.

In this example, the recurrent neutral network is used for recognizing the gesture of sign language.
The recognition rate (validity rate) becomes a big problem. This recognition rate is affected by the number of hidden layer nodes. We need to find the optimal number of nodes with the best recognition rate. In order to find this optimal number of nodes, it is necessary to learn the same input / output data (input and teacher signal) with various numbers of nodes and evaluate the recognition rate with the same learning progress. is there.

In parallel with the number of nodes in the hidden layer, the higher the learning progress, the higher the recognition rate does not necessarily mean. If overrunning (too much learning), the recognition rate increases. Since it decreases, it is necessary to find the optimal learning progress.

In the learning carried out by changing the number of nodes, for example, when the number of nodes is 100, first, the learning progress (the error between the teacher signal and the actual output value) is set to 0.9. Then, when learning is satisfied, the weight at this time is saved, and the learning progress is set to 0.8, learning is performed, and when the learning progress reaches 0.05, the learning is terminated.

Next, the number of nodes is changed and the same is done. Finally, the recognition rate is calculated using the stored weight of the learning progress of each node number, and the pair of the number of hidden layers and the learning progress that gives the best recognition result is calculated. If this is used for future recognition, the best recognition rate will be obtained.

Here, the learning progress is a value for which the difference (= error) between the actual output and the teacher signal is desired to fall within a certain range. Further, the recognition rate evaluation is a test pattern after completion of learning (determining whether or not the result when inputting unlearned data is correct.) There is an error (error) between the actual output and the teacher signal. , For learning time series data,
When it becomes smaller than the learning progress value set for all patterns, all time series, and all output nodes, it means that learning has advanced to that value.

A specific example of this embodiment will be described below. Regarding learning of the same output pattern, the recognition rate when learning is performed by changing the number of hidden layer nodes and the learning progress,
A comparison will be made between the conventional method and the method of the present invention.

In the conventional method, even if the learning progress is set to 0.7, there are many cases in which the learning does not converge (the learning progress is not satisfied, the learning does not end), and even if it is satisfied, it takes a huge amount of time. End up. In most cases, the first three places in the time series do not satisfy the progress of learning, and all the rest satisfy the progress of learning.

The recognition rate evaluation in such a case means that the evaluation is performed while the learning progress is 0.7 or 0.6. With this, it is difficult to determine where to end the learning, and it is also unknown whether the optimum recognition rate can be obtained. Hereinafter, the case of the conventional method and the case of carrying out the method of the present invention will be compared. Conventional method Number of hidden layers 100 50 30 Learning progress 0.9 0.9 0.9 Learning frequency 1 1 1 Recognition rate (%) 0 0 0 Learning progress 0.8 0.8 0.8 0.8 Number of learning 53 78 76 Recognition rate (%) 40 75 80 Progress of learning 0.7 (Not converged) 0.7 (Not converged) 0.7 (Not converged) Number of learning 500 500 500 500 Recognition rate (%) 80 85 85 85 Learning Progression of 0.7 (not converged) 0.7 (not converged) 0.7 (not converged) Number of learning 1000 1000 1000 1000 Recognition rate (%) 90 90 90 85 Progress of learning 0.7 (not converged) 0.7 (not converged) 0.7 (not converged) Number of learning 1500 1500 1500 Recognition rate (%) 85 85 85 Progress of learning 0.7 (not converged) 0.7 (not converged) 0. 7 (not converged) Number of learning 2000 2000 2000 2000 Recognition rate (%) 85 85 85 85 learning progress 0.7 (not converged) 0.7 (not converged) 0.7 (not converged) learning count 2500 2500 2500 recognition rate (%) 85 85 85 85 learning progress 0 .7 (not converged) 0.7 (not converged) 0.7 (not converged) Number of learning 3000 3000 3000 Recognition rate (%) 85 80 85 Number of nodes is 100, learning progress is 0.7, learning 10 times
When 00, the number of nodes is 50, the learning progress is 0.7, and the number of learning is 10
An optimal recognition rate of 90% was obtained by using the network weight when 00.

As a result, any one of them will be used. The present invention Number of hidden layers 100 50 30 Progress of learning 0.9 0.9 0.9 Number of learning 1 1 1 Recognition rate (%) 0 0 0 Progress of learning 0.8 0.8 0.8 0.8 Number of learning 2 2 2 Recognition rate (%) 0 0 0 Learning progress 0.7 0.7 0.7 0.7 Number of learning 3 3 3 Recognition rate (%) 0 0 0 Learning progress 0.6 0.6 0.6 Number of learning 4 4 4 Recognition rate (%) 0 0 0 Progress of learning 0.5 0.5 0.5 0.5 Number of learning 239 288 297 Recognition rate (%) 70 60 60 Progress of learning 0.4 0.4 0.4 Number of learning 250 289 300 Recognition rate (%) 90 80 80 Progress of learning 0.3 0.3 0.3 Number of learning 307 416 346 Recognition rate (%) 90 90 90 85 Progress of learning 0.2 0.2 0.2 Number of learning 403 841 771 Recognition rate (%) 90 90 95 Learning progress 0.1 0.1 0.1 Number of learning 1580 2070 1980 Recognition rate (%) 90 85 90 90 Optimal number of nodes, learning progress of 0.2, learning rate of 771 95% was obtained. According to the above, the optimal number of hidden layer nodes obtained by the conventional method is different from that of the present invention, and the recognition rate is also different. The result obtained by the method of the present invention was able to obtain a higher recognition rate as compared with the conventional method.

This result also shows that the learning progress is accurately obtained, so that the optimal number of hidden layer nodes and the learning progress can be accurately and easily obtained. Compared with the number of times of learning required to obtain the optimum recognition rate in the conventional method, the number of times of learning in the present invention was smaller, and a good recognition rate could be obtained. In other words, the optimal recognition rate can be obtained in a short time, which leads to higher experiment rate.

Further, according to the conventional method, the present invention can solve the problem that the progress of the learning does not proceed from 0.8 rank in one direction and it is difficult to know where to finish the learning. The recognition rate under the optimum condition of the conventional method, which is obtained in such a phenomenon, is not reliable. If you learn more, the recognition rate may improve, and if you stop it when you are less, there is still anxiety that the recognition rate was good. The recognition rate depends on the accuracy. According to the present invention, the anxiety of this point is eliminated, and the one with the highest recognition rate can be obtained.

[0097]

According to the present invention, the progress of learning can be accurately obtained, and as a result, the number of times of learning required to obtain an optimum recognition rate can be reduced. In other words, the optimal recognition rate can be obtained in a short time, which leads to higher experiment rate.

The present invention can also solve the problem of where to end learning. Then, by ignoring the output that is not necessary for the progress of learning, the one with the highest recognition rate can be obtained.

[Brief description of drawings]

FIG. 1 is a principle diagram of a neural network learning device of the present invention.

FIG. 2 is a block diagram showing an embodiment of the present invention.

FIG. 3 is a diagram showing input / output characteristics of the network.

FIG. 4 is a diagram showing the structure of a network.

FIG. 5 is a diagram showing a teaching signal, an output result, and a learning progress in the embodiment.

FIG. 6 is a diagram showing an example in which unlearned data is input and recognized after learning.

FIG. 7 is a block diagram of a history judgment unit

FIG. 8 is a diagram showing another example of the present invention.

FIG. 9 is a diagram showing an example of an Elman type recurrent neural network.

FIG. 10 is a diagram showing a conventional neural network learning device.

FIG. 11 is a diagram showing a teaching signal, an output result, and a learning progress in a conventional example.

[Explanation of symbols]

 NW neural network (N) output node (A) learning unit 1 adding unit 2 maximum value selecting unit 3 threshold setting unit 4 evaluation unit 10 input / output pattern holding unit 11 learning execution unit 12 learning parameter holding unit 13 weight updating unit 14 Combination weight storage unit 21 Output monitor unit 22 Learning target selection unit 23 Evaluation constant setting unit 24 Reference value setting unit

Claims (3)

[Claims] 1. A neural network (N) having a plurality of output nodes (N) each of which outputs time-series data with respect to an input and which shows a specific judgment result.
W), an input / output pattern holding unit (10) for holding an input to the neural network and a teacher signal which is a desired output to the neural network, and an input pattern of the input / output pattern holding unit (10) is input to the neural network. A learning execution unit (11) that calculates an output value, a learning parameter holding unit (12) that holds a predetermined learning rule and a learning parameter, and a learning execution unit (11) based on the learning rule and the learning parameter. And a weight updating unit (13) that compares the output of the network with the teacher signal of the input / output pattern holding unit (10) and updates the weight of the network, and determines the difference between the output of the network and the teacher signal. The neural network that determines the progress of learning that is completed and that the learning is completed when the progress of this learning becomes a certain value or less. In the learning device, an output monitor section (21) for monitoring the output of the neural network is provided, and the output of the neural network is monitored by the output monitor section (21) to improve the learning progress. 2. A neural network learning device, characterized in that a learning target selection means (22) is provided for ignoring the output of the learning progress evaluation target when it is not required. 2. The learning target selection means (22) has an evaluation constant setting unit (23) for presetting the number of output data to be excluded from the targets of learning progress evaluation, and the evaluation constant setting unit (23). 2. The neural network learning apparatus according to claim 1, wherein the output data is ignored by the number of evaluation constants set in. 3. The neural network is provided with an input layer, a hidden layer, and an output layer each consisting of a plurality of nodes, and in learning, the learning is repeated with the number of nodes in the hidden layer being variable, and the number of nodes and the learning progress are 2. The neural network according to claim 1, wherein a relationship between the recognition rate and the recognition rate is obtained to obtain a set of the number of hidden layer nodes corresponding to the optimum recognition rate and a learning progress rate, and thereafter the set is used for recognition. Network learning device.