KR20170081887A - Method of determining the final answer by convolution of the artificial neural networks - Google Patents

Abstract

A method of determining a final correct answer using convolution in an artificial neural network according to an embodiment of the present invention includes the steps of receiving vectors having an arbitrary dimension, Dividing a plurality of candidate answers into sub-regions of a predetermined size, learning an artificial neural network for each of the partial regions, determining a plurality of correct answers from the vectors obtained as learning results, and And selecting a final correct answer from the plurality of correct answers.

Description

TECHNICAL FIELD [0001] The present invention relates to a method of determining a final answer using convolution in an artificial neural network,

The present invention relates to an artificial neural network technique, and more particularly, to a method for determining a final correct answer using convolution in an artificial neural network.

In general, the Artificial Neural Network in machine learning and cognitive science is a statistical learning algorithm inspired by biological neural networks (the animal's central nervous system, especially the brain). An artificial neural network refers to the entire model that has artificial neurons (nodes) that form a network of synapses by changing the binding strength of synapses through learning.

This artificial neural network technique can improve the performance of classification problem by using convolution. Specifically, the input total observation information is divided into sub-regions. At this time, the sub-regions are allowed to overlap and set to include all or part of the entire observation information. Each sub-region is learned by constructing a separate neural network, forming a convolutional layer.

In each neural network of learned sub-regions, an output vector is generated, which is changed to a representative value or a representative vector by a pooling technique. The input vector of the next layer is generated by generating a new vector following the modified representative vectors. In this way, the convolutional layer can represent upper patterns as a combination of sub-patterns that appear locally in the lower layer.

In contrast to conventional artificial neural networks or kernel machines iteratively transforming input values to ensure non-linearity, learning through this convolutional layer directly implements the abstraction process from lower to higher patterns It is possible to do it. This is not only convenient for visualizing the neural network configuration, but it can also mitigate the data sparseness that occurs when input values are given in very large dimensions.

In the conventional technique, after several stages of abstraction, classification is performed through a fixed decision layer using only a simple logistic regression layer at the final stage. This method is based on the assumption that the output information learned at the top layer immediately before the decision is pattern-related information that directly affects the problem to be solved.

However, if there is no sufficient knowledge or empirical knowledge about the types of patterns and search spaces on the top layer, data sparseness due to large dimensions also occurs in the top layer.

Also, in the process of applying the convolutional layer to the decision layer to obtain the data sparseness mitigation capability provided by the convolutional layer, the pooling technique for the existing representative vector generation is directly used for decision making. .

For example, existing pooling techniques can be divided into max pooling and average pooling. Both of these methods are for designating representative values of the patterns of the respective sub-regions, and the discrete information required in the decision step is lost.

It is an object of the present invention to provide a method for determining a final correct answer of an artificial neural network using convolution for dimension reduction of top-layer patterns and data sparseness reduction.

According to an aspect of the present invention, there is provided a method for determining a final correct answer using convolution in an artificial neural network, comprising: receiving vectors having an arbitrary dimension; inputting the input vectors into a sub- regions, learning the artificial neural network for each of the partial regions, determining a plurality of correct answers from the vectors obtained as a result of the learning, and extracting final correct answers from the plurality of correct answers .

According to the embodiment of the present invention, by using the convolution method in the final correct answer determination process of the artificial neural network, data sparseness due to dimensional reduction is alleviated by learning a wide range of inputs with a separate neural network having a low dimension.

In addition, according to the embodiment of the present invention, an artificial neural network is constructed for sub-regions of input in the step of finally determining the correct answer in the artificial neural network, the correct answer candidates are directly determined at the output of each neural network, By determining the final candidate (final correct answer) from the candidates, it is possible to make error correction in a way that generates one correct answer from the entire input, and even if the correct answer decision from the local patterns is not correct, The artificial neural network can mitigate the difficulty of learning the values that are not the maximum value in the probability vector for determining the correct answer, thereby estimating the best correct answer prediction result through the patterns that can be observed even in the case of the prediction failure .

In addition, according to the embodiment of the present invention, it is possible to learn not only the performance improvement in the final layer but also the most abstract patterns independently appearing in position, thereby reducing the number of stacking required or reducing data, Can improve performance.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram of a final correct answer determining apparatus using convolution of an artificial neural network according to an embodiment of the present invention; FIG.
BACKGROUND OF THE INVENTION Field of the Invention [0001] The present invention relates to an artificial neural network,
3 is a flowchart of a method for determining a final correct answer using convolution in an artificial neural network according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. And is provided to fully convey the scope of the invention to those skilled in the art, and the present invention is defined by the claims. It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. It is noted that " comprises, " or "comprising," as used herein, means the presence or absence of one or more other components, steps, operations, and / Do not exclude the addition.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are given to the same or similar components, and in the following description of the present invention, Detailed explanations of the detailed description will be omitted when the gist of the present invention can be obscured.

1 is a structural diagram of a final correct answer determining apparatus using convolution in an artificial neural network according to an embodiment of the present invention.

1, an apparatus 100 for determining a final correct answer using a convolution in an artificial neural network according to an embodiment of the present invention includes an input unit 110, a generating unit 120, a learning unit 130, A selection unit 140, and a selection unit 150. Here, the final correct answer determination apparatus 100 may be implemented in a decision layer that finally determines a decision among the plurality of layers of the artificial neural network.

Hereinafter, the operation of the final correct answer determining apparatus 100 using convolution in the artificial neural network according to the embodiment of the present invention will be described in detail with reference to FIG.

The input unit 110 receives a vector having an arbitrary dimension composed of discrete or continuous values. For example, the vector input to the input unit 110 may be a six-dimensional vector (first through sixth vectors 11 through 16) as shown in FIG. 2, and the marked circular image may represent the respective element values of the vector .

The generator 120 divides the vector input from the input unit 110 into sub-regions of a predetermined size. Here, the predetermined size may be preset by the developer or user of the artificial neural network in advance. Also, the sub-regions may be connected to each other, or may be areas that are not connected to each other. By separating the sub-regions, the sub-regions can learn redundantly. For reference, a general ensemble scheme can be used to distinguish and generate sub-regions.

In FIG. 2, sub-regions having three element values can be consistently generated. For example, in FIG. 2, the first to third vectors 11 to 13 may be divided into one sub-region, and the second to fourth vectors 12 to 14, the third to fifth vectors 13 to 15 ) And the fourth to sixth vectors 14 to 16 are each divided into one sub-region, so that a total of four sub-regions 21 to 24 can be generated.

The learning unit 130 learns the artificial neural networks 31 to 34 that receive each of the sub-regions 21 to 24 generated by the generator 120 as inputs. That is, the learning unit 130 learns each of the artificial neural networks 31 to 34 according to the sub-regions 21 to 24.

Here, the artificial neural network can be configured in various ways. For example, the linear combination of the element values of the sub-region input vector may be performed by stacking or repeating several layers of artificial neural networks that yield results through various activation functions such as sigmoid function or tanh function recurrent network).

In addition, the learning unit 130 determines the dimension of the output vector for each sub-region and the range of the output value. Here, the dimension may be the number of all classes to be finally determined through the artificial neural network. A neural network that follows the shape of an artificial neural network that learns sub-regions by sub-regions by inputting sub-regions generated by the generator 120 for each element of the output vector may be formed.

The determination unit 140 determines candidates 41 to 44 of the final correct answer value to be determined by the network from the vector obtained as the learning result of the learning unit 130. [ For example, when the vector represented by the learning unit 130 has a dimension by the number of selectable discrete all correct answer values, and the value of each dimension represents the selection probability of each correct answer, the determination unit 140 determines a plurality of values (Correct answer candidates) 41 to 44 for each partial region can be determined through the correct answer determination method using the softmax function that determines the dimension having the maximum value among the answers as the correct answers.

At this time, the determination unit 140 may generate the correct candidates (the final correct answer value for each subregion) 41 to 44 by the number of the sub-regions generated in the generation unit 120. [ For example, as the four partial regions 21 to 24 are generated by the generating unit 120 in FIG. 2, the four determining candidates 41 to 44 can be determined by the determining unit 140.

The selecting unit 150 selects one correct answer 51 by selecting correct answers from the correct answers candidates 41 to 44 determined by the determining unit 140. [ At this time, the selection unit 150 can select the final correct answer by using a method of selecting (correcting) the correct answer candidates with higher importance according to the importance of each sub-region that appears most frequently or arbitrarily. Alternatively, the selecting unit 150 may select a final correct answer by applying voting algorithms or methods for creating an ensemble model.

For example, in order to select a final correct answer from the correct candidates 41 to 44, the selecting unit 150 selects the frequency of the candidates for each correct candidate, the probability that the neural network of each sub-region generates a candidate, Selects the final correct answer 51 using the prior probability of each sub-region designated by the user, or selects the final correct answer 51 using the ensemble model through widely known classifiers.

As described above, according to the embodiment of the present invention, by using the convolution method in the final correct answer determination process of the artificial neural network, it is possible to learn a wide range of inputs by using a separate neural network having a low dimension so that data sparseness do.

In addition, according to the embodiment of the present invention, an artificial neural network is constructed for sub-regions of input in the step of finally determining the correct answer in the artificial neural network, the correct answer candidates are directly determined at the output of each neural network, By determining the final candidate (final correct answer) from the candidates, it is possible to make error correction in a way that generates one correct answer from the entire input, and even if the correct answer decision from the local patterns is not correct, The artificial neural network can mitigate the difficulty of learning the values that are not the maximum value in the probability vector for determining the correct answer, thereby estimating the best correct answer prediction result through the patterns that can be observed even in the case of the prediction failure .

In addition, according to the embodiment of the present invention, it is possible to learn not only the performance improvement in the final layer but also the most abstract patterns independently appearing in position, thereby reducing the number of stacking required or reducing data, Can improve performance.

FIG. 3 is a flowchart illustrating a final correct answer determination method using convolution in an artificial neural network according to an embodiment of the present invention.

Hereinafter, unless otherwise noted, it is assumed that the apparatus is operated by the final correct answer determination apparatus 100 implemented in the decision-making layer that finally determines the doctors in the artificial neural network.

First, a vector having an arbitrary dimension composed of discrete or continuous values is input (S310). For example, the input vector may be a six-dimensional vector (first to sixth vectors 11 to 16) as shown in FIG. 2, and the indicated circular image may represent the respective element values of the vector.

In operation S320, the input vector is divided into sub-regions of a predetermined size and generated (S320). Here, the predetermined size may be preset by the developer or user of the artificial neural network in advance. In FIG. 2, it can be divided into sub-regions 21 to 24 consistently of three element values. For example, in FIG. 2, the first to third vectors 11 to 13 may be divided into one sub-region, and the second to fourth vectors 12 to 14, the third to fifth vectors 13 to 15 ) And the fourth to sixth vectors 14 to 16 are each divided into one sub-region, so that a total of four sub-regions 21 to 24 can be generated.

Thereafter, the artificial neural networks 31 to 34 are learned by inputting the sub-regions 21 to 24 generated in step S320 (S330). That is, each of the plurality of artificial neural networks 31 to 24 corresponding to each of the sub-regions 21 to 24 is learned.

Here, the artificial neural network can be configured in various ways. For example, the linear combination of the element values of the sub-region input vector may be performed by stacking or repeating several layers of artificial neural networks that yield results through various activation functions such as sigmoid function or tanh function recurrent network).

At this time, the dimension of the output vector for each sub-region and the range of the output value can be determined. Here, the dimension may be the number of all classes to be finally determined through the artificial neural network. For example, a neural network that follows the form of an artificial neural network may be formed by inputting the sub-regions generated in step S320 for each element of the output vector and learning for each sub-region.

The candidate 41 to 44 of the final correct answer value to be determined by the network is determined from the vector obtained as the learning result of step S330 (S340). For example, when the vector representing the learning result of the artificial neural network has dimensions by the number of selectable discrete all correct answer values, and the value of each dimension represents the selection probability of each correct answer, the dimension having the maximum value among the plurality of values The final correct answer value (correct answer candidate) 41 to 44 for each partial area can be determined through the correct answer determination method using the softmax function determined as the correct answer.

At this time, it is possible to generate (determine) correct answer candidates (final correct answer value for each partial area) by the number of the sub-regions generated in step S320. For example, as four sub-regions 21 to 24 are generated in FIG. 2, four correct answers 41 to 44 can be generated.

The final correct answer 51 is selected from the correct answers candidates 41 to 44 determined in step S340 (S350). At this time, the final correct answer can be selected by selecting a correct answer candidate having the highest importance depending on the importance of each sub-region, which is most frequently displayed or arbitrarily given. Alternatively, a voting algorithm or an ensemble model can be applied to select the final correct answer.

For example, in order to select the final correct answer from the correct candidates, the frequency with which each candidate candidate appears as a candidate, the probability that the neural network of each sub-region generates a candidate, the prior probability of each sub- Prior probabilities are used to select the final correct answer, or the ensemble model is used to select the final correct answer through widely known classifiers.

As described above, according to the embodiment of the present invention, by using the convolution method in the final correct answer determination process of the artificial neural network, it is possible to learn a wide range of inputs by using a separate neural network having a low dimension so that data sparseness do.

In addition, according to the embodiment of the present invention, an artificial neural network is constructed for sub-regions of input in the step of finally determining the correct answer in the artificial neural network, the correct answer candidates are directly determined at the output of each neural network, By determining the final candidate (final correct answer) from the candidates, it is possible to make error correction in a way that generates one correct answer from the entire input, and even if the correct answer decision from the local patterns is not correct, The artificial neural network can mitigate the difficulty of learning the values that are not the maximum value in the probability vector for determining the correct answer, thereby estimating the best correct answer prediction result through the patterns that can be observed even in the case of the prediction failure .

In addition, according to the embodiment of the present invention, it is possible to learn not only the performance improvement in the final layer but also the most abstract patterns independently appearing in position, thereby reducing the number of stacking required or reducing data, Can improve performance.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It is to be understood that the invention may be embodied in other specific forms. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. The scope of the present invention is defined by the appended claims rather than the detailed description, and all changes or modifications derived from the scope of the claims and their equivalents should be construed as being included within the scope of the present invention.

100: Final correct decision device
110: input unit 120:
130: learning unit 140:
150:

Claims (1)

Receiving vectors having arbitrary dimensions;
Dividing the input vectors into sub-regions of a predetermined size;
Learning an artificial neural network for each of the partial areas;
Determining a plurality of correct answers candidates from the vectors obtained as the learning result;
Selecting a final correct answer from the plurality of correct answers;
And determining a final correct answer using convolution in the artificial neural network.

Images