JP2012018450A - Neural network system, construction method of neural network system and control program of neural network system - Google Patents

Abstract

【Task】
Providing a neural network system that can output the accuracy of estimation along with the estimation results in an expert system that uses a neural network that performs estimation processing such as estimating disease from the history or medical examination information and predicting changes in the situation. To do.
[Solution]
Generate multiple neural networks with different components such as the number of nodes in the input layer, the number of intermediate layers, the number of nodes constituting the intermediate layer, the initial value of each weighting factor, and the learning method (propagation function between nodes) The same learning process is performed on them. Inferring unknown patterns with the above-mentioned multiple neural networks, and outputting the certainty of the judgment result numerically from the variation in the inference [selection figure]

Description

  The present invention relates to a neural network system including a plurality of neural networks, for example, a neural network system using a neural network that estimates a disease from medical history information and medical examination information and predicts a change in the situation, and A network system, its construction method, and a control program are provided.

Neural network technology is used as a means for estimating causes from outside phenomena and predicting future transition in dealing with human diseases and plant failures. A neural network is composed of a plurality of neuron circuits. The neuron circuit is a circuit having multiple inputs (X ₁ , X ₂ ,..., X _k ) and one output (y) as illustrated in FIG. Each of the multiple inputs (X ₁ , X ₂ ,..., X _k ) is associated with a weight coefficient 113 ( W ₁ , W ₂ , .W _k ). The product of (X _i ) and the weighting factor ( W _i ) is obtained. The calculation results of the calculation units 112 are added by the addition circuit 114, normalized by the threshold means 115 such as a sigmoid function or a snake side function, and output 117 (y).
FIG. 7B is an example of a neural network, and is configured by combining a plurality of neuron circuits shown in FIG. The neural network includes an input layer 152 to which a set of measurement data such as diagnostic data [X ₁ , X ₂ ,..., X _p ] 151 is input, and one or more outputs 155 corresponding to disease names and the like. Output layer 154, and at least one intermediate layer 153 located between the input layer 152 and the output layer 154.

The i-th node (i) of the input layer 152 distributes the input (X _i ) to the nodes of the intermediate layer 153. The nodes constituting the intermediate layer 153 and the output layer 154 are the neuron circuit shown in FIG. 7A, which processes the signal from the preceding node according to the above-described operation and transmits it to the succeeding node or outputs it from the output terminal. .
In order to make a diagnosis using a neural network, for example, for each symptom or health check item (fever, cough, headache, blood pressure, etc.), a data string corresponding to the item is input pattern [X ₁ , X ₂ ,. .., X _p ] are input to the input layer 152. Further, a cause (such as a disease name) to be estimated is assigned to each node of the output layer 154. For example, “cold” is assigned to output 1, “food poisoning”, etc. are assigned to output 2. In this case, when an input pattern representing a “cold” symptom is input from the input unit 151, “1” is output only from the first node to which “cold” is assigned, and other causes (disease names) are assigned. The neural network is trained so that “0” is output from the node that is set.

FIG. 8 is a diagram showing an outline of neural network learning. In the figure, there is only one node in the output layer. For example, when the input pattern is recognized as “cold”, “1” means YES, and when it is not recognized as “cold”, it means NO. Learning to output “0”. A large number of patterns such as symptoms recognized as “cold” and patterns such as symptoms not recognized as “cold” are prepared as learning patterns 161. In addition, an output of the correct answer, that is, an output “A _Z (“ 1 ”or“ 0 ”)” indicating “cold” or “not cold” is added to each learning pattern as correct data 162. Each weighting coefficient of the neuron circuit before starting learning is set to a predetermined initial value or a random value. The factor [A ₁ ] of the learning pattern 161 is input to the input unit X ₁ of the neural network, the factor [A ₂ ] is input to the input unit X _2, and each factor constituting one learning pattern 161 is input to the corresponding input unit of the neural network. input. Each element of the input learning pattern 161 is sequentially calculated by the neuron circuits of the intermediate layer 152 and the output layer 154, and outputs an output signal [Z] to the output terminal 155.

The comparison circuit 163 compares [Z] output to the output terminal 155 with the correct answer data [A _Z ] added to the learning pattern 161. The weight changing means 164 changes the value of the weight coefficient of each neuron circuit according to a predetermined method so that the output [Z] and the correct answer data [A _Z ] match. The learning operation described above is performed on each prepared learning pattern, the weighting coefficient that outputs a correct value for all the learning patterns is determined, and the learning ends. When an unknown symptom pattern is input to the learned neural network, the value of each factor of the symptom pattern is calculated according to the weighting coefficient of each neuron circuit, and whether or not it corresponds to “cold” is determined as “YES” ”Or“ NO ”is output from the output layer 154. The following are known documents related to neural networks and their learning.

JP-A-2005-319301 JP-A-2005-293241

The conventional neural network shown in FIG. 7 and FIG. 8 or the neural network described in the prior art document etc. is data indicating whether or not an input pattern such as an input measurement value corresponds to a specific event. Is output. For example, a neural network that has been learned to predict whether or not diabetes will develop in the next few years from input medical checkup data is “becoming diabetic” or “not diabetic” for the medical checkup data to be determined. The prediction result of is output.
However, the risk of developing a specific disease is thought to change continuously. In addition, when preparing a preventive measure for a disease, it is created taking into consideration the degree of risk, such as whether the risk of developing the disease is very high or the risk is low. However, as described above, the neural network only outputs two values, “onset” and “not onset”, and has only a limited significance in creating a diagnosis or countermeasure.
In the present invention, for example, when predicting the onset of diabetes, a neural network capable of presenting the possibility of onset by numerical values, such as “the possibility of becoming diabetic is 70%” and “the possibility of becoming diabetic is 30%”. Is to provide.

  The configuration of the neural network is determined by factors such as the number of nodes in the input layer, the number of intermediate layers, the number of nodes constituting the intermediate layer, the initial value of each weighting factor, and the learning method (propagation function between nodes). It is possible to generate a neural network having various configurations depending on the difference of elements. Even in the case of neural networks having different constituent elements, it is often possible to provide a neural network that outputs the same determination result for an unknown pattern by performing the same learning on them. However, when an unknown pattern is located in the vicinity of the boundary between the pattern area determined as “YES” and the pattern area determined as “NO”, different neural networks with different components may output different determination results. The present invention makes it possible to output the certainty of the determination result as a numerical value by utilizing the above characteristics of the neural network.

  The present invention prepares a plurality of neural networks having different components learned by the same learning pattern, causes the plurality of neural networks to recognize a pattern to be determined, and determines the ratio of “YES” and “NO” as a determination result. In this configuration, the certainty of the determination result is output as a numerical value. When the present invention is used to determine one disease, an output such as “the probability of the disease is 70%” becomes possible. In addition, when there are a plurality of nodes in the output layer and each node is associated with a different disease and learned, the possibility of each disease can be digitized and output in the order of the highest possibility. .

The figure which shows one embodiment of the neural network which comprises the neural network system which concerns on this invention The figure which shows the operation | movement flow of one embodiment of the neural network system which concerns on this invention. The figure which shows the detail of the learning part in FIG. The figure which shows the detail of the verification part in FIG. The figure which shows the detail of the recognition part in FIG. Network configuration diagram of a neural network system according to the present invention Neural network and neuron circuit diagram Diagram showing the concept of neural network learning

FIG. 1 is a diagram showing an outline of a neural network constituting a neural network system according to the present invention. The intermediate layer of the neural network according to the present invention is not limited to a single layer, but will be described below as a single layer for the sake of simplicity. The neural network shown in FIG. 1 includes one input layer 152, one intermediate layer 153, and one output layer 154. Each term of the pattern [X ₁ , X ₂ ,..., X _i ,..., X _p ] composed of p factors is input from the input terminal 151 to each node of the input layer. Each node in the input layer 152 distributes input data to each node in the intermediate layer 153. A weight calculation unit exists in an information transmission path connecting each node of the input layer 152 and each node of the intermediate layer 153. For example, a weight calculator (W _ij ) is provided between the node (i) of the input layer 152 and the node (j) of the intermediate layer 153, and the data “X _i from the node (i) of the input layer 152 is provided. Is multiplied by a weight coefficient “ W _ij ” possessed by the weight calculation unit (W _ij ) and transmitted to the node (j).

The node (j) of the intermediate force layer 153 adds the information transmitted from the weight calculation units (W _1j ),..., (W _pj ), and normalizes it according to a predetermined method. ). The information transmission path connecting the node (j) of the intermediate force layer 153 and the node (z) of the output layer 154 is provided with a weight calculation unit (W _jz ), and data from the node (j) of the intermediate force layer 153 Is multiplied by a weight coefficient “ W _jz ” possessed by the weight calculator (W _jz ) and transmitted to the node (z) of the output layer 154. The node (z) adds the information transmitted from the weight calculation units (W _1z ),..., (W _mz ), normalizes it according to a predetermined method, and outputs it as a recognition result “Z”.
In the learning stage, the recognition result “Z” is input to the feedback means 162 and compared with the correct answer “A _Z ” shown in FIG. 8, and is used to correct the weighting factor of each weight calculator.

The neural network is represented by a function having p factors [X ₁ , X ₂ ,..., X _i ,..., X _p ] of the input pattern as variables and the output [Z] as a calculated value. In the neural network learning, a parameter (constant) of a function that outputs an intended value [Z] is selected from _p variables [X ₁ , X ₂ ,..., X _i ,. It can be said processing. Therefore, it is possible to generate an arbitrary number of different types of neural networks by realizing the function of the neural network by a computer program, and it is also possible to execute estimation processing using them in parallel. is there.

1 is a control unit that selects the number of nodes of the neural network and a feedback method for learning, and has the following functions.
1. 1. Selection of the number of nodes in the input layer 152 2. Selection of the number of nodes in the intermediate layer 153 3. Selection of feedback method (propagation function) Link load selection

[Selection of the number of nodes in the input layer 152]
As a plurality of factors ( _p in X _{1 to} X _{p in} FIG. 1) constituting the input pattern, there are a factor that is essential for estimation and a factor that can be arbitrarily selected not to use. The form control means 170 determines whether to use a selectable factor or not, and determines the input layer 152 at the input node corresponding to the required factor and the selected factor. With this function, it is possible to generate a plurality of forms of neural networks in which the number of nodes in the input layer 152 and the factors used are different.

[Selection of the number of nodes in the intermediate layer 153]
The number of intermediate layers of the neural network and the number of nodes belonging to each intermediate layer can be arbitrarily selected. In this embodiment, the number of nodes belonging to each intermediate layer is randomly selected from 0 to 25. When the number of nodes is determined, a signal path and a weight calculation unit (W _ij ) are set between each node (j) of the intermediate layer 153 and each node (i) of the input layer 152. In addition, an initial value of the weight coefficient ( W _ij ) of the weight calculation unit (W _ij ) is set.
A signal path and a weight calculation unit (W _jz ) are set between each node (j) of the intermediate layer 153 and the node (z) of the output layer 154, and an initial value of the weight coefficient ( W _jz ) of the weight calculation unit Set. The initial values of the weighting factors ( W _ij ) and ( W _jz ) are set at random. Since the number of nodes in the intermediate layer and the initial value of the weighting factor are set at random, it is possible to easily generate a large number of neural networks having different configurations.

[Select feedback method (propagation function)]
As a feedback form of a neural network, that is, a learning method, many methods such as a linear function and a radial function are known. In this embodiment, a linear function, radial function, general regression, feedback processing means (program) by error back propagation is provided, and when generating a neural network, one processing means is randomly selected from the feedback processing means, The feedback means 162 is configured.

[Select link load]
The determination of the link load is to determine the parameters of the above-mentioned propagation function, and there are many mathematical methods for the determination method. In the present embodiment, random selection was made from backpropagation (error back propagation method), conjugate gradient descent method, K-means algorithm (center assignment), nearest neighbor method, and pseudo inverse matrix (linear least square optimization). Based on the method, the parameters of the propagation function are determined.

Even if the neural networks have different configurations, a neural network that performs the same estimation operation can be obtained by learning them using the same learning pattern. When an unknown pattern with a high similarity to a typical pattern estimated to be "YES" or "NO" is input to a neural network with a different configuration that has undergone the same learning, many neural networks output the same estimation result To do.
However, if the unknown pattern is significantly different from any of the above typical patterns, that is, if the unknown pattern is located near the boundary of the region of the pattern that is estimated to be “YES” or “NO”, the neural network Different estimation results may be output. The ratio of “YES” (or the ratio of “NO”) to all estimation results represents the likelihood of the estimation results. For example, if 70 percent of multiple neural networks that are trained to predict the onset of diabetes predict that they will “become diabetic” and 30 percent predict that they will not be diabetic, 70 percent ".
In the present invention, a plurality of neural networks having different configurations are prepared and the same learning is performed, and unknown data is estimated by the plurality of neural networks that have been learned. From the ratio of “YES” or “NO” in the estimation result, the probability of the estimation result is digitized and output.

  FIG. 2 is a diagram showing an example of a construction process and an estimation process of the neural network system according to the present invention. In the present embodiment, M (for example, several ten thousand) neural networks having different configurations are generated, and the same learning is performed using the same learning pattern. The M neural networks that have finished learning are subjected to determination of a plurality of verification patterns, the accuracy rate of each neural network is obtained, and N (for example, 100) neural networks with high estimation accuracy are selected. ing.

The neural network generation unit 401 in FIG. 2 differs in configuration by randomly changing all or part of the number of nodes in the input layer, the number of nodes in the intermediate layer, the feedback method (propagation function), and the link load selection method. M neural networks are generated. The generated data defining each neural network is stored in the neural network storage unit 402 as a data block.
The learning pattern storage unit 403 stores a plurality of, for example, 200 pairs of learning patterns and their correct answer data used for neural network learning. When predicting the onset of metabolic syndrome after 6 years, the age, gender, body mass index, systolic blood pressure (SBP), diastolic blood pressure (DBP), blood, Glucose, neutral fat, HDL cholesterol, LDL cholesterol, AST value, ALT value, HMW-adiponectin, total adiponectin, glycated albumin, total cholesterol, free fatty acid, insulin, HOMA-IR, presence or absence of smoking, etc. Data indicating whether or not a person has been diagnosed with metabolic syndrome in a health examination six years later is stored in pairs.

The learning unit 404 learns M neural networks stored in the neural network storage unit 402 using pairs of learning patterns and correct data stored in the learning pattern storage unit 403. The learned neural network is stored in the learned neural network storage unit 405.
FIG. 3 is a conceptual diagram of a process of learning M neural networks using a learning pattern, which is executed after the process 421 of generating M neural networks. In step 421, M (432-1 to 432 -M) neural networks having different configurations are generated and stored in the neural network storage unit 402. Further, the storage unit 402 stores information relating to a feedback method and a link load selection method corresponding to the neural networks 432-1 to 432-M.
In step 422, the learning unit 404 learns the neural networks 432-1 to 432-M stored in the neural network storage unit 402.
The learning pattern 430 read from the learning pattern storage unit 403 is input to the neural networks 432-1 to 432-M. The output estimated by each neural network from the learning pattern 430 is compared with the correct answer 431 read out from the learning pattern storage unit 403 by the comparison units 433-1 to 433-M. The comparison result is fed back to each neural network via feedbacks 434-1 to 434-M and used to correct the weighting factor. The correction of the weighting factor is performed based on the feedback method and link load selection method stored in the neural network storage unit 402 corresponding to the neural networks 432-1 to 432-M.
The above operation is repeated until all learning of the neural networks 432-1 to 432-M is completed. The data of the neural networks 432-1 to 432-M for which learning has been completed is stored in the learned neural network storage unit 405.

The verification pattern storage unit 406 in FIG. 2 stores a pair of verification pattern and correct data. The verification pattern is composed of the same factors as the learning pattern. For example, the verification pattern can be created by using a part of data collected in a health checkup as a learning pattern and the rest as a verification pattern. The verification unit 407 inputs a verification pattern to each of the neural networks stored in the learned neural network storage unit 405, compares the output with correct data, and stores the correct answer rate in the verification result storage unit 407.
The selection unit 408 reads the correct rate of the M neural networks stored in the verification result storage unit 407, selects N with a high correct rate, and stores it in the selected neural network storage unit 409.

FIG. 4 is a conceptual diagram of processing for obtaining the correct answer rate of each neural network executed by the verification unit 407.
In step 441, the verification unit 407 reads the verification pattern 450 from the verification pattern storage unit 406 and inputs it to the neural networks 432-1 to 432-M stored in the learned neural network storage unit 405. The output of each neural network is compared with the correct answer 451 read from the verification pattern storage unit 405 by the comparison units 453-1 to 453-M. The comparison result is stored in the correct answer rate management means 454-1 to 454-M as correct / incorrect answer data. When the verification of all the verification patterns is completed, the correct rate management means 454-1 to 454-M calculate and store the correct rate from the stored correct / incorrect answer data, and end step 441.
In step 442, the selection unit 408 in FIG. 2 verifies the accuracy rate stored in the accuracy rate management means 454-1 to 454-M, selects N neural networks with high accuracy rates, for example, 100, and selects the selected neural network. Store in the network storage unit 409. When the process ends, the generation of the neural network according to the present invention ends.

The recognition unit 412 in FIG. 2 receives the data 410 to be determined read from an external device such as a computer keyboard or a data storage device via the determination pattern input unit 411 and stores it in the selected neural network storage unit 406. Input to the N neural networks. The accuracy calculation unit 413 receives N determination results output from the N neural networks, calculates a ratio of the determination result “YES” (or “NO”), and outputs the calculated ratio.
FIG. 5 is a conceptual diagram of processing for obtaining the recognition accuracy for the determination pattern executed by the recognition unit 412 and the accuracy calculation unit 413.
In step 461, the recognizing unit 412 inputs the determination pattern 470 received via the determination pattern input unit 411 to each of the neural networks 471-1 to 471 -N stored in the selected neural network storage unit 409. The result is transmitted to the accuracy calculation unit 413.
In step 462, the accuracy calculation unit 413 outputs the determination result ratio “YES” (or “NO”) as the determination accuracy 473.

Although the present invention is different in internal configuration, it is possible to estimate the cause of an unknown pattern by a plurality of neural networks learned by the same learning pattern, and to quantify and output the probability of the estimation together with the estimation result It is what.
The embodiment shown in FIG. 1 to FIG. 5 has a configuration in which there is one node in the output layer, and one event, for example, “becoming diabetic or not” is estimated. However, as shown in FIG. 7B, a configuration in which a plurality of nodes in the output layer are provided and different events are assigned to each node is also possible. For example, by assigning the disease A to the node 1 in the output layer, the disease B... To the node 2, the probability of the disease A is X percent and the probability of the disease B is Y percent from one determination pattern. It is possible to obtain a plurality of estimation results simultaneously.

The neural network system according to the present invention can be configured as one independent computer system, or can be configured as a network configuration connected by a LAN or the Internet. FIG. 6 shows an example in which the devices constituting the neural network system according to the present invention are connected by a LAN. One or a plurality of personal computers 301 (only one is shown in FIG. 6), a server 310, and other devices such as an inspection device 320 are connected by a LAN 300.
The personal computer 301 includes a processing device 302 that executes a program, a local file device 303 that stores a program or data executed by the processing device 302, an input device 304 such as a keyboard, a mouse, and a card reader, and an output such as a display device and a printer. A device 305 and the like.

The server 310 executes the instructed program and transmits / receives data to / from other devices connected to the network. The server 310 generates a neural network and stores programs and data necessary for learning. Data file device 312 for ANN generation, neural network storage unit 402 in FIG. 2, learned neural network storage unit 405, ANN file device 313 to be selected neural network storage unit 409, pattern file for storing learning pattern 430 and verification pattern 450 The apparatus 314 includes a console input / output device (not shown).
The program executed by the server processing unit 311 includes a program for generating a function representing a neural network corresponding to the generation unit 401 of the neural network shown in FIG. 2, a program for performing learning processing of a function corresponding to the learning unit 404, and verification. A selection program for selecting a function with a high accuracy rate from functions corresponding to the learning finished corresponding to the unit 407 and the selection unit 408, an inference program for performing an inference using the selected functions corresponding to the recognition unit 412 and the accuracy calculation unit 413, and the like There are programs. The LAN 300 is connected to a device such as an inspection device 320 and can be used from the personal computer 301.

When generating the neural network system, the server processing unit 311 is instructed to generate the neural network system from the input device 304 of the personal computer 301 or the console input device of the server 310. In the instruction, a learning pattern and a verification pattern file stored in the pattern file device 314 are specified together with an instruction regarding the performance of the neural network system.
The server processing unit 311 executes generation, learning, verification, and selection of a neural network in accordance with an instruction from the input device, and stores the selected neural network in the selected neural network storage unit 409 of the ANN file device 313.
When the user uses the neural network system from the personal computer 301, data to be determined using a keyboard or other input device 304 is transmitted to the server 310. The inference program of the server processing unit 311 executes a determination data determination process (412 in FIG. 2) using the neural network stored in the selected neural network storage unit 409, and the result is output from the output device of the personal computer 301. Output (413 in FIG. 2).

The present invention relates to an inference system used for medical diagnosis or the like using a neural network, and the system is constituted by a plurality of neural networks in which the same learning is performed, although the internal configuration is different.
In diagnosis and the like, information such as the degree of accuracy of diagnosis and the degree of risk of suffering from a disease is a major decision factor for treatment and the like. However, the prediction obtained from the conventional neural network is either “affected” or “not affected”, and it has been impossible to obtain the accuracy of the prediction, such as the probability of being affected.
In the present invention, prediction is performed using a plurality of neural networks, and the accuracy of prediction is obtained from the variation in prediction. The user can know not only the prediction of “YES” or “NO” but the accuracy of the prediction of “YES” or the probability of occurrence of the “YES” situation.

  In addition, as a configuration to select multiple neural networks, many neural networks with different numbers of input layer nodes, intermediate layer nodes, feedback methods (propagation functions), and link load selection methods are created for learning and verification. It is possible to adopt a configuration for selecting a neural network having high prediction performance. By adopting this configuration, it is possible to improve prediction performance and accuracy of prediction accuracy.

Claims (5)

N neural networks with different internal configurations (N is 2 or more),
Learning means for performing the same learning on the N neural networks;
Determination data input means for inputting data to be determined to the N neural networks for which learning has been completed;
Accuracy calculation means for calculating a specific determination ratio from the determination results of the N neural networks with respect to the data to be determined;
A neural network system comprising: output means for outputting the value calculated by the accuracy calculation means as the accuracy of the specific determination. The neural network system according to claim 1, wherein
Each of the plurality of neural networks has an input layer, an intermediate layer, and an output layer;
The learning is determined by the feedback propagation function and the link weight,
A neural network system, wherein an internal configuration differs depending on at least one of the number of nodes in the input layer, the number of intermediate layer nodes, the feedback propagation function, and the load selection method of the link. The neural network system according to claim 1 or 2, wherein
A neural network generating means for generating M (M is greater than N) neural networks having different internal configurations;
The learning means performs the same learning on the M neural networks,
A neural network system comprising selection means for selecting N neural networks having high estimation accuracy from the M neural networks having been learned. A neural network generation step for generating M neural networks having different internal configurations (M is greater than N);
A learning step for performing the same learning on the M neural networks;
A selection step of selecting N neural networks with high estimation accuracy from the M neural networks that have been learned;
A determination data input step of inputting data to be determined to the N neural networks;
A probability calculating step of calculating a specific determination ratio from the determination results of the N neural networks with respect to the data to be determined;
A method for controlling a neural network, comprising: an output step for outputting the value calculated in the accuracy calculation step as the accuracy of the specific determination. A neural network generation step for generating M neural networks having different internal configurations (M is greater than N);
A learning step for performing the same learning on the M neural networks;
A selection step of selecting N neural networks with high estimation accuracy from the M neural networks that have been learned;
A determination data input step of inputting data to be determined to the N neural networks;
A probability calculating step of calculating a specific determination ratio from the determination results of the N neural networks with respect to the data to be determined;
A control program for a neural network, which causes a computer to execute an output step of outputting the value calculated in the accuracy calculation step as the accuracy of the specific determination.

Images