JP2022129061A - Information processing system, information processing apparatus, information processing method, and program - Google Patents

Abstract

Kind Code: A1 An inference model is learned while ensuring privacy of medical data and confidentiality of the inference model. An information processing system (800) includes a first information processing device (900) operated by a teacher data provider and a second information processing device (1000) managed by an inference model provider. The first information processing device 900 includes a teacher data acquisition unit 902 that acquires teacher data, and the teacher data as a first part composed of an input layer of an inference model and an intermediate layer that is part of an intermediate layer. It has a first learning unit 903 that performs a first learning process by inputting to the model. The second information processing device 1000 performs the second learning process by inputting the output from the first learning process to the second partial model configured from a layer different from the first partial model in the inference model. and a second learning unit 1002 that performs [Selection drawing] Fig. 8

Description

The present invention relates to an information processing system, an information processing apparatus, an information processing method, and a program for performing learning processing on an inference model.

Machine learning technology is applied to medical data such as medical images acquired by medical imaging equipment (modality) and medical information acquired from medical information systems, and inferences about specific diseases (disease detection, benign/malignant discrimination, prognosis prediction) , risk prediction, etc.) are known.

Non-Patent Document 1 discloses a technique of learning a model consisting of an encoder/decoder using medical images as an input, and performing feature extraction using an inference model (encoder).

Behzad Bozorgtabar et al. , "SALAD: Self-Supervised Aggregation Learning for Anomaly Detection on X-RayS", International Conference on Medical Image Computing and Computer-ASSiSted Intervention-MICCAI20.

In the technique described in Non-Patent Document 1, learning of an inference model is performed by an information processing device on the side of the inference model provider. It is difficult to protect privacy because it is necessary to transmit to the information processing device of the third party. On the other hand, if the inference model is stored in an information processing device managed by the user of the inference model, it prevents the user from duplicating the inference model and distributing it to a third party, or from illegally using the inference model by altering it. Since it is difficult to do so, the confidentiality of the inference model cannot be guaranteed.

An object of the present invention is to provide an information processing system capable of learning an inference model while ensuring the privacy of teacher data and the confidentiality of an inference model.

To solve the above problems, an information processing system according to one aspect of the present invention includes: a first information processing device operated by an administrator of training data; and two information processing devices, and performing a learning process for learning an inference model based on a neural network composed of an input layer, an intermediate layer and an output layer, wherein the first information processing device is a teacher data acquisition unit that acquires teacher data; and inputting the teacher data to a first partial model composed of the input layer of the inference model and an intermediate layer that is part of the intermediate layers. The second information processing device has a first learning unit that performs a first learning process by using and a second learning unit that performs a second learning process by inputting to a second partial model composed of layers.

According to the present invention, the inference model can be learned while ensuring the privacy of the teacher data and the confidentiality of the inference model.

1 is a diagram showing the configuration of an information processing system according to a first embodiment; FIG. Schematic diagram of an inference model according to the first embodiment. 2 is a diagram showing the hardware configuration of the first information processing apparatus according to the first embodiment; FIG. FIG. 4 is a flowchart showing inference processing of the information processing system according to the first embodiment; The figure which shows the structure of the information processing system which concerns on 2nd Embodiment. Schematic diagram of an inference model according to the second embodiment. FIG. 11 is a flowchart showing inference processing of the information processing system according to the second embodiment; The figure which shows the structure of the information processing system which concerns on 3rd Embodiment. Schematic diagram of an inference model according to the third embodiment. FIG. 11 is a flowchart showing learning processing of the information processing system according to the third embodiment; Schematic diagram of an information processing system according to a fourth embodiment. Schematic diagram of an inference model according to the fourth embodiment. The flowchart which shows the learning process of the information processing system which concerns on 4th Embodiment.

The present invention can be preferably applied to medical data such as raw data (signal data) acquired by a modality and diagnostic medical image data generated by image reconstruction from raw data. Modalities include, for example, X-ray CT devices, MRI devices, SPECT devices, PET devices, and electrocardiographs. The inference target data and teacher data may be not only medical data but also information related to patient privacy, such as age, gender, and disease information.

An inference process based on an inference model in the information processing system of the present invention will be described below in terms of a first embodiment and a second embodiment. In the third and fourth embodiments, the inference model learning process in the information processing system of the present invention will be described. As will be described later, the inference model used in the inference process is not limited to the inference model generated through the learning process of the third and fourth embodiments of the present invention. The inference model used in the inference process is a trained inference model that has been trained based on machine learning or deep learning by a known technique or the learning process described in the present invention. Here, the learned inference model may be a learned inference model that has been subjected to a learning process that satisfies a predetermined condition. good. Therefore, as additional learning of an inference model that has already been trained by a known method, a learning process may be performed in a learning process described later, or the learning process may be performed in reverse order.

Preferred embodiments of the present invention will now be described with reference to the accompanying drawings.

[First embodiment]
Here, the information processing system 1 of the present invention will be explained using FIG. An information processing system 1 according to the present invention comprises a first information processing device 2, a second information processing device 3, and a network 4 that communicably connects the two information processing devices.

Here, the configuration of the information processing system 1 according to the present invention will be described with reference to FIG. The information processing system 1 comprises a first information processing device 2 and a second information processing device 3 . The first information processing device 2 and the second information processing device 3 each have a partial model that is a part of an inference model that performs inference processing on medical data to be inferred and outputs an execution result. there is The inference model here is a trained model based on a neural network composed of an input layer, an intermediate layer, and an output layer. Through learning processing, parameters for outputting an inference result are determined, and a model in which the parameters and the network model are paired is defined as an inference model.

The first information processing device 2 performs a first inference using a first partial model composed of an input layer and at least a part of the intermediate layers among the learned inference models described above. The second information processing device 3 performs the second inference using the second partial model configured from a layer different from the first partial model among the learned inference models described above. Run. The configuration of each information processing device will be described below.

Here, the first information processing device 2 is an information processing device that can be operated by, for example, a medical worker who is a user of an inference model who has authority to manage medical data to be inferred. On the other hand, the second information processing device 3 is an information processing device owned by a model provider who has the authority to manage the inference model used for inference. The second information processing device 3 resides in a server outside the first information processing device 2 and is configured to be communicable via the network 4 .

The first information processing device 2 includes an acquisition unit 11 that acquires medical data to be inferred, and a learned neural network that is configured from an input layer, an intermediate layer, and an output layer that performs inference processing on the medical data. of the inference model, a first partial model composed of an input layer and at least a partial intermediate layer of the intermediate layers is used to perform the first inference processing on the inference target medical data and the first inference result of the first inference processing is transferred to a first inference unit 12 configured from a layer different from the layer constituting the first partial model of the inference processing. and an output unit 13 for outputting to the second information processing device 3, which is another information processing device having two partial models.

The first information processing device 2 also has a storage unit 10 that stores a first partial model, which is a part of a learned inference model, and medical data to be inferred. It also has an inference result obtaining unit 14 for obtaining a second inference result by the second information processing apparatus 3, which is another information processing apparatus, and a display control unit 15 for displaying the obtained inference result on a display device. .

(storage unit 10)
The storage unit 10 stores a first partial model including an input layer of a learned inference model and medical data to be inferred. The storage unit 10 associates and stores a network corresponding to the partial model and a learned parameter corresponding to the network as a first partial model. In addition, medical data to be inferred may be medical data automatically transferred from a modality or an external image server. Note that the part of the trained inference model refers to a continuous part from one layer to another layer, but is not limited to this, a continuous part from one neuron to another neuron, or an isolated neuron There may be. Also, the partial model may be a plurality of non-adjacent parts in the learned inference model.

(Acquisition unit 11)
The acquisition unit 11 acquires inference target medical data from the storage unit 10 and transmits the acquired inference target medical data to the first inference unit 12 .

(First reasoning unit 12)
The first inference unit 12 acquires the first partial model from the storage unit 10 and performs first inference using the first partial model on medical data to be inferred. A first inference result from the first partial model is sent to the output unit 13 . In this embodiment, the first partial model is a first partial model composed of an input layer and at least a part of the intermediate layers among the learned inference models, and the output unit 13 sends the output from the hidden layer. Here, the output of the hidden layer is tensor information, and if the inference model is a CNN-based model, the output is a feature map.

(Output unit 13)
The output unit 13 transmits the first inference result to the second information processing device 3, which is another information processing device. When there are a plurality of inference models, the information of the inference model corresponding to the partial model used in the first inference is output to the second information processing device.

(Inference result acquisition unit 14)
The inference result acquisition unit 14 acquires the inference result of the second inference processing for the inference target medical image data from the second information processing device 3 . After acquiring the inference result, the inference result acquisition unit 14 transmits the inference result to the display control unit 15 .

(Display control unit 15)
The display control unit 15 controls display of the inference result acquired by the inference result acquisition unit 14 on the display device. The display device is a display attached to the information processing device, a mobile terminal of a hospital official via an external server, or the like.

Here, the first information processing device 2 may be configured by a computer including a processor, memory, storage, and the like. In this case, by loading the program stored in the storage into the memory and executing the program by the processor, the storage unit 10, the acquisition unit 11, the first inference unit 12, the output unit 13, the inference result acquisition unit 14, Functions and processes such as the display control unit 15 are realized. However, not limited to this configuration, for example, all or part of the configuration of the first information processing device 2 may be realized by a specially designed processor (such as ASIC) or FPGA. Alternatively, part of the arithmetic processing may be executed by a processor such as GPU or DSP. Further, the first information processing device 2 may be composed of a single piece of hardware, or may be composed of a plurality of pieces of hardware. For example, using cloud computing or distributed computing, a plurality of computers may work together to realize the functions and processing of the first information processing device 2 .

FIG. 3 shows an example of a specific configuration of the first information processing device 2. As shown in FIG. In this example, the local information processing device 2 has a CPU 20 , a GPU 21 , a RAM 22 , a ROM 23 and a storage device 24 , which are connected by a system bus 25 . A display device 26 and an input device 27 such as a mouse and a keyboard are connected to the local information processing device 2 .

Since the first information processing device 2 is configured as described above, the user of the inference model who is the administrator of the medical data does not need to transmit the medical data to be inferred to the external information processing device. , can protect the privacy of medical data. Also, the provider of the inference model can secure the confidentiality of the inference model by installing only a part of the inference model in the first information processing device 2 .

On the other hand, the second information processing device 3 exists in a server outside the first information processing device 2, and performs inference processing on medical data. Using a first partial model composed of an input layer and at least a part of intermediate layers among intermediate layers, of the learned inference model based on the learned inference for the medical data to be inferred Using the result of the first inference process in the model as an input, the second inference process is performed using the second partial model composed of a layer different from the layer constituting the first partial model in the inference process. It is composed of a second inference unit 71 that performs processing. The second information processing apparatus also has a storage unit 70 for storing 3 second partial models.

In this embodiment, the second partial model is composed of an intermediate layer different from the intermediate layer forming the first inference model, and an output layer, among the trained inference models. Note that the network configuration of the partial models described above is an example, and the number of partial models and the number of information processing apparatuses can be changed as appropriate.

(storage unit 70)
The storage unit 70 associates and stores a network corresponding to the second partial model and a learned parameter corresponding to the network. A partial model refers to a continuous portion from one layer to another layer, but is not limited to this, and may be a continuous portion from one neuron to another neuron or an isolated neuron. A partial model may also be a plurality of non-adjacent parts in an inference model.

(Second reasoning unit 71)
The second inference unit 12 acquires the second partial model from the storage unit 10 and performs second inference using the second partial model on medical data to be inferred. Then, the inference result of the second inference is transmitted to the first information processing device 2 . Here, when the selection information of the learned inference model is acquired from the first information processing device 2, the second partial model corresponding to the corresponding learned inference model is acquired, and the second inference model is acquired. to implement. Here, in the present embodiment, since the second partial model is a model having an output layer, the inference result is output in the second information processing device. On the other hand, the second partial model may consist only of the intermediate layer and the output of the intermediate layer may be sent to the first information processing device 2 .

The second information processing device 3 is configured as described above, so that the provider of the inference model can install only a part of the learned inference model in the first information processing device 2 and use the learned inference model. A part of the inference model can be kept in the information processing device 3 owned and managed by itself, and the confidentiality of the inference model can be secured.

By the information processing system composed of the first information processing device 2 and the second information processing device 3 described above, while protecting the privacy of the inference target medical data, inference can be performed on the inference target medical data. The confidentiality of the inference model to be performed can be secured.

Here, the inference process by the information processing system 1 in this embodiment will be described with reference to FIG.

In step S40, the acquisition unit 11 in the first information processing device 2 acquires inference target medical data. After acquiring the inference target medical data, the acquisition unit 11 transmits the acquired inference target medical data to the first inference unit 12, and proceeds to the next step.

In step S41, the first inference unit 12 in the first information processing device 2 uses a first partial model composed of an input layer and at least a part of the intermediate layers to generate an inference target. medical data, the first inference processing is executed. After executing the first inference processing, the first inference unit 12 transmits the inference result of the first inference processing to the output unit 13, and proceeds to the next step.

In step S42, the output unit 13 in the first information processing device 2 outputs the inference result of the first inference to the second information processing device 2, and proceeds to the next step.

In step S43, the second inference unit 71 in the second information processing device 3 performs second inference processing using a second partial model configured from a layer different from the layer configuring the first partial model. to implement. In this embodiment, the second partial model includes an intermediate layer of the inference model that is different from the intermediate layer that forms the first partial model. The second partial model further has an output layer, and outputs an inference result for inference target medical data. The second inference unit 71 advances the processing to the next step after transmitting the inference result for the medical data to be inferred to the first information processing device 2 .

The second inference unit 71 may store the inference result of the second inference process in the storage unit 70 and present the inference result in response to access from the outside. Moreover, the output destination is not limited to the first information processing apparatus 2, and the result may be transmitted to a designated information terminal or a contact.

In step S<b>44 , the inference result acquisition unit 14 in the first information processing device 2 acquires the second inference result from the second inference unit 71 in the second information processing device 3 . After acquiring the second inference result, the inference result acquisition unit 14 transmits the second inference result to the display control unit 15, and proceeds to the next step.

In step S45, the display control unit 15 causes the display device 25 to display the inference result for the inference target medical data. The inference result displayed on the display device 25 is a series of inference processing (first inference processing and second inference processing).

The display control unit 15 may display the medical data to be inferred and the inference result in association with each other, or may display the information of the inference model used for the inference.

Therefore, the information processing system 1 can perform inference while protecting the privacy of inference-targeted medical data and ensuring the confidentiality of an inference model that performs inference on the inference-targeted medical data. can.

[Second embodiment]
In the first embodiment, of a trained inference model based on a neural network that performs inference processing on medical data and is composed of an input layer, an intermediate layer, and an output layer, the input layer and part of the intermediate layer and a second partial model composed of a layer different from the first partial model among the learned inference models. The information processing system comprising the second information processing device 3 having the .

In this embodiment, the first information processing device 2 further includes a third inference unit that performs inference using a third partial model including an output layer, and outputs to the first information processing device side. The configuration of an information processing system capable of An inference model in this embodiment will be described with reference to FIG.

It should be noted that differences from the first embodiment will be particularly described here, and overlapping portions will be omitted as appropriate.

An information processing system 1 according to the present embodiment includes a first information processing device 2, a second information processing device 3, and a network 4 that communicably connects the respective information processing devices, as in the first embodiment. be done. The information processing system 1 is also composed of three partial models.

The first information processing device 2 on the user side has an input layer of a trained inference model based on a neural network that performs inference processing on medical data and is composed of an input layer, an intermediate layer, and an output layer. and a part of the intermediate layers, and further a third partial model composed of a part of the intermediate layers and an output layer.

On the other hand, in the second information processing device 3 on the inference model manager side, among the intermediate layers of the inference model, between the intermediate layer of the first partial model and the intermediate layer of the third partial model has a second partial model consisting of at least part of the intermediate layer of Note that the number of partial models is variable, and the partial model having the input layer and the partial model having the output layer are the first information processing device 2 on the user side who has the authority to manage medical data to be inferred. should be prepared for As for the first information processing apparatus 2, the number thereof does not matter as long as the information processing apparatus is managed by the user.

With this information processing system 1, it is possible not only to secure the confidentiality of the inference model for the medical data while protecting the privacy of the inference target medical data, but also to output the inference results on the user side. Privacy regarding results can also be protected.

For example, in the case where the inference model implements high image quality, if the output layer is in an information processing device that is not on the user side, the image obtained by improving the image quality of the medical data to be inferred is sent to the second information processing device. 3, it is difficult to protect the privacy of medical data. Therefore, the first information processing device 2 has a partial model having an output layer, and the privacy of the output can be protected by performing inference using the partial model.

The information processing apparatus 1 according to this embodiment includes a storage unit 10, an acquisition unit 11, a first inference unit 12, an output unit 13, an inference result acquisition unit 14, a display control unit 15, and the third partial model described above. It has at least a third inference unit 51 that performs a third inference using

The information processing apparatus 2 according to this embodiment also includes a storage unit 70 and a second inference unit 71 . The second partial model used in the second inference unit 71 is an intermediate layer between the intermediate layer of the first partial model and the intermediate layer of the third partial model among the learned inference models. It consists of a network composed of layers.

The inference process in this embodiment will be described below with reference to FIG.

Since the flow up to step S42 is the same as that of the first embodiment, the explanation is omitted.

In step S<b>73 , the second information processing device 3 receives the inference result of the first inference unit 12 as input and performs second inference using the second partial model. The second partial model is composed of a network composed of intermediate layers between the intermediate layer of the first partial model and the intermediate layer of the third partial model among the learned inference models. When the second inference unit 71 transmits the output from the intermediate layer based on the second partial model as the result of the second inference to the first information processing device, the process proceeds to the next step.

In step S74, the third inference unit 51 in the first information processing device 2 receives the second inference result and makes a third inference using the third partial model. The third inference unit 51 transmits the result of the third inference to the inference result acquisition unit 14, and proceeds to the next step.

In step S<b>75 , the inference result acquisition unit 14 acquires the third inference result as the inference result for the inference target medical data, and transmits the acquired inference result to the display control unit 15 .

Although the explanation of step S45 is omitted, the inference result displayed on the display device 25 is a series of inference processes ( (first inference processing, second inference processing, and third inference processing).

According to this embodiment, while protecting the privacy of inference target medical data, it is possible not only to secure the confidentiality of an inference model for the medical data, but also to perform inference processing while protecting the privacy of inference results.

In the third and fourth embodiments, the inference model learning process in the present invention will be described below. Note that the inference model used in the inference process described above is not limited to the inference model generated through the learning process of the third and fourth embodiments. Further, the inference model in the following embodiments may be an inference model that has not undergone learning processing or a learned inference model that has undergone learning processing.

[Third embodiment]
In the third embodiment, a configuration for learning an information processing system 800 of the present invention will be described with reference to FIG. The information processing system 800 includes a first information processing device 900 that is an information processing device on the user side of the inference model, a second information processing device 1000 that is an information processing device on the side of the provider of the inference model, and an information processing device. It consists of a network 1100 connecting between them. A network configuration of an inference model corresponding to each information processing device will be described with reference to FIG.

The first information processing device 900 performs inference processing on medical data, and includes an inference model based on a neural network that is configured from an input layer, an intermediate layer, and an output layer. It has a first partial model configured with at least a part of the intermediate layer. The second information processing apparatus 1000 also has a second partial model having a second partial model composed of a layer different from the layer constituting the first partial model in the inference model. Also, in this embodiment, the second partial model is a partial model composed of part of the intermediate layer and the output layer. In this way, the first partial model including the input layer is provided in the first information processing device 900 on the user side, and the second partial model, which is a part of the inference model, is provided in the second model of the inference model provider. is provided in the information processing apparatus 1000, the learning process of the inference model can be performed while protecting the privacy of the medical data and securing the confidentiality of the inference model. Also, here, the first partial model may be configured as a network for public use, and the second partial model as a network for confidentiality. The model provider can further enhance the confidentiality of the inference model by using a confidential network as the second partial model.

The first information processing device 900 has a storage unit 901 that stores teacher data and information on an inference model. Further, it includes a teacher data acquisition unit 902 that acquires teacher data from the storage unit 900 and a first learning unit 903 that learns the first partial model based on the acquired teacher data. Note that the storage unit 900 may be configured by a storage device or the like managed by the user of the inference model. Further, when the learning process of the first partial model is completed, the first learning unit 903 stores the information of the learned first partial model in the storage unit 901 .

The second information processing device 1000 has a storage unit 1001 that stores information on an inference model, and a second learning unit 1002 that learns a second partial model.

Here, the learning process performed by the learning unit refers to forward propagation of teacher data to the partial model, and updating of the parameters of the partial model using error information obtained by the error backpropagation method. The teacher data is composed of learning data and a correct label. The learning data is, for example, medical image data, and the correct label is information indicating an object appearing in the medical image data. Also, the correct label may be set as correct image data indicating what an object is captured in each pixel. Further, the first information processing device 900 transmits the model selection information to the second information processing device 900, so that when at least one of the first partial model and the second partial model exists in plural, Also in , an appropriate model can be selected.

An example of the learning process of the information processing system 800 according to this embodiment will be described below with reference to the flowchart of FIG.

In step S50, the teacher data acquisition unit 902 acquires from the storage unit 901 teacher data in which the learning data and the correct label are paired. The teacher data acquisition unit 902 transmits the information of the learning data to the first learning unit 903 and the correct label to the second information processing device 1000, and proceeds to the next step.

In step S<b>51 , the first learning unit 903 acquires the learning data transmitted from the teacher data acquisition unit 902 and the information of the first partial model from the storage unit 901 . Here, the first learning unit 903 may transmit information indicating the acquired first partial model to the second information processing apparatus 1000 .

In step S<b>52 , the second learning unit 1002 acquires the information of the second partial model from the storage unit 1001 and the information of the correct label from the teacher data acquisition unit 902 .

In step S53, the first learning unit 903 inputs the learning data to the first partial model, forward propagates it, and performs the first learning process, which is a part of the learning process. When the first learning process ends, the data generated by the first learning process, such as tensors, is sent to the second learning unit 1002 .

In step S54, the second learning unit 1002 inputs the parameters transmitted from the first learning unit 903 to the second partial model, forward propagates them, and executes the second learning process, which is a part of the learning process. do.

In step S55, the second learning unit 1002 compares the output of the second partial model by forward propagation of the second partial model including the output layer in the network configuration with the correct label, and uses the loss function to calculate the error Get information about Also, the second learning unit 1002 determines here whether or not the learning is completed. The second learning unit 1002 determines the end of the learning process depending on whether the calculated error information is less than a predetermined value or whether the learning process has been performed a predetermined number of times. When the second learning unit 1002 determines that the learning process has ended, the flow ends. On the other hand, when it is determined to continue the learning process, the process proceeds to step S56. Note that step S55 may be determined by the first learning unit 903 before starting the first learning process.

In step S56, the second learning unit 1002 updates the parameters of the second partial model based on the error information calculated in step S55. Here, parameters refer to weights and biases, for example. Then, the error information is transmitted from the intermediate layer close to the output layer side to the input layer side by back propagation (error back propagation method). When the second learning unit 1002 transmits the error information from the intermediate layer close to the input layer side constituting the second partial model to the first learning unit 903, the process proceeds to the next step.

In step S<b>57 , first learning section 903 updates the parameters of the first partial model based on the error information transmitted from second learning section 1002 . After updating the parameters of the first partial model, the process proceeds to step S53. Incidentally, as described in step S55, the first learning unit 903 may determine the end of the learning process at this timing.

The information processing system 800 according to the present invention is configured in this way, so that the inference model can be learned while ensuring the privacy of the medical data and the confidentiality of the inference model. Note that the number of partial models is not limited to two, and the present invention can be applied as long as the partial model including the input layer exists in the information processing device on the user side of the inference model. Also, when there are a plurality of inference models, information for selecting a partial model corresponding to each of the plurality of inference models may be transmitted from the first learning unit 903 to the second learning unit 1002 . A partial model may be selected by the user or may be selected by the information processing device according to input data.

[Fourth embodiment]
In the third embodiment, the first information processing device has a first partial model composed of an input layer and at least a part of the intermediate layers, which is different from the first partial model. The learning process in the information processing system in which the second information processing device has the second partial model composed of layers has been described.

In this embodiment, the first information processing apparatus further includes a third partial model including at least an intermediate layer different from the first partial model and the second partial model, and an output layer. Learning of the system will be explained with reference to FIG. Also, the network configuration of the inference model in this embodiment will be described with reference to FIG. Note that the description of the overlapping parts with the third embodiment will be omitted as appropriate.

An information processing system 1200 according to this embodiment includes a first information processing device 1300 that is an information processing device on the user side of an inference model, and a second information processing device 1400 that is an information processing device on the side of the provider of an inference model. and a network 1100 connecting the information processing apparatuses.

The first information processing device 1300 performs inference processing on medical data, and in an inference model based on a neural network composed of an input layer, an intermediate layer, and an output layer, the input layer and the intermediate layer It has a first partial model configured with at least a part of the intermediate layer. Furthermore, the first information processing device 1300 has a third partial model of the inference model that includes at least the output layer.

The second information processing device 1400 has a second partial model composed of at least a part of the intermediate layers of the inference model.

According to the configuration of this embodiment, the first information processing apparatus 1300 has a third partial model that further includes an output layer. Inference model learning processing can be performed without sending to 1400 . Furthermore, the provider of the inference model can secure the confidentiality of the inference model by having a second partial model including at least a part of the intermediate layers constituting the inference model. In this embodiment, if the first information processing apparatus 1300 has a partial model including an input layer and a partial model including an output layer, the number of partial models, the number of information processing apparatuses, and the like can be appropriately designed. be.

The first information processing device 1300 has a storage unit 1301 that stores teacher data and information on an inference model. Also, a teacher data acquisition unit 902 that acquires teacher data from the storage unit 1301, a first learning unit 1303 that learns the first partial model based on the acquired teacher data, and a third model that learns the third partial model. and three learning units.

The second information processing device 1400 includes a storage unit 1001 that stores information on an inference model, and a second learning unit that learns a second partial model.

Here, the learning process means forward propagation of the learning data that constitutes the teacher data to the partial model, and back propagation of the error information between the correct label and the output value from the output layer (error backpropagation method). shows a series of processes for updating the parameters of The training data consists of learning data and correct labels.

An example of the learning process of the information processing system 800 according to this embodiment will be described below with reference to the flowchart of FIG. 13 . Here, especially the difference from FIG. 10 will be explained.

In step S<b>130 , the third learning unit 1304 acquires third partial model information from the storage unit 1301 and correct label information from the teacher data acquisition unit 1302 .

In step S132, the second learning unit 1402 inputs the parameters transmitted from the first learning unit 903 to the second partial model, forward propagates them, and performs the second learning process, which is a part of the learning process. do.

In step S133, the third learning unit 1304 inputs the parameters transmitted from the second learning unit 1402 to the third partial model, forward propagates them, and executes the third learning process, which is a part of the learning process. do.

In step S134, the third learning unit 1304 compares the output of the third partial model by forward propagation of the third partial model including the output layer in the network configuration with the correct label, and uses the loss function to calculate the error Get information about Also, the third learning unit 1304 determines here whether or not the learning is completed. The third learning unit 1304 determines completion of the learning process based on whether the calculated error information is less than a predetermined value or whether the learning process has been performed a predetermined number of times. When the third learning unit 1304 determines that the learning process has ended, the flow ends. On the other hand, when the third learning unit 1304 determines to continue the learning process, the process proceeds to step S53. Note that the end determination of the learning process in step S134 may be determined by the first learning unit 1303 before the start of the first learning process.

In step S135, the third learning unit 1304 updates the parameters of the third partial model based on the error information calculated in step S134. Here, parameters refer to weights and biases, for example. The third learning unit 1304 transmits the error information from the intermediate layer close to the output layer side to the input layer side by backpropagation of the error information. When the third learning unit 1304 transmits the error information from the intermediate layer close to the input layer side constituting the third partial model to the second learning unit 1402, the process proceeds to the next step.

In step S<b>136 , second learning section 1402 updates the parameters of the second partial model based on the error information transmitted from third learning section 1304 . The second learning unit 1402 transmits the error information from the intermediate layer close to the output layer side to the input layer side by backpropagation of the error information, and transfers the output from the intermediate layer close to the input layer side to the first After transmitting to the learning unit 1303, the process proceeds to the next step.

In step S<b>137 , first learning section 1303 updates the parameters of the first partial model based on the error information transmitted from second learning section 1402 . After updating the parameters of the first partial model, the process proceeds to step S53. Incidentally, as described in step S134, the first learning unit 1303 may determine the end of the learning process at this timing.

The information processing system 1200 according to the present invention is configured in this way, so that the inference model learning process can be performed while ensuring the privacy of the medical data and the confidentiality of the inference model. Furthermore, it is not necessary to transmit the learning data and the correct label that constitute the training data from the information processing device on the user side of the inference model, so that the confidentiality of the medical data can be further ensured.

Note that the inference model learned by the third and fourth embodiments may be used as the inference model that performs the inference processing in the first and second embodiments. Also, the third and fourth learning processes are effective as a technique for additional learning of an inference model that performs inference processes.

[Modification 1]
This modification can be applied to the third embodiment and the fourth embodiment. In the learning process described above, the learning process for updating the parameters of the partial models that make up the inference model has been described, but additional learning may be performed for the purpose of updating the parameters of only some of the partial models. For example, fixing the parameters of the second partial model and updating the parameters of the first partial model, or fixing the parameters of the first partial model and updating the parameters of the second partial model. be done.

If the first learning unit 903 updates the parameters of the first partial model and the second learning unit 1002 does not update the parameters of the second partial model, the first partial model can be customized according to the data you want to input. For example, the first learning unit 903 performs additional learning specialized for the characteristics of the modality for acquiring medical data, a specific imaging range, etc. for the first partial model. This has the effect of preventing unintended modification of the model.

When the second learning unit 1002 updates the parameters of the second partial model and the first learning unit 903 does not update the parameters of the first partial model, the first model under the control of the provider of the inference model It can be expected to improve the accuracy and robustness of the second partial model.

Further, by the additional learning described above, a plurality of first partial models and second partial models may be created, or a plurality of partial models may be appropriately combined and used.

[Modification 2]
In the learning process described above, the case where the partial model configured including the output layer is in the first information processing device and the case where it is in the second information processing device have been described.

In this modified example, a configuration will be described in which whether the partial model including the output layer is to be placed in the information processing device on the user side of the inference model or in the provider side of the inference model is set according to the output of the inference model. .

For example, if the output of the inference model is equivalent to the input learning data, such as increasing the resolution of the input data, the inference model including the output layer is placed in the information processing device on the user side of the inference model. On the other hand, when the output is an inference model for classifying and detecting medical data, the inference model including the output layer is placed in the information processing device of the inference model provider. By constructing a partial model according to the output of the inference model, learning processing can be performed while balancing machine resources and confidentiality of medical data. It should be noted that constructing a partial model according to the output of the inference model is also effective in inference processing. For example, a first partial model that performs the first inference may be selected from a plurality of first models according to the medical data to be inferred, or a plurality of second partial models may be selected from the inference target medical data. A second classification model may be selected that makes a second inference depending on the medical data.

[Modification 3]
In the above-described learning processing, the case where the inference model learning processing is performed using the error backpropagation method has been described.

In this modified example, in the learning process of the inference model, a case will be described in which the learning process is performed by a learning method other than the error backpropagation method.

For example, a method such as Synthetic Gradient, which trains a model that estimates the expected gradient for each layer, a method such as Feedback Alignment, which uses a fixed random matrix when backpropagating the error, and a target rather than an error. A method such as Target Prop that propagates , or any other method may be used.

(Other examples)
The present invention is also realized by executing the following processing. That is, the software (program) that realizes the functions of the above-described embodiments is supplied to a system or device via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or device reads the program. This is the process to be executed.

Claims (21)

a first information processing device operated by a teacher data provider; and a second information processing device managed by an inference model provider capable of communicating with the first information processing device via a network; An information processing system that performs learning processing for learning an inference model based on a neural network composed of an input layer, an intermediate layer, and an output layer,
The first information processing device is
a teacher data acquisition unit that acquires teacher data;
first learning for performing a first learning process by inputting the teacher data into a first partial model composed of the input layer of the inference model and a partial intermediate layer of the intermediate layers; has a part
The second information processing device inputs the output from the first learning process to a second partial model configured from a layer different from the first partial model in the inference model, thereby obtaining a second and a second learning unit that performs a learning process of the information processing system. the first information processing device further has a third partial model including the output layer;
2. The information processing system according to claim 1, further comprising a third learning unit that performs a third learning process by inputting an output from the second learning process to the third partial model. . The third learning unit is characterized in that it acquires a correct label that constitutes the teacher data from the teacher data acquisition unit, and acquires error information based on the correct label and the output from the output layer. The information processing system according to claim 2, wherein: 4. The information processing system according to claim 3, wherein the third learning unit updates parameters of the third partial model based on error backpropagation of the error information. The second learning unit updates the parameters of the second partial model based on the error information transmitted from the third learning unit,
5. The information processing system according to claim 4, wherein the first learning unit updates parameters of the first partial model based on error information transmitted from the second learning unit. . In the second information processing device, the second partial model includes an output layer,
Acquiring a correct label constituting the teacher data from the teacher data acquiring unit in the first information processing device, and acquiring error information based on the correct label and the output from the output layer. 2. The information processing system according to claim 1. 7. The information processing system according to claim 6, wherein said second learning unit updates parameters of said second partial model based on said error information. the first partial model is a network for disclosure to a provider of the training data;
8. The information processing system according to any one of claims 1 to 7, wherein said second partial model is a network for confidentiality from a provider of said training data. 9. The second learning unit in the second information processing apparatus generates the second partial model by additional learning in which the parameters of the first partial model are fixed. The information processing system according to any one of . 10. The first learning unit in the first information processing apparatus generates the first partial model by additional learning in which the parameters of the second partial model are fixed. The information processing system according to any one of . 11. The information processing system according to claim 1, wherein said second information processing device is an information processing device operated by a provider of said inference model. 12. The information processing system according to any one of claims 1 to 11, wherein said teacher data is medical data. An information processing device operated by an administrator of training data, the information processing device comprising:
a teacher data acquisition unit that acquires teacher data;
Of a neural network-based inference model composed of an input layer, an intermediate layer, and an output layer, for a first partial model composed of the input layer and at least a portion of the intermediate layers , performs a first learning process by inputting the teacher data,
a first learning unit for outputting the inference model to another device having a second partial model having a second partial model composed of layers different from the layers constituting the first partial model;
An information processing device comprising: The information processing device further has a third partial model including the output layer,
and a third learning unit that performs a third learning process by inputting an output from a second learning process using the second partial model to the third partial model. Item 14. The information processing device according to item 13. The third learning unit acquires a correct label constituting the teacher data from the teacher data acquisition unit, acquires error information based on the correct label and the output from the output layer, 15. The information processing apparatus according to claim 14, wherein the parameters of the three partial models are updated. An information processing device operated by a provider of an inference model, the information processing device comprising:
An inference model based on a neural network that performs inference processing on medical data and is composed of an input layer, an intermediate layer, and an output layer. Using the output from the first learning process by another device having the first partial model for the second partial model composed of layers different from the layers constituting the first partial model An information processing apparatus, comprising: a second learning unit that performs second learning by using the second learning unit. 17. The information processing apparatus according to claim 16, wherein said second learning unit updates the parameters of said second partial model based on error information calculated by said other apparatus. a teacher data acquisition step for acquiring teacher data;
Of a neural network-based inference model composed of an input layer, an intermediate layer, and an output layer, for a first partial model composed of the input layer and at least a portion of the intermediate layers and a first learning step of performing a first learning process by inputting the teacher data. Of an inference model based on a neural network composed of an input layer, an intermediate layer, and an output layer, a first partial model composed of the input layer and at least a portion of the intermediate layers is constructed. An information processing method, comprising: a second learning step of performing second learning using an output from the first learning process on a second partial model composed of a layer different from the layer. A program for causing a computer to execute the information processing method according to claim 18 or 19. An information processing system comprising a first information processing device operated by a teacher data provider and a second information processing device capable of communicating with the first information processing device via a network,
The neural network comprises a first neural network and a second neural network in an intermediate layer, the first neural network is stored in the first information processing device, and the second neural network is the second neural network. is stored in the information processing device of
Learning processing is performed in the first neural network stored in the first information processing device and the second neural network stored in the second information processing device using teacher data. An information processing system characterized by

Images