TW202004572A - Learning device, learning method, program, learned model, and bone metastasis detection device - Google Patents

Abstract

A learning device (1) for generating a model of a neural network for detecting a bone metastasis region from a scintigram of a subject, the learning device (1) being provided with an input unit (10) for inputting, as teacher data, scintigrams of a plurality of subjects and correct answer labels of a bone metastasis region and a non-bone-metastasis region in the scintigrams, a patch image creation unit (13) for cutting out a region in which a bone of a subject appears from the scintigrams of the plurality of subjects and creating a patch image, and a learning unit (15) for learning a model of a neural network used to detect a bone metastasis region of a bone scintigram using the patch image and a correct answer label corresponding thereto as teacher data.

Description

Learning device, learning method, program product, memory medium with memory completed model and bone metastasis detection device

The invention relates to a technique for detecting a bone metastasis area from a subject's scintillation chart.

As a related study on the detection of bone metastasis on a bone scintillation chart, a study by Kazuyuki Kagami and others (Non-Patent Document 1) can be cited. In Non-Patent Document 1, after segmentation (bone classification of the whole body), in each bone region (8 regions), information of high concentration is detected using information such as average or standard deviation. In addition, in Non-Patent Document 2, a CAD (Computer Aided Diagnosis) system "BONENAVI version 2.1.7" (FUJIFILM RI Pharma Co., Ltd., Tokyo, Japan) is used by artificial neural network (ANN: Artificial Neural Networks) analyzed 225 prostate cancer patients (bone metastasis cases: 124 cases, normal cases: 101 cases) after analyzing bone scintillation images of whole-body images, and showed their analysis results. In the BONENAVI system, two imaging marks of ANN and BSI (Bone Scan Index) are output. The ANN value indicates the possibility of bone metastasis. The range of ANN is a continuous value of 0~1. "0" refers to the possibility that there is no bone metastasis. "1" refers to the high probability of bone metastasis. BSI represents the amount of metastatic tumors (the ratio of bone metastatic regions to the whole body bone). The detection result was that Sensitivity was 82% (102/124) and Specificity was 83% (84/101).

[Prior Technical Literature] [Non-patent literature]

[Non-Patent Document 1] Kawakami et al. "Introduction of Bone Scintigraphy Diagnostic Support Software "BONENAVI"" Journal of Nuclear Medicine, 63(0): 41-51, 2011

[Non-Patent Document 2] M, Koizumi, K. Motegi, M. Koyama, T. Terauchi, T. Yuasa, and J. Yonese, "Diagnostic performance of a computer-assisted diagnosis system for bone scintigraphy of newly developed skeletal metastasis in prostate cancer patients: search for low-sensitivity subgroups" Annals of Nuclear Medicine, 31 (7): 521-528, 2017

The bone metastasis detection support system on the bone scintillation diagram mainly includes the recognition processing of anatomical structures and the emphasis (feature extraction) or detection processing of abnormal parts. Eventually, these processes are aggregated to determine the site suspected of bone metastasis, and the result is presented to the doctor.

In the above-mentioned conventional technique, the bone metastasis area is displayed on the scintillation graph of the subject, but it is observed that there is a non-bone metastasis area (non-malignant lesion area such as fracture, inflammation, etc.) having a similar concentration value as the bone metastasis area. The detection is a region of bone metastasis (so-called "over-picking"), which reduces the detection accuracy. Therefore, the present invention provides a technique for maintaining the detection rate of the bone metastasis area and reducing excessive pickup.

One aspect of the present invention is a learning device that generates a model for neural networks such as bone metastasis detection from a subject's scintillation diagram; it includes: an input section that inputs a plurality of subjects' flickers The correct labels of the bone metastasis area and the non-bone metastasis area in the map and each scintillation map are used as guidance data; and the learning department uses the above guidance data to carry out a model of a neural network model for detecting the bone metastasis area of the bone scintillation map Learn.

Moreover, another aspect of the present invention is a learning method that generates a model for a neural network such as a bone metastasis region detected from a subject's scintillation graph; it has the following steps: input a plurality of subjects The scintillation map and the correct labeling of the bone metastasis area and non-bone metastasis area in each scintillation map as guidance data; and use the guidance data to learn the neural network model for detecting the bone metastasis area of the bone scintillation diagram Steps.

In addition, another aspect of the present invention is a program product that is used to generate a neural network model such as a bone metastasis region detected from a subject's scintillation graph; it performs the following steps: input a plurality of subjects The scintillation graph of the examiner and the correct labeling of the bone metastasis area and non-bone metastasis area in each scintillation map as the guidance data; and use the guidance data to carry out the neural network model for detecting the bone metastasis area of the bone scintillation diagram Steps of learning.

In addition, another aspect of the present invention is a memory medium with a learning completion model, which is used to make the computer function by detecting the bone metastasis area from the subject's scintillation diagram; it is composed of a neural network This type of neural network has a convolution layer and a deconvolution layer, and this type of neural network includes a structure in which the feature map obtained by the convolution layer is input to the deconvolution layer. The scintillation graphs of multiple subjects and the correct labels of the bone metastasis area and non-bone metastasis area in each scintillation graph are used as the guidance data for learning, which is based on the scintillation pattern of the subject input to the above neural network The method of detecting the bone metastasis area makes the computer function.

In this way, by using a neural network model of correct labeling of bone metastatic and non-bone metastatic regions, the model can be used to properly detect bone metastatic regions from the subject's scintillation graph.

1, 2, 3, 4 ‧‧‧ learning device

10, 21, 40, 51 ‧‧‧ input

11, 22, 41, 52 ‧‧‧ Control Department

12, 23, 44, 55 ‧‧‧ Concentration Standardization Processing Department

13, 24, 45, 56 ‧‧‧ patch image production department

14‧‧‧ Patch image reversal section

15, 46‧‧‧ Learning Department

16, 26, 47, 58 ‧‧‧ Memory Department

17, 27, 48, 59 ‧‧‧ output

18‧‧‧ Guidance Data Analysis Department

19‧‧‧ Patch image selection section

20、50‧‧‧Bone metastasis detection device

25、57‧‧‧Deduction Department

42、53‧‧‧Image reversal section

43, 54‧‧‧ Front and rear image alignment

A‧‧‧ Patch image

A’‧‧‧ Patch image

B‧‧‧ Patch image

B’‧‧‧ Patch image

C‧‧‧ Patch image

D‧‧‧ Patch image

R‧‧‧Region

FIG. 1 is a diagram showing the structure of a learning device according to the first embodiment.

Figure 2A is a diagram showing the subject's flashing diagram and the correct label.

FIG. 2B is a diagram showing an example of a patch image.

FIG. 2C is a diagram showing another example of the patch image.

Figure 3 is a diagram showing the structure of a neural network-like model.

FIG. 4 is a diagram showing the configuration of the bone metastasis detection device according to the first embodiment.

FIG. 5 is a diagram showing an example of a patch image cut out from a scintillation chart.

6 is a diagram showing the operation of the learning device according to the first embodiment.

Fig. 7 is a diagram showing the operation of the bone metastasis detection device according to the first embodiment.

8 is a diagram showing the structure of a learning device according to a second embodiment.

9 is a diagram showing the structure of a learning device according to a modification.

FIG. 10 is a graph (FROC, free-response receiver operating characteristic) of the relationship between Sensitivity and FP (P) obtained through experiments.

11 is a diagram showing the structure of a learning device according to a third embodiment.

12 is a diagram showing a scintillation graph of a subject input to the learning device of the third embodiment.

FIG. 13 is a diagram showing the configuration of a neural network model used in the learning device of the third embodiment.

Fig. 14 is a diagram showing the configuration of a bone metastasis detection device according to a third embodiment.

15 is a diagram showing the operation of the learning device according to the third embodiment.

Hereinafter, the learning device and the bone metastasis detection device according to the embodiment of the present invention will be described with reference to the drawings. In addition, in the following description, the numerical values described as conditions such as dimensions are only examples in a preferred form, and are not intended to limit the present invention.

The learning device of the embodiment is a model that generates a neural network model for detecting bone metastasis from a subject's scintillation chart, and is provided with: an input unit that inputs a plurality of subjects' scintillation charts and each of the scintillation charts The correct label of the bone metastasis area and the non-bone metastasis area is used as guidance data; and the learning department uses the guidance data to learn the neural network model such as the bone metastasis area for detecting the bone scintillation map. Here, although the non-bone metastasis area is an area having a similar concentration value as the bone metastasis area, it is an area that does not cause bone metastasis. Non-bone metastasis areas include non-malignant lesion areas (fractures, inflammation, etc.). Furthermore, the bone metastasis area is also called "abnormal accumulation".

In this way, by using a neural network model of correct labeling of the bone metastatic region and the non-bone metastatic region, the model can be used to properly detect the bone metastatic region from the subject's scintillation graph.

The learning device of the embodiment may also include a patch image creation unit that generates a patch image by cutting out the area of the subject's bone from the scintillation pictures of a plurality of subjects, and the learning unit uses the patch image and its corresponding The correct label is used as a guide for learning.

In the neural network-like model, the memory size required for learning increases as the image size becomes larger. According to the configuration of the embodiment, it is possible to reduce the memory size required for learning by generating a patch image obtained by cutting out the area shot to the bone of the subject and using the patch image for learning . In addition, since the appearance of the bone metastasis area is not so dependent on the shape of the organ, it is possible to learn even when the whole organ is not photographed. Therefore, the patch image is suitable for the guidance data for the learning of neural network models such as bone metastasis areas.

In the learning device of the embodiment, the patch image creation unit may scan a window of a predetermined size on the subject's scintillation graph, and cut out the window when the subject's bone is photographed in the window The area serves as a patch image. By scanning the window in this way and cutting out the patch image, the self-experienced scintillation picture cut out the patch image without omission.

The learning device of the embodiment may be included in the patch image created by the patch image creation unit to obtain a patch image including a bone metastatic region or a non-bone metastatic region and a region not including a bone metastatic region and a non-bone metastatic region A guide data analysis unit for the composition ratio of any patch image.

The inventors and others made inferences using models generated by learning from various guidance materials, and studied the conditions under which neural network models that can properly detect bone metastasis regions can be generated. As a result, it was found that the content of the patch image constituting the guidance material (the composition ratio of the patch image including the bone metastatic region or the non-bone metastatic region and the patch image not including either the bone metastatic region or the non-bone metastatic region) and The accuracy of the neural network model is related. According to the implementation form, the guidance materials can be adjusted in such a way as to enable proper learning by analyzing the guidance materials for learning and displaying the contents of the guidance materials.

The learning device of the embodiment may also include a method in which the composition ratio determined by the guidance data analysis unit is included in a predetermined range, and the bone image transfer region and the non-bone are not extracted from the patch image created by the patch image creation unit The patch image selection unit of the patch image of any one of the transition areas.

If the composition ratio of the patch image including the bone metastasis area is too small, the accuracy of the model obtained by learning may be deteriorated. Therefore, the composition ratio of the patch image including the bone metastasis area is adjusted by the configuration of the embodiment Get bigger.

The learning device of the embodiment may further include a patch image inverting unit that inverts at least a part of the patch image created by the patch image creation unit from left to right or upside down.

By inverting the patch image in this way, it is possible to increase the variation of the guidance data, thereby generating a model with higher accuracy through learning. Furthermore, in the case of inverting the patch image, the patch image after inversion can be used, or both the patch image after inversion and the patch image before inversion can be used as guidance materials.

In the learning device of the embodiment, the neural network may also include the following structure, that is, it has an Encoder-Decoder (encoder-decoder) structure, and the feature map obtained by the Encoder (encoder) structure is input to Decoder (decoder) structure.

According to this configuration, an encoder structure is used to capture the global features of the image, and the feature map obtained during the Encode process is input to the decoder structure, thereby also learning the local features. By capturing the spatial breadth of the bone metastasis area, the position information of the bone metastasis area can be properly obtained.

In the bone metastasis detection device of the embodiment, it may be a neural network including a first network part having an encoder-decoder structure and a second network part having an encoder-decoder structure. Structure, the input part inputs the scintillation pictures taken from the front and back and their correct labels for each subject, and the learning part inputs the scintillation pictures of the subjects taken from the front into the input layer of the first network part, and the second network part The input layer is input from the back to take a picture of the subject's flashing picture and learn.

In addition, in the bone metastasis detection device of the embodiment, the neural network may include a first network part having an encoder-decoder structure and a second network part having an encoder-decoder structure. The input unit inputs the scintillation pictures taken from the front and back and their correct labels for each subject, and the learning unit inputs the first cut out from the scintillation picture taken from the front of the subject to the input layer of the first network part. The patch image, and the second patch image corresponding to the first patch image cut out from the scintillation image of the subject taken from the rear is input to the input layer of the second network part to learn.

By using a neural network such as a structure combining two network parts with an encoder-decoder structure, the scintillation picture taken from the front and the scintillation picture taken from the rear are processed simultaneously rather than independently It can generate a neural network model that improves the recognition accuracy of bone metastatic regions and non-bone metastatic regions.

In the bone metastasis detection device of the embodiment, a non-malignant lesion area may be included in the non-bone metastasis area, and the input unit accepts a plurality of subjects with correct labels of each of the bone metastasis area and the non-malignant lesion area The scintillation graph is used as guidance data. The learning department uses the guidance data to learn a model of a neural network such as detecting each of the bone metastasis area and the non-malignant lesion area.

The bone metastasis detection device of the embodiment includes: a memory section having a learning completion model such as a neural network learned by the learning device; an input section which inputs a scintillation image of the subject; a patch image creation section , The patch image is made from the scintillation map; the inference section, which inputs the patch image to the input layer of the learning completed model read out from the memory section, and finds the bone transfer region contained in the patch image; and the output section, Its output represents data of bone metastasis area. According to this configuration, the detection rate of the bone metastasis area can be maintained, and excessive pickup can be reduced.

The program product of the embodiment can also be used to detect the bone metastasis area from the subject's scintillation diagram; it causes the computer to perform the following steps: the step of inputting the subject's scintillation diagram; the step of making the patch image from the scintillation diagram; The self-memory has the learning part of the learning completion model of a neural network such as that learned by the above learning device, reads the learning completion model, inputs the patch image to the input layer of the learning completion model, and finds the patch image Steps of bone metastasis area; and step of outputting data representing bone metastasis area.

The program product of the embodiment is used to detect the bone metastasis area from the scintillation diagram of the subject; it causes the computer to perform the following steps: input the steps of taking two scintillation pictures of the subject from before and after; A step reversed in the horizontal direction; self-memory has a learning part of the learning completion model pre-generated by learning using the guidance data to read out the learning completion model and input 2 flashing pictures to the input layer of the learning completion model, Steps to find the area of bone metastasis included in the scintillation diagram; and step of outputting data representing the area of bone metastasis. By inverting one of the two scintillation pictures taken from the front and back into the same direction in this way, inputting the two scintillation pictures to the input layer of the learning completed model for inference, it is possible to accurately detect the bone metastasis area.

Hereinafter, the learning device and the bone metastasis detection device of the embodiment will be described with reference to the drawings.

(First embodiment)

FIG. 1 is a diagram showing the structure of a learning device 1 according to the first embodiment. The learning device 1 of the first embodiment is a device that generates a neural network model for detecting a bone metastasis region from a scintillation graph of a subject through learning. The neural network model generated by the learning device 1 of the present embodiment is a model for classifying the scintillation graph area of the subject into three types of bone metastatic area, non-bone metastatic area, and background. In this embodiment, the category of non-bone metastasis areas includes not only non-malignant lesion areas, but also physiologically concentrated areas such as kidneys and bladder.

The learning device 1 has: an input section 10 that inputs guidance data; a control section 11 that learns a neural network-like model based on the guidance data; a memory section 16 that memorizes a model generated by learning; and an output section 17, It outputs the model stored in the memory section 16 to the outside.

FIG. 2A is a diagram showing an example of guidance data input to the input unit 10. Guidance materials include the subject's flashing pictures and the correct labels assigned to the subject's flashing pictures. In this example, the size of the flicker picture is 512×1024 [pixels]. The correct label is to specify for each pixel whether the attention pixel is a pixel corresponding to the cluster, or a background (other than cluster) pixel. In the case of pixels corresponding to aggregation, it is further specified which of the bone metastasis area, injection leakage or urine leakage, and non-bone metastasis area. Furthermore, injection leakage or urine leakage will be excluded from the detection target of the bone metastasis detection device 20 of this embodiment.

Next, the control unit 11 will be described. The control unit 11 includes a density normalization processing unit 12, a patch image creation unit 13, a patch image inversion unit 14, and a learning unit 15.

The concentration normalization processing section 12 has a function of normalizing the concentration value, which is used to suppress the unevenness of the concentration value of the normal bone area that differs for each subject. The concentration standardization processing unit 12 performs standardization of the concentration value through the process of concentration range adjustment, identification of normal bone level, and grayscale standardization. The adjustment of the density range is, for example, linear conversion such that the cumulative upper 0.2% pixel value of the density histogram other than the density value 0 of the input image becomes 1023, and the cumulative upper 98% pixel value becomes 0.

The identification of normal bone level is to use the multiple threshold method for the density histogram of the image after the density range adjustment except for the density value 0. The threshold value is set to 1% from the cumulative upper 1% to 25% of the pixel value of the histogram. After binarizing the image after the range adjustment in each threshold value, a 4-link mark is performed. According to the result, the area with an area of more than 10 [pixels] and less than 4900 [pixels] is selected. Secondly, the average concentration value in the calculated area is sorted in descending order, and the transition point (the boundary between the normal area and the abnormal area) is determined. The position where two consecutive average density values become less than 3% of the peak pixel is taken as the transition point. The peak pixel is the maximum value of the average density value of each area.

Then, in gradation normalization, the average value P of the average density values of five consecutive points including the transition point is obtained. Finally, normalization is carried out by multiplying the normalized coefficient F=k/P with the adjusted image of the density range. Here, the constant k is set to 358.4, but this value is determined by experiment (Ito Tatsuya "Development of Abnormal Clustering Detection Processing on Bone Scintillation Chart" Tokyo Agricultural University Bachelor Thesis, 2015).

The patch image creation unit 13 has a function of cutting out a scintillation image from the subject and creating a patch image. In this example, the size of the patch image is 64×64 [pixels]. Scan the 64×64[pixels] window at 2[pixels] intervals on the subject’s scintillation graph (512×1024[pixels]), and include the aggregation mark (bone metastasis area or non-bone metastasis area) in the (1) window Area), or (2) when the bone area is included and the aggregate is not included, the image patch is cut out. The patch image creation unit 13 cuts out image patches from the input bone scintillation pictures of the plurality of subjects. 2B and 2C are diagrams showing examples of image patches cut out from the scintillation graph of the subject and correct labels corresponding thereto.

The patch image inverting unit 14 has a function of inverting a part of the patch images in the created patch image to the left and right.

The learning unit 15 has a function of using a patch image to learn a neural network model such as a bone metastasis region detected from a scintillation map. In this embodiment, U-Net, which is one of FCN (Fully Convolutional Networks), is used as a neural network-like model.

FIG. 3 is a diagram showing an example of a neural network model used in this embodiment. FIG. 3 shows an example of the structure when a patch with a patch size of 64×64 [pixel] is input. The neural network model used in this embodiment has an encoder-decoder structure. In the encoder structure, convolution and pooling are performed repeatedly to extract the global characteristics of the image. Through the decoder structure, the global structure is restored to the original size image, but in this process, by combining the features obtained in the process of encoding (Encode), the local features are also learned.

Also, the neural network model used in this embodiment has Bottleneck (bottleneck) as one of the residual blocks (K. He, X. Zhang, S. Ren, and J. Sun "Deep residual learning for image "recognition" arXiv: 1512.03385, 2015) to select higher-level features.

The structure of the neural network model shown in FIG. 3 will be described in detail. In this example, the input image is grayscale, and the input dimension is 64×64×1. First, make the number of channels 32 in the convolutional layer and pass the bottleneck. After that, 2×2 MAX pooling (maximum pooling) is performed, and the bottleneck is passed by doubling the number of channels. Repeating these layers a total of 4 times, the final feature map size of the encoder becomes 4×4×512.

Then, a deconvolution layer is used to double the size of the feature map. Then, the output of the deconvolution layer is concatenated with the feature map of the encoder, and passes through the bottleneck. The same layer is repeated 4 times in the same way as the encoder, and the size of the final feature map of the decoder becomes 64×64×32. Finally, the 1×1 convolutional layer becomes the number of output categories, that is, 3 channels (background, bone metastasis area, and non-bone metastasis area), and becomes 64×64×3. In addition, zero padding (Zero Padding) is performed in all 3×3 convolutional layers. After the convolutional layer, there is Batch Normalization (S. Ioffe, and C. Szegedy, “Batch Normalization: Accelerating Deep Network Training by Reducing "Internal Covariate Shift" (arXiv: 1502.03167, 2015) and ReLU function.

The learning unit 15 uses a patch image (including those obtained by inverting the patch image inversion unit 14 to the right and left) and its correct label to perform a neural network model learning. Learning is performed by evaluating the error (loss function) between the probability p _i and the correct probability of the output obtained when the patch image is input to the neural network-like model by Softmax function conversion. The Softmax function and loss function are shown below.

In addition, the learning unit 15 uses the verification data set (verification set) to verify the network obtained by the learning. Save the learning model obtained by using the guide data to learn any number of repetitions, and use the validation set to search all learning models for the parameters of the learning model. The sum of the excessive pick-up FP(P) of the pixel unit and the missing FN(P) of the pixel unit, that is, FP(P)+FN(P) is used as the evaluation value to determine the number of repetitions of learning. The learning unit 15 memorizes the model generated by learning in the memory unit 16.

The configuration of the learning device 1 of the present embodiment has been described above, but examples of the hardware of the learning device 1 include a CPU (Central Processing Unit) and RAM (Random Access Memory). , ROM (Read Only Memory, read-only memory), hard drive, monitor, keyboard, mouse, communication interface and other computers. The program product having modules for realizing the above-mentioned functions is stored in RAM or ROM, and the learning device 1 is realized by using the CPU to execute the program product. Such programming products are also included in the scope of the present invention.

FIG. 4 is a diagram showing the configuration of the bone metastasis detection device 20. The bone metastasis detection device 20 has an input unit 21 which inputs a scintillation image of the subject; a control unit 22 which detects a bone metastasis region from the subject's scintillation image; and a memory unit 26 which stores the memory by the learning device 1 The learning completion model obtained by learning; and the output section 27, which outputs data of the detected bone metastasis area.

The control unit 22 includes a density normalization processing unit 23, a patch image creation unit 24, and an inference unit 25. The concentration normalization processing unit 23 is the same as the concentration normalization processing unit 12 included in the learning device 1. The patch image creation unit 24 has a function of cutting out 64×64 [pixels] patch images from the scintillation image of the input subject. The basic configuration is the same as the patch image creation unit 13 included in the learning device 1, but the interval at which patch images are cut out is different. That is, in the learning device 1, the cutout is performed at 2 [pixels] intervals, but in the bone metastasis detection device 20, the patch image is cut out at 32 [pixels] intervals.

The inference unit 25 reads the learning completed model from the learning completed model memory unit 26, and inputs the patch image to the input layer of the learning completed model, and finds that each pixel of the patch image belongs to the background, the bone metastatic region, and the non-bone metastatic region. Probability of category.

FIG. 5 is a diagram showing an example of a patch image cut out from a scintillation chart. As shown in FIG. 5, the patch image is cut out from the subject's scintillation image in such a way that the adjacent patch images overlap half of each other. Therefore, for example, the region R is where the patch images A to D overlap, and the feature map of the pixels in the region R is obtained by each of the four patch images A to D. The inference unit 25 takes the average of the feature maps obtained by each of the four patch images. Then, the inference unit 25 converts the reconstructed output into a probability by the Softmax function, and determines the category with the highest probability for each pixel, and uses it as the final output.

FIG. 6 is a diagram showing the operation of the learning device 1. The learning device 1 inputs a plurality of subjects' scintillation graphs and the corresponding correct labels (background, bone metastasis area, non-bone metastasis area) as guidance materials (S10). The learning device 1 normalizes the density of the input scintillation map (S11), and creates a patch image from the normalized scintillation map (S12). The learning device 1 inverts a part of the patch image in the created patch image to the left and right (S13). Then, the learning device 1 uses the patch image and the corresponding correct label to learn the neural network-like model (S14), and memorizes the neural network-like model obtained by learning in the memory section 16 (S15). In addition, when the learning completion model is used in the bone metastasis detection device 20, the learning model stored in the memory unit 16 is read out and output to other devices.

FIG. 7 is a diagram showing the operation of the bone metastasis detection device 20. The bone metastasis detection device 20 inputs a scintillation image of the subject of the inspection object (S20). The bone metastasis detection device 20 normalizes the density of the input scintillation chart (S21), and creates a patch image from the normalized scintillation chart (S22). The bone metastasis detection device 20 reads out the neural network model such as the completed learning from the memory section 26, and inputs a patch image to the input layer of the neural network model such as the readout, and checks the bone of each pixel included in the patch image The transition area is detected (S23). The bone metastasis detection device 20 integrates the detection results of pixels in the area where a plurality of patch images overlap (S24). The bone metastasis detecting device 20 outputs the final result of the bone metastasis area obtained (S25).

The learning device 1 of the first embodiment uses the scintillation graph of the subject and the correct label corresponding to it to learn the neural network model. By using this learning to complete the model, it is possible to reduce the so-called "over-picking" and properly detect the bone metastasis area.

In addition, the learning device 1 of the first embodiment can reduce the memory size required for learning by learning using the patch image cut from the subject's scintillation graph. In addition, since the generation site of the bone metastasis area is not related to the shape of the organ, even if it is divided into patch images for learning, appropriate learning can be performed.

In addition, the learning device 1 of the first embodiment increases the change of the guidance data by inverting a part of the patch images from the plurality of patch images, thereby obtaining a reliable learning result. In addition, in this embodiment, the example in which the patch image is reversed left and right is cited, but the patch image may be reversed up and down. The method of using the patch image after being inverted upside down is suitable for the case where the anatomical structure of the background bone is vertically symmetrical (for example, to study the gathering of limbs extending in the vertical direction, etc.).

(Second embodiment)

FIG. 8 is a diagram showing the configuration of the learning device 2 of the second embodiment. The neural network model generated by the learning device 2 of the second embodiment is the same as the first embodiment, and is used to classify the scintillation area of the subject into three categories: bone metastasis area, non-bone metastasis area and background Model. The basic configuration of the learning device 2 of the second embodiment is the same as the learning device 1 of the first embodiment, but the learning device 2 of the second embodiment includes guidance data analysis for analyzing the contents of a plurality of image patches as guidance materials Department 18. Among the plurality of image patches, there is a patch image including a bone metastatic region or a non-bone metastatic region, and a patch image not including any one of a bone metastatic region and a non-bone metastatic region. The guidance data analysis unit 18 obtains a patch image including a bone metastasis region or a non-bone metastasis region and a patch not including a bone metastasis region or a non-bone metastasis region included in a plurality of patch images as guidance data The composition ratio of the image. The output unit 17 outputs data that generates the composition ratio of the patch image of the learning completed model stored in the memory unit 16.

By outputting the composition ratio of the patch image used to generate the learning completed model in this way, how to change the guidance information can be obtained when a new learning completed model is generated without changing the detection accuracy of the bone metastasis region using the learning completed model Tips for learning. In the present embodiment, an example in which the guidance data is changed by a user who observes the composition ratio of the patch image is cited, but further progress may be made, and the learning device 2 changes the guidance data based on the composition ratio of the patch image.

9 is a diagram showing a learning device 3 according to a modification of the second embodiment. The learning device 3 of the modified example includes a patch image selection unit 19 in addition to the configuration of the learning device 2 of the second embodiment. The patch image selection unit 19 has a function of selecting a patch image for learning based on the analysis result of the guidance data analysis unit 18. According to research by the present inventors and others, it is considered that if there are too many patch images that do not include any one of the bone metastatic region and the non-bone metastatic region, an appropriate model cannot be generated. Therefore, the learning device 3 of the modified example selects the patch image for learning when the composition ratio of the patch image that does not include the bone metastatic region or the non-bone metastatic region is more than a predetermined threshold value, instead of using all Does not include patch images of bone metastatic areas or non-bone metastatic areas. Thereby, the possibility of generating a model with better detection accuracy of the bone metastasis area is increased.

(Third Embodiment)

FIG. 11 is a diagram showing a learning device 4 of the third embodiment. The learning device 4 of the third embodiment uses Butterfly-Net as a model of a neural network such as a learning object. Butterfly-Net has a structure that combines two network parts with encoder-decoder structure. About Butterfly-Net, it has been described in detail in "Btrfly Net: Vertebrae Labelling with Energybased Adversarial Learning of Local Spine Prior" by Anjany Sekuboyina and others, MICCAI 2018.

The learning device 4 has: an input section 40 that inputs guidance data; a control section 41 that learns a neural network-like model based on the guidance data; a memory section 47 that memorizes a model generated by learning; and an output section 48, It outputs the model stored in the memory 47 to the outside. Furthermore, the learning device 4 of the third embodiment generates a region for scintillation of the subject to be classified as a bone metastasis area, a non-malignant lesion area (fracture, inflammation, etc.), and other areas (kidney, bladder, etc. physiological accumulation Area, injection leakage, urine leakage, background) 3 types of models. In this embodiment, the category where the physiologically concentrated region is included in other regions is classified as a category different from the non-malignant lesion region.

The learning device 4 of this embodiment uses a scintillation graph of the subject taken from the front (hereinafter, referred to as "front image") and a scintillation graph of the subject taken from the rear (hereinafter, referred to as "rear image") ), and the correct label assigned to each scintillation picture as a guide. 12 is a diagram showing an example of a front image and a rear image. Furthermore, the rear image is inverted in the horizontal direction.

The control unit 41 includes an image inverting unit 42, a front and rear image alignment unit 43, a density normalization processing unit 44, a patch image creation unit 45, and a learning unit 46.

The image reversing unit 42 has a function of reversing the rear image. When the image inversion unit 42 performs inversion, the correct label applied to the rear image is also inverted. The front and rear image alignment unit 43 performs alignment of the front image and the reversed rear image. In addition, although the example in which the rear image is reversed and aligned with the front image is given here, of course, the front image may be reversed and aligned with the rear image.

The concentration normalization processing unit 44 has a function of normalizing the concentration value to suppress the unevenness of the concentration value of the normal bone area that differs for each subject. The concentration standardization processing unit 44 normalizes the concentration value by the process of concentration range adjustment, identification of normal bone level, and grayscale standardization. The density normalization processing unit 44 converts the input density I _in of the scintillation graph into the density I _{normalized normalized} by the following formula (3).

為黃金比例 among them,

Golden ratio

The patch image creation unit 45 has a function of cutting out a scintillation image from the subject and creating a patch image. In this embodiment, the patch image creation unit 45 cuts out patch images at corresponding positions from the front image and the rear image, and generates a pair of front and rear patch images. In FIG. 12, the patch image A obtained from the front image and the patch image A′ obtained from the rear image are a pair of patch images. In addition, the patch image B and the patch image B′ are also paired patch images.

In this example, the size of the patch image is 64×64 [pixels]. Scan the 64×64[pixels] window at 2[pixels] intervals on the subject’s scintillation image (512×1024[pixels]), and include (1) the aggregation mark (bone metastasis area or Non-bone metastasis area), or (2) When the bone area is included and the aggregation is not included, an image patch is cut out. In the case where the patch image is cut out in accordance with the conditions of (1)(2) above in either the front image or the back image, a pair is cut out from the other of the front image or the back image Patch image.

The learning unit 46 has a function of using a patch image to learn a neural network model such as a bone metastasis region detected from a scintillation map. In this embodiment, Butterfly-Net having a structure in which two U-Nets are combined is used as a neural network-like model.

13 is a diagram showing an example of Butterfly-Net used in this embodiment. Butterfly-Net has a downward convex structure on the upper side, and has the same structure as the network shown in FIG. 3. Butterfly-Net has an upwardly convex structure underneath, and has the same structure as the network shown in FIG. 3 (only drawn upside down). Butterfly-Net is a combination of 2 U-Nets at the position of 128 feature maps of 8×8 each.

Also, the neural network model used in this embodiment uses a bottleneck as one of the residual blocks (K. He, X. Zhang, S. Ren, and J. Sun "Deep residual learning for image recognition" arXiv : 1512.03385, 2015), to select the more advanced features. In this manual, the Butterfly-Net thus improved is called "ResButterfly-Net".

In this example, the input image is grayscale, and the input dimension is 64×64×1. First, make the number of channels 32 in the convolutional layer and pass the bottleneck. After that, the maximum pooling of 2×2 is carried out, and the bottleneck is passed by doubling the number of channels. The process of repeating these layers three times is performed by each of the upper and lower U-Net. Then, at the position where the feature map with the size of 8×8×128 is obtained, the upper and lower two U-Net feature maps are combined, and then the bottleneck and maximum pooling are performed twice. Finally, 2×2× is obtained by encoding Feature mapping of 512 size.

Then, after passing the bottleneck, deconvolution is performed to double the size of the feature map. Then, the output of the deconvolution is concatenated with the feature map of the encoder to pass the bottleneck. After performing the same layer twice as the encoder, the result is copied, the feature maps of the upper and lower encoders are connected, and the process of deconvolution through the bottleneck is repeated 3 times. Finally, in the 1×1 convolutional layer, the number of output categories is 3 channels (bone metastasis area, non-malignant lesion area, and other areas), which becomes 64×64×3. In addition, FIG. 13 shows a bone metastatic legion and a non-malignant lesion, and the part other than the bone metastatic legion and the non-malignant lesion is another area.

The learning section 46 uses the paired patch images and their correct labels to learn the neural network-like model. The learning is performed by evaluating the error (loss function) between the probability p _i and the correct probability of the output obtained when a pair of patch images are input to a neural network-like model using Softmax function conversion. The loss function is shown below.

Here, w _c is the weighting of category c to reduce the influence of different pixel counts.

FIG. 14 is a diagram showing the configuration of a bone metastasis detection device 50 according to the third embodiment. The bone metastasis detection device 50 has: an input unit 51 which inputs a scintillation image of the subject; a control unit 52 which detects a bone metastasis region from the subject's scintillation image; a memory unit 58 which memorizes by the learning device 4 The learning completed model obtained by learning; and an output section 59, which outputs data of the detected bone metastasis area.

The control unit 52 includes an image inverting unit 53, a front and rear image alignment unit 54, a density normalization processing unit 55, a patch image creation unit 56, and an inference unit 57. The image inverting unit 53, the front-back image alignment unit 54 and the density normalization processing unit 55 are the same as the image inverting unit 42, the front-back image alignment unit 43 and the density normalization processing unit 44 included in the learning device 4. The patch image creation unit 56 has a function of cutting out the patch image from the scintillation image (front image and rear image) of the input subject. The patch image creation unit 56 cuts out corresponding areas from the front and rear scintillation images to generate a pair of patch images. Furthermore, the patch image creation unit 56 may change the interval at which patch images are cut out with respect to the patch image creation unit 45 included in the learning device 4.

The inference section 57 reads the learning completed model from the learning completed model memory section 58 and inputs a pair of patch images to the input layer of the learning completed model to find that each pixel of the patch image belongs to the bone metastasis area, non-malignant lesion area, and others The probability of each category of the area.

15 is a diagram showing the operation of the learning device 4. The learning device 4 inputs a plurality of scintillation images of the subjects (front image and rear image) and corresponding correct labels (bone metastasis area, non-malignant lesion area, other area) as guidance data (S30). The learning device 4 reverses the rear image (S31), and aligns the front image and the reversed rear image (S32). Next, the learning device 4 normalizes the density of the input front image and the rear image (S33), and cuts out the areas corresponding to the front and rear images to generate a pair of patch images (S34).

Then, the learning device 4 uses the patch image and the corresponding correct label to perform learning of the neural network-like model (S35). As mentioned above, in the learning here, a pair of front and back patch images are input to the input layer of Butterfly-Net, and the learning is based on the categories and correct data output from the output layer. The learning device 4 memorizes the neural network model obtained by learning in the memory section 47 (S36). In addition, when the learning completion model is used in the bone metastasis detection device 50, the learning model stored in the memory unit 47 is read out and output to other devices.

The learning device 4 of the third embodiment is configured to use a neural network model such as Butterfly-Net as a learning model, and input a pair of patch images before and after to its input layer for learning. Therefore, by simultaneously acquiring high correlation The patch image is processed to produce a neural network model that can accurately detect the bone metastasis area.

Furthermore, in the learning device 4 of the third embodiment, it is also the same as the first embodiment, and by inverting the patch image and inverting a part of the patch images from the plurality of patch images, Increase the change of guidance materials. However, in this embodiment, the patch image used as the guidance material is a pair of front and rear images. Therefore, when the patch image in front or back is inverted, the other patch image is also inverted in the same direction. turn. By inverting the patch image in this way and adding guidance materials, reliable learning can be performed.

In addition, in the learning device 4 of the third embodiment described above, an example of generating a learning completion model classified into a category different from that of the first embodiment is given, but of course it is also possible to produce a model classified into the same category as the first embodiment Learn to complete the model. Conversely, in the first embodiment or the second embodiment described above, injection leakage or urine leakage is excluded, but physiological accumulation such as kidney or bladder, injection leakage, urine leakage, and background can also be set to other regions to generate classification It is the same type of model as the third embodiment.

[Example]

(Example 1)

An example of detecting a bone metastasis region using the learning completion model generated by the learning device 1 of the first embodiment will be described. As the learning completion model, the bone completion region is detected by using the learning completion model generated using the guidance data obtained by inverting the patch image and the learning completion model generated using the guidance data not inverting the patch image.

(Sample used for experiment)

‧Standardized image of concentration value of anterior bone scintillation chart: 103 cases

‧Image size: 512×1024[pixels]

‧Resolution: 2.8×2.8[mm/pixel]

‧Patch size: 64×64[pixels]

(Evaluation method)

‧3-fold cross-validation (learning: 68 cases, verification: 17 cases, test: 17-18 cases)

Furthermore, the verification data is used to determine the number of repetitions of learning.

(Evaluation value)

‧FP(P): excessive picking up of pixel units

‧FN(P): omission of pixel units in bone metastasis area

‧Sensitivity: detection rate of the regional unit of bone metastasis area = (number of detected bone metastasis areas)/(number of bone metastasis areas)

‧FROC curve: Sensitivity vs. FP(P) or FP(R)

‧FP(P)/background+FN(P)/bone metastasis area

(Learning conditions of the model)

‧Optimizer: Adam (α=0.001, β ₁ =0.9, β ₂ =0.999)

‧Batch size 64

‧Number of repetitions: 10000 times

‧Select the network with the least number of FP(P)+FN(P) through verification

(Comparative example)

As a comparative example, the MadaBoost (C. Domingo and O. Watanabe "MadaBoost: A modification of AdaBoost" Proc. Thirteenth Annual Conference on Computational Learning Theory, pp. 180-189 , 2000) to compare the detected results. The algorithm of MadaBoost uses the method developed by the researcher of the inventors (Nan Yongtai "Improvement of Bone Metastasis Detection Processing on Bone Scintillation Chart" 3rd Cancer Nuclear Medicine Image Analysis Software Development Conference).

(Experimental results)

FIG. 10 is a FROC curve showing the relationship between sensitivity and FP(P) obtained through experiments. The FROC curve shown in FIG. 10 indicates that as the sensitivity on the vertical axis becomes higher, the “over-picking” that picks up the non-bone metastatic region as the bone metastatic region accordingly increases. In the method of the embodiment, for example, if the sensitivity is set to 0.8, the over pickup is 200 pixels or less. Compared with the case where over-picking of 500 pixels or more occurs in the conventional method using MadaBoost, in the embodiment, over-picking can be suppressed. In FIG. 10, the U-Net (Flip) graph shows the result of detection using the learning completion model generated by using the guidance data obtained by inverting a part of the patch image, and the U-Net graph shows that the use is not performed. A diagram of the results of the detection of the model of learning completion generated by the inversion of the patch image.

(Example 2)

An example of detecting a bone metastasis region using the learning completion model generated by the learning device 4 of the third embodiment will be described. As the learning completion model, ResButterfly-Net described in the third embodiment and the bottleneck of ResButterfly-Net are replaced with Butterfly-Net of the convolution layer. In addition, U-Net is used as a comparative example.

(Sample used for experiment)

‧52-95 year old Japanese men with prostate cancer: 246 cases

(Evaluation method)

‧3-fold cross-validation (learning: 164 cases, verification: 41 cases, test: 41 cases)

Furthermore, the verification data is used to determine the best number of repetitions for learning.

(Evaluation value)

‧FP(P): excessive picking up of pixel units

‧FP(R): Excessive pickup of regional units

‧FP(P)+FN(P): over-picking of pixel units and omission of pixel units in bone metastasis area

(Learning conditions of the model)

‧Optimizer: Adam (α=0.001, β ₁ =0.9, β ₂ =0.999)

‧Lot size 256

‧Number of repetitions: set to a maximum of 50,000 times, and make the best number of repetitions when the total number of misclassified pixels becomes the minimum.

(Experimental results)

Table 1 shows the evaluation values when the sensitivity of the bone metastatic region is 0.9. The top is the result of the front image, and the bottom is the result of the rear image.

[Table 1]

As shown in Table 1, if ResButterfly-Net and Butterfly-Net are used as the learning model, compared with the model using U-Net, it can be confirmed that there are fewer errors in hotspot detection in multiple indicators.

The above embodiments and examples include the technical ideas shown in (1) to (11) below.

(1) A learning device that generates a model of a neural network such as an abnormal cluster for detecting scintillation graphs from subjects; it includes: an input unit that inputs a plurality of subject scintillation graphs and each flicker The correct labels of normal aggregation and abnormal aggregation in the figure are used as guidance data; and the learning department uses the above guidance data to learn the model of neural networks such as abnormal aggregation of bone scintillation graphs.

(2) The learning device according to (1), which includes a patch image creation unit that cuts out the area where the bones of the subject are photographed from the scintillation pictures of the plurality of subjects to create a patch image, and the learning unit uses The above patch image and the corresponding correct label are used as guidance materials for learning.

(3) The learning device according to (2), wherein the patch image creation part scans a window of a predetermined size on the scintillation graph of the subject and cuts out the subject's bone in the window The area of the window is the patch image.

(4) The learning device according to (2) or (3), which is provided in the patch image created by the patch image creation unit, and obtains a patch image including normal aggregation or abnormal aggregation and does not include normal aggregation And the guide data analysis part of the composition ratio of the patch image of any of the abnormal aggregation.

(5) The learning device according to claim 4, which includes a method for extracting the patch image created by the patch image creation unit in such a manner that the composition ratio determined by the above-mentioned guidance data analysis unit is included in a predetermined range. A patch image selection unit including a patch image of either normal aggregation or abnormal aggregation.

(6) The learning device according to any one of (1) to (5), which includes a patch for inverting at least a part of a patch image of the patch image created by the patch image creation unit from left to right or upside down Image reversal section.

(7) The learning device according to any one of (1) to (6), wherein the neural network described above includes an encoder-decoder structure, and the feature map obtained by the encoder structure is input to the decoding The structure of the device.

(8) A learning method that generates a neural network model for detecting abnormal clusters from the scintillation graph of the subject; it has the following steps: input a plurality of scintillation graphs of the subject and each scintillation graph The correct labeling of normal aggregation and abnormal aggregation is used as guidance data; and the use of the above guidance data to learn the model of neural network to detect abnormal aggregation of bone scintillation graph.

(9) A program product that is used to generate a model of a neural network such as an abnormal cluster for detecting scintillation from the subject; it performs the following steps: input a plurality of scintillation of the subject and each flicker The steps of using the correct labels for normal and abnormal aggregation in the figure as guidance data; and using the above guidance data to learn the model of neural network model for detecting abnormal aggregation of bone scintillation graph.

(10) A memory medium with a learning completion model, which is used to make the computer function by the way of detecting abnormal aggregation of the subject's scintillation graph; it is composed of a neural network, which has Convolutional layer and inverse convolutional layer, and this type of neural network includes the structure of inputting the feature map obtained by the convolutional layer to the deconvolutional layer. The above-mentioned memory medium with the learned completion model will flash a plurality of subjects The correct labels of normal aggregation and abnormal aggregation in the graph and each scintillation graph are used as guidance data for learning, and the computer is made functional by detecting abnormal aggregation from the scintillation graph of the subject input to the above neural network.

(11) An anomalous aggregation detection device, comprising: a memory unit whose memory has a learning completion model of a neural network or the like learned by the learning device of any one of (2) to (7); an input unit, It inputs the scintillation image of the subject; the patch image creation unit, which creates the patch image from the above scintillation image; the inference unit, which inputs the patch image to the input layer of the learning completed model read from the memory unit, and Find the abnormally concentrated area included in the patch image; and the output section, which outputs data indicating the abnormally concentrated area.

This application claims priority based on Japanese Application No. 2018-096186 filed on May 18, 2018, and incorporates all the contents disclosed therein.

1‧‧‧Learning device

10‧‧‧Input

11‧‧‧Control Department

12‧‧‧Concentration Standardization Department

13‧‧‧ Patch image production department

14‧‧‧ Patch image reversal section

15‧‧‧Learning Department

16‧‧‧ Memory Department

17‧‧‧ Output

Claims (16)

A learning device that generates a model of a neural network for detecting a bone metastasis region from a scintillation graph of a subject; it includes: an input unit that inputs a plurality of scintillation graphs of each subject and each scintillation graph The correct label of the bone metastasis area and the non-bone metastasis area is used as guidance data; and the learning department uses the above guidance data to learn a neural network model such as a bone metastasis area for detecting a bone scintillation graph. As in the learning device of claim 1, it is provided with a patch image creation unit that creates a patch image by cutting out the area where the subject's bone is captured from the scintillation images of the plurality of subjects, and the learning unit uses the patch image Images and the corresponding correct labels are used as guidance materials for learning. A learning device according to claim 2, wherein the patch image creation unit scans a window of a predetermined size on the scintillation graph of the subject, and cuts out the window when the bone of the subject is photographed in the window The area is used as the patch image. The learning device according to claim 2 or 3, which includes a guidance data analysis unit that obtains a bone metastasis region or a non-bone metastasis region from the patch image created by the patch image creation unit The ratio of the patch image to the patch image that does not include the bone metastasis area and the non-bone metastasis area. The learning device according to claim 4 is provided with a patch image selection unit that includes the composition ratio determined by the above-mentioned guidance data analysis unit in a predetermined range so as to include the patch image The patch image created by the image creation unit extracts the patch image that does not include the bone metastatic region and the non-bone metastatic region. The learning device according to claim 2 is provided with a patch image inverting unit that inverts at least a part of the patch image of the patch image created by the patch image creating unit from side to side or from top to bottom. The learning device according to any one of claims 1 to 3, wherein the aforementioned neural network includes the following structure, that is, has an Encoder-Decoder (encoder-decoder) structure, and will be obtained by the encoder structure The feature map is input to the decoder structure. The learning device according to claim 1, wherein the above-mentioned neural network has a structure combining the first network part with the encoder-decoder structure and the second network part with the encoder-decoder structure, and the input The department enters the scintillation pictures taken from the front and back and their correct labels for each subject. The learning section inputs the scintillation pictures of the subjects taken from the front into the input layer of the first network part, and enters the second network part The input of the input layer is taken from behind to learn the flashing picture of the subject. The learning device according to claim 2 or 3, wherein the aforementioned neural network has a structure combining the first network part with the encoder-decoder structure and the second network part with the encoder-decoder structure, The input unit inputs the scintillation images taken from the front and back and their correct labels for each subject, and the learning unit inputs the first patch cut from the scintillation images of the subject taken from the front to the input layer of the first network part An image, and the second patch image corresponding to the first patch image cut out from the scintillation image of the subject taken from the rear is input to the input layer of the second network part to learn. The learning device according to any one of claims 1 to 3, wherein the non-bone metastasis area includes a non-malignant lesion area, and the input unit accepts the correct label with each of the bone metastasis area and the non-malignant lesion area The scintillation graphs of a plurality of subjects are used as guidance data. The learning section uses the guidance data to learn a model of a neural network such as detecting each of the bone metastasis area and the non-malignant lesion area. A learning method that generates a neural network model for detecting bone metastasis areas from the scintillation graph of the subject; it has the following steps: enter the scintillation graph of multiple subjects and the bones in each scintillation graph The step of using the correct label of the transfer area and the non-bone metastasis area as guidance data; and the step of learning the neural network model such as the bone metastasis area for detecting the bone scintillation graph using the above guidance data. A program product that is used to generate a model of a neural network such as a bone metastasis area detected from a subject's scintillation chart; it performs the following steps: input a plurality of subject scintillation charts and each scintillation chart The correct labeling of the bone metastasis area and the non-bone metastasis area is used as guidance data; and the step of learning the neural network model such as the bone metastasis area used to detect the bone scintillation graph using the above guidance data. A memory medium with a model for learning and learning, which is used to make the computer function by detecting the bone metastasis area by the scintillation diagram of the subject; it is composed of a neural network with a convolution layer And a deconvolution layer, and this type of neural network includes a structure that inputs the feature map obtained by the convolution layer to the deconvolution layer. The above-mentioned memory medium with the learned completion model will include a plurality of subjects' scintillation maps and each The correct labels of the bone metastasis area and the non-bone metastasis area in the scintillation map are used as the guide data for learning, and it enables the computer to function by detecting the bone metastasis area from the scintillation map of the subject input to the above neural network. A bone metastasis detection device, comprising: a memory part having a memory completion model learned by a neural network such as a learning device of request item 2 or 3; an input part which inputs a scintillation graph of a subject; a patch map The image production unit creates a patch image from the scintillation image; the inference unit inputs the patch image to the input layer of the learning completed model read from the memory unit and obtains the bones included in the patch image A transfer area; and an output section, the output of which indicates data of the bone transfer area. A program product that is used to detect the bone metastasis area from the subject's scintillation diagram; it causes the computer to perform the following steps: the step of entering the subject's scintillation diagram; the step of making a patch image from the above scintillation diagram; The memory of the learning completion model of a neural network or the like learned by the learning device of the request item 2 or 3 reads the learning completion model and inputs the patch image to the input layer of the learning completion model to obtain the above The step of the bone metastasis area included in the patch image; and the step of outputting the data representing the bone metastasis area. A program product that is used to detect the bone metastasis area from the subject's scintillation diagram; it causes the computer to perform the following steps: input the steps of taking two scintillation pictures of the subject from before and after; make two of the scintillation diagrams A step reversed in the horizontal direction; self-memory has a learning completion model pre-generated by learning using the guidance data to read out the learning completion model and input 2 flashes to the input layer of the learning completion model Figure, the step to find the bone metastasis area included in the above scintillation diagram; and the step to output the data representing the bone metastasis area.

Images