Disclosure of Invention
To overcome the problems in the related art, an object of the present disclosure is to provide a method, an apparatus, a storage medium, and an electronic device for recognizing a medical image.
In order to achieve the above object, according to a first aspect of an embodiment of the present disclosure, there is provided a method for recognizing a medical image, the method including:
inputting a medical image to be recognized into a current image recognition model, and acquiring sample data from image recognition data output by the current image recognition model according to a verification result of the image recognition data output by the current image recognition model, wherein the current image recognition model is a first image recognition model trained in advance, the verification result is determined by using a preset confidence threshold and a judgment result input by a doctor, and the sample data is data obtained by modifying image recognition data with a wrong verification result by a medical image expert;
when the number of the sample data reaches a first preset number, retraining the first image recognition model through the first preset number of the sample data to obtain a second image recognition model;
comparing the recognition performance of the first image recognition model with that of the second image recognition model to obtain an updated image recognition model, wherein the updated image recognition model is the second image recognition model or a third image recognition model, and the third image recognition model is an image recognition model which is obtained by training an initial model selected from a model library and has better recognition performance than the first image recognition model;
and identifying the currently input medical image through the updated image identification model.
Optionally, after the identification of the currently input medical image by the updated image identification model, the method further includes:
and taking the updated image recognition model as the current image recognition model, circularly executing the step from inputting the medical image to be recognized into the current image recognition model, so as to obtain sample data from the image recognition data output by the current image recognition model according to the verification result of the image recognition data output by the current image recognition model, and recognizing the currently input medical image through the updated image recognition model.
Optionally, before the medical image to be recognized is input into the current image recognition model, so as to obtain sample data from the image recognition data output by the current image recognition model according to the verification result of the image recognition data output by the current image recognition model, the method further includes:
training the first image recognition model through a second preset amount of training data, wherein the training data comprises: sample medical images and diagnostic information corresponding to the sample medical images, the diagnostic information including: disease diagnosis, at least one of lesion area and severity of disease condition.
Optionally, the inputting the medical image to be recognized into the current image recognition model to obtain sample data from the image recognition data output by the current image recognition model according to the verification result of the image recognition data output by the current image recognition model includes:
taking a first medical image as an input of the current image recognition model to acquire first image recognition data output by the current image recognition model, wherein the first image recognition data comprises: the first medical image, diagnosis information corresponding to the first medical image, and a confidence of the diagnosis information include: at least one of a disease diagnosis, a diseased area, and a severity of a disease condition;
when the confidence coefficient is smaller than the confidence coefficient threshold value, determining that the verification result of the first image identification data is wrong;
outputting the first image identification data to a first interactive interface so that a medical image expert modifies the first image identification data into second image identification data through the first interactive interface;
taking the second image identification data as the sample data;
outputting the second image identification data serving as final identification data to a second interactive interface, so that a doctor can obtain the final identification data through the second interactive interface;
or,
when the confidence coefficient is larger than or equal to the confidence coefficient threshold value, outputting the first image identification data to a second interactive interface so that a doctor can verify the first image identification data through the second interactive interface;
when the first image identification data acquired through the second interactive interface passes the verification of the doctor, determining the first image identification data as the final identification data;
when the first image identification data acquired through the second interactive interface fails to pass the verification of the doctor, outputting the first image identification data to the first interactive interface, so that a medical image expert modifies the first image identification data into third image identification data through the first interactive interface;
taking the third image identification data as the sample data;
and outputting the third image identification data serving as the final identification data to a second interactive interface, so that the doctor can obtain the final identification data through the second interactive interface.
Optionally, the comparing the recognition performances of the first image recognition model and the second image recognition model to obtain an updated image recognition model includes:
comparing the recognition performance of the first image recognition model with that of the second image recognition model, wherein the recognition performance is a recognition rate obtained by testing the image recognition models through a preset medical image data set;
when the recognition performance of the first image recognition model is smaller than that of the second image recognition model, taking the second image recognition model as the updated image recognition model; or,
when the recognition performance of the first image recognition model is greater than or equal to that of the second image recognition model, training an initial model selected from the model library to obtain a third image recognition model with recognition performance superior to that of the first image recognition model;
and taking the third image recognition model as the updated image recognition model.
According to a second aspect of the embodiments of the present disclosure, there is provided an apparatus for recognizing a medical image, the apparatus including:
the system comprises a sample acquisition module, a data processing module and a data processing module, wherein the sample acquisition module is used for inputting a medical image to be identified into a current image identification model so as to acquire sample data from image identification data output by the current image identification model according to a verification result of the image identification data output by the current image identification model, the current image identification model is a first image identification model trained in advance, the verification result is determined by utilizing a preset confidence threshold and a judgment result input by a doctor, and the sample data is obtained by modifying the image identification data with the wrong verification result by a medical image expert;
the model retraining module is used for retraining the first image recognition model through the sample data of the first preset quantity to obtain a second image recognition model when the quantity of the sample data reaches a first preset quantity;
the model updating module is used for comparing the recognition performance of the first image recognition model with that of the second image recognition model to obtain an updated image recognition model, wherein the updated image recognition model is the second image recognition model or a third image recognition model, and the third image recognition model is an image recognition model which is obtained by training an initial model selected from a model library and has better recognition performance than the first image recognition model;
and the image identification module is used for identifying the currently input medical image through the updated image identification model.
Optionally, the apparatus further comprises:
and the cyclic updating module is used for taking the updated image identification model as the current image identification model, circularly executing the step from inputting the medical image to be identified into the current image identification model, acquiring sample data from the image identification data output by the current image identification model according to the verification result of the image identification data output by the current image identification model, and identifying the currently input medical image through the updated image identification model.
Optionally, the apparatus further comprises:
the model training module is used for training the first image recognition model through a second preset amount of training data, wherein the training data comprise: sample medical images and diagnostic information corresponding to the sample medical images, the diagnostic information including: disease diagnosis, at least one of lesion area and severity of disease condition.
Optionally, the sample acquiring module includes:
the image analysis submodule is configured to use a first medical image as an input of the current image recognition model to obtain first image recognition data output by the current image recognition model, where the first image recognition data includes: the first medical image, diagnosis information corresponding to the first medical image, and a confidence level of the diagnosis information, wherein the diagnosis information includes: disease diagnosis, at least one of lesion area and severity of disease condition;
the first data judgment sub-module is used for determining that the verification result of the first image identification data is wrong when the confidence coefficient is smaller than the confidence coefficient threshold value;
the first data output submodule is used for outputting the first image identification data to a first interactive interface so that a medical image expert modifies the first image identification data into second image identification data through the first interactive interface;
the first data acquisition submodule is used for acquiring the second image identification data as the sample data;
the second data output submodule is used for outputting the second image identification data serving as final identification data to a second interactive interface, and a doctor acquires the final identification data through the second interactive interface;
or,
the second data judgment sub-module is used for outputting the first image identification data to a second interactive interface when the confidence coefficient is greater than or equal to the confidence coefficient threshold value, so that a doctor can verify the first image identification data through the second interactive interface;
the first data verification sub-module is used for determining the first image identification data as the final identification data when the first image identification data acquired through the second interactive interface passes the verification of the doctor;
the second data verification sub-module is used for outputting the first image identification data to the first interactive interface when the first image identification data acquired through the second interactive interface fails to be verified by the doctor, so that a medical image expert modifies the first image identification data into third image identification data through the first interactive interface;
the second data acquisition submodule is used for taking the third image identification data as the sample data;
and the third data output submodule is used for outputting the third image identification data serving as the final identification data to a second interactive interface, and is used for acquiring the final identification data by a doctor through the second interactive interface.
Optionally, the model updating module includes:
the performance comparison submodule is used for comparing the identification performance of the first image identification model with the identification performance of the second image identification model, wherein the identification performance is an identification rate obtained by testing the image identification model through a preset medical image data set;
a first model updating submodule, configured to, when the recognition performance of the first image recognition model is smaller than the recognition performance of the second image recognition model, use the second image recognition model as the updated image recognition model; or,
a model obtaining sub-module, configured to train an initial model selected from the model library when the recognition performance of the first image recognition model is greater than or equal to the recognition performance of the second image recognition model, so as to obtain a third image recognition model whose recognition performance is superior to that of the first image recognition model;
and the second model updating submodule is used for taking the third image recognition model as the updated image recognition model.
According to a third aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, which when executed by a processor implements the steps of the medical image recognition method provided by the first aspect of the embodiments of the present disclosure.
According to a fourth aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including:
a memory having a computer program stored thereon;
a processor for executing the computer program in the memory to implement the steps of the medical image identification method provided by the first aspect of the embodiments of the present disclosure.
By the technical scheme, the medical image to be recognized can be input into the current image recognition model, so that sample data can be obtained from the image recognition data output by the current image recognition model according to the verification result of the image recognition data output by the current image recognition model, the current image recognition model is a first image recognition model trained in advance, the verification result is determined by utilizing a preset confidence threshold and a judgment result input by a doctor, and the sample data is data obtained by modifying the image recognition data with the wrong verification result by a medical image expert; when the number of the sample data reaches a first preset number, retraining the first image recognition model through the first preset number of the sample data to obtain a second image recognition model; comparing the recognition performance of the first image recognition model with that of the second image recognition model to obtain an updated image recognition model, wherein the updated image recognition model is the second image recognition model or a third image recognition model, and the third image recognition model is an image recognition model which is obtained by training an initial model selected from a model library and has better recognition performance than the first image recognition model; and identifying the currently input medical image through the updated image identification model. The image recognition model can be continuously trained and updated through the medical image and the recognition result while the medical image recognition result is obtained, and the growing property, the adaptability and the accuracy of the image recognition model are ensured.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
Fig. 1 is a flowchart illustrating a medical image recognition method according to an exemplary embodiment, as shown in fig. 1, the method including:
step 101, inputting a medical image to be recognized into a current image recognition model, and acquiring sample data from image recognition data output by the current image recognition model according to a verification result of the image recognition data output by the current image recognition model.
The current image recognition model is a first image recognition model trained in advance, the verification result is determined by using a preset confidence threshold and a judgment result input by a doctor, and the sample data is obtained by modifying image recognition data with an incorrect verification result by a medical image expert.
In addition, the medical image to be identified may be an X-ray developed image, a CT (computed Tomography) image, a PET (Positron Emission Tomography) image, an NMRI (Nuclear Magnetic Resonance Imaging), a medical ultrasonography image, and the like. The doctor may be a medical worker who is responsible for diagnosis and treatment or a worker who is responsible for managing the medical image photographing apparatus. The medical image experts are medical workers or researchers capable of accurately identifying medical images, and the medical image experts can consider that identification results made by the medical image experts on the medical images are correct.
For example, the current image recognition model may be a neural network model having a predetermined number of layers (preferably 25 layers). The image recognition model can be applied to an image recognition system or an information management system of a hospital, for example, the image recognition model can be directly connected with the shooting devices of the medical images of multiple types to receive the medical images generated by the shooting devices; and/or, the system is connected with a device for inputting and scanning the entity picture so as to acquire the medical image input by the operator, and then the obtained result is output to the electronic device of the doctor.
It should be noted that the first image recognition model may be understood as an image recognition model that has been initially trained (in the setup phase of the whole image recognition system or the information management system) and is already put into use. That is, the first image recognition model (in cooperation with the verification of the doctor) already has the capability of recognizing and analyzing the medical image involved in the actual medical procedure, and outputting the diagnosis information including the disease diagnosis result, the lesion region, the severity of the disease, and the like. However, due to the diversity of medical images, the image recognition model formed by one-time training may have poor adaptability and poor recognition effect. Therefore, in step 101, while obtaining the correct recognition result (i.e. image recognition data) of the medical image through the first image recognition model, the confidence threshold value that fails to pass the system and the incorrect recognition result (i.e. the image recognition data whose verification result is incorrect) verified by the doctor are collected. After the wrong recognition results are obtained, the wrong recognition results are modified into correct recognition results through a modification interface provided for a medical image expert. These correct recognition results are used, on the one hand, for retraining the first image recognition model in the following step 102 (after reaching a certain number) and, on the other hand, for providing the correct recognition results to the patient via the doctor (after all, the doctor needs to treat the patient according to the recognition results).
102, when the quantity of the sample data reaches a first preset quantity, retraining the first image recognition model through the sample data of the first preset quantity to obtain a second image recognition model.
For example, in an actual operation process, the step 101 may identify a small number of medical images each time, and retrain a large amount of the sample data for the first image identification model, so that, in the step 102, the number of the sample data to be acquired needs to be confirmed, and when the number of the sample data is larger than the first preset number (for example, 500), retrain the model to acquire the second image identification model. It is understood that the second image recognition model is consistent with the configuration of the first image recognition model (which may be the number of layers of the neural network).
Step 103, comparing the recognition performance of the first image recognition model and the second image recognition model to obtain an updated image recognition model.
The updated image recognition model is the second image recognition model or a third image recognition model, and the third image recognition model is an image recognition model which is obtained by training the initial model selected from the model base and has better recognition performance than the first image recognition model.
For example, the identification performance may be obtained by testing the first image identification model and the second image identification model on a preset medical image data set respectively. When the performance of the second image recognition model is superior to that of the first image recognition model, the second image recognition model can be directly used for replacing the first image recognition model, and the updated image recognition model is obtained; when the performance of the first image recognition model is superior to that of the second image recognition model, the structure of the first image recognition model itself may be considered to be not suitable for the medical image corresponding to the first preset number of sample data, and at this time, an initial model having a different structure from that of the first image recognition model may be selected from the model library, and a model having a recognition performance superior to that of the first image recognition model may be obtained through data training as the third image recognition model and may be used as the updated image recognition model. The image recognition models with different structures can be understood as image recognition models with neural networks with different layers.
The initial model is obtained by training a pre-obtained data set, and when a new data set exists, the model can be retrained, so that a new model different from the initial model is obtained. The method for generating the third image recognition model through data training is similar to the method for generating the first image recognition model or the second image recognition model, and the difference is that the structure of the third image recognition model is different from the structure of the first image recognition model or the second image recognition model, but the model structures of the first image recognition model and the second image recognition model are the same and the model parameters are different. The involved recognition model consists of two parts, namely a model structure (the number of layers of a neural network, the number of neurons in each layer, and the connection relation between the neurons in each two layers) and model parameters (weight parameters for connection between the neurons). It should be noted that, the process of model retraining and updating is to keep the model structure unchanged, and only try to change the model parameters, and then try to change the model structure and change the model parameters, so as to obtain a better-performance recognition model for a new data set.
And 104, identifying the currently input medical image through the updated image identification model.
Illustratively, the image recognition model in the image recognition system or the information management system described above at this time has been updated to the second image recognition model or the third image recognition model. Thereafter, the input medical image can be continuously identified through the updated image identification model.
In summary, the present disclosure can input a medical image to be recognized into a current image recognition model, so as to obtain sample data from image recognition data output by the current image recognition model according to a verification result of the image recognition data output by the current image recognition model, where the current image recognition model is a first image recognition model trained in advance, the verification result is determined by using a preset confidence threshold and a judgment result input by a doctor, and the sample data is data obtained by modifying image recognition data of which the verification result is incorrect by a medical image specialist; when the number of the sample data reaches a first preset number, retraining the first image recognition model through the first preset number of the sample data to obtain a second image recognition model; comparing the recognition performance of the first image recognition model with that of the second image recognition model to obtain an updated image recognition model, wherein the updated image recognition model is the second image recognition model or a third image recognition model, and the third image recognition model is an image recognition model which is obtained by training an initial model selected from a model library and has better recognition performance than the first image recognition model; and identifying the currently input medical image through the updated image identification model. The image recognition model can be continuously trained and updated through the medical image and the recognition result while the medical image recognition result is obtained, and the growing property, the adaptability and the accuracy of the image recognition model are ensured.
Fig. 2 is a flowchart illustrating another medical image recognition method according to the embodiment shown in fig. 1, and as shown in fig. 2, after the step 104, the method may further include:
step 105, taking the updated image recognition model as the current image recognition model, and circularly executing the steps from inputting the medical image to be recognized into the current image recognition model, so as to obtain sample data from the image recognition data output by the current image recognition model according to the verification result of the image recognition data output by the current image recognition model, and recognizing the currently input medical image through the updated image recognition model.
For example, in the actual medical image recognition process, the updated image recognition model may be used as the current image recognition model, the input medical image is continuously recognized, and the model updating process from step 101 to step 104 is executed in a loop during the process, so as to implement continuous dynamic transformation of the image recognition model.
Fig. 3 is a flowchart illustrating a further medical image recognition method according to the embodiment shown in fig. 2, and as shown in fig. 3, before the step 101, the method may further include:
and 106, training the first image recognition model through a second preset amount of training data.
Wherein the training data comprises: sample medical images and diagnostic information corresponding to the sample medical images, the diagnostic information comprising: disease diagnosis, at least one of lesion area and severity of disease condition.
The step 106 is an initialization training step occurring in the above-mentioned setup phase of the whole image recognition system or the information management system, or when the image recognition model is not in use. After the initialization training step, the image recognition model has a basic image recognition function.
Fig. 4 is a flowchart illustrating a method for acquiring medical image sample data according to the embodiment shown in fig. 3, where, as shown in fig. 4, the step 101 may include: step 1011-.
In step 1011, the first medical image is used as the input of the current image recognition model to obtain the first image recognition data output by the current image recognition model.
Wherein, the first image identification data comprises: the first medical image, the diagnosis information corresponding to the first medical image, and the confidence of the diagnosis information include: at least one of a disease diagnosis result, a diseased region, and a severity of a disease. It should be noted that the confidence may be the confidence of the whole diagnostic information, or multiple confidences corresponding to all conclusions in the diagnostic information, for example, the confidence may include the confidence of the presence or absence of a disease (disease diagnosis result), the confidence that the pixel points in the lesion region are lesion pixel points, and the confidence that the severity of the disease is early, middle or late. It should be noted that, when there are a plurality of confidence levels, the following confidence level thresholds are also set correspondingly, and are compared respectively during comparison, and when any one of the confidence levels does not satisfy the corresponding confidence level threshold, it is determined that the verification result of the first image recognition data is an error.
In step 1012, when the confidence is smaller than the confidence threshold, it is determined that the verification result of the first image recognition data is an error.
Step 1013, the first image recognition data is output to a first interactive interface, so that the medical image expert modifies the first image recognition data into second image recognition data through the first interactive interface.
In step 1014, the second image identification data is used as the sample data.
Step 1015, outputting the second image identification data as final identification data to a second interactive interface for the doctor to obtain the final identification data through the second interactive interface.
In step 1016, when the confidence level is greater than or equal to the confidence level threshold, the first image recognition data is output to a second interactive interface, so that the doctor can verify the first image recognition data through the second interactive interface.
Step 1017, when the first image identification data acquired through the second interactive interface passes the verification of the doctor, determining that the first image identification data is the final identification data.
Step 1018, when the first image identification data obtained through the second interactive interface fails the verification of the doctor, outputting the first image identification data to the first interactive interface, so that the medical image specialist modifies the first image identification data into third image identification data through the first interactive interface.
In step 1019, the third image identification data is used as the sample data.
Step 1020, outputting the third image recognition data as the final recognition data to a second interactive interface for the doctor to obtain the final recognition data through the second interactive interface.
Illustratively, after the medical image is input into the first image recognition model, the output first image recognition data needs to be verified twice, first, the first image recognition data is verified for the first time through a preset confidence threshold in the system, and then, the doctor verifies the first image recognition data for the second time by outputting the first image recognition data to a first interactive interface (actually, an electronic device with a display and input function). When the first image identification data is not verified in any one of the two verifications, the verification result of the first image identification data is determined to be an error, the erroneous first image identification data is output to the second interactive interface, the medical image expert modifies the erroneous first image identification data, and the modified image identification data is the sample data.
Note that the recognition result received by the doctor is image recognition data pushed by the system, and the doctor does not actually know what processing the image recognition data has been subjected to. In this embodiment, the identification result modified by the medical image specialist and transmitted to the doctor is set to be correct, so as to avoid unnecessary verification of the image identification data modified by the medical image specialist by the doctor, a label "confirmed by the specialist" may be added to the identification result modified by the medical image specialist and transmitted to the doctor, and when the doctor determines that the label exists in the received identification result, the identification result may be directly transmitted to the patient, or the next treatment may be performed based on the identification result.
Fig. 5 is a flowchart illustrating an updating method of a medical image recognition model according to the embodiment shown in fig. 3, and as shown in fig. 5, the step 103 may include: steps 1031 and 1032, or steps 1031, 1033 and 1034.
Step 1031, comparing the recognition performances of the first image recognition model and the second image recognition model.
The identification performance is an identification rate obtained by testing the image identification model through a preset medical image data set.
Step 1032, when the recognition performance of the first image recognition model is smaller than that of the second image recognition model, the second image recognition model is used as the updated image recognition model.
Step 1033, when the recognition performance of the first image recognition model is greater than or equal to the recognition performance of the second image recognition model, training the initial model selected from the model library to obtain the third image recognition model with better recognition performance than the first image recognition model.
For example, the model library may be a model library storing a plurality of already constructed candidate image recognition models with different neural network layer numbers, and the first image recognition model is also included therein. In the process of obtaining the third image recognition model, all the candidate image recognition models in the model library except the first image recognition model can be retrained and tested through the preset medical image data set, and the corresponding recognition rate is obtained. And comparing the first image recognition model with the recognition rate of the first image recognition model respectively to obtain the third image recognition model.
Step 1034, the third image recognition model is used as the updated image recognition model.
In summary, the present disclosure can input a medical image to be recognized into a current image recognition model, so as to obtain sample data from image recognition data output by the current image recognition model according to a verification result of the image recognition data output by the current image recognition model, where the current image recognition model is a first image recognition model trained in advance, the verification result is determined by using a preset confidence threshold and a judgment result input by a doctor, and the sample data is data obtained by modifying image recognition data of which the verification result is incorrect by a medical image specialist; when the number of the sample data reaches a first preset number, retraining the first image recognition model through the first preset number of the sample data to obtain a second image recognition model; comparing the recognition performance of the first image recognition model with that of the second image recognition model to obtain an updated image recognition model, wherein the updated image recognition model is the second image recognition model or a third image recognition model, and the third image recognition model is an image recognition model which is obtained by training an initial model selected from a model library and has better recognition performance than the first image recognition model; and identifying the currently input medical image through the updated image identification model. The method can add the modification and judgment steps of doctors and medical image experts in the process of identifying the medical image, and continuously train and update the image identification model in real time through the medical image and the correct identification result while obtaining the medical image identification result, thereby ensuring the growth, adaptability and accuracy of the image identification model.
Fig. 6 is a block diagram illustrating an apparatus for recognizing medical images according to an exemplary embodiment, and as shown in fig. 6, the apparatus 600 may include:
a sample obtaining module 610, configured to input a medical image to be identified into a current image identification model, so as to obtain sample data from image identification data output by the current image identification model according to a verification result of the image identification data output by the current image identification model, where the current image identification model is a first image identification model trained in advance, the verification result is determined by using a preset confidence threshold and a judgment result input by a doctor, and the sample data is data obtained by modifying image identification data with an incorrect verification result by a medical imaging specialist;
a model retraining module 620, configured to retrain the first image recognition model according to the first preset number of sample data when the number of the sample data reaches a first preset number, so as to obtain a second image recognition model;
a model updating module 630, configured to compare the recognition performance of the first image recognition model with that of the second image recognition model to obtain an updated image recognition model, where the updated image recognition model is the second image recognition model or a third image recognition model, and the third image recognition model is an image recognition model with recognition performance better than that of the first image recognition model obtained by training an initial model selected from a model library;
and the image recognition module 640 is configured to recognize the currently input medical image through the updated image recognition model.
Fig. 7 is a block diagram of another medical image recognition apparatus according to the embodiment shown in fig. 6, and as shown in fig. 7, the apparatus 600 further includes:
a cyclic update module 650, configured to take the updated image recognition model as the current image recognition model, and cyclically execute the steps from inputting the medical image to be recognized into the current image recognition model, to obtain sample data from the image recognition data output by the current image recognition model according to the verification result of the image recognition data output by the current image recognition model, to recognizing the currently input medical image through the updated image recognition model.
Fig. 8 is a block diagram of an apparatus for recognizing medical images according to the embodiment shown in fig. 7, wherein the apparatus 600 further includes:
a model training module 660, configured to train the first image recognition model through a second preset amount of training data, where the training data includes: sample medical images and diagnostic information corresponding to the sample medical images, the diagnostic information comprising: disease diagnosis, at least one of lesion area and severity of disease condition.
Fig. 9 is a block diagram illustrating a sample acquisition module according to the embodiment shown in fig. 8, wherein the sample acquisition module 610, as shown in fig. 9, comprises:
the image analysis sub-module 6101 is configured to use the first medical image as the input of the current image recognition model to obtain the first image recognition data output by the current image recognition model, where the first image recognition data includes: the first medical image, the diagnosis information corresponding to the first medical image and the confidence of the diagnosis information comprise: disease diagnosis, at least one of lesion area and severity of disease condition;
a first data determining sub-module 6102, configured to determine that the verification result of the first image identification data is an error when the confidence level is less than the confidence level threshold;
a first data output sub-module 6103, configured to output the first image identification data to a first interactive interface, so that the medical image expert modifies the first image identification data into a second image identification data through the first interactive interface;
a first data obtaining sub-module 6104 for obtaining the second image identification data as the sample data;
a second data output sub-module 6105, configured to output the second image identification data as final identification data to a second interactive interface, so that a doctor can obtain the final identification data through the second interactive interface;
or,
a second data determining sub-module 6106, configured to output the first image identification data to a second interactive interface when the confidence level is greater than or equal to the confidence level threshold, so that the doctor can verify the first image identification data through the second interactive interface;
a first data verification sub-module 6107, configured to determine that the first image identification data is the final identification data when the first image identification data acquired through the second interactive interface passes the verification of the doctor;
a second data verification sub-module 6108, configured to output the first image identification data to the first interactive interface when the first image identification data acquired through the second interactive interface fails to be verified by the doctor, so that the medical image specialist modifies the first image identification data into third image identification data through the first interactive interface;
a second data obtaining sub-module 6109, configured to use the third image identification data as the sample data;
the second data output sub-module 6110 is configured to output the third image identification data as final identification data to the second interactive interface, so that the doctor can obtain the final identification data through the second interactive interface.
FIG. 10 is a block diagram illustrating a model update module according to the embodiment shown in FIG. 8, such as the model update module 630 shown in FIG. 10, including:
a performance comparison sub-module 631, configured to compare recognition performances of the first image recognition model and the second image recognition model, where the recognition performances are recognition rates obtained by testing the image recognition models through a preset medical image data set;
a first model updating submodule 632, configured to take the second image recognition model as the updated image recognition model when the recognition performance of the first image recognition model is smaller than the recognition performance of the second image recognition model; or,
a model obtaining sub-module 633, configured to train an initial model selected from the model library when the recognition performance of the first image recognition model is greater than or equal to the recognition performance of the second image recognition model, so as to obtain the third image recognition model with recognition performance superior to that of the first image recognition model;
the second model updating sub-module 634 is configured to use the third image recognition model as the updated image recognition model.
In summary, the present disclosure can input a medical image to be recognized into a current image recognition model, so as to obtain sample data from image recognition data output by the current image recognition model according to a verification result of the image recognition data output by the current image recognition model, where the current image recognition model is a first image recognition model trained in advance, the verification result is determined by using a preset confidence threshold and a judgment result input by a doctor, and the sample data is data obtained by modifying image recognition data of which the verification result is incorrect by a medical image specialist; when the number of the sample data reaches a first preset number, retraining the first image recognition model through the first preset number of the sample data to obtain a second image recognition model; comparing the recognition performance of the first image recognition model with that of the second image recognition model to obtain an updated image recognition model, wherein the updated image recognition model is the second image recognition model or a third image recognition model, and the third image recognition model is an image recognition model which is obtained by training an initial model selected from a model library and has better recognition performance than the first image recognition model; and identifying the currently input medical image through the updated image identification model. The method can add the modification and judgment steps of doctors and medical image experts in the process of identifying the medical image, and continuously train and update the image identification model in real time through the medical image and the correct identification result while obtaining the medical image identification result, thereby ensuring the growth, adaptability and accuracy of the image identification model.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
Fig. 11 is a block diagram illustrating an electronic device 1100 in accordance with an example embodiment. As shown in fig. 11, the electronic device 1100 may include: a processor 1101, a memory 1102, multimedia components 1103, input/output (I/O) interfaces 1104, and communication components 1105.
The processor 1101 is configured to control the overall operation of the electronic device 1100, so as to complete all or part of the steps in the above-mentioned medical image recognition method. The memory 1102 is used to store various types of data to support operation at the electronic device 1100, such as instructions for any application or method operating on the electronic device 1100, as well as application-related data, such as contact data, messaging, pictures, audio, video, and so forth. The Memory 1102 may be implemented by any type or combination of volatile and non-volatile Memory devices, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk, or optical disk. The multimedia components 1103 may include screen and audio components. Wherein the screen may be, for example, a touch screen and the audio component is used for outputting and/or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may further be stored in the memory 1102 or transmitted through the communication component 1105. The audio assembly also includes at least one speaker for outputting audio signals. The I/O interface 1104 provides an interface between the processor 1101 and other interface modules, such as a keyboard, mouse, buttons, and the like. These buttons may be virtual buttons or physical buttons. The communication component 1105 provides for wired or wireless communication between the electronic device 1100 and other devices. Wireless Communication, such as Wi-Fi, bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination of one or more of them, so that the corresponding Communication component 1105 may include: Wi-Fi module, bluetooth module, NFC module.
In an exemplary embodiment, the electronic Device 1100 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components for performing the above-mentioned medical image recognition method.
In another exemplary embodiment, a computer readable storage medium comprising program instructions, such as the memory 1102 comprising program instructions, executable by the processor 1101 of the electronic device 1100 to perform the medical image identification method described above is also provided.
Preferred embodiments of the present disclosure are described in detail above with reference to the accompanying drawings, however, the present disclosure is not limited to the specific details of the above embodiments, and other embodiments of the present disclosure may be easily conceived by those skilled in the art within the technical spirit of the present disclosure after considering the description and practicing the present disclosure, and all fall within the protection scope of the present disclosure.
It should be noted that the various features described in the above embodiments may be combined in any suitable manner without departing from the scope of the invention. Meanwhile, any combination can be made between various different embodiments of the disclosure, and the disclosure should be regarded as the disclosure of the disclosure as long as the combination does not depart from the idea of the disclosure. The present disclosure is not limited to the precise structures that have been described above, and the scope of the present disclosure is limited only by the appended claims.