CN112116625B - Automatic cardiac CT image segmentation method, device and medium based on contradiction labeling method - Google Patents

Abstract

The invention relates to a method, a device and a medium for automatically segmenting cardiac CT images based on a contradictory labeling method, which comprises the following steps: performing low-precision contradiction labeling on the heart CT image to obtain a first contradiction labeling set; taking similar annotations included in contradiction annotation pictures in different frames but adjacent pictures in the CT images of the plurality of hearts as a second contradiction annotation set; preheating a deep neural network based on U-Net through a precise annotation data part, and mixing a first contradiction annotation set and a second contradiction annotation set to obtain a mixed contradiction annotation set; training a full convolution neural network through the mixed contradiction labeling set, and segmenting the foreground and the background of the heart CT image until the full convolution neural network achieves convergence; and counting gray level histograms of foreground and background of CT heart series images segmented by the U-Net network, segmenting a threshold, setting a region of interest of the heart CT images, and segmenting. The beneficial effects of the invention are as follows: the network training process is faster; the time cost is relatively low, and the cost is reduced.

Description

Method, device and medium for automatic segmentation of cardiac CT images based on contradictory labeling method

Technical Field

The present invention relates to the field of computers, and in particular to a method, a device and a medium for automatically segmenting cardiac CT images based on a contradictory labeling method.

Background Art

According to the World Health Organization, tens of millions of people die from cardiovascular disease (heart disease) each year, accounting for an estimated 31% of all deaths worldwide. Cardiac computed tomography (CT) images are widely used in radiotherapy plans, including heart disease, and automatic segmentation of cardiac CT images has become a hot research topic in recent years. Complete cardiac segmentation based on medical CT images refers to tissue segmentation of the entire sequence of CT cardiac images. The segmentation results are of great significance in assisting doctors in diagnosing cardiovascular diseases and guiding doctors in surgery.

Due to the complexity of cardiac samples and the serious interference of noise, the boundaries of cardiac tissue structures in CT images are often blurred. Currently, the two main technologies known for automatic segmentation of cardiac CT medical images are: active contour model technology, which is widely used to segment lung CT images; and machine learning methods, such as convolutional neural networks (CNN). There have been some research reports on how to segment a single atrium or ventricle from a cardiac image, but there are few reports on the segmentation results of the entire cardiac CT sequence image. At present, most full-sequence CT cardiac segmentation requires semi-automatic segmentation under the guidance of professional doctors through human-computer interaction, and it costs a lot of money to label a mature, complete and convincing public data set.

Due to the diversity and complexity of cardiac samples, it is extremely difficult to annotate related medical images. Therefore, the annotation and accurate segmentation of the tissue structure of CT cardiac images has become a major challenge.

Summary of the invention

The purpose of the present invention is to solve at least one of the technical problems existing in the prior art, and to provide a method, device and medium for automatic segmentation of cardiac CT images based on contradictory labeling method, which is simple to implement and reduces costs while ensuring labeling accuracy.

The technical solution of the present invention includes an automatic segmentation method for cardiac CT images based on a contradictory labeling method, which is characterized by: S100, low-precision contradictory annotations are performed on cardiac CT images to obtain contradictory annotation maps of two similar annotation information of the same cardiac CT image, and the contradictory annotations included in the contradictory annotation map in the same frame are used as a first contradictory annotation set; S200, similar annotations included in the contradictory annotation maps in different frames but adjacent images of multiple cardiac CT images are used as a second contradictory annotation set; S300, a U-Net-based deep neural network is preheated through a precise annotation data part, and the first contradictory annotation set and the second contradictory annotation set are mixed to obtain a mixed contradictory annotation set; S400, a fully convolutional neural network is trained through the mixed contradictory annotation set to segment the foreground and background of the cardiac CT image until the fully convolutional neural network reaches convergence; S500, grayscale histograms of the foreground and background of the CT cardiac series images segmented by the U-Net network are counted, the lowest valley between the two peaks of the histogram is used as a segmentation threshold, and the region of interest of the cardiac CT image is set and segmented.

According to the method for automatic segmentation of cardiac CT images based on the contradictory labeling method, the contradictory annotations in the first contradictory labeling set are 5% of all the labeled points in the same frame of the plurality of cardiac CT images.

According to the method for automatic segmentation of cardiac CT images based on contradictory labeling, the similar annotation information of the second contradictory labeling set is 30% of all the annotation points of different frames but adjacent images in the plurality of cardiac CT images.

According to the method for automatic segmentation of cardiac CT images based on the contradictory labeling method, the low-precision contradictory annotation is set to: contradictory annotation performed by an annotation set corresponding to an annotation point number lower than that of a standard annotation set, or contradictory annotation performed by an annotation set that has not been fully authenticated by experts in the relevant field.

According to the method for automatic segmentation of cardiac CT images based on contradictory labeling, the pre-training includes: performing high generalization segmentation training with a mixed contradictory labeling set.

According to the method for automatic segmentation of cardiac CT images based on contradictory labeling, the fully convolutional neural network model is set to a U-Net fully convolutional neural network model.

According to the method for automatic segmentation of cardiac CT images based on contradictory labeling, S400 includes: training the U-Net full convolutional neural network model with the mixed contradictory labeling set is a decreasing learning, and the learning rate of the decreasing learning is 10%.

According to the method for automatic segmentation of cardiac CT images based on contradictory labeling, S500 includes: using a statistical U-Net fully convolutional neural network model to segment the foreground and background discrete data of a CT cardiac series image, using a grayscale histogram hist as the model output for statistical purposes, obtaining two corresponding peaks, using the trough between the two peaks, setting an interest threshold and completing automatic segmentation of the region of interest of the cardiac CT image, wherein the trough is 10% of the first peak.

The technical solution of the present invention also includes an automatic segmentation device for cardiac CT images based on the contradictory labeling method, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and is characterized in that the processor implements any of the method steps described when executing the computer program.

The technical solution of the present invention also includes a computer-readable storage medium, which stores a computer program, and is characterized in that when the computer program is executed by a processor, any of the method steps described is implemented.

The beneficial effects of the present invention are: the segmentation of the target region of interest of the whole CT cardiac series image meets the requirements; and the relevant neural network is trained by the contradictory annotation method, and the whole network training process is faster. This is because the contradictory annotation method does not require a large number of annotation sets, and the annotation accuracy does not need to be too high. Therefore, the time cost of preparing the annotation data set is relatively low, that is, the cost can be reduced while completing the segmentation.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is further described below in conjunction with the accompanying drawings and embodiments;

FIG. 1 is a general flow chart according to an embodiment of the present invention.

FIG. 2 is a schematic diagram of a contradictory annotation set according to an embodiment of the present invention.

FIG. 3 is a schematic diagram before and after showing contradiction marking according to an embodiment of the present invention.

FIG. 4 is a diagram showing an example of a CT tomography image according to an embodiment of the present invention.

FIG. 5 is a diagram showing the result of segmenting the region of interest according to an embodiment of the present invention.

FIG. 6 is a diagram showing a device medium according to an embodiment of the present invention.

DETAILED DESCRIPTION

This section will describe in detail the specific embodiments of the present invention. The preferred embodiments of the present invention are shown in the accompanying drawings. The purpose of the accompanying drawings is to supplement the description of the text part of the specification with graphics, so that people can intuitively and vividly understand each technical feature and the overall technical solution of the present invention, but it cannot be understood as a limitation on the scope of protection of the present invention.

In the description of the present invention, “several” means one or more, “more” means more than two, “greater than”, “less than”, “exceed”, etc. are understood as not including the number itself, and “above”, “below”, “within”, etc. are understood as including the number itself.

In the description of the present invention, the consecutive numbering of the method steps is for the convenience of review and understanding. Combined with the overall technical scheme of the present invention and the logical relationship between the various steps, adjusting the implementation order between the steps will not affect the technical effect achieved by the technical scheme of the present invention.

FIG1 shows an overall flow chart according to an embodiment of the present invention. The process includes: S100, low-precision contradictory annotation of cardiac CT images, obtaining contradictory annotation graphs of two similar annotation information of the same cardiac CT image, and taking the contradictory annotations included in the contradictory annotation graph in the same frame as the first contradictory annotation set; S200, taking the similar annotations included in the contradictory annotation graphs in different frames but adjacent images of multiple cardiac CT images as the second contradictory annotation set; S300, preheating the U-Net-based deep neural network through the precise annotation data part, mixing the first contradictory annotation set and the second contradictory annotation set to obtain a mixed contradictory annotation set; S400, training the full convolutional neural network through the mixed contradictory annotation set, segmenting the foreground and background of the cardiac CT image, until the full convolutional neural network reaches convergence; S500, counting the grayscale histograms of the foreground and background of the CT cardiac series images segmented by the U-Net network, taking the lowest valley between the two peaks of the histogram as the segmentation threshold, setting the region of interest of the cardiac CT image and performing segmentation. Among them, the annotation of the cardiac CT image is generally a cardiac CT series image.

With respect to the technical solution of FIG. 1 , the present invention also proposes the following implementation means:

1. Prepare the contradictory annotation map of two similar annotation information of the same image (single image) from the CT heart series images, which contains about 5% of the contradictory annotations as the contradictory annotation set 1

2. Prepare similar annotation information from different images in the CT heart image series, including about 30% contradictory annotations (the annotation information of similar images is quite different) as contradictory annotation set 2.

3. Mix the two conflicting annotation sets prepared earlier. First, use the images with high annotation accuracy and a very small learning rate for warm-up. Then, perform full training with the complete conflicting dataset.

4. Use the prepared contradictory annotated dataset to train the U-Net network to segment the foreground and background of the CT heart series images until the network reaches convergence (the loss remains basically unchanged). The entire training process of the U-Net model is about 600 epochs, and the learning rate is decayed 3 times with 0.1, 0.01, and 0.001 respectively, using the SGD optimizer with Momentum. Finally, the segmentation accuracy of the foreground and background training of the CT heart series images reaches more than 98%; the training is terminated.

5. Since the contradictory labeling method uses contradictory labeled data, this method has good generalization. Therefore, it is not necessary to set a validation group during training. Setting a validation group does not conflict with the contradictory labeling method, but the validation group contains more contradictory data, which will hinder the evaluation of true accuracy.

6. Statistical U-Net network segmented the foreground and background discreteness of CT heart series images. After experimentation, it was found that using hist as the statistical method of model output, two peaks can be obtained. 10% is the valley between the two peaks. Set the threshold to complete the automatic segmentation of the region of interest of the heart CT image.

Fig. 2 is a schematic diagram of contradictory annotation sets according to an embodiment of the present invention, which are 2a (heart CT image), 2b (annotation set 1), and 2c (annotation set 2) from left to right.

The contradictory labeling method aims to overcome the difficulty of labeling a large number of high-precision CT cardiac image sequences. Since there are few public datasets for CT cardiac segmentation on the Internet, there is a lack of sufficient labeled datasets for the development of segmentation algorithms; and the preparation of labeled datasets for CT cardiac segmentation requires professional knowledge of CT cardiac images to produce high-precision labeled data. The data to be labeled is not only large in quantity but also diverse and complex in samples, and it takes a long time to complete accurately. Therefore, the labeling of CT cardiac images is very difficult. The contradictory labeling method is designed to allow for lower labeling accuracy, different labeling results may not be completely consistent, and the data volume is small. By using small data to train the network; this not only reduces the difficulty of labeling and reduces the annotation cost, but also avoids the problem of overfitting of network training usually caused by small training datasets. That is, the design concept of the contradictory labeling method is to train the neural network by labeling certain small datasets with contradictions. The contradictory data used in the design of the contradictory labeling method mainly includes two types: contradictory data of the same image and contradictory data of different images:

The same image uses two different, contradictory but similar annotation sets, as shown in Figure 2. Images (2b) and (2c) in Figure 2 are both segmentation annotation sets of the same image (2a). They are similar to each other, but also slightly different. A careful observation shows that the two annotation data sets shown in images (2b) and (2c) are basically the same, but there are indeed individual or minor differences, such as the protruding part in the lower left corner and the shape division of the middle area, which show that the two annotation point sets are not exactly the same. These differences are the contradictions between the two annotation sets. Of course, these differences or contradictions will not and should not be too large. The purpose of this annotation method is to force the neural network to pay more attention to the commonalities between images when learning. And through the automatic learning of the neural network, the two annotation results are combined to find better edge information than the annotation results (for example, making the edges smoother). Using two different, contradictory but similar annotation sets also prevents over-training of the network and prevents problems such as overfitting of the network caused by small data sets and insufficient annotation accuracy.

FIG3 is a schematic diagram of contradictory annotations before and after according to an embodiment of the present invention. Each column is 3a (cardiac CT image) and 3b (annotated image) from left to right. Different or adjacent images contain different segmentation annotation sets. CT cardiac sequence images have a high degree of similarity between adjacent layers, and similar faults in different cardiac sequence images also have certain similarities. The different annotation sets of similar images can better force the neural network to learn the commonalities between images. The errors of segmentation are corrected by automatically learning the neural network to prevent overfitting, etc. Of course, this will lead to some loss of segmentation accuracy, but the experimental results show that the so-called loss of segmentation accuracy is acceptable under the premise that the annotation data set itself is not accurate. In some details, these losses in segmentation accuracy can also be used to correct the segmentation annotation set. Another advantage of selecting similar images instead of identical images is that the available data set can be expanded. If only different annotations are performed on the same images, overfitting may occur in actual training.

The above two types of contradictory labeled data reflect the essence of the contradictory labeling method. The design concept is to use the contradictory labeling method to annotate only a relatively small amount of low-precision data annotations and quickly train a model with good generalization ability. In order to verify the effectiveness of the contradictory labeling method, we constructed the above two types of contradictory labeled data and experimentally trained a model to verify our theory. In order to simply verify the contradictory labeling method, the experimental training process did not include other data augmentation functions, nor did it use early stopping and other methods. The original U-Net model was used for training until the model reached extreme convergence (the loss remained basically unchanged). This verifies the generalization ability of the model trained by the above two types of contradictory data. The experimental results show that with only a small number of low-precision labeled images, the neural network trained by the contradictory labeling method can automatically generate the foreground part of the medical image (that is, the main area of interest to the doctor).

Low-precision annotation set or high-precision, high and low are relative concepts in themselves; the definition of annotation set accuracy will have different ranges according to different processing objects and segmentation purpose requirements, and there is no unified definition. All "datasets that have not been fully approved by experts in related fields and have no endorsement from relevant institutions or experts" should belong to low-precision annotation sets. The low-precision annotation set used in "Automatic Segmentation Method of Region of Interest in Cardiac CT Images Based on Contradictory Annotation Method" refers to a low-precision annotation set that has not been fully approved by experts in related fields, has relatively few annotation points, and the results of two manual annotations in the same picture can be seen to be different by naked eye, or the annotation results in different pictures can be seen to be different by naked eye. These two types of low-precision annotation sets are the contradictory annotation datasets used by the contradictory annotation method training network.

A new CT cardiac image segmentation method is designed based on the aforementioned contradictory annotation method. This method can well divide CT cardiac images into foreground (ROI) and background. This new method uses the contradictory annotation method as the core to construct a contradictory annotation set to complete network training, and then completes the segmentation of the region of interest through the threshold. The segmentation threshold is 10%, and the discrete situation of the foreground and background can basically meet the segmentation requirements.

Through the analysis and statistics of the model output results, we found that the trained CT heart segmentation model has the following characteristics: Due to the limitation of the accuracy of the original data annotation, the model training has a certain impact on the effect of the segmentation details. However, due to the enhanced generalization ability, the results of the model training have a good effect on the segmentation of the image foreground and background. Therefore, the model can obtain the complete foreground/region of interest segmentation results. At the same time, another key point is that the contradiction training can be used as a pre-training strategy to improve the generalization strategy of the limit fitting of the precision standard data.

Fig. 4 shows an example of a CT tomographic image according to an embodiment of the present invention. Each column from left to right is 4a (a tomographic image on the 50th layer), 4b (a tomographic image on the 140th layer), and 4c (a tomographic image on the 270th layer).

Referring to Figure 4, some images of the CT heart dataset were manually annotated and used as training sets to train the neural network; ultimately a high segmentation accuracy was achieved.

The test data set briefly introduces the specific process of using CT heart data and the annotation method of contradiction in the experiment (the principle of selecting contradictions). The CT heart data used in the experiment is about 300 frames of tomography for each heart. About 220 frames of images contain contrast agents. The image to be segmented is an image containing contrast agents. The image is shown in Figure 4:

The white area is the contrast agent area. In this experiment, we finally obtained 420 valid CT heart annotated images. These 420 images contain contradictory annotations from two similar annotations of the same image, which contain about 5% contradictory annotations. The second type is similar annotation information from different images, which contains about 30% contradictory annotations (the annotation information of similar images is quite different).

Results and analysis, as shown in FIG5 , is a result diagram of the region of interest segmentation according to an embodiment of the present invention, wherein the columns from left to right are 5a (cardiac CT image), 5b (contradiction labeling method training result), and 5c (region of interest segmentation result).

The original U-Net model was used to train 420 data. At the same time, data outside the annotation set was used as a test to verify the training results. Since the contradictory labeling method uses contradictory annotated data, in order to avoid the verification group correcting the training results, we did not set up a verification group during the training process. We hope that in this way, we can show the generalization effect of the contradictory labeling method as much as possible.

The entire U-Net model was trained for a total of 600 epochs, and the learning rate was decayed 3 times by 10% each time. The final segmentation accuracy of the training was 98.06%. After verification by the test set, the extraction of the region of interest was very complete, showing a good segmentation effect. The extraction and segmentation results of the region of interest are shown in Figure 5.

The training data set is annotated by ourselves. Since it is a low-precision data set, it is temporarily impossible to complete the quantitative comparative analysis of the segmentation results; it is temporarily impossible to use the contradictory labeling method in other existing segmentation methods. Since the data set currently annotated with the contradictory labeling method has certain contradictory segmentation results, the upper limit of the final training accuracy cannot reach 100%. To meet the law of contradiction, we cannot make different annotation results of the same CT heart image be judged as correct at the same time. That is, using contradictory attributes to stimulate the unexplainable automatic learning characteristics of deep neural networks. In order to obtain good segmentation results.

FIG6 is a diagram of a device medium according to an embodiment of the present invention. The device includes a memory 100 and a processor 200, wherein the processor 200 stores a computer program, and the computer program is used to execute: low-precision contradictory annotation of cardiac CT images, and the contradictory annotations included in the contradictory annotation map are used as the first contradictory annotation set; the similar annotations included in the contradictory annotation map are used as the second contradictory annotation set; the first contradictory annotation set and the second contradictory annotation set are mixed to obtain a mixed contradictory annotation set, and the mixed contradictory annotation set is pre-trained; the full convolutional neural network is trained by the mixed contradictory annotation set to segment the foreground and background of the cardiac CT image until the full convolutional neural network reaches convergence; the foreground and background of the CT cardiac series images segmented by the U-Net network are statistically discrete, and the region of interest of the cardiac CT image is set and segmented. Among them, the memory 100 is used to store data.

The embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in the technical field without departing from the purpose of the present invention.

Claims (7)

1. An automatic cardiac CT image segmentation method based on a contradictory labeling method is characterized by comprising the following steps of:

S100, performing low-precision contradiction annotation on a heart CT image to obtain contradiction annotation graphs of two similar annotation information of the same heart CT image, and taking the contradiction annotation included in the contradiction annotation graph in the same frame graph as a first contradiction annotation set; the contradiction annotation in the first contradiction annotation set is 5% of all annotation points of the same frame image of the plurality of heart CT images;

S200, taking similar annotations included in contradiction annotation pictures in different frames but adjacent pictures in the plurality of cardiac CT images as a second contradiction annotation set; similar annotation information of the second contradictory annotation set is 30% of all annotation points of different frames but adjacent images in the plurality of cardiac CTs;

S300, preheating a deep neural network based on U-Net through a precise annotation data part, and mixing the first contradiction annotation set and the second contradiction annotation set to obtain a mixed contradiction annotation set;

s400, training a full convolution neural network through the mixed contradiction labeling set, and segmenting the foreground and the background of the heart CT image until the full convolution neural network achieves convergence; the training comprises: performing high generalization segmentation training by using a mixed contradiction labeling set; combining the labeling results of the first contradiction labeling set and the labeling results of the second contradiction labeling set through automatic learning of the full convolution neural network to find out edge information better than the labeling results;

s500, counting gray level histograms of foreground and background of the CT heart series image segmented by the U-Net network, taking the lowest valley between the double peaks of the histograms as a segmentation threshold, setting a region of interest of the heart CT image and segmenting.

2. The automatic segmentation method of cardiac CT images based on the contradictory labeling method according to claim 1, wherein the low-precision contradictory labeling is set as: and performing contradiction labeling by using a labeling set corresponding to the number of labeling points with the labeling points lower than that of the standard labeling set or performing contradiction labeling by using the labeling set which is not completely identified by experts in the related fields.

3. The automatic cardiac CT image segmentation method based on the contradictory labeling method according to claim 1, wherein the full convolutional neural network model is set as a U-Net full convolutional neural network model.

4. The automatic segmentation method for cardiac CT images based on contradictory labeling according to claim 3, wherein S400 comprises:

The mixed contradiction labeling set training U-Net full convolution neural network model is descending learning, and the learning rate of the descending learning is 10%.

5. The automatic segmentation method for cardiac CT images based on contradictory labeling according to claim 3, wherein S500 comprises:

And (3) counting foreground and background discrete data of the CT heart series image by a U-Net full convolution neural network model, counting by taking a gray-scale histogram hist as model output, obtaining two corresponding peaks, setting an interest threshold value by using a valley between the two peaks, and completing automatic segmentation of a heart CT image interest region, wherein the valley is 10% of a first peak.

6. An automatic segmentation device for cardiac CT images based on contradictory labeling, the device comprising a memory, a processor and a computer program stored in said memory and executable on said processor, characterized in that the processor implements the method steps of any one of claims 1-5 when executing said computer program.

7. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the method steps of any of claims 1-5.