CN107203989A - End-to-end chest CT image dividing method based on full convolutional neural networks - Google Patents

Abstract

The invention discloses an end-to-end chest CT image segmentation method based on a fully convolutional neural network. First, k groups of chest CT images are clinically scanned, and each group of CT images is separated into each slice as a training sample; then for each training Samples and test samples are manually segmented by professional medical staff, and the image is divided into four parts: lung, trachea, skin and background; next, an end-to-end fully convolutional neural network is constructed to perform the labeled chest CT training data Training to obtain the trained parameter model; separate each slice of the scanned CT image, and input the trained model one by one to obtain the segmented output model; finally combine the output models to obtain the segmented chest CT image model. The convolutional neural network model proposed by the present invention uses image neighborhood content for feature extraction, which can perform dense prediction on chest CT images, and simplifies the process of image segmentation.

Description

End-to-End Chest CT Image Segmentation Method Based on Fully Convolutional Neural Network

technical field

The invention belongs to the field of medical image processing, and in particular relates to an end-to-end chest CT image segmentation method based on a fully convolutional neural network.

Background technique

With the continuous and rapid development of medical imaging and computer technology, the use of computer technology to analyze clinical image data has improved the probability of successful disease prevention and treatment. Computed tomography (CT) is most commonly used in the diagnosis and detection of thoracic diseases. Since CT can provide high-resolution scanning images for various organs or tissues of the chest, it is very important to make full use of CT scanning images for the detection of lung diseases, such as lung cancer, pulmonary nodules and other diseases. In the computer-aided diagnosis system, accurate segmentation of lung CT scan images is the basis and premise of subsequent chest function analysis and three-dimensional image data reconstruction. Accurate segmentation can not only increase the accuracy of disease diagnosis, but also reduce the time of subsequent irrelevant calculations.

Because the lungs are filled with air, the CT value of the lungs is very low compared to other tissues and organs of the chest, but because the difference in density between the trachea and the surrounding tissue is not too large, the CT value of the trachea is relatively low relative to the lungs CT value is higher. However, due to the similar density of the trachea and chest skin, the contrast between the trachea and the skin in chest CT images is very low, and it is difficult to directly identify the trachea from the CT image. Under special physiological and pathological conditions, the morphological differences of various parts of chest CT are more obvious, which makes the segmentation of chest CT images a real challenge.

The segmentation methods of chest CT images generally include threshold method, region growing method and methods combined with other theories. The threshold method can easily segment the lungs and other organs based on the different CT values of chest CT images, but it is really difficult to segment the trachea whose CT value is similar to that of the skin. In the chest CT image segmentation of the region growing method, it is necessary to set the seed point of the region growing first. When the lung or trachea appears to bifurcate and cause the trachea image in the CT image to be discontinuous, the segmentation is difficult to achieve satisfactory results. Combining other theoretical methods for chest CT image segmentation also has many problems, such as the processing time of the algorithm is too long, which cannot meet the real-time requirements of the application in the computer-aided diagnosis system. In view of the particularity of chest CT image segmentation in computer-developed diagnostic systems, the image segmentation system needs to correctly classify chest mucosal adhesion nodules, blood vessels and other tissues, and also needs to achieve fast and accurate CT image segmentation. The key to image segmentation.

There are many problems in the segmentation of chest CT images using traditional methods, such as pleural adhesive nodules on the edge of the lung and damage to the edge of the lung parenchyma. Aiming at these problems, many related studies have appeared: such as detecting Concave-convex points, divide all the pixels in the image into convex points, concave points and smooth points, you can use the raised points along the contour to draw a straight line to correct the edge of the lung; curvature analysis method, the curvature comparison in this method Larger places represent the trachea or large blood vessels. The image segmentation method of the snake model combines the snake model and physiological anatomy to segment chest CT images. However, these segmentation methods also have certain shortcomings. The detection of concave and convex points is more sensitive to the deformation model. The parameters and initialization of each model in the curvature analysis method will have certain errors. The Snake model does not have a standardized segmentation process, and the segmentation results will appear. Certain error.

Contents of the invention

Aiming at the problems existing in chest CT image segmentation, the present invention combines deep learning technology and proposes a chest CT image segmentation method with end-to-end structural properties. The model includes the interaction interval between the convolutional layer and other layers, which can automatically extract the features of the input image to use the neighborhood information of the content in the image, and through the setting of the pooling layer, it can expand the network without increasing the amount of calculation. The receptive field of the convolution kernel obtains more information, and uses this information to classify the pixels in the image, and then segment the entire CT image.

In order to achieve the above-mentioned purpose, the technical solution proposed by the present invention is an end-to-end chest CT image segmentation method based on a fully convolutional neural network, specifically comprising the following steps:

Step 1: Clinically scan k groups of chest CT images, and separate each group of CT images according to each slice as a training sample;

Step 2: For each training sample and test sample, professional medical staff will manually segment the image into four parts: lung, trachea, skin and background;

Step 3: Construct an end-to-end fully convolutional neural network to train the marked chest CT training data to obtain a trained parameter model;

Step 4: Separate each slice of the scanned CT image, and input the trained model one by one to obtain a segmented output model;

Step 5: Combine the output models to finally obtain the segmented chest CT image model.

Further, the training in the above step 3 specifically includes the following steps:

A. Set up three convolutional layers in front of the model. The relationship between the input and output of the convolutional layer is:

y _ij ＝f _ks ({x _{si+δi,sj+δj} } _{0≤δi,δj≤k} )

x _ij is the data vector at position (i, j) in a certain layer, y _ij is the data vector at a certain position in the next layer, k is the kernel size, s is the stride or subsampling factor;

B. Set a layer of nonlinear activation unit after each convolutional layer. The activation function selected here is ReLU. The nonlinear calculation is in-situ calculation without increasing the storage space of the model. The input-output relationship is:

y _ij = max(0, x _ij )

x _ij here is the amount of data at the same position as the output y _ij ;

C. Set a pooling layer after each nonlinearly activated convolutional layer;

D. Set the convolutional layer after the third pooling layer, which maintains the spatial structure of the output feature map of the previous convolutional layer;

E. A deconvolution layer is set in the network, which maps the feature map in the network to the size of the original image. The deconvolution layer is the reverse transmission process of the data in the convolution layer, which can be regarded as an interpolation process , can also be regarded as a sparse filter to amplify, and the output produced is:

Both i and j here are based on the 0-value f' _ij as the output feature map;

F. Set the loss function of the network, define the loss function as the sum of each dimension in the space, and the form is:

l(x:Θ) is the total loss, l'(x _ij ; Θ) is the loss of each point, Θ is the parameter in the entire network, the loss here is Softmax loss, and the output is the loss of 4 categories, the formula as follows:

The value of k here is 4, which is divided into four parts: lungs, trachea, skin and background. The loss function of each category is equivalent to:

The loss here is the cross entropy loss;

G. Use the processed data to train the defined network on the deep learning platform. After a period of training, save the trained model.

Preferably, the maximum pooling with a stride of 2 is selected in the above step C.

The convolutional layer in the above step D is two 1×1 convolutional layers.

Compared with prior art, the beneficial effect of the present invention:

Compared with the existing technology of chest CT image segmentation, the present invention has the following advantages:

(1) The present invention proposes a convolutional neural network model utilizing image neighborhood content for feature extraction, and the depth model can be used for intensive prediction of chest CT images;

(2) The nonlinear deep learning model that learns directly from the image to the labeled image has end-to-end structural characteristics, which simplifies the process of image segmentation;

(3) Experimental results show that applying the model constructed by the present invention to the segmentation work of chest CT images can effectively divide chest CT images into lungs, trachea, skin and background 4 parts. A new approach is provided for the CT image segmentation required for the diagnosis of chest diseases.

Description of drawings

Fig. 1 is the overall flow chart of the present invention.

Figure 2 is a chest CT image and the segmented label image.

detailed description

The present invention is described in further detail now in conjunction with accompanying drawing. The drawings described here are used to provide a further understanding of the present invention and constitute a part of the application. The schematic embodiments of the present invention and their illustrations are only used to explain the present invention, and do not constitute improper limitations to the present invention.

As shown in Figure 1, the present invention comprises the following steps:

Step 1: Clinically scan k groups of chest CT images, and separate each group of CT images according to each slice as a training sample;

In the process of obtaining chest CT images, CT scanning is a spiral scan. The scanning process is generally that when the X-ray light source and detector rotate around the patient, the patient's bed moves slowly along the direction of the rotation axis. . Modern CT uses two-dimensional multi-row detectors to collect cone beam data, and some systems are equipped with multiple X-ray light sources and multiple two-dimensional detectors. The helical trajectory of the X-ray light source is achieved by rotating the light source around the patient and simultaneously moving the patient along the axis of rotation. The imaging of chest CT images is related to the attenuation coefficient, and the attenuation coefficient is related to the properties of the material, and the attenuation coefficient is usually recorded as μ. It is defined as the logarithm of the ratio of the input to output intensity of the glazing per unit length. Usually, the bone attenuation coefficient value is larger, while the soft tissue attenuation coefficient value is smaller. The attenuation coefficient of material is also related to the energy of X-rays. When the energy of X-rays is higher, the attenuation coefficient is smaller. The resulting CT image size is 512×512, and the number of scanned slices is not equal.

Step 2: For each training sample and test sample, professional medical staff will manually segment the image into four parts: lung, trachea, skin and background;

Step 3: Construct an end-to-end fully convolutional neural network to train the marked chest CT training data to obtain a trained parameter model;

Step 4: Separate each slice of the scanned CT image, and input the trained model one by one, and obtain a segmented output model;

Step 5: Combine the output models to finally obtain the segmented chest CT image model.

As shown in FIG. 2 , the first row of images is the chest image to be segmented, and the second row is the corresponding segmented chest CT image.

The construction of the deep learning model in the above step 3, the constructed model includes the following parts:

(1) Three convolutional layers are set in front of the model, and the relationship between the input and output of the convolutional layer is:

y _ij ＝f _ks ({x _{si+δi,sj+δj} } _{0≤δi,δj≤k} )

x _ij is the data vector at position (i, j) in a certain layer, y _ij is the data vector at a certain position in the next layer, k is the kernel size, s is the stride or subsampling factor;

(2) Set a layer of nonlinear activation unit after each convolutional layer. The activation function selected here is ReLU. The nonlinear calculation is in-situ calculation without increasing the storage space of the model. The input-output relationship is:

y _ij = max(0, x _ij )

Here, x _ij is the amount of data at the same position as the output y _ij .

(3) A pooling layer is set after each non-linearly activated convolutional layer. Here, the maximum value pooling with a stride of 2 is selected.

(4) Two 1×1 convolutional layers are set after the third pooling layer, and the convolutional layer maintains the previous output and maintains the spatial structure of the feature map.

(5) A deconvolution layer is set in the network, which maps the feature map in the network to the size of the original image. The deconvolution layer is the reverse transmission process of the data in the convolution layer, which can be regarded as an interpolation The process can also be regarded as a sparse filter to amplify, and the output produced is:

Both i and j here are based on the 0-valued f′ _ij as the output feature map.

(6) Set the loss function of the network, and define the loss function as the sum of each dimension in the space, in the form of:

l(x:Θ) is the total loss, l'(x _ij ;Θ) is the loss of each point, and Θ is the parameter in the entire network. The loss here is Softmax loss, and the output is the loss of 4 categories. The formula is as follows:

The value of k here is 4, which is divided into four parts: lungs, trachea, skin and background. The loss function of each category is equivalent to:

The loss here is the cross entropy loss;

(7) Use the processed data to train the defined network on the deep learning platform, and after a period of training, save the trained model.

It should be noted that the present invention is not limited to the specific technical solutions described in the above embodiments, and all technical solutions formed by equivalent replacements belong to the protection scope of the present invention.

Claims (4)

1. based on the end-to-end chest CT image segmentation method of full convolutional neural network, it is characterized in that the method comprises the following steps: Step 1: Clinically scan k groups of chest CT images, and separate each group of CT images according to each slice as a training sample; Step 2: For each training sample and test sample, professional medical staff perform manual segmentation to divide the image into four parts: lung, trachea, skin and background; Step 3: Construct an end-to-end fully convolutional neural network to train the marked chest CT training data to obtain a trained parameter model; Step 4: Separate each slice of the scanned CT image, and input the trained model one by one to obtain a segmented output model; Step 5: Combine the output models to finally obtain the segmented chest CT image model. 2. the end-to-end chest CT image segmentation method based on full convolution neural network according to claim 1, is characterized in that the training in step 3 specifically comprises the following steps: A. Set up three convolutional layers in front of the model. The relationship between the input and output of the convolutional layer is: y _ij ＝f _ks ({x _{si+δi,sj+δj} } _{0≤δi,δj≤k} ) x _ij is the data vector at position (i, j) in a specific layer, y _ij is the data vector at a certain position in the next layer, k is the kernel size, s is the stride or subsampling factor; B. Set a layer of nonlinear activation unit after each convolutional layer. The activation function selected here is ReLU. The nonlinear calculation is in-situ calculation without increasing the storage space of the model. The input-output relationship is: y _ij = max(0, x _ij ) x _ij here is the amount of data at the same position as the output y _ij ; C. Set a pooling layer after each nonlinearly activated convolutional layer; D. Set the convolutional layer after the third pooling layer, which maintains the spatial structure of the output feature map of the previous convolutional layer; E. A deconvolution layer is set in the network, which maps the feature map in the network to the size of the original image. The deconvolution layer is the reverse transmission process of the data in the convolution layer, which can be regarded as an interpolation process , can also be regarded as a sparse filter to amplify, and the output produced is: Both i and j here are based on the 0-value f' _ij as the output feature map; F. Set the loss function of the network, define the loss function as the sum of each dimension in the space, and the form is: l(x:Θ) is the total loss, l'(x _ij ; Θ) is the loss of each point, Θ is the parameter in the entire network, the loss here is Softmax loss, and the output is the loss of 4 categories, the formula as follows: The value of k here is 4, which is divided into four parts: lungs, trachea, skin and background. The loss function of each category is equivalent to: The loss here is the cross entropy loss; G. Use the processed data to train the defined network on the deep learning platform. After a period of training, save the trained model. 3. The end-to-end chest CT image segmentation method based on a full convolutional neural network according to claim 2, characterized in that what is selected in the C step is that the stride is a maximum pooling of 2. 4. The end-to-end chest CT image segmentation method based on a full convolutional neural network according to claim 2, wherein the convolutional layer in the D step is a two-layer 1 * 1 convolutional layer.