CN108898160A - Breast cancer tissue's Pathologic Grading method based on CNN and image group Fusion Features - Google Patents

Abstract

The present invention relates to CNN and image classification identification technology fields, more particularly to breast cancer tissue's Pathologic Grading method based on CNN and image group Fusion Features.The present invention proposes that the CNN model merged by construction feature judges breast cancer tissue's pathology grade of molybdenum target image, gray feature, textural characteristics and the wavelet character extracted using molybdenum target tumor region, Feature Selection is carried out by LASSO logistic regression model, select the feature big with breast cancer tissue pathology rank correlation, Fusion Features are carried out in the newly added full articulamentum of network by the high-level semantics features for extracting CNN and the image group feature filtered out again, and the CNN model that fitting obtains Fusion Features is used to identify breast cancer tissue's pathology grade.The present invention can the breast molybdenum target image directly to patient scan carry out breast cancer tissue's pathology grade locating for analytical judgment patient, further shortened while guaranteeing discrimination precision differentiate the time.

Description

Breast cancer histopathological grading method based on fusion of CNN and radiomics features

technical field

The invention relates to the technical field of CNN and image classification and recognition, in particular to a histopathological grading method for breast cancer based on fusion of CNN and radiomics features.

Background technique

Breast cancer is the most common cancer among women and the second most common cause of death among women. The global incidence of breast cancer has been on the rise since the late 1970s, and many patients died of breast cancer. Mammography mammography is the first choice, the easiest and most reliable non-invasive detection method for judging breast diseases at present, and its high resolution is helpful for early detection of breast cancer.

In recent years, with the development of big data and high-performance computing, CNN (Convolutional Neural Network) has achieved remarkable results in the field of computer vision, and the recognition rate in natural image classification has exceeded the human recognition level. CNN extracts image features through multi-layer convolution and pooling, and then updates parameters through the backpropagation algorithm, which changes the limitations of human experience in previous manual design features.

The histopathological grading of breast cancer (SBR grading) is mainly based on the combined evaluation of image morphological characteristics in three aspects: the index of cancer cell mitosis, the difference of mammary ducts, and the atypia of cancer cell nuclei. The histopathological grading of breast cancer It has an important relationship with the prognosis of patients. In the same clinical stage, the 5-year survival rate of patients decreases with the increase of histopathological grade. The classification of SBR of breast cancer is mainly based on observing the differentiation of cancer cells in pathological sections of patients under a microscope. At present, doctors cannot directly judge the classification from conventional mammography images.

Contents of the invention

In view of the above problems, the present invention proposes a breast cancer histopathological grading method based on the fusion of CNN and radiomics features, which can directly analyze the mammogram images of patients, and automatically combine the artificially designed radiomics features with CNN The extracted high-level semantic features of the image are fused on the newly added fully connected layer, and the CNN model after the feature fusion is trained to obtain the histopathological grade of breast cancer of the patient, which provides a basis for further disease discrimination and prognosis analysis.

In order to achieve the above object, the present invention adopts the following technical solutions:

A breast cancer histopathological grading method based on the fusion of CNN and radiomics features, including the following steps:

Step 1: Extract the tumor area of the mammography image, calculate the grayscale, texture and wavelet features on the extracted mammography tumor area, and extract a total of 180-dimensional radiomics feature vectors through the above calculation; the extracted mammography The target image tumor area is made into breast tumor area mammography image samples of the same size, and the image samples are divided into training set, verification set and test set;

Step 2: For the extracted 180-dimensional radiomics feature vector, use the LASSO logistic regression model to perform feature screening, and use the screened radiomics features to perform feature fusion;

Step 3: Use the pre-trained CNN model for migration learning, train the CNN classification model, add a new fully connected layer on the basis of the original CNN classification model, and transfer the CNN classification model before the fully connected layer to the new fully connected layer. The output and screened radiomics features are fused, and retrained on the basis of the CNN classification model parameters, the fused CNN classification model parameters are updated, and the fused CNN classification is performed according to the classification effect of the model on the verification set The model parameters were adjusted to obtain a feature-fused CNN model, which was used for breast cancer histopathological grading on mammography images.

Further, after the step 3, it also includes:

Use the test set to verify the classification accuracy of the CNN model obtained by feature fusion.

Further, said step 1 includes:

Step 1.1: Extract the ROI of the tumor area of the mammography image to obtain the ROI image, calculate 14 gray features, 22 texture features and 144 wavelet features of the ROI image, and extract a total of 180-dimensional radiomics feature vectors;

Step 1.2: Expand the scale of the ROI image through data enhancement methods;

Step 1.3: Scale the ROI images after data scale expansion to the same size to meet the input requirements of the CNN model.

Further, said step 3 includes:

Step 3.1: Take the mammography image samples of the breast tumor area in the training set as the input of the CNN model, perform transfer learning on the CNN model pre-trained on the ImageNet natural image dataset, and train the CNN classification model;

Step 3.2: Add a new fully-connected layer on the basis of the original CNN grading model. On the new fully-connected layer, output the high-level semantic features of the breast tumor area mammography image before the fully-connected layer of the CNN grading model and use LASSOlogistic regression The radiomics features screened by the model are fused, and retrained on the basis of the CNN classification model parameters, the fused CNN classification model parameters are updated, and the fused CNN classification model parameters are updated according to the classification effect of the model on the verification set. Make adjustments to obtain the CNN model of feature fusion.

Compared with the prior art, the present invention has the beneficial effects:

The present invention proposes to judge the histopathological grade of breast cancer in mammography images by constructing a feature-fused CNN model, using the grayscale features, texture features and wavelet features extracted from the mammography tumor area, and performing feature screening through the LASSO logistic regression model to select For features that are highly correlated with the histopathological grade of breast cancer, the high-level semantic features extracted by CNN and the screened radiomics features are fused in the newly added fully connected layer of the network, and the feature-fused CNN is obtained by fitting. The model was used to identify breast cancer histopathological grades. The invention can directly analyze the mammography target image scanned by the patient to determine the histopathological grade of breast cancer of the patient, and further shorten the discrimination time while ensuring the discrimination accuracy.

Description of drawings

FIG. 1 is a basic flow chart of a breast cancer histopathological grading method based on fusion of CNN and radiomics features according to an embodiment of the present invention.

FIG. 2 is a basic flowchart of a breast cancer histopathological grading method based on fusion of CNN and radiomics features according to another embodiment of the present invention.

Fig. 3 is a mammography image of different irradiation positions in the histopathological grading method of breast cancer based on fusion of CNN and radiomics features according to an embodiment of the present invention.

Detailed ways

The present invention will be further explained below in conjunction with accompanying drawing and specific embodiment:

Embodiment one:

As shown in Figure 1, a kind of breast cancer histopathological grading method based on CNN and radiomics feature fusion of the present invention comprises the following steps:

Step S101: Extract the tumor area of the mammography image, and calculate grayscale, texture and wavelet features on the extracted mammography tumor area, and extract a total of 180-dimensional radiomics feature vectors through the above calculation; the extracted mammography The target image tumor area is made into mammary tumor image samples of the same size, and the image samples are divided into training set, verification set and test set.

Step S102: For the extracted 180-dimensional radiomics feature vector, the LASSO logistic regression model is used for feature screening, and the screened radiomics feature is used for feature fusion.

Step S103: Use the pre-trained CNN model for transfer learning, train the CNN classification model, add a new fully connected layer on the basis of the original CNN classification model, and transfer the CNN classification model before the fully connected layer to the new fully connected layer. The output and screened radiomics features are fused, and retrained on the basis of the CNN classification model parameters to obtain the fused CNN model, and the fused CNN classification model parameters are adjusted according to the classification effect of the model on the verification set. Adjustments were made to obtain a feature-fused CNN model for breast cancer histopathological grading on mammography images.

Embodiment two:

As shown in Figure 2, another breast cancer histopathological grading method based on CNN and radiomics feature fusion of the present invention includes the following steps:

Step S201: Extract the tumor area of the mammography image, calculate the grayscale, texture and wavelet features on the extracted mammography tumor area, and extract a total of 180-dimensional radiomics feature vectors through the above calculation; the extracted mammography The target image tumor area is made into mammary tumor image samples of the same size, and the image samples are divided into training set, verification set and test set.

The step S201 includes:

Step S2011: extract the ROI from the tumor area of the mammography image to obtain the ROI image, calculate 14 grayscale features, 22 texture features and 144 wavelet features of the ROI image, and extract a total of 180-dimensional radiomics feature vectors;

The grayscale features are grayscale maximum value, minimum value, mean value, median value, variance, kurtosis, energy, entropy, mean absolute variance, skewness, standard deviation, uniformity, grayscale range, root mean square total 14 features (see Aerts H J W L, Velazquez E R, Leijenaar R T H, et al. Decoding tumor phenotype by noninvasive imaging using a quantitative radiomics approach [J]. Nature communications, 2014, 5: 4006);

The texture feature is a 9-dimensional feature derived based on the gray level co-occurrence matrix, namely energy, contrast, entropy, homogeneity, correlation, variance, and average, difference, autocorrelation (see Weszka J S, Dyer CR, Rosenfeld A. A comparative study of texture measures for terrain classification [J]. IEEE transactions on Systems, Man, and Cybernetics, 1976(4): 269-285) and 13-dimensional features derived from the gray run matrix, namely Short Run Emphasis, Long Run Emphasis, Gray-LevelNonuniformity, Run-Length Nonuniformity, Run Percentage, Low Gray-Level RunEmphasis, High Gray-Level Run Emphasis, Short Run Low Gray-Level Emphasis, ShortRun High Gray-Level Emphasis, Long Run Low Gray-Level Emphasis, Long Run High Gray-Level Emphasis, Gray-Level Variance, Run-Length Variance (see Galloway MM. Texture analysis using gray level run lengths [J]. NASA STI/Recon Technical Report N, 1974, 75; Chu A, Sehgal C M ,Greenleaf J F.Use of gray value distribution of run lengths for texture analysis[J].Pattern RecognitionLetters,1990,11(6):415-419) a total of 22-dimensional features;

The wavelet features are grayscale features and texture features calculated on 4 wavelet components, 144 features in total.

Step S2012: Enlarge the scale of the ROI image through a data enhancement method; as an implementable manner, the scale of the ROI image can be expanded through a data enhancement method of random translation, rotation, flip, and multi-scale scaling;

Step S2013: uniformly scale the ROI images after data scale expansion to the same size to meet the input requirements of the CNN model.

Step S202: For the extracted 180-dimensional radiomics feature vector, use the LASSO logistic regression model to perform feature screening, select features that are highly correlated with the histopathological grade of breast cancer, and use the screened radiomics features to perform feature fusion .

LASSO regression is to add L1 regularization term on the basis of least squares fitting to improve the accuracy of linear regression model. Its penalty function is the absolute value of regression coefficient, which can make some parameter estimation results equal to zero, so it is helpful for characteristics choose. Histopathological grading is a binary classification problem. Logistic regression analysis is a generalized linear model commonly used for binary classification or one-to-many classification. It normalizes the response of simple linear regression to 0 and 1, so LASSO can be regressed Linear regression in the model is replaced by logistic regression to select features for binary classification. The objective function of LASSO logistic regression optimization is as follows:

Among them, n is the number of samples, Xi is an m× _n size of the original data, that is, each sample has m features, y _i is the corresponding response value of each sample, ω is the linear regression coefficient, b is the linear The cutoff value for the regression, λ is a non-negative regularization parameter used to control the sparsity of the regression coefficients. Inputting the extracted radiomics features into the LASSO logistic regression model can perform radiomics feature screening.

Step S203: Use the pre-trained CNN model for migration learning, train the CNN classification model, add a new fully connected layer on the basis of the original CNN classification model, and transfer the CNN classification model before the fully connected layer to the new fully connected layer. The output and screened radiomics features are fused, and retrained on the basis of the CNN classification model parameters, the fused CNN classification model parameters are updated, and the fused CNN classification is performed according to the classification effect of the model on the verification set The model parameters were adjusted to obtain a feature-fused CNN model, which was used for breast cancer histopathological grading on mammography images.

The step S203 includes:

Step S2031: using the mammography image samples of the breast tumor area in the training set as the input of the CNN model, performing transfer learning on the CNN model pre-trained on the ImageNet natural image data set, and training the CNN classification model;

Step S2032: Add a new fully-connected layer on the basis of the original CNN grading model, and output the high-level semantic features of the breast tumor region mammography image before the fully-connected layer of the CNN grading model on the new fully-connected layer with LASSO logistic The radiomics features screened by the regression model are fused, and retrained on the basis of the parameters of the CNN classification model, and the parameters of the fused CNN classification model are updated. According to the classification effect of the model on the verification set, the fused CNN classification model is The parameters are adjusted to obtain the CNN model of feature fusion.

Step S204: Use the test set to verify the classification accuracy of the obtained feature-fused CNN model.

As a possible implementation, the mammography image data set used has a total of 204 cases, and each case contains an axial (craniocaudal, CC) image and a lateral oblique (mediolateral oblique, MLO) image, as shown in Figure 3, Part (a) in Figure 3 is the axial mammography image, and part (b) in Figure 3 is the inclinometric mammography image. The standard DICOM format is used for mammography image storage, and its resolution (width×height) is 3328×4096 and 2560×3328. The tumor areas in all mammography images are delineated by the hospital's professional radiologists, and all cases are accompanied by accurate diagnosis results from the hospital's pathology department to determine their pathological grades. Different mammography imaging histopathological grading algorithms were tested on the collected mammography image data sets, and the classification accuracy and AUC value were used to quantitatively evaluate the classification performance. The results are shown in Table 1. Compared with GoogLeNet (see Szegedy C, Liu W, Jia Y, et al.Going deeper with convolutions[C]. IEEE Conference on Computer Vision and Pattern Recognition, 2015:1-9) and the classification effect of the traditional random forest classifier have been significantly improved, the classification accuracy rate reached 0.7500, and the AUC value reached 0.8051.

Table 1 Classification performance of breast cancer mammography imaging pathology grading algorithm

classification algorithm Classification accuracy AUC GoogLeNet 0.7031 0.7049 random forest 0.6029 0.6618 Feature Fusion Algorithm 0.7500 0.8051

What is shown above is only a preferred embodiment of the present invention. It should be pointed out that for those of ordinary skill in the art, some improvements and modifications can also be made without departing from the principles of the present invention. It should be regarded as the protection scope of the present invention.

Claims (4)

1. The histopathological grading method of breast cancer based on CNN and radiomics feature fusion, is characterized in that, comprises the following steps: Step 1: Extract the tumor area of the mammography image, calculate the grayscale, texture and wavelet features on the extracted mammography tumor area, and extract a total of 180-dimensional radiomics feature vectors through the above calculation; the extracted mammography The target image tumor area is made into breast tumor area mammography image samples of the same size, and the image samples are divided into training set, verification set and test set; Step 2: For the extracted 180-dimensional radiomics feature vector, use the LASSO logistic regression model to perform feature screening, and use the screened radiomics features to perform feature fusion; Step 3: Use the pre-trained CNN model for migration learning, train the CNN classification model, add a new fully connected layer on the basis of the original CNN classification model, and transfer the CNN classification model before the fully connected layer to the new fully connected layer. The output and screened radiomics features are fused, and retrained on the basis of the CNN classification model parameters, the fused CNN classification model parameters are updated, and the fused CNN classification is performed according to the classification effect of the model on the verification set The model parameters were adjusted to obtain a feature-fused CNN model, which was used for breast cancer histopathological grading on mammography images. 2. the breast cancer histopathological grading method based on CNN and radiomics feature fusion according to claim 1, is characterized in that, after described step 3, also comprises: Use the test set to verify the classification accuracy of the CNN model obtained by feature fusion. 3. the breast cancer histopathological grading method based on CNN and radiomics feature fusion according to claim 1, is characterized in that, described step 1 comprises: Step 1.1: Extract the ROI of the tumor area of the mammography image to obtain the ROI image, calculate 14 gray features, 22 texture features and 144 wavelet features of the ROI image, and extract a total of 180-dimensional radiomics feature vectors; Step 1.2: Expand the scale of the ROI image through data enhancement methods; Step 1.3: Scale the ROI images after data scale expansion to the same size to meet the input requirements of the CNN model. 4. the breast cancer histopathological grading method based on CNN and radiomics feature fusion according to claim 1, is characterized in that, described step 3 comprises: Step 3.1: Take the mammography image samples of the breast tumor area in the training set as the input of the CNN model, perform transfer learning on the CNN model pre-trained on the ImageNet natural image dataset, and train the CNN classification model; Step 3.2: Add a new fully-connected layer on the basis of the original CNN grading model. On the new fully-connected layer, output the high-level semantic features of the breast tumor area mammography image before the fully-connected layer of the CNN grading model and use LASSOlogistic regression The radiomics features screened by the model are fused, and retrained on the basis of the CNN classification model parameters, the fused CNN classification model parameters are updated, and the fused CNN classification model parameters are updated according to the classification effect of the model on the verification set. Make adjustments to obtain the CNN model of feature fusion.