CN111739000B - A system and device for improving the accuracy of left ventricular segmentation in multiple cardiac views - Google Patents

Abstract

The invention provides a system and a device for improving the left ventricle segmentation accuracy of a plurality of heart views based on deep learning, which comprises: a data acquisition module configured to: acquiring picture data of the echocardiograms of a plurality of different views to form an original image data set; acquiring an echocardiogram to be processed as data to be segmented; a pre-processing module configured to: preprocessing an original image data set to form an experimental data set; a training module configured to: constructing a deep neural network training model, inputting an experimental data set into the training model for training, stopping training the training model and storing model parameters when the loss function value in the training model is not reduced any more; a data processing module configured to: inputting an echocardiogram to be processed into a training module for storing model parameters to obtain an endocardium and epicardium segmentation result; the training precision of the image processing of different heart chambers is improved, and further the heart view segmentation precision is improved.

Description

A system and device for improving the accuracy of left ventricular segmentation in multiple cardiac views

technical field

The invention belongs to the technical field of medical detection, and in particular relates to a system and a device for improving the accuracy of left ventricle segmentation in multiple cardiac views based on deep learning.

Background technique

The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

With the development of medical technology, a variety of medical imaging data have appeared. How to use these medical imaging data to diagnose diseases correctly, quickly and to the greatest extent has become a hot spot in today's society.

Machine learning techniques enable researchers to develop and utilize sophisticated models to classify or predict various abnormalities or diseases, or to identify and segment medical lesions. Today, the development of machine learning technology is gradually mature and perfect. Deep learning is a new field of machine learning research, and its motivation lies in the establishment and simulation of the human brain to analyze and study neural networks and simulate the mechanism of the human brain to interpret data. Therefore, in recent years, more and more researchers have begun to pay attention to processing technologies such as pattern recognition, classification and segmentation in medical image processing.

In clinical application, echocardiography is an important means for doctors to judge heart disease. In clinical treatment, the characteristics of left ventricular motion status in echocardiography are the primary basis for doctors to diagnose heart disease. By segmenting the left ventricle, important medical metrics such as ejection fraction can be calculated. In the clinical ultrasound diagnosis process, the central apical two-chamber, the apical three-chamber, and the apical four-chamber of echocardiography all contain complete left ventricular information, but because of the different detection positions of the ultrasound probe, the left ventricle in different chambers is different. The shape is not the same. At the same time, the echocardiogram also contains a lot of noise, and the existing segmentation algorithms cannot accurately segment the left ventricle. Therefore, it is very important to provide a measure to improve the accuracy of medical anatomical structure segmentation. This will greatly reduce the workload of doctors and improve the accuracy of diagnosis.

SUMMARY OF THE INVENTION

In order to solve the above problems, the present invention proposes a left ventricular myocardium segmentation system and device based on a deep learning segmentation method, which can automatically segment the left ventricular myocardium under different views.

In a first aspect, the present invention provides a deep learning-based system for improving the accuracy of left ventricle segmentation in multiple cardiac views, including:

The data acquisition module is configured to: collect image data of echocardiograms of several different views to form an original image data set; collect the echocardiograms to be processed as data to be segmented;

a preprocessing module, configured to: preprocess the original image data set to form an experimental data set;

The training module is configured to: construct a deep neural network training model, input the experimental data set into the training model for training, when the loss function value in the training model no longer decreases, the training model stops training and saves the model parameters.

The data processing module is configured to: input the data to be segmented into the training module storing the model parameters to obtain the segmentation result of the endocardium and endocardium.

In a second aspect, the present invention also provides a deep learning-based device for improving the accuracy of left ventricle segmentation in multiple cardiac views, including: a RetinaNet network and the system for improving the segmentation accuracy of cardiac views as described in the first aspect, The data to be segmented is input into the RetinaNet network to obtain different cardiac view recognition results and left ventricle detection results, and the detection results are input into the segmentation network of the system to improve the accuracy of cardiac view segmentation for segmentation.

Compared with the prior art, the present invention has the following beneficial effects:

1. The present invention proposes a left ventricular myocardium segmentation system under different heart views based on deep learning. The system adopts a preprocessing module and a training module to form an experimental data set and data to be processed, and can automatically segment the myocardium. The inner and outer membranes do not need the doctor to manually outline, reducing the doctor's workflow.

2. The training module of the present invention combines segmentation and detection to improve the training accuracy for different ventricular image processing, thereby improving the segmentation accuracy of cardiac views.

3. The present invention adopts a preprocessing module to reduce the influence of noise in echocardiography, and by establishing a training model and using a segmentation network, the left ventricle can be accurately segmented, and the segmentation accuracy is improved.

Description of drawings

The accompanying drawings that form a part of the present application are used to provide further understanding of the present application, and the schematic embodiments and descriptions of the present application are used to explain the present application and do not constitute improper limitations on the present application.

1 is a schematic diagram of forming an experimental data set in Embodiment 1 of the present invention;

FIG. 2 is a schematic diagram of a system for improving the accuracy of left ventricle segmentation in multiple cardiac views based on deep learning in Embodiment 1 of the present invention.

3 is a schematic diagram of an apparatus for improving the accuracy of left ventricle segmentation in multiple cardiac views based on deep learning of the present invention.

Detailed ways:

The present invention will be further described below with reference to the accompanying drawings and embodiments.

It should be noted that the following detailed description is exemplary and intended to provide further explanation of the application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

It should be noted that the terminology used herein is for the purpose of describing specific embodiments only, and is not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly dictates otherwise, the singular is intended to include the plural as well, furthermore, it is to be understood that when the terms "comprising" and/or "including" are used in this specification, it indicates that There are features, steps, operations, devices, components and/or combinations thereof.

Example 1

In order to solve the above problems, as shown in Figures 1-3, the present invention proposes a left ventricular myocardium segmentation system and device based on a deep learning segmentation method, which can automatically segment the left ventricular myocardium in different views.

A deep learning-based system for improving the accuracy of left ventricle segmentation in multiple cardiac views, including:

The data acquisition module is configured to: collect image data of echocardiograms of several different views to form an original image data set; collect the echocardiograms to be processed as data to be segmented;

a preprocessing module, configured to: preprocess the original image data set to form an experimental data set;

The training module is configured to: construct a deep neural network training model, input the experimental data set into the training model for training, when the loss function value in the training model no longer decreases, the training model stops training and saves the model parameters.

The data processing module is configured to: input the data to be segmented into the training module storing the model parameters to obtain the segmentation result of the endocardium and endocardium.

Further, the deep neural network training model is a deep convolutional neural network, including a plurality of convolution kernels, the parameters of which include parameter values of the plurality of convolution kernels, and the loss function is a pixel-by-pixel cross-entropy loss function.

Further, the training model includes a segmentation network, and the experimental data set is input into the segmentation network for training; the segmentation network includes a potential expression extraction module, a fully convolutional connection module, and a segmentation sub-network module that are sequentially communicated and connected.

Further, the fully convolutional connection module is a neural network composed of multiple convolution kernels, and its parameters include parameter values of the multiple convolution kernels.

Further, the latent expression extraction module has two input ports, which are a port connecting the experimental data set and a port connecting the real segmentation annotation of the image to be trained. The sparse features extracted by the latent expression extraction module can represent the input information. The images to be trained are echocardiograms marked for segmentation, that is, the data to be segmented.

Further, the segmentation sub-network module adopts FCN, UNet or Segnet, the Segnet segmentation network includes an encoder and a decoder, the encoder network uses the first 13 layers of the VGG network, and each encoder layer corresponds to a decoder layer. The encoder part consists of several convolutional layers, BN layers, RELU and pooling layers. The pooling layer here uses 2x2 max-pooling, which will result in a larger loss of boundary details.

Further, the preprocessing of the original image data set includes:

(1) Remove the patient, hospital and scan information in the original image data set, and only retain the fan-shaped area of the ultrasound image;

(2) Adjust the image size to maintain the original ultrasound image aspect ratio.

Further, the image data is the image data of the second apical chamber, the third apical chamber, and the fourth apical chamber.

Further, the forming of the original image data set includes: annotating the picture data, and combining the picture data of different echocardiograms.

Further, the labeling is: labeling the ventricular epicardium in different views in the exported picture, and recording it in the form of "picture name, view type, and ventricular contour coordinates".

Further, the ventricular epicardium is marked as: the outline of the ventricular epicardium is depicted in the ultrasound images of different views.

Example 2

A deep learning-based cardiac multi-view left ventricular myocardium segmentation system, including:

Acquisition module: Take the patient as a unit, acquire the medical images of the apical two chambers and the four apical chambers of the echocardiography, mark the contours of the left ventricular myocardium in different views, and make them into the original image data set;

Preprocessing module: preprocess the data set to obtain the experimental data set;

Training module: Input the experimental data set into the deep learning RetinaNet network to obtain the heart view recognition results and left ventricle detection results, and input the detection results to the segmentation network to obtain the segmentation results.

Further, the method for making the original image data set is specifically:

Taking the patient as a unit, first derive the pictures of the apical two-chamber, the apical three-chamber, and the apical four-chamber from the echocardiogram, and delineate the outline of the left ventricle and epicardium in the ultrasound images of different views;

The image information of multiple patients is combined to form an img file, and the label information is formed into a label file in the form of "image name view category left ventricle contour coordinates". The img file and the label file are the original image dataset.

The segmentation network method is specifically:

The segmentation network mainly includes three sub-modules: latent expression extraction, fully convolutional connection and segmentation sub-network, and the three modules are connected in a cascade structure;

The latent expression extraction module has two inputs in the training phase, connecting the original input data and labeling the segmented images respectively;

The fully convolutional connection module concatenates the input image and the latent expression vector;

The segmentation sub-network can utilize existing image segmentation networks.

Specifically, on a patient-by-patient basis, medical images of the apical two-chamber and apical four-chamber echocardiography were collected, and the contours of the inner and outer membranes of the left ventricle in different views were marked to make the original image dataset.

The original image dataset is preprocessed to obtain the experimental dataset.

The experimental data set is input into the segmentation network for training. When the loss function value of the training model is no longer reduced, the model training stops and the model parameters are saved.

It also includes a data processing module: input the ultrasound images to be processed into the training model, and load the saved model parameters to obtain the segmentation result of the left ventricle of the heart.

First, the corresponding equipment was used to collect echocardiographic images, and with the support of hospital data, echocardiographic images of each experimental object were collected. After the images are collected, the collected images are processed into the experimental data set after going through the preprocessing stage.

The process of constructing the experimental data set is shown in Figure 1, including three parts: data collection, data labeling and data preprocessing;

Data acquisition includes acquiring echocardiographic images of patients, and exporting the data sets of apical two-chamber, apical three-chamber, and apical four-chamber into picture format on a patient-by-patient basis.

Data Annotation Manually annotate the left ventricular epicardium in different views in the exported pictures, and record them in the form of "picture name view type left ventricle contour coordinates".

Data preprocessing mainly includes the following steps:

(1) The irrelevant information is removed, and only the fan-shaped area of the ultrasound image is retained;

(2) Image size adjustment, in order to maintain the original ultrasound image aspect ratio, the image size needs to be adjusted;

The experimental data set is input into the RetinaNet network to obtain different cardiac view recognition results and left ventricle detection results, and the detection results are input into the segmentation network for segmentation to improve the segmentation accuracy.

The invention proposes a left ventricular myocardium segmentation system under different heart views based on deep learning. The system can automatically segment the inner and outer layers of the myocardium, and does not require a doctor to manually outline, thereby reducing the doctor's workflow.

In other embodiments, the present invention also provides:

A device for improving the accuracy of left ventricle segmentation of multiple heart views based on deep learning, comprising: a RetinaNet network and the system for improving the accuracy of heart view segmentation as described in the above-mentioned embodiments, inputting the data to be segmented into the RetinaNet network to obtain Different heart view recognition results and left ventricle detection results, and the detection results are input into the segmentation network of the system to improve the accuracy of cardiac view segmentation to achieve high-precision left ventricle endocardium segmentation, as shown in Figure 3.

Although the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, they do not limit the scope of protection of the present invention. Those skilled in the art should understand that on the basis of the technical solutions of the present invention, those skilled in the art do not need to pay creative work. Various modifications or deformations that can be made are still within the protection scope of the present invention.

Claims (6)

1. a system for improving the accuracy of left ventricle segmentation of multiple heart views based on deep learning, is characterized in that, comprising: The data acquisition module is configured to: collect image data of echocardiograms of several different views to form an original image data set; collect the echocardiograms to be processed as data to be segmented; a preprocessing module, configured to: preprocess the original image data set to form an experimental data set; The training module is configured to: build a training model, input the experimental data set into the training model for training, when the loss function value in the training model no longer decreases, the training model stops training and saves the model parameters; The data processing module is configured to: input the data to be segmented into the training module that saves the model parameters, and obtain the segmentation result of the endocardium and endocardium; The training model includes a segmentation network, and the experimental data set is input into the segmentation network for training; the segmentation network includes a potential expression extraction module, a fully convolutional connection module, and a segmentation sub-network module that are sequentially connected in communication; the potential expression extraction module It has two input ports, which are the port connecting the data to be divided and the port connecting the experimental data set; The image data is the image data of the second apical chamber, the third apical chamber, and the fourth apical chamber; the forming the original image data set includes: annotating the picture data, combining the picture data of different echocardiograms; Data annotation included delineating the ventricle and epicardium in the ultrasound images in different views. 2. The system for improving the accuracy of left ventricle segmentation in multiple cardiac views based on deep learning as claimed in claim 1, wherein the fully convolutional connection module is a neural network composed of multiple convolution kernels. The parameters include parameter values for multiple convolution kernels. 3. The system for improving the accuracy of left ventricle segmentation in multiple cardiac views based on deep learning according to claim 1, wherein the preprocessing of the original image data set comprises: (1) Remove the patient, hospital and scan information in the original image data set, and only retain the fan-shaped area of the ultrasound image; (2) Adjust the image size to maintain the original ultrasound image aspect ratio. 4. The system for improving the accuracy of left ventricle segmentation in multiple cardiac views based on deep learning as claimed in claim 1, wherein the labeling of the picture data comprises: performing an epicardial analysis on the ventricular epicardium in different views in the derived picture. Annotate and record it in the form of "picture name, view type, ventricle contour coordinates". 5. The system for improving the accuracy of left ventricle segmentation in multiple cardiac views based on deep learning according to claim 1, wherein the image data is an apical two-chamber, an apical three-chamber, and an apical four-chamber. image data. 6. A device for improving the accuracy of left ventricle segmentation of multiple heart views based on deep learning, characterized in that, comprising: RetinaNet network and the deep learning-based improving multiple hearts according to any one of claims 1-5 The system for the segmentation accuracy of the left ventricle of the view inputs the data to be segmented into the RetinaNet network to obtain the recognition results of different cardiac views and the detection results of the left ventricle, and inputs the detection results to the segmentation network of the system to improve the segmentation accuracy of the cardiac view for segmentation.

Images