CN111815569B

CN111815569B - Image segmentation method, device, equipment and storage medium based on deep learning

Info

Publication number: CN111815569B
Application number: CN202010544386.8A
Authority: CN
Inventors: 曹桂平
Original assignee: Guangzhou Shiyuan Electronics Thecnology Co Ltd
Current assignee: Guangzhou Shiyuan Electronics Thecnology Co Ltd
Priority date: 2020-06-15
Filing date: 2020-06-15
Publication date: 2024-03-29
Anticipated expiration: 2040-06-15
Also published as: CN111815569A

Abstract

The embodiment of the invention discloses an image segmentation method, device, terminal equipment and storage medium based on deep learning. Obtaining a training set based on a segmentation marking operation of sample image data in organ image data obtained by scanning an organ; generating an initial image segmentation model, inputting the training set into the initial image segmentation model for model training to obtain an image segmentation model for image structure segmentation, wherein the initial image segmentation model is obtained by adding a regular constraint branch parallel to a decoder branch of a U-Net frame at a central layer of the U-Net frame, and the regular constraint branch is used for carrying out regular constraint on an encoder branch of the U-Net frame; and inputting the organ image data to be segmented into the image segmentation model so as to carry out segmentation marking on the organ image data to be segmented. By placing additional constraints on encoder weights in the U-Net network, more robust features in the image data can be extracted.

Description

Image segmentation method, device, equipment and storage medium based on deep learning

Technical Field

The embodiment of the invention relates to the technical field of image processing, in particular to an image segmentation method, device, terminal equipment and storage medium based on deep learning.

Background

In recent years, brain image data can be conveniently obtained through a noninvasive imaging technology, and quantitative analysis can be carried out on the anatomical structure and internal lesions of the brain through imaging analysis, and the quantitative analysis can be used as an effective basis for disease diagnosis and treatment. Magnetic resonance imaging (Magnetic Resonance Imaging, MRI) techniques are an effective means of performing brain image analysis. The automatic segmentation of each anatomical structure (such as gray matter, white matter, cerebrospinal fluid and the like) of the brain nuclear magnetic resonance image is the basis for quantitative analysis of each tissue region of the brain, and is also a key basic step of focus analysis, three-dimensional visualization and surgical navigation. Therefore, the method for automatically dividing the brain structure is significant in research accuracy, rapidness and high efficiency.

Nuclear magnetic resonance images of the brain can show complex anatomical structures inside the brain. The structure segmentation difficulty of the brain MRI image is greatly increased due to the problems of certain boundary blurring, low contrast, low spatial resolution, partial volume effect, huge morphological differences of various tissue structures and the like of the image.

The traditional brain structure segmentation method mainly comprises a manual segmentation method and a semi-automatic segmentation method. The manual segmentation method is time-consuming, labor-consuming, tedious in process and expensive in cost, the segmentation result is seriously affected by personal experience, and the segmentation result is difficult to reproduce. The semi-automatic segmentation method mainly comprises the following steps: gray scale-based methods (threshold method, region growth, fuzzy clustering, etc.), probability map-based methods, active contour-based methods (geodesic method, level set method, etc.), which still rely on priori interactive operations of operators, the segmentation accuracy and efficiency are difficult to reach the practical application level.

Currently, in the field of medical image segmentation, image segmentation methods based on deep learning are rapidly developed. Compared with the traditional method, the image segmentation method based on the deep learning has great advantages in segmentation precision and full-automatic segmentation.

However, the inventor finds that when the brain nuclear magnetic resonance image is segmented by the image segmentation method based on the deep learning, the accuracy of an output result is reduced when limited perturbation of input information occurs due to the design reason of a neural network structure in the conventional image segmentation method based on the deep learning, and the robustness of the whole segmentation system is poor. In addition, this phenomenon is common, and there is a problem of insufficient robustness in data segmentation of other organs and modalities in addition to brain MRI image segmentation.

Disclosure of Invention

The invention provides an image segmentation method, device, terminal equipment and storage medium based on deep learning, which are used for solving the technical problem of insufficient robustness in the prior art when data segmentation with obvious region division in an image is carried out.

In a first aspect, an embodiment of the present invention provides an image segmentation method based on deep learning, including:

obtaining a training set based on a segmentation marking operation of sample image data in organ image data obtained by scanning an organ;

generating an initial image segmentation model, inputting the training set into the initial image segmentation model for model training to obtain an image segmentation model for image structure segmentation, wherein the initial image segmentation model is obtained by adding a regular constraint branch parallel to a decoder branch of a U-Net frame at a central layer of the U-Net frame, and the regular constraint branch is used for carrying out regular constraint on an encoder branch of the U-Net frame;

and inputting the organ image data to be segmented into the image segmentation model so as to carry out segmentation marking on the organ image data to be segmented.

In a second aspect, an embodiment of the present invention further provides an image segmentation apparatus based on deep learning, including:

a training set generation unit configured to obtain a training set based on a segmentation marker operation on sample image data in organ image data obtained by scanning an organ;

the model training unit is used for generating an initial image segmentation model, inputting the training set into the initial image segmentation model for model training so as to obtain an image segmentation model for image structure segmentation, wherein the initial image segmentation model is obtained by adding a regular constraint branch which is parallel to a decoder branch of a U-Net frame at a central layer of the U-Net frame, and the regular constraint branch is used for carrying out regular constraint on an encoder branch of the U-Net frame;

and the segmentation marking unit is used for inputting the organ image data to be segmented into the image segmentation model so as to carry out segmentation marking on the organ image data to be segmented.

In a third aspect, an embodiment of the present invention further provides a terminal device, including:

one or more processors;

a memory for storing one or more programs;

the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the deep learning based image segmentation method as described in the first aspect.

In a fourth aspect, an embodiment of the present invention further provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor implements the image segmentation method based on deep learning according to the first aspect.

The image segmentation method, the device, the terminal equipment and the storage medium based on deep learning are used for obtaining a training set based on the segmentation marking operation of sample image data in the organ image data, wherein the organ image data is obtained by scanning an organ; generating an initial image segmentation model, inputting the training set into the initial image segmentation model for model training to obtain an image segmentation model for image structure segmentation, wherein the initial image segmentation model is obtained by adding a regular constraint branch parallel to a decoder branch of a U-Net frame at a central layer of the U-Net frame, and the regular constraint branch is used for carrying out regular constraint on an encoder branch of the U-Net frame; and inputting the organ image data to be segmented into the image segmentation model so as to carry out segmentation marking on the organ image data to be segmented. The regular constraint branches parallel to the decoder branches are added in the central layer of the U-Net framework, the regular constraint branches and the encoder of the U-Net framework are combined into the variable self-encoder, the variable self-encoder and the U-Net framework share the weight of the encoder in the integral model, and the weight of the encoder in the U-Net network can be additionally constrained under the action of a loss function of the variable self-encoder, so that the integral model can extract more robust features in image data.

Drawings

Fig. 1 is a flowchart of an image segmentation method based on deep learning according to a first embodiment of the present invention;

FIG. 2 is a schematic illustration of MRI image data of a brain;

FIG. 3 is a schematic illustration of artificial segmentation markers for MRI image data of the brain;

fig. 4 is a schematic structural diagram of the image segmentation model according to the present embodiment;

FIG. 5 is a schematic diagram of a multi-scale structure centrally disposed in the present solution;

FIG. 6 is a result of segmentation of brain MRI image data in the MRBrains18 dataset based on the present approach;

FIG. 7 is a graph showing the segmentation of brain MRI image data in an IBSR based on the present approach;

fig. 8 is a schematic structural diagram of an image segmentation apparatus based on deep learning according to a second embodiment of the present invention;

fig. 9 is a schematic structural diagram of a terminal device according to a third embodiment of the present invention.

Detailed Description

The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are for purposes of illustration and not of limitation. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.

It should be noted that, for the sake of brevity, this specification is not exhaustive of all of the alternative embodiments, and after reading this specification, one skilled in the art will appreciate that any combination of features may constitute an alternative embodiment as long as the features do not contradict each other.

For example, in one embodiment of the first embodiment, one technical feature is described: the preprocessing of the organ image data mainly includes preprocessing of the gray scale density and/or gray scale range to obtain organ image data having the same gray scale density and/or gray scale range, and in another embodiment of the first embodiment, another technical feature is described: the encoder downsamples 3 times for the content features of the brain MRI image data. Since the above two features are not contradictory, after reading the specification of the present application, it should be appreciated by those skilled in the art that an embodiment having both features is also an alternative embodiment, i.e. performing the preprocessing of the organ image related to the gray density and the gray range, and then performing the downsampling 3 times during the training process.

It should be noted that the embodiment of the present invention is not a set of all the technical features described in the first embodiment, in which some technical features are described for the optimal implementation of the embodiment, and if the combination of several technical features described in the first embodiment can achieve the design of the present invention, the embodiment may be used as an independent embodiment, and of course may also be used as a specific product form.

The following describes each embodiment in detail.

Example 1

Fig. 1 is a flowchart of an image segmentation method based on deep learning according to an embodiment of the present invention. The image segmentation method based on deep learning provided in the embodiment can be performed by various operation devices for image segmentation, the operation devices can be implemented in a software and/or hardware mode, and the operation devices can be formed by two or more physical entities or one physical entity

Specifically, referring to fig. 1, the image segmentation method based on deep learning specifically includes:

step S101: the training set is obtained based on a segmentation marking operation on sample image data in the organ image data obtained by scanning the organ.

The organ image data applied in the present embodiment is mainly an image with similar distribution of regions corresponding to tissue structures in the same type of image, for example, fundus image, brain MRI image data, and in the present embodiment, brain MRI image data is taken as an example for explanation. The MRI image data of the brain is obtained by scanning the brain of a plurality of persons through a nuclear magnetic resonance scanner, and when the MRI image data of the brain is specifically generated, various data mode requirements can be required, such as T1 weighted imaging (as shown in fig. 2), T1lR weighted imaging, T2 weighted imaging, and the like. For imaging under different data mode requirements, additional registration operation is needed when image preprocessing is performed, and in the scheme, only T1 weighted imaging data is used for improving the segmentation efficiency and reducing the segmentation cost. Of course, in the case where the data division processing capability is sufficient, the integrated processing of the multiple data patterns may also be employed.

In the case where the registration operation is not required by using only the T1-weighted imaging data, preprocessing of the organ image data is mainly preprocessing of the gray density and/or gray range correlation to obtain organ image data having a uniform gray density and/or gray range. During the imaging process, the brain MRI image data is affected by factors such as a coil, and finally acquired brain MRI image data can show irregular and slow gray density changes, so that interference of digital image segmentation and quantitative analysis is caused. In the scheme, the brain MRI image data with consistent gray scale density is obtained by carrying out offset field correction on the brain MRI image data, and particularly, the offset field correction can be carried out by adopting an N4 Bias Field Correction method. Further, since the image gradation range of each image slice obtained by the magnetic resonance imaging is not uniform, a gradation normalization operation is required, and specifically, the image gradation distribution can be normalized to (0, 1) by:

I＝(I-I _min )/(I _max -I _min )

wherein I represents the gray matrix of the current sequence image, I _min ，I _max Representing the minimum and maximum values of the gray matrix of the current image, respectively.

On the basis of the preprocessed organ image data, a plurality of preprocessed organ image data are selected as sample image data to receive segmentation marking operation to obtain a training set. The specific marking operation is completed by manual operation, the marking content comprises segmented target areas, such as brain gray Matter (Cortical Gray Matter, GM), white Matter (White Matter) and cerebrospinal fluid (Cerebrospinal Fluid) in the case of brain MRI image data, and a gold standard image (group Truth) corresponding to the organ image data can be obtained through manual marking of the segmented images. The golden standard image is shown in fig. 3, wherein 10 is a cerebrospinal fluid region, 11 is a grey brain matter region, 12 is a white brain matter region, and a black region outside a brain structure region is a background region.

Step S102: generating an initial image segmentation model, inputting the training set into the initial image segmentation model for model training to obtain an image segmentation model for image structure segmentation, wherein the initial image segmentation model is obtained by adding a regular constraint branch parallel to a decoder branch of a U-Net frame at a central layer of the U-Net frame, and the regular constraint branch is used for carrying out regular constraint on an encoder branch of the U-Net frame.

As shown in fig. 4, the main body of the image segmentation model in this solution is a U-Net framework, i.e. a U-shaped network formed by two branches on the left in fig. 4, where the left branch in the U-shaped network is an encoder branch, the right branch is a decoder branch, and the lowest is the central layer of the U-shaped network. In the scheme, the architecture of the integral neural network is based on a U-shaped network, a regular constraint branch (the rightmost branch in fig. 4) is further added at the central layer of the U-shaped network, the regular constraint branch is parallel to the decoder branch, and the regular constraint branch is used for carrying out regular constraint on the encoder branch.

In the image segmentation model constructed by the scheme, the method can be regarded as the comprehensive realization of the U-shaped network and the variable self-encoder. The network parameters of the encoder branches are shared by the U-shaped network and the variable-component self-encoder, namely the encoder branches and the decoder branches form a complete U-shaped network, the encoder branches are constrained by the loss functions of the decoder branches, meanwhile, the encoder branches and the regular constraint branches form a complete variable-component self-encoder, and the encoder branches are constrained by the reconstruction errors of the variable-component self-encoder. Thus, the encoder branches are constrained in two ways, where the regular constrained branches enable capturing features of the image through reconstruction errors. Moreover, the regular constraint branch only plays a role in the training process of the image segmentation model, and does not participate in the related data processing process of the segmentation markers when the organ image data is segmented after model training is completed. The regular constraint branches are equivalent to the idea of adding image generation countermeasure in the image segmentation model, and the encoder branches are prompted to extract more effective information.

In the whole, the U-shaped network and the variable self-encoder share the weights of the encoder branches, and the weights of the encoder branches can be additionally restrained under the action of the loss function of the regular constraint branches, so that the encoder branches extract more robust features, and further the overall model has better robustness when the viscera image data are segmented after training is completed.

Based on the overall architecture of the image segmentation model, the training process of the image segmentation model in step S102 may be further implemented through steps S1021-S1026.

Step S1021: an initial image segmentation model is generated.

When the image segmentation model is initialized, the parameter details of the image segmentation model can be further initialized based on the overall model architecture, which is suitable for different types of organ image data.

For the loss function of the regular constraint branch, the posterior probability distribution q can be used _φ (z|x) and likelihood function p _θ And (z|x) calculating KL divergence, wherein x is an input image, z is a hidden variable in a self-encoder of a variation formed by the regular constraint branch and the encoder branch, and phi and theta respectively correspond to parameters to be learned by a network in the encoder branch and the decoder branch.

Specifically, the loss function L _VAE (θ, φ) is calculated by the following formula:

wherein,represents the reconstructed mean square error, D _KL (q _φ (z|x)||p _θ (z)) represents the a priori distribution p _θ (z) and posterior probability distribution q _φ The inverse KL divergence of (z|x), in the model, it is desirable that the two distributions are as close as possible, so this constraint is used. Since the variable self-encoder and the U-network share the weights of the encoder branches, the variable self-encoder can be used for the encoding of the data signal under the action of the loss function of the variable self-encoder (namely, the loss function of the regular constraint branch)The weights of the encoder branches are additionally constrained, so that the encoder branches extract more robust features, i.e., the robustness of the overall model is improved.

Furthermore, gaussian noise is added between the center layer and the regular constraint branches. The regular constraint branches are added with Gaussian noise in the hidden layer, so that the anti-interference performance of the model is improved, the generalization performance is improved, and the problem of sample imbalance is relieved to a certain extent. The hidden layer refers to the low-dimensional feature expression of the network, namely the central part of the network, as shown in fig. 4, gaussian noise is specifically added between the center of the network and the regular constraint branches, and from the mathematical expression, the gaussian noise passes through N (μ, σ) in fig. 4 ² ) (i.e., normal distribution) constraints.

In a specific implementation, the loss function of the decoder may be a binary cross entropy loss. Namely:

wherein y represents group Truth, p _i Representing the output value of the logical layer.

If necessary, a multi-scale structure can be arranged in the central layer. The multi-scale structure, i.e., the cavity convolution module (Atrous Conv Module), is shown in fig. 5. The cavity convolution can increase the receptive field of the extracted features, and the segmentation performance of the algorithm on objects under different scales is improved. In this scheme, through the setting of different hole convolution steps (r=1, r=2, r= 5,r =7), object features under different scales can be obtained.

The center layer may also include a Droupout operation at the same time to prevent overfitting and increase the generalization performance of the model.

The number of neurons of each convolutional layer of the encoder branch is the same for the content features of the brain MRI image data. Unlike most U-networks, this scheme increases the number of neurons in the initial layer, reducing the number of deep neurons. Meanwhile, the same neuron number is set at the encoder branch of the network in consideration of the fact that deep neurons are relatively small in memory occupation, calculation amount and the like. When analyzing brain MRI image data, it was found that high resolution images contained much more image detail information than low resolution images. Similarly, in the feature map, the high resolution feature map contains more rich and complex information, and more neurons are needed to learn the features; in low resolution feature maps, which contain less information, the effective features therein can be learned by fewer neurons.

In addition, the encoder downsamples 3 times for the content features of the brain MRI image data. As the network layer deepens, parameters, training difficulty, training cost and the like of the network are greatly increased. In brain structure segmentation tasks, the overall structural features are relatively consistent, although there are large differences in brain structure from person to person, and these salient features are already learned by the shallow network. Compared with a deep network, the shallow network gradient return is more effective, and is beneficial to improving the segmentation precision. Of course, for other types of organ image data, a more appropriate number of downsampling times may be set after specific analysis of the image features.

The special treatment of the scheme in the aspects of network layer depth and neuron number can reduce network parameters to 20% of the traditional U-shaped network through the two strategies, reduce calculated amount and algorithm running time, improve segmentation precision and speed, and have great significance to practical application.

Step S1022: and inputting the training set into the initial image segmentation model to perform model training to obtain a transition image segmentation model.

Generally, multiple training is required to complete the generation of the final image segmentation model based on the initial image segmentation model. In the scheme, an image segmentation model obtained after each training is defined as a transition segmentation model. That is, the training process for the initial image segmentation model and the transition image segmentation model in the scheme is the same, the difference is mainly that the training set is updated and changed, and the two definitions are just the differences in names made for the scheme expression clearly.

Step S1023: and inputting the organ image data without segmentation markers into the transition image segmentation model to carry out segmentation markers, and correcting and determining segmentation marker results.

In the practical model training process, sample data in a training set constructed based on sample image data may not be too much, and in order to reduce manual operation, in the scheme, organ image data without segmentation marks is input into a transition image segmentation model for segmentation marks, so that the transition image segmentation model is tested, and samples in the training set are increased. Specifically, there are two results of the correction determination of the division mark result, the first result is that the correction is not required, and the second result is that the correction is required. If the first result, it may be generally determined that the transition image segmentation model passes the test, step S1026 may be performed; if the second result is the second result, the segmentation marker result is corrected, and the corrected segmentation marker result may be added to the training set for further training, that is, step S1024 is performed. By means of the scheme for correcting the segmentation marking result, reliable segmentation marking of the organ image data can be obtained rapidly through manual confirmation and a small amount of correction, and a training set is increased rapidly.

Step S1024: and adding and updating the corrected segmentation mark result to the training set.

Step S1025: and training and updating the transition image segmentation model according to the training set added and updated.

The updating of the training set and the updating of the transition image segmentation model are similar to the training set construction and the training process of the initial image data segmentation model described above, and will not be repeated here.

Step S1026: and when the segmentation marking result is determined accurately, taking the current transition image segmentation model as an image segmentation model.

It should be noted that, steps S1021 to S1026 are described in the whole model training process, and the numbers of the steps do not represent absolute constraints on the execution sequence. Each step can be specifically adjusted according to the execution progress of model training; for example, in step S1023, if the segmentation marker result is determined to be accurate without correction, it may be considered that the test result of the current transition image segmentation model has reached the preset performance test standard, and the transition image segmentation model is taken as the final image segmentation model, that is, step S1026 is performed after step S1023. In particular, there may be multiple steps circulating between each step; for example, in steps S1023 to S1025, it may be the case that the organ image data of the undivided marker is input into the transition image segmentation model a plurality of times in succession, and the obtained segmentation marker results all need to be corrected, and in this case steps 1023 to S1025 need to be circularly performed to complete the segmentation markers, the segmentation marker result correction, the training set update, and the update of the transition image segmentation model for each of the undivided organ images. If the progress of the model training process is smooth, determining a judgment operation based on the correction in step S1023, wherein some steps may not even be performed; for example, in the transitional image segmentation model obtained after step S1022, if it is determined that the correction is not necessary by the correction determination of the segmentation marker result of the organ image data not segmented in step S1023, step S1026 may be directly executed without executing step S1024 and step S1025.

In addition, in the specific implementation process, in order to improve the segmentation accuracy and precision of the image segmentation model, the method can be further limited to determining that the segmentation marking result is accurate when the segmentation marking result of a plurality of continuous non-segmented image data is judged to be not corrected, and correspondingly confirming the image segmentation model.

Step S103: and inputting the organ image data to be segmented into the image segmentation model so as to carry out segmentation marking on the organ image data to be segmented.

In practical application, the image data of the viscera to be segmented can be input into the image segmentation model and automatically predicted by the image segmentation model, so that an accurate segmentation result can be quickly obtained. Fig. 6 and 7 are the results of segmenting brain MRI image data in the mrbrain s18 dataset and brain MRI image data in the IBSR (Internet Brain Segmentation Repository, international brain segmentation database), respectively, based on the present approach. Where Image represents organ Image data to be segmented, group trunk represents a result of manually performing segmentation marking on the organ Image data to be segmented, and prediction represents a segmentation marking result of the organ Image data to be segmented by the Image segmentation model.

Example two

Fig. 8 is a schematic structural diagram of an image segmentation apparatus based on deep learning according to a second embodiment of the present invention. Referring to fig. 8, the image segmentation apparatus based on deep learning includes: a training set generation unit 201, a model training unit 202, and a segmentation marking unit 203.

Wherein, the training set generating unit 201 is configured to obtain a training set based on a segmentation marking operation on sample image data in organ image data, where the organ image data is obtained by scanning an organ; a model training unit 202, configured to generate an initial image segmentation model, input the training set into the initial image segmentation model to perform model training, so as to obtain an image segmentation model for image structure segmentation, where the initial image segmentation model is obtained by adding a regular constraint branch parallel to a decoder branch of a U-Net frame at a center layer of the U-Net frame, and the regular constraint branch is used to perform a regular constraint on an encoder branch of the U-Net frame; a segmentation marking unit 203 for inputting the organ image data to be segmented into the image segmentation model to perform segmentation marking on the organ image data to be segmented.

On the basis of the above embodiment, the model training unit 202 includes:

the model initialization module is used for generating an initial image segmentation model;

the initial training module is used for inputting the training set into the initial image segmentation model to perform model training to obtain a transition image segmentation model;

the intermediate test module is used for inputting the organ image data without segmentation marks into the transition image segmentation model to carry out segmentation marks, and correcting and determining segmentation mark results;

the training set updating module is used for adding and updating the corrected segmentation mark result to the training set;

the model updating module is used for training and updating the transition image segmentation model according to the training set added with the update;

and the model determining module is used for determining accuracy when the segmentation marking result is determined, and taking the current transition image segmentation model as an image segmentation model.

On the basis of the embodiment, the loss function of the regular constraint branch passes through the posterior probability distribution q _φ (z|x) and likelihood function p _θ And (z|x) calculating KL divergence, wherein x is an input image, z is a hidden variable in a self-encoder of a variation formed by the regular constraint branch and the encoder branch, and phi and theta respectively correspond to parameters to be learned by a network in the encoder branch and the decoder branch.

On the basis of the above embodiment, the loss function L _VAE (θ, φ) is calculated by the following formula:

L _VAE (θ,φ)＝-Ε _z～qφ(z|x) logp _θ (x|z)+D _KL (q _φ (z|x)||p _θ (z))

wherein,represents the reconstructed mean square error, D _KL (q _φ (z|x)||p _θ (z)) represents the a priori distribution p _θ (z) and posterior probability distribution q _φ Inverse KL divergence of (z|x).

On the basis of the above embodiment, gaussian noise is added between the center layer and the regular constraint branches.

On the basis of the above embodiment, the loss function of the decoder is a binary cross entropy loss.

On the basis of the above embodiment, the organ image data is brain MRI image data;

correspondingly, the training set generating unit 201 includes:

the image preprocessing module is used for preprocessing the viscera image data to obtain viscera image data with consistent gray density and/or gray range;

the segmentation marking module is used for selecting a plurality of preprocessed organ image data as sample image data to receive segmentation marking operation to obtain a training set.

On the basis of the above embodiment, the gray density is preprocessed by offset field correction; the gray scale range is preprocessed by gray scale normalization.

On the basis of the above embodiments, the central layer comprises a multi-scale structure.

On the basis of the above embodiments, the central layer comprises a Droupout operation.

On the basis of the above embodiment, the number of neurons of each convolution layer of the encoder is the same.

On the basis of the above embodiment, the number of downsampling of the encoder is 3.

The image segmentation device based on deep learning provided by the embodiment of the invention is contained in living body detection equipment, can be used for executing any image segmentation method based on deep learning provided in the first embodiment, and has corresponding functions and beneficial effects.

Example III

Fig. 9 is a schematic structural diagram of a terminal device according to a third embodiment of the present invention, where the terminal device is a specific hardware presentation scheme of the image segmentation device based on deep learning. As shown in fig. 9, the terminal device includes a processor 310, a memory 320, an input means 330, an output means 340, and a communication means 350; the number of processors 310 in the terminal device may be one or more, one processor 310 being taken as an example in fig. 9; the processor 310, memory 320, input means 330, output means 340 and communication means 350 in the terminal device may be connected by a bus or other means, for example by a bus connection in fig. 9.

The memory 320 is a computer readable storage medium, and may be used to store a software program, a computer executable program, and a module, such as program instructions/modules corresponding to the image segmentation method based on deep learning in the embodiment of the present invention (for example, the training set generating unit 201, the model training unit 202, and the segmentation marking unit 203 in the image segmentation apparatus based on deep learning). The processor 310 executes various functional applications of the terminal device and data processing by running software programs, instructions and modules stored in the memory 320, i.e., implements the above-described deep learning-based image segmentation method.

Memory 320 may include primarily a program storage area and a data storage area, wherein the program storage area may store an operating system, at least one application program required for functionality; the storage data area may store data created according to the use of the terminal device, etc. In addition, memory 320 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some examples, memory 320 may further include memory located remotely from processor 310, which may be connected to the terminal device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The input means 330 may be used to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the terminal device. The output device 340 may include a display device such as a display screen.

The terminal equipment comprises the image segmentation device based on the deep learning, can be used for executing any image segmentation method based on the deep learning, and has corresponding functions and beneficial effects.

Example IV

The embodiments of the present invention also provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are configured to perform the relevant operations in the deep learning based image segmentation method provided in any of the embodiments of the present application, and have corresponding functions and beneficial effects.

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product.

Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein. The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory. The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, etc., such as Read Only Memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises an element.

Note that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, while the invention has been described in connection with the above embodiments, the invention is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the invention, which is set forth in the following claims.

Claims

1. The image segmentation method based on the deep learning is characterized by comprising the following steps of:

and inputting the organ image data to be segmented into the image segmentation model for image structure segmentation so as to carry out segmentation marking on the organ image data to be segmented.

2. The method of claim 1, wherein generating an initial image segmentation model, inputting the training set into the initial image segmentation model for model training to obtain an image segmentation model for image structure segmentation, comprises:

generating an initial image segmentation model;

inputting the training set into the initial image segmentation model to perform model training to obtain a transition image segmentation model;

inputting the organ image data without segmentation marks into the transition image segmentation model for segmentation marks, and correcting and determining segmentation mark results;

adding and updating the corrected segmentation mark result to the training set;

training and updating the transition image segmentation model according to the training set added with the update;

and when the segmentation marking result is determined accurately, taking the current transition image segmentation model as an image segmentation model.

3. The method of claim 1, wherein the loss function of the canonical constraint branch passes a posterior probability distribution q _φ (z|x) and likelihood function p _θ And (z|x) calculating KL divergence, wherein x is an input image, z is a hidden variable in a variable self-encoder formed by the regular constraint branch and the encoder branch, z obeys standard normal distribution, and phi and theta respectively correspond to parameters to be learned by a network in the encoder branch and the decoder branch.

4. A method according to claim 3, characterized in that the loss function L _VAE (θ, φ) is calculated by the following formula:

5. The method of claim 1, wherein the loss function of the decoder is a binary cross entropy loss.

6. The method of claim 1, wherein the organ image data is brain MRI image data;

correspondingly, the training set is obtained based on the segmentation marking operation of the sample image data in the viscera image data, and the training set comprises the following components:

preprocessing the organ image data to obtain organ image data with consistent gray density and/or gray range;

and selecting a plurality of preprocessed organ image data as sample image data to receive segmentation marking operation to obtain a training set.

7. The method of claim 6, wherein the gray density is pre-processed by offset field correction; the gray scale range is preprocessed by gray scale normalization.

8. Image segmentation device based on degree of depth study, characterized by comprising:

and the segmentation marking unit is used for inputting the organ image data to be segmented into the image segmentation model for image structure segmentation so as to carry out segmentation marking on the organ image data to be segmented.

9. A terminal device, comprising:

one or more processors;

a memory for storing one or more programs;

the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the deep learning based image segmentation method of any of claims 1-7.

10. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements the deep learning based image segmentation method according to any one of claims 1-7.