CN110599528A - Unsupervised three-dimensional medical image registration method and system based on neural network - Google Patents

Abstract

The invention provides an unsupervised three-dimensional medical image registration method and system based on a neural network, comprising the following steps of: collecting an image; preprocessing the acquired three-dimensional medical image; training a neural network to obtain a trained neural network model; and inputting the medical image to be registered into the trained neural network model for registration to obtain and output a registered image. Wherein the invention is based on a deformation fieldDeformation fieldDeformation fieldAnd based on the deformed imageDeformed imageDeformed imageDeformed imageComputing a fixed image I using a loss function_FAnd the deformed imageAnd performing back propagation optimization on the neural network until the calculated loss function value is not reduced or the network training reaches a preset training iteration number, and finishing the neural network training to obtain a trained neural network model. The invention is used for realizing medical image registration.

Description

A method and system for unsupervised 3D medical image registration based on neural network

technical field

The invention relates to the field of medical image registration, in particular to an unsupervised three-dimensional medical image registration method and system based on a neural network, which is mainly used for registration of three-dimensional human brain images.

Background technique

Medical image registration refers to seeking one or a series of spatial transformations for a medical image to make it spatially consistent with the corresponding points on another medical image. This consistency means that the same anatomical point on the human body has the same spatial position on the two matching images. Usually, the image to be registered is called a moving image (Moving Image), and the transformed target image is called a fixed image or a reference image (Fixed Image).

There are many open source software available for segmentation and registration of medical images, for example: FreeSurfer can be used to analyze and visualize structural and functional neuroimaging data from slices or time series, which can be used in skull stripping, B1 bias field correction, Gray and white matter segmentation and morphological difference measurement have good performance; FSL is similar to Freesufer, which can comprehensively analyze FMRI, MRI and DTI brain imaging data; ITK software package can be used for multi-dimensional image segmentation and registration; NiftyReg realizes The rigid body, affine and nonlinear registration methods for nifti images support GPU operation at the same time; elasti× is an open source software based on ITK, which includes common algorithms for processing medical image registration; ANTS is currently a better registration tool, Deformable registration of diffeomorphisms can be achieved. The above registration tools are all based on the traditional registration method to fit the deformation of the image. In addition, many current registration methods based on deep learning have also been proposed, such as: DIRNet, BIRNet, voxelmorph, etc., use neural networks to obtain image transformation parameters, and then use transformation networks to transform floating images to obtain registration results.

Although the above registration tools and methods have achieved good results in image registration, there are still the following problems:

(1) Some methods require manual labeling and supervision information, and the registration requirements are high and the registration speed is relatively low. The marking of feature points and feature regions of medical images and the acquisition of image supervision information need professional medical imaging doctors to complete, which is extremely difficult for registration workers without relevant medical experience. At the same time, different doctors and colleagues There may be differences in the feature marks obtained by a doctor at different times, and the process of artificial marking is time-consuming and laborious, and the subjective judgment of the doctor has a great impact on the registration results.

(2) The registration accuracy is relatively low. Although some methods proposed in recent years have made great progress in registration results, the accuracy still needs to be improved.

Therefore, the present invention provides a neural network-based unsupervised three-dimensional medical image registration method and system for solving the above problems.

Contents of the invention

In view of the above-mentioned deficiencies in the prior art, the present invention provides an unsupervised three-dimensional medical image registration method and system based on a neural network, which is used to realize rapid registration of medical images. Also used to improve registration accuracy.

In a first aspect, the present invention provides a neural network-based unsupervised three-dimensional medical image registration method, comprising steps:

L1, image acquisition: acquisition of 3D medical images from public datasets OASIS and ADNI, and/or: acquisition of 3D medical images from CT, MRI, or DICOM interfaces of ultrasound imagers;

L2, preprocessing the acquired 3D medical images: including image segmentation, cropping, normalization and affine alignment, and selecting any image from the affine aligned images as the fixed image I _F , and the rest of the images as Floating image I _M ; the cropped images have the same size;

L3, training the neural network based on the fixed image I _F and the floating image I _M obtained after preprocessing, to obtain a trained neural network model;

L4, inputting the medical image to be registered into the above-mentioned trained neural network model for registration, obtaining and outputting a registration image of the medical image to be registered;

Wherein in the step L3, based on the fixed image I _F obtained after preprocessing and the floating image I _M training neural network, obtain the trained neural network model, including:

S1. Input the fixed image I _F and the floating image I _M obtained after preprocessing into the neural network as the input layer of the neural network, and each set of input data includes the fixed image I _F and a floating image I _M ;

S2. Down-sampling the fixed image I _F and the floating image I _M input in the input layer, and output the feature map;

The downsampling includes 3 downsampling processes, a convolution calculation process with a convolution kernel size of 3×3×3 and a LeakyReLU activation function calculation process after the 3 downsampling processes; the 3 Each down-sampling process corresponds to three down-sampling process layers; the three down-sampling process layers are recorded as the first down-sampling process layer, the second down-sampling process layer and the third down-sampling process layer according to the execution order of the down-sampling process Process layer; each downsampling process layer includes a convolution layer with a convolution kernel size of 3×3×3, a LeakyReLU activation function layer and a maximum pooling layer;

S3. Perform feature reweighting on the feature maps output by the first downsampling process layer, the second downsampling process layer, and the third downsampling process layer corresponding to the downsampling process layer, respectively, to obtain three weighted features The graphs are: the first weighted feature map, the second weighted feature map, and the third weighted feature map;

S4. Perform 1×1×1 convolution on the feature map output in step S2, and output the deformation field from the floating image I _M to the fixed image I _F

S5. Input the feature map output in step S2 into the upsampling layer for upsampling. The upsampling layer includes 3 upsampling process layers, and each upsampling process layer includes an UpSampling layer and a convolution kernel with a size of 3 × 3 × 3 convolutional layers, the convolution kernel size in each upsampling process layer is 3 × 3 × 3 convolutional layers, respectively, with a LeakyReLU activation function layer; the 3 upsampling process layers correspond to the above Three upsampling processes of upsampling; the three upsampling process layers are recorded as the first upsampling process layer, the second upsampling process layer and the third upsampling process layer in sequence according to the order in which the three upsampling processes occur Sampling process layer;

Wherein, after the feature map output by the UpSampling layer of the first upsampling process layer is fused with the third weighted feature map, it is used as the convolutional layer with a convolution kernel size of 3×3×3 in the first upsampling process layer Input; after the feature map output by the UpSampling layer of the second upsampling process layer is fused with the second weighted feature map, it is used as the convolution layer whose convolution kernel size is 3×3×3 in the second upsampling process layer Input; after the feature map output by the UpSampling layer of the third upsampling process layer is fused with the first weighted feature map, it is used as the convolution layer whose convolution kernel size is 3×3×3 in the third upsampling process layer enter;

S6. Perform 1×1×1 convolution on the feature maps output by the first upsampling process layer, the second upsampling process layer, and the third upsampling process layer, and output the first upsampling process layer and the second upsampling process layer The deformation fields from the floating image I _M to the fixed image I _F corresponding to the process layer and the third upsampling process layer are the deformation fields in turn deformation field and deformation field

S7, the floating image I _M and the deformation field of the above-mentioned output Input to the spatial transformation network, transform the floating image I _M and the above output deformation field Input to the space transformation network, the floating image I _M and the above output deformation field Input to the space transformation network, the floating image I _M and the above output deformation field Input to the space transformation network, respectively through the space transformation of the space transformation network, correspondingly obtain the deformed image corresponding to the floating image I _M , which is the transformed image successively deformed image deformed image and the transformed image

S8. Deformation field based on the above output deformation field deformation field And the deformed image based on the above deformed image deformed image deformed image Using the loss function to calculate the fixed image I _F and the deformed image Between the loss function value, and optimize the neural network by backpropagation, until the calculated loss function value no longer becomes smaller or the network training reaches the preset number of training iterations, the neural network training is completed, and the training good neural network model;

The calculation expression of the loss function is:

In this formula ①, Indicates the calculated loss function value, α and β are constants and α+β=1, is a regular term, λ is a regularization control constant parameter, Represents a pre-specified three-dimensional medical image obtained by downsampling the fixed image _IF ; a three-dimensional medical image The size, in turn, corresponds to the warped image Equal, 3D medical images The resolutions of are successively reduced and are smaller than the resolution of the fixed image I _F , Denotes the fixed image I _F and the deformed image The similarity measure between Represents the above three-dimensional medical image with the warped image The similarity measure between Represents the above three-dimensional medical image with the warped image The similarity measure between Represents the above three-dimensional medical image with the warped image The similarity measure between; The same similarity measure function is used.

Further, the implementation steps of feature reweighting described in step S3 include:

Step S31, write down that each feature map to be reweighted in the sampling output is a feature map X, and this X∈R ^(H×W×D) , perform slice processing on the feature map X in its D dimension, Use the global average pooling strategy to perform global average pooling on each slice x∈R ^(H×W) obtained by slice processing, and obtain the slice descriptor z of each slice x on the D dimension of the feature map X, each The specific formula of the slice descriptor z is as follows:

In the formula (i, j) represents the pixel point on the slice x, and x(i, j) represents the gray value of the slice x at the pixel point (i, j);

Step S32, obtaining the weight s of each slice x on the D dimension of the feature map X, wherein the calculation formula of the weight s of each slice x is as follows:

s=σ(δ(z)),

Wherein, σ represents the sigmoid activation function, δ is the ReLU activation function, and z is the slice descriptor of the slice x obtained in step S31;

Step S33, correspondingly load each weight s obtained in step S32 to its corresponding slice, and obtain each slice on the D dimension of the feature map X × the corresponding reweighted slice The feature reweighting calculation formula corresponding to each slice x is as follows:

In this formula, F _scale (x, s) represents the multiplication operation between the slice x and its corresponding weight s;

Step S34, the reweighted slice corresponding to each slice x on the D dimension of the feature map X obtained in step S33 Corresponding to obtain the reweighted feature map corresponding to the feature map X

Further, the similarity measurement function adopts a cross-correlation function.

Further, the space transformation network adopts the STN space transformation network.

Further, the preprocessing also includes data enhancement;

The data enhancement includes: performing warping transformation on each obtained floating image respectively to obtain the warped transformed image corresponding to each obtained floating image; the obtained warped transformed images are newly added floating image.

In a second aspect, the present invention provides a neural network-based unsupervised three-dimensional medical image registration system, including:

The image acquisition unit acquires three-dimensional medical images from public datasets OASIS and ADNI, and/or: acquires three-dimensional medical images from CT, MRI or DICOM interfaces of ultrasound imagers;

The image preprocessing unit performs preprocessing on the obtained 3D medical image, including image segmentation, cropping, normalization processing and affine alignment, and selects any image from the affine aligned image as a fixed image I _F , The rest of the images are used as floating images I _M ; wherein the cropped images have the same size;

The neural network training unit is based on the fixed image I _F obtained after preprocessing and the floating image I _M to train the neural network to obtain a trained neural network model;

An image registration unit, inputting the medical image to be registered into the above-mentioned trained neural network model for registration, obtaining and outputting a registration image of the medical image to be registered;

Wherein, the neural network training unit includes:

Input module, the fixed image I _F and floating image I _M obtained after preprocessing are input into neural network as the input layer of neural network, and each group of input data all comprises described fixed image I _F and a described floating image I _M ;

The downsampling module is used to downsample the fixed image _IF and the floating image _IM input in the input layer, and output the feature map; the downsampling includes 3 downsampling processes and one after the 3 downsampling processes The convolution calculation process with a convolution kernel size of 3×3×3 and a LeakyReLU activation function calculation process; the three downsampling processes correspond to the three downsampling process layers; the three downsampling process layers described above follow the downsampling process The execution sequence of is recorded as the first downsampling process layer, the second downsampling process layer and the third downsampling process layer; each downsampling process layer includes a convolution layer with a convolution kernel size of 3×3×3 , a LeakyReLU activation function layer and a maximum pooling layer;

The reweighting module performs feature reweighting on the feature maps output by the LeakyReLU activation function layer in the first downsampling process layer, the second downsampling process layer, and the third downsampling process layer corresponding to downsampling, and obtains three weighted The feature maps of , in order: the first weighted feature map, the second weighted feature map, and the third weighted feature map;

The first deformation field output module performs 1×1×1 convolution on the feature map output by the downsampling module, and outputs the deformation field from the floating image I _M to the fixed image I _F

The upsampling module inputs the feature map output by the downsampling module into the upsampling layer for upsampling. The upsampling layer includes 3 upsampling process layers, and each upsampling process layer includes an UpSampling (i.e. upsampling) layer And a convolution layer with a convolution kernel size of 3×3×3, and a LeakyReLU activation function layer after each upsampling process layer with a convolution kernel size of 3×3×3; 3 The upsampling process layer corresponds to the three upsampling processes of the upsampling; the three upsampling process layers are recorded as the first upsampling process layer, the second upsampling process layer, and the second upsampling process layer according to the order in which the three upsampling processes occur. The sampling process layer and the third upsampling process layer; wherein, after the feature map output by the UpSampling layer of the first upsampling process layer is fused with the third weighted feature map, it is used as the convolution kernel size in the first upsampling process layer It is the input of the convolutional layer of 3×3×3; after the feature map output by the UpSampling layer of the second upsampling process layer is fused with the second weighted feature map, it is used as the convolution kernel size in the second upsampling process layer It is the input of the convolutional layer of 3 × 3 × 3; after the feature map output by the UpSampling layer of the third upsampling process layer is fused with the first weighted feature map, it is used as the convolution kernel size in the third upsampling process layer It is the input of the convolutional layer of 3×3×3;

The second deformation field output module performs 1×1×1 convolution on the feature maps output by the first upsampling process layer, the second upsampling process layer and the third upsampling process layer, and outputs the first upsampling process layer , the deformation field from the floating image I _M to the fixed image I _F corresponding to the second upsampling process layer and the third upsampling process layer, which are the deformation fields in turn deformation field and deformation field

A spatial transformation module that transforms the floating image I _M and the above-mentioned output deformation field Input to the spatial transformation network, transform the floating image I _M and the above output deformation field Input to the space transformation network, the floating image I _M and the above output deformation field Input to the space transformation network, the floating image I _M and the above output deformation field Input to the space transformation network, respectively through the space transformation of the space transformation network, correspondingly obtain the deformed image corresponding to the floating image I _M , which is the transformed image successively deformed image deformed image and the transformed image

Neural network optimization module, deformation field based on the above output deformation field deformation field And the deformed image based on the above deformed image deformed image deformed image Using the loss function to calculate the fixed image I _F and the deformed image Between the loss function value, and optimize the neural network by backpropagation, until the calculated loss function value no longer becomes smaller or the network training reaches the preset number of training iterations, the neural network training is completed, and the training good neural network model;

The calculation expression of the loss function is:

In this formula ①, Indicates the calculated loss function value, α and β are constants and α+β=1, is a regular term, λ is a regularization control constant parameter, Represents a pre-specified three-dimensional medical image obtained by downsampling the fixed image _IF ; a three-dimensional medical image The size, in turn, corresponds to the warped image Equal, 3D medical images The resolutions of are successively reduced and are smaller than the resolution of the fixed image I _F , Denotes the fixed image I _F and the deformed image The similarity measure between Represents the above three-dimensional medical image with the warped image The similarity measure between Represents the above three-dimensional medical image with the warped image The similarity measure between Represents the above three-dimensional medical image with the warped image The similarity measure between; The same similarity measure function is used.

Further, the reweighting module includes:

The descriptor acquisition module records that each feature map to be reweighted in the sampling output is X, X∈R ^(H×W×D) , slices the feature map X on its D dimension, and uses The global average pooling strategy performs global average pooling on each slice x∈R ^(H×W) obtained from the slice processing, and obtains the slice descriptor z of each slice x on the D dimension of the feature map X, and each slice The specific formula of the descriptor z is as follows:

In the formula (i, j) represents the pixel point on the slice x, and x(i, j) represents the gray value of the slice x at the pixel point (i, j);

The slice weight calculation module obtains the weight s of each slice x on the D dimension of the feature map X, wherein the calculation formula of the weight s of each slice x is as follows:

s=σ(δ(z)),

Among them, σ represents the sigmoid activation function, δ is the ReLU activation function, and z is the slice descriptor of the slice x obtained by the descriptor acquisition module;

The weighting module loads each weight s obtained by the slice weight calculation module to its corresponding slice, and obtains each slice on the D dimension of the feature map X × the corresponding reweighted slice The feature reweighting calculation formula corresponding to each slice x is as follows:

In this formula, F _scale (x, s) represents the multiplication operation between the slice x and its corresponding weight s;

Weighted image acquisition module, based on the feature map X obtained by the weighting module, each slice x on the D dimension corresponds to the reweighted slice Corresponding to obtain the reweighted feature map corresponding to the feature map X

Further, the similarity measurement function adopts a cross-correlation function.

Further, the space transformation network adopts the STN space transformation network.

Further, the preprocessing in the image preprocessing unit also includes data enhancement; wherein the data enhancement includes: performing bending transformation on each obtained floating image to obtain the corresponding The warped transformed images of ; the obtained warped transformed images are newly added floating images.

The beneficial effect of the present invention is that,

(1) The unsupervised three-dimensional medical image registration method and system based on the neural network provided by the present invention all use an unsupervised registration method, and do not need any marker information and registration supervision information in the registration process, reducing the number of markers The demand for data and the error of human subjective judgment help medical workers who have no relevant medical experience to perform image registration to a certain extent, and to a certain extent improve the speed of registration and save registration time. To a certain extent, it saves manpower and material resources.

(2) The neural network-based unsupervised three-dimensional medical image registration method and system provided by the present invention, through feature reweighted fusion and loss function with multi-level loss supervision function, the feature map obtained by the down-sampling path is based on its contribution The degree weighting is fused into the upsampling path, while supervising the model loss from different resolution angles, achieving more effective feature reuse and model supervision, and improving the registration accuracy to a certain extent.

In addition, the design principle of the present invention is reliable, the structure is simple, and has very wide application prospects.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, In other words, other drawings can also be obtained from these drawings on the premise of not paying creative work.

Fig. 1 is a schematic flowchart of a method according to an embodiment of the present invention.

Fig. 2 obtains described deformation field in the method shown in Fig. 1 deformation field deformation field deformation field Schematic diagram of the process instructions.

Fig. 3 is a schematic block diagram of a system according to one embodiment of the present invention.

Detailed ways

In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described The embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

Key terms appearing in the present invention are explained below.

1 and 2 are schematic flowcharts of a method according to an embodiment of the present invention. This embodiment takes the registration of 3D human brain images as an example.

As shown in Figure 1, the method 100 includes:

The first step is image acquisition.

Download public datasets of human brain images from public datasets OASIS and ADNI.

During specific implementation, those skilled in the art can also obtain the required three-dimensional medical image from the DICOM interface of the CT, MRI or ultrasonic imager.

The second step, preprocessing:

The 3D medical image obtained in the first step is preprocessed.

The human brain data collected in the first step includes redundant parts such as the neck, mouth, nasal cavity, and skull, and their sizes and grayscales are different. For this purpose, standard preprocessing is performed on these image data. First, the image is segmented to separate the human brain from the original data, and the resulting human brain image is cropped to make it consistent in size. Then normalization is performed, and the voxel values are normalized to [0, 1]. Affine alignment is performed afterwards. Then select one image from the affine-aligned data as a fixed image I _F , and the rest as a floating image I _M .

In addition, in order to enhance the robustness and generalization ability of the neural network model, the preprocessing in this step also includes data enhancement, that is, performing bending transformation on each floating image obtained in the above preprocessing process, and obtaining All images in are new floating images. That is, the new image obtained by the bending transformation also belongs to the floating image obtained by preprocessing in this step.

Wherein, the bending transformation described in this embodiment can adopt three different degrees of bending transformation to realize data enhancement. In actual implementation, the number of bending transformations of different degrees can also be increased or decreased according to the actual situation.

So far, all the floating images obtained by preprocessing in this step constitute the training set of the neural network.

The third step is to train the neural network to obtain the trained neural network model. The neural network is trained based on the fixed image _IF obtained after preprocessing and the training set, and a trained neural network model is obtained.

The fourth step is to input the 3D medical image to be registered into the neural network model for registration, and finally obtain and output the registration image of the 3D medical image to be registered.

When the present invention is in use, image acquisition is carried out first, then the images collected are preprocessed, then the neural network is trained based on the fixed image I _F obtained by the preprocessing and the training set, and the trained neural network model is obtained, and then the registration is performed. The 3D medical images are input into the trained neural network model for registration, and finally the corresponding registration images are obtained and output.

This method has a better effect when the number of related images to be registered (human brain images to be registered in this embodiment) is relatively large. Specifically, after the neural network model has been trained, the images to be registered can be respectively input into the trained neural network model, and registration images corresponding to the images to be registered can be obtained. Specifically, after the neural network model is trained, each time an image to be registered is input, a corresponding registration image can be output; after the last registration image is output, the trained neural network model can continue to Input the next image to be registered until the image registration of all images to be registered is completed.

Wherein in the third step, the training neural network is obtained to obtain a trained neural network model, including:

S1. Input the fixed image I _F and the floating image I _M obtained after preprocessing into the neural network as the input layer of the neural network, and each set of input data includes the fixed image I _F and a floating image I _M .

Each set of input data I _F and I _M is sent to the neural network input layer after being losslessly spliced into a 2-channel 3D image.

S2. Down-sampling the input fixed image _IF and floating image _IM in the input layer, and output the feature maps of the input fixed image _IF and floating image _IM .

The downsampling includes 3 downsampling processes, a convolution calculation process with a convolution kernel size of 3×3×3 and a LeakyReLU activation function calculation process after the 3 downsampling processes; the 3 Each down-sampling process corresponds to three down-sampling process layers; the three down-sampling process layers are recorded as the first down-sampling process layer, the second down-sampling process layer and the third down-sampling process layer according to the execution order of the down-sampling process Process layer; each downsampling process layer includes a convolution layer with a convolution kernel size of 3×3×3, a LeakyReLU activation function layer, and a maximum pooling layer.

S3. Perform feature reweighting on the feature maps output by the first downsampling process layer, the second downsampling process layer, and the third downsampling process layer corresponding to the downsampling process layer, respectively, to obtain three weighted features The maps are, in order: the first weighted feature map, the second weighted feature map, and the third weighted feature map.

The implementation steps of feature reweighting described in the step S3 include:

Step S31, write down that each feature map to be reweighted in the sampling output is a feature map X, and this X∈R ^(H×W×D) , perform slice processing on the feature map X in its D dimension, Use the global average pooling strategy to perform global average pooling on each slice x∈R ^(H×W) obtained by slice processing, and obtain the slice descriptor z of each slice x on the D dimension of the feature map X, each The specific formula of the slice descriptor z is as follows:

In the formula (i, j) represents the pixel point on the slice x, and x(i, j) represents the gray value of the slice x at the pixel point (i, j);

Step S32, obtaining the weight s of each slice x on the D dimension of the feature map X, wherein the calculation formula of the weight s of each slice x is as follows:

s=σ(δ(z)),

Wherein, σ represents the sigmoid activation function, δ is the ReLU activation function, and z is the slice descriptor of the slice x obtained in step S31;

Step S33, correspondingly load each weight s obtained in step S32 to its corresponding slice, and obtain each slice on the D dimension of the feature map X × the corresponding reweighted slice The feature reweighting calculation formula corresponding to each slice x is as follows:

In this formula, F _scale (x, s) represents the multiplication operation between the slice x and its corresponding weight s;

Step S34, the reweighted slice corresponding to each slice x on the D dimension of the feature map X obtained in step S33 Corresponding to obtain the reweighted feature map corresponding to the feature map X

For example, for the feature map X∈R ^(H×W×D) , the feature map X obtained in step S33 has slices on the D dimension of X ₁ , X ₂ , ..., X _D , slices X ₁ , X ₂ , ..., The reweighted feature maps corresponding to X _D are Then there is a reweighted feature map corresponding to the feature map X

S4, S4, perform 1×1×1 convolution on the feature map output in step S2, and output the deformation field from the floating image I _M to the fixed image I _F

S5. Input the feature map output in step S2 into the upsampling layer for upsampling. The upsampling layer includes 3 upsampling process layers, and each upsampling process layer includes an UpSampling layer and a convolution kernel with a size of 3 × 3 × 3 convolutional layers, the convolution kernel size in each upsampling process layer is 3 × 3 × 3 convolutional layers, respectively, with a LeakyReLU activation function layer; the 3 upsampling process layers correspond to the above Three upsampling processes of upsampling; the three upsampling process layers are recorded as the first upsampling process layer, the second upsampling process layer and the third upsampling process layer in sequence according to the order in which the three upsampling processes occur Sampling process layer.

Wherein, after the feature map output by the UpSampling layer of the first upsampling process layer is fused with the third weighted feature map, it is used as the convolutional layer with a convolution kernel size of 3×3×3 in the first upsampling process layer Input; after the feature map output by the UpSampling layer of the second upsampling process layer is fused with the second weighted feature map, it is used as the convolution layer whose convolution kernel size is 3×3×3 in the second upsampling process layer Input; after the feature map output by the UpSampling layer of the third upsampling process layer is fused with the first weighted feature map, it is used as the convolution layer whose convolution kernel size is 3×3×3 in the third upsampling process layer enter.

S6. Perform 1×1×1 convolution on the feature maps output by the first upsampling process layer, the second upsampling process layer, and the third upsampling process layer, and output the first upsampling process layer and the second upsampling process layer The deformation fields from the floating image I _M to the fixed image I _F corresponding to the process layer and the third upsampling process layer are the deformation fields in turn deformation field and deformation field

S7, the floating image I _M and the deformation field of the above-mentioned output Input to the spatial transformation network, transform the floating image I _M and the above output deformation field Input to the space transformation network, the floating image I _M and the above output deformation field Input to the space transformation network, the floating image I _M and the above output deformation field Input to the space transformation network, respectively through the space transformation of the space transformation network, correspondingly obtain the deformed image corresponding to the floating image I _M , which is the transformed image successively deformed image deformed image and the transformed image

S8. Deformation field based on the above output deformation field deformation field And the deformed image based on the above deformed image deformed image deformed image Using the loss function to calculate the fixed image I _F and the deformed image Between the loss function value, and optimize the neural network by backpropagation, until the calculated loss function value no longer becomes smaller or the network training reaches the preset number of training iterations, the neural network training is completed, and the training Good neural network models.

Wherein, the calculation expression of the loss function is:

In this formula ①, Indicates the calculated loss function value, α and β are constants and α+β=1, is a regular term, λ is a regularization control constant parameter, Represents a pre-specified three-dimensional medical image obtained by downsampling the fixed image _IF ; a three-dimensional medical image The size, in turn, corresponds to the warped image Equal, 3D medical images The resolutions of are successively reduced and are smaller than the resolution of the fixed image I _F , Denotes the fixed image I _F and the deformed image The similarity measure between Represents the above three-dimensional medical image with the warped image The similarity measure between Represents the above three-dimensional medical image with the warped image The similarity measure between Represents the above three-dimensional medical image with the warped image The similarity measure between; The same similarity measure function is used.

Optionally, the similarity measurement function described in this embodiment uses a cross-correlation function; the space transformation network uses an STN space transformation network.

Optionally, the method 100 described in this embodiment may be implemented using a U-Net neural network as a basic neural network structure.

Optionally, the fusion of the feature maps involved in the present invention may be splicing and fusion. In this embodiment, U-Net-style channel dimension splicing and fusion can be used. In addition, when those skilled in the art realize the fusion of the feature maps involved in the present invention, they can also choose other fusion methods for fusion according to the actual situation. fusion.

In this embodiment, when downsampling, the downsampling factor of the maximum pooling layer is 2×2×2, correspondingly, it can be preset Obtained by reducing the fixed image I _F by 1/2, Obtained by reducing the fixed image I _F by 1/4, Obtained by reducing the fixed image I _F by 1/8, resolution > resolution > >0.

In summary, the neural network-based unsupervised 3D medical image registration method provided by the present invention, the use of its loss function, realizes the unsupervised registration of 3D medical images, and does not require any label information and registration information during the registration process. Supervising information reduces the need for labeled data and errors in human subjective judgments. To a certain extent, it helps medical workers who have no relevant medical experience to perform image registration, and to a certain extent, it improves the speed of registration and saves registration time. time, but also save manpower and material resources to a certain extent.

Referring to FIG. 3 , a neural network-based unsupervised three-dimensional medical image registration system 200 of the present invention includes:

The image acquisition unit 201 acquires three-dimensional medical images from the public data sets OASIS and ADNI;

The image preprocessing unit 202 performs preprocessing on the acquired three-dimensional medical image, including image segmentation, cropping, normalization processing and affine alignment, and selects any image from the affine aligned image as a fixed image I _F , and the rest of the images are used as floating images I _M ; wherein the cropped images have the same size;

The neural network training unit 203 trains the neural network based on the fixed image _IF and the floating image _IM obtained after preprocessing to obtain a trained neural network model;

Image registration unit 204, input the medical image to be registered into the above-mentioned trained neural network model for registration, obtain and output the registration image of the medical image to be registered;

Wherein, the neural network training unit 203 includes:

Input module, the fixed image I _F and floating image I _M obtained after preprocessing are input into neural network as the input layer of neural network, and each group of input data all comprises described fixed image I _F and a described floating image I _M ;

The downsampling module is used to downsample the fixed image _IF and the floating image _IM input in the input layer, and output the feature map; the downsampling includes 3 downsampling processes and one after the 3 downsampling processes The convolution calculation process with a convolution kernel size of 3×3×3 and a LeakyReLU activation function calculation process; the three downsampling processes correspond to the three downsampling process layers; the three downsampling process layers described above follow the downsampling process The execution sequence of is recorded as the first downsampling process layer, the second downsampling process layer and the third downsampling process layer; each downsampling process layer includes a convolution layer with a convolution kernel size of 3×3×3 , a LeakyReLU activation function layer and a maximum pooling layer;

The reweighting module performs feature reweighting on the feature maps output by the LeakyReLU activation function layer in the first downsampling process layer, the second downsampling process layer, and the third downsampling process layer corresponding to downsampling, and obtains three weighted The feature maps of , in order: the first weighted feature map, the second weighted feature map, and the third weighted feature map;

The first deformation field output module performs 1×1×1 convolution on the feature map output by the downsampling module, and outputs the deformation field from the floating image I _M to the fixed image I _F

The upsampling module inputs the feature map output by the downsampling module into the upsampling layer for upsampling, the upsampling layer includes 3 upsampling process layers, each upsampling process layer includes an UpSampling layer and a convolution kernel A convolutional layer with a size of 3×3×3, and a convolution kernel with a size of 3×3×3 in each upsampling process layer is respectively equipped with a LeakyReLU activation function layer; the 3 upsampling process layers correspond to The three upsampling processes of the upsampling; the three upsampling process layers are recorded as the first upsampling process layer, the second upsampling process layer and the first upsampling process layer in the order in which the three upsampling processes occur. Three upsampling process layers; wherein, after the feature map output by the UpSampling layer of the first upsampling process layer is fused with the third weighted feature map, the size of the convolution kernel in the first upsampling process layer is 3×3× The input of the convolutional layer of 3; after the feature map output by the UpSampling layer of the second upsampling process layer is fused with the second weighted feature map, the size of the convolution kernel in the second upsampling process layer is 3×3× The input of the convolutional layer of 3; after the feature map output by the UpSampling layer of the third upsampling process layer is fused with the first weighted feature map, the size of the convolution kernel in the third upsampling process layer is 3×3× The input of the convolutional layer of 3;

The second deformation field output module performs 1×1×1 convolution on the feature maps output by the first upsampling process layer, the second upsampling process layer and the third upsampling process layer, and outputs the first upsampling process layer , the deformation field from the floating image I _M to the fixed image I _F corresponding to the second upsampling process layer and the third upsampling process layer, which are the deformation fields in turn deformation field and deformation field

A spatial transformation module that transforms the floating image I _M and the above-mentioned output deformation field Input to the spatial transformation network, transform the floating image I _M and the above output deformation field Input to the space transformation network, the floating image I _M and the above output deformation field Input to the space transformation network, the floating image I _M and the above output deformation field Input to the space transformation network, respectively through the space transformation of the space transformation network, correspondingly obtain the deformed image corresponding to the floating image I _M , which is the transformed image successively deformed image deformed image and the transformed image

Neural network optimization module, deformation field based on the above output deformation field deformation field And the deformed image based on the above deformed image deformed image deformed image Using the loss function to calculate the fixed image I _F and the deformed image Between the loss function value, and optimize the neural network by backpropagation, until the calculated loss function value no longer becomes smaller or the network training reaches the preset number of training iterations, the neural network training is completed, and the training good neural network model;

The calculation expression of the loss function is:

In this formula ①, Indicates the calculated loss function value, α and β are constants and α+β=1, is a regular term, λ is a regularization control constant parameter, Represents a pre-specified three-dimensional medical image obtained by downsampling the fixed image _IF ; a three-dimensional medical image The size, in turn, corresponds to the warped image Equal, 3D medical images The resolutions of are successively reduced and are smaller than the resolution of the fixed image I _F , Denotes the fixed image I _F and the deformed image The similarity measure between Represents the above three-dimensional medical image with the warped image The similarity measure between Represents the above three-dimensional medical image with the warped image The similarity measure between Represents the above three-dimensional medical image with the warped image The similarity measure between; The same similarity measure function is used.

Wherein, the reweighting module includes:

The descriptor acquisition module records that each feature map to be reweighted in the sampling output is X, X∈R ^(H×W×D) , slices the feature map X on its D dimension, and uses The global average pooling strategy performs global average pooling on each slice x∈R ^(H×W) obtained from the slice processing, and obtains the slice descriptor z of each slice x on the D dimension of the feature map X, and each slice The specific formula of the descriptor z is as follows:

In the formula (i, j) represents the pixel point on the slice x, and x(i, j) represents the gray value of the slice x at the pixel point (i, j);

The slice weight calculation module obtains the weight s of each slice x on the D dimension of the feature map X, wherein the calculation formula of the weight s of each slice x is as follows:

s=σ(δ(z)),

Among them, σ represents the sigmoid activation function, δ is the ReLU activation function, and z is the slice descriptor of the slice x obtained by the descriptor acquisition module;

The weighting module loads each weight s obtained by the slice weight calculation module to its corresponding slice, and obtains each slice on the D dimension of the feature map X × the corresponding reweighted slice The feature reweighting calculation formula corresponding to each slice x is as follows:

In this formula, F _scale (x, s) represents the multiplication operation between the slice x and its corresponding weight s;

Weighted image acquisition module, based on the feature map X obtained by the weighting module, each slice x on the D dimension corresponds to the reweighted slice Corresponding to obtain the reweighted feature map corresponding to the feature map X

Optionally, the similarity measurement function adopts a cross-correlation function.

Optionally, the space transformation network is an STN space transformation network.

Optionally, the preprocessing in the image preprocessing unit 202 also includes data enhancement; wherein the data enhancement includes: performing bending transformation on each obtained floating image to obtain each obtained floating image The images after warp transformation corresponding to the image; each obtained warp transformed image is a newly added floating image.

For the same and similar parts among the various embodiments in this specification, refer to each other. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for the relevant parts, refer to the description in the method embodiment.

Although the present invention has been described in detail in conjunction with preferred embodiments with reference to the accompanying drawings, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, those skilled in the art can make various equivalent modifications or replacements to the embodiments of the present invention, and these modifications or replacements should be within the scope of the present invention/any Those skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present invention, and all should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (10)

1. An unsupervised three-dimensional medical image registration method based on a neural network is characterized by comprising the following steps of:

l1, image acquisition: acquiring three-dimensional medical images from public data sets OASIS and ADNI, and/or: acquiring a three-dimensional medical image from a DICOM interface of a CT, MRI or ultrasonic imager;

l2, preprocessing the acquired three-dimensional medical image: the method comprises image segmentation, clipping, normalization processing and affine alignment, and any one image is selected from images after affine alignment to be used as a fixed image I_FThe rest of the image is taken as a floating image I_M(ii) a Wherein the size of the cut images is consistent;

l3 based on the fixed image I obtained after preprocessing_FAnd a floating image I_MTraining a neural network to obtain a trained neural network model;

l4, inputting the medical image to be registered into the trained neural network model for registration to obtain and output a registration image of the medical image to be registered;

wherein in step L3, based on the fixed image I obtained after preprocessing_FAnd a floating image I_MTraining a neural network to obtain a trained neural network model, comprising:

s1, fixing image I obtained after preprocessing_FAnd a floating image I_MInputting the neural network as an input layer of the neural network, wherein each set of input data comprises the fixed image I_FAnd one said floating image I_M；

S2, inputting fixed image I in input layer_FAnd a floating image I_MPerforming down sampling to output a characteristic diagram;

the down-sampling comprises 3 down-sampling processes, a convolution calculation process with convolution kernel size of 3 multiplied by 3 and a LeakyReLU activation function calculation process after the 3 down-sampling processes; the 3 downsampling processes correspond to 3 downsampling process layers; the 3 down-sampling process layers are sequentially marked as a first down-sampling process layer, a second down-sampling process layer and a third down-sampling process layer according to the execution sequence of the down-sampling process; each downsampling process layer comprises a convolution layer with convolution kernel size of 3 multiplied by 3, a LeakyReLU activation function layer and a maximum pooling layer;

s3, respectively carrying out feature re-weighting on feature graphs output by LeakyReLU activation function layers in a first downsampling process layer, a second downsampling process layer and a third downsampling process layer corresponding to downsampling to obtain three weighted feature graphs, which are sequentially: a first weighted feature map, a second weighted feature map, and a third weighted feature map;

s4, performing 1 × 1 × 1 convolution on the feature map output in the step S2, and outputting a floating image I_MTo a fixed picture I_FDeformation field of

S _5, inputting the feature map output in the step S2 into an UpSampling layer for UpSampling, wherein the UpSampling layer comprises 3 UpSampling process layers, each UpSampling process layer comprises an UpSampling layer and a convolutional layer with the convolutional kernel size of 3 multiplied by 3, and a LeakyReLU activation function layer is arranged behind the convolutional layer with the convolutional kernel size of 3 multiplied by 3 in each UpSampling process layer; the 3 upsampling process layers correspond to the 3 upsampling processes of the upsampling; the 3 upsampling process layers are sequentially marked as a first upsampling process layer, a second upsampling process layer and a third upsampling process layer according to the sequence of the 3 upsampling processes;

after the feature map output by the UpSampling layer of the first UpSampling process layer is fused with the third weighted feature map, the feature map is used as the input of the convolutional layer with the convolutional kernel size of 3 multiplied by 3 in the first UpSampling process layer; fusing a feature map output by the UpSamplling layer of the second up-sampling process layer with the second weighted feature map, and taking the feature map as the input of the convolution layer with the convolution kernel size of 3 multiplied by 3 in the second up-sampling process layer; fusing a feature map output by an UpSampling layer of a third up-sampling process layer with the first weighted feature map, and taking the feature map as the input of a convolution layer with the convolution kernel size of 3 multiplied by 3 in the third up-sampling process layer;

s6, respectively carrying out 1 multiplied by 1 convolution on the feature maps output by the first up-sampling process layer, the second up-sampling process layer and the third up-sampling process layer, and outputting the floating images I corresponding to the first up-sampling process layer, the second up-sampling process layer and the third up-sampling process layer_MTo a fixed picture I_FIn turn is a deformation fieldDeformation fieldAnd deformation field

S7, floating the image I_MAnd the deformation field of the above outputInputting into a spatial transformation network, rendering floating images I_MAnd the deformation field of the above outputInput to the spatial transformation network, floating image I_MAnd the deformation field of the above outputInput to the spatial transformation network, floating image I_MAnd the deformation field of the above outputInputting the image data into the space transformation network, and correspondingly obtaining a floating image I through the space transformation of the space transformation network_MThe corresponding deformed images are sequentially deformed imagesDeformed imageDeformed imageAnd the deformed image

S8 deformation field based on the outputDeformation fieldDeformation fieldAndbased on the deformed image obtainedDeformed imageDeformed imageDeformed imageComputing a fixed image I using a loss function_FAnd the deformed imagePerforming back propagation optimization on the neural network until the calculated loss function value is not reduced or the network training reaches a preset training iteration number, and finishing the neural network training to obtain the trained neural network model;

the computational expression of the loss function is:

in the formula (i), the first and second groups,represents the calculated loss function value, α and β are both constants, α + β is 1,is a regularization term, λ is a regularization control constant parameter,representing a predetermined map defined by said fixed mapLike I_FDown-sampling the obtained three-dimensional medical image; three-dimensional medical imageSequentially with the deformed imageEquivalent, three-dimensional medical imageAre successively lower and are each smaller than the fixed image I_FThe resolution of (a) of (b),representing said fixed image I_FAnd the deformed imageA measure of the degree of similarity between the two,representing the three-dimensional medical imageAnd the deformed imageA measure of the degree of similarity between the two,representing the three-dimensional medical imageAnd the deformed imageA measure of the degree of similarity between the two,representing the three-dimensional medical imageAnd the deformed imageA measure of similarity between;the same similarity metric function is used.

2. The method for unsupervised three-dimensional medical image registration based on neural network as claimed in claim 1, wherein the step of implementing feature re-weighting in step S3 comprises:

step S31, recording each feature graph output in the down sampling and to be subjected to feature re-weighting is a feature graph X, wherein X is equal to R^{(H ×W×D)}Slicing the feature map X in the D dimension of the feature map X, and processing each slice X ∈ R obtained by slicing by using a global average pooling strategy^(H×W)And performing global average pooling to obtain a slice descriptor z of each slice X in the D dimension of the feature map X, wherein a specific formula of each slice descriptor z is as follows:

wherein (i, j) represents a pixel point on the slice x, and x (i, j) represents a gray value of the slice x at the pixel point (i, j);

step S32, obtaining a weight S of each slice X in the dimension D of the feature map X, where the calculation formula of the weight S of each slice X is as follows:

s＝σ(δ(z))，

wherein σ represents a sigmoid activation function, δ is a ReLU activation function, and z is a slice descriptor of the slice x obtained in step S31;

step S33, correspondingly loading each weight S obtained in step S32 to the corresponding slice, to obtain each slice X corresponding weighted slice in the D dimension of the feature map XWherein the feature re-weighting calculation formula corresponding to each slice x is as follows:

in the formula, F_scale(x, s) represents the multiplication between a slice x and its corresponding weight s;

step S34, based on the feature map X obtained in step S33, the reweighed slice corresponding to each slice X in the D dimension thereofCorrespondingly obtaining the re-weighted feature map corresponding to the feature map X

3. The unsupervised three-dimensional medical image registration method based on neural network as claimed in claim 1, wherein said similarity measure function is a cross-correlation function.

4. The unsupervised three-dimensional medical image registration method based on neural network as claimed in claim 1, wherein the spatial transformation network employs STN spatial transformation network.

5. The method of claim 1, wherein the preprocessing further comprises data enhancement;

the data enhancement comprises the following steps: respectively performing bending transformation on each obtained floating image to obtain a bent transformed image corresponding to each obtained floating image; the resulting warped images are newly added floating images.

6. An unsupervised three-dimensional medical image registration system based on a neural network, comprising:

an image acquisition unit acquiring three-dimensional medical images from the public data sets OASIS and ADNI, and/or: acquiring a three-dimensional medical image from a DICOM interface of a CT, MRI or ultrasonic imager;

the image preprocessing unit is used for preprocessing the acquired three-dimensional medical image, comprises image segmentation, cutting, normalization processing and affine alignment, and selects any one image from the images subjected to affine alignment as a fixed image I_FThe rest of the image is taken as a floating image I_M(ii) a Wherein the size of the cut images is consistent;

a neural network training unit based on the fixed image I obtained after preprocessing_FAnd a floating image I_MTraining a neural network to obtain a trained neural network model;

the image registration unit is used for inputting the medical image to be registered into the trained neural network model for registration to obtain and output a registration image of the medical image to be registered;

wherein, the neural network training unit comprises:

an input module for preprocessing the obtained fixed image I_FAnd a floating image I_MInputting the neural network as an input layer of the neural network, wherein each set of input data comprises the fixed image I_FAnd one said floating image I_M；

A down-sampling module for fixed image I input in the input layer_FAnd a floating image I_MPerforming down sampling to output a characteristic diagram; the down-sampling comprises 3 down-sampling processes, a convolution calculation process with convolution kernel size of 3 x 3 and a LeakyReLU laser after the 3 down-sampling processesA live function calculation process; 3 downsampling processes correspond to 3 downsampling process layers; the 3 down-sampling process layers are sequentially marked as a first down-sampling process layer, a second down-sampling process layer and a third down-sampling process layer according to the execution sequence of the down-sampling process; each downsampling process layer comprises a convolution layer with convolution kernel size of 3 multiplied by 3, a LeakyReLU activation function layer and a maximum pooling layer;

the reweighting module is used for respectively performing characteristic reweighting on the characteristic graphs output by the LeakyReLU activation function layer in the first downsampling process layer, the second downsampling process layer and the third downsampling process layer corresponding to downsampling to obtain three weighted characteristic graphs, and the three weighted characteristic graphs are sequentially: a first weighted feature map, a second weighted feature map, and a third weighted feature map;

a first deformation field output module for performing 1 × 1 × 1 convolution on the feature map output by the down-sampling module to output a floating image I_MTo a fixed picture I_FDeformation field of

The up-sampling module is used for inputting the characteristic diagram output by the down-sampling module into an up-sampling layer for up-sampling, the up-sampling layer comprises 3 up-sampling process layers, each up-sampling process layer comprises an UpSamplling layer and a convolution layer with the convolution kernel size of 3 multiplied by 3, and a LeakyReLU activation function layer is respectively arranged behind the convolution layer with the convolution kernel size of 3 multiplied by 3 in each up-sampling process layer; the 3 upsampling process layers correspond to the 3 upsampling processes of the upsampling; the 3 upsampling process layers are sequentially marked as a first upsampling process layer, a second upsampling process layer and a third upsampling process layer according to the sequence of the 3 upsampling processes; after the feature map output by the UpSampling layer of the first UpSampling process layer is fused with the third weighted feature map, the feature map is used as the input of the convolutional layer with the convolutional kernel size of 3 multiplied by 3 in the first UpSampling process layer; fusing a feature map output by the UpSamplling layer of the second up-sampling process layer with the second weighted feature map, and taking the feature map as the input of the convolution layer with the convolution kernel size of 3 multiplied by 3 in the second up-sampling process layer; fusing a feature map output by an UpSampling layer of a third up-sampling process layer with the first weighted feature map, and taking the feature map as the input of a convolution layer with the convolution kernel size of 3 multiplied by 3 in the third up-sampling process layer;

a second deformation field output module, which respectively performs 1 × 1 × 1 convolution on the feature maps output by the first upsampling process layer, the second upsampling process layer and the third upsampling process layer, and outputs the floating images I corresponding to the first upsampling process layer, the second upsampling process layer and the third upsampling process layer_MTo a fixed picture I_FIn turn is a deformation fieldDeformation fieldAnd deformation field

A spatial transformation module for transforming the floating image I_MAnd the deformation field of the above outputInputting into a spatial transformation network, rendering floating images I_MAnd the deformation field of the above outputInput to the spatial transformation network, floating image I_MAnd the deformation field of the above outputInput to the spatial transformation network, floating image I_MAnd the deformation field of the above outputInput to the spatial transformation network, respectively via the null of the spatial transformation networkInter-transforming, corresponding to obtain floating image I_MThe corresponding deformed images are sequentially deformed imagesDeformed imageDeformed imageAnd the deformed image

Neural network optimization module, deformation field based on the above outputDeformation fieldDeformation fieldAnd based on the deformed image obtainedDeformed imageDeformed imageDeformed imageComputing a fixed image I using a loss function_FAnd the deformed imagePerforming back propagation optimization on the neural network until the calculated loss function value is not reduced or the network training reaches a preset training iteration number, and finishing the neural network training to obtain the trained neural network model;

the computational expression of the loss function is:

in the formula (i), the first and second groups,represents the calculated loss function value, α and β are both constants, α + β is 1,is a regularization term, λ is a regularization control constant parameter,representing a predetermined image I fixed by said_FDown-sampling the obtained three-dimensional medical image; three-dimensional medical imageSequentially with the deformed imageEquivalent, three-dimensional medical imageAre successively lower and are each smaller than the fixed image I_FThe resolution of (a) of (b),representing said fixed image I_FAnd the deformed imageA measure of the degree of similarity between the two,representing the three-dimensional medical imageAnd the deformed imageA measure of the degree of similarity between the two,representing the three-dimensional medical imageAnd the deformed imageA measure of the degree of similarity between the two,representing the three-dimensional medical imageAnd the deformed imageA measure of similarity between;the same similarity metric function is used.

7. The unsupervised neural network-based three-dimensional medical image registration system of claim 6, wherein the re-weighting module comprises:

a descriptor obtaining module for recording that each feature graph to be subjected to feature re-weighting output in the down-sampling is X, and X belongs to R^{(H ×W×D}) Slicing the feature map X in the D dimension of the feature map X, and processing each slice X ∈ R obtained by slicing by using a global average pooling strategy^(H×W)And performing global average pooling to obtain a slice descriptor z of each slice X in the D dimension of the feature map X, wherein a specific formula of each slice descriptor z is as follows:

wherein (i, j) represents a pixel point on the slice x, and x (i, j) represents a gray value of the slice x at the pixel point (i, j);

the slice weight calculation module is used for acquiring the weight s of each slice X on the D dimension of the feature map X, wherein the calculation formula of the weight s of each slice X is as follows:

s＝σ(δ(z))，

wherein, σ represents a sigmoid activation function, δ is a ReLU activation function, and z is a slice descriptor of a slice x obtained by the descriptor obtaining module;

a weighting module for correspondingly loading each weight s obtained by the slice weight calculation module on the corresponding slice to obtain each slice on the D dimension of the characteristic diagram X and the weighted sliceWherein the feature re-weighting calculation formula corresponding to each slice x is as follows:

in the formula, F_scale(x, s) represents slice x and its corresponding weight sA multiplication operation between;

a weighted image acquisition module for obtaining a weighted slice corresponding to each slice X on D dimension of the feature image X based on the weighted imageCorrespondingly obtaining the re-weighted feature map corresponding to the feature map X

8. The unsupervised three-dimensional medical image registration system based on neural network as claimed in claim 6, wherein said similarity measure function is a cross-correlation function.

9. The unsupervised three-dimensional medical image registration system based on neural network as claimed in claim 6, wherein the spatial transformation network employs STN spatial transformation network.

10. The unsupervised three-dimensional medical image registration system based on neural network as claimed in claim 6, wherein said preprocessing in the image preprocessing unit further comprises data enhancement; wherein said data enhancement comprises: respectively performing bending transformation on each obtained floating image to obtain a bent transformed image corresponding to each obtained floating image; the resulting warped images are newly added floating images.