CN104305957B - Wear-type molecular image navigation system - Google Patents

Abstract

A head-mounted molecular image navigation system, including: a multi-spectral light source module, used to irradiate visible light and near-infrared light to the detection area; a signal acquisition module, used to collect near-infrared fluorescence images and visible light images of imaging objects; head-mounted The system support module is used to carry the multi-spectral light source module and the signal acquisition module to adjust the irradiation of the multi-spectral light source module to the detection area; the image processing module is used to collect near-infrared light images and The visible light image is fused, and the fused image is output. According to the embodiment of the present invention, the flexible use of the equipment in the application of the image system is effectively realized, and the application space of the optical molecular image navigation is expanded.

Description

Head-mounted molecular image navigation system

technical field

The invention relates to an imaging system, in particular to a head-mounted molecular image navigation system.

Background technique

As a new method and means of non-invasive visualization imaging technology, molecular imaging essentially reflects the changes in the biological molecular level and the overall function of organisms caused by changes in molecular regulation. Therefore, it is an important technology to study the life activities of genes, biomacromolecules and cells at the molecular level in vivo. The basic research of technology has become one of the hotspots and difficulties in the field of molecular imaging.

Molecular imaging equipment combines traditional medical imaging technology with modern molecular biology, and can observe physiological or pathological changes from the cellular and molecular levels. It has the advantages of non-invasive, real-time, in vivo, high specificity, high sensitivity, and high-resolution imaging. . The use of molecular imaging technology, on the one hand, can greatly speed up the development of drugs and shorten the time for preclinical drug research; provide more accurate diagnosis, make the treatment plan best match the patient's genetic map, and help pharmaceutical companies develop personalized treatments Drugs; on the other hand, it can be applied in the field of biomedicine to achieve in vivo quantitative analysis, image navigation, molecular typing and other goals. However, the system using this method is relatively complicated, and the ease of operation and user comfort need to be further improved.

Therefore, the present invention proposes a head-mounted molecular image navigation system, which detects in-body targets in molecular images by means of multi-spectral excitation, and enhances the scope of application.

Contents of the invention

The invention provides a head-mounted molecular image navigation system, comprising:

The multi-spectral light source module is used to irradiate visible light and near-infrared light to the detection area;

A signal acquisition module, configured to acquire near-infrared fluorescence images and visible light images of imaging objects;

The head-mounted system support module is used to carry the multi-spectral light source module and the signal acquisition module, so as to adjust the irradiation of the multi-spectral light source module to the detection area;

The image processing module is used to perform image fusion on the collected near-infrared light image and visible light image, and output the fusion image.

Embodiments of the invention have the following technical effects:

1. Realize molecular image navigation and molecular imaging through head-mounted methods, which improves convenience while realizing functions.

2. The method of projection imaging can guide the operator to pre-judge the imaging range, thus increasing the function of human-computer interaction.

3. Using the function of voice recognition can facilitate the operator to further free his hands in the process of using the system, so as to control the head-mounted molecular image navigation system more accurately.

4. Due to the eigenvalue extraction method of threshold decomposition, the signal-to-background ratio is significantly improved, which is helpful for operators to operate accurately in real time according to image guidance.

Description of drawings

1 is a schematic structural view of a head-mounted system support module according to an embodiment of the present invention;

2 is a block diagram of a head-mounted molecular image navigation system according to an embodiment of the present invention;

FIG. 3 is a flowchart of an image processing method of the head-mounted molecular image navigation system according to an embodiment of the present invention.

detailed description

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be described in further detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

Embodiments of the present invention provide a head-mounted molecular image navigation system based on excited fluorescence imaging in molecular images.

FIG. 1 is a schematic structural diagram of a head-mounted system support module according to an embodiment of the present invention. FIG. 2 is a block diagram of a head-mounted molecular image navigation system according to an embodiment of the present invention. As shown in Figure 2, the head-mounted molecular image navigation system may include a multi-spectral light source module 110, used to provide a plurality of different spectral bands of light, so as to irradiate the subject; an optical signal acquisition module 120, used to collect the subject in real time The fluorescence excitation image and visible light image of the object to be inspected; the head-mounted system support module 130 is used to adjust the comfort of the operator when wearing it, and ensure the safe and effective imaging; the image processing module 140 is used to perform image segmentation and feature extraction , image registration and other processing to realize the fusion of visible light image and fluorescence image and output the fusion image.

Next, the operations of the multispectral light source module 110 , the optical signal acquisition module 120 , the head mounted system support module 130 and the image processing module 140 will be described in detail respectively.

The multispectral light source module 110 may include a cold light source 111 , a near-infrared laser 112 and a light source coupler 113 . The cold light source 111 is used to emit visible light to the object under inspection. The cold light source 111 may be provided with a first bandpass filter to transmit visible light with a wavelength of 400-650nm. The near-infrared laser 113 is configured to emit near-infrared light having a central wavelength of, for example, 785 nm. The excitation light source can be extracted through an optical fiber. It is known to those skilled in the art that the embodiments of the present invention are not limited to the above-mentioned implementation manners, and other manners known in the art may also be used to emit visible light and near-infrared light. When the detection area is excited, based on the spectral separation method, the cold light source 111 and the near-infrared laser 112 are simultaneously emitted by a single optical fiber. Specifically, the light emitted by the visible light source and the near-infrared light source is coupled at the light exit. A light source coupler 113 is provided at the coupling. The light source coupler 113 can be a diverging lens, which changes the light source from a straight point light source into a cone beam light, so that the irradiation area can be enlarged, so as to achieve uniform irradiation of the excitation light source on the detection area. For example, an optical lens may be provided at the light exit of the near-infrared laser 112 , and the optical lens is reversely coupled to the output end of the laser to achieve output with a large divergence angle of the light source. A mechanical fixing method may be used to fix one end of the optical fiber and the optical lens together, and connect the other end of the optical fiber to the head-mounted system support module 130 .

The optical signal acquisition module 120 may include a camera 121 , a lens 122 and a coordinate projector 123 . The camera 121 is configured to collect near-infrared fluorescence signals and visible light signals. Among them, the cold light source illuminates the background during the acquisition process. For example, the reference parameters required for near-infrared light signal acquisition can be set as follows: at 800nm, the quantum efficiency is higher than 30%, the frame rate is greater than 30fps, and the size of the image source (that is, the smallest photosensitive unit point image source of the camera 121) is greater than 5 Microns. Preferably, a second bandpass filter is placed between the camera 121 and the lens 122 so as to transmit near-infrared light with a wavelength of 810-870nm. When the camera 121 is in operation, the coordinate projector 123 can project a circular outline to the detection area (not shown in the figure), which is the maximum range of the field of view, so that the operator can obtain the detection area of the system, and at the same time it is convenient for the operator The excitation range of the multispectral light source module 110 is obtained.

As shown in FIG. 1 , the head-mounted system support module 130 may include a head-mounted system bracket 131 . The head-mounted system bracket 131 is used to carry the light source module 110 and the signal collection module 120 . Preferably, the head-mounted system support module 130 may also include a voice recognition and control module 132 . The voice recognition control module 132 may include a microphone, a voice recognition unit and a control unit (not shown), so as to control the operations of the multi-spectral light source module 110, the coordinate projector 123 and other modules through the operator's voice. The voice recognition and control module 132 can be implemented using voice recognition technology known in the art.

The visible light image and the near-infrared fluorescence image of the subject from the optical signal acquisition module 120 are respectively input to the image processing module 140 . The image processing module 140 is implemented by back-end computer processing, and the acquisition and light source control can also be manually controlled by the back-end. The image processing module 140 first preprocesses the input near-infrared fluorescence image, so as to obtain the characteristic distribution of the fluorescence image according to fluorescence specificity. Preprocessing can include noise removal, feature extraction, and dead point compensation. Of course, preprocessing known in the art can also be performed on the visible light image. The feature extraction of the input near-infrared fluorescence image can be performed by threshold segmentation. For example, for a pixel in the near-infrared fluorescence image whose gray value G/background noise gray value G _{n is} higher than 1.5, multiply the gray value of the pixel by 2, and for pixels whose G/G _n is lower than 1.5 point, divide the gray value of the pixel by 2. According to this threshold segmentation method, feature points can be enhanced. For regions of interest whose grayscale value is greater than the preset threshold, these regions of interest can be converted into pseudo-color images through a grayscale image-to-pseudo-color image adjustment algorithm known in the art, so as to further mark the feature points and the features of the region. Position, so that the operator can guide the operation according to the image. The image processed by the image processing module 140 is a fused image, which has a display and projection interface on the general-purpose computer, which is convenient for the operator to realize the output display of the image. At the same time, the video signal can be fed back to the head-mounted system, and the fusion image can be visualized by placing the screen in front of the mirror.

Then, by using the obtained optical characteristic distribution of the fluorescence image, image fusion is performed on the visible light image of the fluorescence input, so as to obtain a fusion result image for output. Specifically, the image fusion of the fluorescence image and the visible light image includes registering the fluorescence image and the visible light image by using the optical property distribution of the fluorescence image. This registration operation will be described in detail below.

The distribution of optical properties of the fluorescence image is fluorescence-specific, while the visible light image is a high-resolution structural image. Image registration according to an embodiment of the present invention takes advantage of the above properties. When performing registration, the morphology theory can be used to correct the minimized energy function of the distribution of the optical properties of the fluorescence image so that its shape is close to that of the imaged tissue. Registration can be performed using the following equation (1).

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; \&Delta;</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&beta;</span>}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; u</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; W</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

In formula (1), d is the discrete Laplace operator, U is the position vector, and n surface points are selected as the main marking points, p _i and a _i are the imaging surface marking points respectively, W ⁱ =(p _i -a _i ) Moving the vector, the vector U _P is obtained by minimizing E(U), then is the position of the surface after deformation.

In order to obtain a more accurate and high-resolution fusion image, the image coincidence degree shown in the following formula (2) is used as the registration effect evaluation standard when performing registration.

$\frac{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; A</span>}<span\; class="notranslate">\; \&cap;</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; |</span>}{<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; A</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; |</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

Among them, A is the normalized gray value matrix of the visible light image, and B is the normalized gray value matrix of the fluorescence image. The closer the calculation result is to 1, the better the image registration effect is.

Fig. 3 shows a flowchart of an image processing method according to an embodiment of the present invention. As shown in FIG. 3 , in step 301 , spatial motion detection is performed on the preprocessed visible light image sequence and fluorescence image sequence, so as to filter out mismatched micro-displacement frames, and obtain visible light image sequence M1 and fluorescence image sequence M2.

Optionally, in step 303, an image pyramid P1 is formed for the high-resolution visible light image sequence M1 obtained in step 301, so as to reduce the amount of data, thereby improving the real-time performance of image processing. Specifically, the Gaussian pyramid is used to down-sample the image so as to generate the i+1th layer according to the i-th layer of the pyramid. The i-th layer is first convolved with a Gaussian kernel, and then all even rows and columns are removed. Of course, the size of the newly obtained image will become a quarter of the previous image. In this case, the image is first doubled in every dimension, and the new rows (even-numbered rows) are filled with zeros. It is then convolved with the specified filter (actually a filter that doubles in each dimension) to estimate an approximation of the "missing" pixels. The entire pyramid is generated by looping over the input image as described above.

In step 305, edge detection is performed on the obtained image pyramid P1 and the fluorescence image sequence M2 by using, for example, a gradient edge detection method using the Roberts operator, to obtain image edges E1 and E2 respectively. Of course, step 303 can also be skipped when the image processing capability is high, and edge detection is directly performed on the visible light image sequence M1 and the fluorescence image sequence M2.

In step 307, saliency-based sparse sampling is performed on the obtained image edges E1 and E2 respectively. The same method can be used to perform saliency-based sparse sampling on the image edges E1 and E2 respectively. Here, the compressed sensing sparse sampling technology is used to sample the coefficients of E1 and E2, so as to obtain the sampling outputs S1 and S2 respectively.

In step 308, registration is performed on the sampled outputs S1 and S2 obtained in step 307. In addition to using the above formulas (1) and (2) for registration, point cloud registration can be used to further optimize the registration results. For details on point cloud registration, please refer to "Xue Yaohong et al., Research on Point Cloud Data Registration and Surface Subdivision Technology, National Defense Industry Press, 2011", which will not be repeated in this article.

Preferably, the image processing method according to the present invention may further include step 309 . In step 309, algorithm convergence verification is performed on the result of point cloud registration, so as to ensure the stability and reliability of the calculation process.

Preferably, the processing of steps 301, 303, 305, and 309 can be performed by a smaller image GPU or FPGA, and at the same time, a central processing unit CPU with stronger computing power is used to perform the registration step 308, thereby further optimizing system performance At the same time, the required hardware size is reduced.

The specific embodiments described above have further described the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (10)

1. a wear-type molecular image navigation system, including:

Multispectral light source module, for irradiating visible ray and near infrared light to search coverage；

Signal acquisition module, for gathering near-infrared fluorescent image and the visible images of imaging object；

Head-mounted system supports module, is used for carrying described multispectral light source module and described signal acquisition module, to adjust State the multispectral light source module range of exposures to described search coverage；

Image processing module, for the near infrared light image gathered and visible images are carried out image co-registration, and exports fusion Image；

Wherein, described image processing module carries out image co-registration include near-infrared fluorescent image and the visible images of collection: The optical characteristics obtaining described near-infrared fluorescent image by making following energy function formula minimize is distributed:

$E\left(U\right)=\left|\right|{\mathrm{\&Delta;}}_{d}U|{|}^2+\mathrm{\&beta;}\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}||U^i-W^i|{|}^2$

Wherein, d is discrete Laplace operator, and U is position vector, selects n surface point as main mark point, p_i、a_iRespectively For imaging surface labelling point, Wⁱ=(p_i-a_i) motion-vector, obtain vector U by minimizing E (U)_P, thenFor table Position after facial disfigurement, Δ d represents first derivative, UⁱRepresent the position vector of i-th labelling point position,Represent mark The imaging surface border of note point p position,Representing the surface-boundary of whole characteristics of image, β represents weight coefficient.

System the most according to claim 1, wherein, described multispectral light source module includes:

Visible light source, for launching visible ray to detected object；

Near infrared laser, for launching near infrared light to detected object；With

Light source coupler；

Wherein, described light source coupler couples described visible ray and near infrared light, and coupling is optically coupled to by simple optical fiber Described head-mounted system supports module.

System the most according to claim 2, wherein, described head-mounted system supports module and includes:

Head-mounted system support, is used for carrying described multispectral light source module and described signal acquisition module；And

Speech control module, for controlling the operation of multispectral light source module, to form the search coverage of expected range.

System the most according to claim 1, wherein, the near-infrared fluorescent image gathered is carried out by described image processing module Feature extraction, including:

For image intensity value G/ background noise gray value G_nPixel higher than 1.5, is multiplied by 2 by the gray value of described pixel； For G/G_nPixel less than 1.5, by the gray value of described pixel divided by 2.

System the most according to claim 1, wherein, described image processing module to gather near-infrared fluorescent image and can See that light image carries out image co-registration, including by the picture registration degree shown in following formula as registration effect assessment standard:

$\frac{2\left|A\&cap;B\right|}{\left|A\right|+\left|B\right|}$

Wherein, A is visible images Normalized Grey Level value matrix, and B is fluoroscopic image Normalized Grey Level value matrix.

System the most according to claim 4, wherein, described image processing module is come further to the reddest by point cloud registering Outer fluoroscopic image and visible images carry out image co-registration.

7. it is applied to an image processing method for wear-type molecular image navigation system described in claim 1, including:

Detection Method in Optical Image Sequences and near-infrared fluorescent image sequence are carried out space motion detection, in order to filter unmatched small Displacement frame (301)；

The described Detection Method in Optical Image Sequences detected through space motion is carried out down-sampling, obtains image pyramid (303)；

Use gradient edge detection method, respectively the image pyramid obtained and near-infrared fluorescent image sequence are carried out edge inspection Survey, obtain image border (305)；

The image border obtained is carried out respectively sparse sampling based on significance, thus respectively obtains sampling output (307)；With And

Sampling output to obtaining performs registration to carry out image co-registration (308).

Method the most according to claim 7, wherein, described registration includes by the picture registration degree shown in following formula as joining Quasi-effect assessment standard:

$\frac{2\left|A\&cap;B\right|}{\left|A\right|+\left|B\right|}$

Wherein, A is visible images Normalized Grey Level value matrix, and B is near-infrared fluorescent image normalization gray value matrix.

Method the most according to claim 7, also includes using point cloud registering further to near-infrared fluorescent image and can See that light image carries out image registration.

Method the most according to claim 9, also includes that the result to point cloud registering carries out Algorithm Convergence checking.