WO2016095168A1 - Tablet computer-based body data visualization method for surgical navigation - Google Patents

Abstract

A tablet computer-based body data visualization method for surgical navigation. A navigation system adopted in the method is mainly composed of a tablet computer, a navigator, an infrared locator, a reference frame and an adapter. Point-to-point real-time communication is achieved between the tablet computer and the navigator via wireless communication technology, and navigation information is transmitted and displayed on the tablet computer. By means of the method, image information about a cross section, a sagittal plane and a coronal plane corresponding to a navigation probe tip can be displayed on a tablet computer in real time, and a section image obtained by intersecting an extended plane of the tablet computer with the skull of a patient is displayed in real time. A doctor can move the tablet computer, so as to intuitively see a corresponding section image, which assists the doctor in locating tumors and lesions.

Description

Volume data visualization method for surgical navigation based on tablet computer Technical field

The invention belongs to the field of medical instruments, and in particular relates to a method for visualizing body data which can be used for surgical navigation based on a tablet computer.

Background technique

The prior art discloses that the neuronavigation system can assist the doctor in planning the surgical approach, determining the location and boundary of the lesion, and avoiding the loss of surrounding normal tissue and organs, which has become an indispensable surgery in the field of neurosurgery. Equipment; its working principle is based on the patient's own CT, MRI and other imaging data before surgery, establish an image guidance space, and register it with the real patient's space. The position of the surgical instrument is tracked in real time using a spatial locator and converted into an image space, and a virtual surgical instrument is superimposed and displayed on the patient's image. Thus, by positioning the virtual surgical instrument in the image space relative to the image, the position of the real surgical instrument relative to the display of the patient's anatomy is used to assist the physician in locating the surgical target and guiding the surgical procedure.

The traditional surgical navigation system utilizes virtual reality technology to superimpose virtual surgical instruments into the patient space, and the doctor performs surgical operations by observing the navigation screen. This kind of virtual reality technology makes doctors immersed in the virtual environment created by computer from the sensory effect, often lacking the feeling of “seeing is believing”. At the same time, during the operation, the doctor has to observe the position of the surgical instrument relative to the patient. The field of view is switched back and forth between the patient's surgical site and the navigation screen. Since the navigation screen has a certain distance from the operating table, this navigation mode is not conducive to the doctor's focus on the operation of the surgical site, and brings errors to the surgical positioning.

Another disadvantage of the conventional surgical navigation system is that it can only display the patient images of the cross-sectional, sagittal, and coronal planes corresponding to the tip of the navigation probe, and cannot obtain an arbitrary cross-sectional image of the patient's volume data field. When the patient is lying on the side and can not get the positive position, it will have a certain impact on the doctor's judgment.

In clinical practice, there is a need for a new neuronavigation system that not only solves the problem of switching back and forth between fields of vision, but also reduces positioning errors, reduces surgery time, reduces the risk of surgery, and does not cause inconvenience to doctors. The invention intends to provide a mobile wireless communication technology, which can transplant a navigation screen to a tablet computer located beside the doctor, and use a spatial locator to track the tablet computer, obtain a cross-sectional image of the plane of the tablet computer, and display it on the screen at the same time. Volume data visualization method.

At present, the mobile navigation-based medical navigation device related to the present invention includes: a DASH navigation system developed by BrainLab, Germany, which has been certified by CE and FDA and has been used for preliminary Bed test, but the above equipment is expensive and complicated to operate.

Summary of the invention

It is an object of the present invention to provide a method for visualizing body data based on tablet-based surgical navigation, and more particularly to a method for surgical navigation based on an arbitrary cross-section visualization of a skull portion of a tablet computer.

In the present invention, the surgical navigation system used is mainly composed of a tablet computer, a navigation device and an auxiliary infrared locator, a reference frame and an adapter, and the surgical navigation system is mainly composed of a tablet computer and an excelim-04 or higher navigator. The tablet computer is equipped with a reference frame for infrared tracking, the screen resolution is not less than 800X600, supports multi-touch and handwriting input, supports Wi-Fi wireless connection mode, and has a power life of not less than 8 hours. The navigation device includes a host computer, a display, an infrared locator, and other related peripheral devices; the tablet computer and the navigation device realize real-time communication in a point-to-point manner through wireless communication technology, and transmit and display the navigation information on the tablet computer.

The method of the invention mainly realizes two intraoperative navigation modes; one is to display the image information of the cross section, the sagittal plane and the coronal plane corresponding to the needle tip of the navigation probe in real time on the tablet; the other is real time display A cross-sectional image of the extended face of the tablet and the small feet of the patient's head.

The method of the present invention is accomplished by image scanning, surgical planning, spatial registration, and intraoperative navigation (as shown in the steps of FIG. 1).

In the method of the present invention, the navigation system employed includes two intra-operative navigation modes:

(1) Tablet computer displays traditional navigation information

In this mode, the tablet is used to receive and display traditional navigation information in real time. The image information displayed by the tablet computer is the same as that of the remote display screen, and is refreshed in real time as the information on the navigation screen is updated (as shown in FIG. 2);

(2) Tablet computer displays arbitrary cross-section information

Because the patient is generally lying on the side of the operation, it is not easy to obtain the patient's positive position image. Therefore, an adapter that can be used for infrared tracking is installed on the tablet, placed near the patient's head, and the position information of the plane where the tablet is located is tracked by the spatial locator. The position information is used to generate an image of the patient's skull section, which is transmitted and displayed on the tablet; when the position of the tablet is changed, any cross-sectional image of the skull can be obtained, thereby facilitating the doctor to obtain the position and depth information of the brain tumor (eg, Figure 3 and Figure 4).

Specifically, the method for visualizing a volume data for a tablet-based surgical navigation according to the present invention is characterized in that an arbitrary cross-sectional image of a patient's skull is acquired and displayed on a tablet according to position information of the tablet computer. On the computer screen, assisted surgery: it includes:

(1) Extending the screen of the tablet by its boundaries;

(2) transforming the orientation information of the extended surface into the image space;

(3) obtaining a corresponding cross-sectional image by resampling according to the position of the extended surface in the image space;

(4) The cross-sectional image is transmitted and displayed on the tablet screen.

More specifically, the method for visualizing volume data for tablet-based surgical navigation according to the present invention includes the following steps:

(1) Install an adapter for infrared tracking on the tablet, track the three reflective balls on the adapter with an optical locator, establish a three-dimensional adapter coordinate system according to the position coordinates of the three reflective balls, and automatically obtain optical positioning. The transformation between the world coordinate system and the adapter coordinate system where the instrument is located, T _aw (shown in Figure 4), is specifically expressed as a 4 × 4 homogeneous matrix M ₁ ;

平移l个单位，得到4个相应的角点A’，B’，C’，D’，并按以下公式计算出4个新的角点在适配器坐标系下的坐标：(2) Using the navigation probe and the spatial locator to obtain the coordinates of the probe tip on the four corner points of the tablet screen (points A, B, C, D in Fig. 3), that is, the four corner points in the adapter coordinates. Coordinates in the system: A(x ₁ , y ₁ , z ₁ ), B(x ₂ , y ₂ , z ₂ ), C(x ₃ , y ₃ , z ₃ ), D(x ₄ , y ₄ , z ₄ ), and then calculate the horizontal width l of the tablet, with the right border of the tablet as the axis (as shown by the long dotted line in Figure 3), press A, B, C, D 4 corners

Panning 1 unit, get 4 corresponding corner points A', B', C', D', and calculate the coordinates of 4 new corner points in the adapter coordinate system according to the following formula:

为

方向上的单位向量，计算公式如下：among them

for

The unit vector in the direction is calculated as follows:

(3) Fix a reference frame next to the patient, track the four reflective balls on the reference frame with an optical locator, establish a three-dimensional coordinate system according to the position coordinates of the above four reflective balls, define the patient coordinate system, and use the spatial locator Obtaining a coordinate transformation T _pw between the locator coordinate system and the patient coordinate system, specifically represented as a 4×4 homogeneous matrix M ₂ , and the transformation between the adapter coordinate system and the patient coordinate system can be expressed as M ₁ *M ₂ ;

(4) Before the image is scanned, a marker point that can be used for scanning is attached to the patient's scalp, and a three-dimensional image coordinate system is established on the obtained patient data after scanning, and a spatial registration method based on the marker point is used to obtain the patient coordinate system. The transformation relationship T _{pi of the} image coordinate system is specifically expressed as a 4×4 homogeneous matrix M ₃ , and finally the transformation M=M ₁ *M ₂ *M _{3 of the} adapter coordinate system to the image coordinate system is obtained;

(5) Transform the coordinates of the four new corner points A', B', C', D' in the adapter coordinate system into the image coordinate system according to the transformation relationship M, and create a rectangular frame with the four points as the vertices. According to the position of the rectangular frame in the image space, re-sampling in the frame to obtain a two-dimensional image, that is, the corresponding cross-sectional image;

(6) The cross-sectional image is transmitted and displayed on the screen of the tablet through wireless communication technology (WiFi, etc.).

In the method of the present invention, all the calculation processes are performed in the host computer, and finally the resampled cross-sectional image is transmitted and displayed on the tablet computer; the mobile tablet computer, the position information of the extended surface thereof also changes, on the tablet screen The realistic cross-sectional image is updated in real time as the orientation of the tablet changes, thereby achieving the goal of visualizing any section.

The method of the present invention can obtain an arbitrary cross-sectional image by re-sampling the cross-sectional image by real-time updating the position information of the extended surface, and refresh the display in real time on the tablet screen. Compared with the conventional surgical navigation system, the present invention makes the present invention The doctor can better focus on the surgical site, without extensive field of view switching in the display screen and surgical area, not only can display the standard cross-sectional image, but also display any cross-sectional image through the orientation of the tablet computer to assist the doctor to locate the tumor and lesions. , bringing great convenience and help to the operation.

Compared with the prior art, the invention has the following advantages:

(1) The present invention enables a tablet computer and a navigator to perform a point-to-point wireless connection through a wireless network, and completes real-time transmission and display of image data, thereby realizing a mobile navigation operation.

(2) The invention can not only display the images of the cross-section, the sagittal plane and the coronal plane corresponding to the tip of the navigation probe on the tablet computer, but also obtain an arbitrary cross-sectional image of the patient's skull according to the position information of the tablet computer, which is convenient for positioning. Tumor, assist the doctor in the operation.

(3) The present invention solves the problem of frequent visual field switching between a doctor's traditional navigation screen and a surgical field, and reduces the registration error caused by human operation.

(4) The invention facilitates the doctor to better understand the orientation information of the navigation image: with the traditional navigation system, the doctor can only obtain images of three positive-axis planes, and the orientation on the navigation screen, such as front, back, left and right, and the actual patient. The orientation of the invention is detached; in the present invention, the doctor can move the tablet computer and visually see the corresponding cross-sectional image by moving forward, moving up, etc., thereby better understanding the orientation information of the image.

DRAWINGS

1 is a schematic view showing the working flow of the surgical navigation system of the present invention;

Figure 2 shows the process of transferring navigation information and displaying it on the tablet screen during surgery;

Figure 3 shows the process of generating and displaying a cross-sectional image;

Figure 4 shows the transformation relationship between the various coordinate systems;

Figure 5 shows the cross-sectional image of the plane of intersection of the extended surface of the tablet and the skull model on the tablet screen.

detailed description

Example 1

As shown in FIG. 1 to FIG. 5, using the tablet-based visualization method of the body data for surgical navigation of the present invention, clinical practice is carried out by the following steps:

(1) Install an adapter for infrared tracking on the tablet, track the three reflective balls on the adapter with an optical locator, establish a three-dimensional adapter coordinate system according to the position coordinates of the three reflective balls, and automatically obtain optical positioning. The transformation between the world coordinate system and the adapter coordinate system where the instrument is located, T _aw (shown in Figure 4), is specifically expressed as a 4 × 4 homogeneous matrix M ₁ ;

平移l个单位，得到4个相应的角点A’，B’，C’，D’，并按以下公式计算出4个新的角点在适配器坐标系下的坐标：(2) Using the navigation probe and the spatial locator to obtain the coordinates of the probe tip on the four corner points of the tablet screen (points A, B, C, D in Fig. 3), that is, the four corner points in the adapter coordinates. Coordinates in the system: A(x ₁ , y ₁ , z ₁ ), B(x ₂ , y ₂ , z ₂ ), C(x ₃ , y ₃ , z ₃ ), D(x ₄ , y ₄ , z ₄ ), and then calculate the horizontal width l of the tablet, with the right border of the tablet as the axis (as shown by the long dotted line in Figure 3), press A, B, C, D 4 corners

Panning 1 unit, get 4 corresponding corner points A', B', C', D', and calculate the coordinates of 4 new corner points in the adapter coordinate system according to the following formula:

为

方向上的单位向量，计算公式如下：among them

for

The unit vector in the direction is calculated as follows:

(3) Fix a reference frame next to the patient, track the four reflective balls on the reference frame with an optical locator, establish a three-dimensional coordinate system according to the position coordinates of the above four reflective balls, define the patient coordinate system, and use the spatial locator Obtaining a coordinate transformation T _pw between the locator coordinate system and the patient coordinate system, specifically represented as a 4×4 homogeneous matrix M ₂ , and the transformation between the adapter coordinate system and the patient coordinate system can be expressed as M ₁ *M ₂ ;

(4) Before the image is scanned, a marker point that can be used for scanning is attached to the patient's scalp, and a three-dimensional image coordinate system is established on the obtained patient data after scanning, and a spatial registration method based on the marker point is used to obtain the patient coordinate system. The transformation relationship T _{pi of the} image coordinate system is specifically expressed as a 4×4 homogeneous matrix M ₃ , and finally the transformation M=M ₁ *M ₂ *M _{3 of the} adapter coordinate system to the image coordinate system is obtained;

(5) Transform the coordinates of the four new corner points A', B', C', D' in the adapter coordinate system into the image coordinate system according to the transformation relationship M, and create a rectangular frame with the four points as the vertices. According to the position of the rectangular frame in the image space, re-sampling in the frame to obtain a two-dimensional image, that is, the corresponding cross-sectional image;

(6) The cross-sectional image is transmitted and displayed on the screen of the tablet through wireless communication technology (WiFi, etc.).

The results show that the doctor can move the tablet and visually view all the corresponding cross-sectional images by moving forward, moving up, etc., so as to better understand the orientation information of the image, not only the corresponding position of the navigation probe tip can be displayed on the tablet. The images of the cross-section, sagittal and coronal planes can also obtain arbitrary cross-sectional images of the patient's skull according to the position information of the tablet, which is convenient for positioning the tumor and assisting the doctor in the operation.

Example 2

Using the tablet-based visualization method of the body data for surgical navigation of the present invention, clinical practice is carried out by the following steps:

(1) Install an induction coil that can be positioned by 6 degrees of freedom by the electromagnetic tracking system on the tablet, and define an adapter coordinate system on the induction coil. The electromagnetic tracking system automatically obtains the world coordinate system and the adapter coordinate system where the electromagnetic tracking system is located. The transformation T _aw between them is specifically expressed as a 4×4 homogeneous matrix M ₁ ;

平移l个单位，得到4 个相应的角点A’，B’，C’，D’，并按以下公式计算出4个新的角点在适配器坐标系下的坐标：(2) Obtain the coordinates of the four corner points (points A, B, C, D in Fig. 3) on the screen of the tablet by prior calibration, that is, the coordinates of the four corner points in the adapter coordinate system: A (x ₁ , y ₁ , z ₁ ), B(x ₂ , y ₂ , z ₂ ), C(x ₃ , y ₃ , z ₃ ), D(x ₄ , y ₄ , z ₄ ), and then calculate the tablet Horizontal width l, with the right edge of the tablet as the axis (as shown by the long dotted line in Figure 3), press A, B, C, D 4 corners

Panning 1 unit, get 4 corresponding corner points A', B', C', D', and calculate the coordinates of 4 new corner points in the adapter coordinate system according to the following formula:

为

方向上的单位向量，计算公式如下：among them

for

The unit vector in the direction is calculated as follows:

(3) Fix a reference frame next to the patient (including a 6-degree-of-freedom tracking induction coil that can be tracked by the electromagnetic tracking system), establish a three-dimensional coordinate system based on the above-mentioned induction coil, that is, the patient coordinate system, and obtain its own coordinates using the electromagnetic tracking system. The coordinate transformation T _pw between the system and the patient coordinate system is specifically expressed as a 4×4 homogeneous matrix M ₂ , and the transformation between the adapter coordinate system and the patient coordinate system can be expressed as M ₁ *M ₂ ;

(4) Before the image is scanned, a marker point that can be used for scanning is attached to the patient's scalp, and a three-dimensional image coordinate system is established on the obtained patient data after scanning, and a spatial registration method based on the marker point is used to obtain the patient coordinate system. The transformation relationship T _{pi of the} image coordinate system is specifically expressed as a 4×4 homogeneous matrix M ₃ , and finally the transformation M=M ₁ *M ₂ *M _{3 of the} adapter coordinate system to the image coordinate system is obtained;

(5) Transform the coordinates of the four new corner points A', B', C', D' in the adapter coordinate system into the image coordinate system according to the transformation relationship M, and create a rectangular frame with the four points as the vertices. According to the position of the rectangular frame in the image space, re-sampling in the frame to obtain a two-dimensional image, that is, the corresponding cross-sectional image;

(6) The cross-sectional image is transmitted and displayed on the screen of the tablet through wireless communication technology (WiFi, etc.).

The results show that the doctor can move the tablet and visually view all the corresponding cross-sectional images by moving forward, moving up, etc., so as to better understand the orientation information of the image, not only the corresponding position of the navigation probe tip can be displayed on the tablet. The images of the cross-section, sagittal and coronal planes can also obtain arbitrary cross-sectional images of the patient's skull according to the position information of the tablet, which is convenient for positioning the tumor and assisting the doctor in the operation.

Claims (7)

A method for visualizing volume data for surgical navigation based on a tablet computer, characterized in that, according to the position information of the tablet computer, an arbitrary cross-sectional image of the patient's skull is obtained and displayed on the screen of the tablet computer, which comprises the steps of: (1) Extending the screen of the tablet according to its boundary, installing an adapter for infrared tracking on the tablet, or installing an induction coil that can be positioned by 6 degrees of freedom by the electromagnetic tracking system; (2) transforming the orientation information of the extended surface into the image space; (3) obtaining a corresponding cross-sectional image by resampling according to the position of the extended surface in the image space; (4) The cross-sectional image is transmitted and displayed on the tablet screen. The method of claim 1 wherein said step (1): Install an adapter for infrared tracking on the tablet, use the optical locator to track the three reflective balls on the adapter, create a three-dimensional adapter coordinate system, and automatically obtain the transformation between the world coordinate system and the adapter coordinate system where the optical locator is located. T _aw , specifically expressed as a 4×4 homogeneous matrix M ₁ ; Use the navigation probe and the spatial locator to obtain the coordinates of the four corner points A, B, C, and D of the probe tip on the tablet screen, that is, the coordinates of the four corner points in the adapter coordinate system: A (x ₁ , y ₁ , z ₁ ), B(x ₂ , y ₂ , z ₂ ), C(x ₃ , y ₃ , z ₃ ), D(x ₄ , y ₄ , z ₄ ), and then calculate the level of the tablet Width l, with the right edge of the tablet as the axis, press A, B, C, D4 corners

Translate l units to obtain 4 corresponding corner points A', B', C', D', and calculate the coordinates of the four corner points of the extended surface in the adapter coordinate system according to the following formula:

among them

for

The unit vector in the direction is calculated as follows:

The method of claim 1 wherein said step (2): A reference frame is fixed next to the patient, and the four reflective balls on the reference frame are tracked by the optical locator, and a three-dimensional coordinate system is established according to the position coordinates of the above four reflective balls, which is defined as the patient coordinate system, and the positioning is obtained by using the spatial locator. The coordinate transformation T _pw between the instrument coordinate system and the patient coordinate system is specifically expressed as a 4×4 homogeneous matrix M ₂ , and the transformation between the adapter coordinate system and the patient coordinate system can be expressed as M ₁ *M ₂ ; Alternatively, a reference frame containing a 6-degree-of-freedom tracking induction coil that can be tracked by the electromagnetic tracking system is fixed next to the patient, a three-dimensional coordinate system is established according to the induction coil, that is, the patient coordinate system, and the electromagnetic tracking system is used to obtain its own coordinate system and The coordinate transformation T _pw between the patient coordinate systems is specifically expressed as a 4×4 homogeneous matrix M ₂ , and the transformation between the adapter coordinate system and the patient coordinate system can be expressed as M ₁ *M ₂ ; Before the image is scanned, a marker point for scanning is determined in the patient's head. After the scan, a three-dimensional image coordinate system is established for the obtained patient data, and a spatial registration method is used to obtain a transformation relationship T _pi between the patient coordinate system and the image coordinate system. Specifically represented as a 4×4 homogeneous matrix M ₃ , the transformation M=M ₁ *M ₂ *M _{3 of the} adapter coordinate system to the image coordinate system is finally obtained. The method of claim 1 wherein said step (3): Transform the coordinates of the four new corner points A', B', C', D' in the adapter coordinate system into the image coordinate system according to the transformation relationship M, and create a rectangular frame with the four points as the vertices, according to The position of the rectangular frame in the image space is resampled in the frame to obtain a two-dimensional image, which is the corresponding cross-sectional image. The method of claim 1 wherein said step (4): The obtained cross-sectional image is transmitted and displayed on the screen of the tablet through wireless communication technology. The method of claim 1 wherein the location information of the extended surface of the mobile tablet changes; the actual cross-sectional image on the tablet screen is updated in real time as the orientation of the tablet changes. The method according to claim 1, wherein in the step (1), an induction coil that can be positioned by 6 degrees of freedom by an electromagnetic tracking system is mounted on the tablet, and an adapter coordinate system is defined on the induction coil. The electromagnetic tracking system is used to automatically obtain the transformation T _aw between the world coordinate system and the adapter coordinate system where the electromagnetic tracking system is located, which is specifically expressed as a 4×4 homogeneous matrix M ₁ .

Images