TWI741889B - Method and system for register operating space - Google Patents

Abstract

A system for register operating space includes a first positioning mark, a local camera, a second positioning mark, a global camera and a computer system. The first positioning mark is disposed on a patient. The local camera captures a first image covering the first positioning mark. The second positioning mark is disposed on the local camera. The global camera captures a second image covering the second positioning mark. The focal length of the global camera is shorter than the focal length of the local camera. The computer system is communicatively connected to the area camera and the global camera to provide a navigation interface based on the first image and the second image.

Description

Surgical space registration system and method

This disclosure relates to a surgical image registration system and method using a regional camera and a global camera.

With the increase in the proportion of the aging population, coupled with the influence of modern life characteristics, such as obesity, sedentary, etc., the proportion of spine lesions has increased year by year; after conservative treatment fails, it is often necessary to rely on implants to help the spine to relieve pain and reduce pain. Maintain basic functions. The spine is responsible for the basic function of protecting the central nervous system, but the implants that can be used are quite narrow, such as pedicle screws, which may damage the central nervous system. Although there are minimally invasive orthopedic surgery on the market, the position of the spine changes with the posture of the person. Therefore, how to accurately track the position of the spine during the operation is a topic of concern to those skilled in the art.

The embodiment of the disclosure proposes a surgical space registration system, which includes a first positioning mark, an area camera, a second positioning mark, a global camera, and a computer system. The first positioning mark is used to be set on the patient. The area camera is used to shoot the first image covering the first positioning mark. The second positioning mark is set on the area camera. The global camera is used to shoot a second image covering the second positioning mark, wherein the focal length of the global camera is smaller than the focal length of the area camera. The computer system is communicatively connected to the area camera and the global camera to provide a navigation interface based on the first image and the second image.

In some embodiments, a calibration procedure includes: shooting a third image covering the first positioning mark through the area camera; identifying the first positioning mark in the third image through a computer system to calculate the distance between the area camera and the first positioning mark The first conversion model; the fourth image covering the first positioning mark and the second positioning mark is captured by the global camera; the first positioning mark and the second positioning mark in the fourth image are recognized by the computer system to calculate the global camera and the second positioning mark A second conversion model between positioning marks, and a third conversion model between the global camera and the second positioning mark is calculated; the area is calculated by the computer system according to the first conversion model, the second conversion model, and the third conversion model The fourth conversion model between the camera and the second positioning mark.

In some embodiments, the surgical space registration system further includes an X-ray imaging machine and a third positioning mark. The third positioning mark is arranged on the X-ray imaging machine, wherein the second image also covers at least the third positioning mark. The computer system is also used to identify the first positioning mark in the first image to calculate a fifth conversion model between the area camera and the first positioning mark. The computer system is also used to identify the second positioning mark in the second image to calculate the sixth conversion model between the global camera and the second positioning mark, and to identify the third positioning mark in the second image to calculate the global camera and X The seventh conversion model between light imaging machines. The computer system is used to calculate the eighth conversion model between the first positioning mark and the X-ray imaging machine according to the fourth conversion model, the fifth conversion model, the sixth conversion model and the seventh conversion model, and to provide navigation according to the eighth conversion model interface.

In some embodiments, the X-ray imaging machine is a C-arm X-ray machine. The C-arm X-ray machine includes a transmitting end and a receiving end, and the above-mentioned third positioning mark is provided on the receiving end.

In some embodiments, the number of third positioning marks is greater than one, and each third positioning mark corresponds to a launch angle of the transmitting end. From another perspective, the embodiment of the present disclosure proposes a method for registering a surgical space, which is suitable for a computer system. The surgical space registration method includes: shooting a first image covering a first positioning mark through an area camera, where the first positioning mark is used to be set on the patient; shooting a second image covering a second positioning mark through a global camera, where the second image The positioning mark is set on the area camera, and the focal length of the global camera is smaller than the focal length of the area camera; and a navigation interface is provided according to the first image and the second image.

Regarding the “first”, “second”, etc. used in this text, it does not particularly mean the order or sequence, but only to distinguish elements or operations described in the same technical terms.

Fig. 1 is a schematic diagram illustrating a surgical space registration system according to an embodiment. 1, the surgical space registration system 100 includes a computer system 110, a global camera 120, an area camera 130, a positioning mark 131, a positioning mark 141 to 143, an X-ray imaging machine 150 and a positioning mark 61 to 163.

The global camera 120 and the area camera 130 can capture visible light images. In some embodiments, the global camera 120 and the area camera 130 may also include an infrared emitter, an infrared sensor, a dual camera, a structured light sensing device, or any device that can sense the depth of the scene.

The positioning marks 141 to 143 are used to be set on the patient, such as on the vertebra 140. When the posture of the patient changes, the positions of the positioning marks 141 to 143 will also change, so the positioning marks 141 to 143 can be used to track the position of the vertebra 140. In other embodiments, the positioning marks 141 to 143 may also be set on other body parts of the patient, and the present disclosure is not limited thereto. In this embodiment, the positioning marks 141 to 143 are in the field of view of the area camera 130. In other words, the area camera 130 can capture images covering the positioning marks 141 to 143, and the positioning marks 141 to 143 have a specific pattern or patterns. Identify. The positioning mark 131 is set on the area camera 130, and the positioning mark 131 is used to track the position of the area camera 130.

In this embodiment, the X-ray imaging machine is a C-arm X-ray machine. The C-arm X-ray machine has a transmitting end 152 and a receiving end 151, wherein the positioning marks 161 to 163 are arranged on the receiving end 151. The emitting end 152 can adjust the angle of emitting X-rays, and the positioning marks 161 to 163 correspond to different emitting angles of the emitting end 152 respectively.

The positioning mark 131 and the positioning marks 161 to 163 are in the field of view of the global camera 120. In other words, the global camera 120 can capture images covering the aforementioned positioning marks. In this embodiment, the focal length of the global camera 120 is smaller than the focal length of the area camera 130. The area camera 130 is used to shoot the positioning marks 141 to 143 at a close distance, thereby providing a clear image, which is used to track the positioning mark 141 ~143 position. The global camera 120 is used to capture an image of the entire scene, and the image is used to determine the relative positional relationship between the positioning marks.

The computer system 110 is used to communicate with the global camera 120 and the area camera 130, for example, through any wired or wireless communication means. The computer system 110 is used to provide a navigation interface based on the images captured by the two cameras. In some embodiments, the navigation interface may be displayed on the screen of the computer system, but in some embodiments, it may also be displayed on any head-mounted device, tablet computer, or transparent display, and the disclosure is not limited thereto. Any technologies such as virtual reality, augmented reality, alternative reality, or mixed reality can be used to form the aforementioned navigation interface, and the present disclosure is not limited thereto. For example, the patient can take a computer tomography (CT) before surgery, and then use image processing technology to segment the vertebrae in the computer tomography to generate corresponding virtual objects, in which three vertebrae have corresponding positioning Mark 141~143. The above-mentioned virtual objects can be displayed in the navigation interface. If the positioning marks 141 to 143 move during the operation, the computer system 110 can change the position of the virtual objects in the navigation interface, so there is no need to repeatedly shoot X-ray images. The images taken by the global camera 120 and the area camera 130 are used to calculate the positions of the positioning marks 141 to 143 (the vertebra 140) in the entire scene. First, a calibration procedure must be performed.

，此轉換模型例如是一矩陣，其包含了在三維空間中位移、旋轉等資訊。在上述辨識過程中，是將定位標記141在影像中的座標轉換為在三維空間中的轉換模型，此轉換可根據相機130的資訊(例如包括焦距)來執行，此轉換也可以經由合適的電腦視覺技術或校正程序來求得。舉例來說，可透過區域相機130拍攝具有特定圖案的旋轉盤，進而求得影像座標與三維空間之間的轉換，本揭露並不限制如何計算此轉換。 Fig. 2 is a schematic diagram illustrating a calibration procedure according to an embodiment. Please refer to FIG. 2, the area camera 130 will shoot an image covering the positioning mark 141, and the computer system will recognize the positioning mark 141 in this image to calculate the conversion model between the area camera 130 and the positioning mark 141

The conversion model is, for example, a matrix, which contains information such as displacement and rotation in a three-dimensional space. In the above recognition process, the coordinates of the positioning mark 141 in the image are converted into a conversion model in three-dimensional space. This conversion can be performed based on the information of the camera 130 (including the focal length), and the conversion can also be performed by a suitable computer. Obtained by vision technology or calibration procedures. For example, a rotating disk with a specific pattern can be photographed through the area camera 130 to obtain the conversion between the image coordinates and the three-dimensional space. The present disclosure does not limit how to calculate this conversion.

，並且計算出全域相機120與定位標記131之間的轉換模型

。類似地，轉換模型

、

包括了在三維空間中位移、旋轉等資訊，關於全域相機120的影像座標與三維空間之間的轉換可由合適的電腦視覺技術來求得。 The global camera 120 can shoot images covering the positioning mark 141 and the positioning mark 131, and then the computer system can recognize the positioning mark 141 and the positioning mark 131 in the image to calculate the conversion model between the global camera 120 and the positioning mark 141

, And calculate the conversion model between the global camera 120 and the positioning mark 131

. Similarly, the conversion model

,

Including information such as displacement and rotation in the three-dimensional space, the conversion between the image coordinates of the global camera 120 and the three-dimensional space can be obtained by appropriate computer vision technology.

。全域相機120與定位標記141之間的轉換模型

可拆成三部分，如以下數學式1所示。由數學式1可以推導出數學式2，根據轉換模型

、

與

可計算出轉換模型

。 [數學式1]

[數學式2]

It is worth noting that the position of the positioning mark 131 is not equal to the position of the area camera 130, and a conversion model is needed between the positioning mark 131 and the area camera 130.

. Conversion model between global camera 120 and positioning mark 141

It can be divided into three parts, as shown in the following mathematical formula 1. Mathematical formula 2 can be derived from Mathematical Formula 1, according to the conversion model

,

and

The conversion model can be calculated

. [Math 1]

[Math 2]

In this calibration procedure, the positions of the global camera 120, the area camera 130 and the positioning mark 141 can be arranged arbitrarily. The relative position between the positioning mark 131 and the area camera 130 is fixed after the calibration procedure.

，此轉換模型可以如以下數學式3分解。 [數學式3]

Please refer back to FIG. 1, after the above-mentioned calibration procedure, the positions of the positioning marks 141 to 143 relative to the X-ray imaging machine 150 can be calculated. Specifically, taking the positioning mark 162 and the positioning mark 141 as an example, the conversion model between the positioning mark 141 and the positioning mark 162 is

, This conversion model can be decomposed as the following mathematical formula 3. [Math 3]

。另外，轉換模型

已經在校正程序中取得，將轉換模型

取反矩陣便可以得到轉換模型

。全域相機120拍攝了涵蓋定位標記131與定位標記162的影像，辨識此影像中的定位標記131可以計算出轉換模型

，而辨識出此影像中的定位標記162可以計算出

。將轉換模型

、

、

與

代入至數學式3可以計算出轉換模型

。如此一來，電腦系統可以根據此轉換模型

提供導航介面，例如計算出定位標記141相對於接收端162的位置，進而在導航介面中合適的位置顯示一虛擬物件來表示對應的脊椎骨。 The area camera 130 captures an image covering the positioning mark 141, and the conversion model can be obtained by identifying the positioning mark 141 in the image

. In addition, the conversion model

Has been obtained in the calibration procedure, the model will be converted

Invert the matrix to get the conversion model

. The global camera 120 captures an image covering the positioning mark 131 and the positioning mark 162, and the conversion model can be calculated by recognizing the positioning mark 131 in the image

, And recognizing the positioning mark 162 in this image can calculate

. Will convert the model

,

,

and

Substitute into Mathematical formula 3 to calculate the conversion model

. In this way, the computer system can transform the model according to this

A navigation interface is provided, for example, the position of the positioning mark 141 relative to the receiving end 162 is calculated, and then a virtual object is displayed at a suitable position in the navigation interface to represent the corresponding spine.

Fig. 3 is a flowchart illustrating a method for registering a surgical space according to an embodiment. Referring to FIG. 3, in step 301, a first image covering a first positioning mark is captured through an area camera, where the first positioning mark is set on the patient. In step 302, a second image covering the second positioning mark is captured through the global camera, where the second positioning mark is used to be set on the area camera, and the focal length of the global camera is smaller than the focal length of the area camera. In step 303, a navigation interface is provided according to the first image and the second image. However, each step in FIG. 3 has been described in detail as above, and will not be repeated here. It is worth noting that each step in FIG. 3 can be implemented as multiple program codes or circuits, and the present invention is not limited thereto. In addition, the method in FIG. 3 can be used in conjunction with the above embodiments, or can be used alone. In other words, other steps can also be added between the steps in FIG. 3.

In the above method, an area camera and a global camera are set. The global camera can provide images of the entire scene, and the area camera can provide more accurate images of the surgical site, thereby ensuring both accuracy and camera field of view.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the relevant technical field can make some changes and modifications without departing from the spirit and scope of the present invention. The protection scope of the present invention shall be subject to those defined by the attached patent application scope.

,

,

,

,

,

,

,

,

:轉換模型 100: Surgical space registration system 110: Computer system 120: Global camera 130: Area camera 131: Positioning mark 140: Spine 141~143: Positioning mark 150: X-ray imaging machine 151: Receiving end 152: Transmitting end 161~163: Positioning mark

,

,

,

,

,

,

,

,

: Conversion model

In order to make the above-mentioned features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail in conjunction with the accompanying drawings. Fig. 1 is a schematic diagram illustrating a surgical space registration system according to an embodiment. Fig. 2 is a schematic diagram illustrating a calibration procedure according to an embodiment. Fig. 3 is a flowchart illustrating a method for registering a surgical space according to an embodiment.

100: Surgical space registration system

110: computer system

120: Global Camera

130: area camera

131: Location mark

140: Spine

141~143: Positioning mark

150: X-ray imaging machine

151: receiving end

152: Transmitter

161~163: positioning mark

,

,

,

,

:轉換模型

,

,

,

,

: Conversion model

Claims (8)

An operating space registration system, comprising: a first positioning mark, which is set on a patient; an area camera, which shoots a first image covering the first positioning mark; and a second positioning mark, which is set on the area camera; A global camera that shoots a second image covering the second positioning mark, wherein the focal length of the global camera is less than the focal length of the area camera; and a computer system that is communicatively connected to the area camera and the global camera, according to the The first image and the second image provide a navigation interface, wherein the computer system executes a calibration procedure, and the calibration procedure includes: shooting a third image covering the first positioning mark through the area camera; The first positioning mark in the third image is used to calculate a first conversion model between the area camera and the first positioning mark; the global camera is used to capture an image covering the first positioning mark and the second positioning mark A fourth image; identify the first positioning mark and the second positioning mark in the fourth image through the computer system to calculate a second conversion model between the global camera and the first positioning mark, and calculate A third conversion model between the global camera and the second positioning mark; and using the computer system to calculate the area camera and the second conversion model according to the first conversion model, the second conversion model, and the third conversion model position A fourth transformation model between the marks, wherein the first transformation model, the second transformation model, the third transformation model, and the fourth transformation model are each a matrix, and the matrix is used to perform displacement or Rotate, the fourth conversion model is a multiplication of the first conversion model, the second conversion model, and the third conversion model. The surgical space registration system according to claim 1, further comprising; an X-ray imaging machine; and at least one third positioning mark arranged on the X-ray imaging machine, wherein the second image also covers the at least one third positioning mark Positioning mark, wherein the computer system also recognizes the first positioning mark in the first image to calculate a fifth conversion model between the area camera and the first positioning mark, wherein the computer system also recognizes the second The second positioning mark in the image is used to calculate a sixth conversion model between the global camera and the second positioning mark, and the at least one third positioning mark in the second image is recognized to calculate the global camera and the second positioning mark. A seventh conversion model between the X-ray imaging machine, wherein the computer system calculates the first positioning mark and the seventh conversion model according to the fourth conversion model, the fifth conversion model, the sixth conversion model and the seventh conversion model An eighth conversion model between X-ray imaging machines, and the navigation interface is provided according to the eighth conversion model. The surgical space registration system as described in claim 2, wherein The X-ray imaging machine is a C-arm X-ray machine. The C-arm X-ray machine includes a transmitting end and a receiving end. The at least one third positioning mark is arranged on the receiving end. The surgical space registration system according to claim 3, wherein the number of the at least one third positioning mark is greater than one, and each of the third positioning marks is located at a launch angle of the transmitting end. An operation space registration method is suitable for a computer system. The operation space registration method includes: shooting a first image covering a first positioning mark through an area camera, wherein the first positioning mark is set on a patient; The global camera shoots a second image covering a second positioning mark, where the second positioning mark is set on the area camera, and the focal length of the global camera is smaller than the focal length of the area camera; a calibration procedure is performed, and the calibration The procedure includes: shooting a third image covering the first positioning mark through the area camera; identifying the first positioning mark in the third image through the computer system to calculate the distance between the area camera and the first positioning mark A first conversion model of a; a fourth image covering the first positioning mark and the second positioning mark is captured through the global camera; the first positioning mark and the second positioning mark in the fourth image are identified through the computer system Positioning mark to calculate the global camera and the first positioning mark And calculate a third conversion model between the global camera and the second positioning mark; and use the computer system according to the first conversion model, the second conversion model, and the The third conversion model calculates a fourth conversion model between the area camera and the second positioning mark, wherein the first conversion model, the second conversion model, the third conversion model, and the fourth conversion model are respectively A matrix for displacement or rotation in a three-dimensional space, the fourth transformation model being the multiplication of the first transformation model, the second transformation model and the third transformation model; and according to the first image and The second image provides a navigation interface. The surgical space registration method according to claim 5, wherein the second image further covers at least one third positioning mark, and the at least one third positioning mark is set on an X-ray imaging machine, and the surgical space registration method further includes: Identify the first positioning mark in the first image to calculate a fifth conversion model between the area camera and the first positioning mark; identify the second positioning mark in the second image to calculate the global A sixth conversion model between the camera and the second positioning mark, and identifying the at least one third positioning mark in the second image to calculate a seventh conversion model between the global camera and the X-ray imager ; And according to the fourth conversion model, the fifth conversion model, the sixth conversion model and the seventh conversion model to calculate the first positioning mark and the X-ray imaging machine An eighth conversion model between the two, and the navigation interface is provided according to the eighth conversion model. The surgical space registration method according to claim 6, wherein the X-ray imaging machine is a C-arm X-ray machine, the C-arm X-ray machine includes a transmitting end and a receiving end, and the at least one third positioning mark is provided on the Receiving end. The surgical space registration method according to claim 7, wherein the number of the at least one third positioning mark is greater than one, and each of the third positioning marks is located at an emission angle of the transmitting end.

Images