JP2024092349A - Surgical Observation System - Google Patents

Abstract

To provide a surgical observation system that enables a doctor, when observing a location other than the center of a monitor screen, to focus on that location simply by directing his/her line of sight toward that location without moving a surgical site camera.
[Solution] The line of sight camera 18 detects the position of the image plane 5a and the line of sight of the main doctor D1, and the observation point detection unit 21 detects the observation point P1 on the image plane 5a that the main doctor D1 is actually looking at. Then, the autofocus unit 11 moves the movable lens 8 of the objective optical system 9 to maximize the contrast value of the observation point P1, so that the focus is set to the observation point P1. Therefore, even if the doctor D1 wants to observe a location other than the center of the image plane 5a of the monitor 5, he or she can focus on that location by simply directing his or her line of sight to that location without moving the surgical site camera 2.
[Selected figure] Figure 5

Description

The present invention relates to a surgical observation system.

In recent years, a method has been adopted in which a camera is installed near the operating table to capture the area being operated on, the image captured by the camera is displayed on a monitor, and the doctor performs the surgery while watching the image displayed on the monitor. This method has been made possible by technological advances in cameras and monitors, which have dramatically improved their resolution.

The doctor can perform surgery while observing the image of the surgical area displayed on the monitor with his head raised (head-up position), so there is no need to look down and focus on the eyepiece of the microscope as is the case with surgery performed under a microscope, and surgery can be performed in a comfortable position.

The camera is equipped with an autofocus mechanism that focuses on the center of the monitor screen, and the doctor performs surgery by moving the camera so that the area he or she wants to see is at the center of the screen (see, for example, Patent Document 1).

Relisted No. 2019-225693

However, in this type of conventional technology, the camera's autofocus mechanism is designed to focus on the center of the monitor's image screen, so if the doctor wants to view the periphery of the image screen other than the center, he or she must move the camera each time.

The present invention was made with a focus on such related technologies, and aims to provide a surgical observation system that allows a doctor to focus on a location other than the center of the monitor screen by simply directing his or her line of sight toward that location without moving the camera.

According to a first technical aspect of the present invention, the present invention is characterized by comprising an objective optical system including a movable lens for varying focal length that can be moved freely along an optical axis inside; a focusing means for moving the movable lens; a surgical site camera equipped with an image pickup element for capturing an image of the surgical site and outputting a captured image signal; a monitor that receives the image signal output from the surgical site camera and displays an image based on the image signal on an image plane and is provided with a marker for recognizing the image plane as seen by the doctor; a gaze camera that detects the position of the image plane as seen by the doctor by image recognition of the marker on the monitor and detects the doctor's gaze by image recognition of the movement of the pupil of one of the doctor's left or right eye; an observation point detection unit that detects the position of the monitor's image plane detected by the gaze camera and the position of the doctor's observation point on the image plane from the doctor's gaze; and an autofocus unit that outputs a signal to the focusing means to move the movable lens in the focus direction so as to maximize the contrast value of the image at the observation point of the monitor detected by the observation point detection means.

According to a second technical aspect of the present invention, a gaze camera is provided on either the left or right side of the dedicated glasses worn by the doctor, and a half mirror is integrally incorporated on the side of the dedicated glasses where the gaze camera is provided, within a range that does not interfere with the doctor's viewing of the monitor. The half mirror has a concave inner surface that can reflect reflected light from the eye closer to the gaze camera and can be received by the gaze camera, and is large enough to allow the gaze camera to capture the entire monitor. The gaze camera captures an image of the monitor through the half mirror, and the marker is recognized as an image to detect the position of the image surface as seen by the doctor, and the inner surface of the half mirror receives reflected light from the doctor's eye into the gaze camera, and the movement of the pupil of that eye is recognized as an image to detect the doctor's gaze.

According to the first technical aspect of the present invention, the eye camera detects the position of the image plane and the doctor's eye line, the observation point detection unit detects the observation point on the image plane at the point of the doctor's eye line, and the autofocus unit moves the movable lens of the objective optical system to maximize the contrast value of the observation point, so that the focus is set to the observation point. Therefore, even when the doctor observes a location other than the center of the monitor's image plane, he or she can focus on that location simply by directing his or her eye line in that location without moving the surgical site camera.

According to a second technical aspect of the present invention, an eye gaze camera is installed in special glasses worn by the doctor, and the eye gaze camera captures the movement of the doctor's eyes from close up and recognizes the image, so the doctor's gaze can be detected reliably.

FIG. 1 is an overall perspective view showing a surgical observation system. FIG. 4 is a side view showing the internal structure of the surgical site camera. FIG. Top view with the doctor's gaze directed towards the center of the monitor. Plan view of the doctor's gaze directed toward an observation point other than the center of the monitor. FIG.

Figures 1 to 6 show a preferred embodiment of the present invention.

The patient lies on operating table 1, and the surgical area G where the surgery is performed is located on the abdomen. At the front of operating table 1 are main doctor D1, who will perform the surgery, and his two assistants N1 and N2. On the other side of operating table 1 are sub-doctor D2, who will assist with the surgery, and his one assistant N3.

A surgical site camera 2 is supported by a stand device 3 above the surgical site G. The position of the surgical site camera 2 can be freely changed by moving the stand device 3. The surgical site camera 2 can capture stereoscopic images in 4K. The surgical site camera 2 has operation handles 4 on the left and right, and the main doctor D1 can hold these operation handles 4 to move the surgical site camera 2 or change its direction. In addition, the operation handle 4 has multiple buttons, and by operating the buttons, manual focusing and magnification can be performed.

On the opposite side of the operating table 1, next to the sub-doctor D2, there is installed an LCD panel type monitor 5 capable of 4K stereoscopic display. This monitor 5 is for the main doctor D1, who can perform surgery while checking the stereoscopic image displayed on this monitor 5 from the front side across from the operating table 1. Another monitor 6 for the sub-doctor D2 is also installed near the operating table 1.

Inside the surgical site camera 2, an objective optical system 9 consisting of a fixed lens 7 and a movable lens 8 is provided. The surgical site G to be observed is located on the optical axis K of the objective optical system 9. The movable lens 8 can be moved along the optical axis K by the focusing means 10, and the focal length of the objective optical system 9 can be changed by moving the movable lens 8. The focusing means 10 is connected to an autofocus unit 11, and the movable lens 8 moves under the control of a control signal S4 from the autofocus unit 11, which will be described later.

The light beam L from the surgical site G passes through the objective optical system 9, then passes through the variable magnification optical system 12, splitting into two light beams L on the left and right, and then passes through the imaging lens 13 and is directed to a pair of left and right image sensors 14.

The video signal S1 of a pair of left and right images with binocular parallax captured by the image sensor 14 is output to the autofocus unit 11, and from there is output to and displayed on the monitors 5 and 6. Of the two monitors 5 and 6, the monitor 5 for the main doctor D1 has markers M at the four corners to detect the position of its image surface 5a.

The main doctor D1, the sub-doctor D2, and the assistants N1 to N3 wear special glasses 15, 16. The special glasses 15, 16 are equipped with polarized lenses 17 on the left and right, and by looking at the image surface 5a through these polarized lenses 17, the surgical site G can be observed in three dimensions.

Only the dedicated glasses 15 for the main doctor D1 are provided with a small gaze camera 18 near the left ear, facing the direction in which the main doctor D1 is looking. A half mirror 19 is built into the dedicated glasses 15 on the left side so as not to interfere with the main doctor D1's viewing of the monitor 5. This half mirror 19 has the ability to reflect 60% of visible light and transmit 40%. The inner surface of the half mirror 19 is concave so that it can reflect reflected light R from the left eye 20 and introduce it into the gaze camera 18, and is sized to allow the gaze camera 18 to capture the entire monitor 5.

The gaze camera 18 photographs the monitor 5 through the half mirror 19 and recognizes the marker M, making it possible to detect the position of the image surface 5a as seen by the main doctor D1. Furthermore, the inner surface of the half mirror 19 takes in reflected light R from the left eye 20 of the doctor D1, and by recognizing the movement of the pupil 20a of the left eye 20, the gaze of the main doctor D1 can be detected. The gaze camera 18 is mounted on the dedicated glasses 15, and the gaze camera 18 photographs and recognizes the movement of the pupil 20a of the main doctor D1 from close up, making it possible to reliably detect the gaze of the main doctor D1.

The position information of the image plane 5a of the monitor 5 detected by the gaze camera 18 and the gaze information of the main doctor D1 are each output as a gaze signal S2 to the observation point detection unit 21. The observation point detection unit 21 can detect the position (coordinate position X, Y) of the observation point P1 of the main doctor D1 on the image plane 5a based on the input gaze signal S2. The position signal S3 of the observation point P1 detected by this observation point detection unit 21 is output to the autofocus unit 11.

A foot switch 22 is provided under the feet of the main doctor D1, and this foot switch 22 is connected to the observation point detection unit 21. Only when the main doctor D1 operates this foot switch 22 with his foot does the observation point detection unit 21 operate and output a position signal S3 to the autofocus unit 11.

Once observation point P1 is identified, the autofocus unit 11 then detects the contrast value of observation point P1 on the image plane 5a, and outputs a control signal S4 to the focusing means 10 to move the movable lens 8 to maximize the contrast value. Since the contrast value of the image plane 5a naturally increases as the image plane 5a becomes more correctly focused, the focus can be adjusted to observation point P1 by moving the movable lens 8 until the maximum contrast at observation point P1 is detected.

As shown in Figure 3, conventionally, the focus was on the center point P of the image plane 5a. Therefore, even if the main doctor D1 tried to change his line of sight to look at an observation point P1 other than the center point P, if the elevation difference of the observation point P1 was different from that of the center point P, the focus would not be on the observation point P1 and clear observation would not be possible.

However, according to this embodiment, even if the observation point P1 is different from the central point P, the autofocus unit 11 moves the movable lens 8 to maximize the contrast value of the observation point P1, so that the observation point P1 is in focus. Therefore, even if the observation point P1 is other than the central point P of the image plane 5a, the main doctor D1 can clearly observe the observation point P1 at the end of his line of sight by simply changing his line of sight without moving the surgical site camera 2.

If the head position of the main doctor D1 remains the same, the focus can be adjusted to the observation point P1 at the line of sight no matter how the line of sight is changed in that state. If the head position of the main doctor D1 changes, the focus can be adjusted to the observation point P1 corresponding to the line of sight at the head position at that time by operating the foot switch 22 again.

2: Surgical site camera 5: Monitor 5a: Image surface 8: Movable lens 9: Objective optical system 10: Focusing means 11: Autofocus section 14: Image sensor 15: Dedicated glasses 18: Line-of-sight camera 19: Half mirror 20: Left eye 20a: Pupil 21: Observation point detection section G: Surgical site K: Optical axis R: Reflected light M: Marker P: Center point P1: Observation point D1: Main doctor D2: Sub-doctor

Claims (2)

An objective optical system including a movable lens for varying a focal length that can be moved freely along an optical axis, a focusing means for moving the movable lens, and a surgical site camera that includes an image pickup device for capturing an image of the surgical site and outputs a captured image signal.
a monitor that receives a video signal output from the surgical site camera, displays an image based on the video signal on an image screen, and is provided with a marker for identifying the image screen as seen by a doctor;
a gaze camera that detects the position of the image plane as seen by the doctor by image recognition of the marker on the monitor, and detects the gaze of the doctor by image recognition of the movement of the pupil of one of the doctor's left or right eye;
an observation point detection unit that detects the position of the monitor's image plane detected by the line of sight camera and the position of the doctor's observation point on the image plane based on the doctor's line of sight;
and an autofocus unit that outputs a signal to the focusing means to move a movable lens in a focusing direction so as to maximize the contrast value of an image at the observation point on the monitor detected by the observation point detection means. The eye camera is installed on either side of the special glasses worn by the doctor.
A half mirror having a concave inner surface capable of reflecting light reflected from the eye closer to the gaze camera and introducing it into the gaze camera and a size allowing the gaze camera to capture an image of the entire monitor is integrally incorporated on the side of the dedicated glasses where the gaze camera is provided, so as not to interfere with the doctor's viewing of the monitor;
2. A surgical observation system according to claim 1, characterized in that the monitor is photographed by an eye camera through a half mirror and the marker is image-recognized to detect the position of the image plane as seen by the doctor, and the doctor's line of sight is detected by taking the light reflected from the doctor's eye into the field of view camera via the inner surface of the half mirror and image-recognizing the movement of the pupil of the eye.

Images