WO2025244127A1 - Fluorescent endoscopic apparatus - Google Patents

Abstract

A processor 9 constituting a fluorescent endoscopic apparatus 1 alternately turns on first and second laser diodes 321, 322, detects differences in brightness level between pixels of a first captured image obtained by capturing one of first and second fluorescent lights with an imaging element 523 at a timing when the first laser diode 321 is turned on and the corresponding pixels of a second captured image obtained by capturing the one fluorescent light with the imaging element 523 at a timing when the second laser diode 322 is turned on, and determines, on the basis of the differences, a region where the one fluorescent light has been emitted.

Description

Fluorescence endoscope device

The present invention relates to a fluorescence endoscope device.

Conventionally, a fluorescent endoscope device is known that irradiates an object of observation (such as a human subject) with visible light, such as excitation light or white light, emitted from a light source device, and observes the fluorescence emitted from a fluorescent reagent contained in the object of observation in response to the irradiation of the excitation light (see, for example, Patent Document 1).

The fluorescence endoscope device described in Patent Document 1 is capable of observing first and second fluorescence in two different wavelength bands corresponding to two types of first and second fluorescent reagents. Specifically, the fluorescence endoscope device includes a first light source that emits visible light (hereinafter referred to as white light), a second light source that emits first excitation light corresponding to the first fluorescent reagent, and a third light source that emits second excitation light corresponding to the second fluorescent reagent. The fluorescence endoscope device also includes first and second image capture elements. The first image capture element is irradiated onto the observation object and captures the white light reflected by the observation object. The second image capture element captures both the first fluorescence emitted from the first fluorescent reagent contained in the observation object when irradiated with the first excitation light, and the second fluorescence emitted from the second fluorescent reagent contained in the observation object when irradiated with the second excitation light.

International Publication No. 2017/047140

A known first fluorescent reagent is one that emits a first fluorescent light having a peak wavelength near 700 nm when irradiated with a first excitation light. A known second fluorescent reagent is one that emits a second fluorescent light having a peak wavelength near 800 nm when irradiated with a second excitation light. The first fluorescent reagent is also sensitive to the second excitation light, and may emit the first fluorescent light even when irradiated with the second excitation light. The second fluorescent reagent is also sensitive to the first excitation light, and may emit the second fluorescent light even when irradiated with the first excitation light. The use of such first and second fluorescent reagents in the fluorescence endoscope device described in Patent Document 1 presents the following problems.
That is, the first and second fluorescence are captured by the same first image capturing element, and therefore, when attempting to simultaneously observe the first and second fluorescence, it is difficult to determine whether a high-luminance region in the captured image obtained by imaging with the first image capturing element is due to the first fluorescence or the second fluorescence.

Here, in order to distinguish between the first and second fluorescent lights, the following optical system and three image pickup elements may be used.
The optical system is an optical system such as a dichroic prism that separates the reflected white light from the object of observation into the first fluorescent light and the second fluorescent light, and the three image capturing elements capture the reflected white light, the first fluorescent light, and the second fluorescent light separated by the optical system.
However, in a configuration using the above-described optical system and three image pickup elements, the optical system becomes large.

Therefore, there is a demand for technology that can avoid increasing the size of the optical system and easily distinguish between the first and second fluorescent lights.

The present invention was made in consideration of the above, and aims to provide a fluorescence endoscope device that avoids an increase in the size of the optical system and makes it easy to distinguish between the first and second fluorescence.

In order to solve the above-mentioned problems and achieve the object, the fluorescence endoscope device of the present invention comprises a first excitation light source that emits first excitation light for exciting a first fluorescent reagent having a peak fluorescence emission wavelength in the vicinity of 700 nm, a second excitation light source that emits second excitation light for exciting a second fluorescent reagent having a peak fluorescence emission wavelength in the vicinity of 800 nm, an image sensor that captures first fluorescence emitted from the first fluorescent reagent contained in an observation object when irradiated with the first excitation light, and second fluorescence emitted from the second fluorescent reagent contained in the observation object when irradiated with the second excitation light, and a processor that controls the operation of the first excitation light source, the second excitation light source, and the image sensor, wherein one of the first excitation light source and the second excitation light source excites one of the first fluorescent reagent and the second fluorescent reagent. the first excitation light includes the third excitation light and the fourth excitation light; and a second laser diode having a wavelength different from the third excitation light and emitting fourth excitation light for exciting the one fluorescent reagent, wherein the first excitation light includes the third excitation light and the fourth excitation light; the processor alternately turns on the first laser diode and the second laser diode, and calculates the difference in brightness levels between corresponding pixels in a first captured image obtained by capturing an image of one of the first and second fluorescent lights using the image sensor when the first laser diode is turned on, and a second captured image obtained by capturing an image of the one fluorescent light using the image sensor when the second laser diode is turned on, and determines the area from which the one fluorescent light is emitted based on the difference.

The fluorescence endoscope device according to the present invention avoids the need for a large optical system and makes it easy to distinguish between the first and second fluorescence.

FIG. 1 is a diagram showing the configuration of a fluorescence endoscope apparatus according to an embodiment. FIG. 2 is a diagram showing the absorption spectrum of the first fluorescent reagent. FIG. 3 is a diagram showing the absorption spectrum of the second fluorescent reagent. FIG. 4 is a diagram illustrating the wavelengths of the second to fourth excitation lights. FIG. 5 is a block diagram showing the configuration of the camera head and the control device. FIG. 6 is a diagram showing the configuration of the imaging unit. FIG. 7 is a diagram showing the operation of the light source device in the second fluorescence observation mode. FIG. 8 is a diagram showing a white image generated in the second fluorescence observation mode. FIG. 9 is a diagram showing a fluorescent image generated in the second fluorescent observation mode. FIG. 10 is a diagram showing the operation of the light source device in the first fluorescence observation mode. FIG. 11 is a diagram illustrating the area determination process executed by the control device. FIG. 12 is a diagram showing a fluorescent image before the region determination process is executed. FIG. 13 is a diagram showing a fluorescent image after region determination processing has been executed. FIG. 14 is a diagram showing the operation of the light source device in the third fluorescence observation mode. FIG. 15 is a diagram showing a fluorescent image after region determination processing has been executed.

Below, a description will be given of a mode for carrying out the present invention (hereinafter referred to as an embodiment) with reference to the drawings. Note that the present invention is not limited to the embodiment described below. Furthermore, in the drawings, the same parts are designated by the same reference numerals.

[Configuration of Fluorescence Endoscope Device]
FIG. 1 is a diagram showing the configuration of a fluorescence endoscope apparatus 1 according to an embodiment.
The fluorescence endoscope device 1 is an endoscope device that uses an endoscope to perform fluorescence observation of an observation target (inside a living body). As shown in Fig. 1, the fluorescence endoscope device 1 includes an insertion section 2, a light source device 3, a light guide 4, a camera head 5, a first transmission cable 6, a display device 7, a second transmission cable 8, a control device 9, and a third transmission cable 10.

In this embodiment, the insertion section 2 is a rigid endoscope. That is, the insertion section 2 has an elongated shape that is either entirely rigid or partially flexible and partially rigid, and is inserted into the object to be observed. Inside this insertion section 2 is an optical system (not shown) that is configured using one or more lenses and focuses the normal light and first and second excitation light reflected by the object to be observed, as well as the first and second fluorescence emitted from the first and second fluorescent reagents contained in the object to be observed. For ease of explanation, the normal light and first and second excitation light reflected by the object to be observed, as well as the first and second fluorescence emitted from the first and second fluorescent reagents contained in the object to be observed, will be referred to below as the subject image.

Furthermore, an excitation light cut filter 22 (Figure 1) is provided at the base end (eyepiece 21) of the insertion section 2 to remove the first and second excitation light contained in the focused subject image. Note that the excitation light cut filter 22 is not limited to being provided in the insertion section 2, and may also be provided within the camera head 5.

One end of a light guide 4 is connected to the light source device 3. The light source device 3 includes first to third light sources 31 to 33, as shown in FIG.
The first light source 31 supplies normal light (hereinafter referred to as white light) including a wavelength band of visible light to one end of the light guide 4 under the control of the control device 9 .
The third light source 33 corresponds to a second excitation light source according to the present invention. The third light source 33 supplies second excitation light to one end of the light guide 4 for exciting a second fluorescent reagent contained in the observation target.
The first and third light sources 31 and 33 may be configured by LEDs (Light Emitting Diodes) or laser diodes.

The second light source 32 corresponds to the first excitation light source according to the present invention. This second light source 32 supplies first excitation light to one end of the light guide 4 for exciting a first fluorescent reagent contained in the observation target. The first excitation light includes third and fourth excitation light beams described below. As shown in FIG. 1 , this second light source 32 includes a first laser diode 321 and a second laser diode 322.
The first laser diode 321 emits third excitation light for exciting the first fluorescent reagent contained in the observation object.
The second laser diode 322 emits a fourth excitation light for exciting the first fluorescent reagent contained in the observation object.

Here, the first fluorescent reagent has the following properties.
2 is a diagram showing the absorption spectrum of the first fluorescent reagent, in which the horizontal axis represents wavelength [nm] and the vertical axis represents absorbance of the absorption spectrum.
The peak wavelength of the absorption spectrum of the first fluorescent reagent is near 680 nm, as shown in Figure 2. Therefore, in order to increase the intensity of the first fluorescence emitted from the first fluorescent reagent, it is preferable to use excitation light having a wavelength near 680 nm as the first excitation light. Furthermore, when excited by the first excitation light, the first fluorescent reagent emits first fluorescence having a wavelength near 700 nm.

The second fluorescent reagent has the following properties:
3 is a diagram showing the absorption spectrum of the second fluorescent reagent, in which the horizontal axis represents wavelength [nm] and the vertical axis represents absorbance of the absorption spectrum.
The peak wavelength of the absorption spectrum of the second fluorescent reagent is near 800 [nm], as shown in Figure 3. Therefore, in order to increase the intensity of the second fluorescence emitted from the second fluorescent reagent, it is preferable to use excitation light having a wavelength near 800 [nm] as the second excitation light. Furthermore, when excited by the second excitation light, the second fluorescent reagent emits second fluorescence having a wavelength near 800 [nm].

Figure 4 is a diagram explaining the wavelengths of the second to fourth excitation light. Specifically, Figure 4 shows the wavelengths of the second to fourth excitation light relative to the absorption spectra of the first and second fluorescent reagents shown in Figures 2 and 3. In Figure 4, the curve indicated by the symbol "CL1" represents the absorption spectrum of the first fluorescent reagent. The curve indicated by the symbol "CL2" represents the absorption spectrum of the second fluorescent reagent. The wavelength indicated by the symbol "P2" represents the wavelength of the second excitation light. The wavelength indicated by the symbol "P3" represents the wavelength of the third excitation light. The wavelength indicated by the symbol "P4" represents the wavelength of the fourth excitation light.

As can be seen from the absorption spectrum of the second fluorescent reagent shown by curve CL2 in Figure 4, the second fluorescent reagent is also sensitive to excitation light having a wavelength near 680 [nm], i.e., the first excitation light, and even when irradiated with the first excitation light, it emits the second fluorescence. For this reason, when observing the first and second fluorescence simultaneously, it is difficult to distinguish between the first and second fluorescence. In this embodiment, as shown in Figure 4, the third excitation light has a wavelength P3 of 675 [nm]. Furthermore, the fourth excitation light has a wavelength P4 of 690 [nm].

Furthermore, the second excitation light has a wavelength P2 of 800 nm. Therefore, as can be seen from the absorption spectrum of the first fluorescent reagent shown by curve CL1 in Figure 4, the first fluorescent reagent is insensitive to the second excitation light, which has a wavelength of 800 nm, and does not emit the first fluorescence even when irradiated with the second excitation light.

In this embodiment, the light source device 3 is configured as a separate unit from the control device 9, but this is not limiting, and the light source device 3 may also be configured to be provided in the same housing as the control device 9.

One end of the light guide 4 is detachably connected to the light source device 3. The other end of the light guide 4 is detachably connected to the insertion section 2. The light guide 4 propagates the white light and first and second excitation light supplied from the light source device 3 (first to third light sources 31 to 33) from one end to the other, supplying them to the insertion section 2. The white light and first and second excitation light supplied to the insertion section 2 are emitted from the tip of the insertion section 2 and irradiated onto the observation object. The white light and first and second excitation light irradiated onto the observation object and reflected by the observation object, as well as the first and second fluorescence (subject images) emitted from the first and second fluorescent reagents contained in the observation object, are each focused by the optical system within the insertion section 2.

The camera head 5 is detachably connected to the proximal end (eyepiece 21 ( FIG. 1 )) of the insertion section 2. The camera head 5 captures an image of a subject after light is collected by the insertion section 2 and the first and second excitation lights have been removed by the excitation light cut filter 22. For ease of explanation, the image signal obtained by capturing an image with the camera head 5 will hereinafter be collectively referred to as a captured image. Furthermore, the subject image after the first and second excitation lights have been removed by the excitation light cut filter 22 will be referred to as an excitation light-removed subject image.
The detailed configuration of the camera head 5 will be described later in the section "Configuration of the Camera Head."

One end of the first transmission cable 6 is detachably connected to the control device 9. The other end of the first transmission cable 6 is detachably connected to the camera head 5. The first transmission cable 6 transmits captured images output from the camera head 5 to the control device 9, and also transmits control signals, synchronization signals, clocks, power, etc. sent from the control device 9 to the camera head 5.

Note that the captured images and other data transmitted from the camera head 5 to the control device 9 via the first transmission cable 6 may be transmitted as optical signals or electrical signals. The same applies to the transmission of control signals, synchronization signals, and clocks from the control device 9 to the camera head 5 via the first transmission cable 6.

The display device 7 is composed of a display using liquid crystal or organic EL (Electro Luminescence) or the like, and displays images based on video signals from the control device 9 under the control of the control device 9.

One end of the second transmission cable 8 is detachably connected to the display device 7. The other end of the second transmission cable 8 is detachably connected to the control device 9. The second transmission cable 8 then transmits the video signal processed by the control device 9 to the display device 7.

The control device 9 includes controllers such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit), and comprehensively controls the operations of the light source device 3, the camera head 5, and the display device 7. Note that the control device 9 is not limited to a CPU or an MPU, and may include an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), or the like.
The detailed configuration of the control device 9 will be explained later in the section "Configuration of the Control Device."

One end of the third transmission cable 10 is detachably connected to the light source device 3. The other end of the third transmission cable 10 is detachably connected to the control device 9. The third transmission cable 10 transmits control signals from the control device 9 to the light source device 3.

[Camera head configuration]
Next, the configuration of the camera head 5 will be described.
FIG. 5 is a block diagram showing the configuration of the camera head 5 and the control device 9.
As shown in FIG. 5 , the camera head 5 includes a lens unit 51 , an imaging unit 52 , and a communication unit 53 .

The lens unit 51 is composed of one or more lenses. The lens unit 51 forms an image of the subject from which excitation light has been removed onto the imaging surfaces of the first and second image sensors 522 and 523, respectively.

FIG. 6 is a diagram showing the configuration of the imaging unit 52.
The imaging unit 52 is a part that generates a captured image by capturing an image of a subject from which excitation light has been removed. As shown in Figures 5 and 6, the imaging unit 52 includes a prism 521, first and second image sensors 522 and 523, and a signal processing unit 524.

The prism 521 has a roughly cubic shape, combining two triangular prism-shaped light-transmitting members 521a (Figure 6). A dichroic filter 521b is provided at the interface between the two light-transmitting members 521a.

The dichroic filter 521b has the property of transmitting light in wavelength bands around 700 nm and 800 nm, and reflecting light in other wavelength bands. Therefore, of the subject image that has been incident on the prism 521 and from which the excitation light has been removed, most of the reflected white light WL (Figure 6) after the first and second excitation lights have been removed by the excitation light cut filter 22 is reflected by the dichroic filter 521b. Then, as shown in Figure 6, the reflected white light WL travels toward the first image sensor 522. On the other hand, of the subject image that has been incident on the prism 521 and from which the excitation light has been removed, most of the first and second fluorescence FL1, FL2 (Figure 6) after the first and second excitation lights have been removed by the excitation light cut filter 22 is transmitted through the dichroic filter 521b (prism 521). The first and second fluorescent lights FL1 and FL2 then travel toward the second image sensor 523.

The first and second image sensors 522, 523 receive incident light and convert it into an electrical signal (analog signal). Examples of these first and second image sensors 522, 523 include a CMOS (Complementary Metal Oxide Semiconductor), which is a rolling shutter type image sensor in which multiple pixels are arranged two-dimensionally in horizontal line units, or a CCD (Charge Coupled Device), which is a global shutter type image sensor.

Then, under the control of the control device 9, the first image sensor 522 captures the light reflected by the dichroic filter 521b. For ease of explanation, the image captured by the first image sensor 522 will be referred to as a white image below.

Here, as shown in Figures 5 and 6, a color filter 522a is provided on the light-receiving surface of the first image sensor 522.

Color filter 522a is a color filter in which three filter groups are arranged in a specific format (e.g., Bayer array) according to the wavelength bands of light (red, green, blue) that they transmit. Specifically, color filter 522a has an R filter group that primarily transmits light in the red wavelength band, a B filter group that primarily transmits light in the blue wavelength band, and a G filter group that primarily transmits light in the green wavelength band.

Furthermore, the second imaging element 523 corresponds to the imaging element according to the present invention. This second imaging element 523 captures light transmitted through the prism 521 under the control of the control device 9. For ease of explanation, the image generated by the second imaging element 523 will be referred to as a fluorescent image below. Note that the light receiving surface of the second imaging element 523 does not have a color filter 522a like the first imaging element 522. In other words, the second imaging element 523 is a so-called monochrome image sensor.

Note that the number of pixels in the white image and the number of pixels in the fluorescent image may be different or the same.

Under the control of the control device 9, the signal processing unit 524 performs signal processing on the captured images (analog signals) generated by the first and second image sensors 522 and 523, and outputs the captured images (digital signals).
For example, the signal processing unit 524 performs signal processing such as removing reset noise from the captured image (analog signal) generated by the first and second image sensors 522 and 523, multiplying the analog signal by an analog gain that amplifies the analog signal (hereinafter referred to as analog gain adjustment), and A/D conversion.

The communication unit 53 functions as a transmitter that transmits the captured images sequentially output from the imaging unit 52 to the control device 9 via the first transmission cable 6.

The communication unit 53 may transmit the white image and the fluorescent image to the control device 9 in sequence, or may transmit the white image and the fluorescent image simultaneously.

[Configuration of the control device]
Next, the configuration of the control device 9 will be described with reference to FIG.
The control device 9 corresponds to a processor according to the present invention. As shown in FIG. 4 , the control device 9 includes a communication unit 91, a processing module 92, a control unit 93, an input unit 94, an output unit 95, and a storage unit 96.

The communication unit 91 functions as a receiver that receives captured images sequentially transmitted from the camera head 5 (communication unit 53) via the first transmission cable 6.

Under the control of the control unit 93, the processing module 92 processes the captured images sequentially transmitted from the camera head 5 (communication unit 53) and received by the communication unit 91. As shown in FIG. 5, this processing module 92 includes an image processing unit 921 and a display control unit 922.

The image processing unit 921 performs image processing on the input captured image (the captured image received by the communication unit 91). Examples of this image processing include optical black subtraction processing (clamping processing), white balance adjustment processing, demosaic processing, color correction matrix processing, gamma correction processing, YC processing that converts RGB signals into luminance color difference signals (Y, Cb/Cr signals), digital gain adjustment that multiplies by digital gain, noise removal, and filter processing that enhances structure.

Note that the image processing performed on the white image and the image processing performed on the fluorescent image may be different image processing, or may be the same image processing.

Under the control of the control unit 93, the display control unit 922 generates a video signal for displaying the captured image after image processing has been performed by the image processing unit 921. The display control unit 922 then outputs the video signal to the display device 7 via the second transmission cable 8.

The control unit 93 is realized by a controller such as a CPU or MPU executing various programs stored in the memory unit 96, and controls the operation of the light source device 3, camera head 5, and display device 7, as well as the operation of the entire control device 9. Note that the control unit 93 is not limited to a CPU or MPU, and may also include an ASIC, FPGA, GPU, etc.

The input unit 94 is configured using operation devices such as a mouse, keyboard, and touch panel, and accepts user operations by a user such as a surgeon. The input unit 94 then outputs an operation signal corresponding to the user operation to the control unit 93.

The output unit 95 is composed of a speaker, printer, etc., and outputs various information.

The memory unit 96 stores programs executed by the control unit 93, information necessary for the control unit 93's processing, etc.

[Operation of the fluorescence endoscope device]
Next, the operation of the above-described fluorescence endoscope device 1 will be described.
In this embodiment, the fluorescence endoscope device 1 is set to one of first to third fluorescence observation modes in response to, for example, a user operation on the input unit 94. The first fluorescence observation mode is a mode in which fluorescence observation is performed using the first fluorescence. The second fluorescence observation mode is a mode in which fluorescence observation is performed using the second fluorescence. The third fluorescence observation mode is a mode in which fluorescence observation is performed using both the first and second fluorescence.
The operation of the fluorescence endoscope device 1 will be described below for each of the first to third fluorescence observation modes, assuming that the observation target contains both the first and second fluorescent reagents.

[Operation of the Fluorescence Endoscope Device in the Second Fluorescence Observation Mode]
7A and 7B are diagrams showing the operation of the light source device 3 in the second fluorescence observation mode. Specifically, Fig. 7A is a time chart showing the operating state of the first light source 31. Fig. 7B is a time chart showing the operating state of the third light source 33. Fig. 7C is a time chart showing the operating state of the second light source 32.
In the second fluorescence observation mode, the control unit 93 controls the operations of the light source device 3 and the first and second image pickup elements 522 and 523 as follows.

As shown in Figure 7(a), the control unit 93 continuously lights up the first light source 31, causing the first light source 31 to constantly emit white light. Note that Figure 7(a) shows a period shorter than the cycle in which dimming control is performed, so the white light is emitted at a constant intensity.

Furthermore, as shown in FIG. 7(b), the control unit 93 continuously lights up the third light source 33, causing the third light source 33 to constantly emit the second excitation light. Note that FIG. 7(b) shows a period shorter than the cycle in which dimming control is performed, so the second excitation light is emitted at a constant intensity.

Furthermore, the control unit 93 keeps the second light source 32 turned off at all times, as shown in Figure 7(c).

Then, the control unit 93 causes the first and second image sensors 522 and 523 to perform imaging operations for each specific frame period.

FIG. 8 is a diagram showing a white image F1 generated in the second fluorescence observation mode.
Specifically, the first image sensor 522 sequentially captures the reflected white light WL for each specific frame period, and sequentially generates white images F1 (FIG. 8).

Fig. 9 is a diagram showing a fluorescent image F2 generated in the second fluorescent observation mode. Note that Fig. 9 is an image obtained by capturing the same subject as the white image F1 shown in Fig. 8, and the area Ar1 indicated by the solid line is an area where the intensity of the second fluorescent light FL2 is high. In Fig. 9, areas other than the area Ar1 are indicated by dashed lines.
Furthermore, the second image sensor 523 sequentially captures images of the second fluorescence FL2 for each specific frame period, and sequentially generates fluorescence images F2 ( FIG. 9 ). Here, the wavelength P2 of the second excitation light is 800 nm. Furthermore, as shown by the curve CL1 in FIG. 4 , the first fluorescent reagent is insensitive to the second excitation light, and does not emit the first fluorescence FL1 even when irradiated with the second excitation light. Therefore, the region Ar1 in the fluorescence image F2 shown in FIG. 9 is a region emitted only by the second fluorescence FL2.

Under the control of the control unit 93, the image processing unit 921 performs image processing on the white image F1 and the fluorescent image F2 received by the communication unit 91. The image processing unit 921 also generates a superimposed image by, for example, superimposing the white image F1 and the fluorescent image F2 using known alpha blending or additive blending. The display control unit 922 then generates a video signal corresponding to the superimposed image and outputs it to the display device 7. As a result, the superimposed image is displayed on the display device 7.

[Operation of the Fluorescence Endoscope Device in the First Fluorescence Observation Mode]
10A and 10B are diagrams showing the operation of the light source device 3 in the first fluorescence observation mode. Specifically, (a) of Fig. 10 is a time chart showing the operating state of the first light source 31. (b) of Fig. 10 is a time chart showing the operating state of the third light source 33. (c) of Fig. 10 is a time chart showing the operating state of the second light source 32.
In the first fluorescence observation mode, the control unit 93 controls the operations of the light source device 3 and the first and second image pickup elements 522 and 523 as follows.

As shown in Figure 10(a), the control unit 93 continuously lights up the first light source 31, causing the first light source 31 to constantly emit white light. Note that Figure 10(a) shows a period shorter than the cycle in which dimming control is performed, so the white light is emitted at a constant intensity.

Furthermore, the control unit 93 keeps the third light source 33 turned off at all times, as shown in Figure 10 (b).

Furthermore, as shown in FIG. 10(c), the control unit 93 alternately lights up the first and second laser diodes 321, 322 for each specific frame period, and alternately emits the third and fourth excitation light beams. Here, in FIG. 10(c), the timing at which the first laser diode 321 lights up is timing T1. Also, the timing at which the second laser diode 322 lights up is timing T2. Note that FIG. 10(c) shows a period shorter than the cycle at which dimming control is performed, and therefore the third and fourth excitation light beams are each emitted at a constant intensity.

Then, the control unit 93 causes the first and second image sensors 522 and 523 to perform imaging operations for each specific frame period.

The first image sensor 522 sequentially captures the reflected white light WL for each specific frame period and sequentially generates a white image F1 (Figure 8).

11 is a diagram illustrating the region determination process executed by the control device 9. Specifically, (a) of FIG. 11 is an enlarged view of a portion of FIG. 4. (b) of FIG. 11 is a diagram illustrating the intensities of the first fluorescence FL1 emitted from the first fluorescent reagent by the third and fourth excitation lights. (c) of FIG. 11 is a diagram illustrating the intensities of the second fluorescence FL2 emitted from the second fluorescent reagent by the third and fourth excitation lights. FIG. 12 is a diagram illustrating a fluorescence image F2A before the region determination process is executed. Note that FIG. 12 is an image obtained by capturing the same subject as the white image F1 shown in FIG. 8, and the region Ar2 indicated by the solid line is a region where the fluorescence intensity of at least one of the first and second fluorescence FL1 and FL2 is high. Furthermore, in FIG. 12, regions other than the region Ar2 are indicated by dashed lines. FIG. 13 is a diagram corresponding to FIG. 12 and illustrates a fluorescence image F2B after the region determination process has been executed.

Furthermore, the second image sensor 523 sequentially captures the first and second fluorescence FL1 and FL2 for each specific frame period, and sequentially generates fluorescence images F2A (Figure 12). Here, the wavelength P3 of the third excitation light is 675 [nm]. The wavelength P4 of the fourth excitation light is 690 [nm]. The second fluorescent reagent is also sensitive to the third and fourth excitation lights, as shown by curve CL2 in Figure 11(a), and emits the second fluorescence FL2 even when irradiated with the third and fourth excitation lights. Therefore, it is difficult to determine whether area Ar2 in the fluorescence image F2A shown in Figure 12 is emitted by the first or second fluorescence FL1 or FL2. The control device 9 then executes a region determination process to determine whether the region Ar2 is emitted by the first or second fluorescence FL1, FL2, as shown below.

Under the control of the control unit 93, the image processing unit 921 performs image processing on the white image F1 and the fluorescent image F2A received by the communication unit 91. Under the control of the control unit 93, the image processing unit 921 also calculates the difference in brightness levels between corresponding pixels in the fluorescent image F2A (first captured image) generated by the second image sensor 523 at timing T1 (FIG. 10(c)) when the first laser diode 321 is turned on, and the fluorescent image F2A (second captured image) generated by the second image sensor 523 at timing T2 (FIG. 10(c)) when the second laser diode 322 is turned on. Here, the brightness level can be, for example, a brightness value or a pixel value. The image processing unit 921 then determines the region from which the first fluorescent light FL1 is emitted based on the magnitude of the difference. Figure 13 illustrates an example where region Ar21 of region Ar2 is determined to be the region from which the first fluorescent light FL1 is emitted.

Specifically, as can be seen by comparing Figure 11(b) and Figure 11(c), there is a difference between the difference D1 in intensity of the first fluorescence FL1 emitted from the first fluorescent reagent by the third and fourth excitation lights, and the difference D2 in intensity of the second fluorescence FL2 emitted from the second fluorescent reagent by the third and fourth excitation lights. The image processing unit 921 then uses the difference between these differences D1 and D2 to determine that area Ar21 is an area from which the first fluorescence FL1 is emitted, based on the magnitude of the difference described above.

Furthermore, the image processing unit 921 determines that area Ar22, other than area Ar21, of area Ar2 is the area from which the second fluorescence FL2 is emitted. The image processing unit 921 then generates fluorescence image F2B ( FIG. 13 ) by lowering the brightness value of area Ar22 in fluorescence image F2A. Note that, for ease of explanation, the lowered brightness value of area Ar22 is indicated by a dashed line in FIG. 13 .

The image processing unit 921 then superimposes the white image F1 and the fluorescent image F2B, for example, using known alpha blending or additive blending, to generate a superimposed image. The display control unit 922 then generates a video signal corresponding to the superimposed image and outputs it to the display device 7. As a result, the superimposed image is displayed on the display device 7.

[Operation of the Fluorescence Endoscope Device in the Third Fluorescence Observation Mode]
14A and 14B are diagrams showing the operation of the light source device 3 in the third fluorescence observation mode. Specifically, (a) of Fig. 14 is a time chart showing the operating state of the first light source 31. (b) of Fig. 14 is a time chart showing the operating state of the third light source 33. (c) of Fig. 14 is a time chart showing the operating state of the second light source 32.
In the third fluorescence observation mode, the control unit 93 controls the operations of the light source device 3 and the first and second image pickup elements 522 and 523 as follows.

As shown in Figure 14(a), the control unit 93 continuously lights up the first light source 31, causing the first light source 31 to constantly emit white light. Note that Figure 14(a) shows a period shorter than the cycle in which dimming control is performed, so the white light is emitted at a constant intensity.

Furthermore, as shown in Figures 14(b) and 14(c), the control unit 93 turns on the third light source 33 and the first and second laser diodes 321 and 322 in sequence for each specific frame period, causing the second to fourth excitation light beams to be emitted in sequence. Here, in Figure 14(c), the timing at which the first laser diode 321 turns on is timing T1. Also, the timing at which the second laser diode 322 turns on is timing T2. Note that Figures 14(b) and 14(c) show cycles that are shorter than the cycle at which dimming control is performed, so the second to fourth excitation light beams are each emitted at a constant intensity.

Then, the control unit 93 causes the first and second image sensors 522 and 523 to perform imaging operations for each specific frame period.

The first image sensor 522 sequentially captures the reflected white light WL for each specific frame period and sequentially generates a white image F1 (Figure 8).

FIG. 15 corresponds to FIGS. 12 and 13 and shows a fluorescent image F2B after the region determination process has been executed.
The second image sensor 523 sequentially captures the first and second fluorescence FL1 and FL2 for each specific frame period, and sequentially generates a fluorescence image F2A (FIG. 12).

Under the control of the control unit 93, the image processing unit 921 performs image processing on the white image F1 and the fluorescent image F2A received by the communication unit 91. Under the control of the control unit 93, the image processing unit 921 also performs area determination processing, similar to the first fluorescence observation mode. Specifically, the image processing unit 921 calculates the difference in brightness levels between corresponding pixels in the fluorescent image F2A (first captured image) generated by the second image sensor 523 at timing T1 ((c) in FIG. 14) when the first laser diode 321 is turned on, and the fluorescent image F2A (second captured image) generated by the second image sensor 523 at timing T2 ((c) in FIG. 14) when the second laser diode 321 is turned on. Here, the brightness level can be, for example, a brightness value or a pixel value. The image processing unit 921 then determines the area from which the first fluorescent light FL1 and the area from which the second fluorescent light FL2 are emitted, based on the magnitude of the difference. Figure 15 illustrates an example in which area Ar21 of area Ar2 is determined to be the area from which the first fluorescence FL1 is emitted, and area Ar22 is determined to be the area from which the second fluorescence FL2 is emitted. The image processing unit 921 also generates a fluorescence image F2B (Figure 15) by distinguishing areas Ar21 and Ar22 in the fluorescence image F2A, for example, by using different colors.

The image processing unit 921 then superimposes the white image F1 and the fluorescent image F2B, for example, using known alpha blending or additive blending, to generate a superimposed image. The display control unit 922 then generates a video signal corresponding to the superimposed image and outputs it to the display device 7. As a result, the superimposed image is displayed on the display device 7.

According to the present embodiment described above, the following effects are achieved.
In the fluorescence endoscope device 1 according to this embodiment, the control device 9 alternately lights up the first and second laser diodes 321 and 322. The control device 9 calculates the difference in brightness levels between corresponding pixels in a fluorescence image F2A generated by the second image sensor 523 at time T1 when the first laser diode 321 is turned on and a fluorescence image F2A generated by the second image sensor 523 at time T2 when the second laser diode 322 is turned on. The control device 9 then uses the difference D1 in intensity between the first fluorescence FL1 emitted from the first fluorescent reagent by the third and fourth excitation lights and the difference D2 in intensity between the second fluorescence FL2 emitted from the second fluorescent reagent by the third and fourth excitation lights to determine the region from which the first fluorescence FL1 is emitted based on the magnitude of this difference. This facilitates distinguishing between the first and second fluorescence FL1 and FL2.
Furthermore, the optical system (prism 521 including dichroic filter 521b) that separates the reflected white light WL from the first and second fluorescent lights FL1 and FL2 can also be configured compactly.
As described above, the fluorescence endoscope device 1 according to this embodiment can easily distinguish between the first and second fluorescence FL1 and FL2 while miniaturizing the optical system.

(Other embodiments)
Although the embodiments for carrying out the present invention have been described above, the present invention should not be limited to only the above-described embodiments.
In the above-described embodiment, one of the wavelengths of the third and fourth excitation light may be set to the same wavelength as the peak wavelength in the absorption spectrum of the first fluorescent reagent. In this case, the difference between the differences D1 and D2 is made clear, and the region from which the first fluorescent light FL1 is emitted can be determined with high accuracy.

The fluorescence image F2B may be a fluorescence image whose contrast has been increased by performing the above-described region determination process, as well as HDR (high dynamic range) described below.
Fluorescence image F2B is a fluorescence image (third captured image) in which contrast has been enhanced by HDR, a technique for expressing a range from dark to bright areas by combining a bright image and a dark image. The bright image is fluorescence image F2A (second captured image) generated by the second image sensor 523 at timing T2 ((c) in FIG. 10 ) when the second laser diode 321 is turned on. The dark image is fluorescence image F2A (first captured image) generated by the second image sensor 523 at timing T1 ((c) in FIG. 10 ) when the first laser diode 321 is turned on.

In the above-described embodiment, one of the excitation light sources according to the present invention is the second light source 32, but this is not limited to this and it may also be the third light source 33.

REFERENCE SIGNS LIST 1 Fluorescence endoscope device 2 Insertion section 3 Light source device 4 Light guide 5 Camera head 6 First transmission cable 7 Display device 8 Second transmission cable 9 Control device 10 Third transmission cable 21 Eyepiece section 22 Excitation light cut filter 31 First light source 32 Second light source 33 Third light source 51 Lens unit 52 Imaging section 53 Communication section 91 Communication section 92 Processing module 93 Control section 94 Input section 95 Output section 96 Storage section 321 First laser diode 322 Second laser diode 521 Prism 521a Light-transmitting member 521b Dichroic filter 522 First imaging element 522a Color filter 523 Second imaging element 524 Signal processing section 921 Image processing section 922 Display control section Ar1, Ar2, Ar21, Ar22 Areas CL1, CL2 Curves D1, D2 Difference F1 White image F2, F2A, F2B Fluorescence images FL1 First fluorescence FL2 Second fluorescence P2 to P4 Wavelengths T1, T2 Timing WL Reflected light of white light

Claims (5)

a first excitation light source that emits first excitation light for exciting a first fluorescent reagent having a peak fluorescence emission wavelength in the vicinity of 700 nm;
a second excitation light source that emits second excitation light for exciting a second fluorescent reagent having a fluorescence emission peak wavelength in the vicinity of 800 nm;
an image capturing element configured to capture an image of a first fluorescence emitted from the first fluorescent reagent contained in the observation object in response to irradiation with the first excitation light and a second fluorescence emitted from the second fluorescent reagent contained in the observation object in response to irradiation with the second excitation light;
a processor that controls operations of the first excitation light source, the second excitation light source, and the image sensor;
One of the first excitation light source and the second excitation light source is
a first laser diode that emits third excitation light for exciting one of the first fluorescent reagent and the second fluorescent reagent;
a second laser diode that emits fourth excitation light having a wavelength different from that of the third excitation light for exciting the one fluorescent reagent;
The first excitation light is
the third excitation light and the fourth excitation light,
The processor:
The first laser diode and the second laser diode are alternately turned on;
a fluorescence endoscope device that calculates a difference in brightness levels between corresponding pixels in a first captured image obtained by causing the image sensor to capture an image of one of the first fluorescent light and the second fluorescent light at a timing when the first laser diode is turned on, and a second captured image obtained by causing the image sensor to capture an image of the one fluorescent light at a timing when the second laser diode is turned on, and determines an area from which the one fluorescent light is emitted based on the difference. The processor:
2. The fluorescence endoscope apparatus according to claim 1, wherein the region from which said one of the fluorescence beams is emitted is determined based on the magnitude of said difference. The processor:
2. The fluorescence endoscope apparatus according to claim 1, wherein a third captured image with enhanced contrast is generated based on the first captured image and the second captured image. The one excitation light source is
2. The fluorescence endoscope apparatus according to claim 1, wherein the first excitation light source is a fluorescent light source. One of the third excitation light and the fourth excitation light is
2. A fluorescence endoscope apparatus according to claim 1, wherein the wavelength is the same as the peak wavelength in the absorption spectrum of said one of the fluorescent reagents.