CN117442144A

CN117442144A - Endoscope system and imaging method thereof

Info

Publication number: CN117442144A
Application number: CN202210854858.9A
Authority: CN
Inventors: 请求不公布姓名
Original assignee: Shenzhen Konuositeng Technology Co ltd
Current assignee: Shenzhen Konuositeng Technology Co ltd
Priority date: 2022-07-18
Filing date: 2022-07-18
Publication date: 2024-01-26

Abstract

The invention discloses an endoscope system and an imaging method thereof, wherein the endoscope system comprises a light source device, an image sensor and an image processing device, wherein the light source device is configured to enable a visible light source and an excitation light source to respectively emit monochromatic visible light and excitation light under a fluorescence imaging mode so as to synchronously irradiate an object to be observed, and the excitation light is used for exciting the object to be observed to emit fluorescence; the image sensor includes at least two channels with different wavelength range sensing capabilities arranged side by side, one of the at least two channels for receiving monochromatic visible light and at least one other for receiving fluorescence; the image processing device is used for processing the signal to generate image data. According to the invention, in the fluorescent imaging mode, the imaging mode of a plurality of light sources and a single light path is adopted, so that dislocation and double images of an output image can be avoided, and the time delay and smear of image display are reduced, thereby reducing the operation fatigue and operation risk of doctors.

Description

Endoscope system and imaging method thereof

Technical Field

The invention relates to the technical field of medical instruments, in particular to an endoscope system and an imaging method thereof.

Background

Minimally invasive surgery has the advantages of small damage to patients, less pain of patients between operations, short postoperative recovery time and the like, is widely applied, and has become the development direction of various fields of surgical medicine. Endoscopes are one of the important surgical instruments in minimally invasive surgery, and are capable of providing information of an affected area (e.g., tumor tissue) in real time during the surgery, and a surgeon performs corresponding treatment (e.g., resection) accordingly. In the operation process, the affected area is fluorescently marked, the fluorescence image reflects the information of the size, the outline and the like of the affected area, and the visible light image reflects the information of the tissue background, the scene position and the like. The two feature information are fused, the fluorescence image and the visible light image of the affected area are displayed in real time, the advantages of the fluorescence image and the visible light image can be complemented, a good operation visual field is provided for a doctor, the doctor is helped to judge the features such as the shape of the affected area, and therefore the purpose of distinguishing normal and healthy tissues is achieved, and operation efficiency is greatly improved.

Disclosure of Invention

In the summary, a series of concepts in a simplified form are introduced, which will be further described in detail in the detailed description. The summary of the invention is not intended to define the key features and essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

To at least partially solve the above problems, the present invention provides an endoscope system including:

the light source device comprises a visible light source and an excitation light source, and is configured to enable the visible light source and the excitation light source to respectively emit monochromatic visible light and excitation light in a fluorescence imaging mode so as to synchronously irradiate an object to be observed, wherein the excitation light is used for exciting the object to be observed to emit fluorescence;

an image sensor configured to convert received light into a corresponding signal, the image sensor comprising at least two channels with different wavelength range sensing capabilities arranged side by side, one of the at least two channels for receiving the monochromatic visible light and at least one other for receiving the fluorescence light; and

image processing means for processing the signals to generate image data.

Optionally, the at least two channels comprise at least two of a red signal channel, a green signal channel, and a blue signal channel.

Optionally, the image processing device is configured to generate background image data from the signal converted by the monochromatic visible light in a fluorescence imaging mode, generate fluorescence image data from the signal converted by the fluorescence, and combine the background image data and the fluorescence image data to output a composite image frame.

Optionally, the image processing apparatus is further configured to estimate a signal expected to be received by a channel other than the channel for receiving the monochromatic visible light in a scene illuminated with light of a wavelength range corresponding to its sensing capability, and to generate background image data from the monochromatic visible light-converted signal and the predicted signal.

Optionally, the light source device is configured such that the monochromatic visible light and the excitation light irradiate the object to be observed with coincident light paths.

Optionally, the light source device is further configured to turn off an excitation light source in a white light imaging mode, and make the visible light source emit white light to irradiate the object to be observed, and the at least two channels respectively receive light in a wavelength range corresponding to a sensing capability thereof in the white light.

Optionally, the at least two channels include a red signal channel, a green signal channel, and a blue signal channel.

Optionally, the visible light source includes a white light source, the light source device further includes a spectroscope for reflecting the excitation light and a filter for filtering the white light emitted from the white light source to form the monochromatic visible light, the filter is movable between a monochromatic light passing position for shielding an optical path of the white light and a white light passing position for avoiding the optical path of the white light, and the light source device is configured such that the optical path of the excitation light reflected by the spectroscope coincides with the optical path of the white light after passing through the filter.

Optionally, the visible light source includes a monochromatic light source and a white light source, the light source device further includes a first spectroscope for reflecting monochromatic visible light emitted by the monochromatic light source and a second spectroscope for reflecting the excitation light, the light source device is configured such that an optical path of the monochromatic visible light reflected by the first spectroscope coincides with an optical path of the excitation light reflected by the second spectroscope, and the white light source is juxtaposed with the first spectroscope and the second spectroscope.

Optionally, a control device is further included, the control device being configured to control the endoscope system to switch between the fluorescence imaging mode and the white light imaging mode, and to control the light source device to turn on the visible light source and the excitation light source in the fluorescence imaging mode and to turn off the excitation light source in the white light imaging mode.

Optionally, the monochromatic visible light comprises blue light having a wavelength peak of about 450nm, the excitation light comprises near infrared light having a wavelength peak of about 780nm or about 808nm, the fluorescence comprises fluorescence having a wavelength peak of 830nm, the at least two channels comprise a red signal channel, a green signal channel, and a blue signal channel, the blue signal channel for receiving the blue light, and the red signal channel and/or the green signal channel for receiving the fluorescence.

According to another aspect of the present invention, there is provided an imaging method of an endoscope system, including the steps of:

receiving mixed light from an object to be observed, and converting the mixed light into corresponding signals, wherein the mixed light comprises monochromatic visible light and fluorescence, the mixed light is synchronously received by a first channel and a second channel which are arranged in parallel and have sensing capabilities of different wavelength ranges, and the wavelength of the monochromatic visible light is in a wavelength range corresponding to the sensing capability of the first channel; and

Background image data is generated from the signal converted from the monochromatic visible light, fluorescence image data is generated from the signal converted from the fluorescence, and the background image data and the fluorescence image data are combined to output a composite image frame.

Optionally, the first channel includes one of a red signal channel, a green signal channel, and a blue signal channel, and the second channel includes another one or two of the red signal channel, the green signal channel, and the blue signal channel.

Optionally, the method further comprises: and generating the monochromatic visible light and excitation light and synchronously irradiating the object to be observed, wherein the excitation light excites the object to be observed to emit the fluorescence.

Optionally, generating the background image data from the signal converted by the monochromatic visible light includes: estimating signals expected to be received by channels other than the channel for receiving the monochromatic visible light in a scene irradiated by light of a wavelength range corresponding to the sensing capability of the channels according to the signals converted by the monochromatic visible light; and generating background image data from the monochromatic visible light converted signal and the predicted signal.

Alternatively, the monochromatic visible light comprises blue light having a wavelength peak of about 450nm, the excitation light comprises near infrared light having a wavelength peak of about 780nm or about 808nm, and the fluorescence comprises fluorescence having a wavelength peak of about 830 nm; and the first channel comprises a blue signal channel and the second channel comprises a red signal channel and/or a green signal channel.

Optionally, the mixed light further includes excitation light that excites the object to be observed to emit the fluorescence, the mixed light is synchronously received by the first channel, the second channel, and the third channel which are arranged side by side and have different wavelength range sensing capabilities, wherein generating fluorescent image data according to the signal converted by the fluorescence includes: generating fluorescence image data from the signal converted from the fluorescence; and correcting the fluorescence image data according to the signal converted by the excitation light, resulting in corrected fluorescence image data.

Optionally, the first channel, the second channel, and the third channel each include one of a red signal channel, a green signal channel, and a blue signal channel.

According to the endoscope system and the imaging method thereof, in the fluorescent imaging mode, the imaging mode of a plurality of light sources and a single light path is adopted, and the visible light and fluorescence generated by the excitation light are sensed by the same image sensor, so that the problem that the visible light imaging and the fluorescence imaging are difficult to align is effectively solved, and dislocation and double image of an output image are avoided; and the visible light source and the excitation light source irradiate the object to be detected synchronously, the image sensor can synchronously receive visible light and fluorescence for imaging, the problem of frame rate reduction caused by pulse irradiation is avoided, and the delay and smear of image display are reduced. Therefore, the invention can reduce the operation fatigue degree and operation risk of doctors. In addition, as no additional sensor is needed for fluorescence imaging, the size of the handle end of the endoscope is miniaturized, and the design and manufacturing cost is reduced.

Drawings

The following drawings are included to provide an understanding of the invention and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the invention and their description to explain the principles of the invention.

In the accompanying drawings:

FIG. 1 is a schematic view of an endoscope system according to a preferred embodiment of the present invention;

FIG. 2 is a schematic diagram of the camera module shown in FIG. 1;

FIG. 3 is a schematic diagram of an imaging principle of the endoscope system shown in FIG. 1 in a white light imaging mode;

FIG. 4 is a schematic view of the imaging principles of the endoscope system shown in FIG. 1 in a fluoroscopic imaging mode;

FIG. 5 is a graph of quantum efficiency of a signal path of the image sensor shown in FIG. 3;

fig. 6 is a schematic structural view of an example of the light source device shown in fig. 1;

fig. 7 is a schematic structural view of another example of the light source device shown in fig. 1;

FIG. 8 is a spectrum diagram of the first spectroscope shown in FIG. 7;

FIG. 9 is a spectral diagram of the second beam splitter shown in FIG. 7;

FIG. 10 is a flow chart of an imaging method according to one embodiment of the invention;

fig. 11 is a flowchart of an imaging method according to another embodiment of the present invention.

Reference numerals illustrate:

10-to-be-observed object 110 light source device

111 white light source 112 monochromatic light source

113 excitation light source 114 first spectroscope

115 second beam splitter 116 filter

120 camera module 121 image sensor

122 lens 123 optical fiber

124 red signal channel 125 green signal channel

126 blue signal channel 130 image processing device

101 visible light source

Detailed Description

In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the invention may be practiced without one or more of these details. In other instances, well-known features have not been described in detail in order to avoid obscuring the invention.

In the following description, a detailed description will be given for the purpose of thoroughly understanding the present invention. It will be apparent that embodiments of the invention may be practiced without limitation to the specific details that are familiar to those skilled in the art. Preferred embodiments of the present invention are described in detail below, however, the present invention may have other embodiments in addition to these detailed descriptions.

It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the present invention. As used herein, the singular is intended to include the plural unless the context clearly indicates otherwise. Furthermore, it will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

Ordinal numbers such as "first" and "second" cited in the present invention are merely identifiers and do not have any other meaning, such as a particular order or the like. Also, for example, the term "first component" does not itself connote the presence of "second component" and the term "second component" does not itself connote the presence of "first component".

It should be noted that the terms "upper", "lower", "front", "rear", "left", "right", "inner", "outer", and the like are used herein for illustrative purposes only and are not limiting.

The current imaging schemes for fusing fluorescent images and visible light images in an endoscope system can be categorized into two types.

The method is a multi-light source multi-light path scheme, wherein visible light imaging and fluorescence imaging are acquired by independent sensors, and then the acquired visible light images and fluorescence images are fused or overlapped. The disadvantage of this scheme is that the quantity of sensor increases, leads to endoscope handle end size too big, is unfavorable for miniaturization, and increases manufacturing cost, exists visible light formation of image and fluorescence formation of image simultaneously and has the light path problem that is difficult to align, leads to dislocation, ghost image problem can appear when visible light image overlaps with the fluorescence image.

The other type is a multi-light source single-light path scheme, wherein the visible light imaging and the fluorescence imaging share one sensor, and the visible light image and the fluorescence image are acquired in a time-sharing manner by controlling the on-off period of the visible light source and the excitation light source for generating fluorescence. The disadvantage of this solution is the large delay, which is particularly easy to cause the desynchronization of the left and right images when applied to the endoscope for 3D imaging, and the images are displayed with smear.

The above schemes all increase visual fatigue and operation risks of doctors during operation to different degrees.

Exemplary embodiments according to the present invention will now be described in more detail with reference to the accompanying drawings. These exemplary embodiments may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. It should be appreciated that these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of these exemplary embodiments to those skilled in the art.

The embodiment of the invention provides an endoscope system which can display images of visible light imaging and fluorescence imaging superposition in a fluorescence imaging mode, enhance the display effect of a surgical scene and provide surgical guidance for doctors. The fluorescent imaging may be a display effect of highlighting the fluorescent portion in other colors (such as green) in a black-and-white background, or a display effect of highlighting the fluorescent portion in other colors (such as green) in a color background.

In some embodiments, the endoscope system is further provided with a white light imaging mode. The endoscope system is switchable between a white light imaging mode and a fluorescence imaging mode. White light imaging may be a color display effect.

The technical scheme of the embodiment of the invention can be applied to an endoscope system for two-dimensional imaging and an endoscope system for three-dimensional imaging.

For convenience of explanation, the endoscope system of the present invention will be described below in connection with the embodiment shown in fig. 1 to 9, in which the endoscope system has a switchable white light imaging mode and a fluorescent imaging mode.

As shown in fig. 1 and 2, the endoscope system includes a light source device 110, an image capturing module 120, and an image processing device 130, which can be connected to each other by signals.

The light source device 110 may include a visible light source 101 and an excitation light source 113. The visible light source 101 may selectively emit white light and monochromatic visible light. The excitation light source 113 may emit excitation light. In the white light imaging mode, the visible light source 101 is turned on and the visible light source 101 is made to emit white light, and the excitation light source 113 is in an off state. In the fluorescence imaging mode, the visible light source 101 is turned on and the visible light source 101 is caused to emit monochromatic visible light, while the excitation light source 113 is turned on to emit excitation light. The light emitted from the light source device 110 is used to irradiate the object to be observed 10. The object to be observed 10 generally comprises a cell, tissue, organ or body part of a human body. In particular cells, tissues, organs or body parts in diseased areas of the human body. In other embodiments, the endoscope system may not be set to a white light imaging mode, where the visible light source 101 emits only monochromatic visible light.

The monochromatic visible light may be visible light in any suitable wavelength range, such as blue, red, green, etc. The excitation light may be any suitable light for exciting a fluorescent agent to fluoresce, such as infrared light, preferably near infrared light. As one illustrative example, monochromatic visible light may include blue light having a wavelength in the range of 425-475 nm. Alternatively, the wavelength peak of the blue light is about 450nm, and may be 440nm, 445nm, 455nm, or 460nm. As an illustrative example, the excitation light may include near infrared light having a wavelength in the range of 760 to 810 nm. Alternatively, the near infrared light may have a wavelength peak of about 780nm or about 808nm, and may be 770nm, 775nm, 785nm, 790nm, or 805nm. As an illustrative example, the fluorescence may include fluorescence in a wavelength range between 810 and 850 nm. Alternatively, the fluorescence may have a wavelength peak of about 830nm, but may be 820nm, 825nm, 835nm, or 840nm.

The object to be observed 10 contains a fluorescent agent. The excitation light is used to excite the object to be observed 10 to fluoresce. The fluorescent agent may be any fluorescent agent suitable for use in humans, and as an example, the fluorescent agent may include indocyanine green (ICG). Indocyanine green has the characteristic of emitting near infrared fluorescence of about 830nm under near infrared light irradiation of about 780 nm.

The light source device 110 is configured to cause the visible light source 101 and the excitation light source 113 to emit monochromatic visible light and excitation light, respectively, to simultaneously irradiate the object to be observed 10 in the fluorescence imaging mode. In this scenario, the visible light source 101 and the excitation light source 113 are on simultaneously, the visible light source 101 providing background illumination, and the excitation light source 113 providing fluorescence excitation energy.

The light source device 110 is also configured to turn off the excitation light source 113 and cause the visible light source 101 to emit white light to irradiate the object to be observed 10 in the white light imaging mode. In this solution, only the visible light source 101 is turned on.

The camera module 120 is used for capturing images, and may include an image sensor 121 and a lens 122. And an optical fiber 123 is disposed around the image sensor 121. The light emitted from the light source device 110 can be conducted and irradiated to the object to be observed 10 via the optical fiber 123. The light is reflected by the object to be observed 10, and then captured by the lens 122, thereby entering the image sensor 121.

The endoscope system is provided with an image sensor 121 for receiving light reflected via the object to be observed 10. In both the white light imaging mode and the fluorescent imaging mode, light reflected via the object to be observed 10 is received by the same image sensor 121.

The image sensor 121 is configured to convert the received light into a corresponding signal. The image sensor 121 may include at least two channels with different wavelength range sensing capabilities arranged side by side. In the fluorescence imaging mode, one of the at least two channels is for receiving monochromatic visible light and at least one other is for receiving fluorescence. Thus, the visible light and the fluorescence can be synchronously sensed and imaged by the same image sensor 121, a sensor for receiving the fluorescence does not need to be additionally arranged, a pulse-type irradiation light source does not need to be arranged, the problem that the visible light imaging and the fluorescence imaging are difficult to align is solved, and the problem of frame rate reduction caused by the pulse-type irradiation is avoided.

As one illustrative example, as shown in fig. 3 and 4, the at least two channels may include a red signal channel 124, a green signal channel 125, and a blue signal channel 126. In this example, the monochromatic visible light is blue light, so that blue signal channel 126 is used to receive blue light, and red signal channel 124 and/or green signal channel 125 are used to receive fluorescence. It will be appreciated that the correspondence of light to channels may vary depending on the wavelength of monochromatic visible light. For example, in other examples, the monochromatic visible light may be red or green, with the corresponding channel being for receiving monochromatic visible light and the other channels being for receiving fluorescence. As another illustrative example, the image sensor 121 may include two channels, one for receiving monochromatic visible light and the other for receiving fluorescence. In this example, the wavelength of the monochromatic visible light for illuminating the object to be observed falls within a wavelength range corresponding to the sensing capability to one of the channels. For example, the two channels may include any two of a red signal channel 124, a green signal channel 125, and a blue signal channel 126. In one example, the two channels include a red signal channel 124 and a blue signal channel 126, and the monochromatic visible light may be red or blue light. In one example, the two channels include a red signal channel 124 for receiving blue light and a blue signal channel 126 for receiving fluorescent light, where the monochromatic visible light is blue light. It is understood that the monochromatic visible light may be changed according to the kind of the channel, and the correspondence of the light and the channel may be changed according to the wavelength of the monochromatic visible light.

In the present embodiment, since the endoscope system further includes a white light imaging mode, the image sensor 121 preferably employs an image sensor including a red signal channel 124, a green signal channel 125, and a blue signal channel 126. In the white light imaging mode, all three channels can receive visible light in a corresponding wavelength range and generate corresponding signals.

As one illustrative example, the image sensor 121 may include, but is not limited to, a charge coupled device (Charge Coupled Device, CCD for short) or a complementary metal oxide semiconductor (Complementary MetalOxide Semiconductor, CMOS for short) image sensor, or the like.

The image processing device 130 is used to process the signal from the image sensor 121 to generate image data. And the image data may be further output to a display device for display.

As one illustrative example, when the endoscope system is switched to the white light imaging mode, the image sensor 121 converts the light components respectively received according to the respective channels into corresponding signals and transmits the signals to the image processing apparatus 130, and the image processing apparatus 130 generates image frames according to the received signals. For example, in the example shown in fig. 3, the image sensor 121 converts red light components, green light components, and blue light components in white light received by the red signal channel 124, the green signal channel 125, and the blue signal channel 126, respectively, into corresponding signals, and the image processing apparatus 130 is capable of generating full-color image frames from these red, green, and blue signals.

As one illustrative example, when the endoscope system is switched to the fluorescence imaging mode, the image sensor 121 converts the monochromatic visible light and the fluorescence received respectively according to the channels into corresponding signals and transmits the signals to the image processing apparatus 130, and the image processing apparatus 130 is capable of generating background image data from the signals converted from the monochromatic visible light and generating fluorescence image data from the signals converted from the fluorescence and combining the background image data and the fluorescence image data to output a composite image frame. For example, in the example shown in fig. 4, the monochromatic visible light is blue light, the image sensor 121 converts blue light and fluorescence received by the blue signal channel 126 and the red signal channel 124 (or the green signal channel 125), respectively, into corresponding signals, the image processing device 130 is capable of generating a black-and-white background image frame from the blue light signals, generating fluorescence image data from the fluorescence signals, and superimposing the fluorescence image data in a specific color such as green into the black-and-white background image frame to obtain a composite image frame. Thus, the enhanced display effect of highlighting a specific area (e.g., a lesion) on a black-and-white background image can be obtained in real time during an operation.

Further, the image processing apparatus 130 can also obtain the pure fluorescence signal received by the channel for receiving fluorescence according to a certain operation on the light-converted signal received by each channel before generating fluorescence image data from the fluorescence signal, and can improve the accuracy of the fluorescence image data generated subsequently. In actual operation, the red signal channel 124 (or the green signal channel 125) receives a small amount of blue light in addition to the fluorescence light, so that the signal directly converted from the light received by the red signal channel 124 (or the green signal channel 125) is actually a fluorescence signal doped with blue background noise. Similarly, blue signal path 126 receives a small amount of fluorescence in addition to blue light. In view of the specific relationship between the sensitivity of each channel in the image sensor 121 to light of different wavelengths (refer to the quantum efficiency curves of the three channels shown in fig. 5), the image processing apparatus 130 may be further configured to obtain a pure fluorescent signal from which blue background noise is removed by performing a certain operation on the signal directly converted from light received by the red signal channel 124 (or the green signal channel 125) and the signal directly converted from light received by the blue signal channel 126. The image processing device 130 may then generate fluorescence image data from the pure fluorescence signal and superimpose the fluorescence image data in a specific color, such as green, onto a black and white background image frame to obtain a composite image frame.

Further, when the endoscope system is switched to the fluorescence imaging mode, the image processing apparatus 130 is also capable of estimating a signal expected to be received by a channel other than the channel for receiving monochromatic visible light in a scene illuminated with light of a wavelength range corresponding to its sensing capability, and generating background image data from the monochromatic visible light converted signal and the predicted signal. This is achieved by the fact that the sensitivity of the channels in the image sensor 121 to light of different wavelengths has a specific relationship. For example, in the example shown in fig. 4, the image sensor 121 includes a red signal channel 124, a green signal channel 125, and a blue signal channel 126, and quantum efficiency curves of the three channels are shown in fig. 5, where three curves represent sensitivity of the three channels to light of different wavelengths, respectively. In the example shown in fig. 4, the monochromatic visible light is blue light, and the red and green signal channels 124 and 125 can receive a small amount of blue light in addition to fluorescence, i.e., the signals directly converted from the light received by the red and green signal channels 124 and 125 are actually fluorescence signals doped with blue background noise. In this case, the image processing apparatus 130 may be further configured to remove the pure fluorescent signal amount calculated above from the signal directly converted from the light received by the red signal channel 124, obtain a blue light signal in the red signal channel 124, and estimate a corresponding red signal according to the relationship shown in fig. 5 from this blue light signal; the signal directly converted from the light received by the green signal channel 125 is removed from the pure fluorescence signal amount calculated above, a blue light signal in the green signal channel 125 is obtained, and a corresponding green signal is estimated from this blue light signal in accordance with the relationship shown in fig. 5. Subsequently, the image processing device 130 can generate a color background image frame from the blue light signal of the blue signal channel 126 and the estimated red and green signals, and superimpose the fluorescence image data generated from the pure fluorescence signal on the color background image frame in a specific color such as green to obtain a composite image frame. Thus, an enhanced display effect of highlighting a specific object to be observed 10 (e.g., human tissue) on a color background image can be obtained in real time during surgery. It will be appreciated that the "color" herein may be a false color with incomplete color information, but does not affect the physician's normal surgical procedure.

It will be appreciated that in the description herein, a "small amount of blue light" is relative to the fluorescence received by the red signal channel 124 and/or the green signal channel 125, and a "small amount of fluorescence" is relative to the blue light received by the blue signal channel 126.

In some embodiments, the endoscope system may also include a control device (not shown), or alternatively, be communicatively coupled (not shown) with an external control device. The control device is configured to send control signals to the light source device 110, the image capturing module 120, and the image processing device 130, respectively. For example, when the endoscope system is switched to the fluorescence imaging mode, the control device controls the light source device 110 to turn on the visible light source 101 and the excitation light source 113 to emit monochromatic visible light and laser light, respectively, and to remain normally on, and controls the image processing device 130 to perform corresponding image processing. For another example, when the endoscope system is switched to the white light imaging mode, the control device controls the light source device 110 to turn on the visible light source 101 to emit white light and remain normally on, and controls the image processing device 130 to perform corresponding image processing. The control device and the image processing device 130 may be integrated into the same computing device, or may be provided in different computing devices. The computing device may include at least one processor for executing instructions and at least one memory for storing instructions and data.

The light source device 110 is also configured such that monochromatic visible light and excitation light illuminate the object to be observed in coincident light paths.

Specifically, the light source device 110 includes an excitation light source and a visible light source. The visible light source may comprise a white light source and/or a monochromatic light source. Wherein the white light source may be used in a white light imaging mode and a fluorescence imaging mode of the endoscope system; the monochromatic light source may be used in a fluorescence imaging mode of an endoscopic system. According to the light path of the excitation light emitted by the excitation light source and the light path of the visible light emitted by the visible light source, the light source device 110 may be further provided with a reflective mirror and/or a spectroscope, so that the light path of the excitation light and the light path of the visible light coincide after passing through the reflective mirror and/or the spectroscope, that is, the mixed light is formed before the excitation light and the visible light reach the object to be observed.

As one illustrative example, as shown in fig. 6, the light source device 110 includes a white light source 111 and an excitation light source 113. The light source device 110 further includes a filter 116 for filtering white light emitted from the white light source 111 to form monochromatic visible light. The filter 116 may be configured to be movable between a monochromatic light passing position at which an optical path of white light is blocked and a white light passing position at which an optical path of white light is blocked. In the white light imaging mode, the filter 116 is located at a white light passing position, and the light path of the white light is not blocked by the filter 116. In the fluorescent imaging mode, the filter 116 is located at a monochromatic light passing position, and white light is filtered by the filter 116 and monochromatic visible light is formed. The light source device 110 further includes a spectroscope 115 for reflecting excitation light. The transmittance of the beam splitter 115 for light of different wavelengths may be selected according to the required excitation light wavelength, for example, as shown in fig. 9, the beam splitter 115 is capable of reflecting near infrared light having a wavelength range of 725 to 800nm and allowing other light other than this wavelength range to pass. As shown in fig. 6, the beam splitter 115 is located at the intersection of the white light path emitted by the white light source 111 and the excitation light path emitted by the excitation light source 113, and the mirror surface of the beam splitter 115 is disposed at an angle to the excitation light path, so that the excitation light path reflected by the beam splitter 115 coincides with the white light path.

As an illustrative example, the white light source 111 in fig. 6 may be replaced with a monochromatic light source, in which case the filter 116 is not required.

As an illustrative example, the light paths of white light or monochromatic visible light extend on a first straight line, and the light paths of excitation light reflected by the spectroscope extend on the first straight line. The optical path of the excitation light before the beam splitter extends on a third straight line. The first line is perpendicular to the third line.

As one illustrative example, as shown in fig. 7, the light source device 110 includes a white light source 111, a monochromatic light source 112, and an excitation light source 113. The light source device 110 further includes a first spectroscope 114 for reflecting monochromatic visible light emitted from the monochromatic light source 112 and a second spectroscope 115 for reflecting excitation light. The transmittance of the first beam splitter 114 and the second beam splitter 115 for light of different wavelengths may be selected according to the respective desired wavelengths of light. For example, as shown in FIG. 8, the first dichroic mirror 115 is capable of reflecting blue light having a wavelength less than 475nm and allowing other light greater than 475nm to pass. For example, as shown in fig. 9, the second beam splitter 115 can reflect near infrared light having a wavelength range of 725 to 800nm and allow other light except for this wavelength range to pass. The first beam splitter 114 and the second beam splitter 115 are arranged such that the optical path of the monochromatic visible light reflected via the first beam splitter 114 coincides with the optical path of the excitation light reflected via the second beam splitter 115. In one example, as shown in fig. 7, the first beam splitter 114 is located at the intersection of the monochromatic light path emitted by the monochromatic light source 112 and the excitation light path emitted by the excitation light source 113, the second beam splitter 115 is located at the intersection of the white light path emitted by the white light source 111 and the excitation light path emitted by the excitation light source 113, the first beam splitter 114 and the second beam splitter 115 are disposed in parallel and inclined with respect to the white light source 111, and the light path formed by the white light passes through the two beam splitters.

As an illustrative example, the optical path of the white light extends on a first straight line. The optical path of the monochromatic visible light reflected by the first spectroscope 114 and the optical path of the excitation light reflected by the second spectroscope 115 both extend on a first straight line. The monochromatic light source 112 and the excitation light source 113 are arranged in parallel and at a distance, and the first beam splitter 114 and the second beam splitter 115 are arranged in parallel and at a distance. The optical path of the monochromatic visible light before the first beam splitter 114 extends on a second straight line. The optical path of the excitation light before the second beam splitter 115 extends on a third straight line. The first line is perpendicular to the second line and the third line. The second line is parallel to and spaced apart from the third line.

In actual operation, after the excitation light reaches the object to be observed, a part of the excitation light is reflected to the image sensor 121 via the object to be observed. In some embodiments, a filter (not shown) may be provided in front of the image sensor 121 or the lens 122 for reflecting the excitation light. Thus, the interference of the excitation light on imaging can be directly avoided.

In other embodiments, instead of providing a filter for reflecting excitation light, the pure fluorescence signal and the pure excitation light signal may be extracted by calculation by the image processing apparatus, and the pure fluorescence signal may be corrected using the pure excitation signal. This is because, in actual operation, the intensity of the excited fluorescence is positively correlated with the intensity of the excitation light incident on the tissue in addition to the amount of dye contained in the tissue, and since the light of the endoscope is usually emitted from the tip of the probe, the intensity of the excitation light is reduced from the center of the field of view (light emitting portion) to the periphery instead of being uniformly distributed over the entire field of view, and thus the detected fluorescence signal may have a certain error due to the uneven intensity distribution of the excitation light, resulting in that the acquired fluorescence image cannot completely accurately reflect the actual condition of the amount of dye in the tissue. With this embodiment, in the fluorescence mode, the image sensor 121 can synchronously receive fluorescence, monochromatic visible light, and excitation light from the object to be observed 10, and the image processing apparatus 130 can correct the fluorescence image data from the signal converted by the excitation light after generating the fluorescence image data from the fluorescence signal to reduce or eliminate the influence of the excitation light intensity distribution unevenness. Taking the embodiment shown in fig. 4 as an example, the red signal channel 124 and the green signal channel 125 receive fluorescence, a small amount of blue light, and a small amount of excitation light, and the blue signal channel 126 receives blue light, a small amount of fluorescence, and a small amount of excitation light. The image processing device 130 can obtain a pure fluorescence signal and a pure excitation light signal according to a predetermined operation from the signals converted based on the light received by the three channels. The image processing device 130 can estimate the intensity of excitation light incident on the corresponding region of the object to be observed based on the calculated pure excitation light signal, and correct the pure fluorescence signal according to a predetermined operation by the estimated intensity of excitation light. This embodiment is suitable for a scene where higher accuracy of information of a diseased region (particularly judgment of the edge of the region) is required, and is advantageous in more accurately acquiring the amount of fluorescent agent in a tissue during a surgery, thereby improving diagnostic ability and a surgical procedure.

The embodiment of the invention also provides an imaging method of the endoscope system. The endoscope system may be the one described in any of the examples of the above embodiments, and will not be described here. As shown in fig. 10, the imaging method includes steps S120 to S140.

S120: the mixed light from the object to be observed is received and converted into a corresponding signal.

In step S120, the mixed light includes monochromatic visible light and fluorescence. The single-color visible light is reflected after the object to be observed is irradiated by the single-color light source, and the fluorescence is emitted after the object to be observed is irradiated by the excitation light source. The mixed light is received simultaneously by first and second channels having sensing capabilities of different wavelength ranges arranged side by side. The first channel and the second channel are channels in an image sensor of an endoscope system for receiving light and converting into corresponding signals. For example, the image sensor may be the image sensor 121 described in any of the foregoing embodiments, and will not be described herein. In some embodiments, the wavelength of the monochromatic visible light is within a wavelength range corresponding to the sensing capability of the first channel, whereby the first channel is for receiving monochromatic visible light and the second channel is for receiving fluorescence. In other embodiments, the wavelength of the monochromatic visible light is within a wavelength range corresponding to the sensing capability of the second channel, whereby the second channel is for receiving monochromatic visible light and the first channel is for receiving fluorescence.

As one illustrative example, the first channel may include one of a red signal channel, a green signal channel, and a blue signal channel, and the second channel may include another one or two of the red signal channel, the green signal channel, and the blue signal channel. For example, in the example shown in fig. 4, the first channel is a blue signal channel and the second channel is a red signal channel and/or a green signal channel.

S130: background image data is generated from the signal converted from monochromatic visible light, and fluorescence image data is generated from the signal converted from fluorescence.

In step S130, the image processing apparatus of the endoscope system may generate background image data from the signal converted by the monochromatic visible light, and generate fluorescence image data from the signal converted by the fluorescence. For example, the image processing apparatus may be the image processing apparatus 130 as described in any one of the foregoing embodiments, and will not be described herein.

In some examples, generating background image data from a signal converted from monochromatic visible light may include: black and white background image frames are generated from signals converted from monochromatic visible light. For example, in the example shown in fig. 4, the image processing device 130 may generate a black and white background image frame from the blue light signal in the blue signal channel 126.

In other examples, generating background image data from a signal converted from monochromatic visible light may include: a signal expected to be received by a channel other than the channel for receiving the monochromatic visible light in a scene illuminated by light of a wavelength range corresponding to its sensing capability is estimated from the signal converted by the monochromatic visible light, and a color background image frame is generated from the signal converted by the monochromatic visible light and the predicted signal. This is achieved by the fact that the sensitivity of the channels in the image sensor to light of different wavelengths has a specific relationship. For example, in the example shown in fig. 4, the image processing apparatus 130 may estimate the blue light signal in the red signal channel 124 to the corresponding red signal according to the relationship shown in fig. 5, estimate the blue light signal in the green signal channel 125 to the corresponding green signal according to the relationship shown in fig. 5, and then generate a color background image frame from the blue light signal of the blue signal channel 126 and the estimated red and green signals. It will be appreciated that the "color" herein may be a false color with incomplete color information, but does not affect the physician's normal surgical procedure.

In some examples, generating fluorescence image data from the signal converted from fluorescence may include: the light-converted signals received by the respective channels are subjected to a certain operation to obtain a pure fluorescence signal received by the channel for receiving fluorescence, and fluorescence image data is generated from the pure fluorescence signal. For example, in the example shown in fig. 4, the image processing apparatus 130 may obtain a pure fluorescent signal from which blue background noise is removed by a certain operation on the signal directly converted from light received by the red signal channel 124 (or the green signal channel 125) and the signal directly converted from light received by the blue signal channel 126, and then generate a fluorescent image frame from the pure fluorescent signal.

In some examples, the mixed light further includes excitation light reflected via the object to be observed, the excitation light being received by a third channel of the image sensor that is juxtaposed with the first channel and the second channel. Generating fluorescence image data from the signal converted from fluorescence may include: and correcting the fluorescence image data according to the signal converted by the excitation light, thereby obtaining corrected fluorescence image data. For example, in the example shown in fig. 4, the image processing apparatus 130 may obtain a pure fluorescent signal and a pure excitation light signal according to a predetermined operation from signals converted based on light received by the red signal channel 124, the green signal channel 125, and the blue signal channel 126, then estimate excitation light intensity incident on a corresponding region of an object to be observed based on the calculated pure excitation light signal, and correct the pure fluorescent signal according to a predetermined operation from the estimated excitation light intensity, generating a corrected fluorescent image frame.

S140: the composite image frame is output by combining the background image data and the fluorescence image data.

In step S140, the image processing apparatus of the endoscope system may output a composite image frame by combining the background image data and the fluorescent image data obtained as described above. For example, the image processing apparatus may be the image processing apparatus 130 as described in any one of the foregoing embodiments, and will not be described herein.

In some embodiments, as shown in fig. 11, the imaging method may further include step S110.

S110: monochromatic visible light and excitation light are generated and the object to be observed is irradiated simultaneously.

In step S110, monochromatic visible light is generated by a visible light source in the light source device, and excitation light is generated by an excitation light source in the light source device, which can excite an object to be observed to emit fluorescence. For example, the light source device may be the light source device 110 as described in any of the foregoing embodiments, and will not be described herein. The visible light source 101 and the excitation light source 113 in the light source device 110 emit monochromatic visible light and excitation light, respectively, to irradiate the object to be observed 10 simultaneously. The monochromatic visible light may be visible light in any suitable wavelength range, such as blue, red, green, etc. The excitation light may be any suitable light for exciting a fluorescent agent to fluoresce, such as infrared light, preferably near infrared light.

As an illustrative example, as shown in fig. 4, the monochromatic visible light may include blue light having a wavelength ranging between 425 and 475nm, alternatively, the blue light may have a wavelength peak of about 450nm, and may be 440nm, 445nm, 455nm, or 460nm; the excitation light may include near infrared light having a wavelength in the range of 760 to 810nm, alternatively, the near infrared light may have a wavelength peak of about 780nm or about 808nm, and may be 770nm, 775nm, 785nm, 790nm or 805nm; the fluorescence may include fluorescence having a wavelength in the range of 810 to 850nm, alternatively, the peak wavelength of the fluorescence may be about 830nm, and may be 820nm, 825nm, 835nm, or 840nm.

In some embodiments, as shown in FIG. 11, the imaging method further includes steps S210-S230.

S210: the monochromatic visible light and the excitation light are turned off, generating white light and irradiating the object to be observed.

In step S210, white light is generated by the visible light source in the light source device. For example, the light source device may be the light source device 110 as described in any of the foregoing embodiments, and will not be described herein. The visible light source 101 in the light source device 110 emits white light to illuminate the object to be observed 10.

S220: white light from the object to be observed is received and converted into a corresponding signal.

In step S220, white light is received by a channel in the image sensor. For example, the image sensor may be the image sensor 121 described in any of the foregoing embodiments, and will not be described herein. For example, as shown in fig. 3, the image sensor 121 includes a red signal channel 124, a green signal channel 125, and a blue signal channel 126, each for receiving light in a wavelength range corresponding to the respective sensing capability and converting into a corresponding signal.

S230: the signal is processed to generate image data and a full-color image frame is output.

In step S230, the image processing apparatus of the endoscope system may generate image data from the signals converted in the respective channels and output full-color image frames. For example, the image processing apparatus may be the image processing apparatus 130 as described in any one of the foregoing embodiments, and will not be described herein.

The sequence of steps of the method of the embodiments of the present invention may be adjusted, combined, or pruned as desired.

The endoscope system and the imaging method thereof provided by the invention have the beneficial effects that:

in a fluorescence imaging mode, an imaging mode of a plurality of light sources and a single light path is adopted, and visible light and fluorescence generated by excitation light are sensed by the same image sensor, so that the problem that the visible light imaging and the fluorescence imaging are difficult to align is effectively solved, and dislocation and double image of an output image are avoided; and the visible light source and the excitation light source irradiate the object to be detected synchronously, the image sensor can synchronously receive visible light and fluorescence for imaging, the problem of frame rate reduction caused by pulse irradiation is avoided, and the delay and smear of image display are reduced.

Therefore, the invention can reduce the operation fatigue degree and operation risk of doctors. In addition, as no additional sensor is needed for fluorescence imaging, the size of the handle end of the endoscope is miniaturized, and the design and manufacturing cost is reduced.

Unless defined otherwise, technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing particular implementations only and is not intended to be limiting of the invention. Features described herein in one embodiment may be applied to another embodiment alone or in combination with other features unless the features are not applicable or otherwise indicated in the other embodiment.

The present invention has been described by way of the above embodiments, but it should be understood that the above embodiments are for illustrative and explanatory purposes only and that the invention is not limited to the above embodiments, but is capable of numerous variations and modifications in accordance with the teachings of the invention, all of which fall within the scope of the invention as claimed.

Claims

1. An endoscope system, comprising:

image processing means for processing the signals to generate image data.

2. The endoscope system of claim 1, wherein the at least two channels comprise at least two of a red signal channel, a green signal channel, and a blue signal channel.

3. The endoscope system of claim 1, wherein the image processing device is configured to generate background image data from the signal converted from the monochromatic visible light in a fluorescence imaging mode, generate fluorescence image data from the signal converted from the fluorescence, and combine the background image data and the fluorescence image data to output a composite image frame.

4. An endoscope system according to claim 3, wherein said image processing device is further configured to estimate a signal expected to be received by channels other than the channel for receiving said monochromatic visible light in a scene illuminated with light of a wavelength range corresponding to its sensing capability, and to generate background image data from said monochromatic visible light converted signal and said predicted signal.

5. The endoscope system according to claim 1, wherein the light source device is configured such that the monochromatic visible light and the excitation light irradiate the object to be observed with coincident light paths.

6. The endoscope system according to claim 1, wherein the light source device is further configured to turn off an excitation light source in a white light imaging mode and cause the visible light source to emit white light to illuminate the object to be observed, the at least two channels respectively receiving light of a wavelength range corresponding to a sensing capability thereof in the white light.

7. The endoscope system of claim 6 wherein the at least two channels comprise a red signal channel, a green signal channel, and a blue signal channel.

8. The endoscope system according to claim 6, wherein the visible light source includes a white light source, the light source device further includes a spectroscope for reflecting the excitation light and a filter for filtering white light emitted from the white light source to form the monochromatic visible light, the filter is movable between a monochromatic light passing position that blocks an optical path of the white light and a white light passing position that avoids the optical path of the white light, the light source device is configured such that an optical path of the excitation light reflected via the spectroscope coincides with an optical path of the white light after passing via the filter.

9. The endoscope system according to claim 6, wherein the visible light source includes a monochromatic light source and a white light source, the light source device further includes a first spectroscope for reflecting monochromatic visible light emitted from the monochromatic light source and a second spectroscope for reflecting the excitation light, the light source device is configured such that an optical path of the monochromatic visible light reflected via the first spectroscope coincides with an optical path of the excitation light reflected via the second spectroscope, and the white light source is juxtaposed with the first spectroscope and the second spectroscope.

10. The endoscope system of claim 6, further comprising a control device configured to control the endoscope system to switch between the fluorescence imaging mode and a white light imaging mode, and to control the light source device to turn on the visible light source and the excitation light source in the fluorescence imaging mode and to turn off the excitation light source in the white light imaging mode.

11. The endoscope system of any of claims 1-10, wherein the monochromatic visible light comprises blue light having a wavelength peak of about 450nm, the excitation light comprises near-infrared light having a wavelength peak of about 780nm or about 808nm, the fluorescence comprises fluorescence having a wavelength peak of about 830nm, the at least two channels comprise a red signal channel, a green signal channel, and a blue signal channel, the blue signal channel for receiving the blue light, the red signal channel and/or the green signal channel for receiving the fluorescence.

12. An imaging method of an endoscope system, comprising the steps of:

13. The imaging method of claim 12, wherein the first channel comprises one of a red signal channel, a green signal channel, and a blue signal channel, and the second channel comprises another one or two of a red signal channel, a green signal channel, and a blue signal channel.

14. The imaging method of claim 12, further comprising: and generating the monochromatic visible light and excitation light and synchronously irradiating the object to be observed, wherein the excitation light excites the object to be observed to emit the fluorescence.

15. The imaging method of claim 12, wherein generating background image data from the signal converted by the monochromatic visible light comprises:

estimating signals expected to be received by channels other than the channel for receiving the monochromatic visible light in a scene irradiated by light of a wavelength range corresponding to the sensing capability of the channels according to the signals converted by the monochromatic visible light; and

Background image data is generated from the monochromatic visible light converted signal and the predicted signal.

16. The imaging method of claim 14, wherein the imaging device comprises a lens,

the monochromatic visible light includes blue light having a wavelength peak of about 450nm, the excitation light includes near infrared light having a wavelength peak of about 780nm or about 808nm, and the fluorescence includes fluorescence having a wavelength peak of about 830 nm; and is also provided with

The first channel comprises a blue signal channel and the second channel comprises a red signal channel and/or a green signal channel.

17. The imaging method of claim 12, wherein the mixed light further comprises excitation light that excites the object to be observed to emit the fluorescence light, the mixed light being synchronously received by first, second, and third channels disposed side-by-side with different wavelength range sensing capabilities, generating fluorescence image data from the signal converted by the fluorescence light comprising:

generating fluorescence image data from the signal converted from the fluorescence; and

and correcting the fluorescence image data according to the signal converted by the excitation light to obtain corrected fluorescence image data.

18. The imaging method of claim 17, wherein the first channel, the second channel, and the third channel each comprise one of a red signal channel, a green signal channel, and a blue signal channel.