JP2018134413A - Imaging apparatus - Google Patents

Abstract

An imaging apparatus capable of improving the visibility of a fluorescent image when a composite image is created by synthesizing a visible image of a subject and a fluorescent image. An image processing unit 31 in a control unit 30 specifies a dominant hue in a visible image by a dominant hue specifying unit 37, and R, G, B for the dominant hue by a colored hue specifying unit 38. Value, the sum of the absolute values of the differences is equal to or greater than the set value, and the hue with the maximum luminance ratio is specified as the colored hue, and the near-infrared image that is the fluorescent image is specified by the fluorescent image coloring unit 39 as the colored hue Coloring is performed with the color hue specified by the unit 38, and the synthesized image is synthesized by the synthesizing unit 36 with the near-infrared image colored by the visible image and the fluorescent image coloring unit 39. [Selection] Figure 3

Description

  The present invention relates to an imaging apparatus that irradiates a fluorescent dye injected into a subject with excitation light and images fluorescence generated from the fluorescent dye.

  In recent years, a technique called near-infrared fluorescence imaging has been used for surgery. In this near-infrared fluorescence imaging, indocyanine green (ICG) as a fluorescent dye is injected into the affected area. When this indocyanine green is irradiated with near infrared light of 750 to 810 nm (nanometer) as excitation light, indocyanine green emits fluorescence in the near infrared region having a peak at about 850 nm. This fluorescence is photographed with a camera capable of detecting near-infrared light, and the image is displayed on a display unit such as a liquid crystal display panel. According to this near-infrared fluorescence imaging, it is possible to observe blood vessels, lymph vessels, and the like existing at a depth of about 20 mm from the body surface.

  Patent Document 1 discloses a near-infrared fluorescence intensity distribution image obtained by irradiating an indocyanine green excitation light to a living organ to which indocyanine green is administered, and indocyanine green administration. Compared with the cancer lesion distribution image obtained by applying X-rays, nuclear magnetic resonance or ultrasound to the previous test organ, it is detected by the intensity distribution image of near-infrared fluorescence, A data collection method is disclosed in which data of a region that is not detected in a cancer lesion distribution image is collected as cancer secondary lesion region data.

  Patent Document 2 discloses that a subject to which an angiographic contrast agent is administered is alternately irradiated with excitation light and visible light, and a fluorescent image and a visible image irradiated with excitation light by an imaging unit are alternately displayed. A surgical support method is disclosed in which a blood vessel image is extracted by performing threshold processing on a fluorescent image with a predetermined threshold, and a composite image in which a visible image and the extracted blood vessel image are superimposed is obtained.

International Publication No. 2009/139466 JP 2009-226072 A

  As described in Patent Document 2, in the case of creating a composite image obtained by combining a visible image and a fluorescent image, the contrast agent region in the fluorescent image is expressed as lightness, and thus becomes a black and white image. And the fluorescence image thereof are combined, but it may be difficult to distinguish between the visible image and the fluorescence image after the combination.

  That is, since the fluorescent image is displayed in white on the visible image, its outline is not clear. In particular, the outline becomes unclear as the brightness difference decreases. For this reason, it is impossible to accurately determine how far the fluorescent region in the fluorescent image is. In such a case, there arises a problem that it is difficult to accurately perform the treatment on the patient.

  The present invention has been made to solve the above-described problem, and an imaging apparatus capable of improving the visibility of a fluorescent image when a composite image is created by synthesizing a visible image of a subject and a fluorescent image. The purpose is to provide.

  In the first invention, the excitation light source that irradiates the subject with excitation light for exciting the fluorescent dye injected into the subject, the visible light source that irradiates visible light toward the subject, and the excitation light An imaging apparatus comprising: a camera capable of detecting fluorescence and visible light for photographing fluorescence generated from the fluorescent dye by irradiation and visible light reflected from the surface of the subject. A dominant hue identifying unit that identifies a dominant hue in a visible image of the subject photographed by: a colored hue identifying unit that identifies a hue different from the hue identified by the dominant hue identifying unit as a colored hue; A fluorescent image coloring unit for coloring a fluorescent image of the subject photographed by the camera with a colored hue identified by the colored hue identifying unit; and the object photographed by the camera. By combining the body visible image and the fluorescence image colored fluorescence image colored with the coloring unit, characterized by comprising a synthesizing unit creating a synthesized image.

  In the second invention, the dominant hue specifying unit specifies a dominant hue using an image obtained by subtracting an image of a fluorescent region in the fluorescent image from the visible image.

  In the third invention, the dominant hue specifying unit specifies a hue that appears most frequently as a dominant hue using a histogram.

  According to the first invention, when a composite image is created by synthesizing a visible image of a subject and a fluorescent image, the fluorescent image and the visible image serving as a background can be clearly identified. It becomes possible to improve the property.

  According to the second invention, it is possible to appropriately specify the dominant hue in the background image by excluding the fluorescent image from the visible image.

  According to the third invention, it is possible to easily identify the dominant hue by using the histogram.

1 is a schematic diagram of an imaging apparatus according to the present invention. 2 is a perspective view of an illumination / photographing unit 12. FIG. It is a block diagram which shows the main control systems of the imaging device which concerns on this invention. It is a flowchart of the image synthetic | combination process which synthesize | combines a visible image and a near-infrared image with the imaging apparatus which concerns on this invention, and produces a synthesized image.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic diagram of an imaging apparatus according to the present invention.

  The imaging apparatus includes an input unit 11 such as a touch panel, and includes a main body 10 including a control unit 30 and the like described later, an illumination / photographing unit 12 supported movably by an arm 13, a liquid crystal display panel, and the like. And a treatment table 16 on which a patient 17 is placed. The illumination / imaging unit 12 is not limited to the one supported by the arm 13, and may be carried by the surgeon in hand.

  FIG. 2 is a perspective view of the illumination / imaging unit 12 described above.

  The illumination / photographing unit 12 includes a camera 21 capable of detecting near-infrared rays and visible light, a visible light source 22 including a large number of LEDs disposed on the outer peripheral portion of the camera 21, and an outer peripheral portion of the visible light source 22. And an excitation light source 23 made up of a number of arranged LEDs. The visible light source 22 emits visible light. The excitation light source 23 emits near infrared light having a wavelength of 780 nm, which is excitation light for exciting indocyanine green.

  FIG. 3 is a block diagram showing a main control system of the imaging apparatus according to the present invention.

  This imaging apparatus is composed of a CPU that performs logical operations, a ROM that stores operation programs necessary for controlling the apparatus, a RAM that temporarily stores data during control, and the like, and a control unit that controls the entire apparatus 30. The control unit 30 includes an image processing unit 31 that executes various image processing described later. The image processing unit 31 includes a combining unit 36 that generates a combined image by combining a visible image and a fluorescent image, a dominant hue specifying unit 37 that specifies a dominant hue, and a coloring hue specifying that specifies a coloring hue. A unit 38 and a fluorescent image coloring unit 39 for coloring the fluorescent image with a colored hue.

  The control unit 30 is connected to the input unit 11 and the display unit 14 described above. The control unit 30 is connected to an illumination / photographing unit 12 including a camera 21, a visible light source 22, and an excitation light source 23. Further, the control unit 30 is also connected to an image storage unit 33 that stores an image taken by the camera 21. The image storage unit 33 includes a near-infrared image storage unit 34 that stores a near-infrared image and a visible image storage unit 35 that stores a visible image. Instead of providing the near-infrared image storage unit 34 and the visible image storage unit 35, a composite image storage unit that stores an image obtained by combining the visible image and the near-infrared image may be provided.

  The operation when performing a surgical operation using the imaging apparatus according to the present invention will be described below. In the following description, a case where breast cancer surgery is performed on the patient 17 will be described.

  When performing breast cancer surgery using the imaging apparatus according to the present invention, indocyanine green is injected into the breast of the patient 17 who is supine on the treatment table 16 by injection. Then, near infrared rays are emitted from the excitation light source 23 toward the subject including the affected part, and visible light is emitted from the visible light source 22. Note that as the near-infrared light emitted from the excitation light source 23, as described above, near-infrared light of 780 nm that acts as excitation light for indocyanine green to emit fluorescence is employed. As a result, indocyanine green generates fluorescence in the near infrared region having a peak at about 850 nm.

  Then, the vicinity of the affected part of the patient 17 is photographed by the camera 21. The camera 21 can detect near infrared light and visible light. An image obtained by irradiating the patient 17 with visible light and capturing the image by the camera 21 becomes a visible image, irradiating the patient 17 with near-infrared light, and an image obtained by photographing the fluorescence from indocyanine green by the camera 21. It becomes an image. The near-infrared image and visible image captured by the camera 21 are sent to the image processing unit 31 shown in FIG. In the image processing unit 31, the near-infrared image and the visible image are converted into image data that can be displayed on the display unit 14. The near-infrared image data is stored in the near-infrared image storage unit 34 in the image storage unit 33. The visible image data is stored in the visible image storage unit 35 of the image storage unit 33.

  In this state, the composition unit 36 in the image processing unit 31 uses the near-infrared image data and the visible image data to create a composite image that combines the visible image and the near-infrared image, as will be described later. To do. Then, the image processing unit 31 displays the near-infrared image, the visible image, and the composite image on the display unit 14 at the same time or selectively in divided regions.

  Next, a composite image creation operation, which is a characteristic part of the present invention, will be described. FIG. 4 is a flowchart of an image composition process for creating a composite image by combining a visible image and a near-infrared image by the imaging apparatus according to the present invention.

  The image processing unit 31 in the control unit 30 shown in FIG. 3 specifies a dominant hue in the visible image by using the dominant hue specifying unit 37, and R, G, B for the dominant hue by the colored hue specifying unit 38. Value, the sum of the absolute values of the differences is equal to or greater than the set value, and the hue with the maximum luminance ratio is specified as the colored hue, and the near-infrared image that is the fluorescent image is specified by the fluorescent image coloring unit 39 as the colored hue Coloring is performed with the color hue specified by the unit 38, and the synthesized image is synthesized by the synthesizing unit 36 with the near-infrared image colored by the visible image and the fluorescent image coloring unit 39.

  In other words, the image processing unit 31 first creates a visible image as a background (background) by subtracting the image of the indocyanine green region that is the fluorescent region in the near-infrared image from the visible image. Then, the dominant hue specifying unit 37 performs color extraction on the visible image, and specifies a dominant hue (hue) in the visible image obtained by subtracting (excluding) the fluorescent region. Here, the dominant hue is also called a dominant color, and is a color that controls the impression of the entire image.

  This dominant hue is, for example, the hue that appears most frequently in the visible image. This dominant hue is identified by using a histogram for the visible image. At this time, since there is a possibility that an image other than the affected part of the patient 17 may be taken in the peripheral region of the visible image, a histogram may be applied to an image near the center of the visible image. Note that the dominant hue may be specified using tone conversion, an average value, an intermediate value, and a mode value in an arbitrary color space.

  Next, the coloring hue specifying unit 38 specifies the coloring hue that should color the near-infrared image indicating indocyanine green. This colored hue is preferably a hue having a high discriminating power with respect to the dominant hue in the previously identified visible image. For this reason, this colored hue has a color difference ΔRGB, which is the sum of absolute values of differences in the R, G, and B values with respect to the dominant hue, equal to or greater than the set value, and the luminance ratio with respect to the dominant hue. The hue with the highest (ratio of luminance L) is selected.

  Here, the color difference ΔRGB described above is expressed by the following equation.

  ΔRGB = [max (Rb, Rf) −min (Rb, Rf)] + [max (Gb, Gf) −min (Gb, Gf)] + [max (Bb, Bf) −min (Bb, Bf)]

  Here, Rb, Gb, and Bb are RGB values of the visible image that becomes the background (background), and Rf, Gf, and Bf are red that is a fluorescent image displayed on the visible image that becomes the background (foreground). This is the RGB value of the outer image. From this equation, the color difference ΔRGB, which is the sum of the absolute values of the differences between the R, G, and B values, is calculated.

  The luminance L is expressed by the following formula. In addition to the following formula, a luminance calculation formula used for the color difference signal may be used.

  L = R1 × 0.2126 + G1 × 0.7152 + B1 × 0.0722

  Here, when sR = R / 255, sB = B / 255, and sG = G / 255, R1 is [sR / 12.92] if sR is 0.03928 or less, and from 0.03928 If it is larger, it is [sR + 0.055) /1.055] to the 2.4th power. Similarly, G1 is [sG / 12.92] if sG is 0.03928 or less, and [sR + 0.055) /1.055] to the 2.4th power if it is greater than 0.03928. B1 is [sB / 12.92] when sB is 0.03928 or less, and [sR + 0.055) /1.055] to the 2.4th power when it is greater than 0.03928.

  Regarding the calculation of the color difference ΔRGB and the luminance L described above, “Web Content Accessibility Guidelines” by W3C (World Wide Web Consortium) which is a standardization body established to promote standardization of various technologies used on the web. WCAG) "1.0 and 2.0.

  Then, the near-infrared image, which is a fluorescent image, is colored with the color hue specified by the color hue specifying unit 38 by the fluorescent image coloring unit 39 in the image processing unit 31. In addition, the brightness (value) at the time of coloring the infrared image at this time employs the same brightness as the brightness of the near-infrared image that is the contrast image, and the maximum value is adopted as the saturation (saturation). Thereby, it becomes possible to recognize the fluorescent region by indocyanine green more suitably on the visible image.

  Thereafter, the synthesizing unit 36 in the image processing unit 31 synthesizes the colored fluorescent image colored by the fluorescent image coloring unit 39 and the visible image by screen synthesis or addition synthesis to create a synthesized image.

  As described above, in the imaging apparatus according to the present invention, the color difference ΔRGB, which is the sum of the absolute values of the differences in the R, G, and B values, is greater than or equal to the set value with respect to the dominant hue as the colored hue. In addition, since the hue having the maximum luminance ratio with respect to the dominant hue is selected, the indocyanine green fluorescent region can be clearly distinguished from the background visible image. For this reason, it becomes possible to perform the treatment etc. with respect to the patient 17 correctly.

  As the setting value for the color difference ΔRGB described above, for example, when the R, G, and B values are 256 gradations and Hue is 360 tone, 500 defined by the WCAG described above can be applied. At this time, ΔHue which is a difference in hue between the colored hue and the dominant hue is 118 to 242. However, it is also possible to select a colored hue with which ΔHue is about 90 to 270 with respect to the dominant hue.

  In the above-described embodiment, the excitation light source 23 that emits near-infrared light having a wavelength of 780 nm is used. However, the excitation light source 23 excites indocyanine green to generate excitation light. What irradiates near-infrared light of about 750 nm to 810 nm that can generate the light can be used.

  Furthermore, in the above-described embodiment, indocyanine green is used as a material containing a fluorescent dye, and the indocyanine green is irradiated with near-infrared light of 780 nm as excitation light, thereby approximately 850 nm from indocyanine green. Although the case of emitting fluorescence in the near-infrared region having a peak at, light other than near-infrared light may be used.

  For example, 5-ALA (5-aminolevulinic acid / 5-aminolevulinic acid) can be used as the fluorescent dye. When 5-ALA is used, 5-ALA that has entered the body of the patient 17 is changed to a protoporphyrin IX / PpIX that is a fluorescent substance. When visible light of about 400 nm is irradiated toward the protoporphyrin, red visible light is irradiated as fluorescence from the protoporphyrin. For this reason, when 5-ALA is used, an excitation light source that emits visible light having a wavelength of about 400 nm may be used.

DESCRIPTION OF SYMBOLS 10 Main body 11 Input part 12 Illumination / imaging | photography part 13 Arm 14 Display part 16 Treatment table 17 Patient 21 Camera 22 Visible light source 23 Excitation light source 30 Control part 31 Image processing part 33 Image storage part 34 Near-infrared image storage part 35 Visible image Storage unit 36 Compositing unit 37 Dominant hue specifying unit 38 Colored hue specifying unit 39 Fluorescent image coloring unit

Claims (3)

An excitation light source that irradiates the subject with excitation light for exciting the fluorescent dye injected into the subject;
A visible light source that emits visible light toward the subject;
A camera capable of detecting fluorescence and visible light for photographing fluorescence generated from the fluorescent dye by irradiation of excitation light and visible light reflected from the surface of the subject;
In an imaging apparatus comprising:
A dominant hue specifying unit for specifying a dominant hue in a visible image of the subject photographed by the camera;
A colored hue specifying unit that specifies a hue different from the hue specified by the dominant hue specifying unit as a colored hue;
A fluorescent image coloring unit for coloring a fluorescent image of the subject photographed by the camera with a coloring hue specified by the coloring hue specifying unit;
Combining a visible image captured by the camera and a colored fluorescent image colored by the fluorescent image coloring unit to create a composite image;
An imaging apparatus comprising: The imaging apparatus according to claim 1, wherein
The dominant hue specifying unit is an imaging device that specifies a dominant hue using an image obtained by subtracting an image of a fluorescent region in the fluorescent image from the visible image. The imaging apparatus according to claim 1 or 2,
The dominant hue specifying unit is an imaging apparatus that specifies a hue that appears most frequently as a dominant hue using a histogram.

Images