JP2014000119A - Endoscope and endoscope system including the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve positioning accuracy in super-resolving processing when acquiring multiple low-resolution images with deviated imaging positions required for super-resolving processing with the use of blurring.SOLUTION: An endoscope includes: an insertion unit 11 to be inserted to the inside of an observation object; an illumination window 16 arranged in the insertion unit and illuminating a subject S inside the observation object; an imaging unit 14 arranged in the insertion unit and imaging a subject; and a marking member 19 having a marker part 17 arranged on a subject and a support wire 18 projected from the insertion unit and supporting the marker part. Especially, the support wire is arranged in the insertion unit to be advanceable/retreatable. The support wire is formed of an elastic material and the marker part is pressurized against a subject by elastic force with the flexure of the support wire.

Description

  The present invention relates to an endoscope that images the inside of an observation target that cannot be directly observed from the outside and an endoscope system including the endoscope, and particularly suitable for performing super-resolution processing on an image obtained by imaging. The present invention relates to an endoscope and an endoscope system including the endoscope.

  In the endoscope, an image sensor is arranged at the distal end of an insertion part that is inserted into an observation target such as a human body, and an image obtained by imaging with the image sensor is displayed on a monitor. So-called electronic endoscopes are widely used. Even in this type of endoscope, it is desirable to form the insertion portion as thin as possible. However, since the size of the image sensor is restricted, an image sensor with high resolution cannot be employed. There has been a problem that it is not possible to sufficiently satisfy the desire to observe internal tissues such as internal organs of the human body with fine images.

  In response to such a demand, the resolution of an image captured by an endoscope is reduced by using a so-called inter-frame super-resolution technique that generates a high-resolution image from a plurality of low-resolution images obtained by imaging. It is performed (patent document 2). In addition, since super-resolution processing requires a plurality of images whose imaging positions are shifted, a plurality of pixels whose imaging positions are shifted using a pixel shifting mechanism that relatively vibrates the imaging element and the objective lens with an actuator. A technique for acquiring the image is also known (Patent Document 1).

JP-A-6-178214 Special table 2010-512173 gazette

  In the inter-frame super-resolution technique, it is necessary to grasp the amount of misalignment between low-resolution images in order to align a plurality of low-resolution images. When the pixel shifting mechanism is used, for example, by controlling the applied voltage in consideration of the hysteresis characteristic of the expansion / contraction amount of the stacked piezoelectric element used in the pixel shifting mechanism, the positional shift amount between images can be managed. Based on this, it is possible to obtain an advantage that alignment with sub-pixel accuracy can be performed. However, in the configuration using the pixel shifting mechanism, it is necessary to provide the pixel shifting mechanism in the insertion portion together with the image sensor, and there is a problem that the insertion portion becomes thick.

  On the other hand, in the case of an endoscope, it is inevitable that the insertion part shakes due to camera shake, that is, the hand holding the endoscope shakes, but the imaging position is shifted by this camera shake, so if this camera shake is used, pixel shifting will occur. Without providing a mechanism, it is possible to acquire a plurality of images whose imaging positions are shifted.

  However, in the case of acquiring a plurality of images whose imaging positions are shifted by using camera shake in this way, unlike the case of using the pixel shifting mechanism, the amount of positional shift between images is not known. Therefore, it is necessary to acquire the amount of misalignment from the image itself. For this purpose, a so-called feature point matching method is used in which feature points are extracted from the image and the amount of misalignment is obtained from the correspondence between the feature points. That's fine. In general inter-frame super-resolution processing, a predetermined standard image and other reference images are defined for a plurality of captured frames, and the positional deviation amount of the reference image with respect to the standard image is set, for example, SURF (Speeded Up Robust Features). And a feature point matching method such as SIFT (Scale-invariant feature transform), and the image is reconstructed by convolving a plurality of images after alignment. However, when the subject is an internal tissue such as an organ, the subject has almost no characteristic texture, and there are few places where the color and shape change greatly. There is a problem in that the effect of increasing the resolution by the inter-frame super-resolution processing is small because it cannot be performed with high accuracy.

  The present invention has been devised to solve such problems of the prior art, and the main object of the present invention is to use a plurality of low-level images that are shifted in imaging positions necessary for super-resolution processing using camera shake. An object of the present invention is to provide an endoscope and an endoscope system including the endoscope that are configured to improve the accuracy of alignment in super-resolution processing when a resolution image is acquired.

  The endoscope according to the present invention includes an insertion unit that is inserted into an observation target, an illumination unit that is provided in the insertion unit and illuminates a subject inside the observation target, and the subject that is provided in the insertion unit. And a marking member having a marker part arranged on the subject and a support part protruding from the insertion part and supporting the marker part.

  In addition, the endoscope system of the present invention is an image that performs super-resolution processing for generating a high-resolution image from the endoscope configured as described above and a plurality of low-resolution images acquired by imaging by the imaging unit. A processing device, and when the image processing device performs alignment of the plurality of low resolution images in the super-resolution processing, a feature point is located at a position corresponding to the marker portion in the low resolution image. It is set as the structure to extract.

  According to the present invention, by arranging the marker unit on the subject, the marker unit is imaged together with the subject by the imaging unit. Thereby, when performing alignment of a plurality of low resolution images acquired by imaging by the imaging unit in the super-resolution processing, feature points are reliably extracted at positions corresponding to the marker portions in the low resolution image. The alignment accuracy can be increased.

Overall configuration diagram showing an endoscope system according to the present embodiment Sectional drawing which shows the front-end | tip part 12 of the insertion part 11 of the endoscope 1 Side view showing the second light source unit 22 The perspective view which shows the marker part 17 The side view which shows the condition which made the marker part 17 contact the to-be-photographed object S The perspective view which shows the front-end | tip part 12 of the insertion part 11 of the endoscope 1 Functional block diagram showing the super-resolution processing unit 32 Explanatory drawing explaining the outline | summary of the reconstruction process performed in the reconstruction process part 43 of the super-resolution process part 32 Explanatory drawing which shows the low-resolution image (captured image) obtained by the imaging by the endoscope 1 The perspective view which shows another embodiment of a marker part The perspective view and sectional drawing which show another embodiment of a marker part The perspective view which shows another embodiment of a marker part The perspective view which shows another embodiment of a marker part The perspective view which shows another embodiment of a marker part

  A first invention made to solve the above-described problems is an insertion unit that is inserted into an observation target, an illumination unit that is provided in the insertion unit and illuminates a subject inside the observation target, and the insertion And a marking member that has a support part that protrudes from the insertion part and supports the marker part. .

  According to this, by arranging the marker portion on the subject, the marker portion is imaged together with the subject by the imaging portion. Thereby, when performing alignment of a plurality of low resolution images acquired by imaging by the imaging unit in the super-resolution processing, feature points are reliably extracted at positions corresponding to the marker portions in the low resolution image. The alignment accuracy can be increased.

  Moreover, 2nd invention is set as the structure by which the said support part was provided in the said insertion part so that advancement / retraction was possible.

  According to this, by extending the support portion from the insertion portion, the marker portion can be brought into contact with the subject without moving the insertion portion itself. In addition, by pulling the support portion into the insertion portion, the insertion portion can be smoothly inserted into and removed from the observation target without damaging the observation target with the marking member or damaging the marking member.

  According to a third aspect of the present invention, the support portion is formed of an elastic material, and the marker portion is pressed against the subject by an elastic force due to bending of the support portion.

  According to this, when the insertion portion is shaken by shaking of the hand, that is, the hand holding the endoscope, the support portion is appropriately bent and the marker portion is pressed against the subject by the elastic force of the support portion. The marker portion can be held stationary on the subject.

  According to a fourth aspect of the present invention, the support portion is formed of a light guide material that guides light from a light source to the marker portion, and the marker portion has a light output surface that emits light guided by the support portion. The configuration.

  According to this, if the light emitted from the marker part is directly incident on the imaging window, a region with high brightness appears in the image of the marker part shown in the captured image, and the light emitted from the marker part is If the light is incident on the subject, a region with high luminance appears on the subject, and a region with higher brightness than the periphery appears in the vicinity of the image of the marker portion shown in the captured image. When an area with high brightness appears in the captured image in this way, feature points are extracted at the position of the area with high brightness. For this reason, it becomes possible to extract feature points more reliably at the position corresponding to the marker portion, and the accuracy of alignment can be further increased.

  According to a fifth aspect of the invention, the support portion is formed of a light guide material that guides light from a light source to the marker portion, and the marker portion includes a fluorescent member that emits light by the light guided by the support portion. The configuration.

  According to this, the brightness of the image of the fluorescent member shown in the captured image is higher than that of the periphery, and the feature point is extracted at the position of the region where the brightness is high. For this reason, it becomes possible to extract feature points more reliably at the position corresponding to the marker portion, and the accuracy of alignment can be further increased.

  Further, in a sixth aspect of the present invention, the marker portion has a contact portion formed in a substantially T-shape extending from the support portion, the contact portion having a substantially circular cross section, The outer peripheral surface is configured to contact the subject.

  According to this, since the object is contacted by the contact surface having a substantially arc-shaped cross section at the contact part, it is possible to avoid the object (organ etc.) from being damaged by the contact with the marker part. Moreover, since the light emission surface and the fluorescent member can be provided at both ends of the contact portion, a plurality of feature points can be extracted, and the alignment accuracy can be further improved.

  According to a seventh aspect of the present invention, the marker portion has a contact portion formed in a state of being bent in a substantially L shape from the support portion, the contact portion having a substantially circular cross section, and an outer peripheral surface thereof. In this configuration, the object is brought into contact with the subject.

  According to this, since the object is contacted by the contact surface having a substantially arc-shaped cross section at the contact part, it is possible to avoid the object (organ etc.) from being damaged by the contact with the marker part. In addition, since the marker portion can be formed only by bending a single linear member having a substantially circular cross section, the manufacture is facilitated.

  According to an eighth aspect of the invention, the marker portion includes a plurality of the fluorescent members and a holding portion that integrally holds the plurality of fluorescent members, and the holding portion is formed of a light transmissive material. The light guided by the support part is configured to enter the fluorescent member through the holding part.

  According to this, since many feature points can be extracted according to the number of arranged fluorescent members, the alignment accuracy can be further improved. In addition, since the holding portion is formed in a plate shape, it can be brought into contact with the subject with a large contact area, so that the subject (such as an organ) can be prevented from being damaged due to contact with the marker portion.

  The ninth invention includes an endoscope configured as described above, an image processing apparatus that performs super-resolution processing for generating a high-resolution image from a plurality of low-resolution images acquired by imaging by the imaging unit, and And the image processing device extracts a feature point at a position corresponding to the marker portion in the low resolution image when aligning the plurality of low resolution images in the super-resolution processing. And

  According to this, since the feature point is reliably extracted at the position corresponding to the marker portion in the low-resolution image, it is possible to improve the alignment accuracy.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is an overall configuration diagram showing an endoscope system according to the present embodiment. The endoscope system includes an endoscope 1 that is inserted into a human body (observation target) and images a subject S (an outer surface of an internal organ or an inner wall of a digestive tract) such as an internal organ of the body; Generated by a controller (control device) 2 that controls the endoscope 1, an image processing unit (image processing device) 3 that performs super-resolution processing on an image captured by the endoscope 1, and the image processing unit 3 And a monitor (display device) 4 for displaying the processed image.

  The endoscope 1 includes an insertion portion 11 that is inserted into the body. In the present embodiment, a so-called rigid endoscope in which the insertion portion 11 is mainly configured in a hard rod shape will be described, but the present invention may be applied to a flexible endoscope using a flexible material for the insertion portion. . In endoscopic surgery using a laparoscope, which is a typical example of a rigid endoscope, for example, a minute hole is made in a part of the abdomen, and the insertion part 11 is inserted from there to observe the inside of the body cavity. Perform surgery. An imaging unit 14 that images a subject (such as an organ) S inside the body through an imaging window 13 is provided at the distal end 12 of the insertion unit 11. In addition, an illumination window (illumination unit) 16 that emits light guided by the optical fiber 15 to illuminate the subject S is provided at the distal end portion 12 of the insertion unit 11.

  Furthermore, the endoscope 1 includes a marking member 19 having a marker portion 17 arranged on the subject S and a support wire (support portion) 18 that protrudes from the distal end portion 12 of the insertion portion 11 and supports the marker portion 17. It has.

  The support wire 18 is pulled out from a pull-out portion 20 provided on the base side of the insertion portion 11 and connected to the controller 2. The insertion hole of the support wire 18 provided in the drawer portion 20 may be used also as a forceps hole for inserting a forceps, and is dedicated to the support wire 18 different from the forceps hole. Also good.

  In particular, in the present embodiment, the support wire 18 is provided in the insertion portion 11 so as to be movable back and forth in the longitudinal direction of the insertion portion 11, and the portion of the support wire 18 drawn out from the pull-out portion 20 of the insertion portion 11 is manually picked. By pushing and pulling, the marker portion 17 can be advanced and retracted.

  Further, the support wire 18 is composed of an optical fiber that sends marker light to the marker portion 17 at the tip, and the optical fiber is covered with a sheath (sheath) that smoothly moves back and forth within the insertion portion 11 and protects the optical fiber. It has become.

  The controller 2 includes a first light source unit 21 that sends illumination light to the illumination window 16 provided in the insertion unit 11 of the endoscope 1 through the optical fiber 15, and a support wire 18 to the marker unit 17 of the marking member 19. The second light source unit 22 for sending the marker light through the optical fiber constituting the optical system, the operation switch 23 for instructing various operations, and the imaging unit 14 provided in the insertion unit 11 of the endoscope 1 are operated. The image capturing control unit 24 that receives image data from the image capturing unit 14 and the main control unit 25 that collectively controls each unit are provided.

  The main control unit 25 operates the imaging control unit 24, the first light source unit 21, and the second light source unit 22 in synchronization. In this embodiment, the imaging sensor 52 (see FIG. 2) provided in the imaging unit 14 performs a global shutter operation at, for example, 30 fps (Frames Per Second) (the shutter cycle is 33.3 ms), but the charge accumulation period (shutter opening time) ) Is set to about 1 ms. The main control unit 25 outputs a light emission instruction to the first light source unit 21 and the second light source unit 22 in synchronization with this timing. As a result, the first light source unit 21 and the second light source unit 22 Light is emitted only during a period corresponding to the charge accumulation period. This realizes high-luminance light emission of the LED, obtains a low-resolution image with less blur, and eliminates the suppression factor of the super-resolution effect.

  On the operation switch 23, instructions such as lighting of the first light source unit 21 and the second light source unit 22, imaging by the imaging unit 14, execution of image processing by the image processing unit 3, and switching of moving images and still images are given. Done.

  The image processing unit 3 is configured by a PC or the like, and includes a storage unit 31 that stores a captured image acquired by the imaging unit 14 provided in the insertion unit 11 of the endoscope 1, and an ultra-high resolution image that is generated from the captured image. A resolution processing unit 32, a display control unit 33 that controls the monitor 4, and a main control unit 34 that collectively controls each unit are provided. The image processing unit 3 is connected to an input unit 35 constituted by a keyboard or the like. Here, drive control (frame rate, exposure time, white balance, etc.) of the imaging unit 14 and a super-resolution processing unit. For example, parameters related to super-resolution processing at 32 are set. The main control unit 25 of the controller 2 and the main control unit 34 of the image processing unit 24 can bidirectionally transmit information, and the main control unit 34 of the image processing unit 24 has already passed the imaging unit 14 ( The value of the control parameter set in the imaging sensor 52 (see FIG. 2) can be referred to, and only the bit corresponding to the control parameter related to the specific drive control of the imaging unit 14 is rewritten, and the imaging unit 14 is rewritten. Can be set to

  FIG. 2 is a cross-sectional view showing the distal end portion 12 of the insertion portion 11 of the endoscope 1. At the distal end portion 12 of the insertion portion 11, the imaging portion 14, the illumination optical fiber 15, and the marking member 19 are accommodated inside a cylindrical casing 51. The illumination light guided by the optical fiber 15 is emitted through an illumination window 16 made of a cover glass. The imaging unit 14 includes an imaging sensor 52 composed of CMOS (Complementary Metal Oxide Semiconductor) and an optical system 53 including a plurality of lenses, and the reflected light from the subject S illuminated by the illumination light is reflected. The light enters the optical system 53 through the imaging window 13 made of a cover glass. The support wire 18 of the marking member 19 is inserted into a guide hole 55 extending in the longitudinal direction of the insertion portion 11 so as to advance and retract.

  As will be described later, a substantially T-shaped marker portion 17 is provided at the tip of the marking member 19. And the marker part 17 is accommodated in the recessed part 81 provided in the same surface as the imaging window 13 and the illumination window 16 at the time of the non-use (refer FIG. 6).

  FIG. 3 is a side view showing the second light source unit 22. The second light source unit 22 collects light emitted from the white LED module 61, a white LED module 61 having an LED element serving as a light source, a heat radiation unit (heat sink) 62 that radiates heat generated by the white LED module 61, and the white LED module 61. A coupling optical system 63 that emits light, and a coupling unit 64 that holds the support wire 18 formed of an optical fiber and makes the light condensed by the coupling optical system 63 incident on the end face of the optical fiber. .

  The first light source unit 21 is similar to this. It is also possible to provide a structure for branching the optical path into two so that the first light source unit 21 and the second light source unit 22 can be shared by one light source unit.

  FIG. 4 is a perspective view showing the marker portion 17. The marker portion 17 has a contact portion 71 formed in a substantially T-shaped state extending from the support wire 18, and the contact portion 71 has a substantially circular cross section, and has an outer peripheral surface on the subject S. It comes to contact. That is, the contact portion 71 is provided in a mode substantially orthogonal to the support wire 18, and the support wire 18 is drawn out from the insertion portion 11, so that the surface on the side opposite to the support wire 18 in the contact portion 71 is the subject S. To touch. Since the contact surface has a substantially arc-shaped cross section and no sharp portion, the subject S can be prevented from being damaged by contact with the marker portion 17.

  In particular, in the present embodiment, the marker portion 17 has a fluorescent member 72 that emits light by light guided by the support wire 18 made of an optical fiber, and the fluorescent member 72 is provided at each end of the contact portion 71. The fluorescent member 72 is obtained by dispersing a fluorescent material in a base material made of a light transmitting material. The contact portion 71 is also formed of a light guide material, and the light guided by the support wire 18 enters the fluorescent member 72 through the contact portion 71 and the fluorescent member 72 emits light.

  FIG. 5 is a side view showing a state in which the marker unit 17 is brought into contact with the subject S. As described above, the support wire 18 is provided in the insertion portion 11 of the endoscope 1 so as to be able to advance and retract. By extending the support wire 18, the marker portion 17 is moved to the subject S without moving the insertion portion 11 itself. Can be contacted.

  Further, the support wire 18 is formed of an elastic material, specifically, an optical fiber made of synthetic resin having elasticity and a sheath, and the marker portion 17 is formed by an elastic force generated when the support wire 18 is bent. Is pressed against the subject S.

  For this reason, when the insertion portion 11 is shaken by shaking, that is, the hand holding the endoscope 1 is shaken, the support wire 18 is appropriately bent and the marker portion 17 is moved to the subject S by the elastic force of the support wire 18. Since it is pressed, the marker portion 17 can be held stationary on the subject S.

  As shown in FIG. 5, the angle of view indicating the imaging range of the imaging unit 14 is included in the illumination range of the illumination unit 16. Therefore, although the marker unit 17 is also irradiated by the irradiation light emitted from the illumination unit 16, the marker unit 17 marks itself with high brightness by irradiating the self-light-emitting element or the subject S from the very vicinity. It functions well as a sign indicating.

  FIG. 6 is a perspective view showing the distal end portion 12 of the insertion portion 11 of the endoscope 1. The distal end portion 12 of the insertion portion 11 is provided with a recess 81 for storing the marker portion 17. When the insertion portion 11 is inserted into or taken out from the human body, the marker portion 17 is inserted. Is stored in the recess 81. Thereby, the insertion part of the human body can be smoothly inserted and removed without damaging the human body with the marking member 19 or damaging the marking member 19.

  FIG. 7 is a functional block diagram showing the super-resolution processing unit 32. FIG. 8 is an explanatory diagram for explaining an overview of the reconstruction processing performed by the reconstruction processing unit 43 of the super-resolution processing unit 32.

  As shown in FIG. 7, the super-resolution processing unit 32 includes a low-resolution image acquisition unit 41, an alignment unit 42, a reconstruction processing unit 43, and a high-resolution image generation unit 44. The super-resolution processing unit 32 is realized by executing a program by a CPU.

  When the operation switch 23 (see FIG. 1) instructs the still image shooting, the main control unit 25 of the controller 2 performs super-resolution to the super-resolution processing unit 32 via the main control unit 34 of the image processing unit 3. Give instructions to perform image processing. The low resolution image acquisition unit 41 acquires a plurality (about 16 to 32) of low resolution images (captured images) necessary for the super-resolution processing from the storage unit 31. The plurality of low-resolution images are images (frame images) generated by performing imaging a plurality of times with the imaging unit 14 provided in the endoscope 1.

  The alignment unit 42 selects one of a plurality of low resolution images as a reference image, and aligns the low resolution images other than the reference image with respect to the reference image. A transformation matrix for coordinate transformation of a low resolution image is obtained. In this alignment, feature point matching is performed. Specifically, feature points are extracted for a reference image and a low-resolution image other than the reference image, and based on the correspondence between the feature points between the images, the reference image is extracted. A projection transformation matrix is obtained for each low-resolution image other than.

  In this feature point matching, it is desired to extract four or more feature points. Therefore, the marker unit 17 may be configured so that four or more feature points are extracted. In many cases, it is possible to extract feature points using a known algorithm known as a method or SIFT, and it is not an absolute requirement to extract four or more feature points.

  As shown in FIG. 8, the reconstruction processing unit 43 first enlarges the reference image at the super-resolution enlargement ratio, and generates a mapping image in which the pixels of the reference image are discretely arranged. Then, using the conversion matrix acquired by the alignment unit 42, the low-resolution image other than the reference image is converted into the coordinate system of the reference image, and the pixels of the low-resolution image other than the reference image are converted based on the converted coordinates. Is plotted on the mapping image. Thereby, a reconstructed image is obtained.

  Here, in general, plotting all low-resolution image pixels on the mapping image does not plot low-resolution image pixels on all mapping image pixels. There are pixels where the pixels of the low resolution image are not plotted.

  In FIG. 8, for convenience of illustration, the low-resolution image has a size of 4 pixels × 4 pixels and the super-resolution enlargement ratio is four times in the vertical and horizontal directions. The magnification of the image is not limited to this.

  In the high resolution image generation unit 44 illustrated in FIG. 7, a high resolution image is generated from the reconstructed image acquired by the reconstruction processing unit 43. Here, first, processing is performed to fill in pixels in which the pixels of the low-resolution image are not plotted in the reconstructed image acquired by the reconstruction processing unit 43 by interpolation, and further, the amount of image blur (PSF: Point Spread Function) , And a blur restoration process is also performed by inverse transformation to output as a high resolution image.

  Now, as shown in FIG. 5, the imaging unit 14 provided at the distal end portion 12 of the insertion unit 11 of the endoscope 1 captures an image of the subject S within the imaging range with a predetermined angle of view. By arranging the marker portion 17 on the subject S, the marker portion 17 is imaged together with the subject S, and a captured image in which the marker portion 17 is reflected on the subject S is obtained.

  Here, when the subject 1 is shaken when the subject S is imaged, that is, when the insertion portion 11 is shaken by shaking the hand holding the endoscope 1, the distal end portion 12 of the insertion portion 11 moves. As described above, the marker portion 17 is biased and pressed in the direction of the subject S by the elastic force of the support wire 18 and is held stationary. The positional relationship with the part 17 does not change.

  FIG. 9 is an explanatory diagram showing a low-resolution image (captured image) obtained by imaging with the endoscope 1. In the low-resolution image obtained by imaging with the endoscope 1, the image of the marker unit 17 arranged on the subject S appears together with the image of the subject S. Although the imaging range of each low-resolution image is shifted due to camera shake, there is no difference in the positional relationship between the image of the marker unit 17 and the image of the subject S. Therefore, if the position of the image of the marker unit 17 is used as a reference, the low resolution image can be accurately aligned.

  As described above, the alignment unit 42 of the super-resolution processing unit 32 performs alignment of other low-resolution images with respect to the reference image by feature point matching. In this alignment, feature points are extracted from the low-resolution image. However, since the image of the marker unit 17 is significantly different from the image of the surrounding subject S, the feature point is reliably extracted at the position of the image of the marker unit 17. For this reason, the alignment process can be performed with high accuracy.

  In particular, in the present embodiment, the fluorescent member 72 is provided in the marker portion 17 and the fluorescent member 72 is caused to emit light with the light guided by the support wire 18, so that the image of the fluorescent member 72 in the captured image is lighter than the surroundings. The feature point is extracted at the position of the region where the brightness is high. In particular, since the fluorescent member 72 emits light uniformly and entirely, the image of the fluorescent member 72 appears with uniform brightness. For this reason, a feature point can be extracted more reliably at a position corresponding to the marker portion 17, and the alignment accuracy can be further increased.

  Next, another embodiment of the marker unit will be described.

  FIG. 10 is a perspective view showing another embodiment of the marker unit. In this example, the marker part 101 has the contact part 102 formed in the state extended from the support wire 18 in the substantially T shape similarly to the example shown in FIG. Although it has a substantially circular cross section and is in contact with the subject S on its outer peripheral surface, and the contact portion 102 is formed of a light guide material, in this case, in particular, the light guided by the support wire 18 Is emitted from the light exit surfaces 103 at both ends thereof.

  In this configuration, light emitted from the light exit surface 103 is incident on the subject S, and a high-luminance region A appears on the subject S. As a result, a region having a higher brightness than the periphery appears in the vicinity of the image of the marker unit 101 shown in the captured image, and feature points are extracted at positions of the regions having the higher brightness. For this reason, it becomes possible to extract feature points more reliably at the position corresponding to the marker unit 101, and the accuracy of alignment can be further increased.

  Further, in this configuration, as in the example shown in FIG. 4, the contact portion 102 contacts the subject S with a contact surface having a substantially arc-shaped cross section, and therefore the subject S is damaged by contact with the marker portion 101. Can be avoided.

  FIG. 11A is a perspective view showing another embodiment of the marker unit. FIG. 11B is a cross-sectional view of the marker portion shown in FIG. In this embodiment, the marker portion 111 has a contact portion 112 formed in a state bent from the support wire 18 into a substantially L shape, and the contact portion 112 has a substantially circular cross section, and has an outer peripheral surface thereof. It contacts the subject S. Further, a chamfered portion 113 is formed on the tip side of the contact portion 112.

  In this configuration, the light guided by the support wire 18 enters the contact portion 112 while being reflected, is reflected by the chamfered portion 113 at the tip of the contact portion 112, and light is emitted from the light exit surface 114 on the side opposite to the subject S. Exit. As a result, a region with high brightness appears in the image of the marker unit 111 shown in the captured image, and a feature point is extracted at the position of the region with high brightness. For this reason, a feature point can be extracted more reliably at a position corresponding to the marker unit 111, and the alignment accuracy can be further improved.

  Further, in this configuration, the subject S is in contact with the contact surface having a substantially arc-shaped cross section on the outer peripheral surface of the contact portion 112, and the chamfered portion 113 is also formed on the distal end side of the contact portion 112. Can be prevented from being damaged by contact with the marker 111.

  FIG. 12 is a perspective view showing another embodiment of the marker unit. In this embodiment, as in the example shown in FIG. 11, the marker portion 121 has a contact portion 122 formed in a state bent from the support wire 18 into a substantially L shape. Although it has a circular cross section and is in contact with the subject S on its outer peripheral surface, particularly here, as in the example shown in FIG. 10, the light guided by the support wire 18 passes through the contact portion 122. The light is emitted from the light exit surface 123 at the tip.

  In this configuration, similarly to the example shown in FIG. 10, the light emitted from the light exit surface 123 enters the subject S, and a high-luminance region A appears on the subject S, so that the subject S in the captured image appears. A region having a higher brightness than the periphery appears in the image, and a feature point is extracted at the position of the higher brightness region. For this reason, a feature point can be extracted more reliably at a position corresponding to the marker portion 121, and the alignment accuracy can be further improved.

  Further, in this configuration, as in the example shown in FIG. 11, the subject S comes into contact with the marker unit 121 because the contact surface has a substantially arc-shaped cross section on the outer peripheral surface of the contact unit 122. Can avoid damage.

  Further, in this configuration, the marker portion 121 can be formed simply by bending a single linear light guide member having an approximately circular cross section (optical fiber covered with a protective sheath). Manufacturing becomes easy.

  FIG. 13 is a perspective view showing another embodiment of the marker unit. In this embodiment, as in the example shown in FIG. 12, the marker portion 131 has a contact portion 132 that is bent from the support wire 18 into a substantially L shape, and the contact portion 132 is substantially Although it has a circular cross section and is in contact with the subject S on its outer peripheral surface, in particular, here, a fluorescent member 133 is attached to the contact portion 132. The fluorescent member 133 has a flat plate shape, and a chamfered portion 134 is formed on the contact surface side.

  In this configuration, similarly to the example shown in FIG. 4, the light guided by the support wire 18 enters the fluorescent member 133 through the contact portion 132 and the fluorescent member 133 emits light. The brightness of the image of the fluorescent member 133 is higher than that of the periphery, and feature points are extracted at the position of the region where the brightness is high. For this reason, it becomes possible to extract the feature points more reliably at the positions corresponding to the marker portions 131, and the alignment accuracy can be further improved.

  Further, in this configuration, as in the example shown in FIG. 12, the object S is contacted by a contact surface having a substantially arc-shaped cross section on the outer peripheral surface of the contact portion 132, and the contact surface side of the fluorescent member 133 is also chamfered. Since the portion 134 is formed, the subject S can be prevented from being damaged due to contact with the marker portion 131.

  FIG. 14 is a perspective view showing another embodiment of the marker unit. In this embodiment, the marker portion 141 includes a plurality (here, four) of fluorescent members 142 and a holding portion 143 that integrally holds the plurality of fluorescent members 142, and the holding portion 143 is a light beam. Light formed of a transmissive material and guided by the support wire 18 enters the fluorescent member 142 through the holding portion 143.

  The holding portion 143 is formed in a circular plate shape, and the support wire 18 is coupled to the center portion. The surface opposite to the support wire 18 is a contact surface that contacts the subject S. The plurality of fluorescent members 142 are arranged at equal intervals on the surface opposite to the contact surface in the outer peripheral portion of the holding portion 143. In this example, four fluorescent members 142 are arranged, but the number of fluorescent members 142 is not limited to this.

  In this configuration, as in the example shown in FIG. 4, the fluorescent member 142 emits light by light incident on the fluorescent member 142 from the holding unit 143, so that the image of the fluorescent member 142 in the captured image has brightness from the periphery. The feature point is extracted at the position of the region where the brightness is high and the brightness is high. For this reason, it becomes possible to more reliably extract feature points at positions corresponding to the marker portions 141, and furthermore, a large number of feature points can be extracted according to the number of arranged fluorescent members 142. The alignment accuracy can be further increased.

  Further, in this configuration, since the holding portion 143 is formed in a plate shape and can be brought into contact with the subject S with a large contact area, the subject S can be prevented from being damaged due to contact with the marker portion 141.

  In this embodiment, one marking member is provided, but a plurality of marking members may be provided. As a result, more feature points can be extracted from the low-resolution image obtained by imaging, so that the alignment accuracy can be further improved.

  In the present embodiment, light is guided to the marker portion by the optical fiber constituting the support wire. However, it is also possible to provide a light source in the marker portion itself.

  Further, in this embodiment, the fluorescent member is provided in the marker portion, and the fluorescent member is caused to emit light with the light guided by the optical fiber constituting the support wire, but the fluorescent member or the reflecting member is provided in the marker portion, Fluorescence may be generated by the illumination light emitted from the illumination unit 16, or the illumination light may be reflected and detected. That is, as shown in FIG. 5, since the imaging range of the imaging unit 14 is illuminated by the illumination unit 16, if a reflecting member (for example, a white ceramic material) or a fluorescent member is provided on the marker unit 17, these members are provided. Can be detected. If comprised in this way, since the light source which supplies light to the marker part 17 becomes unnecessary, cost reduction can be achieved more.

  In the present embodiment, the example in which the image processing unit is connected to the endoscope via the controller has been described. However, the image data output from the endoscope is transmitted to the image processing unit via the communication network. In addition, the image data obtained by the endoscope may be recorded on a recording medium, and the image data of the recording medium may be read by the image processing unit.

  In the present embodiment, the marker unit itself emits light by causing the marker unit to emit light with a fluorescent member, but the marker unit itself may not emit light. In this case, if the marker portion is colored in a color that is significantly different from the color of the body tissue such as the organ that is the subject, for example, a color that is complementary to the color of the body tissue (for example, cyan that is a complementary color of red), Extraction of points becomes easy.

  In the present embodiment, the case where a high-resolution still image is obtained from a plurality of low-resolution images by super-resolution processing has been described. For example, a reference image is taken in the time axis direction with respect to a plurality of captured frame images. If the super-resolution processing is executed by selecting in order, a moving image is obtained as a result. That is, the present invention can be applied to moving images.

  Further, in the present embodiment, the description has been made on the premise that super-resolution processing is performed using a plurality of low-resolution images that have been displaced due to camera shake, but the present invention is not limited to camera shake, for example, the endoscope itself can be vibrated, The relative positional relationship between the imaging optical system and the image sensor may be changed by a simple mechanism such as a bimorph so that a low-resolution image is obtained by forcibly giving a pixel shift. In any case, if alignment is performed using the marker according to the present invention, a high-resolution image can be obtained without using a high-precision alignment mechanism such as a conventional pixel shift.

  An endoscope according to the present invention and an endoscope system provided with the endoscope are capable of performing super-resolution processing when acquiring a plurality of low-resolution images whose imaging positions necessary for super-resolution processing are shifted using camera shake. In particular, the present invention relates to an endoscope for imaging an inside of an observation object that cannot be directly observed from the outside, and an endoscope system including the endoscope. The present invention is useful as an endoscope suitable for performing super-resolution processing and an endoscope system equipped with the endoscope.

DESCRIPTION OF SYMBOLS 1 Endoscope 2 Controller 3 Image processing unit (image processing apparatus)
4 monitor 11 insertion part 12 tip part 13 imaging window 14 imaging part 16 illumination window (illumination part)
17, 101, 111, 121, 131, 141 Marker part 18 Support wire (support part)
19 Marking member 32 Super-resolution processing part 42 Positioning part 71, 102, 112, 122, 132 Contact part 72, 133, 142 Fluorescent member 103, 114, 123 Light exit surface 143 Holding part S Subject

Claims (9)

An insertion part to be inserted inside the observation object;
An illumination unit that is provided in the insertion unit and illuminates a subject inside the observation target;
An imaging unit provided in the insertion unit for imaging the subject;
An endoscope comprising: a marker portion arranged on the subject; and a marking member having a support portion that protrudes from the insertion portion and supports the marker portion.   The endoscope according to claim 1, wherein the support portion is provided in the insertion portion so as to freely advance and retract.   The internal view according to claim 1, wherein the support portion is formed of an elastic material, and the marker portion is pressed against the subject by an elastic force generated by bending of the support portion. mirror. The support part is formed of a light guide material that guides light from a light source to the marker part,
The endoscope according to any one of claims 1 to 3, wherein the marker portion has a light exit surface that emits light guided by the support portion. The support part is formed of a light guide material that guides light from a light source to the marker part,
The endoscope according to any one of claims 1 to 3, wherein the marker unit includes a fluorescent member that emits light by the light guided by the support unit.   The marker portion has a contact portion formed in a substantially T-shape extending from the support portion, and the contact portion has a substantially circular cross section and contacts the subject at an outer peripheral surface thereof. The endoscope according to any one of claims 1 to 5, wherein the endoscope is configured as described above.   The marker portion has a contact portion formed to be bent in a substantially L shape from the support portion, and the contact portion has a substantially circular cross section so that the outer peripheral surface contacts the subject. The endoscope according to any one of claims 1 to 5, wherein the endoscope is provided.   The marker portion includes a plurality of the fluorescent members and a holding portion that integrally holds the plurality of fluorescent members, and the holding portions are formed of a light-transmitting material and guided by the support portion. The endoscope according to claim 5, wherein the light enters the fluorescent member through the holding portion. An endoscope according to any one of claims 1 to 8, and an image processing apparatus that performs super-resolution processing for generating a high-resolution image from a plurality of low-resolution images acquired by imaging by the imaging unit, Prepared,
The image processing device extracts a feature point at a position corresponding to the marker portion in the low-resolution image when aligning the plurality of low-resolution images in the super-resolution processing. Endoscope system.