WO2017017735A1 - Image processing device, display control method and program - Google Patents

Abstract

Provided are an image processing device, a display control method and a program that can prevent a lesion from being overlooked while reducing the labor burden when performing an examination. A processor 5 is provided with the following: an image generation unit 513 that generates a wide band image corresponding to a wide band filter and a narrow band image corresponding to a narrow band filter, on the basis of image data generated by an image pickup element 202a; a calculation unit 514 that calculates a characteristic amount which is included in the narrow band image generated by the image generation unit 513; a determination unit 515 that determines whether the characteristic amount calculated by the calculation unit 514 exceeds a threshold value; and a display control unit 516 that, on the basis of the determination results of the determination unit 515, emphasizes either the wide band image or the narrow band image and causes the same to be displayed on a display device 4.

Description

Image processing apparatus, display control method and program

The present invention relates to an image processing apparatus that performs image processing on image data generated by an imaging apparatus that captures an in-vivo image of a subject, a display control method, and a program.

In recent years, by providing a filter unit in which a plurality of wide band filters having wide band wavelength transmission characteristics in a visible light region and a narrow band filter having narrow band wavelength transmission characteristics are two-dimensionally arrayed in an imaging device, There is known an endoscope which simultaneously acquires a narrow band image capable of observing capillary and mucous membrane micropatterns and a normal color image (see Patent Document 1). In this technique, a normal image and a narrowband image are simultaneously displayed on one display device. As a result, it is possible to improve the diagnostic accuracy of the lesioned part by the user such as a doctor.

In addition, in a display device for displaying an image corresponding to image data generated by the capsule endoscope provided with the above-described image sensor, a normal image selection button for instructing display of a normal image, narrow band image There is known a technique for displaying either a normal image or a narrowband image by providing a narrowband image selection button for instructing display and the user selecting one of two buttons according to observation. (See Patent Document 2).

And combining a normal image including a subject image having information in a white light wavelength band and a narrow band image including a subject image having information in a specific narrow band wavelength band to generate a composite image; A technique for displaying this composite image on a display device is known (see Patent Document 3).

JP 2007-20880 A JP 2008-86759 A JP, 2011-194111, A

However, in Patent Document 1 described above, since the normal image and the narrow band image are simultaneously displayed, the user must perform an operation of the endoscope while making a diagnosis while comparing the two images. There was a problem that the load of the work of the user in was heavy.

Further, in Patent Document 2 described above, when the user finds a suspicious lesion while observing a normal image, the user switches the narrow band image from the normal image by selecting the narrow band image selection button each time. Also in this case, there is a problem that the load of the work of the user at the time of medical examination is large.

Furthermore, in the patent document 3 mentioned above, although the normal image and the narrow band image are compounded and displayed, since each of the normal image and the narrow band image includes important diagnostic information, one synthesized image is included. There was a risk that they could miss the lesion.

The present invention has been made in view of the above, and an image processing apparatus, a display control method, and a program capable of preventing a user from missing a lesion while reducing the burden of the user's work at the time of medical examination. Intended to provide.

In order to solve the problems described above and to achieve the object, an image processing apparatus according to the present invention includes a plurality of wide band filters transmitting light in the primary color wavelength band and a narrow band filter transmitting at least one narrow band light To form a predetermined array pattern, and an individual filter forming the array pattern is generated by an imaging element disposed at a position corresponding to any of a plurality of pixels arranged in a two-dimensional grid. An image processing apparatus for performing predetermined image processing on the acquired image data and displaying an image corresponding to the image data subjected to the image processing on a display device, based on the image data generated by the imaging element An image generation unit generating a wide band image corresponding to the wide band filter and a narrow band image corresponding to the narrow band filter; and the narrow band generated by the image generation unit The wide band image based on a determination result of a calculation unit that calculates a feature amount included in an image, a determination unit that determines whether the feature amount calculated by the calculation unit exceeds a threshold, and a determination result of the determination unit And a display control unit for emphasizing one of the narrowband images more than the other and displaying the narrowband image on the display device.

In the image processing apparatus according to the present invention as set forth in the above invention, the calculating unit calculates the feature amount based on an edge included in the narrow band image.

Further, in the image processing apparatus according to the present invention according to the above-mentioned invention, the calculation unit may set the feature amount based on a first edge included in the wide band image and a second edge included in the narrow band image. To calculate.

The image processing apparatus according to the present invention is characterized in that, in the above-mentioned invention, the calculation unit calculates the number of pixels whose pixel value exceeds a predetermined value in the narrow band image as the feature amount.

Further, in the image processing apparatus according to the present invention, in the above-mentioned invention, the display control section may display the narrow band image display area as the wide band image when the determination section determines that the feature quantity exceeds the threshold value. And a display area larger than the display area.

In the image processing apparatus according to the present invention, in the above-mentioned invention, the display control unit reduces the wide band image when the determination unit determines that the feature amount exceeds the threshold, and the reduced wide band An image is superimposed on the narrow band image and displayed on the display device.

Further, in the image processing apparatus according to the present invention, in the above-mentioned invention, the display control section blinks the narrow band image when the determination section determines that the feature amount exceeds the threshold, and the display apparatus It is characterized by being displayed on.

Further, in the image processing apparatus according to the present invention, in the above-mentioned invention, the display control unit determines that the narrow band image is either a character or a graphic when the determination unit determines that the feature amount exceeds the threshold. To be displayed on the display device.

The image processing apparatus according to the present invention is characterized in that, in the above-mentioned invention, the peak wavelength of light transmitted by the narrow band filter is between 395 nm and 435 nm.

The image processing apparatus according to the present invention is characterized in that, in the above-mentioned invention, the peak wavelength of light transmitted by the narrow band filter is between 790 nm and 820 nm.

Further, in the display control method according to the present invention, a predetermined array pattern is formed by using a plurality of wide band filters transmitting light in the primary color wavelength band and a narrow band filter transmitting at least one narrow band light. Each of the filters forming the array pattern performs predetermined image processing on the image data generated by the imaging device disposed at the position corresponding to any of the plurality of pixels disposed in a grid, and the image is generated. A display control method executed by an image processing apparatus that causes an image corresponding to processed image data to be displayed on a display device, wherein a wide band image corresponding to the wide band filter is generated based on the image data generated by the image sensor. An image generating step of generating a narrow band image corresponding to the narrow band filter, and the narrow band image generated in the image generating step The wide band image and the broadband image are calculated based on a determination step of calculating the included feature amount, a determination step of determining whether the feature amount calculated in the calculation step exceeds a threshold, and a determination result of the determination step. And D. a display control step of emphasizing one of narrowband images more than the other and displaying the same on the display device.

In the program according to the present invention, a predetermined arrangement pattern is formed using a plurality of wide band filters transmitting light in the primary color wavelength band and a narrow band filter transmitting at least one at least one narrow band light; Each of the filters forming the array pattern performs predetermined image processing on image data generated by an imaging element arranged at a position corresponding to any one of a plurality of pixels arranged in a grid pattern, In an image processing apparatus for displaying an image corresponding to image data subjected to image processing on a display device, a wide band image corresponding to the wide band filter and the narrow band filter based on the image data generated by the imaging device An image generation step of generating a corresponding narrowband image, and a feature included in the narrowband image generated in the image generation step Of the wide band image and the narrow band image on the basis of the calculation step of calculating L, the determination step of determining whether the feature value calculated in the calculation step exceeds the threshold, and the determination result of the determination step. And a display control step of emphasizing one of the other than the other and displaying the display on the display device.

According to the present invention, it is possible to prevent missing of a lesion while preventing the burden of work at the time of medical examination.

FIG. 1 is a view schematically showing an entire configuration of an endoscope system according to a first embodiment of the present invention. FIG. 2 is a block diagram showing the functions of the main parts of the endoscope system according to the first embodiment of the present invention. FIG. 3 is a view schematically showing a configuration of a color filter according to Embodiment 1 of the present invention. FIG. 4 is a view showing the relationship between the transmittance and the wavelength of each of the filters constituting the color filter according to Embodiment 1 of the present invention. FIG. 5 is a flowchart showing an outline of display processing performed by the processor according to the first embodiment of the present invention on a display device. FIG. 6 is a view schematically showing a filter used when the calculation unit of the processor according to the first embodiment of the present invention calculates the horizontal feature amount of the narrow band image. FIG. 7 is a view schematically showing a filter used when the calculation unit of the processor according to the first embodiment of the present invention calculates the feature amount in the vertical direction of the narrow band image. FIG. 8 is a diagram illustrating an example of an image displayed by the display device when the determination unit of the processor according to the first embodiment of the present invention determines that the value is smaller than the threshold. FIG. 9 is a diagram showing an example of an image displayed by the display device when the determination unit of the processor according to the first embodiment of the present invention determines that the value is not smaller than the threshold. FIG. 10 is a diagram showing an example of another image displayed by the display device when the determination unit of the processor according to the first embodiment of the present invention determines that the value is smaller than the threshold. FIG. 11 is a diagram showing an example of another image displayed by the display device when the determination unit of the processor according to the first embodiment of the present invention determines that it is not smaller than the threshold. FIG. 12 is a diagram showing an example of still another image displayed by the display device when the determination unit of the processor according to the first embodiment of the present invention determines that it is not smaller than the threshold. FIG. 13 is a diagram showing an example of still another image displayed by the display device when the determination unit of the processor according to the first embodiment of the present invention determines that it is not smaller than the threshold. FIG. 14 is a flowchart showing an outline of display processing performed by the processor according to the second embodiment of the present invention on a display device. FIG. 15 is a view schematically showing a configuration of a color filter according to Embodiment 3 of the present invention. FIG. 16 is a diagram showing the relationship between the transmittance and the wavelength of each of the filters constituting the color filter according to Embodiment 3 of the present invention. FIG. 17 is a flowchart showing an outline of display processing performed by the processor according to the third embodiment of the present invention on a display device.

Hereinafter, modes for carrying out the present invention (hereinafter, referred to as “embodiments”) will be described. In the present embodiment, a medical endoscope system for capturing and displaying an image of a body cavity of a subject such as a patient will be described as an example. Further, the present invention is not limited by the following embodiments. Furthermore, in the description of the drawings, the same parts will be described with the same reference numerals.

Embodiment 1
[Configuration of Endoscope System]
FIG. 1 is a view schematically showing an entire configuration of an endoscope system according to a first embodiment of the present invention.

The endoscope system 1 illustrated in FIG. 1 includes an endoscope 2 (endoscope) that captures an in-vivo image of a subject by inserting the tip into a body cavity of the subject, and a tip of the endoscope 2 The light source device 3 generates illumination light emitted from the light source, the display device 4 displays an image corresponding to the image data captured by the endoscope 2, and the in-vivo image captured by the endoscope 2 is subjected to predetermined image processing And a processor 5 (control device) that controls the operation of the entire endoscope system 1 as a whole. In the first embodiment, the processor 5 functions as an image processing apparatus.

The endoscope 2 has an elongated insertion portion 21 having flexibility, an operation portion 22 connected to the proximal end side of the insertion portion 21 and receiving input of various operation signals, and an insertion portion from the operation portion 22 21 includes a universal cord 23 which extends in a direction different from the extending direction and incorporates various cables connected to the processor 5 and the light source device 3.

The insertion portion 21 is connected to a proximal end side of the bending portion 25 and a distal end portion 24 having a built-in imaging device (imaging portion) described later, a bendable bending portion 25 formed of a plurality of bending pieces, and flexibility And a flexible flexible tube portion 26 having a property.

The operation unit 22 includes a bending knob 221 that bends the bending unit 25 in the vertical and horizontal directions, a treatment instrument insertion unit 222 that inserts a treatment tool such as a biological forceps, a laser knife and an inspection probe into a body cavity, the light source device 3, In addition to the processor 5, there are provided a plurality of switches 223 which are operation input units for inputting operation instruction signals of peripheral devices such as air supply means, water supply means, gas supply means and the like. The treatment tool inserted from the treatment tool insertion portion 222 is exposed from the opening (not shown) via the tip portion 24.

The universal cord 23 incorporates at least a light guide to be described later and a collecting cable. The universal cord 23 has a connector portion 27 (see FIG. 1) which is detachable from the light source device 3. The connector portion 27 has a coiled coil cable 27a extended, and has an electrical connector portion 28 detachably attachable to the processor 5 at the extended end of the coil cable 27a. The connector unit 27 is internally configured using an FPGA (Field Programmable Gate Array).

The light source device 3 is configured using, for example, a halogen lamp or a white LED (Light Emitting Diode). Under the control of the processor 5, the light source device 3 emits illumination light from the tip end side of the insertion portion of the endoscope 2 toward the subject.

Under the control of the processor 5, the display device 4 displays an image corresponding to an image signal subjected to image processing by the processor 5 and various information related to the endoscope system 1. The display device 4 is configured using a liquid crystal, a display panel such as an organic EL (Electro Luminescence), or the like.

The processor 5 performs predetermined image processing on the RAW image data input from the endoscope 2 and outputs the raw image data to the display device 4. The processor 5 is configured using a CPU or the like.

Next, the function of the main part of the endoscope system 1 will be described. FIG. 2 is a block diagram showing the function of the main part of the endoscope system 1. The details of the configuration of each part of the endoscope system 1 and the paths of electric signals in the endoscope system 1 will be described with reference to FIG.

[Configuration of Endoscope]
First, the main part of the endoscope 2 will be described.
As shown in FIG. 2, the endoscope 2 includes an optical system 201, an imaging unit 202, an A / D conversion unit 203, and a light guide path 204.

The optical system 201 receives the reflected light of the illumination light emitted by the light source device 3 on the imaging surface of the imaging unit 202 to form an object image. The optical system 201 is configured using one or more lenses, a prism, and the like.

The imaging unit 202 receives an object image formed on the light receiving surface by the optical system 201 under the control of the processor 5 and performs photoelectric conversion to generate image data (RAW image data) of the object. The generated image data is output to the A / D converter 203. Specifically, under the control of the processor 5, the imaging unit 202 captures an image of the subject at a reference frame rate, for example, a frame rate of 60 fps, and generates image data of the subject. The imaging unit 202 photoelectrically converts light received by a plurality of pixels arranged in a two-dimensional grid, and generates an electrical signal. An imaging element 202 a such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) A plurality of first band-pass filters (hereinafter referred to as "wide band filters") for transmitting light of the primary color wavelength band, and a narrow band having a maximum value outside the range of the wavelength band of light passing through the first band-pass filter And a color filter 202 b in which a filter unit including a second band-pass filter (hereinafter referred to as “narrow band filter”) for transmitting light of the above is arranged to correspond to a plurality of pixels.

FIG. 3 is a view schematically showing the configuration of the color filter 202b. As shown in FIG. 3, the color filter 202 b includes two wide band filters R transmitting red components, eight wide band filters G transmitting green components, two wide band filters B transmitting blue components, and a narrow band. The four narrow band-pass filters X1 transmitting the light of (1) are configured using a filter unit in which a predetermined arrangement pattern is formed. The color filters 202b are disposed at positions corresponding to any of the plurality of pixels of the imaging element 202a in which the individual filters forming the above-described array pattern are arrayed in a secondary grid. Here, the peak wavelength of the wavelength band of the narrow band light in the first embodiment is between 395 nm and 435 nm. The image data generated by the imaging unit 202 using the color filter 202 b configured as described above is subjected to predetermined image processing (for example, interpolation such as demosaicing processing) by the processor 5 described later, whereby a color wide band is generated. Converted into images and narrowband images.

FIG. 4 is a diagram showing the relationship between the transmittance and the wavelength of each of the filters constituting the color filter 202b. 4, the curve L _B represents the relationship between the transmittance and the wavelength of the broadband filter B, the curve L _G represents the relationship between the transmittance and the wavelength of the broad band filter G, the curve L _R is the transmittance of the broadband filter R The curve L _X1 shows the relationship between the transmittance and the wavelength of the narrowband filter X1. Further, in FIG. 4, the peak wavelength of the narrow band filter X1 is described as being between 395 nm and 435 nm.

As shown in FIG. 4, the spectral characteristics of the narrow band filter X1 are such that the width of the wavelength transmission band of the narrow band filter X1 is narrower than each of the wide band filter R, the wide band filter B, and the wide band filter G.

Returning to FIG. 2, the description of the configuration of the endoscope 2 will be continued.
The A / D conversion unit 203 performs A / D conversion on the analog image data input from the imaging unit 202, and outputs the digital image data subjected to the A / D conversion to the processor 5.

The light guide path 204 is configured using an illumination lens and a slide guide, and propagates the illumination light emitted by the light source device 3 toward a predetermined area.

Processor Configuration
Next, the main part of the processor 5 will be described.
The processor 5 includes an image processing unit 51, a recording unit 52, and a control unit 53.

The image processing unit 51 subjects the digital image data input from the endoscope 2 to predetermined image processing and outputs the image data to the display device 4. The image processing unit 51 includes a separation unit 511, a demosaicing unit 512, an image generation unit 513, a calculation unit 514, a determination unit 515, and a display control unit 516.

When digital RAW data is input from the endoscope 2, the separation unit 511 separates the RAW data into mosaic-like channels, and outputs the signal value of the RAW data separated to each of the channels to the demosaicing unit 512. Do.

The demosaicing unit 512 performs demosaicing processing using the signal value of each channel separated by the separation unit 511, thereby generating each of the R image, the G image, the B image, and the X1 image, and this R image, G Each of the image, the B image, and the X1 image is output to the image generation unit 513.

The image generation unit 513 generates a color wide-band image using the R image, the G image, and the B image generated by the demosaicing unit 512, and generates a pseudo image using the G image and the X1 image generated by the demosaicing unit 512. Generate a narrowband image of color.

The calculating unit 514 calculates the feature amount of the narrowband image generated by the image generating unit 513. Specifically, the calculation unit 514 calculates the feature amount based on the edge included in the narrowband image.

Determination unit 515 determines whether the feature amount of the narrowband image calculated by calculation unit 514 is smaller than a threshold, and outputs the determination result to display control unit 516.

The display control unit 516 controls the display mode of the display device 4. Further, based on the determination result of the determination unit 515, the display control unit 516 highlights one of the wide band image and the narrow band image generated by the image generation unit 513 and causes the display device 4 to display the image. Specifically, the display control unit 516 enlarges the display area of either the wide band image or the narrow band image generated by the image generation unit 513 and causes the display device 4 to display the enlarged image. For example, when the determination unit 515 determines that the threshold is exceeded by the determination unit 515, the display control unit 516 causes the display device 4 to display the display area of the narrowband image larger than the display area of the broadband image.

The recording unit 52 records image data generated by the endoscope 2, a program executed by the endoscope system 1, and information being processed. The recording unit 52 is configured using a non-volatile memory, a volatile memory, or the like.

The control unit 53 controls the respective units constituting the endoscope system 1 in an integrated manner. The control unit 53 is configured using a CPU or the like. The control unit 53 controls the emission timing of the illumination light of the light source device 3, the imaging timing of the imaging unit 202 of the endoscope 2, and the like.

Processor Processing
Next, display processing performed by the processor 5 on the display device 4 will be described. FIG. 5 is a flowchart showing an outline of the display processing performed by the processor 5 on the display device 4.

As shown in FIG. 5, when the digital RAW data is input from the endoscope 2, the separation unit 511 separates the RAW data into mosaic-like channels (step S101). Specifically, the separation unit 511 separates the signal value of the RAW data for each channel corresponding to each of the wide band filter R, the wide band filter G, the wide band filter B, and the narrow band filter X1.

Subsequently, the demosaicing unit 512 generates each of the R image, the G image, the B image, and the X1 image by performing the demosaicing process using the signal value of each channel separated by the separation unit 511 (step S102). ). Here, the method of demonstration may be performed by a known linear interpolation, or may be performed on the signal values of the wide band filter R and the wide band filter B with reference to the signal value of the wide band filter G.

Thereafter, the image generation unit 513 generates a color wide-band image using the R image, the G image, and the B image generated by the demosaicing unit 512 (step S103), and the G image and the X1 image generated by the demosaicing unit 512 To generate a pseudo-color narrowband image (step S104).

Subsequently, the calculating unit 514 calculates the feature amount of the narrowband image generated by the image generating unit 513 (step S105). Specifically, the calculation unit 514 calculates the feature amount based on the edge included in the narrowband image. More specifically, the calculating unit 514 sets the pixel value of each pixel of the narrowband image as X (i, j), and calculates the edge of the narrowband image using a Sobel filter. The Sobel filter is a filter for calculating an edge by multiplying each of nine pixel values of upper, lower, left, and right around the target pixel of the narrow band image by coefficients and summing the results. For example, when the total value in the horizontal direction is gHS, the calculation unit 514 multiplies each of the nine pixel values in the upper, lower, left, and right around the target pixel of the narrowband image by the coefficient of the filter F1 shown in FIG. The horizontal total value gHS is calculated by summing the results. Further, when the total value in the vertical direction is gVS, the calculation unit 514 multiplies the respective coefficients of the filter F2 shown in FIG. 7 and sums up the results to calculate the total value gVS in the vertical direction. Thereafter, when the edge of the pixel of interest (center pixel) is g (i, j), the calculator 514 calculates the edge g (i, j) according to the following equation (1).
g (i, j) = (gHS ² + gVS ² ) ^1/2 (1)
Subsequently, the calculation unit 514 calculates the sum GHx of the edges of the narrowband image according to the following equation (2).
GHx = Σg (i, j) (2)
Here, Σ is the sum of all g (i, j).
As described above, the calculating unit 514 calculates the sum GHx of the edges of the narrowband image as the feature amount, based on the edges included in the narrowband image.

Subsequently, the determination unit 515 determines whether the feature amount of the narrowband image calculated by the calculation unit 514 is smaller than a threshold (step S106). If the determination unit 515 determines that the feature amount of the narrowband image calculated by the calculation unit 514 is smaller than the threshold (step S106: Yes), the processor 5 proceeds to step S107 described later. On the other hand, when the determination unit 515 determines that the feature amount of the narrowband image calculated by the calculation unit 514 is not smaller than the threshold (step S106: No), the processor 5 proceeds to step S108 described later.

In step S107, the display control unit 516 causes the display device 4 to display the wide band image generated by the image generation unit 513 larger than the narrow band image generated by the image generation unit 513. Specifically, as shown in FIG. 8, the display control unit 516 causes the display device 4 to display the wide band image P1 larger than the narrow band image P2 in the display area 40 of the display device 4. After step S107, the processor 5 proceeds to step S109 described later.

In step S108, the display control unit 516 causes the display device 4 to display the narrowband image generated by the image generation unit 513 larger than the wide band image generated by the image generation unit 513. Specifically, as shown in FIG. 9, the display control unit 516 causes the display device 4 to display the narrowband image P2 larger than the wide band image P1 in the display area 40 of the display device 4. As a result, when the feature amount of the narrowband image P2 exceeds the predetermined threshold, the user is focused on the narrowband image P2 because the narrowband image P2 is displayed larger and emphasized than the wide band image P1. It is possible to prevent missing of a lesion of a subject. After step S108, the processor 5 proceeds to step S109 described later.

In step S109, when the observation of the subject is ended (step S109: Yes), the processor 5 ends the present process. On the other hand, when the observation of the subject is not finished (step S109: No), the processor 5 returns to the above-described step S101.

According to the first embodiment of the present invention described above, when the display control unit 516 determines that the feature amount included in the narrow band image exceeds the threshold value by the determination unit 515, the narrow band image P2 is selected from the wide band image P1. The display area is enlarged and displayed on the display device 4. As a result, the narrow band image P2 is emphasized and displayed on the display device 4 only when necessary while the examination based on the wide band image P1 is based, so that the narrow band image P2 is focused, so the work at the time of the user's examination It is possible to prevent missing of the lesion while minimizing the burden of

Further, in the first embodiment of the present invention, when the display control unit 516 determines that the feature amount of the narrowband image P2 is smaller than the threshold by the determination unit 515, the wide band image P1 is displayed larger than the narrowband image P2 When the determination unit 515 determines that the feature amount of the narrowband image P2 is equal to or more than the threshold while displaying on the device 4, the narrowband image P2 is displayed on the display device 4 larger than the wide band image P1. The display method may be appropriately changed in accordance with the determination result of the determination unit 515. Specifically, when the determination unit 515 determines that the feature amount of the narrowband image P2 is smaller than the threshold, the display control unit 516 reduces the narrowband image P2, and the reduced narrowband image P2 is a wideband image. While superimposing on P1 and displaying in the display area 40 of the display device 4 (see FIG. 10), if the determining unit 515 determines that the feature amount of the narrowband image P2 is equal to or greater than the threshold, the broadband image P1 is reduced. The reduced wide band image P1 may be superimposed on the narrow band image P2 and displayed in the display area 40 of the display device 4 (see FIG. 11).

In the first embodiment of the present invention, the display control unit 516 may blink the image displayed by the display device 4 according to the determination result of the determination unit 515. For example, as shown in FIG. 12, when the display device 4 displays the wide band image P1 and the narrow band image P2 in parallel in the same display area 40, the determination unit 515 sets the feature amount of the narrow band image P2 to a threshold value. When it is determined as above, the edge T1 of the narrow band image P2 may be blinked and displayed on the display device 4 (FIG. 12 → FIG. 13). Of course, the display control unit 516 may cause the display device 4 to display each of the wide band image P1 and the narrow band image P2 in a blinking manner. As a result, the user can intuitively understand that the narrow band image P2 includes a lesion. Of course, the display control unit 516 may warn the fact that the feature amount of the narrowband image P2 is equal to or more than the threshold value by displaying characters, figures, and the like in the display area of the display device 4.

Second Embodiment
Next, a second embodiment of the present invention will be described. The endoscope system according to the second embodiment has the same configuration as the endoscope system 1 according to the first embodiment described above, and the process to be performed is different. In the following, processing performed by the endoscope system according to the second embodiment will be described. The same components as those in the first embodiment described above are denoted by the same reference numerals and description thereof is omitted.

Processor Processing
FIG. 14 is a flowchart showing an outline of display processing performed by the processor 5 according to the second embodiment of the present invention on the display device 4. In FIG. 14, steps S201 to S204 correspond to the above-described steps S101 to S104, respectively.

In step S205, the calculating unit 514 calculates the feature amount included in the narrow band image. Specifically, the calculating unit 514 calculates an edge by the same method as that of the first embodiment described above, and calculates the sum GHx of the luminance of the edge as a feature amount.

Subsequently, the calculation unit 514 calculates the feature amount based on the edge included in the wide band image (step S206). Specifically, the calculating unit 514 calculates the edge of the wide band image and calculates the sum GHg of the edges as the feature amount, as in the method of calculating the edge of the narrow band image according to the first embodiment described above.

After that, the calculation unit 514 calculates a value (GHx / GHg) obtained by dividing the feature amount of the narrowband image calculated in step S205 by the feature amount of the wide band image calculated in step S206 described above (step S207).

Subsequently, the determination unit 515 determines whether the value (GHx / GHg) calculated by the calculation unit 514 in step S207 described above is smaller than a threshold (step S208). If the determination unit 515 determines that the value calculated by the calculation unit 514 is smaller than the threshold (step S208: Yes), the processor 5 proceeds to step S209 described later. On the other hand, when the determination unit 515 determines that the value calculated by the calculation unit 514 is not smaller than the threshold (step S208: No), the processor 5 proceeds to step S210 described later.

Steps S209 to S211 correspond to the above-described steps S107 to S109, respectively.

According to the second embodiment of the present invention described above, when the display control unit 516 determines that the feature amount included in the narrow band image exceeds the threshold value by the determination unit 515, the wide band image P2 is selected from the wide band image P1. The display area is enlarged and displayed on the display device 4. As a result, the narrow band image P2 is emphasized and displayed on the display device 4 only when necessary while the examination based on the wide band image P1 is based, so that the narrow band image P2 is focused, so the work at the time of the user's examination It is possible to prevent missing of the lesion while minimizing the burden of

Third Embodiment
Next, a third embodiment of the present invention will be described. The third embodiment differs in the configuration of the color filter 202b in the endoscope system 1 according to the above-described first embodiment, and in the processing executed by the processor. In the following, after the configuration of the color filter according to the third embodiment is described, processing performed by the processor according to the third embodiment will be described. The same components as those in the first embodiment described above are denoted by the same reference numerals and description thereof is omitted.

FIG. 15 is a view schematically showing a configuration of a color filter 202c according to Embodiment 3 of the present invention. As shown in FIG. 15, the color filter 202c is configured using a filter unit of an arrangement in which four narrow band-pass filters X2 transmitting narrow band light are replaced by one set instead of the above-described narrow band-pass filter X1. . Here, the wavelength band peak wavelength of the narrow band light in the third embodiment is between 790 nm and 820 nm. The image data generated by the imaging element 202a using the color filter 202c configured in this way is subjected to predetermined image processing (for example, interpolation such as demosaicing processing) by the processor 5, whereby a color broadband image and an image are generated. It is converted to an infrared narrow band image.

FIG. 16 is a diagram showing the relationship between the transmittance and the wavelength of each of the filters constituting the color filter 202c. 16, the curve L _B represents the relationship between the transmittance and the wavelength of the broadband filter B, the curve L _G represents the relationship between the transmittance and the wavelength of the broad band filter G, the curve L _R is the transmittance of the broadband filter R The curve L _X2 shows the relationship between the transmittance of the narrowband filter X2 and the wavelength. Further, in FIG. 16, the peak wavelength of the narrow band filter X2 is described as being between 790 nm and 820 nm.

As shown in FIG. 16, the spectral characteristics of the narrow band filter X2 are such that the width of the wavelength transmission band is narrower than each of the wide band filter R, the wide band filter B and the wide band filter G, and has the maximum value of the transmission spectrum on the long wavelength side . Furthermore, the narrow band-pass filter X2 transmits light (infrared light) having the maximum value of the transmission spectrum outside the range of the wavelength band of light (visible light) transmitted through the wide band filter.

Processor Processing
Next, display processing performed by the processor 5 on the display device 4 will be described. FIG. 17 is a flowchart showing an outline of the display processing performed by the processor 5 on the display device 4.

In FIG. 17, steps S301 to S304 correspond to steps S101 to S104 in FIG. 5 described above.

In step S305, the calculating unit 514 calculates, as a feature amount, the number of pixels exceeding a predetermined value for the pixel value of each pixel of the narrowband image generated by the image generating unit 513.

Subsequently, the determination unit 515 determines whether the number of pixels calculated by the calculation unit 514 is smaller than a threshold (step S306). Here, the threshold is 20% with respect to the total number of pixels in the narrowband image. The threshold can be changed as appropriate. If the determination unit 515 determines that the number of pixels calculated by the calculation unit 514 is smaller than the threshold (step S306: Yes), the processor 5 proceeds to step S307 described later. On the other hand, when the determination unit 515 determines that the number of pixels calculated by the calculation unit 514 is not smaller than the threshold (step S306: No), the processor 5 proceeds to step S308 described later.

Steps S307 to S309 correspond to steps S107 to S109 in FIG. 5 described above.

According to the third embodiment of the present invention described above, the peak wavelength of light transmitted by the narrow band-pass filter X2 constituting the color filter 202c is between 790 nm and 820 nm, and the calculation unit 514 sets the pixel value to a predetermined value. Since the number of pixels exceeded is calculated as a feature amount, narrow band image is displayed by emphasizing the infrared narrow band image P2 and displaying it on the display device 4 only when necessary while making a consultation with the wide band image P1. By focusing on the user, it is possible to prevent the user from missing a lesion while minimizing the burden of work at the time of medical examination.

(Other embodiments)
In the present invention, the wide-band color filter is configured of the primary color filter, but for example, complementary color filters (Cy, Mg, Ye) transmitting light having complementary wavelength components may be used. Furthermore, even if a color filter (R, G, B, Or, Cy) configured by a primary color filter and a filter (Or, Cy) that transmits light having orange and cyan wavelength components is used. Good. Furthermore, a color filter (R, G, B, W) configured by a primary color filter and a filter (W) that transmits light having a white wavelength component may be used.

Further, in the present invention, the color filters are provided with narrow band filters that transmit one type of wavelength band, but a plurality of narrow band filters may be provided in the color filters. For example, the narrow band filter X1 of the first embodiment described above and the narrow band filter X2 of the third embodiment described above may be provided.

Further, in the present invention, the image processing apparatus has been described as a processor used as an endoscope system. However, for example, a capsule endoscope that can be inserted into a body cavity of a subject can be applied.

Furthermore, in the present specification, in the description of each operation flowchart described above, the operation is described using “first”, “next”, “follow”, “after”, etc. for convenience, but the operation is performed in this order Does not mean that it is essential to

Further, the method of each process by the image processing apparatus in the above-described embodiment, that is, the process shown in each flowchart can be stored as a program that can be executed by a control unit such as a CPU. In addition, memory cards (ROM cards, RAM cards, etc.), magnetic disks (floppy disks (registered trademark), hard disks, etc.), optical disks (CD-ROM, DVD, etc.), storage in external storage devices such as semiconductor memory etc. Can be distributed. Then, a control unit such as a CPU can read the program stored in the storage medium of the external storage device, and can execute the above-described processing by controlling the operation by the read program.

The present invention is not limited to the above-described embodiment and modification as it is, and at the implementation stage, the constituent elements can be modified and embodied without departing from the scope of the invention. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the above-described embodiment. For example, some components may be deleted from all the components described in the above-described embodiment and modifications. Furthermore, the components described in each embodiment and modification may be combined as appropriate.

Further, in the specification or the drawings, the terms described together with the broader or synonymous different terms at least once can be replaced with the different terms anywhere in the specification or the drawings. Thus, various modifications and applications are possible without departing from the spirit of the invention.

REFERENCE SIGNS LIST 1 endoscope system 2 endoscope 3 light source device 4 display device 5 processor 51 image processing unit 52 recording unit 53 control unit 201 optical system 202 imaging unit 202 a imaging device 202 a and 202 c color filter 203 A / D conversion unit 204 light guide path 511 separation unit 512 demosaicing unit 513 image generation unit 514 calculation unit 515 determination unit 516 display control unit

Claims (12)

A plurality of wide band filters transmitting light in the primary wavelength band and a narrow band filter transmitting at least one narrow band light form a predetermined array pattern, and the individual filters forming the array pattern Predetermined image processing is performed on image data generated by an imaging device arranged at a position corresponding to any of a plurality of pixels arranged in a two-dimensional grid, and the image data subjected to the image processing is handled. An image processing apparatus for displaying an image to be displayed on a display device;
An image generator configured to generate a wide band image corresponding to the wide band filter and a narrow band image corresponding to the narrow band filter based on the image data generated by the image sensor;
A calculation unit that calculates a feature amount included in the narrow band image generated by the image generation unit;
A determination unit that determines whether the feature value calculated by the calculation unit exceeds a threshold value;
A display control unit which causes one of the wide band image and the narrow band image to be displayed more emphatically than the other on the basis of the determination result of the determination unit;
An image processing apparatus comprising: The image processing apparatus according to claim 1, wherein the calculation unit calculates the feature amount based on an edge included in the narrow band image. The calculation unit calculates a first edge included in the wide band image and a second edge included in the narrow band image, and the calculation is performed based on the first edge and the second edge. The image processing apparatus according to claim 1, wherein a feature amount is calculated. The image processing apparatus according to claim 1, wherein the calculation unit calculates, as the feature amount, the number of pixels in which the pixel value exceeds a predetermined value in the narrow band image. The display control unit causes the display region of the narrowband image to be larger than the display region of the wide band image and causes the display device to display the display region when the determination unit determines that the feature amount exceeds the threshold. The image processing apparatus according to any one of claims 1 to 4, characterized in that The display control unit reduces the wide band image when the determination unit determines that the feature value exceeds the threshold, and the reduced wide band image is superimposed on the narrow band image and displayed on the display device. The image processing apparatus according to any one of claims 1 to 4, characterized in that: The display control unit blinks the narrow band image and causes the display device to display the narrow band image when the determination unit determines that the feature amount exceeds the threshold. The image processing apparatus according to any one of the preceding claims. When the determination unit determines that the feature quantity exceeds the threshold, the display control unit causes the display device to display the narrowband image with any one of characters and graphics superimposed thereon. The image processing apparatus according to any one of claims 1 to 4. The image processing apparatus according to any one of claims 1 to 8, wherein a peak wavelength of light transmitted by the narrow band filter is between 395 nm and 435 nm. The image processing apparatus according to any one of claims 1 to 8, wherein a peak wavelength of light transmitted by the narrow band filter is between 790 nm and 820 nm. A predetermined array pattern is formed using a plurality of wide band filters transmitting light in the primary color wavelength band and a narrow band filter transmitting at least one narrow band light, and the individual filters forming the array pattern are gratings. Predetermined image processing is performed on image data generated by an imaging element arranged at a position corresponding to any of a plurality of pixels arranged in a shape, and an image corresponding to the image data subjected to the image processing is A display control method executed by an image processing apparatus displayed on a display device,
An image generation step of generating a wide band image corresponding to the wide band filter and a narrow band image corresponding to the narrow band filter based on the image data generated by the image sensor;
Calculating at least one feature amount included in the narrow band image generated at the image generation step;
A determination step of determining whether the feature value calculated in the calculation step exceeds a threshold value;
A display control step of emphasizing either one of the wide band image and the narrow band image from the other and displaying it on the display device based on the determination result of the determination step;
A display control method comprising: A predetermined array pattern is formed using a plurality of wide band filters transmitting light in the primary color wavelength band and a narrow band filter transmitting at least one narrow band light, and the individual filters forming the array pattern are gratings. Predetermined image processing is performed on image data generated by an imaging element arranged at a position corresponding to any of a plurality of pixels arranged in a shape, and an image corresponding to the image data subjected to the image processing is In the image processing apparatus displayed on the display device,
An image generation step of generating a wide band image corresponding to the wide band filter and a narrow band image corresponding to the narrow band filter based on the image data generated by the image sensor;
Calculating at least one feature amount included in the narrow band image generated at the image generation step;
A determination step of determining whether the feature value calculated in the calculation step exceeds a threshold value;
A display control step of emphasizing either one of the wide band image and the narrow band image from the other and displaying it on the display device based on the determination result of the determination step;
A program characterized by causing

Images