JP2020110250A - Fundus image processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a fundus image processing apparatus capable of performing a vein identification process in a fundus image with higher accuracy. SOLUTION: A grayscale fundus image mainly composed of R component is created by using the R component of a color fundus image composed of three color components of RGB, and a grayscale fundus image mainly composed of R component is created by using G component. Based on the grayscale conversion means 10 to be performed, the grayscale fundus image mainly composed of R component, and the grayscale fundus image mainly composed of G component, the blood vessel enhancement mainly composed of R component is set with a higher brightness value as the probability of being a blood vessel region is higher. A blood vessel region enhancing means 20 for obtaining an image and a blood vessel-enhanced image mainly composed of G component, and a vein-enhanced image creating means 30 for obtaining a vein-enhanced image in which a portion having a high probability of being a vein is emphasized based on the blood vessel-enhanced image mainly composed of R component. And 40, there is an arterial-enhanced image creating means 40 for obtaining an arterial-enhanced image in which a portion having a high probability of being an artery is emphasized based on a blood vessel-enhanced image mainly composed of G component. [Selection diagram] Fig. 2

Description

The present invention relates to a fundus image processing technique, and more particularly to a technique for obtaining an image capable of identifying arteries and veins using the fundus image.

In so-called metabolic syndrome such as hypertension, dyslipidemia, diabetes and arteriosclerosis that are typical lifestyle-related diseases, blood pressure, lipids and blood glucose can be easily measured and self-examination is possible. However, regarding arteriosclerosis measurement (commonly known as blood vessel age test), at present, it is a cardiovascular specialist equipped with test equipment such as PWV test (pulse wave velocity), ABI test (ankle brachial blood pressure ratio), carotid artery echo test, etc. It is not possible to measure unless it is a clinic.

On the other hand, a method of easily measuring hypertension and arteriosclerosis by measuring the diameter of the fundus blood vessel using a fundus camera owned by an ophthalmology clinic is also known. The fundus is the only place in the human body where blood vessels can be directly observed, and assuming that the hardening of the fundus arteries progresses in the same manner as the arteries of the whole body, it is possible to measure arteriosclerosis by taking a fundus photograph. Fundus cameras are becoming smaller and cheaper, and special lenses that can be used with smartphones are already sold overseas, and the infrastructure for self-photographing fundus photographs is being established.

In view of such a situation, the applicant has a technique of emphasizing a blood vessel portion from a fundus image obtained by photographing (see Patent Document 1) and a technique of classifying a specified blood vessel region into one of arteriovenous (patented) Reference 2) was proposed.

JP, 2018-23602, A JP, 2018-102586, A

In the technique described in Patent Document 2, after using the technique of specifying the fundus blood vessels that is also used in Patent Document 1, a hue value is calculated for each pixel in the specified blood vessel region, and the hue value is fixed for each pixel. A method of performing attribute classification of arteriovenous according to a threshold value is used. Therefore, the attributes determined for each pixel are likely to vary. In addition, the fundus region is physically a spherical surface, and significant unevenness in brightness and hue occurs between the central part and the peripheral part of the fundus image that is two-dimensionally photographed with a single light source. Therefore, the blood vessel in the central portion is easily determined to be an artery, and the blood vessel in the peripheral portion is easily determined to be a vein. Therefore, even if the threshold value for determining the arteriovenous is manually adjusted, there is a region that is difficult to be appropriately separated. Therefore, there is a case where an operation of manually correcting the image for which the arteriovenous determination is performed is necessary.

Therefore, an object of the present invention is to provide a fundus image processing apparatus capable of performing vein identification processing in a fundus image with higher accuracy.

The present disclosure includes a plurality of means for solving the above problems, and if one example is given,
A device for processing a color fundus image to enhance a blood vessel region,
Grayscale conversion means for creating a grayscale fundus image mainly composed of R components by using R components of a color fundus image composed of three color components of RGB,
Blood vessel region emphasizing means for obtaining a blood vessel emphasized image mainly composed of R component, which has a higher luminance value as the probability of being a blood vessel region is higher, based on the grayscale fundus image mainly composed of R component;
A vein-enhanced image creating means for obtaining a vein-enhanced image in which a portion having a high probability of being a vein is emphasized based on the R-component-based blood vessel-enhanced image;
Have.

In addition, the present disclosure is
A method for a computer to process a color fundus image to create an image in which blood vessel regions are emphasized,
Creating a grayscale fundus image mainly composed of R components using R components of a color fundus image composed of three color components of RGB, and creating a grayscale fundus image mainly composed of G components using G components;
Based on the R-component-based grayscale fundus image and the G-component-based grayscale fundus image, the R-component-based blood vessel-enhanced image and the G-component-based image in which a higher brightness value is set as the probability of a blood vessel region is higher Obtaining a blood vessel weighted image of
Obtaining a vein-enhanced image in which a portion having a high probability of being a vein is emphasized based on the R-component-based blood vessel-enhanced image;
Obtaining an artery-emphasized image in which a portion having a high probability of being an artery is emphasized based on the blood vessel-emphasized image mainly composed of the G component;
Assigning the arterial weighted image to the R component and the vein weighted image to the B component to create a color arteriovenous identification image;
Have.

According to the present invention, it is possible to perform the vein identification process in the fundus image with higher accuracy.

1 is a hardware configuration diagram of a fundus image processing device 100 according to an embodiment of the present invention. It is a functional block diagram showing the composition of the fundus image processing device concerning one embodiment of the present invention. It is a flow chart which shows the processing outline of the fundus image processing device concerning one embodiment of the present invention. 4 is a functional block diagram showing details of a blood vessel region emphasizing means 20. FIG. It is a figure which shows the relationship between the pixel (x, y) on a gray scale fundus image and the neighboring pixel for calculating average value Mean (x, y). It is a flowchart which shows the detail of the linear component emphasis process of step S300. It is a figure which shows an example of the linear structuring element used by an opening process. It is a figure which shows an example of the circular structuring element used by an opening process. It is a figure which shows the concrete pixel value of a linear structuring element when direction d=1. It is a figure which shows the concrete pixel value of a linear structuring element when the direction d=2. It is a figure which shows the gray scale fundus image and the blood vessel emphasis image obtained by the process of the fundus image processing apparatus which concerns on this embodiment. It is a figure which shows the vein emphasis image, the arterial emphasis image, and the arteriovenous identification image obtained by the process of the fundus image processing apparatus which concerns on this embodiment. It is a figure for comparing the arteriovenous identification image obtained by the prior art and this embodiment in the case of a normal person. It is a figure for comparing the arteriovenous identification image obtained by the prior art and this embodiment in the case of glaucoma. It is a figure for comparing the arteriovenous identification image obtained by this embodiment with the prior art in the case of a cataract.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.
<1. Device configuration>
FIG. 1 is a hardware configuration diagram of a fundus image processing apparatus according to an embodiment of the present invention. The fundus image processing apparatus 100 according to the present embodiment can be realized by a general-purpose computer, and as shown in FIG. 1, a CPU (Central Processing Unit) 1 and a RAM (Random Access Memory) that is a main memory of the computer. 2, a large-capacity storage device 3 such as a hard disk and a flash memory for storing programs and data executed by the CPU 1, an instruction input I/F (interface) 4 such as a keyboard and a mouse, a data storage medium, etc. A data input/output I/F (interface) 5 for data communication with an external device and a display unit 6 which is a display device such as a liquid crystal display are provided and are connected to each other via a bus.

FIG. 2 is a functional block diagram showing the configuration of the fundus image processing apparatus according to this embodiment. In FIG. 2, 10 is a gray scale conversion means, 20 is a blood vessel region emphasis means, 30 is a vein weighted image generation means, 40 is an artery weighted image generation means, 50 is an arteriovenous identification image generation means, 60 is a fundus image storage means, 70 is an arteriovenous identification image storage means.

The grayscale conversion unit 10 performs a predetermined operation on the RGB three primary color components of the color fundus image, and performs negative/positive inversion so that the blood vessel candidate region has high brightness, thereby converting into two types of grayscale fundus images. I do. The blood vessel region emphasizing means 20 emphasizes the blood vessel region with respect to each of the grayscale fundus image mainly composed of the R component and the grayscale fundus image mainly composed of the G component, and a higher luminance value is set as the probability of being the blood vessel region is higher. Processing for obtaining a blood vessel emphasized image mainly composed of R component and a blood vessel emphasized image mainly composed of G component is performed.

The vein-enhanced image creating means 30 performs a predetermined product operation between the R-component-based blood vessel-enhanced image and the G-component-based blood vessel-enhanced image to create a grayscale vein-enhanced image. The artery-enhanced image creating means 40 performs a predetermined difference calculation between the R-component-based blood vessel-enhanced image and the G-component-based blood vessel-enhanced image to perform a process of creating a grayscale artery-enhanced image. The arteriovenous identification image creating means 50 performs a process of creating a color arteriovenous identification image by allocating a grayscale arterial emphasis image to the R component and a grayscale vein emphasis image to the B component.

In the grayscale conversion unit 10, the blood vessel region emphasizing unit 20, the vein emphasizing image creating unit 30, the arterial emphasizing image creating unit 40, and the arteriovenous identification image creating unit 50, the CPU 1 executes a program stored in the storage device 3. It is realized by The fundus image storage unit 60 is a storage unit that stores a color fundus image that is an object of extraction of a blood vessel region and is captured in color using a visible light/light source type fundus camera, and is realized by the storage device 3. When the color fundus image is read by the fundus image processing apparatus and is processed as it is, the RAM 2 functions as the fundus image storage unit 60. The color fundus image is image data recorded by three components of RGB, and is the image of the fundus of the subject. In this embodiment, a full-color fundus image recorded with 8-bit 256 gradations for each color of RGB is used. The arteriovenous vein identification image storage means 70 is a storage means for storing the arteriovenous vein identification image generated by the arteriovenous vein identification image creating means 50, and is realized by the storage device 3.

Each component shown in FIG. 2 is actually realized by installing a dedicated program in hardware such as a computer and its peripheral devices as shown in FIG. That is, the computer executes the contents of each means according to the dedicated program. In this specification, a computer means a device that has an arithmetic processing unit such as a CPU and is capable of data processing, and includes not only a general-purpose computer such as a personal computer but also a portable terminal such as a smartphone and a tablet. Including.

The storage device 3 shown in FIG. 1 is provided with a dedicated program for operating the CPU 1 and causing the computer to function as a fundus image processing device. By executing this dedicated program, the CPU 1 functions as the grayscale conversion unit 10, the blood vessel region emphasizing unit 20, the vein emphasizing image creating unit 30, the arterial emphasizing image creating unit 40, and the arteriovenous identification image creating unit 50. Will be realized. Further, the storage device 3 not only functions as the fundus image storage unit 60 and the arteriovenous identification image storage unit 70, but also stores various data necessary for processing as the fundus image processing device.

<2. Processing operation>
<2.1. Pretreatment>
First, a color fundus image to be processed is prepared. As a color fundus image, if there is an image file captured in color by a digital fundus camera, it can be used as it is. In addition, if it is an old one recorded on a photographic medium by an analog fundus camera, analog color negative/positive film, photographic paper, instant photo etc. that have been stored will be read in color by a scanner and digitalized. Get a color fundus image file. In general, a color fundus image is obtained by photographing in color using a visible light/light source type fundus camera. The acquired color fundus image is stored in the fundus image storage unit 60 of the fundus image processing apparatus. In this embodiment, a full-color image of R, G, and B component 8-bit 256 gradations is prepared as a color fundus image.

<2.2. Processing outline>
Next, the image forming method according to the embodiment of the present invention will be described together with the processing operation of the fundus image processing apparatus shown in FIGS. An image creating method according to an embodiment of the present invention processes a fundus image to create an image such as a vein-enhanced image, an artery-enhanced image, or an arteriovenous identification image. FIG. 3 is a flow chart showing an outline of processing of the fundus image processing apparatus according to one embodiment of the present invention and showing an image creating method according to one embodiment of the present invention. As described above, the color fundus image to be processed is full-color image data of 8-bit 256 gradations for each RGB color. Therefore, assuming that the color fundus image having the number of pixels Xs in the x direction and the number of pixels Ys in the y direction is c=0 (Red), 1 (Green), and 2 (Blue) indicating the color components, Image(x, y) , C)=0 to 255 (x=0,..., Xs-1; y=0,..., Ys-1; c=0, 1, 2).

First, the grayscale conversion unit 10 performs grayscale conversion on the color fundus image to create a grayscale fundus image (step S100).

The grayscale conversion means 10 performs grayscale conversion on the color fundus image Image(x, y, c) by generally performing the process according to the following [Formula 1].

[Formula 1]
Gr(x,y)=Image(x,y,0)*Wr+Image(x,y,1)*Wg+Image(x,y,2)*Wb

In the above [Formula 1], Image (x, y, 0) represents the R (red, red) component of the color fundus image, and Image (x, y, 2) is the color fundus image. Of these, the B (blue, blue) component is shown. Further, Wr represents the weight of the R component, Wg represents the weight of the G component, and Wb represents the weight of the B component. Wr, Wg, and Wb are all real numbers.

Further, the grayscale conversion means 10 performs negative/positive inversion on the image Gr(x, y) in the grayscale format by generally performing the processing according to the following [Equation 2] to obtain the grayscale fundus image Gray. Get (x,y). Negative/positive inversion means a process of converting a larger original pixel value into a smaller value. Here, the reason for performing the negative/positive inversion on the fundus image is to color the blood vessel candidate region of the grayscale fundus image in a pseudo color by increasing the brightness of the blood vessel candidate region and decreasing the brightness of the background non-blood vessel candidate region. This is to make it easier. By performing the negative/positive inversion, the background portions (non-blood vessel candidate areas) do not mix even if a plurality of grayscale fundus images are combined, so that the positional relationship of the blood vessel candidate areas can be easily grasped.

[Formula 2]
Gray(x,y)=255-Gr(x,y)

Since the wall of the fundus blood vessel is almost transparent, the arteriovenous veins have different colors due to the difference in the color of blood flowing inside. Arterial blood shows a bright red color due to hemoglobin of red blood cells that adsorb oxygen in the lungs, and venous blood shows bluish-purple color due to the removal of oxygen from hemoglobin of red blood cells after gas exchange in the periphery and the dissolution of carbon dioxide in the blood. .. Generally, in the RGB full-color fundus image of a healthy person, there is a significant difference in the luminance of the color-separating components in the order of R>G>>B, and the strongest R component has the characteristic that the contrast of red-rich arteries is low. The weakest B component is that the contrast of arteries rich in red is relatively high. Therefore, when the difference component between the R component and the B component is taken, an image in which the contrast of the artery is suppressed and the vein pattern is emphasized is obtained. On the other hand, the intermediate G component is characterized by high contrast in all blood vessels regardless of arteriovenous, and the G component is most suitable for uniformly extracting all fundus blood vessels. However, in the fundus image of a patient with cataract, the blood vessel image of the R component and the G component becomes unclear compared to a healthy person, and the B component is enhanced instead. Therefore, when the B component is subtracted from the G component and the R component, The pattern contrast may be improved. (As the pathology of cataract progresses, it becomes difficult to extract blood vessel patterns and identify arteries and veins from color fundus images, and angiographic examination is required). In this embodiment, two types of gray scale fundus images are created from color fundus images. The first is a grayscale fundus image mainly composed of G components, and the second is a grayscale fundus image mainly composed of R components.

The creation of the first grayscale fundus image mainly composed of the G component will be described. “Mainly G component” means mainly including the G component of the original color fundus image. Therefore, it may contain other R component or B component. In this embodiment, as the grayscale fundus image mainly composed of the G component, a grayscale fundus image including only the G component which does not include the R component and the B component is created. In the above [Formula 1], in the present embodiment, the grayscale image Gr(x, y) calculated with the weight of each component set to Wr=0.0, Wg=1.0, and Wb=0.0 is the G component. The main grayscale image Grg(x, y) is used. Since Wg=1.0, the G component is directly reflected in the grayscale image Grg(x, y). Since Wr=Wb=0.0, the R component and the B component are not reflected at all in the grayscale image Grg(x,y). Therefore, in the above [Formula 1], only the G component of each pixel is extracted to obtain the image Grg(x, y) in the gray scale format. As described above, generally, in a fundus image of a healthy person, all blood vessel images appear in the G component.

Further, the grayscale conversion means 10 gives the image Grg(x, y) in the grayscale format mainly composed of the G component as the image Gr(x, y) in the grayscale format in the above [Formula 2], thereby performing the negative/positive inversion. The grayscale fundus image Grayg(x, y) mainly including the G component is obtained. By performing the negative/positive inversion, the pixel value of the blood vessel region whose brightness is lower than that of the surroundings is converted to a high pixel value.

The second generation of the R-component-based grayscale fundus image will be described. “Mainly R component” means that mainly the R component of the original color fundus image is included. Therefore, other G component or B component may be included. In the present embodiment, as the grayscale fundus image mainly composed of the R component, a grayscale fundus image mainly composed of the R component in which the B component is reflected in the opposite direction is created. The grayscale conversion unit 10 performs the grayscale conversion on the color fundus image Image(x, y, c) by similarly performing the processing according to the above [Formula 1].

In the above [Formula 1], in the present embodiment, the weight of each component is set to Wr=1.0 and Wg=0.0, and Wb is calculated as a negative value in the range of −2.0 to 0.0. The scale image Gr(x, y) is a grayscale image Grr(x, y) mainly composed of R components. In the grayscale image Grr(x,y), the smaller Wb, the more the artery is emphasized. Since Wr=1.0, the R component is reflected as it is in the grayscale image Grr(x, y). Since Wg=0, the G component is not reflected at all in the grayscale image Grr(x, y). Since Wb is a negative value, the B component is reflected in the opposite direction in the grayscale image Grr(x,y). Wb may take a value of -2.0, and in this case, the value of twice the B component is reduced. However, since the value of the B component is small in the original color fundus image, the value of the grayscale image Grr(x, y) often does not become a negative value even if the value is doubled.

That is, in step S100, the grayscale conversion means 10 subtracts the B component of the color fundus image from the R component of the color fundus image by a predetermined ratio Wb to create a grayscale image mainly composed of the R component. ..

Further, the grayscale conversion means 10 gives the image Grr(x, y) in the grayscale format mainly composed of the R component as the image Gr(x, y) in the grayscale format in the above [Formula 2], thereby performing the negative/positive inversion. A grayscale fundus image Grayr(x, y) mainly including the R component is obtained.

By performing the processing according to the above [Formula 2] and performing the negative/positive inversion, the pixel value of the blood vessel region having a lower brightness than the surroundings is converted to a high pixel value. In this way, the grayscale fundus image Grayr(x, y) mainly composed of the R component is obtained.

When two types of gray scale fundus images are obtained, the blood vessel region emphasizing unit 20 emphasizes the blood vessel region with respect to each of the R component-based grayscale fundus image and the G component-based grayscale fundus image, and The higher the certain probability is, the higher the brightness value is set, and the processing to create the blood vessel emphasized image mainly composed of the R component and the blood vessel emphasized image mainly composed of the G component is performed. FIG. 4 is a functional block diagram showing details of the blood vessel region emphasizing means 20. In FIG. 4, 21 is an image flattening means, 22 is a linear component enhancing means, and 23 is a pixel gradation converting means. As shown in FIG. 4, the blood vessel region enhancing unit 20 includes an image flattening unit 21, a linear component enhancing unit 22, and a pixel gradation converting unit 23.

The image flattening unit 21 calculates an average value of pixel values (luminance values) of neighboring pixels included in a predetermined range for each pixel of the grayscale fundus image, and calculates the pixel value of the pixel from the predetermined value. By adding the values obtained by subtracting the average value, a flattened image in which the brightness of all the blood vessel candidate regions is made uniform above a certain level is created. The linear component enhancing means 22 performs an opening process on the flattened image using a predetermined structuring element to enhance the linear components included in all the blood vessel candidate regions, and the granular components not included in the blood vessel candidate regions. The process of creating a linear component emphasized image in which is suppressed. The pixel gradation converting means 23 converts the pixel values that do not satisfy the predetermined condition, that is, the pixel values of the pixels of the background portion that are not included in the blood vessel candidate region equal to or less than the predetermined threshold value, in the linear component-enhanced image after conversion. Of the blood vessel-emphasized image is replaced with the minimum value, and the pixel value of the pixel included in the blood vessel candidate region satisfying the predetermined condition, that is, the pixel value exceeding the predetermined threshold value is near the minimum gradation value. The gradation of the pixel is corrected so as to fall within the range from the value of to the maximum value, and a process of creating a blood vessel emphasized image (Tray) is performed.

First, the image flattening means 21 calculates an average value of pixel values of neighboring pixels included in a predetermined range, subtracts the average value from the pixel values of the pixels, and adds a predetermined uniform value Mid. , A flattened image is created (step S200). The reason why the predetermined uniform value Mid is added is to prevent the pixel value from becoming a negative value. In step S200, the same processing is performed on both the G component-based grayscale fundus image Grayg(x, y) and the R component-based grayscale fundus image Grayr(x, y). Hereinafter, the G component-based grayscale fundus image Grayg(x, y) and the R component-based grayscale fundus image Grayr(x, y) are collectively referred to as a grayscale fundus image Gray(x, y). In the following processing, the processing performed on Gray(x, y) indicates the processing performed on each of Grayg(x, y) and Grayr(x, y). In step S200, a grayscale flattened image Gray'(x, y) is obtained by performing processing according to the following [Equation 5].

[Formula 5]
Mean(x,y)=[Σ _j=-m+1,m Σ _i=-m+1,m Gray(x+i,y+j)}/(4m ² )
Gray'(x,y)=Gray(x,y)-Mean(x,y)+Mid

In the above [Formula 5], m is an integer of 1 or more, and the subscript “j=−m+1,m” “i=−m+1,m” of j is j from −m+1 to m, and i is It indicates that the sum of 4m ² pixels from −m+1 to m is calculated. That is, Mean (x, y) represents the average value of the pixel values of the neighboring pixels included in the range of the front, rear, left, and right m pixels from the pixel (x, y). The number of neighboring pixels is preferably in the range of 1/3000 to 1/7000 of all pixels of the image. In this embodiment, m=8 and the number of neighboring pixels is 256 (=4 m ² =2 m×2 m). This corresponds to about 1/50,000 when the gray scale fundus image Gray(x, y) has 1280×1024 pixels.

Using this average value Mean(x, y), the average value Mean(x, y) is subtracted from the pixel value Gray(x, y) of each pixel (x, y) by the second equation of [Equation 5]. A flattened image Gray'(x, y) is obtained by adding a predetermined value Mid to the calculated value. The predetermined value Mid is a median value of gradation that can be taken by the pixel. In the present embodiment, since the image can take a value of 0 to 255, there are two median values of 127 and 128, but Mid=128 for convenience.

When calculating the average value Mean(x, y), the sum of the pixel values of the neighboring pixels may be directly obtained as shown in the first expression of [Equation 5], but this method is used. , The processing load is high. Therefore, in the present embodiment, the image flattening unit 21 is configured to calculate the average value Mean(x, y) at high speed with a small processing load by executing the processing according to the following [Equation 6]. There is.

[Formula 6]
When j=0, S(i,0)=Σ _ii=0,i Gray(ii,0)
When j>0, S(i,j)=S(i,j−1)+Σ _ii=0,i Gray(ii,j)
Mean(x, y)=[S(x+m, y+m)-S(x-m, y+m)-S(x+m, ym)+S(x-m, ym)]/(4m< ² >).

In the first expression of the above [Equation 6], S(i,j) is the pixel value Gray of the rectangular area from the first pixel (upper left corner x=0, y=0 of the image) to the pixel (i,j). It is the sum of (i+1)×(j+1) pieces of (ii, jj).

FIG. 5 is a diagram showing a relationship between a pixel (x, y) on the gray scale fundus image Gray(x, y) and a neighboring pixel for calculating the average value Mean(x, y). As surrounded by a large thick frame, in order to calculate the average value Mean(x, y), a total of 2 m×the upper left pixel (x−m+1, y−m+1) to the lower right pixel (x+m, y+m). It is necessary to add 2m pixel values. At this time, the total sum value S(x, y) of the pixel values of the (x+1)×(y+1) rectangular pixels from the first pixel of the image to the pixel (x, y) is calculated as It is calculated using a formula. Then, the sum of the pixel values of the 2m×2m rectangular areas is the sum of the pixel values of the (x+m+1)×(y+m+1) rectangular areas of the first pixel (0,0) to the lower right pixel (x+m,y+m). From the sum S(x+m, y+m) to the top pixel (0,0) to the pixel (x+m,y−m) at the upper right, the sum S((++m+1)×(y−m+1) of pixel values of rectangular areas x+m, y−m) and the sum S(x of pixel values of (x−m+1)×(y+m+1) rectangular areas from the first pixel (0, 0) to the lower left pixel (x−m, y+m). -M, y+m) is subtracted to obtain pixel values of (x-m+1) x (y-m+1) rectangular areas from the first pixel (0, 0) to the upper left pixel (x-m, ym). Is equivalent to the sum of the sums S(x-m, ym).

Then, the average value Mean(x, y) is added or subtracted from the total sum value of the pixels in the rectangular area from the first pixel (0, 0) to the specified four pixels, as shown in the second equation of [Equation 6]. It can be calculated at high speed with a small processing load. Here, the end pixels of the four rectangular areas to which the sum total value is added and subtracted are surrounded by small thick frames (x+m, y+m), (x−m, y+m), (x+m, ym) in FIG. There are 4 pixels of (x-m, ym). The average value Mean (x, y) of the neighboring pixels is calculated from each pixel (x, y) of the gray scale fundus image Gray (x, y), and a predetermined value Mid is added to the flattened image Gray' (x, y). y) is obtained. That is, a flattened image Grayg′(x,y) mainly composed of the G component and a flattened image Grayr′(x,y) mainly composed of the R component are obtained. In addition, in FIG. 3, it is described as a flattened image (G) and a flattened image (R), respectively.

Eventually, the image flattening unit 21 executes (0, 0) for the grayscale fundus image Gray(x, y) whose image size is Xs×Ys by executing the process according to [Equation 6]. 0) to each pixel (x, y), a process of calculating a sum value S(x, y) of pixel values of (x+1)×(y+1) rectangular areas is performed, and S(x+m, y+m)−S is performed. The value obtained by dividing the value of (x−m, y+m)−S(x+m, ym)+S(x−m, ym) by 2 m×2 m is calculated as an average value. Although an extra process for calculating the cumulative value S(x, y) in each pixel (x, y) in advance is added, this calculation can be performed only by Xs×Ys addition operations. On the other hand, if an average value Mean(x, y) in each pixel (x, y) is calculated without using the cumulative value S(x, y), 2m×2m additions are performed for each pixel. Calculation is required, and total Xs×Ys×2m×2m additions are required. On the other hand, if the cumulative value S(x, y) is used, the addition/subtraction of 4 times is required for each pixel, and the total Xs×Ys×5 additions/subtractions including the calculation of the cumulative value of the entire image are completed. To do.

After the flattened image is obtained, the linear component emphasizing unit 22 then performs an opening process (contraction/expansion process) on the flattened image to emphasize the linear component, and the linear component emphasized image. Is obtained (step S300). FIG. 6 is a flowchart showing the processing operation of the linear component enhancement in step S300.

First, the linear component emphasizing unit 22 performs an opening process using eight types of linear structuring elements on the flattened image obtained by the image flattening unit 21 (step S310). In the opening process in step S310, a plurality of linear structuring elements are used to create a plurality of linear opening images. In this embodiment, eight linear structuring elements are used as the designated structuring element. This is a linear structuring element, which is a linear structuring element whose directions differ in eight directions. FIG. 7 is a diagram showing an example of the linear structuring element used in this embodiment.

The linear structuring element shown in FIG. 7 serves as a mask and is therefore a binary image. Therefore, in FIG. 7, the value of the pixel described as “0” is “0”, but the value of the pixel described as a numerical value other than “0” is “1” according to each of the eight types of directions d. ", and "0" in the other directions. The numbers other than "0" in FIG. 7 indicate the directions. “1” is the horizontal direction (the horizontal direction in the drawing), and as the number increases by 1, the direction changes at intervals of 22.5 degrees. As for “2”, “4”, “6”, and “8”, the columns of the same number cannot be arranged in a straight line due to the grid-like arrangement of the pixels, and thus are arranged in the neighboring partitions. Three-digit numbers such as "234" and "678" are three directions of "2" "3" "4" and three directions of "6" "7" "8", respectively. This indicates that the pixel value is “1”. Note that "9" indicates the central pixel, and the value of the central pixel is always "1".

As shown in FIG. 7, the linear structuring element has a length twice the radius N of the circular structuring element, a pixel width of 1 pixel, and is arranged at least in the vicinity of a straight line at intervals of 22.5 degrees. Will be defined. Actually, it is preferable that the continuous pixels to which the pixel value “1” is given in each direction be a straight line, but when the total number of pixels of the linear structuring element is small, it is not necessarily a straight line. Therefore, in the example of FIG. 7, in the directions indicated by “2”, “4”, “6”, and “8”, the pixel having the pixel value “1” is not in the straight line but in the vicinity of the straight line at the interval of 22.5 degrees. Will be placed.

The linear structuring element that is actually used has the condition of adding a radius of 7 pixels from the center. That is, it is the logical product (AND) of the linear structuring element shown in FIG. 7 and the circular structuring element shown in FIG. FIG. 8 is a diagram showing an example of the circular structural element used in the present embodiment. As shown in FIG. 8, in this embodiment, a circular structuring element in the form of a 15×15 mask image in which a radius of 7 pixels, that is, a distance of 7 pixels or less from the center is effective is used. As shown in FIG. 8, among the 15×15 pixels, the pixel value at a distance of 7 pixels or less from the center is “1”, and the pixel value of the other pixels exceeding the distance 7 from the center is “0”. By taking the logical product (AND) of the linear structuring element shown in FIG. 7 and the circular structuring element shown in FIG. 8, for example, when the direction d=1, the binary mask as shown in FIG. An image is obtained. When the direction d=2, a binary mask image as shown in FIG. 10 is obtained. In FIGS. 9 and 10, the pixel indicated by “1” is the reference pixel. That is, in the binary image of (2N+1)×(2N+1) pixels in which the reference pixel referred to in the opening process is defined, the reference pixel is defined inside the circle having the radius N, and the pixel width is 2N+1 and the pixel width is Eight directions are defined by arranging one pixel at least in the vicinity of a straight line at intervals of 22.5 degrees.

The linear structuring element shown in FIG. 7 is M(d, u, v)={0, 1} (d=1,..., 8; u=−N,..., 0,..., N). V=-N,..., 0,..., N). Note that N represents a valid radius, and in the example of FIG. 7, N=7. Further, u indicates the horizontal axis in FIG. 7, and v indicates the vertical axis in FIG. 7, respectively. The linear component emphasizing unit 22 uses the linear structuring element M(d, u, v) to perform the process according to the above [Equation 5] and obtains the flattened image Gray′(x, y). On the other hand, the process according to the following [Formula 7] is executed to obtain the contracted image Eray(x, y).

[Formula 7]
Eray(d,x,y)=MIN _{u=-N,N;v=-N,N} [(255-M(d,u,v)*254)*(Gray'(x+u,y+v)+1)- 1]

In the above [Formula 7], MIN indicates that it takes a minimum value, and the subscript “u=−N,N;v=−N,N” of MIN indicates that u and v are from −N to N. It indicates that the calculation in the range is performed. That is, for each pixel Gray'(x, y) of "x=N,..., Xs-N-1; y=N,..., Ys-N-1", "u=-N. , N;v=−N,N″, the conversion is performed to the minimum pixel value of M(u,v)=1 in the adjacent pixels. Therefore, in the example of FIG. 7, the calculation is performed using 225 pixels with (2N+1)×(2N+1) pixels of the linear structuring element, where N=7, and the minimum value is given as Eray(x,y). ..

When the contracted image Eray(d,x,y) is obtained by executing the processing according to the above [Formula 7], the Eray(d,x,y) is replaced with Gray'(x,y), and The processing according to [Equation 7] can be repeatedly executed. The number of repetitions can be set appropriately. However, in order to suppress image quality deterioration, it is usually set to once and not repeated. By performing the process in each of the eight directions d, eight contraction images Eray(d, x, y) with d=1 to 8 are obtained.

When the contraction processing is repeatedly executed and eight kinds of contraction images Eray(d, x, y) are obtained, then the linear component enhancing means 22 uses the linear structuring element used for the contraction processing as the designated structuring element. And perform expansion processing. Specifically, using the linear structuring element M(d,u,v), the process according to the following [Equation 8] is performed on the contracted image Eray(d,x,y), and after the expansion, The image Dray(d,x,y) of is obtained.

[Formula 8]
Dray(d,x,y)=MAX _{u=-N,N;v=-N,N} [M(d,u,v)×Eray(d,x+u,y+v)]

In the above [Formula 8], MAX indicates that it takes the maximum value, and the subscript “u=−N,N;v=−N,N” of MAX indicates that u and v are from −N to N. It indicates that the calculation in the range is performed. That is, for each pixel Eray(d, x, y) of "x=N,..., Xs-N-1; y=N,..., Ys-N-1", "u=- The conversion is performed to the maximum pixel value of M(d, u, v)=1 in the adjacent pixels of N, N; v=-N, N". Therefore, in the example of FIG. 7, the calculation is performed using 225 pixels as (2N+1)×(2N+1) pixels of the linear structuring element, and the maximum value is given as Dray(d,x,y).

When the expanded image Dray(d,x,y) is obtained by executing the processing according to the above [Equation 8], this Dray(d,x,y) is replaced with Eray(d,x,y). Thus, the processing according to the above [Formula 8] can be repeatedly executed. The number of repetitions needs to be the same as the number of repetitions of the contraction processing, and normally the contraction processing is set to 1 and the repetition is not performed. By performing the process in each of the eight directions d, eight types of expanded images Dray(d, x, y) with d=1 to 8 are obtained. The image obtained as a result of the contraction and expansion is obtained as a linear opening image Dray(d,x,y).

Next, the linear component emphasizing unit 22 performs a process of creating an image having the maximum value of the eight types of linear opening images (step S320). Specifically, the processing according to the following [Equation 9] is executed to obtain the grayscale linear component emphasized image Lray(x, y) which is the image of the maximum value.

[Formula 9]
Lray(x,y)=MAX _d=1,8 Delay(d,x,y)

In the above [Formula 9], MAX indicates that it takes the maximum value, and the subscript “d=1, 8” of MAX indicates that all eight types of linear opening images Dray(d, x, y) It indicates that calculation is performed. That is, the process of acquiring the maximum value of the eight types of linear opening images Dray(d, x, y) is performed for each pixel (x, y). As a result, the linear component emphasized image Lray(x, y), which is the image with the maximum value, is obtained. That is, the linear component emphasized image Lrayg(x, y) mainly composed of the G component and the linear component emphasized image Lrayr(x, y) mainly composed of the R component are obtained. In addition, in FIG. 3, it is described as a linear component emphasized image (G) and a linear component emphasized image (R), respectively. In the present embodiment, since processing is performed on eight types of linear opening images according to eight types of directions, it is possible to evenly suppress the extraction of granular components in each direction changed at an equal interval of 22.5 degrees. Since the number of images to be created is only eight according to the direction, the load of arithmetic processing can be suppressed.

After the linear component emphasized image is obtained, the pixel gradation conversion unit 23 then performs gradation conversion (contrast correction) on the linear component emphasized image to obtain a blood vessel emphasized image (step S400). Specifically, first, the pixel gradation conversion unit 23 has a frequency distribution of the pixel value vb for all Xs×Ys pixels of the linear component emphasized image Lray(x, y) having a value of 0 to 255. H(vb) (vb=0,..., 255) is calculated. Then, the minimum value vbmin that satisfies the condition shown in the following [Formula 10] is obtained.

[Formula 10]
Σ _vb=0,vbmin H(vb)≧(Xs×Ys)×α

In the above [Formula 10], the subscript “vb=0, vbmin” of Σ indicates that the sum of vb from 0 to vbmin is obtained. Therefore, the condition shown in the above [Formula 10] indicates that, out of the total number of pixels (Xs×Ys), pixels having a ratio α×100% have a pixel value vbmin or less. As the ratio α, it is necessary to give a ratio of regions other than blood vessels included in the fundus image, and this ratio varies for each acquired fundus image, but is 0.7 to 0.9 on average, typical As a typical example, it is preferable to set 0.8. After all, by using [Equation 10], the minimum pixel value vb= of the linear component emphasized image is set as the threshold value of the pixel value for discriminating the non-blood vessel candidate region and the blood vessel candidate region included in the linear component emphasized image. It is possible to specify the minimum pixel value vbmin among the pixel values in which the total number of pixels counting from 0 exceeds a predetermined ratio α of the total number of pixels of the linear component emphasized image.

When the minimum value vbmin satisfying the condition shown in the above [Formula 10] is obtained, next, the pixel gradation converting means 23, x=0,..., Xs−1; y=0,..., Ys. The processing according to the following [Formula 11] is executed for all the pixels of −1, and the contrast is corrected to obtain the blood vessel emphasized image Tray(x, y).

[Formula 11]
Tray(x,y)={Lray(x,y)-vbmin}×255×β/(255-vbmin)

In the above [Formula 11], β is a brightness scaling value. The brightness scaling value β can be set arbitrarily, but depends on the set ratio α, so when α=0.8, it is preferably 70 to 90, and particularly preferably 80. You can That is, β is preferably set to about 100 times α.

Since Tray(x,y) is a pixel value, when the result of [Equation 11] is a negative value, it is replaced with Tray(x,y)=0, and the result of [Equation 11] becomes 255. If it exceeds, it is replaced with Tray(x, y)=255. Therefore, a pixel that does not satisfy the predetermined condition Lray(x,y)>vbmin is replaced with the minimum value “0”, and a pixel that satisfies the predetermined condition Lray(x,y)>vbmin is less than the minimum value “0”. The pixel value of Tray(x, y) is set so as to fall within the range from the large value “1” to the maximum value. In Tray(x, y), “0”, which is the minimum gradation value, indicates outside the blood vessel candidate region, and the blood vessel candidate region is “1” or more and “255” or less. The minimum value of the blood vessel candidate region is set to a value near the minimum gradation value. In the present embodiment, the minimum gradation value +1 is set as a value near the minimum gradation value, but it may be set higher. Of course, in order to increase the contrast of the blood vessel region, it is preferable that the minimum gradation value is +1. The blood vessel emphasized image Tray(x, y) thus obtained is a gray scale image having a higher value for a pixel having a higher probability of being a blood vessel region.

When a pixel that does not satisfy the predetermined condition Lray(x, y)>vbmin is set to the minimum value, it is preferably set to “0” as described above, but a value other than “0” may be set. Since the fundus area is generally circular, the non-fundus area is also included in the rectangular fundus image. Therefore, the non-fundus area is set to “0” and the non-vascular candidate area in the fundus area is set to “1”. Alternatively, the blood vessel candidate region in the fundus region can be set to "2" to "255". Since the purpose is to increase the contrast, it is, of course, preferable that the minimum value in the blood vessel candidate region is close to “0”.

As a result of the processing in step S400, a blood vessel emphasized image Trayg(x, y) mainly composed of a G component and a blood vessel emphasized image Trayr(x, y) mainly composed of an R component are obtained which are grayscale images. In addition, in FIG. 3, it is described as a blood vessel emphasized image (G) and a blood vessel emphasized image (R).

Next, the vein-enhanced image creating means 30 creates a vein-enhanced image using the blood vessel-enhanced image Trayg(x, y) mainly composed of the G component and the blood vessel-enhanced image Trayr(x, y) mainly composed of the R component. S500). The vein-enhanced image is an image in which a portion having a high possibility of being a vein is emphasized, that is, an image in which a portion having a high probability of being a vein is emphasized. An image obtained by performing a predetermined product operation between the blood vessel emphasized image mainly composed of the R component and the blood vessel emphasized image mainly composed of the G component is a vein emphasized image in which a portion having a high probability of being a vein is emphasized. The vein-enhanced image can be obtained based on the R-component-based blood vessel-enhanced image. It is also possible to use the blood vessel emphasized image mainly composed of the R component as the vein emphasized image as it is. As described above, in venous blood, oxygen is removed from hemoglobin of red blood cells after gas exchange in the periphery, and carbon dioxide is dissolved in the blood to show a blue-purple color. Can withstand use as well. Especially, a blood vessel-enhanced image mainly composed of R component obtained by subtracting B component in which an arterial component is conspicuous can be regarded as a venous component, but in order to suppress the remaining arterial component, a G component mainly including all blood component is mainly composed. An effective method is to take the AND with the blood vessel-emphasized image.

Therefore, in the present embodiment, in order to create a more accurate vein-enhanced image, a predetermined product operation is performed between the R-component-based blood vessel-emphasized image and the G-component-based blood vessel-emphasized image as follows. create. In creating the vein-enhanced image, first, the average value Avrg of pixel values equal to or greater than a predetermined value in the blood vessel-enhanced image Trayg(x,y) mainly composed of the G component and the predetermined value in the blood vessel-enhanced image Trayr(x,y) mainly composed of the R component An average value Avrr of pixel values equal to or larger than the value is calculated. Although the predetermined value can be set as appropriate, in the present embodiment, in both the blood vessel emphasized image Trayg(x, y) mainly composed of the G component and the blood vessel emphasized image Trayr(x, y) mainly composed of the R component, It is set to 128, which is a value at the center of the range from 0 to 255. Therefore, in this embodiment, the average value of the pixel values of the pixels having the pixel value of 128 or more is calculated as Avrg and Avrr.

Next, the process according to the following [Formula 12] is executed to obtain the vein-enhanced image Grayv(x, y).

[Formula 12]
Grayv(x,y)={Trayg(x,y)·Trayr(x,y)·Avrg/Avrr} ^1/2

In the above [Formula 12], the product operation of the blood vessel emphasized image Trayg(x, y) mainly composed of the G component and the blood vessel emphasized image Trayr(x, y) mainly composed of the R component is performed, and Avrg/Avrr is set to the value of the product operation. After multiplication, the square root is obtained to obtain the value of each pixel of the vein-enhanced image Grayv(x,y). That is, in step S500, the vein-enhanced image creating unit 30 calculates the average value of the pixel values of the pixels included in the blood vessel region for each of the R-component-based blood vessel-emphasized image and the G-component-based blood vessel-emphasized image. , And the calculated average values are the R component average value Avrr and the G component average value Avrg, respectively, the value of (G component average value Avrg/R component average value Avrr) becomes the pixel value of each pixel of the blood vessel emphasized image mainly of the R component. After performing correction by multiplying by, a predetermined product operation is performed. Further, in step S500, the vein-enhanced image creating means 30 performs, as a predetermined product operation, the pixel value of the pixel of the R-component-based blood vessel-emphasized image and the pixel value of the pixel corresponding to the pixel of the G-component-dominated blood vessel-emphasized image. And the geometric mean is taken.

In order to perform level matching with the pixel value of the artery-enhanced image Graya(x,y), which will be described later, subsequently, the maximum value of each pixel value of the vein-enhanced image Grayv(x,y) is normalized to a predetermined value (255). Perform processing to Specifically, first, the maximum value MAXv of pixel values in the vein-enhanced image Grayv(x, y) is obtained. Then, the processing according to the following [Equation 13] is executed to obtain the vein-enhanced image Grayv(x, y) normalized to the range of 0 to 255.

[Formula 13]
Grayv(x,y)←512・Grayv(x,y)/MAXv
If the result of the calculation is Grayv(x,y)>255, set Grayv(x,y)=255.

In [Equation 13], the vein-enhanced image Grayv(x is obtained by multiplying 512, which is twice the gradation 256, and multiplying by a predetermined magnification VE (=512/MAXv), which is a value obtained by dividing the maximum value MAXv. , Y) of each pixel value is normalized, and in the case of a pixel having a maximum possible pixel value of 255, the maximum value is set to 255. As described above, the normalized vein-enhanced image Grayv(x, y) is obtained.

Next, the artery-enhanced image creating means 40 creates an artery-enhanced image by using the blood vessel emphasized image Trayg(x, y) mainly composed of the G component and the blood vessel emphasized image Trayr(x, y) mainly composed of the R component. S600). The arterial-enhanced image is an image in which a portion having a high possibility of being an artery is emphasized, that is, an image in which a portion having a high probability of being an artery is emphasized. An image obtained by performing a predetermined difference calculation between a blood vessel emphasized image mainly composed of R component and a blood vessel emphasized image mainly composed of G component is an arterial emphasized image in which a portion having a high probability of being an artery is emphasized. It is difficult to obtain an arterial weighted image from a blood vessel weighted image having a single color component like a vein weighted image, and is obtained by subtracting the vein weighted image from a G component-based blood vessel weighted image including all blood vessel components. be able to. For example, it can be obtained by performing a predetermined difference calculation between the vein-enhanced image calculated by the vein-enhanced image creating means 30 and the blood vessel-enhanced image mainly composed of the G component. However, in the present embodiment, in order to create a more accurate artery-enhanced image, a predetermined difference calculation is performed between the R-component-based blood vessel-enhanced image and the G-component-based blood vessel-enhanced image as follows. create. Either the process of the vein-enhanced image creating unit 30 or the process of the artery-enhanced image creating unit 40 may be executed first. When the vein-enhanced image creating means 30 has already executed the process, the average value Avrg of pixel values equal to or larger than a predetermined value in the calculated blood vessel-enhanced image Tray(x, y) mainly composed of the G component and the blood vessel mainly composed of the R component. An average value Avrr of pixel values that are equal to or larger than a predetermined value in the emphasized image Trayr(x, y) is used. When the artery-enhanced image creating unit 40 performs the processing before the vein-enhanced image creating unit 30, the average value Avrg of the pixel values of the predetermined value or more in the blood vessel emphasized image Trayg(x, y) mainly composed of the G component, the R component mainly The average value Avrr of the pixel values equal to or larger than a predetermined value in the blood vessel emphasized image Trayr(x, y) is calculated.

Next, the process according to the following [Formula 14] is executed to obtain the artery-enhanced image Graya(x, y).

[Formula 4]
Graya(x,y)=Trayg(x,y)−Trayr(x,y)·Avrg/Avrr
If the result of the calculation is Graya(x,y)<0, then Graya(x,y)=0 is set.

In the above [Formula 4], after the blood vessel emphasized image Trayr(x, y) mainly composed of R component is multiplied by Avrg/Avrr, the blood vessel emphasized image Trayg(x, y) mainly composed of G component is used to emphasize the blood vessel emphasized R component. The difference calculation for subtracting the image Trayr(x, y) is performed to obtain the value of each pixel of the artery-enhanced image Graya(x, y). That is, in step S600, the arterial weighted image creating unit 40 calculates the average value of the pixel values of the pixels included in the blood vessel region for each of the blood vessel weighted image mainly composed of the R component and the blood vessel weighted image mainly composed of the G component, Letting the calculated average values be the R component average value Avrr and the G component average value Avrg, the value of (G component average value Avrg/R component average value Avrr) is added to the pixel value of each pixel of the R component-based blood vessel emphasized image. A predetermined difference calculation is performed after the correction for multiplication is performed.

As with the vein-enhanced image Grayv(x,y), the arterial-enhanced image Graya(x,y) is also subjected to the process of normalizing the maximum value of each pixel value to a predetermined value (255). Specifically, first, the maximum value MAXa of the pixel values in the artery-enhanced image Graya(x,y) is obtained. Then, the process according to the following [Formula 15] is executed to obtain the artery-emphasized image Graya(x, y) normalized to the range of 0 to 255.

[Equation 15]
Graya(x,y)←512・Grayv(x,y)/MAXa
When the result of the calculation is Graya(x,y)>255, Graya(x,y)=255.

In [Equation 15], the artery-enhanced image Graya(x is obtained by multiplying 512, which is twice the gradation 256, and multiplying by a predetermined magnification AR (=512/MAXa), which is a value divided by the maximum value MAXa. , Y) of each pixel value is normalized, and in the case of a pixel having a maximum possible pixel value of 255, the maximum value is set to 255. As described above, the normalized artery-enhanced image Graya(x, y) is obtained. Thus, in step S500, the vein-enhanced image creating unit 30 multiplies the pixel value of each pixel of the gray-scale vein-enhanced image by the predetermined magnification VE, and in step S600, the artery-enhanced image creating unit 40 The pixel value of each pixel of the artery-enhanced image is multiplied by a predetermined magnification AR. In [Expression 13] in step S500, the pixel value of each pixel of the grayscale vein-enhanced image is multiplied by a predetermined magnification VE, and in [Expression 15] in step S600, the pixel value of each pixel of the grayscale artery-enhanced image is calculated. By multiplying the value by a predetermined scale factor AR, a process of matching the maximum pixel values of the artery-enhanced image and the vein-enhanced image is performed.

When the vein-enhanced image Grayv(x, y) and the artery-enhanced image Graya(x, y) are obtained, the arteriovenous identification image creating means then creates an arteriovenous identification image (step S700). Specifically, using the grayscale vein-enhanced image Grayv(x, y) and the artery-enhanced image Graya(x, y), the color arteriovenous identification image AVsep(x, y, c) (c=0( Red), 1 (Green), 2 (Blue)) are created. At this time, first, the process according to the following [Formula 16] is executed to obtain the hue conversion image Hues(x, y).

[Formula 16]
Hues(x,y)=Graya(x,y)-Grayv(x,y)

As shown in [Equation 16], Hues(x, y) is a grayscale image obtained by subtracting the vein-enhanced image Grayv(x, y) from the artery-enhanced image Graya(x, y). The hue-converted image Hues(x, y) tends to have a value larger than 0 in the case of arteries and smaller than 0 in the case of veins.

The arteriovenous identification image creating means 50 further obtains an arteriovenous determination result image AVsep(x, y, c) by executing processing according to the following [Numerical Expression 17].

[Formula 17]
AVsep(x,y,1)=Grayg(x,y)/4
When Hues(x, y)> 0 (when the possibility of arteries is high)
AVsep(x,y,0)=(Hues(x,y)/2+L/2).Grayg(x,y)/(L-1)
AVsep(x,y,2)=Hues(x,y)/4
When Hues(x, y) ≤ 0 (when there is a high possibility of veins)
AVsep(x,y,2)=(-Hues(x,y)/2+L/2)Grayg(x,y)/(L-1)
AVsep(x,y,0)=-Hues(x,y)/4

In [Expression 17], Grayg(x, y) is the blood vessel emphasized image Trayg(x, y) mainly composed of the G component, and L is the number of gradations of the pixel value, and is 256, for example. As shown in [Formula 17], the arteriovenous identification image creating means 50 allocates the arterial weighted image to the R component AVsep(x, y, 0) and the vein weighted image to the B component AVsep(x, y, 2). A color arteriovenous identification image AVsep(x, y, c) is created. First, in step S700, the arteriovenous identification image creating means 50 gives a fixed value as G component AVsep(x, y, 1), which is a value obtained by dividing Grayg(x, y) by 4. This is so that Grayg(x, y) can be acquired from the arteriovenous identification image AVsep(x, y, c). Next, when Hues(x, y)>0 where the possibility of an artery is high, the value of Hues(x, y) is raised to L/2 to L−1, and the maximum pixel value ( The result is multiplied by the blood vessel-enhanced image Grayg(x, y) mainly composed of the G component, which is divided by L-1) and is normalized, and is given as the R component AVsep(x, y, 0). At the same time, the B component AVsep(x, y, 2) is fixedly given a value obtained by dividing Hues(x, y) by 4. This is so that Hues(x, y) can be acquired from the arteriovenous identification image AVsep(x, y, c). As described above, the value of the R component AVsep(x, y, 0) becomes significantly larger and becomes red compared to the G component AVsep(x, y, 1) and the B component AVsep(x, y, 2). Since the value of AVsep(x, y, 0) is multiplied by the blood vessel emphasized image mainly composed of the G component, it becomes dark gray in the non-blood vessel candidate area. Further, when Hues(x, y)≦0 in which the possibility of veins is high, the value obtained by inverting the sign of Hues(x, y) is converted so as to be L/2 to L−1, and further converted to the pixel value. The B component AVsep(x, y, 2) is obtained by multiplying the normalized blood vessel emphasized image Grayg(x, y) of the G component by dividing by the maximum value (L-1). At the same time, the R component AVsep(x, y, 0) is given a fixed value that is the value obtained by inverting the sign of Hues(x, y) and dividing the value by 4. This is also to allow Hues(x, y) to be acquired from the arteriovenous identification image AVsep(x, y, c). As described above, the value of the B component AVsep(x, y, 2) becomes significantly larger and becomes blue compared to the G component AVsep(x, y, 1) and the R component AVsep(x, y, 0). Since the value of AVsep(x, y, 2) is multiplied by the blood vessel emphasized image mainly composed of the G component, it becomes dark gray in the non-blood vessel candidate area. After all, the arteriovenous identification image creating means 50 executes the process according to the above [Formula 17] to convert the arterial emphasized image Graya(x, y) into the R component AVsep(x, y, 0) and the vein emphasized image. When assigning Grayv(x, y) to the B component AVsep(x, y, 2), the pixel value of each pixel of the artery-enhanced image Graya(x, y) and the vein-enhanced image Grayv(x, y) is the main component of the G component. The blood vessel emphasized image Trayg(x, y) is multiplied.

Also in the arteriovenous determination result image AVsep(x, y, c), as in the color fundus image Image(x, y, c), variables c=0 (Red), 1 (Green), 2 (Blue) indicating color components. ). In the case of an artery, the Red component becomes large and the Blue component becomes small. In the case of a vein, the Blue component becomes large and the Red component becomes large, and the value of the blood vessel emphasized image is set in the Green component. In order to facilitate the visual identification of veins and veins, the pixel value of the blood vessel emphasized image is greatly reduced and given. Therefore, the determination as to whether or not it is a blood vessel candidate region can be made more clearly for the Red component or the Green component in which the value of the blood vessel emphasized image is multiplied than the Green component. In the obtained arteriovenous determination result image AVsep(x, y, c), when L=256, it can be displayed as a general full-color image, and the distinction between arteries and veins can be visually recognized. ..

<3. Image display example>
A display example of an image created by the fundus image processing apparatus according to this embodiment will be described. 11 and 12 are views showing images obtained in the processing process of the fundus image processing apparatus according to the present embodiment. FIG. 11A is a G component-based grayscale fundus image created by the grayscale conversion means 10, and FIG. 11B is an R component-based grayscale fundus created by the grayscale conversion means 10. It is an image. Wr, Wg, and Wb shown in FIGS. 11A and 11B are weights of the respective color components shown in [Equation 1]. FIG. 11( c) is a grayscale G component-based blood vessel emphasized image obtained by the blood vessel region enhancing unit 20, and FIG. 11( d) is a grayscale R component obtained by the blood vessel region enhancing unit 20. It is a blood vessel emphasis image of the main subject.

FIG. 12A is a grayscale vein-enhanced image created by the vein-enhanced image creating means 30 using a predetermined product operation. FIG. 12B is a grayscale artery-enhanced image created by the artery-enhanced image creating means 40 using a predetermined difference calculation. FIG. 12C is a color arteriovenous identification image created by the arteriovenous identification image creating means 50.

13 to 15 are diagrams for comparing the arteriovenous identification image obtained by the conventional technique with the arteriovenous identification image obtained by the present embodiment. 13 to 15, (a) is a color fundus image to be processed, (b) is an arteriovenous identification image obtained by a conventional technique, and (c) is an arteriovenous identification image obtained by the present embodiment. Is shown. FIG. 13 is an image obtained for a normal subject. Comparing the conventional technique of FIG. 13B and the present embodiment of FIG. 13C, in the conventional technique of FIG. 13B, the hue component is different between the central part and the peripheral part of the fundus and the blood vessel attribute is the central part. While there was a tendency for the arteries to be separated toward the veins in the peripheral portion, the tendency is eliminated in the present embodiment of FIG. 13( c ).

FIG. 14 is an image obtained for a subject with glaucoma. Comparing the prior art of FIG. 14(b) and the present embodiment of FIG. 14(c), in the prior art of FIG. 14(b), the hue component is different between the central part and the peripheral part of the fundus, and especially the upper blood vessel attribute. Tended to be biased toward veins and tend to be separated, whereas such a tendency is eliminated in the present embodiment of FIG. 14(c). FIG. 15 is an image obtained for a subject with cataract. Comparing the prior art of FIG. 15(b) and the present embodiment of FIG. 15(c), in the prior art of FIG. 15(b), the hue component is different between the central part and the peripheral part of the fundus, and in particular, the blood vessel attribute on the right side. Tended to be biased toward the artery and tend to be separated, but in the present embodiment of FIG. 15C, such a tendency is eliminated.

<4. Modifications>
Although the preferred embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications can be made. For example, in the above embodiment, the vein-enhanced image creating unit 30 creates a grayscale vein-enhanced image by performing a predetermined product operation between the R-component-based blood vessel-enhanced image and the G-component-based blood vessel-enhanced image. However, the blood vessel emphasized image mainly composed of the R component may be given as it is as a gray scale vein emphasized image. Further, the artery-enhanced image creating unit 40 performs a predetermined difference calculation between the R-component-based blood vessel-enhanced image and the G-component-based blood vessel-enhanced image to create a grayscale artery-enhanced image. However, a predetermined difference calculation may be performed between the grayscale vein-enhanced image calculated by the above-mentioned predetermined product calculation and the G-component-based blood vessel emphasized image. Specifically, a process of subtracting the grayscale vein-enhanced image calculated by the predetermined product operation from the blood vessel emphasized image mainly composed of the G component is performed.

When the R-component-based blood vessel-emphasized image is used as a grayscale vein-emphasized image as it is, the artery-emphasized image creating unit 40 sets a predetermined value between the R-component-based blood vessel-emphasized image and the G-component-based blood vessel-emphasized image. After performing the product operation to obtain the grayscale product operation image, a predetermined difference operation may be performed between the obtained product operation image and the blood vessel emphasized image mainly composed of the G component. Specifically, a process of subtracting the grayscale product calculation image calculated by the predetermined product calculation from the blood vessel emphasized image mainly composed of the G component is performed. At this time, as the grayscale product calculation image, for example, an image substantially the same as the vein-enhanced image Grayv(x, y) calculated by the vein-enhanced image creating means 30 in the above-described embodiment can be used.

Further, in the above embodiment, the arterial weighted image and the arteriovenous identification image are created, but these do not necessarily have to be created, and only the vein weighted image may be created. The arterial-weighted image and the arteriovenous identification image enable more detailed grasping of the condition of the fundus including the artery, but even the vein-enhanced image alone is useful for grasping the condition of the fundus.

1... CPU (Central Processing Unit)
2... RAM (Random Access Memory)
3... storage device 4... instruction input I/F
5: Data input/output I/F
6... Display unit 10... Gray scale conversion means 20... Blood vessel region emphasis means 21... Image flattening means 22... Linear component emphasis means 23... Pixel gradation conversion means 30. ..Venous-enhanced image creating means 40...arterial-enhanced image creating means 50...arteriovenous identification image creating means 60...fundinal image storage means 70...arteriovenous identification image storage means 100...fundus image Processor

Claims (16)

A device for processing a color fundus image to enhance a blood vessel region,
Grayscale conversion means for creating a grayscale fundus image mainly composed of R components by using R components of a color fundus image composed of three color components of RGB,
Blood vessel region emphasizing means for obtaining a blood vessel emphasized image mainly composed of R component, which has a higher luminance value as the probability of being a blood vessel region is higher, based on the grayscale fundus image mainly composed of R component;
A vein-enhanced image creating means for obtaining a vein-enhanced image in which a portion having a high probability of being a vein is emphasized based on the R-component-based blood vessel-enhanced image;
A fundus image processing apparatus having: The grayscale conversion means further uses the G component to create a grayscale fundus image mainly composed of the G component,
The blood vessel region enhancing means further obtains the blood vessel emphasized image mainly composed of the G component based on the gray scale fundus image mainly composed of the G component,
The vein weighted image creating means,
The fundus image processing apparatus according to claim 1, wherein the vein weighted image is obtained by performing a predetermined product operation between the R component-based blood vessel emphasized image and the G component-based blood vessel emphasized image. .. By performing a predetermined difference calculation between the vein-enhanced image created by the vein-enhanced image creating unit and the G-component-based blood vessel-enhanced image, an arterial-enhanced image in which a portion having a high probability of being an artery is emphasized An arterial weighted image creating means for obtaining
An arteriovenous identification image creating means for creating a color arteriovenous identification image by assigning the arterial weighted image to the R component and the vein weighted image to the B component;
The fundus image processing apparatus according to claim 2, further comprising: The grayscale conversion means further uses the G component to create a grayscale fundus image mainly composed of the G component,
The blood vessel region enhancing means further obtains the blood vessel emphasized image mainly composed of the G component based on the gray scale fundus image mainly composed of the G component,
By performing a predetermined difference calculation between the vein-enhanced image created by the vein-enhanced image creating unit and the G-component-based blood vessel-enhanced image, an arterial-enhanced image in which a portion having a high probability of being an artery is emphasized An arterial weighted image creating means for obtaining
An arteriovenous identification image creating means for creating a color arteriovenous identification image by assigning the arterial weighted image to the R component and the vein weighted image to the B component;
The fundus image processing apparatus according to claim 1, further comprising: Creating an arterial weighted image that obtains an arterial weighted image in which a portion having a high probability of being an artery is emphasized by performing a predetermined difference calculation between the blood vessel weighted image mainly composed of the R component and the blood vessel weighted image mainly composed of the G component Means and
An arteriovenous identification image creating means for creating a color arteriovenous identification image by assigning the arterial weighted image to the R component and the vein weighted image to the B component;
The fundus image processing apparatus according to claim 2, further comprising: The grayscale conversion means further uses the G component to create a grayscale fundus image mainly composed of the G component,
The blood vessel region enhancing means further obtains the blood vessel emphasized image mainly composed of the G component based on the gray scale fundus image mainly composed of the G component,
A product calculation image obtained by performing a predetermined product calculation between the R-component-based blood vessel emphasized image and the G-component-based blood vessel emphasized image, and the G-component-based blood vessel emphasized image An arterial weighted image creating means for obtaining an arterial weighted image in which a portion having a high probability of being an artery is weighted by performing a difference calculation of
An arteriovenous identification image creating means for creating a color arteriovenous identification image by assigning the arterial weighted image to the R component and the vein weighted image to the B component;
The fundus image processing apparatus according to claim 1, further comprising: The grayscale conversion means,
As the grayscale fundus image mainly composed of the R component, negative-positive inversion is performed on an image obtained by subtracting a predetermined ratio of the B component of the color fundus image from the R component of the color fundus image,
7. The fundus image processing apparatus according to claim 2, wherein the grayscale fundus image mainly composed of the G component is created by negative/positive inversion with respect to the G component image of the color fundus image. The blood vessel region emphasizing means,
For each pixel of the R-component-based grayscale fundus image and the G-component-based grayscale fundus image, an average value of pixel values of neighboring pixels included in a predetermined range is calculated, and from the pixel value of the pixel, An image flattening unit that creates a flattened image of each of the R component-based grayscale fundus image and the G component-based grayscale fundus image by subtracting the average value and adding a predetermined value,
For each of the flattened images, a linear component enhancing unit that performs an opening process using a predetermined structuring element to create each linear component enhanced image in which the brightness of the blood vessel candidate region is aligned to a certain level or more,
In each of the linear component emphasized images, the pixel value that does not satisfy the predetermined condition is replaced so as to be the minimum gradation value of the converted blood vessel emphasized image, and the pixel value that satisfies the predetermined condition is Pixel gradation conversion means for correcting the gradation of pixels so as to fall within a range from a value near the minimum value to a maximum value, and creating the blood vessel emphasized image mainly composed of the R component and the blood vessel emphasized image mainly composed of the G component,
The fundus image processing device according to claim 2, further comprising: The linear component emphasizing means performs an opening process on the flattened image using a plurality of linear structuring elements that are linear structuring elements having different directions as the predetermined structuring element to perform a plurality of linear opening images. The fundus image processing apparatus according to claim 8, wherein the linear component emphasized image is created by giving a maximum value for each pixel as a pixel value from the created and obtained plurality of linear opening images. The linear structuring element has a predetermined radius of N (N is an integer of 1 or more), and is a binary image of (2N+1)×(2N+1) pixels in which reference pixels referred to in the opening process are defined, Reference pixels are defined inside the circle having the radius, the pixel width is 1N, the pixel width is 1 pixel, and the reference pixels are arranged at least in the vicinity of a straight line at intervals of 22.5 degrees to define 8 directions. The fundus image processing apparatus according to claim 9. The vein-enhanced image creating means or the artery-enhanced image creating means,
An average value of pixel values of pixels included in a blood vessel region is calculated for each of the R component-based blood vessel emphasized image and the G component-based blood vessel emphasized image, and the calculated average values are respectively calculated as the R component average value and the G component. Assuming the component mean value,
After performing a correction by multiplying the pixel value of each pixel of the blood vessel emphasized image mainly composed of R component by a value of G component average value/R component average value,
The fundus image processing apparatus according to claim 3, wherein the vein-enhanced image or the artery-enhanced image is created. The vein weighted image creating means,
As the predetermined product operation,
The geometric mean of the pixel value of the pixel of the blood vessel emphasized image mainly composed of the R component and the pixel value of a pixel corresponding to the pixel of the blood vessel emphasized image mainly composed of the G component is calculated. The fundus image processing device according to any one of 6 above. The vein-enhanced image creating means multiplies a pixel value of each pixel of the vein-enhanced image by a predetermined magnification VE,
The arterial weighted image creating means multiplies the pixel value of each pixel of the arterial weighted image by a predetermined magnification AR,
The fundus image processing apparatus according to any one of claims 3 to 6, which performs a process of matching maximum pixel values of the vein-enhanced image and the artery-enhanced image. The arteriovenous identification image creating means,
When assigning the arterial weighted image to the R component and the vein weighted image to the B component,
7. The fundus image processing apparatus according to claim 3, wherein the pixel values of the pixels of the artery-enhanced image and the vein-enhanced image are multiplied by the blood vessel-enhanced image mainly composed of the G component. .. A program for causing a computer to function as the fundus image processing device according to any one of claims 1 to 14. A method for a computer to process a color fundus image to create an image in which blood vessel regions are emphasized,
Creating a grayscale fundus image mainly composed of R components using R components of a color fundus image composed of three color components of RGB, and creating a grayscale fundus image mainly composed of G components using G components;
Based on the R-component-based grayscale fundus image and the G-component-based grayscale fundus image, the R-component-based blood vessel-enhanced image and the G-component-based image in which a higher brightness value is set as the probability of a blood vessel region is higher Obtaining a blood vessel weighted image of
Obtaining a vein-enhanced image in which a portion having a high probability of being a vein is emphasized based on the R-component-based blood vessel-enhanced image;
Obtaining an artery-emphasized image in which a portion having a high probability of being an artery is emphasized based on the blood vessel-emphasized image mainly composed of the G component;
Assigning the arterial weighted image to the R component and the vein weighted image to the B component to create a color arteriovenous identification image;
An image creating method having.