KR101269175B1 - Method for detecting an area and area detecting apparatus applying the same - Google Patents

Abstract

An area detection method is disclosed. The method for detecting an area of the eye into which the intraocular lens is inserted by the area detecting apparatus comprises receiving an image of an eye photographed in real time and a template corresponding to the area into which the intraocular lens is to be inserted into the captured eye image. Receiving a user command to set the step, using a template matching algorithm, calculating a correlation coefficient between the set template image and the captured eye image in real time, SAD (sum) for the captured eye image in real time calculating the value of the absolute diference value, comparing the correlation coefficient with the SAD value, and selectively performing a template matching algorithm or SAD algorithm according to the comparison result to insert the intraocular lens in the captured eye image. Detecting the area of interest.

Description

Method for detecting an area and area detecting apparatus applying the same

The present invention relates to an area detection method for detecting an area of an eye into which an intraocular lens is inserted, and an area detection device using the same, and more particularly, selects a template matching algorithm or a sum of absolute difference (SAD) algorithm. The present invention relates to a region detecting method for detecting an insertion region of an intraocular lens and a region detecting apparatus using the same.

With the aging of medical technology and the development of diet, cataract surgery is one of the frequent operations. In cataract patients with astigmatism, eyeglasses, contact lenses, or additional corneal surgery are required to achieve improved vision after surgery. Recently, astigmatism, or toric IOL, was introduced to correct astigmatism. It became possible.

However, it is very important that the toric IOL, astigmatism-corrected intraocular lens, be placed so that the axis of the inserted toric IOL exactly matches the calculated axis. In particular, the correct alignment of the intraocular lens implant has the most important effect on the surgical outcome, and even a small amount of error results in a relatively large loss of astigmatism.

Therefore, the indirect retinal state is detected through ultrasound or electrophysiological examination, but it is not possible to give information necessary for surgery, and surgery is often performed depending on the experience of a doctor in an operating room. Therefore, accurate tracking of the intraocular lens IOL region is very important.

In the case of retinal disease, the retina cannot be observed due to intraocular turbidity before surgery, and the indirect retinal state is determined by ultrasound or electrophysiology. However, this does not necessarily give the information necessary for the operation, there is a problem that the operation in many cases depending on the experience of the doctor in the operating room.

Therefore, in intraocular lens implantation, a search for a method for accurately tracking the insertion region of the intraocular lens is required.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to provide a region detecting method for detecting an insertion region of an intraocular lens and a region detecting apparatus using the same by selectively applying a template matching algorithm or a SAD algorithm. .

According to an embodiment of the present invention, a method for detecting an area of an eye into which an artificial lens is inserted by an area detecting device includes: receiving an image of an eye photographed in real time; Receiving a user command for setting a template corresponding to a region into which the intraocular lens is to be inserted into the captured eye image, by using a template matching algorithm, the captured template image and the captured image in real time Calculating a correlation coefficient between eyeball images, calculating a sum of absolute difference (SAD) value for the eyeball image captured in real time, comparing the correlation coefficient with the SAD value, and a template according to a comparison result Selectively performing a matching matching algorithm or a SAD algorithm to the artificial eye in the captured eye image; And a step of detecting a region in which stagnation is inserted.

In this case, the step of receiving the user command may receive a user command for setting a template corresponding to a region into which the intraocular lens is inserted in one frame of the image of the eyeball captured in real time.

여기서, r은 상관 계수, f(x,y)는 상기 템플릿의 픽셀값, f'는 상기 템플릿의 평균 픽셀값, t(x,y)는 촬상된 안구 영상의 픽셀값, t'은 촬상된 안구 영상의 블록에 평균값을 나타낸다.Meanwhile, the correlation coefficient has a value between -1 and 1, and the calculating of the correlation coefficient may calculate the correlation coefficient by using the following equation.

Where r is the correlation coefficient, f (x, y) is the pixel value of the template, f 'is the average pixel value of the template, t (x, y) is the pixel value of the photographed eye image, and t' is the captured image. The mean value is shown in the block of the eyeball image.

여기서, I_t(i,j)와 I_{t - 1}(i,j)는 촬상된 안구 영상의 시간 t, t+1 에서 휘도(luminance) 화소값, (u,v)는 움직임을 의미하며, N은 블록의 크기이다. In the calculating of the SAD value, the SAD value may be calculated using the following equation.

Here, I _t (i, j) and I _{t - 1} (i, j) are luminance pixel values at time t and t + 1 of the photographed eye image, and (u, v) means movement. N is the size of the block.

The comparing may include normalizing the calculated SAD value and comparing the normalized SAD value with the correlation coefficient.

In this case, the detecting may include performing the template matching algorithm if the correlation coefficient is greater than the normalized SAD value, and performing the SAD algorithm if the correlation coefficient is less than or equal to the normalized SAD value. A region into which the intraocular lens is inserted may be detected in an eyeball image.

Meanwhile, the method may further include displaying the detected area.

According to another embodiment of the present invention, an apparatus for detecting an area of an eye into which an intraocular lens is inserted includes an imager configured to image the eye in real time, and a template corresponding to an area into which the intraocular lens is to be inserted in the captured eye image. an input unit for receiving a user command for setting a template, a template matching algorithm, and calculating a correlation coefficient between the set template image and the eyeball image captured in real time, and the eyeball image captured in real time. A calculation unit for calculating a sum of absolute difference (SAD) value and the correlation coefficient with the SAD value, and selectively performing a template matching algorithm or a SAD algorithm according to the comparison result to photograph the eyeball in real time And a detector configured to detect a region where the intraocular lens is inserted in the image.

In this case, the input unit may receive a user command for setting a template for the region into which the intraocular lens is to be inserted in one frame of the image of the eyeball captured in real time.

여기서, r은 상관 계수, f(x,y)는 상기 템플릿의 픽셀값, f'는 상기 템플릿의 평균 픽셀값, t(x,y)는 촬상된 안구 영상의 픽셀값, t'은 촬상된 안구 영상의 블록에 평균값을 나타낸다.On the other hand, the correlation coefficient has a value between -1 and 1, the calculation unit may calculate the correlation coefficient by using the following equation.

Where r is the correlation coefficient, f (x, y) is the pixel value of the template, f 'is the average pixel value of the template, t (x, y) is the pixel value of the photographed eye image, and t' is the captured image. The mean value is shown in the block of the eyeball image.

여기서, I_t(i,j)와 I_{t - 1}(i,j)는 촬상된 안구 영상의 시간 t, t+1 에서 휘도(luminance) 화소값, (u,v)는 움직임을 의미하며, N은 블록의 크기이다. On the other hand, the calculator may calculate the SAD value using the following equation.

Here, I _t (i, j) and I _{t - 1} (i, j) are luminance pixel values at time t and t + 1 of the photographed eye image, and (u, v) means movement. N is the size of the block.

The detector may normalize the calculated SAD value and compare the normalized SAD value with the correlation coefficient.

In this case, the detection unit performs the template matching algorithm when the correlation coefficient is greater than the normalized SAD value, and performs the SAD algorithm when the correlation coefficient is less than or equal to the normalized SAD value to capture the image in real time. A region into which the intraocular lens is inserted may be detected in an eyeball image.

The display device may further include a display unit displaying the detected area.

According to various embodiments of the present disclosure, by selectively applying a template matching algorithm or an SAD algorithm, the insertion region of the intraocular lens may be detected, and thus, the insertion region of the intraocular lens may be detected with a smaller amount of computation, and the illumination of the surgical environment may be changed. And a strong effect on the background change of the insertion region.

1 is a block diagram illustrating a configuration of an area detection apparatus according to an embodiment of the present invention;
2 is a flowchart illustrating an area detection method according to an embodiment of the present invention;
3 to 6 are diagrams for evaluating the performance of the area detection method according to an embodiment of the present invention;
7 is another graph for evaluating the performance of the area detection method according to an embodiment of the present invention.

Hereinafter, with reference to the drawings will be described the present invention in more detail.

1 is a block diagram illustrating a configuration of an area detection apparatus according to an exemplary embodiment. In detecting the insertion region of the intraocular lens, the area detecting apparatus 100 according to the present embodiment selectively detects the insertion region of the intraocular lens by selectively performing a template matching algorithm or a sum of absolute difference (SAD) algorithm. can do. Referring to FIG. 1, the area detecting apparatus 100 performing such a function may include an image capturing unit 110, an input unit 120, a calculating unit 130, a detecting unit 140, and a display unit 150.

The imaging unit 110 picks up the eyeball in real time. Here, the eyeball includes a region into which the intraocular lens is to be inserted, and the image capturing unit 110 which performs such a function may be implemented as a camera to output an image of the captured eyeball in frame units.

The input unit 120 receives a user command for setting a template corresponding to a region into which an intraocular lens is to be inserted into the captured eye image. In detail, the input unit 120 may receive a user command for setting a template having a width w and a height h with respect to one frame of the eyeball image captured in real time, wherein the template is a template matching algorithm. Can mean an image of a region to be tracked.

Specifically, a template matching algorithm means an algorithm for calculating a correlation between a template and an input image of an image of an area to be tracked, and thus detecting an area to be tracked in the input image. (1) Myung Hwangbo, Jun-Sik Kim, and Takeo Kanade, “Inertial-Aided KLT Feature Tracking for a Moving Camera” International Conference on Intelligent Robots and Systems, October 11-15, 2009).

This template matching consists of two steps. The first step is to analyze the relationship by comparing and comparing the pixels of the image one by one. The second step is to maximize the correlation coefficient by using the similarity of the correlation coefficient. (2) Chaur_Chin Chen and Hsueh-Ting Chu “Similarity Measurement Between Images.” Proceedings of the 29th Annual International Computer Software and Applications Conference (COMPSAC'05), vol. 1, pp. 41-42, 2005).

First, in order to measure the similarity between the template image and the input image, an Euclidian distance is calculated, which can be expressed by Equation 1 below.

In addition, Pearson correlation coefficient, which is a deviation sample correlation coefficient, is used to compare the similarity of correlation coefficients, which is expressed by Equation 2 below.

Here, f (x, y) represents a pixel value of the template image, and f 'represents an average pixel value of the template image. t (x, y) denotes a pixel value of the input image, t 'denotes an average value in a block of the input image, r is a correlation coefficient, and a matching value between -1 and 1 as a result of matching the input image and the template image. The closer to 1, the more similar the template image to the input image.

The biggest advantage of the template matching is an easy and simple algorithm, which is highly accurate when there is little environmental change. However, the disadvantage is that if you have a complex background, there is a possibility of incorrect extraction.

The calculator 130 may calculate a correlation coefficient between the set template image and the eyeball image captured in real time using a template matching algorithm. That is, the calculation unit 130, as shown in Equation 2 above, using the average value of each frame of the eyeball image captured in real time and the average value of the extracted template, between the templates set for each frame of the eyeball image captured in real time Correlation can be calculated. Here, the calculated correlation coefficient is a pierce correlation coefficient having a value between -1 and 1.

The correlation coefficient calculated by the calculator 130 is a normalized value and may have a value between -1 and 1. FIG.

In addition, the calculator 130 may calculate a sum of absolute difference (SAD) value for the eyeball image captured in real time.

In detail, the sum of absolute difference (SAD) algorithm is a method used for a motion estimation method in a video compression standard such as H.264 and MPEG-4. Motion estimation is a method of obtaining motion on a screen in order to compress a video using a correlation between neighboring screens in time, and is performed using a block matching algorithm (BMA). The block matching algorithm is a method of finding a block most similar to each current block in a current frame to be encoded in a previous frame. The most widely used method is a full search block matching algorithm.

On the other hand, in the global search method, all possible candidate blocks in a given search area are compared with the current block, and the method used as a comparison measure is a SAD algorithm, which can be represented by Equation 3 below ([3] H. Cheong). and AM Tourapis, “Fast motion estimation within the H.264 codec,” IEEE Int. Conf.Multimedia and Expo, Baltimore, MD, vol. 3, pp. 517-520, July 2003. [4] Dong-kyun Park, Hyo-moon Cho, Sang-bok Cho, and Jong-hwa Lee, “A Fast Motion Estimation Algorithm for SAD Optimization in Sub-pixel” IEEE International Symposium on ISIC '07. Pp.528-531).

Here, I _t (i, j) and I _{t - 1} (i, j) are respective luminance pixel values at time t, t + 1. Also, (u, v) means motion, and N is the size of a block.

Such SAD has the advantage of finding a block that is similar to the current block relatively accurately even in a complicated background, but has a disadvantage of being sensitive to light changes.

The detector 140 may compare the correlation coefficient and the SAD value, and selectively detect a region into which the intraocular lens is inserted by selectively performing a template matching algorithm or an SAD algorithm according to the comparison result.

To this end, the detector 140 calculates norSAD by normalizing the SAD value calculated by the calculator 130 using Equations 4 to 6 below.

Here, n means the size of the block, n is 30 in a 30 × 30 block.

This normalization process is intended to reduce the amount of computation when the template image is incorrectly found by the SAD algorithm, and to compare with the Pearson correlation coefficient having a value between -1 and 1.

The detector 140 may compare the correlation coefficient and the SAD value, and selectively detect a region into which the intraocular lens is inserted by selectively performing a template matching algorithm or an SAD algorithm according to the comparison result.

In detail, the detector 140 compares the correlation coefficient and the SAD value for each frame of the eyeball image captured in real time based on the output of the calculator 130. If the correlation coefficient is greater than the normalized SAD value, the detector 140 performs a template matching algorithm. If the correlation coefficient is less than or equal to the normalized SAD value, the detection unit 140 performs an SAD algorithm to insert an intraocular lens in the eye image captured in real time. The area to be detected can be detected. Meanwhile, the template matching algorithm and the SAD algorithm are easily understood by those of ordinary skill in the art, and thus, further detailed description thereof will be omitted.

The display unit 150 may display the detected area. That is, the display unit 150 may add and display a region into which the detected intraocular lens is inserted in the eyeball image captured by the imaging unit 110.

Meanwhile, the above-described embodiment has been described as receiving a user command for setting a template, but this is merely an example. In the area detection apparatus according to the present invention, a template for an area into which an intraocular lens is inserted may be automatically set. In detail, the template may be automatically set for a portion in which the brightness of the pixel changes from a low value to a high value in the captured eye image or vice versa.

On the other hand, in order to detect the movement of the pattern in the two-dimensional image, a light flow method is widely used. This method can, in principle, find the relative movements required for eye recognition. However, there is a problem that the motion vector is not accurately detected or the computational cost is high ([5] SS Beauchemin, and JL Barron. “The Computation of Optical Flow”, ACM Computing Surveys, Vol. 27, No. 3, pp. 433-467, Sep. 1995, S. Beauchemin, and JL Barron. “The Computation of Optical Flow”, ACM Computing Surveys, Vol. 27, No. 3, pp. 433-467, Sep. 1995.

The calculation of the light flow is based on two assumptions that the brightness of the object is constant over time and that adjacent points in the image move similarly, which is expressed by Equation 7 below.

 Here, Ix, Iy, and It represent the rate of change of brightness in the x-axis direction, the y-axis direction, and the time axis direction, respectively. However, in Equation 7, since only the direction of the brightness gradient can be obtained, the velocity vectors u and v are estimated by applying a smoothing condition. There are several ways to estimate u and v, but the most common method is:

step 1. Initialize the velocity vectors c (i, j) for all (i, j).

step 2. Next, let k be the number of repetitions and calculate u and v using the following Equations 8 and 9 for all pixels (i, j).

,

를 나타낸다.here,

,

Indicates.

step 3. When the condition of Equation 10 is satisfied, the calculation is stopped, or the process is repeated.

는 최대 허용 오차를 나타낸다.here,

Represents the maximum allowable error.

2 is a flowchart illustrating an area detection method according to an exemplary embodiment.

In operation S210, an image of an eye photographed in real time is received. In this case, a user command for setting a template corresponding to an area into which an intraocular lens is to be inserted is input to one frame of an image captured in real time.

Thereafter, a user command for setting a template corresponding to a region into which the intraocular lens is to be inserted is input to the captured eye image (S220).

The correlation coefficient between the set template image and the eyeball image captured in real time is calculated using a template matching algorithm (S230).

Meanwhile, a sum of absolute diference (SAD) value of an eye image captured in real time is calculated (S240).

The correlation coefficient is then compared with the SAD value (S250). In this case, the calculated SAD value may be normalized and the normalized SAD value may be compared with a correlation coefficient.

According to the comparison result, a template matching algorithm or a SAD algorithm may be selectively performed to detect a region into which an intraocular lens is inserted in the captured eye image.

Specifically, when the correlation coefficient is larger than the normalized SAD value (S260-Y), the template matching algorithm is performed (S270). If the correlation coefficient is smaller than or equal to the normalized SAD value (S260-N), the SAD algorithm is performed. A region into which the intraocular lens is inserted may be detected in the extracted eye image.

In operation S230, the calculated correlation coefficient has a value between −1 and 1 and a correlation coefficient may be calculated using the following equation.

Where r is the correlation coefficient, f (x, y) is the pixel value of the template, f 'is the average pixel value of the template, t (x, y) is the pixel value of the photographed eye image, and t' is the captured image. The mean value is shown in the block of the eyeball image.

In operation S240, the SAD value may be calculated using the following equation.

Here, I _t (i, j) and I _{t - 1} (i, j) are luminance pixel values at time t and t + 1 of the photographed eye image, and (u, v) means movement. N is the size of the block.

On the other hand, the detected area may be displayed. In detail, an area in which the detected intraocular lens is inserted may be added to the captured eye image and displayed.

3 to 6 are diagrams for evaluating the performance of the area detection method according to an embodiment of the present invention. 3 to 6 show results of two different eye images captured at 30 frames per second, and the block is set to 30 × 30.

3 and 4 show images before and after 150 frames of the first eye image, and FIGS. 5 and 6 show images before and after 150 frames of the second eye image, respectively. 3 to 6, it can be seen that even if there is a lot of movement of the area into which the intraocular lens is inserted or another object enters the background of the area into which the intraocular lens is inserted, it is accurately tracked.

7 is another graph for evaluating the performance of the area detection method according to an embodiment of the present invention. FIG. 7 shows a result of comparing template matching using Pearson's correlation coefficient and peak signal to noise ratio (PSNR) of the area detection method according to the present invention.

As shown in FIG. 7, it can be seen that the region detection method according to the present invention has a higher PSNR compared to the method using the template matching using the Pearson correlation coefficient. Through this, the region detection method according to the present invention uses the Pearson correlation coefficient. It can be seen that the estimation is more accurate than template matching.

Although the above has been illustrated and described with respect to preferred embodiments of the present invention, the present invention is not limited to the above-described example, and the general knowledge in the art to which the present invention pertains without departing from the gist of the present invention as claimed in the claims. Anyone with a variety of modifications can be made, as well as such changes fall within the scope of the claims.

100: area detection device
110: imaging unit 120: input unit
130: calculator 140: detector

Claims (14)

In the method for detecting an area of the eye into which the intraocular lens is inserted by the area detecting device,
Receiving an image of an eye photographed in real time;
Receiving a user command for setting a template corresponding to an area into which the intraocular lens is to be inserted into the captured eye image;
Calculating a correlation coefficient between the set template image and the eyeball image captured in real time by using a template matching algorithm;
Calculating a sum of absolute diference (SAD) value for the eye image captured in real time;
Comparing the correlation coefficient with the SAD value; And
And selectively detecting a region into which the intraocular lens is inserted in the captured eye image by performing a template matching algorithm or a SAD algorithm according to the comparison result. The method of claim 1,
The step of receiving the user command,
And a user command for setting a template corresponding to an area into which the intraocular lens is to be inserted with respect to one frame of the eyeball image captured in real time. The method of claim 1,
The correlation coefficient has a value between -1 and 1,
Computing the correlation coefficient,
And calculating the correlation coefficient by using the following equation.

Where r is the correlation coefficient, f (x, y) is the pixel value of the template, f 'is the average pixel value of the template, t (x, y) is the pixel value of the photographed eye image, and t' is the captured image. The average pixel value is shown in the block of the eyeball image. The method of claim 1,
Computing the SAD value,
The SAD value is calculated using the following equation.

Here, I _t (i, j) and I _{t - 1} (i, j) are luminance pixel values at time t and t + 1 of the photographed eye image, and (u, v) means movement. N is the size of the block. The method of claim 1,
Wherein the comparing comprises:
And normalizing the calculated SAD value and comparing the normalized SAD value with the correlation coefficient. The method of claim 5,
Wherein the detecting comprises:
If the correlation coefficient is greater than the normalized SAD value, the template matching algorithm is performed. If the correlation coefficient is less than or equal to the normalized SAD value, the SAD algorithm is performed to insert the artificial lens in the captured eye image. An area detection method characterized by detecting a region to be used. The method of claim 1,
And displaying the detected area. In the device for detecting a region of the eye into which the intraocular lens is inserted,
An imaging unit for capturing the eyeball in real time;
An input unit configured to receive a user command for setting a template corresponding to an area in which the intraocular lens is to be inserted in the captured eye image;
Calculation of a correlation coefficient between the set template image and the eyeball image captured in real time using a template matching algorithm, and calculating a sum of absolute diference (SAD) value for the eyeball image captured in real time. part; And
A detector for comparing the correlation coefficient with the SAD value and selectively performing a template matching algorithm or a SAD algorithm according to a comparison result to detect a region in which the artificial lens is inserted in the real-time eyeball image; Area detection apparatus comprising a. 9. The method of claim 8,
Wherein the input unit comprises:
And a user command for setting a template for a region into which the intraocular lens is to be inserted in one frame of the image of the eyeball captured in real time. 9. The method of claim 8,
The correlation coefficient has a value between -1 and 1,
The calculating unit calculates,
And calculating the correlation coefficient by using the following equation.

Where r is the correlation coefficient, f (x, y) is the pixel value of the template, f 'is the average pixel value of the template, t (x, y) is the pixel value of the photographed eye image, and t' is the captured image. The mean value is shown in the block of the eyeball image. 9. The method of claim 8,
The calculating unit calculates,
The SAD value is calculated using the following equation.

Here, I _t (i, j) and I _{t - 1} (i, j) are luminance pixel values at time t and t + 1 of the photographed eye image, and (u, v) means movement. N is the size of the block. 9. The method of claim 8,
Wherein:
And normalize the calculated SAD value and compare the normalized SAD value with the correlation coefficient. The method of claim 12,
Wherein:
If the correlation coefficient is greater than the normalized SAD value, the template matching algorithm is performed. If the correlation coefficient is less than or equal to the normalized SAD value, the SAD algorithm is performed to perform the artificial lens in the real-time eyeball image. And the area detecting device is inserted. 9. The method of claim 8,
And a display unit for displaying the detected area.

Images