JP2005063000A - Image sharpening method, apparatus and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To clarify an image in which an in-focus area and an out-of-focus area are mixed.
A circle of confusion C indicating a range of surrounding pixels 2 that affect the color information of the pixel of interest 1 is estimated using all the pixels constituting the target image as the pixel of interest 1, and the estimated circle of confusion C is estimated. Based on the above, the color information of the pixel of interest 1 is restored to a state in which the surrounding pixel 2 is not affected by using an inverse filter of the blur function.
[Selection] Figure 1

Description

  The present invention relates to an image sharpening method and apparatus, and a program. More specifically, the present invention relates to a method, apparatus, and program for sharpening an image in which an in-focus area and an out-of-focus area are mixed.

  In accordance with recent progress in image processing technology, images are used for monitoring equipment such as power equipment. Since the image can be obtained in one place, the image is useful for monitoring a system with a wide spread, such as power facilities, and can be expected to be used for early recovery of the accident point and elucidation of the cause of the accident. . The reason for the low quality of the surveillance image is blurring that occurs when the distance between the lens and the subject does not match the focal length of the lens (hereinafter also referred to as “blurring due to defocusing”), or high-speed movement of the target object. There are motion blur caused by it. In a monitoring image of a stationary facility, since there is little possibility of motion blur, blur due to defocusing is a main cause of image quality degradation.

  Conventionally, methods for improving image blur have been studied for the purpose of improving the quality of astronomical images (see, for example, Non-Patent Documents 1 and 2). In a conventional method for improving a blurred image, a process in which a blurred image is generated from an image without blur is modeled using a blur function (also referred to as a point spread function (hereinafter abbreviated as PSF)). Then, using an inverse filter corresponding to the inverse function of the estimated blur function, an image without blur is obtained from the blurred image.

  In addition, a method has been proposed in which images with different focal points are photographed at the same angle, and in-focus pixels are selected from the plurality of images and combined to obtain a clear image without defocus (for example, Non-patent documents 3 and 4).

Rafacl Molina et al, "Image Restoration in Astronomy", IEEE Signal Processing Magazine Vol.18, No.2, pp.14-29, 2001 S. Chaudhuri et al, "Depth from Defocus: A Real Aperture Imaging Approach", Springer, 1998 Naito et al., "Enhancing acquisition of omnifocal images using generation of out-of-focus images from different focal images", IEICE Transactions D-II, Vol. J79-D-II, No.6, pp .1046-1053 1996 Kubota et al., "Positioning method and imaging system for multifocal image processing", Journal of the Institute of Image Information and Television Engineers, Vol. 54, No.2, pp260-267, 2000

  However, in the conventional techniques shown in Non-Patent Documents 1 and 2, etc., a single PSF is used under the assumption that the entire image is uniformly blurred (that is, the same blur is generated throughout the entire image). Since blur is improved by estimation, it cannot be applied to a locally blurred image. For example, in an image that monitors the entire facility over a wide area, the object to be monitored is not clearly determined. Therefore, the focused area is focused on a point of interest of the observer, etc. Region) and unfocused unclear portions (out-of-focus region) are mixed. In such an image, since the precondition that the entire image is uniformly blurred is lost, the techniques of Non-Patent Documents 1 and 2 and the like cannot be expected to improve the blur. When any problem occurs in the equipment in the out-of-focus area, it becomes difficult to quickly grasp and deal with the problem.

  Further, in the prior art disclosed in Non-Patent Documents 3 and 4, etc., a camera must be prepared for each focus to be acquired. For example, if it is desired to acquire an image in accordance with three focal points, three CCDs must be prepared (see Non-Patent Document 4). When applied to an image that monitors the entire facility over a wide area, a considerable number of cameras are required, and it is difficult to put it to practical use.

  Therefore, an object of the present invention is to provide a method, an apparatus, and a program for sharpening an image in which an in-focus area and an out-of-focus area are mixed.

  In order to achieve such an object, the image sharpening method according to claim 1 is directed to a target image in which an in-focus area and an out-of-focus area are mixed, with each of all pixels constituting the target image as a target pixel. Estimate the circle of confusion that indicates the range of surrounding pixels that affect the color information of the pixel of interest, and use the inverse filter of the blur function based on the estimated circle of confusion to change the color information of the pixel of interest to the influence of the surrounding pixels. It is trying to restore to a state without.

  According to another aspect of the present invention, there is provided an image sharpening device that uses, as a target pixel, all the pixels constituting the target image for the target image in which the in-focus area and the out-of-focus area are mixed, and color information of the target pixel. Estimate a circle of confusion that indicates the range of surrounding pixels that affect the color, and use the inverse filter of the blur function based on the estimated circle of confusion to restore the color information of the pixel of interest to a state that does not affect the surrounding pixels I am doing so.

  According to another aspect of the present invention, there is provided a program for sharpening an image, wherein a target image in which a focal point area and an out-of-focus area are mixed is used as a target pixel for all the pixels constituting the target image. Estimate a circle of confusion that indicates the range of surrounding pixels that affect the color, and use the inverse filter of the blur function based on the estimated circle of confusion to restore the color information of the pixel of interest to a state that does not affect the surrounding pixels As a means, the computer is made to function.

  Therefore, according to the image sharpening method, apparatus, and program of the present invention, a circle of confusion is estimated for each pixel of the target image, and an appropriate blur function inverse filter is used for each pixel of the target image. Therefore, the out-of-focus area can be improved without deteriorating the in-focus area, and an image in which the in-focus area and the out-of-focus area are mixed can be clarified. Here, in this specification, the in-focus region does not indicate only a pixel or a pixel group that is completely in focus, but may be a pixel or a pixel group that is approximately in focus. The present invention is also effective for an image having a plurality of out-of-focus conditions with different circles of confusion.

  Further, the invention according to claim 2 is the image sharpening method according to claim 1, in estimating the circle of confusion, the diameter of the circle of confusion is increased from the initial value by a predetermined increment value, and Based on the diameter of the circle of confusion before and after the increase, the correlation coefficient between the two images obtained by temporarily restoring the color information of the pixel of interest is obtained, and the diameter of the circle of confusion where the correlation coefficient is maximized is determined to be a true value. ing. In this case, by using the fact that the value of the correlation coefficient does not change so much before and after the correct value of the diameter of the circle of confusion, an appropriate diameter of the circle of confusion for the pixel of interest can be selected.

  Thus, according to the first to fourth aspects of the present invention, since an image in which the in-focus area and the out-of-focus area are mixed like a facility monitoring image can be clarified, for example, an unclear part of the image (out-of-focus area ), Even if any problem occurs, the unclear part can be clarified and the problem can be made obvious and a quick response can be made. In addition, in order to automatically detect an abnormality by image processing, an object recognition function is required, and there is a possibility that the object recognition ability may be reduced when blur is included locally like a monitoring image. The object recognition rate can be improved by sharpening the locally blurred monitoring image according to the present invention. The present invention is particularly suitable for a field that requires image monitoring over a wide range, such as equipment monitoring images, and that requires a clear image over the entire image.

  Hereinafter, the configuration of the present invention will be described in detail based on embodiments shown in the drawings.

  1 to 14 show an embodiment of an image sharpening method and apparatus and program according to the present invention. This image sharpening method is based on a target image in which an in-focus area and an out-of-focus area are mixed, with all the pixels constituting the target image as the target pixel, and the peripheral pixels that affect the color information of the target pixel. A circle of confusion that indicates a range is estimated, and the color information of the pixel of interest is restored to a state that is not affected by surrounding pixels by using an inverse filter of a blur function based on the estimated circle of confusion.

  For example, the target image in the present embodiment is a gray-scale digital image for the sake of simple processing. Each pixel constituting the target image is assigned a brightness value (luminance value) of 256 gradations from 0 (black) to 255 (white), for example, as color information. However, the target image is not necessarily limited to a grayscale image, and may be a color image. Further, an analog image converted into a digital image may be a target image.

  A process (blur process) in which a blurred image (referred to as a blur image) is generated from an image without blur (referred to as an original image) includes, for example, a blur function (PSF), an original image, and Modeled by adding noise to the convolution of.

However,
x: horizontal coordinate position y of the pixel of interest in the image y; vertical coordinate position g (x, y) of the pixel of interest in the image; luminance value h (x, y) of the pixel of interest in the blurred image; PSF
f (x, y); luminance value n (x, y) of the pixel of interest in the original image; noise

  As an inverse filter of the blur function (PSF), for example, a winner filter is used in the present embodiment. However, as an inverse filter, in addition to the basic Wiener filter, there are many known general inverse filters, restricted least square filters, projection filters, etc. Of course, these inverse filters may be adopted. is there. The restoration of the blurred image to the original image using the Wiener filter is expressed, for example, as Equation 2.

However,
F (u, v); Fourier transform of f (x, y) G (u, v); Fourier transform of g (x, y) H (u, v); h (x, y) Fourier transform H ^* (u, v); complex conjugate of H (u, v) Wn (u, v); noise power spectrum Wf (u, v); original image power spectrum

  The blur due to defocusing is a blur that occurs when the distance between the lens and the subject does not match the focal length of the lens, and the PSF is expressed as Equation 3.

However,
x ″; horizontal coordinate position y ″ with reference to the target pixel in the image; vertical coordinate position d with reference to the target pixel in the image; diameter of the circle of confusion

  The target pixel is affected by the luminance value of the pixel in the circle of confusion with the diameter d. In this specification, pixels other than the target pixel included in the circle of confusion centered on the target pixel are referred to as peripheral pixels. When the circle of confusion circle diameter d is 1 pixel or less (that is, the radius of the circle of confusion is 0.5 pixels or less), the focus is in focus (that is, the number of surrounding pixels is 0), and the luminance value of the pixel of interest is correctly expressed. The When the circle of confusion circle diameter d exceeds 1 pixel, the luminance value of the pixel of interest is affected by the luminance values of surrounding pixels, resulting in a blurred image (unclear image). The relationship between the diameter of the circle of confusion and the blur will be described with reference to FIG. In FIG. 1, the pixel of interest is represented by reference numeral 1, the peripheral pixels are represented by reference numeral 2, and the circle of confusion is represented by reference numeral C. In the case of the circle of confusion C1 shown in FIG. 1, the average of the luminance value of the pixel of interest 1 in the original image and the luminance values of the four peripheral pixels 2a is the luminance value of the pixel of interest 1 in the blurred image. In the case of the circle of confusion C2 having a larger diameter d, the luminance value of the pixel of interest 1 is further affected by the peripheral pixel 2b (the hatched pixel in FIG. 1) and becomes a uniform luminance value in a wider range. The image becomes more blurred.

  FIG. 4 shows an example in which the image is blurred by changing the diameter of the circle of confusion. 4A shows the case where the diameter d of the circle of confusion is 1.4 pixels, FIG. 4B shows the case where the diameter d of the circle of confusion is 2.4 pixels, and FIG. 4C shows the circle of confusion. The case where the diameter d is 3.4 pixels is shown. As described above, when the diameter of the circle of confusion increases, the image becomes unclear due to the influence of the surrounding pixels.

  When a blur image caused by defocusing is targeted, if this circle of confusion is correctly estimated, the blur image can be restored to a clear one using an inverse filter. For example, when the blurred image in FIG. 4A is restored using the Wiener filter described above, a clear image is obtained as shown in FIG. However, if the circle of confusion cannot be estimated correctly, an unclear image is generated by the inverse filter. For example, if the blurred image of FIG. 4A blurred by the circle of confusion circle diameter d = 1.4 is used with the Wiener filter assuming that the circle circle diameter d is 1.8 pixels, there are many high frequency components and noise. The image is as shown in FIG. Furthermore, for example, if the blurred image of FIG. 4A is used with the Wiener filter assuming that the diameter d of the circle of confusion is 3.4 pixels, the image shown in FIG. 7 is completely unclear. In an image in which an in-focus area and an out-of-focus area are mixed, the diameter of the circle of confusion differs depending on the pixel. Therefore, even if an inverse filter is applied to such an image based on a single PSF, a clear image cannot be obtained. I understand that.

  Therefore, in order to sharpen an image in which the in-focus area and the out-of-focus area are mixed, it is important to accurately estimate the diameter of the circle of confusion for each pixel constituting the target image. For example, in the present embodiment, in estimating the circle of confusion, the diameter of the circle of confusion is increased from the initial value by a predetermined increment value, and the color information (luminance value) of the pixel of interest is calculated based on the diameter of the circle of confusion before and after the increase. ) Is temporarily restored, and the circle of confusion circle where the correlation coefficient is maximized is determined to be a true value. This utilizes the fact that the value of the correlation coefficient does not change much before and after the correct value of the circle of confusion circle (ie, the true value). Of course, the radius of the circle of confusion may be increased from the initial value by a predetermined increment value, and the radius of the circle of confusion where the above correlation coefficient is maximized may be determined as a true value. .

  The image sharpening method of the present invention can be implemented as an image sharpening device using, for example, a known computer. For example, as illustrated in FIG. 14, the image sharpening device 10 includes an input unit 11 to which a target image is input, and each of all pixels constituting the target image with respect to the target image input from the input unit 11. Is used as a pixel of interest, and a circle of confusion indicating a range of surrounding pixels that affect the color information of the pixel of interest is estimated, and color information of the pixel of interest is obtained using an inverse filter of a blur function based on the estimated circle of confusion. A central processing unit (CPU) 12 that executes processing for restoring the image to a state free from the influence of surrounding pixels, an output unit 13 that outputs the restored image (also referred to as a restored image), a target image, and a restored image Main storage device (for example, RAM) 14 and external storage device (for example, hard disk) 15 are recorded. The hardware resources are electrically connected through the bus 16, for example. The input unit 11 is not particularly limited. For example, a reading device (CD drive or the like) of a medium on which the target image is recorded, a network interface that receives target image data from an external device through a line, or the like may be used. Further, the output unit 13 is not particularly limited, and for example, a display that displays a restored image, a network interface that transmits restored image data to an external device through a line, or the like may be used. In addition, for example, the image sharpening program of the present invention is recorded in the external storage device 15, and the computer functions as the image sharpening device 10 by being read and executed by the CPU 12.

  2 and 3 show an example of processing executed by the image sharpening apparatus 10 using the image sharpening program according to the present embodiment. Note that the target image is input to the image sharpening device 10 from the input unit 11 and read in advance before the processing in FIGS. 2 and 3. As shown in FIG. 2, in this embodiment, for example, all the pixels constituting the target image are sequentially set as the target pixels (S1 to S4, S6, S7), and the blur improvement process (S5) is performed on the target pixels. I have to. Note that x in FIG. 2 indicates a horizontal coordinate position in the target image, and y indicates a vertical coordinate position in the target image. Further, x_max in FIG. 2 indicates the number of pixels in the horizontal direction in the target image, and y_max indicates the number of pixels in the vertical direction in the target image.

FIG. 3 is a detailed flowchart of the blur improvement process (S5) of FIG. In this blur improving process, first, d representing the size of the diameter of the circle of confusion is initialized (S501). For example, in this embodiment, “1” is substituted for d as an initial value. Next, a local region centered on the pixel of interest (x, y) is cut out (S502). For example, in this embodiment, an area of 32 × 32 pixels centering on the pixel of interest is cut out. When the target pixel is the pixel at the end of the target image, the local area is not necessarily 32 × 32 pixels, and the size of the range that can be cut out. However, the size of the local region, the cutting method, and the like are not limited to the above example. Next, the luminance value of the pixel of interest is restored with an inverse filter using the PSF (Formula 3) when the diameter of the circle of confusion is set to d pixels with respect to the image of the extracted local area, and the image of the local area after the restoration is restored. to save an image _{I 1,} for example, in the main memory 14 or the external storage device 15 (S503). For example, in the present embodiment, the luminance value of only the pixel of interest in the local region is restored using the Wiener filter expressed by Formula 2. Note that the restored luminance value of the target pixel is not yet determined, and this restoration is also referred to as temporary restoration.

Then, the following process is repeated until the circle of confusion diameter d reaches a predetermined maximum value (indicated by d_max in FIG. 3) (S504; No). That is, for the local region image, the luminance value of the pixel of interest is restored by the Wiener filter expressed by Equation 2 using PSF (Equation 3) when the diameter of the circle of confusion is d pixels, and the restored local region to save an image of an area as an image _{I 2,} for example, the main storage device 14 or the external storage device 15 (S505). Note that the restored luminance value of the pixel of interest is not yet determined, and the restoration here is also temporary restoration. Then, the correlation between the image _{I 1} and the image _{I 2,} i.e. obtaining a correlation coefficient between the image _{I 1} and the image _{I 2} (S506). In calculating the correlation coefficient, color information (luminance value) of pixels constituting the image I ₁ and color information (luminance value) of pixels constituting the image I ₂ are used. In the case of this embodiment, since the image I ₁ and the image I ₂ are initially the same, the correlation coefficient is 1. Then, the value of the circle of confusion circle d and the value of the correlation coefficient are recorded in the main storage device 14 or the external storage device 15 (S507). Next, the image _{I 2} and the image _{I 1} (S508). For example, the data of the image I ₂ is overwritten on the recording area where the data of the image I ₁ was recorded. Then, the value of the confusion circle diameter d is increased by a predetermined increment Δd (S509). For example, in the present embodiment, 0.5 pixel is set as Δd. However, the value of Δd is not limited to this example. The processes of S505 to S509 are repeated until the confusion circle diameter d reaches a predetermined maximum value d_max. For example, in this embodiment, 25 pixels are set as d_max, but the value of d_max is not limited to this example. As described above, the confusion circle diameter d is increased by 0.5 pixels from the initial value of 1 pixel, and between the _two images I ₁ and I ₂ temporarily restored based on the increasing and decreasing confusion circle diameter d. The correlation coefficient is obtained.

  When the circle of confusion diameter d becomes 25 pixels or more (S504; Yes), the correlation coefficient is maximized based on the value of the circle of confusion circle and the value of the correlation coefficient recorded in the main storage device 14 or the external storage device 15. The value of the circle of confusion circle diameter is searched, and the value of the diameter of the circle of confusion searched is determined to be a true value. FIG. 11 shows the relationship between the size of the circle of confusion and the value of the correlation coefficient. The horizontal axis in FIG. 11 indicates the diameter of a circle of confusion centered on the pixel of interest. A broken line in FIG. 11 indicates a point where the correlation coefficient is maximized (maximum extreme value). Then, the luminance value of the pixel of interest is obtained by the Wiener filter expressed by Equation 2 using the PSF of the circle of confusion circle determined to be a true value (Equation 3), and the luminance value is used as the luminance value of the pixel of interest in the restored image. (S510).

  The blur improvement process (S5) described above is performed for all the pixels constituting the target image (S1 to S4, S6, S7), and a restored image in which the influence of the peripheral pixels is removed with respect to the luminance value of each pixel is obtained. Note that, when the blur improvement process (S5) is performed for a certain pixel of interest, and a pixel that has been subjected to the blur improvement process is included in a circle of confusion centered on the pixel of interest, the luminance value of the pixel that has been processed In this case, the luminance value before the blur improvement process is used.

  In order to verify the effectiveness of the present invention, the following simulation was performed. A blurred image with a larger diameter of confusion circle as it goes to the right in the image was created and used as the target image. The confusion circle diameter was varied from 2.1 to 9.0. This target image is shown in FIG. You can see that the image is blurred as it goes to the right. FIG. 9 shows a restored image obtained by applying the image sharpening method of the present invention to the target image of FIG. FIG. 10 shows the result of restoring the target image of FIG. 8 using the Wiener filter based on one PSF (that is, a uniform circle of confusion). In FIG. 10, the diameter of the circle of confusion is 5.6, which is the average of 2.1 to 9.0. Compared to FIG. 10, the image of FIG. 9 is clearly clearer. For example, the presence of the worker cannot be confirmed in FIG. 10, but the presence of the worker can be confirmed in FIG. Therefore, it can be confirmed that the present invention functions effectively with respect to an image in which an in-focus area and an out-of-focus area are mixed or an image having a plurality of out-of-focus areas with different confusion circles.

  It is important to improve image quality for visual inspection when considering the problem of blurring as a monitoring system for equipment such as electric power facilities. However, in order to use images as sensors, the effects on object recognition are clarified. It is necessary to keep. Therefore, in this example, the influence of the blurred image on the recognition rate of the object was examined, and it was confirmed that the present invention also contributes to the improvement of the recognition rate of object recognition.

  In order to investigate the effect of blur on object recognition, the effect of blur was evaluated on a method using SVM (Support Vector Machine) that shows strong object recognition ability. Further, COIL (Columbia Object Image Library) provided as a benchmark image for object recognition and identification was used. The COIL is an image database of a total of 7200 images composed of images in 72 directions obtained by photographing 100 objects by changing the orientation by 5 degrees. Of the 72 object images, 8 were selected, a total of 800 were selected and learned using SVM, and object recognition experiments were performed on the remaining 6400 images. In this embodiment, the experiment was conducted using only monochrome information.

  First, the resolution dependency of the object recognition rate was examined as follows. Since the sample image is 128 × 128 pixels, it is enlarged or reduced by a technique using DCT (Discrete Cosine Transform) to obtain an image of 16 × 16 pixels, 32 × 32 pixels, 64 × 64 pixels, 256 × 256 pixels. It was created. Furthermore, object recognition experiments were performed on a total of five types of original image sizes of 128 × 128. The SVM used here was ν-SVM. FIG. 12 shows the experimental results. From this result, it can be seen that the resolution itself does not significantly affect the object recognition rate.

  Next, the effect of the object recognition rate on blurring was investigated. A blur image was created by changing the diameter of the circle of confusion for the original image that did not include the blur, and an object recognition experiment was performed on the blur image. The result of the object recognition experiment is shown in FIG. It can be seen that the object recognition rate decreases as the circle of confusion increases. This means that the object recognition rate becomes worse as the image becomes unclear.

  Next, the effectiveness of the present invention when the object recognition is performed under an image in which the in-focus part and the out-of-focus part are mixed was examined. The experiment was performed as follows. The diameter of the circle of confusion was set to 1.6 pixels to simulate a state in which the focal point was almost in focus at the central part, and 3.7 pixels as a non-focused part in the peripheral part. A COIL image (image size is 32 × 32 pixels) was blurred using such PSF. Next, a restored image obtained by applying the image sharpening method of the present invention to the blurred image and a restored image obtained by applying a Wiener filter based on a conventional uniform PSF are created. An object recognition experiment was conducted.

  As a result, the recognition rate decreased by 1% in the blurred image before restoration, whereas the recognition rate recovered to 0.1% in the restored image sharpened by the present invention. However, in the restored image to which the Wiener filter based on the conventional uniform PSF is applied, the object recognition rate is deteriorated to 1.7%. Thus, it was confirmed that the present invention is not only suitable for visual monitoring but also contributes to an improvement in the recognition rate of object recognition.

  As described above, according to the image sharpening method, apparatus, and program of the present invention, the confusion region of the PSF is estimated for each pixel of the target image to improve the blur. The mixed images can be sharpened. Therefore, for example, if the target image is a facility monitoring image in which an in-focus area and an out-of-focus area are mixed, even if a problem occurs in an unclear portion (out-of-focus area) of the image, according to the present invention, the unclear area Since this problem can be clarified by clarifying such a part, it is possible to respond quickly. In addition, in order to automatically detect an abnormality by image processing, an object recognition function is required, and there is a possibility that the object recognition ability may be reduced when blur is included locally like a monitoring image. The object recognition rate can be improved by sharpening the locally blurred monitoring image according to the present invention. The present invention is particularly suitable for a field that requires image monitoring over a wide range, such as equipment monitoring images, and that requires a clear image over the entire image.

  The above-described embodiment is an example of a preferred embodiment of the present invention, but is not limited thereto, and various modifications can be made without departing from the gist of the present invention. For example, in the above-described embodiment, the target image is a grayscale image, but the target image may be a color image. In this case, for example, the color model is RGB, and red (R), green (G), and blue (B) take values from 0 to 255, and 256 × 256 × 256 colors (= 16777216 colors) can be expressed. 2, the processing shown in FIGS. 2 and 3 may be performed for each of the R value, the G value, and the B value.

  The inverse filter of the blur function (PSF) is not limited to the above-described Wiener filter, and a general inverse filter, a restricted least square filter, a projection filter, or the like may be used.

  Further, in the above-described embodiment, the blur improvement process (S5) is performed by sequentially using the pixels constituting the target image as the target pixel. However, the present invention is not limited to this example, and all the pixels constituting the target image are simultaneously selected as the target pixel. As described above, the blur improvement process (S5) may be performed in parallel for all the pixels.

It is a conceptual diagram for demonstrating the principle of the sharpening method of the image of this invention. It is a flowchart which shows an example of the process in the image sharpening method and apparatus of this invention, and a program. It is a flowchart which shows an example of the process which detailed the blurring improvement process of the flowchart of FIG. An example in which an image is blurred by changing the diameter of a circle of confusion, (A) shows a case where the diameter d of the circle of confusion is 1.4 pixels, and (B) shows a case where the diameter d of the circle of confusion is 2.4 pixels. (C) shows the case where the diameter d of the circle of confusion is 3.4 pixels. An example of an image restored using an inverse filter that correctly estimates a circle of confusion for the image in FIG. An example of an image restored using an inverse filter that failed to estimate the circle of confusion for the image in FIG. FIG. 4A shows another example of an image restored using an inverse filter that has failed to estimate the circle of confusion for the image in FIG. An example of the target image in which the in-focus area and the out-of-focus area are mixed is shown. An example of an image restored using the image sharpening method of the present invention with respect to the image of FIG. 8 is shown. An example of an image restored using the conventional method with respect to the image of FIG. 8 is shown. The relationship between the size of the circle of confusion and the correlation coefficient between the two images restored based on the circles of confusion before and after increasing the diameter of the circle of confusion. The vertical axis indicates the value of the correlation coefficient, and the horizontal axis Indicates the diameter of a circle of confusion centered on the pixel of interest. The relationship between the object recognition rate by SVM and the image resolution (pixel density) is shown, the vertical axis shows the object recognition rate [%], and the horizontal axis shows the number of pixels on one side of the square image. The relationship between the object recognition rate by SVM and the size of the circle of confusion is shown, the vertical axis shows the relative object recognition rate [%] when no blur is set to 1, and the horizontal axis shows the horizontal direction from the pixel of interest. Alternatively, the distance (number of pixels) from the pixel of interest to the peripheral pixels that are located farthest in the vertical direction and are included in the circle of confusion. It is a schematic block diagram which shows an example of the image sharpening apparatus of this invention.

Explanation of symbols

1 pixel of interest 2 peripheral pixel C circle of confusion 10 image sharpening device

Claims (4)

  For a target image in which an in-focus area and an out-of-focus area are mixed, a circle of confusion indicating a range of surrounding pixels that affect the color information of the target pixel is defined by using all the pixels constituting the target image as the target pixel. A clear image is characterized in that the color information of the pixel of interest is restored to a state free from the influence of the surrounding pixels by using an inverse filter of a blur function based on the estimated circle of confusion. Method.   In the estimation of the circle of confusion, the diameter of the circle of confusion is increased by a predetermined increment from the initial value, and the color information of the pixel of interest is temporarily restored based on the diameter of the circle of confusion before and after the increase. 2. The image sharpening method according to claim 1, wherein a correlation coefficient between two images is obtained, and a diameter of a circle of confusion at which the correlation coefficient is maximized is determined as a true value.   For a target image in which an in-focus area and an out-of-focus area are mixed, a circle of confusion indicating a range of surrounding pixels that affect the color information of the target pixel is defined by using all the pixels constituting the target image as the target pixel. An image sharpening apparatus that performs estimation and restores color information of the pixel of interest to a state that is not affected by the peripheral pixels by using an inverse filter of a blur function based on the estimated circle of confusion.   For a target image in which an in-focus area and an out-of-focus area are mixed, a circle of confusion indicating a range of surrounding pixels that affect the color information of the target pixel is defined by using all the pixels constituting the target image as the target pixel. A computer functioning as a means for estimating and restoring the color information of the pixel of interest to a state free from the influence of the surrounding pixels using an inverse filter of a blur function based on the estimated circle of confusion. Image sharpening program.