TWI420416B - An image up-sampling method - Google Patents

Description

Image enlargement method

The present invention relates to an image enlargement method, and more particularly to an image enlargement method suitable for performing digital image enlargement in a computer to obtain a clear enlargement result.

In the increasingly widespread application of digital imaging, and deep into human life, digital image up-sampling has become an important application technology. No matter the industry or the academic world, there is no spare power for the technical research of image enlargement. However, the enlargement of the digital image is to enlarge the image with a small number of pixels into an image with a large number of pixels. In the process of amplification, due to insufficient sample available for reference, pixel loss occurs, thereby causing jagged edges and images. Features such as blurred features, brightness decay, etc. Therefore, how to obtain clear amplification results is the goal pursued by the industry and has become an important issue in image magnification technology.

Among the published techniques, Freeman et al. published an imaging magnification method based on learning based, which can be found in Freeman et al., "Learning low-level vision," IJCV, 2000. Dai et al. published a method called soft edge prior, see Dai et al., "Soft edge smoothness prior for alpha channel super resolution," published in Proceedings of IEEE Conference on Computer Vision, 2007. Chang et al. published a method for magnifying images with neighbor embedding, see Chang et al., "Super-resolution through neighbor embedding," IEEE Conference on Computer Vision and Pattern Recognition, 2004. In addition, Yang et al. published a method for magnifying images with a sparse representation, as seen in Yang et al., "Image super-resolution as sparse representation of raw image patches," IEEE Conference on Computer Vision and Pattern Rcognition. , Vol. 9, pp1-8, June 2008).

The above method respectively proposes a method for magnifying the image quality from different angles, and obtains a clear image. However, the quality of the images obtained is still unsatisfactory. At the same time, these methods do not achieve clear quality when the image magnification is high. In addition, several technologies need to provide an external training set image when magnifying an image, which is difficult to perform automatically.

The co-inventor of this case and Shawmin Lei have proposed a special interpolation method that can be used to magnify digital images. See Sreedevi et al., "An examplar-based approach for texture compaction synthesis and retrieval," IEEE Transactions on Image Processing, Vol. 19, pp. 1307-1318, May 2010. This paper proposes a double-cube interpolation method to magnify the image. In the double-cube interpolation process, the double-interpolation method is further adjusted by using parameters such as an optimal matching threshold, a smoothing control threshold, a local dynamic (brightness) adjustment value, and a neighboring pixel standard deviation value. Although this method can provide a fairly clear magnified image, its application is biased towards images with higher homogeneity or regularity. For non-homogeneous images, the processing effect is still not satisfactory.

Therefore, it is indeed necessary to provide a novel image enlargement method to obtain a highly clear magnified image.

At the same time, it is also necessary to provide an image enlargement method that can perform image enlargement without any additional information.

It is also necessary to provide an image magnification method that can be used to amplify non-homogeneous images and obtain clear magnification results.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a novel image magnifying method in which a highly sharp magnified image can be obtained.

The object of the present invention is also to provide an image enlargement method capable of autonomously performing image enlargement without additional information.

It is also an object of the present invention to provide an image magnification method that can be used to amplify a non-homogeneous image to obtain a clear magnification result.

The present invention provides a three-stage image magnification method for upsampling a digital image in a computer device. In the first stage, the image to be processed is first subjected to downsampling, and in the second stage, an error image is obtained from the reduced sampled image. In the third stage, the image to be processed is gradually enlarged until the desired magnification is obtained, and the enlarged image to be processed is corrected with the erroneous image. The error image is a result of subtracting the image to be processed from the reduced sample image. The error image may also be a result of subtracting the image to be processed and the reduced sample image to the same magnification. The step of correcting the image to be processed may be performed during the amplification process or after the amplification is completed.

In the embodiment of the present invention, the magnification of the downsampling is preferably between 1/10 and 1/1, preferably 1/2.

In an embodiment of the invention, when the error image is enlarged, it may include amplification by interpolation. In some of these embodiments, the amplification method is a bi-cubic interpolation.

In an embodiment of the invention, a step of correcting the error image may be included, the step including a process of sharpening the error image. The sharpening process may include sharpening the error image with at least two kinds of sharpening degrees to obtain at least two different sharpness degree error images, and characterizing the error images of the at least two different sharpening degrees. , the step of adding the error image. The step of adding the at least two different sharpness degree error images to the error image may include a best matching process.

In some examples of the present invention, before the image to be processed is subjected to the enlargement process, a step of sharpening the image to be processed may be further included. The sharpening process may include a step of performing a low-sharpening process on the image.

In some examples of the invention, the magnifying process of the image to be processed may include an interpolation process. Wherein, the interpolation method may include a double cubic interpolation process. The double-cube interpolation process may further include adjusting at least one of a parameter such as an optimal matching threshold, a smoothing control threshold, a local dynamic (brightness) adjustment value, and a neighboring pixel standard deviation value. The steps of the interpolation calculation.

The method of the present invention may further comprise the step of sharpening the resulting image after the correction and amplification have been completed.

The above objects and advantages of the present invention will become more apparent from the detailed description and appended claims.

The image enlargement method of the present invention mainly comprises three major steps. First, get a pending image. Down sampling is performed from the image to be processed in a first step to obtain a reduced sample image with a small number of pixels. Secondly, an error image is extracted from the reduced sample image, representing an error portion generated when the image is enlarged. In the final stage, the image to be processed is enlarged by interpolation, and the wrong part of the enlarged image is removed, and a magnified result with good effect is obtained.

Fig. 1 is a flow chart showing the image enlargement method of the present invention. As shown in the figure, the image I to be processed is first obtained on the 101 computer device. The image I to be processed is sampled at 102 to obtain a reduced-sampled image (down-sampling). When this step is performed, the image to be processed can be sampled in any manner as long as an image with a small number of pixels can be obtained. In practice, the pixels can usually be captured in the image to be processed. The ratio can be from 1/10 to 1/10, depending on the image size (number of pixels), system resource cost, and required resolution. Depending on the conditions. A ratio of more than 1/10 or less than 1/10 is also applicable to the present invention.

In this example, the sampling ratio is set to 1/2, that is, the pixel of the image to be processed, each of two adjacent pixels as a group, one of the pixels is taken, and the obtained pixels are combined into one reduced sampling. Image D, D = 1/2 (I).

In step 103, the reduced-sampled image is enlarged to obtain the mapped image Rb. When zooming in, the reduced-sampled image can be directly enlarged to the original scale to become the mapped image. However, in the present embodiment, the image is enlarged by a method commonly referred to as "bi-cubic interpolation". The bicubic interpolation amplification method calculates the value of the unknown pixel located at the center of the 4×4 matrix by pixels in a 4×4 matrix. The advantage of this method is that a clearer magnified image can be obtained, but the disadvantage is that the calculation is complicated and time consuming. In this example, the reduced-sampled image is enlarged to become a mapped image Rb, Rb=2D.

The receiver obtains the error image. In step 104, the image to be processed I is subtracted from the mapped image Rb to obtain an error image E. E=I-Rb.

In this step, a step of repairing the mapped image Rb may be included. This step is to process the mapped image Rb by best matching. The method is that the mapped image Rb is optimally matched with the image to be processed I, a minimum sharpened image IL, a medium sharpened image IM, and a highest sharpened image IH, so that it can include each image file. The best match. The method of obtaining the minimum sharpened image IL, the medium sharpened image IM, and the highest sharpened image IH includes "unmask sharpening" to obtain different degrees of edge sharpening images. Wherein, according to the obtained edge sharpening degree, the lowest sharpening image IL, the medium sharpening image IM, and the highest sharpening image IH are respectively determined.

In the example of the present invention, the parameter at the time of sharpening can be set as follows: for the lowest sharpened image IL, the radius control parameter (radius) is set to 3.0 (pixel), and the sharpness degree value (amount) is 0.6; For medium sharpened image IM, set its radius control parameter to 3.0 (pixels), and the sharpness degree is 0.9; for the highest sharpened image IH, set its radius control parameter to 3.0 (pixels), and the sharpness value is 1.2. Of course, other parameter settings are also applicable to the present invention as long as three different levels of sharpening images can be obtained. The degree of sharpening is naturally not limited to three. Less than 3 or more than 3 are not available. The image quality required for all-view and the processing cost of the computer are determined. In addition, when performing the optimal matching process, the threshold value can be set to 4, representing the error threshold value, that is, the mean square error. The experiment found that the threshold is a trade-off between the best match result and the signal-to-noise ratio. If the setting is too high, the noise in the image will increase, otherwise it is not easy to find the best match. In this embodiment, the value of the value is set to 4, and the obtained structural similarity (SSIM) of the repaired image is about 0.95, and the peak signal to noise ratio (PSNR) is excellent. In the conventional technology. Of course, as mentioned above, the setting of this value is a result of a trade-off, above or below 4, which can be applied to the present invention, and is not a technical limitation.

In this example, the mapped image Rb is optimally matched with the lowest sharpened image IL, the medium sharpened image IM, and the highest sharpened image IH, that is, a repaired mapped image R is obtained. In step 104, the image to be processed I is subtracted from the mapped image R to obtain an error image E. E=I-R.

Thereafter, the method of the present invention enters the stage of image enlargement. In this stage, the image to be processed I is first enlarged by a certain multiple in step 105. Here, the magnification is not limited. For example, the user can choose to zoom in to the desired multiple at a time, or gradually zoom in to a desired multiple in a few steps. In an embodiment of the invention, the stepwise magnification is used to amplify to the desired multiple in most steps. Under this practice, it is more suitable to enlarge the magnification by 1.10 times to 3.0 times each time. Among them, a better combination can be achieved by amplifying 1.50 times between the processing effect and the computing time. But not any technical limitations.

In this step, the image to be processed that has undergone preliminary processing may also be selected as the image to be processed. For example, one of the sharpened sharpened image IL, the medium sharpened image IM, and the highest sharpened image IH. These images have been sharpened and can provide superior magnification. Among them, in the preferred embodiment of the present invention, the minimum sharpening image IL is used to obtain a superior effect.

Any known amplification technique can be used when performing the amplification process. For example, the interpolation method commonly used in the industry can be applied. However, what kind of amplification technique is used is not a technical limitation of the present invention. As long as the image can be enlarged, a predetermined multiple can be achieved. In the preferred embodiment of the present invention, the above-described bicubic interpolation method is proposed as an enlarged processing method. This method provides excellent magnification quality for sharpened images. However, other amplification methods can still be used in the present invention.

In the processing steps of the above-described double-cube interpolation method, several parameter values can be used to improve the amplification effect. For example, the adjustment parameters proposed in the aforementioned Sreedevi et al. paper, that is, the optimal matching threshold, the smoothing control threshold, the local dynamic adjustment value, the adjacent pixel standard deviation value, etc., can be used to adjust the bicubic interpolation method. Calculate for better image quality. The best matching threshold is used to control the value of the relevant pixel so that it does not fall within the error range. The smoothing control threshold (smooth control thresholod) is used to control the smoothing process so as not to be too high. The local dynamics are used to adjust the brightness of the pixels. The mean error of neighboring pixels is a value obtained after the best matching process is performed on the mapped image.

Next, at step 106, the error image E is also magnified by the same magnification. The amplification method may use any known image enlargement method, which may be the same as or different from the enlargement method of the image to be processed I. In step 107, the error image is subtracted from the image to be processed, and the corrected image Z, Z=I-E is obtained.

At step 108, it is determined whether the magnification of the corrected image Z has reached a desired multiple. If the result of the determination is no, the step returns to 105, and the corrected image is enlarged by a certain magnification. If yes, it is judged that the correction is completed and the next step is entered.

In the present invention, steps 106 and 107 can be interchanged with step 108. It is claimed that the image to be processed can be gradually enlarged to a desired multiple before the corrected image to be processed is corrected by the error shadow. However, due to the gradual enlargement of the processing strategy in this example, the gradual correction may be more applicable.

After the above processing, the image has been enlarged to the desired magnification and corrected to become an image that meets the demand. Here, the obtained image can be further sharpened in step 109 to eliminate the sawtooth portion of the image and further improve the quality. As mentioned previously, the sharpening method can use any known technique. This will not be repeated here.

Fig. 2 shows the results of processing in which three images are enlarged by the method of the present invention. When performing image enlargement, the sharpening software used is the NIP2 software commonly used in the industry. The figure shows that the graphs a, b, and c are each magnified by 36 times to obtain the results of the graphs d, e, and f. It is shown that the enlarged image can still be maintained with clear quality using the method of the present invention. Figure 3 shows a comparison of the effects of the method of the present invention with the prior art, wherein Figure a is the image to be processed, b is the result of a 16-fold magnification by the learning based method, and c is the result of a 16-fold magnification by the soft edge prior method. d is the result of a 16-fold magnification by the neighbor embedding method, e is a result of a 16-fold magnification by the sparse representation method, and f is a result of a 16-fold magnification by the method of the present invention. The method of the present invention is shown to provide a magnified image that is sharper than conventional techniques.

In fact, using the method of the present invention, not only can a clear magnification effect be obtained, but also in the amplification and correction of the image, only the training set image generated by the image to be processed is used, and no external training is needed. The image is collected, so that it can be processed autonomously, without selecting the applicable training set image, and saving the processing time.

The above is a description of an embodiment of the image enlargement method of the present invention, which is used to exemplify the present invention. It is not difficult for those who are accustomed to the art to make various changes and derivatives. It is within the scope of the invention as long as it does not go beyond the scope of the patent application of the present invention.

Fig. 1 is a flow chart showing the image enlargement method of the present invention.

Fig. 2 shows the results of processing in which three images are enlarged by the method of the present invention.

Fig. 3 is a schematic view showing the comparison of the effects of the method of the present invention and the prior art.

Claims (14)

An image enlargement method for upsampling a digital image in a computer device, the method comprising the steps of: obtaining a digital image to be processed; reducing the sampled image to be processed; Obtaining an error image in the image; gradually expanding the image to be processed until a desired magnification is obtained, and correcting the enlarged image to be processed with the erroneous image; wherein the error image is the image to be processed and the reduced sampling a result of image subtraction; the method further comprising the step of correcting the error image, the step comprising a process of sharpening the error image; wherein the sharpening process comprises sharpening with at least 2 sharpening degrees The error image is obtained by obtaining at least two different sharpness degree error images, and adding the characteristics of the at least two different sharpness degree error images to the error image; wherein the at least two different sharpening are performed The feature of the degree of error image, the step of adding the error image, includes a best matching process. The method of claim 1, wherein the error image is a result of subtraction after the image to be processed and the reduced sample image are enlarged to the same magnification. The method of claim 1, wherein the step of correcting the image to be processed is performed during the zooming process. The method of claim 1, wherein the step of correcting the image to be processed is After the enlargement process is completed. The method of claim 1, wherein the downsampling ratio is between 1/10 and 1/1. The method of claim 5, wherein the downsampling magnification is 1/2. The method of claim 1, wherein the error image is enlarged by interpolation. The method of claim 7, wherein the error image is enlarged by a bi-cubic interpolation. The method of claim 1, wherein the image to be processed further includes a step of sharpening the image to be processed before performing the enlargement process. The method of claim 9, wherein the sharpening process comprises the step of subjecting the image to a low sharpening process. The method of claim 1, wherein the enlargement processing of the image to be processed includes an interpolation process. The method of claim 11, wherein the interpolation comprises a double cubic interpolation process. The method of claim 12, wherein the double-cube interpolation process further includes a best matching threshold, a smoothing control threshold, a local dynamic (brightness) adjustment value, a neighboring pixel standard deviation value, and the like. At least one of the parameters adjusts the step of the bicubic interpolation calculation. The method of claim 1, further comprising the step of sharpening the resulting image after the correction and amplification are completed.