CN1267882C - Method and system for improving display resolution in images using sub-pixel sampling and visual error filtering - Google Patents

Abstract

The present invention provides a method and an apparatus for converting a color image of a higher-resolution to an image of a lower-resolution. With the method of the present invention, based on characteristics of spatial frequency responses of the human visual system, a lower-resolution image (116) is formed by separating a higher-resolution image (72) into a luminance channel (76) and chrominance channels (98 & 100) so as to carry out suitable sampling and filtering for each of the luminance channel (76) and the chrominance channels (98 & 100), then by combing the luminance channel (86) and the chrominance channels (111 & 114) after the sampling. As to the chrominance channels (98 & 100), traditional sub-sampling (101 & 103) is carried out to avoid chromatic aliasing, while sub-pixel sampling (80) is performed for the luminance channel (76) to improve resolution of luminance component.

Description

Method, device and system for transforming images

technical field

Embodiments of the present invention relate to the field of displaying high resolution images on a display having a lower resolution, wherein the display uses a tristimulus configuration to display the R, G, and B or other components of the image. Such triad configurations are common in direct view LCD displays, for example, and in such configurations a single pixel consists of 3 side-by-side sub-pixels. Each sub-pixel controls only one of the three primary colors (ie, R, G, and B), and each sub-pixel itself is typically controlled by only the primary color of a digital image display. High-resolution images may be available in memory, or may be available directly from algorithms (vector graphics, certain font designs, and computer graphics).

Background technique

The subject of this application is related to that filed on December 12, 2000 by Scott Daly, entitled "Methods and Systems for Improving Display Resolution using Sub-Pixel Sampling and Visual Error Compensation" method and system for display resolution)" in U.S. Patent Application No. 09/735,454. This patent application is hereby incorporated by reference.

The subject of this application is identical to that filed on December 12, 2000 by Rajesh Reddy KKovvuri and Scott Daly, entitled "Methods and Systems for Improving Display Resolution in Achromatic Images using Sub-Pixel Sampling and Visual Error Compensation (by using sub-pixel Sampling and visual error filtering to improve the display resolution of monochrome images)" in US Patent Application No. 09/735,425. This patent application is hereby incorporated by reference.

The most commonly used method for displaying a high-resolution image on a lower-resolution display is to downsample the pixels 2 of the high-resolution image 4 to the resolution of the low-resolution display 6, as shown in Figure 1 . The R, G, B values of each downsampled color pixel 8 are then transformed into separate R, G, B elements 10 , 12 , 14 for each display pixel 16 . These R, G, B elements 10, 12, 14 of a display pixel 16 are also referred to as sub-pixels. Because the display device does not allow color elements to overlap, the sub-pixels only take one of the three colors R, G, or B, however, the magnitude of the color can vary over the entire gray scale range (eg, 0-255). The sub-pixels typically have an aspect ratio (width:height) of 1:3, so that the resulting pixel 16 is square. The subsampling/transformation technique does not take into account that the R, G, B subpixels of the display are spatially arranged; in fact, they are assumed to overlap in the same way as they do in the high resolution image shown in Figure 1 of.

The pixels of the high-resolution image 4 are shown as three slightly offset stacked squares 8, indicating that their RGB values are associated with the same spatial location (ie, pixel). A display pixel 16 comprising one of each of the R, G, B sub-pixels 10, 12, 14 is shown as part of the lower resolution tristimulus display 6 on Fig. 1 by using darker colored lines. Other display pixels are indicated by lighter dotted lines.

In this example, the high resolution image has a resolution (in both horizontal and vertical scales) that is 3 times greater than the display resolution. Since this direct subsampling technique causes aliasing artifacts, various methods are used, such as averaging a sampled pixel's neighboring unsampled pixels. It should be noted that this general technique of averaging neighboring elements while subsampling is mathematically equivalent to prefiltering a high-resolution image with a rectangular filter. Additionally, it should be noted that the technique of selecting a pixel other than the leftmost (as shown in this figure) can be viewed as a pre-filter that only affects phase. Therefore, most of the processing related to preventing aliasing can be regarded as a filtering operation for high-resolution images, even if only the kernel (Kernel) is applied to the sampled pixel positions.

Monochrome images, as specified in this specification, have no visible color shift. This monochromatic condition can occur when the image contains only one layer or one color channel, or when the image has multiple layers or multiple color channels, but each color layer is the same, thereby producing a single color image.

It has been seen that the foregoing techniques do not take advantage of the potential display resolution. Background information on this can be obtained by referring to R. Fiegenblatt (1989), "Full color imaging on amplitude color mosaic displays", Proc. SPIE V. 1075, 199-205; and J. Kranz and L. Silverstein (1990), "Color matrix display image quality: The effects of luminance and spatial sampling (color matrix display image quality: the effect of luminance and spatial sampling)", SIDSymp.Digest 29 -32, these articles are cited here for reference. For example, in the display shown in Figure 1, when the resolution of display pixel 16 is 1/3 of the resolution of high-resolution image (source image) 4, the resolution of subpixels 10, 12, and 14 is equal to that of the source Resolution (on the horizontal scale). If the display is only used by color-blind people, it is possible to exploit the spatial location of the sub-pixels. This approach is shown in Figure 2 below, where the R, G and B subpixels 10, 12 and 14 are shown taken from the corresponding colors of the different pixels 11, 13 and 15 of the high resolution image. This allows the horizontal resolution to be sub-pixel resolution, which is three times the display's pixel resolution.

But what about viewers of displays that are not colorblind (ie, most viewers)? Fortunately for display engineers, even observers with perfect color 5 vision are color blind at the highest spatial frequencies. This is shown in Figure 3 below, which shows the idealized spatial frequency response of the human visual system.

Here, luminance 17 refers to the monochromatic visual observation of the image being viewed, and chroma 19 refers to the color content, which is processed by the visual system as equal brightness modulations from red to green and blue to yellow. The color difference signals R-G and B-Y of the video image roughly approximate these modulations. For most observers, the bandwidth of the color frequency response is 1/2 the bandwidth of the luminance frequency response. Sometimes the bandwidth of the blue-yellow modulation response is even smaller, down to about 1/3 of the brightness. Sampling involving the conversion of color elements from different image pixels to subpixels of a display pixel tristimulus, as shown in Figure 2, may be referred to as subpixel sampling.

With reference to Fig. 4, on the horizontal direction of display, there is a frequency range, in the Nyquist (Nyquist) frequency of display pixel 16 (display pixel=three-color group pixel, give three-color group Nyquist for each three-color group pixel 0.5 period) and the Nyquist frequency of subpixel elements 10, 12 and 14 (0.5 period per subpixel = 1.5 period/tricolor group pixel). This area is shown as a rectangular area 20 on FIG. 4 . The sinc function obtained by convolving the high-resolution image with a rectangular function whose width is equal to the displayed sampling interval is shown as a shallow dotted line 22 . This is the most common approach taken to model the display MIF (modulation transfer function) when the display is an LCD.

The sinc function resulting from convolving the high-resolution source image with a rectangle equal to the sub-pixel spacing is shown as dashed line 24, which has a higher bandwidth. This is a limitation imposed by the display when considering that sub-pixels are rectangular in one dimension. In the shown rectangular area 20, the sub-pixels can display brightness information, but no color information. In fact, any color information in this region is obfuscated. Thus, in this region, by allowing for color aliasing, we can obtain higher frequency luminance information than is allowed by tricolor (ie, display) pixels. This is the "favorable" region obtained by using sub-pixel sampling.

For font display applications, black and white fonts are typically preprocessed, as shown in FIG. 5 . Standard preprocessing includes hints, which refer to centering font strokes at pixel centers, ie, font stroke specific phase shifts. This is usually followed by low-pass filtering, also known as grayscale anti-aliasing.

The visual frequency response (CSF) shown in Figure 3 is idealized. In fact, they have a finite falling slope, as shown in Fig. 6(a). The luminance CSF 30 is converted from units of cy/deg (cycles per degree) into the display pixel domain (assuming a viewing distance of 1280 pixels). It is shown as a solid line 30 with a maximum frequency close to 1.5 cycles/pixel (display pixel), and it is bandpassed in shape with a tristimulus peak close to 0.2 cycles/pixel. R: G CSF32 is shown as a dotted line, ie low pass, with a maximum frequency close to 0.5 cycles/pixel. The B:Y-modulated CSF 34 is shown as a dot-dash, LPF curve with a similar maximum frequency to the R:G CSF, but with a lower maximum response. The range between the cutoff frequency of chroma CSF 32 and 34 and luma CSF 30 is the region where we can allow color aliasing in order to increase luma bandwidth.

Figure 6(a) also shows the idealized image power spectrum 36 as a function of 1/f, presented on the graph as a straight line with a slope of -1 (since the graph uses a logarithmic axis). This spectrum will repeat at the sampling frequency. These repetitions are shown as the horizontal sample rate for pixels 38 and sub-pixels 40 . The repetition occurring at the lower frequency 38 is due to pixel sampling, and the repetition occurring at the higher frequency 40 is due to sub-pixel sampling. It should be noted that the shape change is due to the fact that we plot it on the logarithmic frequency axis. The frequency of these repeated spectra extending to lower frequencies below the Nyquist frequency is known as aliasing. The leftmost repetition is color aliasing 38 because it is due to the pixel sampling rate, while luminance aliasing 40 occurs at a higher frequency because it involves higher subpixel sampling rates.

In Fig. 6(a), no pre-filtering is applied to the source spectrum. Consequently, the aliasing phenomenon due to pixel sampling (ie color aliasing) extends to very low frequencies 35 . Therefore, even though the color CSF has a narrower bandwidth than the luminance CSF, color artifacts are still visible (depending on the noise and contrast of the display).

In Fig. 6(b), we apply pre-filtering (equal to a rectangular function of three source image pixels) to the source power spectrum, shown as dotted line 22 in Fig. 4, and it can be seen that it affects over 0.5 cycles/ The baseband spectrum of the pixel 42, such that it has a steeper slope than -1, is shown as 44. These repetitions also show the effect of this prefiltering. Even with this filtering, we see that some color aliasing (repetitive spectrum at lower frequencies) occurs at frequencies 46 below the cutoff frequencies of the two chrominance CSFs 32a and 34a. Thus, it can be seen that simple luminance pre-filtering will struggle to remove color aliasing, failing to remove all luminance frequencies above 0.5 cycles/pixel (ie, "favorable" regions).

Since we rely on the difference in bandwidth of the visual system as a function of luminance or chrominance, giving us a luminance bandwidth boost in the "region of interest" 20, one possibility is to Designing pre-filtering: C. Betrisey et al. (2000), "Displaced filtering for patterned displays", SID Symposium digest, 296-299, which is hereby incorporated by reference, and shown in Figure 7.

This technique ideally uses different prefilters depending on which color layer and which color sub-pixel the image is being sampled from. Thus, there are 9 filters. They can be designed by using the human visual difference model described in the following article: X. Zhang and B. Wandell (1996), "A spatial extension of CIELab for digital color image reproduction (CIELab for digital color image reproduction Spatial extension of )", SID Symp. Digest 731-734, which article is hereby incorporated by reference, and is shown in Figure 7. This is done offline, assuming the images are always black and white. In the final implementation, to save computation, a rectangular function is used instead of the resulting filter. In addition, there is still some residual color error that can be seen because the color aliasing phenomenon extends down to frequencies lower than the color CSF cutoff frequency (as seen in Figure 6(b)).

However, the vision model used does not take into account the masking properties of the visual system, which cause luminance-induced chromatic masking when luminance is at moderate to high contrast levels. Thus, in larger fonts, color artifacts along the font edges are obscured by the font's high luminance contrast. However, as the font size decreases, the brightness of the font decreases, and the same color artifacts become very noticeably visible (e.g., for very small fonts, the black/white segments of the font disappear, leaving only localized colored spots).

Contents of the invention

The present invention is created based on the above-mentioned characteristics of the spatial frequency response of the human visual system, in other words, on the fact that luminance CSF has a higher cut-off frequency than chrominance CSF. With the present invention, the low-resolution image is obtained by separating the higher-resolution image into luma data and chrominance data, performing appropriate sampling and filtering on each luma data and chrominance data, and combining the luma data after sampling and chrominance data are formed. As for chrominance data, conventional subsampling is performed to avoid color aliasing, while subpixel sampling is performed on luma data to increase the resolution of the luma component. Also, high pass filtering is performed on the sub-pixel sampled luminance data in order to remove low frequency artifacts that occur during sub-pixel sampling of the luminance data.

A conceptual illustration of the invention is given below. In FIGS. 1 and 2, the individual R, G, and B values of the R, G, and B subpixels 10, 12, and 14 in one display pixel 16 reflect the subsampling shown in FIG. The respective R, G, and B values of color pixel 8 (ie, 11 ) of image 4 of . However, the respective R, G and B values of sub-pixels 10, 12 and 14 are different from the respective R, G and B values of color pixel 8 (11). Therefore, the subpixel sampling shown in FIG. 2 is performed according to the luminance component so that the respective R, G and B values of the subpixels 10 , 12 and 14 reflect the luminance component of the color pixels 11 , 13 and 15 .

Embodiments of the present invention include methods and systems that rely less on filtering and its linearity assumptions, and are able to work on input color images. These embodiments are capable of removing low frequency color artifacts directly after they are caused by sub-pixel sampling. This is achieved by generating an LPF version of the color content of the image that is added to the luminance and color aliasing versions. This is done by removing color artifacts caused by sub-pixel sampling with a color control other than the additive, base color gamut (ie, RGB). In practice, only lower frequency color artifacts need to be canceled out, since high frequency color artifacts are not visible due to the narrower bandwidth of the color CSF, as shown in Figure 6(a).

The method and system of the present invention can be used when a display is viewed at a greater distance than design specifications to obtain a higher resolution luminance signal without seeing color aliasing. These techniques do not need to assume that the source image is text or that the image is monochrome.

Embodiments of the present invention transform a higher resolution image into a lower resolution image while at the same time errors caused by the subsampling process are reduced. When a higher resolution image is not in a format that allows separation of luma and chrominance data, the image is converted to such a format. Many opposing color gamuts are acceptable. Opposing color domain images are segmented, thereby separating the luma and chrominance channels, thereby allowing separate processing.

The luma channel is then converted to an additive color domain (ACD), such as RGB, and the ACD luma image is sub-pixel sampled to preserve luma data while reducing resolution. After sub-pixel sampling, the sub-pixel sampled (SPS) image is transformed back to the opposing color domain (OCD), and split again into separate luma and chrominance channels. The SPS chrominance channels produced by this separation are then high-pass filtered to remove low-frequency artifacts produced during sub-pixel sampling. The SPS luminance channel is typically not modified so that the original luminance data is preserved.

The chroma channels from the original image are low-pass filtered and then sub-sampled to provide chroma data for the lower resolution image. These low pass filtered chroma channels are then combined with high pass filtered subpixel sampled chroma channels generated from the original luma channel. These combined chroma channels are also combined with SPS luma channels to form error-reduced, lower-resolution images, usually in opposing color gamuts. This error-reduced, low-resolution image can be converted to the added color gamut or to some other color gamut compatible with the desired application.

The present invention provides a method for converting a higher resolution image into a lower resolution image with reduced visible errors, said method comprising the acts of converting the higher resolution Opposite Color Domain (OCD) Segmenting an image into a separate original luma channel and one or more original chrominance channels; transforming the original luma channel into an additive color domain (ACD) luma image, performing subpixel sampling on the ACD luma image; converting the The ACD luminance image is converted into an OCD luminance image; the OCD luminance image is divided into independent luminance channels and one or more SPS chrominance channels that have carried out sub-pixel sampling (SPS); high-pass filtering the one or more SPS chrominance channels; and combining the one or more high-pass filtered SPS chrominance channels with the SPS luminance channel to form a lower resolution image with reduced errors.

The present invention also provides an apparatus for converting a higher resolution image into a lower resolution image with reduced visible errors, said apparatus comprising: an Opposite Color Domain (OCD) for converting the higher resolution means for segmenting an image into independent original luma channels and one or more original chrominance channels; for transforming said original luma channels into additive color domain (ACD) luma images and performing sub-processing on said ACD luma images means for pixel sampling; means for converting said ACD luminance image into an OCD luminance image; for dividing said OCD luminance image into independent sub-pixel sampled (SPS) luminance channels and one or more means for SPS chroma channels; a high pass filter for high pass filtering said one or more SPS chroma channels; and for combining said one or more high pass filtered SPS chroma channels with said SPS luma channels Means that combine to form lower resolution images with reduced errors.

The present invention further provides a method for converting a higher resolution image into a lower resolution image with reduced visible errors, said method comprising the act of converting the higher resolution Opposite Color Domain (OCD) Segmenting an image into a separate initial luma channel and a plurality of initial chrominance channels; transforming said initial luma channel into an additive color domain (ACD) luma image, performing subpixel sampling on said ACD luma image; converting said ACD The luminance image is converted into an OCD luminance image; the OCD luminance image is divided into independent luminance channels and SPS chrominance channels that have carried out sub-pixel sampling (SPS); the SPS chrominance channels are high-pass filtered; said initial chroma channel; subsampling said filtered initial chroma channel; combining said subsampled low pass filtered chroma channel with said high pass filtered SPS chroma channel; and The combined chrominance channel is combined with the SPS luma channel to form a lower resolution image with reduced errors.

The present invention further provides a method for displaying a higher resolution image at a lower resolution with reduced visible errors, said method comprising the act of converting the higher resolution RGB image into a higher resolution Opposite Color Domain (OCD) images; splitting the higher resolution OCD image into independent initial luminance channels and multiple initial chrominance channels; transforming the initial luminance channels into RGB luminance images; for the The RGB luminance image performs sub-pixel sampling; the (SPS) RGB luminance image that has been sub-pixel sampled is converted into an SPS-OCD luminance image; the SPS-OCD luminance image is divided into independent SPS luminance and SPS chromaticity channel; high pass filter the SPS chroma channel; low pass filter the initial chroma channel of the higher resolution OCD image; subsample the filtered initial chroma channel; combining a sampled low pass filtered chroma channel with said high pass filtered SPS chroma channel; combining said combined chroma channel with said SPS luma channel to form an error reduced OCD lower resolution image; and transforming said error-reduced OCD lower-resolution image into an error-reduced lower-resolution RGB image.

The present invention further provides a system for converting a higher resolution image into a lower resolution image with reduced visible errors, said system comprising: a first divider for dividing the higher resolution opposite The color domain (OCD) image of is divided into independent original luminance channel and a plurality of original chrominance channels; Subpixel sampler is used for transforming described original luminance channel into added color domain (ACD) luminance image, and to The ACD luminance image performs sub-pixel sampling; a converter is used to convert the ACD luminance image into an OCD luminance image; a second divider is used to divide the OCD luminance image into independent sub-pixel samples (SPS) luma channel and SPS chroma channel; high pass filter for high pass filtering said SPS chroma channel; low pass filter for low pass filtering said initial chroma channel; subsampler for subsampling the filtered initial chroma channel; a first combiner for combining the subsampled low pass filtered chroma channel with the high pass filtered SPS chroma channel; and a second a combiner for combining the combined chroma channels with the SPS luma channels to form a lower resolution image with reduced errors.

Description of drawings

In order to obtain the above and other advantages and objects of the invention, a more particular description of the invention outlined above will be rendered by reference to specific embodiments of the invention as shown in the accompanying drawings. After seeing that these drawings depict only typical embodiments of the invention and are not to be considered as limiting the scope of the invention, the invention will be described and illustrated with additional particularity and detail by using the accompanying drawings, in which:

FIG. 1 is a diagram showing conventional image sampling for a display having a tricolor pixel structure;

FIG. 2 is a diagram showing subpixel image sampling for a display having a tricolor pixel structure;

Figure 3 is a diagram showing an idealized CSF transformed to a digital frequency plane;

4 is a graph showing analysis of pixel Nyquist frequencies and sub-pixel Nyquist frequency regions representing regions of interest;

Figure 5 shows a typical preprocessing technique;

Figure 6(a) is a graph showing analysis using 1/f-power spectra repeated at pixel sampling and sub-pixel sampling frequencies;

Figure 6(b) is a graph showing analysis using 1/f-power spectrum repeated at pixel sampling and sub-pixel sampling frequencies with improvements due to preprocessing;

Figure 7 is a block diagram showing a known use of a vision model;

Figure 8 is a flowchart showing an embodiment of the present invention;

Figure 9 is a flow chart showing a specific embodiment of the present invention; and

Figure 10 is a diagram showing signals maintained by an embodiment of the present invention.

Detailed ways

The presently preferred embodiments of the present invention will be best understood by referring to the drawings, in which like parts are designated by like numerals throughout. The figures listed above are directly referenced as part of this detailed description.

It will be readily seen that the components of the invention may be arranged and designed in a wide variety of different configurations, as generally described and shown in the drawings herein. Accordingly, the following more detailed description of embodiments of the method and system of the present invention is not intended to limit the scope of the invention, but is merely representative of presently preferred embodiments of the invention.

Elements of embodiments of the invention may be implemented in hardware, firmware and or software. Also, it is possible to provide software (program) for implementing the characteristics of the embodiments of the present invention in such a manner that the software is stored in a computer-readable medium. Examples of such media include, in addition to recording media such as information storage devices (semiconductor memory, floppy disk, hard disk, etc.) and optical storage devices (CD-ROM, DVD, etc.) and wireless communication network) communication media (fiber optics, wireless communication lines, etc.) used in the system. Furthermore, it is possible to implement features of embodiments of the invention in computer signals embodied in electronic transmissions. Although the exemplary embodiments disclosed herein may describe only one of these forms, it should be appreciated that a person skilled in the art would be able to implement these elements in any of these forms and still remain within the scope of the invention.

Embodiments of the present invention may be described and claimed in terms of "monochrome". As used in connection with an image in this specification, this term refers to an image that has no visible color variation. A monochrome image can be an image that contains only one layer or one color channel, or an image that has multiple layers or multiple color channels, but each color layer is the same, resulting in a single color image.

Embodiments of the invention may be described and claimed in terms of "RGB" images or fields, or "added color fields" or "added color images." As used in this specification, these terms may refer to any form of multi-component image domain with integral luminance and chrominance information, including but not limited to various RGB domains and CMYK domains.

Embodiments of the invention may be described and claimed in terms of a "YCrCb" image or domain, an "opposite color" domain, image or channel, or a "color difference" domain, image or channel. These terms, as used in this description, may refer to any form having a multi-component image domain including at least one distinct luma channel and chrominance channel, including but not limited to, YCrCb, LAB, YUV and YIQ domains.

Embodiments of the present invention may be used to transform a higher resolution image into a lower resolution image with few visible errors in the transformed image. While these embodiments are typically used in conjunction with a display device to convert an image having a higher resolution than the display, down to a usable resolution of the display, other applications are applicable.

Images transformed by embodiments of the present invention may be in a variety of formats. When these formats are incompatible with the processing of embodiments of the present invention, the images can be converted to a compatible format prior to processing and, when necessary, converted back to the original format after processing.

Embodiments of the present invention may be explained with reference to Figure 8, which shows a diagram summarizing an exemplary embodiment. This process 70 starts with an image existing in an opposing color domain (OCD), such as YCrCb, LAB, YUV, YIQ or similar. When an image exists in an additive color gamut (ACD), such as the RGB or CMYK gamut or some other color space, the image may be transformed into the opposing color gamut before being processed with an embodiment of the present invention. Certain embodiments include the step of converting the image into a compatible format prior to processing.

Once the image is in the opposing color domain 72, with different luma and chrominance channels, the image is "segmented" (step 74) in preparation for processing the luma and chrominance channels separately. "Segmentation" (step 74 ) may include sampling or filtering of the original OCD image 72 , or other methods of separating the luma and chrominance data from the original image 72 . Segmentation may also include image transformations.

After segmentation, the original luminance channel 76 is transformed (step 78) into an ACD luminance image, such as an RGB image. This step, when done, enables the sampling of the luma image in the format or domain in which it will ultimately be displayed. Once the luminance image is transformed (step 78), subpixel sampling is performed on the image (step 80) in order to increase the resolution of the resulting lower resolution image. In this way, the brightness data of each successive pixel in the original higher resolution image is assigned to each corresponding sub-pixel in the lower resolution image.

When the subpixel sampling (step 80) is complete, a subpixel sampled (SPS) luminance image is generated, this SPS luminance image is transformed (step 82) into an OCD image, which may be referred to as an SPS-OCD luminance image. This transformation is performed to further segment (step 84) the SPS luma image into different luma and chrominance channels. The SPS luminance channel 86 is typically left undisturbed until later combined with other channels (step 88). However, SPS chrominance channels 90 and 92 are filtered before further combining.

These SPS chrominance channels 90 and 92 may be divided into a red-green channel 90 and a blue-yellow channel 92 . These channels typically include Cr and Cb channels for YCrCb images, "a" and "b" channels for LAB images, U and V channels for YUV images, I and Q channels for YIQ images, or similar for other color spaces or domains. channel. These chrominance channels 90 and 92 are high pass filtered (steps 94 and 96), removing low frequency artifacts that occur during subpixel sampling.

In some embodiments of the invention, high pass filtering (steps 94 and 96) may be performed by a blur masking method. Blur masks can use low-pass kernels. Typically, the original image is processed with a low-pass kernel, producing a low-pass version of the image. This low-pass version is then subtracted from the original unfiltered image while preserving the median of the image. A successful embodiment uses a Gaussian low pass kernel with a sum value of about 0.3 pixels to about 0.8 pixels. A sum value of 0.6 pixels was found to be particularly successful, and resulted in a cutoff in the frequency domain of about 0.168 cycles/pixel. This gives a good blur mask filter. The derivation for the Gaussian kernel is given below.

The one-dimensional Gaussian function used in some embodiments is given as:

$\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}\sqrt{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; \&pi;</span>}}}{\mathrm{<span\; class="notranslate">\; e</span>}}^{<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}\mathrm{<span\; class="notranslate">\; \&mu;</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

The Fourier transform of this function is given as:

$\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; e</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; \&pi;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

Here, we see that σ (in pixels) in the spatial domain corresponds to 1/π ² σ (in cycles/pixel) in the frequency domain. This relationship can be used to help determine the cutoff frequency of a filter given its σ, or conversely, the space σ of the blur mask for a given frequency, which can be guided by the CSF model.

The two-dimensional Gaussian function used in some embodiments is given as:

$\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; \&pi;</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}}{\mathrm{<span\; class="notranslate">\; e</span>}}^{<span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\frac{{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; +</span>\frac{{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&mu;</span>}}_{\mathrm{<span\; class="notranslate">\; Y</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Since the Gaussian function can be separated in rectangular coordinates, the frequency response of the two-dimensional Gaussian function is similar to formula (2). When considering the importance of σ, that is, σ _x in the time domain is 1/ π ² σ _x , and σ _y in the time domain is 1/π ² σ _y in the frequency domain.

A successful embodiment of the invention employs a Gaussian blur mask filter implemented with a kernel of size 3×3, whose sum (sigma) value is chosen to be 0.6, resulting in a low-pass filter of about 0.2 cycles/pixel Cut-off frequency.

Other embodiments of the invention may use high-pass filters equivalent to the inverse CSFs of the respective opposing color channels. These CSFs can be transformed from the domain of cy/deg (in which they are modeled) to the digital domain of cy/pix. The actual transformation process takes viewing distance into account and allows customization for different applications with specific display resolutions (in pixels/mm) and different expected or intended viewing distances. Due to the method of the present invention, color artifacts will not be visible when viewed no closer than the designed viewing distance. However, brightness resolution will be improved.

For chrominance channels 90 and 92 or for selected channels, such filtering (steps 94 and 96) may be performed according to the amount or strength of artifacts introduced during a particular sampling process, or according to some other criteria.

The low pass filtering of the original OCD chroma channels 98 and 100 (steps 102 and 104) may be performed concurrently with processing in the luma channel 105, or may be performed at some other time. Low-pass filtering is performed on the OCD chroma channels (steps 102 and 104) in order to remove dominant color frequencies greater than the display pixel Nyquist frequency. Thus, these channels can be subsampled by a factor of 1:3 in a conventional manner (steps 101 and 103 ) without color aliasing in the color channel 110 .

Once the filtering operation is complete, the separated channels can be combined. The combination of color channels will vary depending on the color gamut used. In the exemplary embodiment, a high-pass filtered, subpixel-sampled blue-yellow (HPFSPS-B/Y) color channel 97 will be compared to a low-pass filtered, conventionally subsampled blue-yellow (LPFSS-B/Y) color channel 97. /Y) color channels 109 are combined (step 106) to form a single high-low filtered, (HLF)B/Y color channel 111. The high-pass filtered, sub-pixel sampled red-green (HPFSPS-R/G) channel 95 is also combined with the low-pass filtered, conventionally sub-sampled red-green (LPFSS-R/G) channel 107 (step 108) to form a single high and low filtered, (HLF) R/G channel 114.

It should be noted that the methods of embodiments of the present invention may be used in other color spaces and domains that may include other color channels and other numbers of color channels and other variations of luma channels. The combined HLF chroma channels 111 and 114 may also be combined with the SPS luma channel 86 (step 88 ) to form a lower resolution OCD image 116 . The lower resolution OCD image 116 can then be converted to other image formats or domains as required for various purposes.

The methods and systems of these embodiments provide lower resolution images with few visible color artifacts.

In addition, the above embodiments can be modified in various ways. For example, in some cases the low pass filtering of chroma channels 98 and 100 (steps 102 and 104) may be omitted. Also, it is possible to form a lower resolution image 116 by using only the luma channel 76 that has been segmented. In other words, it is possible to perform each step of the luma channel 105 with respect to the luma channel 76, while omitting each step of the color channel 110 and steps 106 and 108 with respect to the chroma channels 98 and 100, in order to combine the SPS luma channel 86 And HPFSPS-R/G channel 95 and HPFSPS-B/Y channel 97. In this way, it is possible to form a lower resolution image 116 .

By referring to FIG. 9, a specific exemplary embodiment of the present invention can be explained. This particular embodiment can be used to process higher resolution RGB images for display on lower resolution display devices. The higher resolution RGB image 120 can optionally be pre-processed (step 122) according to the specific needs of the user or application. Pre-processing (step 122) may include hinting, a type of low-pass filtering, or some other processing technique. Preprocessing (step 122) can also be bypassed altogether.

After any pre-processing (step 122), RGB may be transformed (step 124) into an opposing color domain image, such as LAB, YCrCb, YIQ, YUV or other image domains. In this example, the LAB image domain is used. Once transformed to this domain, the image can be segmented (step 126) into the individual L, a, and b channels of the domain in order to process these channels separately. In this way, the chrominance and luma channels can be processed separately.

The "L" channel 127 is then converted back to the RGB domain (step 128) so that it can be sampled in its final display format. This transformation may involve simply duplicating the L layer or channel into three identical R, G and B layers. A single layer can also be used, however, the actual transform method will depend on the color transform chosen.

Then, sub-pixel sampling is performed on this RGB luminance image (step 130 ) to preserve the horizontal luminance resolution of the original RGB image 120 . After sub-pixel sampling, the sampled image is again transformed (step 132) into an opposing color domain, such as LAB. This sampled LAB image is segmented (step 134) to separate the luma and chrominance channels for further processing of the chrominance channels. Here, luma channel 136 is typically not processed so that the original luma data is preserved. However, the chrominance channels 150 and 152 of the subpixel sampled and segmented image are high pass filtered (steps 146 and 148) to remove low frequency color aliasing that occurs during subpixel sampling.

As in other embodiments described above, this high pass filtering may be performed with a blur mask filter using a Gaussian low pass kernel. In an embodiment using this method, the chroma channels are filtered to produce a low-pass filtered chroma image, and this low-pass filtered chroma image is subtracted from the SPS-RGB chroma image to produce a "high-pass filtered "filtered (HPF) SPS chrominance image or channels 147 and 149. High pass filtering (steps 146 and 148) is typically performed on the "a" and "b" channels, but may be performed on only one channel when conditions permit.

The low pass filtering (steps 138 and 140) of the original "a" and "b" chrominance channels 154 and 156 may be performed concurrently with the processing of the "L" channel, or may be performed at some other time. Low pass filtering (steps 138 and 140) is performed on the "a" and "b" chrominance channels to remove dominant color frequencies greater than the display pixel Nyquist frequency. After low-pass filtering (steps 138 and 140), these channels can be subsampled by a factor of 1:3 (steps 142 and 144) in a conventional manner without color aliasing.

When the channels have been filtered and sampled, they are combined to form a lower resolution image with few errors. High pass filtered luminance “a” channel 147 will be combined with subsampled, low pass filtered “a” channel 143 (step 160 ) to form processed “a” channel 164 . High pass filtered luminance “b” channel 149 will be combined with subsampled, low pass filtered “b” channel 145 (step 158 ) to form processed “b” channel 162 . These chrominance channels 162 and 164 are then combined with the SPS luma channel 136 (step 166 ) to form a lower resolution LAB image 168 with reduced errors.

This error-reduced image can be transformed (step 170) to the RGB domain to produce an error-reduced, lower resolution RGB image 172, which can be output to a display or other device.

The functionality of the processing of an embodiment of the present invention can be described with reference to FIG. 10, which shows the signal held with respect to the luma CSF 180 and the color CSF 182. The color signal that is preserved includes a high-pass region 184 that is undetectable to the color CSF, and a low-pass region 186 that contains the useful color content of the image. Ideally, the frequencies lost from this low-pass color 186 are invisible to the observer because of the limited bandwidth of RG and BY CSF. The HPF color signal 184 is a color aliasing that carries effective luminance information. It should be noted that this technique will work on color images since low frequency color information is preserved. Figure 10 shows that there is no overlap between the two color signals, but depending on the actual filters used, overlap is possible. Other embodiments may include the use of filters that allow overlapping of high-pass 184 and low-pass 186 color signals as shown in FIG. 10 . Overlap can allow greater color bandwidth at the expense of color aliasing.

The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. Therefore, the scope of the present invention is indicated by the appended claims rather than the above description. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.

Claims (14)

CLAIMS 1. A method for transforming a higher resolution image into a lower resolution image with reduced visible errors, said method comprising the acts of: Segmentation of a higher resolution Opposite Color Domain (OCD) image into separate original luma channels and one or more original chrominance channels; transforming the original luminance channel into an additive color domain (ACD) luminance image, performing subpixel sampling on the ACD luminance image; converting the ACD luminance image into an OCD luminance image; The OCD luminance image is divided into independent luminance channels and one or more SPS chrominance channels that have been sub-pixel sampled (SPS); high pass filtering the one or more SPS chrominance channels; and The one or more high pass filtered SPS chrominance channels are combined with the SPS luma channel to form a lower resolution image with reduced errors. 2. A method for transforming a higher resolution image into a lower resolution image with reduced visible errors, said method comprising the acts of: Segmentation of a higher resolution Opposite Color Domain (OCD) image into a separate initial luma channel and multiple initial chrominance channels; transforming the original luminance channel into an additive color domain (ACD) luminance image, performing subpixel sampling on the ACD luminance image; converting the ACD luminance image into an OCD luminance image; The OCD luminance image is divided into independent sub-pixel sampling (SPS) luminance channels and SPS chrominance channels; high pass filtering the SPS chrominance channel; low pass filtering the initial chroma channel; subsampling said filtered initial chroma channel; combining the subsampled low pass filtered chroma channel with the high pass filtered SPS chroma channel; and The combined chrominance channel is combined with the SPS luma channel to form a lower resolution image with reduced errors. 3. The method of claim 2, comprising the process of transforming said additional color gamut image when said higher resolution image replaces an opposing color gamut image and becomes an additive color gamut image An image of an opposing color gamut. 4. The method of claim 2, wherein said added color gamut image is an RGB image. 5. The method of claim 2, wherein said opposing color gamut image is a YCrCb image. 6. The method of claim 2, wherein said opposing color gamut image is a LAB image. 7. The method of claim 2, wherein said high pass filtering comprises blur mask filtering. 8. The method of claim 2, wherein said high pass filtering comprises the act of: filtering the SPS chroma channel through a blur mask filter with a Gaussian low-pass kernel resulting in a low-pass SPS chroma channel; and The SPS low pass chroma channel is subtracted from the SPS chroma channel to produce a high pass filtered SPS chroma channel. 9. The method of claim 2, wherein said chrominance channels include a red-green channel and a blue-yellow channel. 10. The method of claim 2, wherein said chrominance channels comprise Cr and Cb channels of a YCrCb image. 11. The method of claim 2, wherein said chrominance channels comprise "a" and "b" channels of a CIELab image. 12. The method of claim 2, further comprising the act of converting said error-reduced lower resolution image into an RGB image. 13. The method of claim 2, wherein said act of performing subpixel sampling comprises converting said original luminance channel into an additive color domain (ACD) luminance image, and sampling said ACD luminance image. 14. A method for displaying a higher resolution image at a lower resolution with reduced visible errors, the method comprising the acts of: Transform a higher resolution RGB image into a higher resolution Opposite Color Domain (OCD) image; segmenting the higher resolution OCD image into independent initial luma channels and a plurality of initial chrominance channels; transforming the initial luminance channel into an RGB luminance image; performing sub-pixel sampling on the RGB brightness image; Transform the (SPS)RGB luminance image that has been subjected to sub-pixel sampling into an SPS-OCD luminance image; Segmenting the SPS-OCD luminance image into independent SPS luminance and SPS chrominance channels; high pass filtering the SPS chrominance channel; low pass filtering the initial chrominance channel of the higher resolution OCD image; subsampling said filtered initial chroma channel; combining the subsampled low pass filtered chroma channel with the high pass filtered SPS chroma channel; combining the combined chrominance channel with the SPS luma channel to form an error-reduced OCD lower resolution image; and The error-reduced OCD lower resolution image is transformed into an error-reduced lower resolution RGB image.

Images