HK1231195B - Device and method of improving the perceptual luminance nonlinearity-based image data exchange across different display capabilities - Google Patents

Description

Device and method for improving image data exchange between displays of different capabilities based on perceived illumination nonlinearity

This application is a divisional application of the invention patent application with application number 201280060408.7, application date December 6, 2012, and invention name "Apparatus and method for improving image data exchange based on perceived illumination nonlinearity between different display capabilities".

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent Application No. 61/567,579, filed December 6, 2011; U.S. Provisional Patent Application No. 61/674,503, filed July 23, 2012; and U.S. Provisional Patent Application No. 61/703,449, filed September 20, 2012, the entire contents of which are incorporated herein by reference for all purposes.

Technical Field

The present invention relates generally to image data. More particularly, embodiments of the present invention relate to the exchange of image data between different display capabilities based on perceptual nonlinearity.

Background Art

Technological advances have enabled modern display designs to present image and video content with significantly improved quality characteristics compared to the same content presented on less advanced displays. For example, some more modern displays are capable of presenting content with a higher dynamic range (DR) than the standard dynamic range (SDR) of conventional or standard displays.

For example, some modern liquid crystal displays (LCDs) have light units (backlight units, sidelight units, etc.) that provide a light field in which individual components can be modulated separately from the modulation of the liquid crystal orientation state of the active LCD elements. This dual modulation approach is scalable (e.g., to N modulation layers, where N includes integers greater than 2), such as by controllable intermediate layers in the display's electro-optical construction (e.g., multiple layers of separately controllable LCD layers).

In contrast, some existing displays have a dynamic range (DR) significantly narrower than high dynamic range (HDR). Mobile devices, computer tablets, gaming devices, televisions (TVs), and computer monitors using typical cathode ray tubes (CRTs), liquid crystal displays (LCDs) with constant fluorescent white backlighting, or plasma screen technology can have DR rendering capabilities limited to approximately three orders of magnitude. Such existing displays therefore represent standard dynamic range (SDR), sometimes also referred to as "low" dynamic range or "LDR" relative to HDR.

Images captured by HDR cameras can have scene-referred HDR, a dynamic range far greater than that of most, if not all, display devices. Scene-referred HDR images can include large amounts of data and must be converted to post-production formats that facilitate transmission and storage (e.g., HDMI video signals with 8-bit RGB, YCbCr, or Deep Color options; 1.5 Gbps SDI video signals with 10-bit 4:2:2 sampling rates; 3 Gbps SDI with 12-bit 4:4:4 or 10-bit 4:2:2 sampling rates; and other video or image formats). Post-production images can include a dynamic range significantly smaller than that of scene-referred HDR images. Furthermore, when the images are delivered to an end-user's display device for presentation, device-specific and/or manufacturer-specific image transformations occur along the way, causing numerous visually noticeable errors in the rendered image compared to the original scene-referred HDR image.

The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, no approach described in this section should be considered prior art solely by virtue of its inclusion in this section. Similarly, unless otherwise indicated, any issues identified with respect to one or more approaches should not be considered as having been identified in any prior art based on this section.

GB2311432A discloses a method for converting an input frame of image data into multiple output frames having different spatial frequency spectra. After applying a separate gain to each output frame according to a model of the human visual system, the output frames are combined, thereby sharpening or softening the input image.

US 2003/0174284 A1 discloses a contrast sensitivity test performed as two alternating forced-choice tests using grating stimuli of randomly interleaved spatial frequencies.

WO2007/014681A discloses a method for adjusting a display system so that it complies with mandatory standards for a selected parameter range. An image encoded according to a standardized grayscale standard display function is converted into display-specific digital drive levels. The digital drive levels are converted into luminance output according to the display system's intrinsic transfer function curve based on human contrast sensitivity.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example, and not by way of limitation, in the accompanying figures and in which like references indicate similar elements and in which:

FIG1 illustrates an example family of contrast sensitivity function curves across multiple light adaptation levels according to an example embodiment of the present invention;

FIG2 illustrates an example integration path according to an example embodiment of the present invention;

FIG3 illustrates an example grayscale display function according to an example embodiment of the present invention;

FIG4 illustrates a graph depicting Weber scores according to an example embodiment of the present invention;

FIG5 illustrates an example framework for exchanging image data with devices of different GSDFs according to an example embodiment of the present invention;

FIG6 illustrates an example conversion unit according to an example embodiment of the present invention;

FIG7 illustrates an example SDR display according to an example embodiment of the present invention;

8A and 8B illustrate example process flows according to an example embodiment of the present invention;

FIG9 illustrates an example hardware platform upon which a computer or computing device as described herein may be implemented according to an example embodiment of the present invention;

FIG. 10A illustrates a maximum amount of code errors in a unit of JND in a plurality of code spaces, each of the plurality of code spaces having a different one of one or more different bit depths according to some example embodiments;

Figures 10B to 10E illustrate distribution of code errors according to some example embodiments; and

FIG11 illustrates values of parameters in a function model according to an example embodiment.

DETAILED DESCRIPTION

This document describes example embodiments related to the exchange of image data between displays of varying capabilities based on perceived luminance nonlinearity. In the following description, for purposes of explanation and to provide a thorough understanding of the present invention, numerous specific details are set forth. However, it should be understood that the present invention can be practiced without these specific details. In other instances, well-known structures and devices are not described in exhaustive detail to avoid unnecessarily obscuring, obscuring, or obfuscating the present invention.

Example embodiments are described herein according to the following outline:

1. General Overview

2. Contrast Sensitivity Function (CSF) Model

3. Perceptual nonlinearity

4. Digital code value and gray level

5. Model Parameters

6. Variable spatial frequency

7. Function Model

8. Exchange image data based on reference GSDF

9. Convert reference coded image data

10. Example Processing Flow

11. Implementation mechanism - Hardware overview

12. Enumerate example embodiments, equivalents, extensions, alternatives and others

1. General Overview

This summary provides a basic description of some aspects of the exemplary embodiments of the present invention. It should be noted that this summary is not an exhaustive or comprehensive overview of all aspects of the exemplary embodiments. Furthermore, it should be noted that this summary should not be construed as identifying any particularly important aspects or elements of the exemplary embodiments, nor as delimiting any scope of the exemplary embodiments in particular, nor as delimiting the present invention generally. This summary merely presents some concepts related to the exemplary embodiments in a concise and simplified format and should be understood as merely a conceptual prelude to the more detailed description of the exemplary embodiments presented below.

If two illuminance values are not significantly different from each other, human vision may not perceive the difference between them. Conversely, if the illuminance values differ by at least a just noticeable difference (JND), human vision only perceives the difference. Due to the nonlinearity of human vision, the magnitude of each JND is not uniform across a range of light levels, but rather varies with each light level. Furthermore, due to perceptual nonlinearity, the magnitude of each JND is not uniform across a range of spatial frequencies at a specific light level, but rather varies with different spatial frequencies below the cutoff spatial frequency.

Image data encoded using luminance quantization steps of equal or linearly scaled size does not match the perceptual nonlinearity of human vision. Image data encoded using luminance quantization steps at a fixed spatial frequency also does not match the perceptual nonlinearity of human vision. According to these techniques, when assigning codewords to represent quantized luminance values, too many codewords may be distributed in a particular region of the light level range (e.g., bright areas), while too few codewords may be distributed in different regions of the light level range (e.g., dark areas).

In overly dense areas, a large number of codewords may not generate a perceptual difference and are therefore effectively wasted. In sparse areas, two adjacent codewords may generate a perceptual difference much larger than the JND and may generate contour distortion (also known as banding) visual artifacts.

According to the techniques described herein, a contrast sensitivity function (CSF) can be used to determine the JND over a wide range of light levels (e.g., 0 to 12,000 cd/m ² ). In an example embodiment, a peak JND as a function of spatial frequency at a particular light level is selected to represent the amount of human perception at that particular light level. The selection of the peak JND is based on the behavior of human vision, which adapts to an increased level of visual perceptibility when backgrounds with similar, but different illuminance values are viewed, which is sometimes referred to as the crispening effect and/or Whittle's crispening effect in video and image display fields of view, and may be described as such herein. As used herein, the term "light adaptation level" may be used to refer to a light level at which a (e.g., peak) JND is selected/determined assuming that human vision is adapted to that light level. The peak JND as described herein varies with spatial frequency at different light adaptation levels.

As used herein, the term "spatial frequency" may refer to the rate of spatial modulation/variation in an image (wherein the rate is calculated relative to or over spatial distance, as opposed to relative to time). In contrast to conventional approaches that may fix the spatial frequency to a specific value, the spatial frequency as described herein may vary, for example, within or over a range. In some embodiments, the peak JND may be limited to a specific spatial frequency range (e.g., 0.1 to 5.0, 0.01 to 8.0 cycles/degree, or a smaller or larger range).

A reference grayscale display function (GSDF) can be generated based on a CSF model. In some embodiments, a very wide field of view is assumed for the CSF model to generate a reference GSDF that better supports entertainment display fields. A GSDF refers to a set of reference digital code values (or reference codewords), a set of reference grayscale levels (or reference illuminance values), and a mapping between these two sets. In an example embodiment, each reference digital code value corresponds to a human perceptual quantity represented by a JND (e.g., a peak JND at a certain light adaptation level). In an example embodiment, the same number of reference digital code values can correspond to a human perceptual quantity.

The GSDF can be obtained by accumulating JNDs starting from an initial value. In an exemplary embodiment, an intermediate codeword value (e.g., 2048 for a 12-bit code space) is assigned as an initial value to the reference digital code. The initial value of the reference digital code can correspond to an initial reference grayscale level (e.g., 100 cd/m ² ). As the reference digital codes increase one by one, other reference grayscale levels for other values of the reference digital code can be obtained by accumulating positive JNDs (addition), and as the reference digital codes decrease one by one, other reference grayscale levels for other values of the reference digital code can be obtained by accumulating negative JNDs (subtraction). In an exemplary embodiment, when calculating the reference values in the GSDF, a quantity such as a contrast threshold can be used instead of the JND. These quantities actually used in calculating the GSDF can be defined as unitless ratios and can differ from the corresponding JND only by a known or determinable multiplier, division factor, and/or bias.

The code space may be selected to include all reference digital code values in the GSDF. In some embodiments, the code space in which all reference digital code values reside may be one of the following code spaces: a 10-bit code space, an 11-bit code space, a 12-bit code space, a 13-bit code space, a 14-bit code space, a 15-bit code space, or a larger or smaller code space.

Although a large code space (>15 bits) can be used to host all reference digital code values, in certain embodiments, the most efficient code space (e.g., 10 bits, 12 bits, etc.) is used to host all reference digital code values generated in the reference GSDF.

The reference GSDF can be used to encode image data captured or generated, for example, by an HDR camera, studio system, or other system with scene-dependent HDR that has a dynamic range far greater than that of most, if not all, display devices. The encoded image data can be provided to downstream devices using a variety of distribution or transmission methods (e.g., HDMI video signals with 8-bit RGB, YCbCr, or Deep Color options; 1.5 Gps SDI video signals with a 10-bit 4:2:2 sampling rate; 3 Gps SDI with a 12-bit 4:4:4 or 10-bit 4:2:2 sampling rate; and other video or image formats).

In some embodiments, because adjacent reference digital code values in the reference GSDF correspond to grayscale levels within the JND, details distinguishable by human vision can be fully or substantially preserved in image data encoded based on the reference GSDF. A display that fully supports the reference GSDF can present images without banding or contouring artifacts.

Image data encoded based on a reference GSDF (or reference coded image data) can be used to support a variety of lower-capability displays that may not fully support all reference luminance values in the reference GSDF. Because the reference coded image data includes all perceptual detail within the supported luminance range (which can be designed to be a superset of the range supported by the display), the reference digital code values can be optimally and efficiently transcoded into display-specific digital code values in a manner that preserves as much detail as possible that can be supported by the specific display and introduces as few visually noticeable errors as possible. Additionally and/or alternatively, decontouring and dithering can be performed in conjunction with or as part of the transcoding from reference digital code values to display-specific digital code values to further improve image or video quality.

The techniques described herein are color space agnostic. They can be used in the RGB color space, the YCbCr color space, or different color spaces. Furthermore, the techniques for deriving reference values (e.g., reference digital code values and reference grayscale levels) using spatial frequency-dependent JNDs can be applied to different channels (e.g., one of the red, green, and blue channels) in addition to the luminance channel in different color spaces (e.g., RGB), which may or may not include a luminance channel. For example, instead of a reference grayscale, a JND applicable to the blue color channel can be used to derive a reference blue value. Thus, in some embodiments, grayscale can be used instead of color. Additionally and/or alternatively, a different CSF model can be used instead of Barten's model. Thus, different model parameters can be used for the same CSF model.

In some embodiments, mechanisms as described herein form part of a media processing system including, but not limited to, a handheld device, a game console, a television, a laptop computer, a netbook computer, a cellular wireless telephone, an electronic book reader, a point-of-sale terminal, a desktop computer, a computer workstation, a computer kiosk, or various other types of terminals and media processing units.

Various modifications to the preferred embodiments and overall principles and features described herein will be apparent to those skilled in the art. Therefore, the present disclosure is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features described herein.

2. Contrast Sensitivity Function (CSF) Model

The sensitivity of human vision to spatial structure in a presented image can be best described by the contrast sensitivity function (CSF), which describes contrast sensitivity as a function of spatial frequency (or the rate of spatial modulation/change in an image perceived by a human observer). As used herein, contrast sensitivity S can be thought of as the gain in human visual neural signal processing, while the contrast threshold _CT can be determined from the inverse of contrast sensitivity, for example:

Contrast sensitivity = S = 1/ _CT expression (1)

As used herein, the term "contrast threshold" may refer to, or be related to, the minimum value of (relative) contrast necessary for the human eye to perceive a difference in contrast (e.g., a just noticeable difference); in some embodiments, the contrast threshold may also be described as a function of the just noticeable difference over a range of illuminance values divided by the light adaptation level.

In some embodiments, the contrast threshold can be measured directly in an experiment without using any CSF model. However, in some other embodiments, the contrast threshold can be determined based on a CSF model. A CSF model can be constructed using several model parameters, and the CSF model can be used to derive a GSDF whose quantization step size in grayscale depends on and varies with the light level characterized by illuminance value and spatial frequency. Example embodiments can be implemented based on one or more of various CSF models, such as those described in Peter GJ Barten's Contrast Sensitivity of the Human Eye and its Effects on Image Quality (1999) (hereinafter, Barten's model or Barten's CSF model), or Scott Daly's Digital Images and Human Vision , Chapter 17, by A.B. Watson, MIT Press (1933) (hereinafter, Daly's model). With respect to example embodiments of the present invention, the contrast threshold used to generate a reference grayscale display function (GSDF) can be derived experimentally, theoretically, using a CSF model, or a combination thereof.

As used herein, GSDF may refer to a mapping of multiple digital code values (e.g., 1, 2, 3, ..., N) to multiple gray levels ( _L1 , _L2 , _L3 , ..., _LN ), where the digital code values represent index values of contrast thresholds and the gray levels correspond to the contrast thresholds as shown in Table 1.

Table 1

Numeric code value Grayscale (illuminance value) 1 2 3 … … i-1 i i+1 … … N

In an embodiment, a gray level (eg, Li) and an adjacent gray level (eg, Li _{+ 1) corresponding to a digital code value (eg, i) may be calculated in relation to a contrast ratio (eg, C(i)} ) as follows:

Where C(i) represents the contrast ratio limited to the illumination range between _Li and Li ₊₁ . _Lmean (i,i+1) comprises the arithmetic mean or average of two adjacent gray levels _Li and Li ₊₁ . The contrast ratio C(i) is mathematically related to the Weber fraction ΔL/L by a factor of 2. Here, ΔL represents ( _Li+1 - _Li ), and L represents _Li , Li ₊₁ , or one of the intermediate values between _Li and Li ₊₁ .

In some embodiments, the GSDF generator may set the contrast C(i) to a value equal to or proportional to a contrast threshold (e.g., _CT (i)) at illumination levels L _{between Li} and Li _{+ 1} , inclusive, as follows:

C(i)＝kC _T (i) Expression (3)

Where k represents a multiplication constant. With respect to embodiments of the present invention, other descriptive statistics/definitions (e.g., geometric mean, median, mode, variance, or standard deviation) and/or scaling (x2, x3, division or multiplication by a scaling factor, etc.) and/or offset (+1, +2, -1, -2, subtraction or addition of an offset, etc.) and/or weighting (e.g., assigning the same or different weight factors to two adjacent gray levels) may be used to relate the contrast threshold to the contrast for the purpose of calculating the gray levels in the GSDF.

As calculated in expressions (1), (2), and (3), the contrast or contrast threshold may include relative values and therefore may include unitless quantities (e.g., so S may also be unitless).

A CSF model can be constructed from a basic contrast threshold calculation or basic contrast threshold measurement based on the CSF that depicts the CSF model. Unfortunately, human vision is complex, adaptive, and nonlinear, so there is no single CSF curve that describes human vision. Instead, a family of CSF curves can be generated based on the CSF model. Even for the same CSF model, different values of the model parameters produce different plots for the CSF curve family.

3. Perceptual nonlinearity

FIG1 illustrates a family of exemplary CSF curves spanning multiple light adaptation levels. For illustrative purposes only, the highest CSF curve depicted in FIG1 is for a light adaptation level with an illuminance value of 1000 candelas per square meter (cd/m ² or "nits"), with the other highly decreasing curves being for light adaptation levels with illuminance values successively decreasing by a factor of 10. A notable feature that can be read from the CSF curves is that as illuminance increases (light adaptation level increases), the overall contrast sensitivity, including the maximum (or peak) contrast sensitivity, increases. The peak spatial frequency at which the contrast sensitivity on the CSF curves in FIG1 peaks is shifted to a higher spatial frequency. Similarly, the maximum perceptible spatial frequency (cutoff frequency) on the CSF curves, which is the intercept of the CSF curve by the horizontal (spatial frequency) axis, also increases.

In an example embodiment, Barten's CSF model, which takes into account several key effects related to human perception, can be used to derive a CSF function that produces a family of CSF curves as shown in FIG1. An example CSF S(u) (or the inverse of the corresponding contrast threshold mi ₎ according to Barten's CSF model can be calculated as shown in Expression (4) below.

Example model parameters used in expression (4) above include the following listed representations:

○2 (numerical factor) corresponds to binocular vision (4 if monocular vision);

○k represents the signal-to-noise ratio, for example, 3.0;

○T represents the integration time of the eye, for example, 0.1 seconds;

○X ₀ represents the angular size of the object (e.g., a square shape);

○ _Xmax represents the maximum angular size of the integration area of the eye (e.g., 12 degrees)

N _max represents the maximum number of cycles accumulated by probability summation, for example, 15 cycles

○η represents the quantum efficiency of the eye, for example, .03;

○p represents the photon conversion factor;

○E represents the retinal illuminance in, for example, troland;

○Φ ₀ represents the spectral density of neural noise, for example, 3× ^10-8 seconds* ^degrees2 ; and

○μ ₀ represents the maximum spatial frequency for lateral suppression, for example, 7 cycles/degree.

The optical modulation transfer function M _opt can be given as:

where σ represents a model parameter related to the pupil and/or light level.

Barten's CSF model, as discussed above, can be used to describe illuminance-related perceptual nonlinearity. Other CSF models can also be used to describe perceptual nonlinearity. For example, Barten's CSF model does not account for the effect of accommodation, which reduces the cutoff spatial frequency in the high-spatial-frequency region of the CSF. This reduction due to accommodation can be expressed as a function of decreasing viewing distance.

For example, for viewing distances above 1.5 meters, the maximum cutoff spatial frequency as described by Barten's CSF model can be achieved without affecting the effectiveness of Barten's model as a model suitable for describing perceptual nonlinearities. However, for distances less than 1.5 meters, eye accommodation effects begin to become significant, reducing the accuracy of Barten's model.

Therefore, Barten's CSF model may not be optimally adjusted for flat-panel displays that have closer viewing distances, such as 0.5 meters, and smartphones that may have viewing distances as close as 0.125 meters.

In some embodiments, Daly's CSF model that takes into account the effects of eye accommodation can be used. In certain embodiments, Daly's CSF model can be constructed based in part on Barten's CSF S(u) in expression (4) above, for example, by modifying the optical modulation transfer function M _opt in expression (5).

4. Digital code value and grayscale

The GSDF, as shown in Table 1, maps perceptual nonlinearity by using digital code values to represent gray levels associated with contrast thresholds in human vision. The gray levels comprising all mapped luminance values can be distributed so that they are optimally spaced to match the perceptual nonlinearity of human vision.

In some embodiments, when the maximum number of gray levels in the GSDF is sufficiently large relative to the maximum range of luminance values, the digital code values in the GSDF can be used in a manner that achieves a minimum number of gray levels (e.g., less than a total of 4096 digital code values) without causing gray level step transitions to be visible (e.g., visible as false contours or banding in the image; or color shifts in dark areas of the image).

In some other embodiments, a limited number of digital code values can still be used to represent a wide dynamic range of grayscale levels. For example, when the maximum number of grayscale levels in a GSDF is not large enough relative to the maximum range of grayscale levels (e.g., an 8-bit representation of digital code values for a grayscale range from 0 to 12,000 nits), the GSDF can still be used in a manner that achieves a minimum number of grayscale levels (e.g., less than a total of 256 digital code values) to reduce or minimize the visibility of grayscale step transitions. With such a GSDF, the number/degree of perceptible errors/artifacts from step transitions can be evenly distributed throughout the hierarchy of the relatively few grayscale levels in the GSDF. As used herein, the terms "grayscale" or "gray level" are used interchangeably and can refer to the represented luminance value (the quantized luminance value represented in the GSDF).

Grayscale in the GSDF can be derived by stacking or integrating contrast thresholds across light adaptation levels (at different illuminance values). In some embodiments, the quantization step size between grayscale levels can be selected so that the quantization step size between any two adjacent grayscale levels falls within a JND. The contrast threshold at a particular light adaptation level (or illuminance value) can be no greater than the just noticeable difference (JND) at that particular adaptation level. Grayscale can be derived by integrating or stacking fractions of the contrast threshold (or JND). In some embodiments, the number of digital code values is greater than the number sufficient to represent all JNDs within the represented illuminance dynamic range.

A contrast threshold for calculating grayscale levels or contrast sensitivity as a reciprocal can be selected from the CSF curve at different spatial frequencies other than a fixed spatial frequency for a particular light adaptation level (or illuminance value). In some embodiments, each contrast threshold is selected from the CSF curve at a spatial frequency corresponding to a peak contrast sensitivity (e.g., due to Whittle's edge effect) for a light adaptation level. Furthermore, contrast thresholds can be selected from the CSF curve at different spatial frequencies for different light adaptation levels.

An example expression for computing/stacking gray levels in a GSDF is as follows:

JND＝1/S(f， _LA ) Expression (6)

Wherein, f represents the spatial frequency, which may be different from a fixed number according to the techniques described herein; _LA represents the light adaptation level. _Lmin may be the lowest illuminance value among all mapped grayscale levels. As used herein, the term "nit" or its abbreviation "nt" may synonymously or interchangeably relate to or refer to a unit of image intensity, brightness, luminance and/or illuminance that is equivalent to or equal to one (1) candela per square meter (1 nit = 1 nt = 1 cd/m ² ). In some embodiments, _Lmin may include a zero value. In some other embodiments, _Lmin may include a non-zero value (e.g., a certain dark level that may be lower than the level typically achievable by a display device, 10 ^-5 nits, 10 ^-7 nits, etc.). In some embodiments, _Lmin may be replaced by a value other than the minimum initial value that allows stacking calculations to be performed using subtraction or negative addition (such as an intermediate value or a maximum value).

In some embodiments, for example, as shown in Expression (6), stacking of JNDs is performed by summing to derive the grayscale levels in the GSDF. In some other embodiments, integration can be used instead of discrete summation. The integration can be performed along an integration path determined from the CSF (e.g., Expression (4)). For example, the integration path can include peak contrast sensitivities for all light adaptation levels within the (reference) dynamic range of the CSF (e.g., different peak sensitivities corresponding to different spatial frequencies).

As used herein, an integration path may refer to a visible dynamic range (VDR) curve that represents the nonlinearity of human perception and establishes a mapping between a set of digital code values and a set of reference grayscale levels (quantized luminance values). This mapping may be required to meet the following criteria: each quantization step (e.g., the luminance difference between two adjacent grayscale levels in Table 1) is less than the JND above or below the corresponding light adaptation level (luminance value). The instantaneous derivative (in nits/spatial-period) of the integration path at a particular light adaptation level (luminance value) is proportional to the JND for that particular adaptation level. As used herein, the term "VDR" or "visual dynamic range" may refer to a dynamic range wider than the standard dynamic range and may include, but is not limited to, a wide dynamic range up to the instantaneous perceptible dynamic range and color gamut that can be perceived by human vision within a blink of an eye.

Based on the techniques described herein, a reference GSDF can be developed that is independent of any particular display or image processing device. In some embodiments, one or more model parameters other than light adaptation level (illuminance), spatial frequency, and angular size can be set to constant (or fixed) values.

5. Model Parameters

In some embodiments, the CSF model is constructed using conservative model parameter values that cover a wide range of display devices. The use of conservative model parameter values provides a smaller JND than existing standard GSDFs. Therefore, in some embodiments, the reference GSDF according to the techniques described herein can support high-precision illuminance values that exceed the requirements of these display devices.

In some embodiments, the model parameters described herein include a field of view (FOV) parameter. The FOV parameter can be set to a value of 45 degrees, 40 degrees, 35 degrees, 30 degrees, 25 degrees, or another larger or smaller value to support a wide range of display devices and viewing scenarios, including those used in studios, theaters, or high-end entertainment systems.

Model parameters as described herein can include an angular size parameter that can be related to, for example, the field of view. The angular size parameter can be set to a value of 45 degrees × 45 degrees, 40 degrees × 40 degrees, 35 degrees × 35 degrees, 30 degrees × 30 degrees, 25 degrees × 25 degrees, or another larger or smaller value to support a wide range of display devices and viewing scenarios. In some embodiments, a portion of the angular size parameter used to derive the reference GSDF can be set to n degrees × m degrees, where n and m can both be values between 30 and 40, and n and m can be equal or unequal.

In some embodiments, a larger angular size (e.g., 40 degrees x 40 degrees) is used to generate a reference GSDF with more grayscale levels and, therefore, greater contrast sensitivity. The GSDF can be used to support a wide range of viewing and/or display scenarios (e.g., large-screen video displays) that may require a wide viewing angle of ~30 to 40 degrees. A GSDF, whose sensitivity is enhanced by selecting a larger angular size, can also be used to support highly variable viewing and/or display scenarios (e.g., movie theaters). Even larger angular sizes can be selected; however, increasing the angular size well above a certain angular size (e.g., 40 degrees) can yield relatively limited margin benefits.

In some embodiments, the reference GSDF model covers a wide range of illuminances. For example, the grayscale or quantized illuminance values represented by the reference GSDF model range from 0 or approximately 0 (e.g., ^10-7 cd/ ^m2 ) to 12,000 cd/ ^m2 . The lower bound of the illuminance values represented in the reference GSDF model can be ^10-7 cd/ ^m2 , or lower or higher values (e.g., 0, ^10-5 , ^10-8 , ^10-9 cd/ ^m2 , etc.). The GSDF can be used to support a wide range of viewing and/or display scenarios with different ambient light levels. The GSDF can be used to support a wide range of display devices (in theaters, indoors, or outdoors) with different black levels.

The upper bound of the illuminance values represented in the reference GSDF model can be 12,000 cd/m ² , or lower or higher values (6000-8000, 8000-10000, 10000-12000, 12000-15000 cd/m ² , etc.). The GSDF can be used to support a wide range of viewing and/or display scenarios with high dynamic range. The GSDF can be used to support a wide range of display devices with different maximum illuminance levels (HDR TVs, SDR displays, laptops, tablets, handheld devices, etc.).

6. Variable spatial frequency

FIG2 illustrates an example integration path (denoted as VDR) according to an example embodiment of the present invention, which can be used as the integration path for obtaining gray levels in a reference GSDF as described herein. In an embodiment, a VDR curve is used to accurately capture the peak contrast sensitivity of human vision over a high dynamic range of luminance values.

As shown in Figure 2, as the light adaptation level (illuminance value) decreases, peak contrast sensitivity occurs not at a fixed spatial frequency value, but at smaller spatial frequencies. This means that technologies with a fixed spatial frequency (e.g., DICOM) can significantly underestimate the contrast sensitivity of human vision for dark light adaptation levels (low illuminance values). Lower contrast sensitivity results in a higher contrast threshold, which leads to a larger quantization step size in quantizing illuminance values.

Unlike the Digital Imaging and Communications in Medicine (DICOM) standard, the VDR curve according to the techniques described herein does not fix the spatial frequency model parameters to a fixed value, such as 4 cycles per degree. Instead, the VDR curve varies with spatial frequency and accurately captures the peak contrast sensitivity of human vision at multiple light adaptation levels. The VDR curve appropriately accounts for edge effects caused by the adaptability of human vision to a wide range of light adaptation levels and helps generate a high-precision reference GSDF. Here, the term "high-precision" means that perceptual errors caused by quantization of illuminance values are eliminated or significantly reduced, based on a reference GSDF that best and most efficiently captures the nonlinearities of human vision within the constraints of a fixed-size code space (e.g., 10 bits, 12 bits, etc.).

A computational process for computing gray levels in a reference GSDF (e.g., Table 1) may be used. In an exemplary embodiment, the computational process is iterative or recursive, repeatedly determining a contrast threshold (or modulation threshold, e.g., _ml in Expression 4) from a VDR curve and applying the contrast threshold to obtain continuous gray levels in the reference GSDF. The computational process may be implemented using the following Expression (7):

So: and

Expression (7)

Where j-1, j, and j+1 represent the indices of three adjacent digital code values; Lj _-1 , _Lj , and Lj ₊₁ represent the grayscale levels to which digital code values j-1, j, and j+1 are mapped, respectively. _Lmax and _Lmin represent the maximum and minimum illuminance values, respectively, over a JND or a fraction of the JND. Using the JND or a fraction of the JND maintains high accuracy of the reference GSDF.

The contrast threshold _ml associated with the JND can be defined as a relative quantity, for example, the difference between L _max and L _min divided by a specific illuminance value of L _max or L _min , or a value between L _max and L _min (e.g., the average of L _max and L _min ). In some embodiments, _ml can alternatively be defined as the difference between L _max and L _min divided by a multiple (e.g., 2) of the specific illuminance value of L _max or L _min , or a value between L max and L _min . When the illuminance values in the GSDF are quantized into multiple gray levels, _{L max and} L _min can refer to adjacent gray levels among these multiple gray levels. As a result, as shown in Expression (7), L _j can be related to L _j-1 and L _j+1 , respectively, by ml.

In an alternative embodiment, rather than using a linear expression as shown in Expression (7), a nonlinear expression may be used to relate the JND or contrast threshold to the gray level. For example, instead of using a simple ratio for the contrast threshold as shown, an alternative expression based on the standard deviation divided by the mean may be used.

In some embodiments, the reference GSDF covers a range of 0 to 12,000 cd/m ² , where the digital code values are represented as 12-bit integer values. To further improve the accuracy of the reference GSDF, _ml can be multiplied by a fractional value f. In addition, the center digital value L2048 (note that, as in the 12-bit code space compatible with SDI, the digital code values are limited to at least 0 and 4096) can be mapped to 100 cd/m ^2. Expression (7) can be derived from the following expression (8):

and

Expression (8)

Here, the fractional value f is set to 0.918177. In the exemplary embodiment, the minimum allowable value of the digital code is set to codeword (or integer value) 16, which is set to 0 (cd/m ² ). The second lowest digital code value 17 ends at 5.27×10-7 cd/m ² , while the digital code value 4076 ends at 12,000 cd/m ² .

3 illustrates an example GSDF mapping between multiple gray levels (expressed as logarithmic luminance values) and multiple digital code values in a 12-bit code space according to an example embodiment of the present invention.

FIG4 illustrates a graph depicting the Weber score (Delta L/L or ΔL) for grayscale levels based on the example GSDF of FIG3 . The perceptual nonlinearity of human vision, as shown in FIG4 , is represented as a function of illuminance value on a logarithmic illuminance axis. Comparable visual differences (e.g., JNDs) in human vision correspond to larger Delta L/L at lower illuminance values. The Weber score curve asymptotically becomes constant at high illuminance values (e.g., when the Weber law is satisfied at higher illuminance values, the Weber score is 0.002).

7. Function Model

One or more analysis functions can be used to obtain the mapping between digital code values and grayscale levels in a GSDF (a reference GSDF or a device-specific GSDF) as described herein. The one or more analysis functions can be proprietary, standard-based, or extensions of standard-based functions. In some embodiments, a GSDF generator (e.g., 504 of FIG. 5 ) can generate a GSDF in the form of one or more forward lookup tables (LUTs) and/or one or more inverse LUTs based on the one or more analysis functions (or formulas). At least some of these LUTs can be provided to various image data codecs (e.g., 506 of FIG. 5 ) or various display devices for converting between reference grayscale levels and reference digital code levels for the purpose of encoding reference image data. Additionally, optionally, or alternatively, at least some of the analysis functions (whose coefficients are represented as integers or floating point numbers) can be provided directly to an image data codec or various display devices for use in obtaining a mapping between digital code values and grayscale levels in a GSDF as described herein and/or for converting between grayscale levels and digital code levels for the purpose of encoding image data.

In some embodiments, the analysis functions as described herein include the following forward function that can be used to predict digital code values based on corresponding gray levels:

Wherein, D represents the (e.g., 12-bit) value of the digital code, L represents the luminance value or grayscale in nits, n can represent the slope in the middle part of the logD/logL curve as given by Expression (9), m can represent the sharpness of the inflection point of the logD/logL curve, and c1, c2, and c3 can define the endpoints and the middle point of the logD/logL curve.

In some embodiments, the analysis function comprises the following inverse function, which corresponds to the forward function in expression (9) and can be used to predict the illuminance value based on the corresponding digital code value:

The digital code values predicted based on the plurality of illumination values using expression (9) can be compared with the observed digital code values. The observed digital code values can be, but are not limited to, any of the following: numerical calculations based on the CSF model as discussed above. In an embodiment, the deviation between the predicted digital code values and the observed digital code values can be calculated and minimized to derive optimal values for the parameters n, m, c ₁ , c ₂ , and c ₃ in expression (9).

Similarly, the illuminance values predicted based on the plurality of digital code values using expression (10) can be compared with the observed illuminance values. The observed illuminance values can be generated, but are not limited to, by numerical calculations based on the CSF model as discussed above, or by using human vision experimental data. In an embodiment, the deviation between the predicted illuminance values and the observed illuminance values can be derived as a function of the parameters n, m, c ₁ , c ₂ , and c ₃ , and the deviation can be minimized to derive the optimal values of the parameters n, m, c ₁ , c ₂ , and c ₃ in expression (10).

The set of optimal values for the parameters n, m, c ₁ , c ₂ and c ₃ determined by expression (9) may be the same as or different from the set of optimal values for the parameters n, m, c ₁ , c ₂ and c ₃ determined by expression (10). In the event that the two sets differ, one or both of the two sets may be used to generate a mapping between digital code values and luminance values. In some embodiments, the two optimal value sets for the parameters n, m, c ₁ , c ₂ and c ₃ may be coordinated under different circumstances, for example, based on minimization of round-trip errors introduced by performing both forward decoding and reverse decoding operations using expressions (9) and (10). In some embodiments, multiple round trips may be performed to investigate the resulting errors in digital code values and/or luminance values or grayscale levels. In some embodiments, the selection of the parameters in expressions (9) and (10) may be based at least in part on the criterion that no significant errors occur in one, two or more round trips. Examples of no significant round trip error may include, but are not limited to, any of the following errors: errors of less than 0.0001%, 0.001%, 0.01%, 1%, 2%, or other configurable values.

Embodiments include using one or more code spaces of different bit lengths to represent digital control values. Optimized values for the parameters in expressions (9) and (10) can be obtained for each of a plurality of code spaces, each of the plurality of code spaces having a different one of the one or more different bit lengths. Based on the optimized values of expressions (9) and (10), a distribution of code errors (e.g., forward transform errors, inverse transform errors, or round-trip errors in digital code values based on expressions (9) and (10)) can be determined. In some embodiments, a numerical difference of one (1) between two digital code values corresponds to a contrast threshold (or to a JND) at a certain light level between two illuminance values represented by the two digital code values. FIG10A illustrates a maximum amount of code errors in JNDs for a plurality of code spaces, each of the plurality of code spaces having a different one of one or more different precisions (having different bit lengths), according to some example embodiments. For example, based on the functional model as described herein, the maximum code error for a code space having an infinite or unrestricted bit length is 11.252. In contrast, based on the functional model described herein, the maximum code error for a code space of 12 bits (or 4096) is 11.298. This indicates that a code space of 12 bits is an excellent choice for digital code values for the functional model represented by expressions (9) and (10).

FIG10B illustrates the distribution of code errors for a 12-bit (or 4096) long code space under the forward transformation (from luminance values to digital code values) specified by Expression (9) according to an exemplary embodiment. FIG10C illustrates the distribution of code errors for a 12-bit (or 4096) long code space under the reverse transformation (from digital code values to luminance values) specified by Expression (10) according to an exemplary embodiment. Both FIG10B and FIG10C indicate that the maximum code error is less than 12.5.

FIG11 illustrates values of parameters that can be used in expressions (9) and (10) according to example embodiments. In some embodiments, as shown, in a particular implementation of the function model as described herein, these non-integer values can be represented/approximated using integer-based formulas. In some other embodiments, in a particular implementation of the function model as described herein, these non-integer values can be represented using fixed-point values, floating-point values, or one of one or more precisions (e.g., 14 bits, 16 bits, or 32 bits).

Embodiments include using a function model having a formula other than those given in expressions (9) and (10) (which may be a tone mapping curve). For example, the function model described herein may use a cone model having the following Naka-Rushton formula:

Wherein, L represents the illuminance value, n, m and σ represent the model parameters associated with the cone model, and _Ld represents the predicted value that can be encoded with a digital code value. A similar method for obtaining the model parameters by minimizing the deviation can be used to derive the optimal values of the model parameters for expression (11). Figure 10D illustrates the distribution of code errors for a code space of 12 bits in length (or 4096) in the case of the forward transformation (from illuminance values to digital code values) specified by expression (11) according to an example embodiment. In an embodiment, the maximum code error as shown in Figure 10D is 25 JNDs.

In another example, a functional model with the following Raised mu formula can be generated:

Where x represents the illuminance value and y represents the predicted digital code value. The optimal value of the model parameter μ can be obtained by minimizing the deviation. FIG10E illustrates the distribution of code errors for a code space of 12 bits (or 4096) in the case of the forward transformation (from illuminance values to digital code values) specified by expression (12) according to an example embodiment. In an embodiment, the maximum code error as shown in FIG10D is 17 JNDs.

As shown herein, in some embodiments, a function model can be used to predict code values from illuminance values or to predict illuminance values from code values. The formula used by the function model can be reversible. The same or similar processing logic can be implemented to perform forward and inverse transformations between these values. In some embodiments, model parameters, including but not limited to any exponentials, can be represented by formulas based on fixed-point values or integers. Therefore, at least a portion of the processing logic can be efficiently implemented using only hardware, only software, or a combination of hardware and software. Similarly, at least a portion of the LUT generated using the function model or model formula (such as Expressions (9) to (12)) can be efficiently implemented using only hardware, only software, or a combination of hardware and software (including ASICs or FPGAs). In some embodiments, one, two, or more function models can be implemented in a single computing device, a configuration of multiple computing devices, a server, or the like. In some embodiments, the error in the predicted code value can be within 14 code values of the target or observed value over the entire visible dynamic range of the illuminance value. In some embodiments, this applies to both forward and inverse transformations. The same or different sets of model parameters can be used in the forward and inverse transforms. Optimal values of the model parameters can be used to maximize round-trip accuracy. Different code spaces can be used. In a particular embodiment, a 12-bit (4096) code space can be used to manage digital code values with minimal code error across the entire visible dynamic range.

As used herein, a reference GSDF may refer to a GSDF that includes reference digital code values and reference grayscale levels, which are related according to a functional model (whose model parameters can be determined using target values or observed values under the CSF model), determined through numerical calculations based on the CSF model (e.g., any functional representation of the mapping between uncertain digital code values and illuminance values), or determined using data from human vision research. In some embodiments, the device GSDF may also include a mapping between digital code values and grayscale levels that can be analytically represented using a functional model as described herein.

8. Exchange image data based on reference GSDF

For purposes of illustration, the digital code values are described as residing in a 12-bit code space. However, the present invention is not limited thereto. Digital code values having different code spaces (e.g., different bit depths other than 12 bits) can be used in the reference GSDF. For example, a 10-bit integer value can be used to represent the digital code. Instead of mapping the digital code value 4076 to an illuminance value of 12000 cd/m ² in the 12-bit representation of the digital code, the digital code value 1019 can be mapped to an illuminance value of 12000 cd/m ² in the 10-bit representation of the digital code. Thus, these and other variations in code space (bit depth) can be used for the digital code values in the reference GSDF.

A reference GSDF can be used to exchange image data between different GSDFs, each designed for a specific type of image acquisition device or image rendering device. For example, a GSDF implemented for a particular type of image acquisition device or image rendering device may implicitly or explicitly depend on model parameters that do not match the model parameters of a device-specific GSDF or a standard GSDF for another type of image acquisition device or image rendering device.

The reference GSDF can correspond to the curve shape depicted in Figures 3 and 4. Generally speaking, the shape of a GSDF depends on the parameters used to derive or design the GSDF. Therefore, the reference GSDF depends on the reference CSF model and the reference model parameters used to generate the reference GSDF from the reference CSF model. The curve shape of a device-specific GSDF depends on the specific device (including display parameters and viewing conditions if the specific device is a display).

In an example, as shown in FIG3, a display whose supported luminance value range is limited to less than 500 cd/ ^m2 may not experience an increase in slope at high luminance value regions (which occurs for all frequencies when human vision transitions to logarithmic behavior). Driving a display having the curve shape of FIG3 can result in a non-optimal (e.g., suboptimal) distribution of gray levels, where too much gray level is distributed in bright areas and not enough in dark areas.

In another example, a low-contrast display is designed for outdoor use under various daylight conditions. The illuminance range of the display may fall largely or almost entirely within the logarithmic behavior region of FIG. 3 . Driving the low-contrast display with the curve shape of FIG. 3 may also result in a non-optimal (e.g., suboptimal) distribution of gray levels, where too much gray level is distributed in dark areas and too little in bright areas.

According to the techniques described herein, each display can use its specific GSDF (which depends not only on display parameters but also on viewing conditions that affect, for example, actual black levels) to optimally support the perceptual information in image data encoded with a reference GSDF. One or more upstream (e.g., encoding) devices used for the overall encoding of the image data use the reference GSDF to preserve as much perceptual detail as possible. The image data encoded in the reference GSDF is then delivered to one or more downstream (e.g., decoding) devices. In example embodiments, the encoding of the image data based on the reference GSDF is independent of the specific device that will subsequently decode and/or present the image data.

Each device (e.g., a display) has a specific GSDF, in which device-specific grayscale levels are supported/optimized. The specific grayscale levels may be known to the display manufacturer or may be specifically designed by the manufacturer to support the device-specific GSDF (which may or may not be based on a standard). The device's line driver may be implemented using quantized luminance values specific to the device. For each device, optimization may be best performed based on the device-specific quantized luminance values. Furthermore, a dark black level (e.g., a minimum device-specific grayscale level) that serves as the lower limit of the device-specific grayscale range may be set based at least in part on the current ambient light level and/or the device's optical reflectivity (which may be known to the manufacturer). Once the dark black level is set, the device-specific grayscale level may be obtained or set by implicitly or explicitly accumulating (e.g., stacking/integrating) the quantization step size in the device's line driver. Grayscale derivation and/or adjustment may or may not be performed at runtime when the device is simultaneously presenting images.

Thus, according to the techniques as described herein, embodiments of the present invention may include, but are not limited to, encoding image data using a reference GSDF and decoding and rendering the image data using a device-specific GSDF.

[0014] Techniques as described herein can be used to exchange image data between various devices having different GSDFs. FIG5 illustrates an example framework (500) for exchanging image data with devices having different GSDFs according to an example embodiment of the present invention. As shown in FIG5, an adaptive CSF model (502) can be used to generate a reference GSDF (504). The term "adaptive" can refer to the adaptability of the CSF model to the nonlinearities and behavior of human vision. The adaptive CSF model can be constructed based at least in part on a plurality of CSF parameters (or model parameters). The plurality of model parameters include, for example, light adaptation level, display area in degrees of width, noise level, eye accommodation (physical viewing distance), illuminance, or color modulation vector (which can, for example, be related to a test image or image pattern used in the adaptive CSF model (502)).

An upstream (e.g., encoding) device may receive image data to be encoded using a reference GSDF (504) before the image data or its derivatives are sent or distributed to a downstream (e.g., decoding) device. The image data to be encoded may initially be in any of a variety of formats (standard-based, proprietary, extensions thereof, etc.) and/or may be derived from any of a variety of image sources (cameras, image servers, tangible media, etc.). Examples of image data to be encoded include, but are not limited to, original or other high-bit-depth images (530). Original or other high-bit-depth images may come from a camera, a studio system, an art director's system, another upstream image processing system, an image server, a content database, etc. The image data may include, but are not limited to, image data of a digital photograph, a video image frame, a 3D image, a non-3D image, a computer-generated graphic, etc. The image data may include scene-dependent images, device-dependent images, or images having various dynamic ranges. Examples of image data to be encoded may include a high-quality version of the original image that is edited, downsampled, and/or compressed along with metadata into a decoded bitstream for distribution to image receiving systems (downstream image processing systems, such as displays from various manufacturers). The original or other high-bit-depth images may be of the high sampling rates used by professionals, art studios, broadcasters, high-end media production entities, and the like. The image data to be encoded may also be generated in whole or in part by a computer, or even obtained in whole or in part from existing image sources, such as old films and documentaries.

As used herein, the phrase "image data to be encoded" may refer to image data of one or more images; the image data to be encoded may include floating-point image data or fixed-point image data and may be in any color space. In an example embodiment, the one or more images may be in an RGB color space. In another example embodiment, the one or more images may be in a YUV color space. In an example, each pixel in an image as described herein includes floating-point pixel values for all channels defined in the color space (e.g., red, green, and blue color channels in an RGB color space). In another example, each pixel in an image as described herein includes fixed-point pixel values for all channels defined in the color space (e.g., 16-bit or higher/lower bit fixed-point pixel values for the red, green, and blue color channels in an RGB color space). Each pixel may optionally and/or alternatively include downsampled pixel values for one or more of the channels in the color space.

In some embodiments, in response to receiving image data to be encoded, an upstream device in the framework (500) maps luminance values specified by or determined from the image data to reference digital code values in a reference GSDF, and generates reference encoded image data encoded with the reference digital code values based on the image data to be encoded. The mapping operation from luminance values based on the image data to be encoded to reference digital code values may include: selecting digital code values whose corresponding reference grayscale levels (e.g., as shown in Table 1) match or approximate as closely as any other reference luminance value in the reference GSDF to the luminance value specified by or determined from the image data to be encoded; and replacing the luminance value with the reference digital code value in the reference encoded image data.

Additionally, optionally or alternatively, pre-processing and post-processing steps (which may include, but are not limited to, color space conversion, downsampling, upsampling, tone mapping, color grading, decompression, compression, etc.) may be performed as part of generating the reference encoded image data.

In an example embodiment, the framework (500) may include software and/or hardware components (e.g., encoding or formatting unit (506)) configured to encode and/or format the reference encoded image data into one or more decoded bitstreams or image files. The decoded bitstreams or image files may be in a standard-based format, a proprietary format, or an extension of a standard-based format. Additionally and/or alternatively, the decoded bitstreams or image files may include metadata containing one or more parameters of the reference GSDF related to pre-processing or post-processing used to generate the reference encoded image data (e.g., model parameters; minimum illuminance value, maximum illuminance value, minimum digital code value, maximum digital code value, etc. as shown in Table 1, Figures 3 and 4; an identification field identifying one of a plurality of CSFs; a reference viewing distance).

In some embodiments, the framework (500) may include one or more separate upstream devices. For example, at least one of the one or more upstream devices in the framework (500) may be configured to encode image data based on a reference GSDF. The upstream device may include software and/or hardware components configured to perform the functions associated with 502, 504, and 506 of FIG. 5 . The decoded bitstream or image file may be output by the upstream device (502, 504, and 506 of FIG. 5 ) via a network connection, a digital interface, a tangible storage medium, etc., and delivered to other image processing devices for processing or presentation in the image data flow (508).

In some example embodiments, the framework (500) further includes one or more downstream devices that are one or more separate devices. The downstream devices can be configured to receive/access the decoded bitstream or image file output by the one or more upstream devices from the image data stream (508). For example, the downstream devices can include software and/or hardware components (e.g., a decoding or reformatting unit (510)) configured to decode and/or reformat the decoded bitstream and image file and recover/retrieve the reference encoded image data therein. As shown in FIG5 , the downstream devices can include a different set of display devices.

In some embodiments, a display device (not shown) may be designed and/or implemented to support the reference GSDF. If the display device supports every grayscale level in the reference GSDF, high-precision HDR image rendering can be provided. The display device can render image details at a level finer than or equal to that detectable by human vision.

In some embodiments, the raw digital code values in a display device's device-specific GSDF (which can be implemented as digitized voltage values in the display system, such as digital drive levels or DDLs) can correspond to device-specific grayscale levels (or luminance values) that differ from the digital code values in the reference GSDF. The device-specific grayscale levels can be designed to support sRGB, Rec.709, or other specifications (including those using complementary density-dependent representations). Additionally, optionally, or alternatively, the device-specific grayscale levels can be based on the underlying DAC characteristics of the display driver.

In some embodiments, display device A (512-A) can be designed and/or implemented as a device-specific GSDF A (514-A) that supports a visible dynamic range (VDR) display. GSDF A (514-A) can be based on a bit depth of 12 bits (12-bit code space) for device-specific digital code values, a 10,000:1 contrast ratio (CR), and a >P3 domain. GSDF A (514-A) can support grayscale levels within a first sub-range (e.g., 0 to 5,000 cd/m ² ) of the entire range of a reference GSDF (504). Alternatively and/or optionally, GSDF A (514-A) can support the entire range (e.g., 0 to 12,000 cd/m ² ) in the reference GSDF (504), but can include fewer reference grayscale levels than all reference grayscale levels in the reference GSDF (504).

In some embodiments, display device B (512-B) may be designed and/or implemented to support a device-specific GSDF B (514-B) for a dynamic range narrower than the VDR. For example, display device B (512-B) may be a standard dynamic range (SDR) display. As used herein, the terms "standard dynamic range" and "low dynamic range" and/or their corresponding abbreviations "SDR" and "LDR" may be used synonymously and/or interchangeably. In some embodiments, GSDF B (514-B) may support a bit depth of 8 bits for device-specific digital code values, a contrast ratio (CR) of 500-5,000:1, and a color gamut as defined in Rec. 709. In some embodiments, GSDF B (514-B) may provide grayscale levels within a second sub-range (e.g., 0 to 2000 cd/m ² ) of the reference GSDF (504).

In some embodiments, the display device C (512-C) can be designed and/or implemented to support a device-specific GSDF C (514-C) for a dynamic range even narrower than SDR. For example, the display device C (512-C) can be a flat panel display. In some embodiments, the GSDF C (514-C) can support a bit depth of 8 bits for device-specific digital code values, a contrast ratio (CR) of 100-800:1, and a color gamut smaller than that defined in Rec. 709. In some embodiments, the GSDFC (514-C) can provide grayscale levels within a third sub-range (e.g., 0 to 1,200 cd/m ² ) of the reference GSDF (504).

In some embodiments, a display device (e.g., display device D (512-D)) may be designed and/or implemented to support a device-specific GSDF (e.g., GSDF D (514-D)) for a very limited dynamic range that is much narrower than SDR. For example, display device D (512-D) may be an electronic paper display. In some embodiments, GSDF D (514-D) may support a bit depth of 6 bits for device-specific digital code values, a contrast ratio (CR) of 10:1 or less, and a color gamut that is much smaller than the color gamut defined in Rec. 709. In some embodiments, GSDF D (514-D) may provide grayscale levels within a fourth sub-range (e.g., 0 to 100 cd/m ² ) of the reference GSDF (504).

The accuracy of the image rendering can be appropriately scaled down for each of the display devices A to D (512-A to D). In some embodiments, a subset of the gray levels in each of the device-specific GSDFs A to D (514-A to D) can be correlated with or mapped to the reference gray levels supported in the reference GSDF (504) so as to evenly distribute perceptually significant errors across the range of gray levels supported by the display device.

In some embodiments, a display device (e.g., one of 512-A through 512-D) having a device-specific GSDF (e.g., one of 514-A through 514-D) receives/extracts reference encoded image data encoded based on a reference GSDF. In response, the display device or a conversion unit (e.g., one of 516-A through 516-D) therein maps reference digital code values specified in the reference encoded image data to device-specific digital code values native to the display device. This can be performed in one of several ways. In one example, mapping the reference digital code values to the device-specific digital code values includes selecting a device-specific grayscale level (corresponding to the device-specific digital code value) that matches or approximates the reference grayscale level (corresponding to the reference digital code value) as closely as any other device-specific grayscale level. In another example, mapping from a reference digital code value to a device-specific digital code value includes: (1) determining a tone-mapped luminance value based on a reference gray level (corresponding to the reference digital code value) associated with a reference GSDF, and (2) selecting a device-specific gray level (corresponding to the device-specific digital code value) that matches or approximates the tone-mapped luminance value as closely as any other device-specific gray level.

The display device, or a driver chip therein (one of 518-A through 518-D), can then use the display specific digital code value to render an image using a device specific grayscale level corresponding to the display specific code value.

In general, the reference GSDF can be based on a different CSF model than the CSF model on which the display-specific GSDF is based. Conversion/mapping between the reference GSDF and the device-specific GSDF is necessary. Even if the same CSF model is used to generate both the reference GSDF and the device-specific GSDF, different values of the model parameters can be used in the derived GSDF. For the reference GSDF, the model parameter values can be conservatively set to preserve details for a variety of downstream devices, while for the device-specific GSDF, the model parameters can reflect the specific design/implementation and viewing conditions under which the display device will present the image. Conversion/mapping between the reference GSDF and the device-specific GSDF is still necessary because the viewing condition parameters of a specific display device (e.g., ambient light level, optical reflectivity of the display device, etc.) are different from the model parameter values used to derive the reference GSDF. Here, the viewing condition parameters can include those that degrade display quality (e.g., contrast ratio, etc.) and elevate black levels (e.g., minimum grayscale level, etc.). The conversion/mapping between a reference GSDF and a device-specific GSDF according to the techniques as described herein improves image rendering quality (e.g., improving contrast ratio by increasing luminance values at high-value areas, etc.).

9. Convert reference code data

FIG6 illustrates an example conversion unit (e.g., 516) according to some embodiments of the present invention. The conversion unit (516) may be, but is not limited to, one of the multiple conversion units (e.g., 516-A to 516-D) shown in FIG5 (e.g., 516-A). In some embodiments, the conversion unit (516) may receive first definition data for a reference GSDF (REF GSDF) and second definition data for a device-specific GSDF (e.g., GSDF-A (514-A in FIG5)). As used herein, if the device is a display, the terms "device-specific" and "display-specific" may be used synonymously.

Based on the received definition data, the conversion unit (516) concatenates the reference GSDF with the display-specific GSDF to form a conversion lookup table (conversion LUT). The concatenation between the two GSDFs may include comparing the grayscale levels in the two GSDFs and, based on the result of the grayscale comparison, establishing a mapping between the reference digital code value in the reference GSDF and the display-specific code value in the display-specific GSDF.

More specifically, given a reference digital code value in a reference GSDF, its corresponding reference grayscale can be determined based on the reference GSDF. The thus-determined reference grayscale can be used to find a device-specific grayscale in a display-specific GSDF. In an exemplary embodiment, the found device-specific grayscale can match or approximate the reference grayscale as closely as any other display-specific grayscale in the display-specific GSDF. In another exemplary embodiment, a tone-mapped luminance value can be obtained by applying a global or local tone mapping operator to the reference grayscale; the found device-specific grayscale can match or approximate the tone-mapped luminance value as closely as any other display-specific grayscale in the display-specific GSDF.

For device-specific gray levels, the corresponding display-specific digital code values can be identified from the display-specific GSDF. An entry consisting of a reference digital code value and a display-specific code value can be added or defined in the conversion LUT.

The steps described above may be repeated for other reference digital code values in the reference GSDF.

In some embodiments, a conversion LUT may be pre-built and stored prior to receiving and processing image data whose processing will be performed at least in part based on the conversion LUT. In alternative embodiments, the image data to be processed using the conversion LUT is analyzed. The results of the analysis can be used to set or at least adjust the correspondence between reference digital code values and device-specific digital code values. For example, if the image data indicates a particular concentration or distribution of luminance values, the conversion LUT can be configured to preserve a high degree of detail in the concentrated region of luminance values.

In some embodiments, the conversion unit (516) includes one or more software and/or hardware components (comparison subunit (602)) configured to compare the quantization step sizes (e.g., luminance value differences or ΔL between adjacent digital code values) in both the reference GSDF and the display-specific GSDF (514-A). For example, the quantization step size at the reference digital code value in the reference GSDF can be a reference luminance value difference (reference GSDFΔL), while the quantization step size at the display-specific digital code value in the display-specific GSDF can be a display-specific luminance value difference (display-specific GSDFΔL). Here, the display-specific digital code value corresponds to the reference digital code value (or forms a pair with the reference digital code value in the conversion LUT). In some embodiments, the comparison subunit (602) compares the two luminance value differences. This operation is essentially a test that can be performed based on the ΔL value or, alternatively and/or alternatively, based on the relative slopes of the two GSDF curves.

The quantization step size of luminance values in a display-specific GSDF can typically exceed the quantization step size of a reference GSDF because one or more reference grayscale levels from the reference GSDF (e.g., corresponding to a high bit-depth domain, etc.) are merged into the display-specific grayscale levels from the display-specific GSDF (e.g., corresponding to a low bit-depth domain, etc.). In these cases, dithering is used to remove banding artifacts. As part of the overall dithering, dithering (in space and/or in time) is also performed on the local surrounding output pixels. In a sense, the human eye can be represented as a low-pass filter. At least in this sense, averaging the local surrounding pixels as described herein creates a desired output grayscale level that reduces and/or removes visual banding artifacts that might otherwise be present due to the large quantization step size in the display-specific GSDF.

In less common cases, the quantization step size of the luminance values of the reference GSDF may sometimes exceed the quantization step size of the display-specific GSDF. A process based on a decontouring algorithm is used, for example, to synthesize output gray levels based on input gray levels by averaging adjacent input pixels.

Accordingly, if the reference GSDFΔL is greater than the display specific GSDFΔL ("Y" path in Figure 6), the decontouring algorithm flag is set for the entry in the conversion LUT that includes the reference digital code value and the display specific digital code value.

If the reference GSDFΔL is less than the display specific GSDFΔL (in Figure 6, the "N" path), the dither algorithm flag is set for the entry in the conversion LUT that includes the reference digital code value and the display specific digital code value.

If the reference GSDFΔL is equal to the display specific GSDFΔL, neither the decontouring algorithm flag nor the dithering algorithm flag is set for the entry in the conversion LUT that includes the reference digital code value and the display specific digital code value.

The decontouring and dithering algorithm flags may be stored with the entries in the conversion LUT, or may be stored in associated data structures external to the conversion LUT, but operatively linked to the conversion LUT.

In some embodiments, the conversion unit (516) is configured to receive reference encoded image data, which may be in the form of a high bit depth or floating point input image, and map reference digital code values specified in a reference GSDF to display-specific digital code values specified in a display-specific GSDF. In addition to mapping digital code values between GSDFs, the conversion unit (516) may also be configured to perform decontouring or dithering based on the settings of the algorithm flags (decontouring algorithm flag or dithering algorithm flag) discussed above.

As noted, the reference GSDF is likely to contain more details than the display-specific GSDF; therefore, the "Y" path of Figure 6 may not occur, or may occur less frequently. In some embodiments, the "Y" path and related processing can be omitted to simplify the implementation of the conversion unit.

In some embodiments, given a reference digital code value determined for a pixel in the reference encoded image data, the conversion unit (516) looks up the corresponding display-specific digital code value in the conversion LUT and replaces the reference digital code value with the corresponding display-specific digital code value. Additionally and/or alternatively, the conversion unit (516) determines whether a decontouring algorithm or a dithering algorithm should be performed on the pixel based on the presence/setting of an algorithm flag for an entry in the conversion LUT that includes the reference digital code value and the display-specific digital code value.

If it is determined that neither the decontouring algorithm nor the dithering algorithm should be executed (e.g., there is no indication or flag to execute either algorithm), neither decontouring nor dithering is temporarily executed on the pixel.

If it is determined that a decontour algorithm should be executed, the conversion unit (516) may execute one or more decontour algorithms. Executing the one or more decontour algorithms may include receiving input image data of local adjacent pixels and inputting the image data of the local adjacent pixels into the decontour algorithm.

If it is determined that a dithering algorithm should be executed, the conversion unit (516) may execute one or more dithering algorithms.

If the conversion unit (516) determines that decontouring or dithering needs to be performed on a neighboring pixel, the pixel may still be included in the decontouring or dithering. In one example, the device-specific (output) gray level of the pixel may be used to dither the local neighboring pixels. In another example, the reference (input) gray level of the pixel may be used to decontouring the local neighboring pixels.

In some embodiments, the conversion unit (516) outputs the processing results of the previous steps to a downstream processing unit or sub-unit. The processing results include display-specific encoded image data in the form of a display-specific bit-depth output image encoded using digital code values in a display-specific GSDF (e.g., GSDF-A).

FIG7 illustrates an example SDR display (700) implementing 8-bit image processing. The SDR display (700) or a VDR decoding unit (702) therein receives an encoded input. The encoded input includes reference decoded image data in an image data container that can be one of a variety of image data container formats. The VDR decoding unit (702) decodes the encoded input and determines/retrieves reference encoded image data therefrom. The reference encoded image data can include image data for each pixel in a color space (e.g., RGB color space, YCbCr color space, etc.). The image data for each pixel can be encoded using a reference digital code value in a reference GSDF.

Additionally and/or alternatively, the SDR display (700) includes a display management unit (704) that maintains display parameters for the SDR display (700). The display parameters may at least partially define a display-specific GSDF (e.g., GSDF-B of FIG. 5 ) associated with the SDR display (700). The display parameters that define the display-specific GSDF may include the maximum (max) and minimum (min) grayscale levels supported by the SDR display (700). The display parameters may also include the color primitives (primaries) supported by the SDR display, the display size (size), the optical reflectivity of the image rendering surface of the SDR display, and the ambient light level. Some of the display parameters may be pre-configured with fixed values. Some of the display parameters may be measured by the SDR display (700) in real time or near real time. Some of the display parameters may be configured by a user of the SDR display (700). Some of the display parameters may be pre-configured with default values and may be overwritten by measurement or by the user. The display management unit (704) establishes/shapes the perceptual nonlinearity of display-specific grayscales based on a reference GSDF and may additionally and/or alternatively perform tone mapping as part of establishing/shaping the display-specific grayscales. For example, to establish/shape the perceptual nonlinearity of display-specific grayscales based on a reference GSDF, the display management unit (704) may establish a conversion LUT and/or other related metadata (e.g., dithering and decontouring flags, etc.) as shown in FIG5 . The display management unit (704) may implement a cascade operation as discussed above to create a conversion LUT and/or other related metadata (712) associated with one or both of the reference GSDF and the display-specific GSDF. The conversion LUT and/or other related metadata (712) may be accessed and used by other units or subunits in the SDR display (700). Furthermore, the conversion LUT and/or other related metadata may be used as or derived from metadata (714) for performing an inverse operation on the perceptual nonlinearity. As used herein, inverting perceptual nonlinearity may include converting display-specific digital code values to display-specific digital drive levels (e.g., digitized voltage levels in a display device).

Additionally and/or alternatively, the SDR display (700) includes a conversion unit (516) and an 8-bit perceptual quantizer (706) as shown in Figures 5 and 6. In some embodiments, the SDR display (700), or the conversion unit (516) and the 8-bit perceptual quantizer (706) therein, converts the reference-encoded image data into a display-specific bit-depth output image encoded with a display-specific digital code value associated with a display-specific GSDF (e.g., GSDF-A or GSDF-B of Figure 5), and quantizes the display-specific bit-depth output image into perceptually encoded image data in an 8-bit code space. As used herein, the term "perceptual encoding" may refer to an encoding based on a human visual perception model, such as a CSF that generates the reference GSDF.

Additionally and/or alternatively, the SDR display (700) includes a video post-processing unit (708) that may, but is not limited to, perform no image processing operation, perform one or more image processing operations on the perceptually coded image data represented by 8-bit luminance. These image processing operations may include, but are not limited to, compression, decompression, color space conversion, downsampling, upsampling, or color grading. The results of these operations may be output to other parts of the SDR display (700).

In an example embodiment, an SDR display (700) includes an 8-bit inverse perceptual quantizer (710) configured to convert display-specific digital code values from a result of an image processing operation into display-specific digital drive levels (e.g., digitized voltage levels). The display-specific digital drive levels generated by (or converted back from) the digital code values by the inverse perceptual quantizer (710) can specifically support one of several luminance nonlinearities supportable in the SDR display (700). In one example, the inverse perceptual quantizer (710) converts the display-specific digital code values into display-specific digital drive levels to support luminance nonlinearities associated with Rec. 709. In another example, the inverse perceptual quantizer (710) converts the display-specific digital code values into display-specific digital drive levels to support luminance nonlinearities associated with a linear luminance domain or a logarithmic luminance domain (which can be relatively easily integrated with local dimming operations). In another example, the inverse perceptual quantizer (710) converts display-specific digital code values into display-specific digital drive levels to support a display-specific CSF (or its associated GSDF), wherein the display-specific gray levels are optimally positioned for the specific display (700) and can be adjusted for viewing conditions specific to the display (700).

10. Example Processing Flow

FIG8A illustrates an example process flow according to an embodiment of the present invention. In some embodiments, one or more computing devices or components (such as one or more computing devices in the framework (500)) can perform the process flow. In block 802, a computing device receives image data to be encoded.

At block 804, the computing device encodes the image data to be encoded into reference encoded image data based on a reference mapping between a set of reference digital code values and a set of reference grayscale levels. Here, luminance values in the image data to be encoded are represented by the set of reference digital code values. The luminance difference between two reference grayscale levels represented by two adjacent reference digital code values in the set of reference digital code values may be inversely proportional to the peak contrast sensitivity of human vision for a particular light level.

In block 806 , the computing device outputs the reference encoded image data.

In an embodiment, a computing device determines a reference grayscale display function (GSDF) based on a contrast sensitivity function (CSF) model; the reference GSDF specifies a reference mapping between a set of reference digital code values and a set of reference grayscale levels. The CSF model includes one or more model parameters, and these model parameters may have an angular size within a range including one or more of: between 25 degrees x 25 degrees and 30 degrees x 30 degrees, between 30 degrees x 30 degrees and 35 degrees x 35 degrees, between 35 degrees x 35 degrees and 40 degrees x 40 degrees, between 40 degrees x 40 degrees and 45 degrees x 45 degrees, or greater than 45 degrees x 45 degrees.

In one embodiment, a computing device assigns an intermediate luminance value within the range of luminance values supported by the reference grayscale set to an intermediate digital code value in a code space hosting the reference digital code value set, and derives a plurality of submaps by performing one or more of stacking or integration calculations, each submap mapping a reference digital code value in the reference digital code value set to a reference grayscale level in the reference grayscale set. The intermediate luminance value may be selected from a range comprising one or more of: less than 50 nits, between 50 nits and 100 nits (inclusive), between 100 and 500 nits (inclusive), or not less than 500 nits.

In an exemplary embodiment, the value of the upper limit of the reference grayscale set coverage is a dynamic range of the following values: less than 500 nits, between 500 nits and 1000 nits (including 500 and 1000 nits), between 1000 nits and 5000 nits (including 1000 and 5000 nits), between 5000 nits and 10000 nits (including 5000 and 10000 nits), between 10000 nits and 15000 nits (including 10000 and 15000 nits), or greater than 15000 nits.

In an embodiment, the peak contrast sensitivity is determined from a contrast sensitivity curve among a plurality of contrast sensitivity curves determined based on a contrast sensitivity function (CSF) model, wherein the CSF model has model parameters including one or more of a luminance value variable, a spatial frequency variable, or one or more other variables.

In an embodiment, at least two peak contrast sensitivities determined based on at least two contrast sensitivity curves of the plurality of contrast sensitivity curves occur at two different spatial frequency values.

In an embodiment, a computing device converts one or more input images represented, received, sent or stored by image data to be encoded from an input video signal into one or more output images represented, received, sent or stored by reference encoded image data contained in an output video signal.

In an embodiment, the image data to be encoded includes image data encoded in one of the following: a high-resolution high dynamic range (HDR) image format, an RGB color space associated with the Academy Color Encoding Specification (ACES) standard of the Academy of Motion Picture Arts and Sciences (AMPAS), the P3 color space standard of the Digital Cinema Initiatives Alliance, the Reference Input Medium Metrics/Reference Output Medium Metrics (RIMM/ROMM) standard, the sRGB color space, the RGB color space associated with the BT.709 recommendation standard of the International Telecommunication Union (ITU), etc.

In an embodiment, the luminance difference between two reference gray levels represented by two adjacent reference digital code values is less than a just noticeable difference threshold at a specific light level.

In an embodiment, the specific light level is an illuminance value between two illuminance values (inclusive of the two illuminance values).

In an embodiment, the set of reference digital code values comprises integer values in a code space having a bit depth of: less than 12 bits; between 12 bits and 14 bits (inclusive); at least 14 bits; or 14 bits or more.

In an embodiment, the set of reference gray levels may include a set of quantized luminance values.

FIG8B illustrates another example process flow according to an embodiment of the present invention. In some embodiments, one or more computing devices or components (such as one or more computing devices in the framework (500)) can perform the process flow. In block 852, the computing device determines a digital code mapping between a reference set of digital code values and a device-specific set of digital code values. Here, the reference set of digital code values is mapped to a reference set of grayscale levels in the reference mapping, while the device-specific set of digital code values is mapped to a device-specific set of grayscale levels in the device-specific mapping.

At block 854, the computing device receives reference encoded image data encoded with reference digital code values. Luminance values in the reference encoded image data are based on a set of reference digital code values. The luminance difference between two reference grayscale levels represented by two adjacent reference digital code values in the set of reference digital code values can be inversely proportional to the peak contrast sensitivity of human vision for a particular light level.

In block 856, the computing device transcodes the reference encoded image data encoded with the reference digital code value set into device-specific image data encoded with the device-specific digital control code set based on the digital code mapping. Luminance values in the device-specific image data are based on the device-specific digital code value set.

In one embodiment, a computing device determines a set of correspondences between a set of reference digital code values and a set of device-specific digital code values. Here, one correspondence in the set of correspondences relates a reference digital code value in the reference digital code value set to a device-specific digital code value. The computing device also compares a first luminance difference at the reference digital code value with a second luminance difference at the device-specific digital code value, and based on the comparison of the first luminance difference with the second luminance difference, stores an algorithm flag indicating whether a dithering operation, a decontouring operation, or no operation should be performed on the reference digital code value.

In one embodiment, a computing device determines a reference digital code value from reference encoded image data for a pixel and further determines whether an algorithm flag is set for the reference digital code value. In response to determining that the algorithm flag is set to decontouring, the computing device performs a decontouring algorithm on the pixel. Alternatively, in response to determining that the algorithm flag is set to dithering, the computing device performs a dithering operation on the pixel.

In one embodiment, a computing device presents one or more images on a display based on device-specific image data encoded with a set of device-specific digital control codes. The display may be, but is not limited to, one of the following: a visible dynamic range (VDR) display, a standard dynamic range (SDR) display, a tablet computer display, or a handheld device display.

In an embodiment, a device-specific grayscale display function (GSDF) specifies a device-specific mapping between a set of device-specific digital code values and a set of device-specific gray levels.

In an embodiment, a device-specific mapping is derived based on one or more display parameters and zero or more viewing condition parameters.

In an embodiment, the device-specific grayscale set covers an upper limit value of a dynamic range of the following values: less than 100 nits; not less than 100 nits but less than 500 nits; between 500 nits and 1000 nits (inclusive); between 1000 nits and 5000 nits (inclusive); between 5000 nits and 10000 nits (inclusive); or greater than 10000 nits.

In an embodiment, a computing device converts one or more input images represented, received, sent, or stored by reference encoded image data from an input video signal into one or more output images represented, received, sent, or stored by device-specific image data contained in an output video signal.

In an embodiment, the device-specific image data supports image presentation in one of the following: a high-resolution high dynamic range (HDR) image format, an RGB color space associated with the Academy Color Encoding Specification (ACES) standard of the Academy of Motion Picture Arts and Sciences (AMPAS), the P3 color space standard of the Digital Cinema Initiatives Alliance, the Reference Input Medium Measurement/Reference Output Medium Measurement (RIMM/ROMM) standard, the sRGB color space, and the RGB color space associated with the BT.709 recommendation of the International Telecommunication Union (ITU).

In an embodiment, the set of device-specific digital code values comprises integer values in a code space having a bit depth of: 8 bits; more than 8 bits but less than 12 bits; 12 bits or more.

In an embodiment, the set of device specific gray levels may comprise a set of quantized luminance values.

In various embodiments, an encoder, decoder, system, etc. performs any one or part of the foregoing methods as described.

11. Implementation mechanism - Hardware overview

According to one embodiment, the techniques described herein are implemented using one or more special-purpose computing devices. The special-purpose computing devices may be hardwired to perform these techniques, or may comprise digital electronics permanently programmed to perform these techniques (such as one or more application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs)), or may comprise one or more general-purpose hardware processors programmed to execute these techniques according to program instructions in firmware, memory, other storage, or a combination thereof. Such special-purpose computing devices may also incorporate customized hardwired logic, ASICs, or FPGAs through custom programming to implement these techniques. The special-purpose computing devices may be desktop computer systems, portable computer systems, handheld devices, networked devices, or any other device incorporating hardwired and/or program logic to implement these techniques.

For example, Figure 9 is a block diagram illustrating a computer system 900 on which an example embodiment of the present invention may be implemented. The computer system 900 includes a bus 902 or other communication mechanism for transmitting information, and a hardware processor 904 coupled to the bus 902 for processing information. The hardware processor 904 may be, for example, a general-purpose microprocessor.

Computer system 900 also includes a main memory 906, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 902 for storing information and instructions to be executed by processor 904. Main memory 906 may also be used to store temporary variables or other intermediate information during execution of instructions by processor 904. Such instructions, when stored on a non-transitory storage medium accessible to processor 904, transform computer system 900 into a special-purpose machine customized to perform the operations specified in the instructions.

Computer system 900 also includes a read-only memory (ROM) 908 or other static storage device coupled to bus 902 for storing static information and instructions for processor 904. A storage device 910, such as a magnetic or optical disk, is provided and coupled to bus 902 for storing information and instructions.

The computing system 900 may be coupled to a display 912, such as a liquid crystal display, via bus 902 for displaying information to a computer user. An input device 914, including alphanumeric and other keys, is coupled to bus 902 for communicating information and command selections to processor 904. Another type of user input device is a cursor control 916, such as a mouse, trackball, or cursor direction keys, for communicating directional information and command selections to processor 904 and for controlling the movement of a cursor on display 912. This input device typically has two degrees of freedom along two axes (a first axis (e.g., x) and a second axis (e.g., y)), which allows the device to specify a position in a plane.

Computer system 900 may implement the techniques described herein using custom hardwired logic, one or more ASICs or FPGAs, firmware, and/or program logic (which, in combination with the computer system, transforms computer system 900 into a special-purpose machine). According to one embodiment, the techniques described herein are performed by computer system 900 in response to processor 904 executing one or more sequences of one or more instructions contained in main memory 906. Such instructions may be read into main memory 906 from another storage medium, such as storage device 910. Execution of the sequences of instructions contained in main memory 906 causes processor 904 to perform the process steps described herein. In alternative embodiments, hardwired circuitry may be used in place of, or in combination with, software instructions.

As used herein, the term "storage media" refers to any non-transitory medium that stores data and/or instructions that cause a machine to operate in a specific manner. Such storage media may include non-volatile media and/or volatile media. Non-volatile media include, for example, optical or magnetic disks, such as storage device 910. Volatile media include dynamic memory, such as main memory 906. Common forms of storage media include, for example, floppy disks, flexible disks, hard disks, solid-state drives, magnetic tape or any other magnetic data storage medium, CD-ROMs, any other optical data storage medium, any physical medium with a pattern of holes, RAM, PROM and EPROM, FLASH-EPROM, NVRAM, or any other memory chip or cartridge.

Storage media are distinct from, but may be used in combination with, transmission media. Transmission media are involved in the transfer of information between storage media. For example, transmission media include coaxial cables, copper wire, and optical fiber, including the wires that comprise bus 902. Transmission media can also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.

Various forms of media may be involved in carrying one or more sequences of one or more instructions to processor 902 for execution. For example, the instructions may initially be carried on a magnetic disk or solid-state drive of a remote computer. The remote computer may load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 900 may receive the data on the telephone line and use an infrared transmitter to convert the data into an infrared signal. An infrared detector may receive the data carried in the infrared signal and appropriate circuitry may place the data on bus 902. Bus 902 carries the data to main memory 906, from which processor 904 retrieves and executes the instructions. The instructions received by main memory 906 may optionally be stored on storage device 910 before or after execution by processor 904.

Computer system 900 also includes a communication interface 918 coupled to bus 902. Communication interface 918 provides bidirectional data communication coupled to network link 920, which is connected to local area network 922. For example, communication interface 918 can be an integrated services digital network (ISDN) card, a cable modem, a satellite modem, or a modem that provides a data communication connection with a corresponding type of telephone line. As another example, communication interface 918 can be a local area network (LAN) card that provides a data communication connection with a compatible LAN. Wireless links can also be implemented. In any such implementation, communication interface 918 sends and receives electrical signals, electromagnetic signals, or optical signals that carry digital data streams representing various types of information.

Network link 920 typically provides data communication with other data devices through one or more networks. For example, network link 920 may provide a connection to a host computer 924 or data equipment operated by an Internet service provider (ISP) 926 via a local area network 922. ISP 926, in turn, provides data communication services through a wide-area packet data communication network, commonly referred to as the "Internet" 928. Both local area network 922 and Internet 928 use electrical, electromagnetic, or optical signals to carry digital data streams. The signals passing through the various networks and the signals on network link 920 through communication interface 918 are exemplary forms of transmission media that carry digital data to and from computer system 900.

Computer system 900 can send messages and receive data (including program code) through one or more networks, network link 920, and communication interface 918. In the Internet example, server 930 can send requested application code through Internet 928, ISP 926, local area network 922, and communication interface 918.

The received code may be executed as it is received by processor 904 and/or stored in storage device 910 or other non-volatile storage for later execution.

12. Enumerate example embodiments, equivalents, extensions, variations and others

The enumerated example embodiments ("EEE") of the present invention have been described above with respect to image data exchange based on perceived luminance nonlinearity between displays of different capabilities. Therefore, embodiments of the present invention may involve one or more of the examples enumerated in Table 2 below.

Table 2. Enumerated example embodiments

Table 3 below describes the calculation of the perceptual curve EOTF used to convert digital video code values to absolute linear luminance levels at the display point. Also included is the inverse OETF calculation used to convert absolute linear luminance to digital code values.

Table 3. Example specifications for perceptual curve EOTFs

Table 4 below shows exemplary values of 10 bits.

Table 4. Example table of 10-bit values

In the foregoing specification, embodiments of the present invention have been described with reference to numerous specific details that may vary from implementation to implementation. Therefore, the only indication of what is the invention, and what the applicants intend to be the sole and exclusive indication of the invention, is the set of claims, including any subsequent amendments, that issue from this application in the specific form in which such claims issue. Any definitions expressly set forth herein for terms contained in such claims shall govern the meaning of such terms as used in the claims. Accordingly, no limitation, element, property, feature, advantage, or attribute that is not expressly recited in a claim should in any way limit the scope of such claims. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

Claims (14)

1. An image processing method, comprising: The digital code value D of the perception curve of the received image; The normalized sensing curve signal value V is determined from the sensing curve digital code value D; The normalized color component value Y is determined based on the following function model. as well as The absolute value L is determined from the normalized color component value Y, where L = 10000·Y. Wherein, the parameters n, m, _c1 , _c2 , and _c3 are the following predetermined values, as well as 2. The method according to claim 1, wherein the digital code value D of the perception curve is 10 bits. 3. The method according to claim 1, wherein the digital code value D of the perception curve is 12 bits. 4. The method according to claim 1, wherein the relationship between the normalized sensing curve signal value V and the sensing curve digital code value D is provided by the following function: Where b is the bit depth corresponding to the number of bits used to represent the digital code value D of the perception curve. 5. The method according to claim 1, wherein the sensing curve digital code value D is received from a non-transitory storage medium. 6. The method according to claim 1, wherein the normalized color component value Y is a red (R), green (G), or blue (B) component. 7. An image processing method, comprising: The digital code value D of the perception curve of the received image; The normalized sensing curve signal value V is determined from the sensing curve digital code value D; The normalized illuminance value Y is determined based on the following function model. as well as The absolute value L is determined from the normalized illuminance value Y, where L = 10000·Y. Wherein, the parameters n, m, _c1 , _c2 , and _c3 are the following predetermined values, as well as 8. The method according to claim 7, wherein the digital code value D of the perception curve is 10 bits. 9. The method according to claim 7, wherein the digital code value D of the perception curve is 12 bits. 10. The method of claim 7, wherein the relationship between the normalized sensing curve signal value V and the sensing curve digital code value D is provided by the following function: Where b is the bit depth corresponding to the number of bits used to represent the digital code value D of the perception curve. 11. The method of claim 7, wherein the sensing curve digital code value D is received from a non-transitory storage medium. 12. An apparatus comprising: One or more processors, and One or more storage media, storing instructions that, when executed by the one or more processors, cause the method according to any one of claims 1-11 to be performed. 13. An apparatus comprising components for performing the method according to any one of claims 1-11. 14. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause to perform the method according to any one of claims 1-11.