HK40020459B - Video coding with content adaptive spatially varying quantization - Google Patents

Description

Video decoding using content-adaptive spatial variation quantization

This application claims the benefit of U.S. Provisional Application No. 62/571,732, filed October 12, 2017, and U.S. Application No. 16/155,344, filed October 9, 2018, the entire contents of which are incorporated herein by reference.

Technical Field

This invention relates to video decoding and/or video processing.

Background Technology

Digital video capabilities can be incorporated into a wide range of devices, including digital television, digital live broadcast systems, wireless broadcasting systems, personal digital assistants (PDAs), laptops or desktop computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite radio phones (so-called "smartphones"), video conferencing devices, video streaming devices, and the like. Digital video devices implement video decoding technologies, such as those described in the standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264/MPEG-4, Advanced Video Decoding (AVC) Part 10, ITU-T H.265, High Efficiency Video Decoding (HEVC), and extensions to these standards. Video devices can transmit, receive, encode, decode, and/or store digital video information more efficiently by implementing such video decoding technologies.

Video decoding techniques involve spatial (intra-picture) prediction and/or temporal (inter-picture) prediction to reduce or remove inherent redundancy in video sequences. For block-based video decoding, video slices (e.g., video frames or portions of video frames) can be segmented into video blocks (which may also be called tree blocks), decoding units (CUs), and/or decoding nodes. Video blocks in intra-frame decoded (I) slices of a picture are encoded using spatial predictions about reference samples in adjacent blocks within the same picture. Video blocks in inter-frame decoded (P or B) slices of a picture can use spatial predictions about reference samples in adjacent blocks within the same picture or temporal predictions about reference samples in other reference pictures. A picture may be called a frame, and a reference picture may be called a reference frame.

Spatial or temporal prediction produces predictive blocks for the block to be decoded. Residual data represents the pixel difference between the original block and the predictive block. Inter-frame decoded blocks are encoded based on the motion vector of the block pointing to the reference sample forming the predictive block and the residual data indicating the difference between the decoded block and the predictive block. Intra-frame decoded blocks are encoded according to the intra-frame decoding mode and the residual data. For further compression, the residual data can be transformed from the pixel domain to the transform domain, resulting in residual transform coefficients that can then be quantized. The quantized transform coefficients, initially arranged in a two-dimensional array, can be scanned to produce a one-dimensional vector of transform coefficients, and entropy decoding can be applied to achieve even greater compression.

A color gamut can be defined as the total number of color values that can be captured, decoded, and displayed. A color gamut refers to the range of colors a device can capture (e.g., a camera) or reproduce (e.g., a display). Often, color gamuts differ between devices. For video decoding, a predefined color gamut of the video data can be used so that each device in the video decoding process can be configured to process pixel values within the same color gamut. Some color gamuts are defined using a larger range of colors than those traditionally used for video decoding. These color gamuts with a larger range of colors are called wide color gamuts (WCG).

Another aspect of video data is dynamic range. Dynamic range is typically defined as the ratio between the maximum and minimum luminance (e.g., illumination) of a video signal. The dynamic range of commonly used video data in the past is considered to have a standard dynamic range (SDR). Other instances of video data specifications define color data with a large ratio of maximum to minimum luminance. This video data can be described as having a high dynamic range (HDR).

Summary of the Invention

This invention describes an example processing method (and apparatus configured to perform the method) for a decoding (e.g., encoding or decoding) loop applied to a video decoding system. The technology of this invention is applicable to decoding video data representations with perceived discernible differences (e.g., signal-to-noise ratio) across a non-uniform distribution of video data within the dynamic range of the video data. A video encoder can be configured to apply a multi-stage quantization process, wherein the residual is first quantized using effective quantization parameters derived statistically from samples of the block. The residual is then further quantized using a base quantization parameter uniform across the image. A video decoder can be configured to decode the video data using the base quantization parameters. The video decoder can be further configured to estimate the effective quantization parameters from statistically derived samples of the decoded block. The video decoder can then use the estimated effective quantization parameters to determine parameters for other decoding tools (including filters). In this way, signaling overhead is preserved when the effective quantization parameters are not represented as signals, but are estimated at the decoder side.

In one example, the present invention describes a method for decoding video data, the method comprising: receiving an encoded block of video data, the encoded block of video data having been encoded using effective quantization parameters and basic quantization parameters, wherein the effective quantization parameters are a function of quantization parameter offsets added to the basic quantization parameters; determining basic quantization parameters for encoding the encoded block of video data; decoding the encoded block of video data using the basic quantization parameters to establish a decoded block of video data; determining an estimate of the quantization parameter offset for the decoded block of video data based on statistics associated with the decoded block of video data; adding the estimate of the quantization parameter offset to the basic quantization parameters to establish an estimate of the effective quantization parameters; and performing one or more filtering operations on the decoded block of video data according to the estimate of the effective quantization parameters.

In another example, the present invention describes a method for encoding video data, the method comprising: determining a base quantization parameter for a video data block; determining a quantization parameter offset for the video data block based on statistics associated with the video data block; adding the quantization parameter offset to the base quantization parameter to establish an effective quantization parameter; and encoding the video data block using the effective quantization parameter and the base quantization parameter.

In another example, the present invention describes an apparatus configured to decode video data, the apparatus including a memory configured to store encoded blocks of video data, and one or more processors in communication with the memory, the processors being configured to: receive encoded blocks of video data; the encoded blocks of video data being encoded using effective quantization parameters and basic quantization parameters, wherein the effective quantization parameters are a function of quantization parameter offsets added to the basic quantization parameters; determine basic quantization parameters for encoding the encoded blocks of video data; decode the encoded blocks of video data using the basic quantization parameters to establish decoded blocks of video data; determine an estimate of the quantization parameter offset for the decoded blocks of video data based on statistics associated with the decoded blocks of video data; add the estimate of the quantization parameter offset to the basic quantization parameters to establish an estimate of the effective quantization parameters; and perform one or more filtering operations on the decoded blocks of video data according to the estimate of the effective quantization parameters.

In another example, the present invention describes an apparatus configured to encode video data, the apparatus including a memory configured to store blocks of video data, and one or more processors in communication with the memory, the one or more processors being configured to: determine a base quantization parameter for the video data blocks; determine a quantization parameter offset for the video data blocks based on statistics associated with the video data blocks; add the quantization parameter offset to the base quantization parameter to establish a valid quantization parameter; and encode the video data blocks using the valid quantization parameter and the base quantization parameter.

In another example, the present invention describes an apparatus configured to decode video data, the apparatus comprising: means for receiving encoded blocks of video data, the encoded blocks of video data having been encoded using effective quantization parameters and basic quantization parameters, wherein the effective quantization parameters are a function of quantization parameter offsets added to the basic quantization parameters; means for determining the basic quantization parameters for encoding the encoded blocks of video data; means for decoding the encoded blocks of video data using the basic quantization parameters to construct decoded blocks of video data; means for determining an estimate of the quantization parameter offset for the decoded blocks of video data based on statistics associated with the decoded blocks of video data; means for adding the estimate of the quantization parameter offset to the basic quantization parameters to construct an estimate of the effective quantization parameters; and means for performing one or more filtering operations on the decoded blocks of video data based on the estimate of the effective quantization parameters.

In another example, the present invention describes an apparatus configured to encode video data, the apparatus comprising: means for determining a base quantization parameter for a video data block; means for determining a quantization parameter offset for the video data block based on statistics associated with the video data block; means for adding the quantization parameter offset to the base quantization parameter to establish a valid quantization parameter; and means for encoding the video data block using the valid quantization parameter and the base quantization parameter.

In another example, the present invention describes a non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors to: receive an encoded block of video data, the encoded block of video data having been encoded using effective quantization parameters and basic quantization parameters, wherein the effective quantization parameters are a function of quantization parameter offsets added to the basic quantization parameters; determine the basic quantization parameters for encoding the encoded block of video data; decode the encoded block of video data using the basic quantization parameters to establish a decoded block of video data; determine an estimate of the quantization parameter offset for the decoded block of video data based on statistics associated with the decoded block of video data; add the estimate of the quantization parameter offset to the basic quantization parameters to establish an estimate of the effective quantization parameters; and perform one or more filtering operations on the decoded block of video data according to the estimate of the effective quantization parameters.

In another example, the present invention describes a non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors to: determine a base quantization parameter for a video data block; determine a quantization parameter offset for the video data block based on statistics associated with the video data block; add the quantization parameter offset to the base quantization parameter to establish a valid quantization parameter; and encode the video data block using the valid quantization parameter and the base quantization parameter.

Details of one or more examples are set forth in the accompanying drawings and description. Other features, objects, and advantages will be apparent from the description, drawings, and claims.

Attached Figure Description

Figure 1 is a block diagram illustrating an example video encoding and decoding system configured to implement the technology of the present invention.

Figures 2A and 2B are conceptual diagrams illustrating the structure of the Quad-Tab Tree (QTBT) and its corresponding decoding tree unit (CTU).

Figure 3 is a conceptual diagram illustrating the concept of HDR data.

Figure 4 is a conceptual diagram illustrating the example color gamut.

Figure 5 is a flowchart illustrating an example of HDR/WCG representation conversion.

Figure 6 is a flowchart illustrating an example of HDR/WCG inverse conversion.

Figure 7 is a conceptual diagram illustrating an example of an electro-optical transfer function (EOTF) used for video data conversion (including SDR and HDR) from perceived uniform code level to linear illumination.

Figure 8 is a block diagram illustrating an example of a video encoder from which the technology of the present invention can be implemented.

Figure 9 is a block diagram illustrating an example quantization unit of a video encoder that can implement the technology of the present invention.

Figure 10 is a block diagram illustrating an example of a video decoder from which the technology of the present invention can be implemented.

Figure 11 is a flowchart illustrating the instance encoding method.

Figure 12 is a flowchart illustrating the instance decoding method.

Detailed Implementation

This invention relates to the processing and/or decoding of video data with High Dynamic Range (HDR) and Wide Color Gamut (WCG) representations. More specifically, the techniques of this invention involve content-adaptive spatial variation quantization to efficiently compress HDR/WCG video signals without explicit signaling of quantization parameters (e.g., variations in quantization parameters represented by δQP syntax elements). The techniques and apparatus described herein can improve the compression efficiency of video decoding systems used for decoding HDR and WCG video data. The techniques of this invention can be used in the context of advanced video codecs, such as extensions to HEVC or next-generation video decoding standards.

Video decoding standards that include hybrid video decoding standards include ITU-T H.261, ISO/IEC MPEG-1 Visual, ITU-T H.262 or ISO/IEC MPEG-2 Visual, ITU-T H.263, ISO/IEC MPEG-4 Visual, and ITU-T H.264 (also known as ISO/IEC MPEG-4 AVC), which include their Adjustable Video Decoding (SVC) and Multi-View Video Decoding (MVC) extensions. The design of a new video decoding standard (i.e., High Efficiency Video Decoding (HEVC, also known as H.265)) has been finalized by the Joint Collaborating Group on Video Decoding (JCT-VC) of the ITU-T Video Decoding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG). The HEVC draft specification, known as Working Draft 10 (WD10) by Bross et al., entitled "High efficiency video coding (HEVC) text specification draft 10 (for FDIS & Last Call)", was presented at the 12th meeting of the Joint Working Group on Video Decoding (JCT-VC) of ITU-T SG16WP3 and ISO/IEC JTC1 /SC29/WG11, Geneva, Switzerland, January 14-23, 2013, JCTVC-L1003v34. It is available at http://phenix.int-evry.fr/jct/doc_end_user/documents/12_Geneva/wg11/JCTVC-L1003-v34.zip . The finalized HEVC standard is called HEVC Version 1. The finalized HEVC standard document was published in April 2013 as "ITU-T..." The H.265H series refers to the infrastructure of audiovisual and multimedia systems and services—specifically, the decoding of moving video, high-efficiency video coding, and is part of the Telecommunication Standardization Sector of the International Telecommunication Union (ITU). Another version of the finalized HEVC standard was published in October 2014. A copy of the H.265/HEVC specification text can be downloaded from http://www.itu.int/rec/T-REC-H.265-201504-I/en .

ITU-T VCEG (Q6/16) and ISO/IEC MPEG (JTC 1/SC 29/WG 11) are currently investigating the potential need to standardize future video decoding technologies with compression capabilities significantly exceeding those of the current HEVC standard (including its current extensions and recent extensions for screen content decoding and high dynamic range decoding). These groups are working together on this exploratory activity within a joint collaborative effort known as the Joint Video Exploration Group (JVET) to evaluate compression technology designs proposed by experts in this field. The JVET first met between October 19 and 21, 2015. The latest version of the reference software, Joint Exploration Model 7 ( JEM7 ), can be downloaded from https://jvet.hhi.fraunhofer.de/svn/svn_HMJEMSoftware/tags/HM-16.6-JEM-7.0/ . The algorithm description of JEM7 can be found in "Algorithm description of Joint Exploration TestModel 7 (JEM7)" by J. Chen, E. Alshina, G. J. Sullivan, J.-R. Ohm, and J. Boyce (JVET-C1001, Torino, July 2017).

Recently, a new video decoding standard, known as the Versatile Video Coding (VVC) standard, is under development by the Joint Video Experts Group (JVET) of VCEG and MPEG. An early draft of VVC is available in document JVET-J1001 "Versatile Video Coding (Draft 1)" and its algorithm description is available in document JVET-J1002 "Algorithm description for Versatile Video Coding and Test Model 1 (VTM 1)".

Figure 1 is a block diagram illustrating an example video encoding and decoding system 10 from which the technology of the present invention can be utilized. As shown in Figure 1, system 10 includes a source device 12 that provides encoded video data to a destination device 14 for later decoding. Specifically, the source device 12 provides the video data to the destination device 14 via a computer-readable medium 16. The source device 12 and the destination device 14 may include any of a wide range of devices, including desktop computers, notebook computers, tablet computers, set-top boxes, mobile phones (such as so-called "smart" phones), so-called "smart" mats, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, or the like. In some cases, the source device 12 and the destination device 14 may be equipped for wireless communication.

Destination device 14 may receive encoded video data to be decoded via computer-readable medium 16. Computer-readable medium 16 may include any type of media or device capable of moving encoded video data from source device 12 to destination device 14. In one example, computer-readable medium 16 may include communication media enabling source device 12 to transmit encoded video data directly to destination device 14 in real time. The encoded video data may be modulated according to communication standards (such as wired or wireless communication protocols) and transmitted to destination device 14. Communication media may include any wireless or wired communication media, such as radio frequency (RF) spectrum or one or more physical transmission lines. Communication media may form part of a packet-based network (such as a local area network, wide area network, or global area network such as the Internet). Communication media may include routers, switches, base stations, or any other devices that can be used to facilitate communication from source device 12 to destination device 14.

In other instances, computer-readable media 16 may comprise non-transitory storage media, such as hard disks, flash drives, optical discs, digital video discs, Blu-ray discs, or other computer-readable media. In some instances, a network server (not shown) may receive encoded video data from source device 12 and, for example, provide the encoded video data to destination device 14 via network transmission. Similarly, a computing device in a media production facility, such as an optical disc stamping facility, may receive encoded video data from source device 12 and produce an optical disc containing the encoded video data. Therefore, in various instances, computer-readable media 16 can be understood as comprising one or more computer-readable media of various forms.

In some instances, encoded data can be output to a storage device from output interface 22. Similarly, encoded data can be accessed from a storage device via an input interface. The storage device may comprise any of a variety of distributed or locally accessible data storage media, such as hard disk drives, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage media for storing encoded video data. In yet another instance, the storage device may correspond to a file server or another intermediate storage device capable of storing the encoded video generated by source device 12. Destination device 14 can access the stored video data from the storage device via streaming or downloading. The file server can be any type of server capable of storing and transmitting encoded video data to destination device 14. Example file servers include web servers (e.g., for websites), FTP servers, network attached storage (NAS) devices, or local disk drives. Destination device 14 can access the encoded video data via any standard data connection, including an Internet connection. It may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., DSL, cable modem, etc.), or a combination of both suitable for accessing encoded video data stored on a file server. The transmission of encoded video data from the storage device may be streaming, downloading, or a combination thereof.

The technology of this invention is not limited to wireless applications or setups. It can be applied to video decoding supporting any of a variety of multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, internet streaming video transmission (e.g., via HTTP Dynamic Adaptive Streaming (DASH)), digital video encoded to data storage media, decoding digital video stored on data storage media, or other applications. In some instances, system 10 can be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting, and/or video telephony.

In the example of Figure 1, source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Destination device 14 includes an input interface 28, a dynamic range adjustment (DRA) unit 19, a video decoder 30, and a display device 32. According to the invention, the DRA unit 19 of source device 12 can be configured to implement the techniques of the invention, including signaling and related operations applied to video data in a specific color space to achieve more efficient compression of HDR and WCG video data. In some instances, the DRA unit 19 can be separate from the video encoder 20. In other instances, the DRA unit 19 can be part of the video encoder 20. In other instances, the source and destination devices can include other components or arrangements. For example, source device 12 can receive video data from an external video source 18 (such as an external camera). Similarly, destination device 14 can interface with an external display device, rather than including an integrated display device.

The system 10 illustrated in Figure 1 is only one example. The techniques for processing and decoding HDR and WCG video data can be performed by any digital video encoding and/or video decoding device. Furthermore, some examples of the techniques of this invention can also be performed by a video preprocessor and/or a video postprocessor. A video preprocessor can be any device configured to process video data before encoding (e.g., before HEVC, VVC, or other encoding). A video postprocessor can be any device configured to process video data after decoding (e.g., after HEVC, VVC, or other decoding). Source device 12 and destination device 14 are merely examples of such decoding devices where source device 12 generates decoded video data for transmission to destination device 14. In some instances, devices 12, 14 can operate in a substantially symmetrical manner, such that each of devices 12, 14 includes video encoding and decoding components, as well as a video preprocessor and a video postprocessor (e.g., DRA unit 19 and anti-DRA unit 31, respectively). Therefore, system 10 can support one-way or two-way video transmission between video devices 12, 14 for purposes such as video streaming, video playback, video broadcasting, or video telephony.

The video source 18 of the source device 12 may include a video capture device, such as a video camera, a video archive containing previously captured video, and/or a video feed interface for receiving video from a video content provider. Alternatively, the video source 18 may generate computer graphics-based data as source video, or a combination of live video, archived video, and computer-generated video. In some cases, if the video source 18 is a video camera, then the source device 12 and the destination device 14 may form a so-called camera phone or video phone. However, as mentioned above, the techniques described in this invention are applicable to video decoding and video processing, and generally, can be applied to wireless and/or wired applications. In each case, the captured, pre-captured, or computer-generated video may be encoded by the video encoder 20. The encoded video information may then be output from the output interface 22 to the computer-readable medium 16.

The input interface 28 of the destination device 14 receives information from the computer-readable medium 16. The information on the computer-readable medium 16 may include grammatical information defined by the video encoder 20 (which is also used by the video decoder 30), the grammatical information including descriptive blocks and other characteristics and/or processed grammatical elements of the decoding unit (e.g., group of pictures (GOP)). The display device 32 displays the decoded video data to the user and may include any of a variety of display devices, such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) display, or another type of display device.

The video encoder 20 and video decoder 30 can each be implemented as any of a variety of suitable encoder or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the technical portion is implemented in software, the device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the technology of the present invention. Each of the video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, and any of the encoders or decoders may be integrated as part of a combined encoder/decoder (codec) in the respective device.

DRA unit 19 and anti-DRA unit 31 can each be implemented as any of a variety of suitable encoder circuits, such as one or more microprocessors, DSPs, ASICs, FPGAs, discrete logic, software, hardware, firmware, or any combination thereof. When the technical portion is implemented in software, the device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the technology of the present invention.

In some instances, the video encoder 20 and the video decoder 30 operate according to video compression standards such as ITU-TH.265/HEVC, VVC, or other next-generation video decoding standards.

In HEVC and other video decoding standards, video sequences typically contain a series of images. Images are also referred to as "frames." An image may contain three sample arrays, denoted as _SL , _SCb , and _SCr . _SL is a two-dimensional array (i.e., a block) of luminance samples. _SCb is a two-dimensional array of Cb chroma samples. _SCr is a two-dimensional array of Cr chroma samples. Chroma samples may also be referred to as "chroma" samples in this document. In other cases, images may be monochrome and may contain only luminance sample arrays.

The video encoder 20 can generate a set of decoding tree units (CTUs). Each CTU may include a decoding tree block for luminance samples, two corresponding decoding tree blocks for chrominance samples, and a syntax structure for the samples used to decode the decoding tree block. In a monochrome image or an image with three separate color planes, a CTU may include a single decoding tree block and a syntax structure for the samples used to decode the decoding tree block. The decoding tree block may be an N×N block of samples. A CTU may also be referred to as a "tree block" or a "maximum decoding unit" (LCU). HEVC's CTUs can be broadly similar to macroblocks in other video decoding standards such as H.264/AVC. However, a CTU is not necessarily limited to a specific size and may contain one or more decoding units (CUs). A slice may contain an integer number of CTUs ordered consecutively in a raster scan.

This invention may use the terms "video unit" or "video block" to refer to one or more blocks of a sample, and the grammatical structure of the sample used to decode one or more blocks of the sample. Instance types of video units may include CTUs, CUs, PUs, transform units (TUs) in HEVC, or macroblocks, macroblock partitions, etc., in other video decoding standards.

To generate a decoded CTU, the video encoder 20 can recursively perform quartic tree partitioning on the decoded tree block of the CTU to divide the decoded tree block into decoded blocks, hence the name "decoded tree unit". A decoded block is an N×N block of samples. A CU may include a decoded block of luminance samples and two corresponding decoded blocks of chrominance samples of an image having luminance sample arrays, Cb sample arrays, and Cr sample arrays, as well as a syntax structure of samples for decoding said decoded blocks. In a monochrome image or an image with three separate color planes, the CU may include a single decoded block and a syntax structure of samples for decoding said decoded block.

The video encoder 20 can segment the decoding blocks of the CU into one or more prediction blocks. A prediction block can be a rectangular (i.e., square or non-square) block of samples to which the same prediction is applied. The prediction unit (PU) of the CU can include prediction blocks of luminance samples of the image, two corresponding prediction blocks of chrominance samples, and a syntax structure for predicting the prediction block samples. In a monochrome image or an image with three separate color planes, the PU can include a single prediction block and a syntax structure for predicting the prediction block samples. The video encoder 20 can generate predictive luminance, Cb, and Cr blocks for each PU of the CU.

In JEM7, a Quadtree Binary Tree (QTBT) partitioning structure can be used instead of the HEVC quadtree partitioning structure described above. The QTBT structure removes the concept of multiple partition types. That is, the QTBT structure removes the separation of CU, PU, and TU concepts and supports greater flexibility in the shape of the CU partition. In a QTBT block structure, the CU can have a square or rectangular shape. In one example, the CU is the first partition according to the quadtree structure. The leaf nodes of the quadtrees are further partitioned using a binary tree structure.

In some instances, two splitting types exist: symmetrical horizontal splitting and symmetrical vertical splitting. Binary leaf nodes are called CUs, and the segments (i.e., CUs) are used for prediction and transform processing without any further segmentation. This means that CUs, PUs, and TUs have the same block size in the QTBT decoding block structure. In JEM, CUs sometimes consist of decoding blocks (CBs) with different color components. For example, in the case of P and B slices in a 4:2:0 chroma format, a CU contains one luminance CB and two chroma CBs, and sometimes it consists of CBs with a single component. For example, in the case of I slices, a CU contains only one luminance CB or only two chroma CBs.

In some instances, the video encoder 20 and video decoder 30 can be configured to operate according to the JEM/VVC standard. According to JEM/VVC, the video decoder (such as the video encoder 20) segments the image into multiple CUs. An example QTBT structure of JEM contains two levels: a first level of segmentation based on a quartic tree, and a second level of segmentation based on a binary tree. The root node of the QTBT structure corresponds to a CTU. The leaf nodes of the binary tree correspond to decoding units (CUs).

In some instances, the video encoder 20 and the video decoder 30 may use a single QTBT structure to represent each of the illuminance and chroma components, while in other instances, the video encoder 20 and the video decoder 30 may use two or more QTBT structures, such as one QTBT structure for the illuminance component and another QTBT structure for the two chroma components (or two QTBT structures for the respective chroma components).

The video encoder 20 and video decoder 30 can be configured to use quadtree segmentation according to HEVC, QTBT segmentation according to JEM/VVC, or other segmentation structures. For illustrative purposes, the description of the present invention is presented with respect to QTBT segmentation. However, it should be understood that the technology of the present invention can also be applied to video decoders configured to use quadtree segmentation or other types of segmentation.

Figures 2A and 2B are conceptual diagrams illustrating the Quadtree Binary Tree (QTBT) structure 130 and the corresponding decoding tree unit (CTU) 132. Solid lines represent quadtree splits, and dotted lines indicate binary tree splits. In each split (i.e., non-leaf) node of the binary tree, a flag is signaled to indicate which split type (i.e., horizontal or vertical) is used, where in this example, 0 indicates a horizontal split and 1 indicates a vertical split. For quadtree splits, there is no need to indicate the split type because the quadtree node splits the block horizontally and vertically into four sub-blocks of equal size. Therefore, the video encoder 20 can encode, and the video decoder 30 can decode, syntax elements (such as split information) for the region tree level (i.e., solid lines) of the QTBT structure 130 and syntax elements (such as split information) for the prediction tree level (i.e., dashed lines) of the QTBT structure 130. The video encoder 20 can encode, and the video decoder 30 can decode, video data (such as prediction and transform data) for the CU represented by the end-leaf nodes of the QTBT structure 130.

Generally, the CTU 132 in Figure 2B can be associated with parameters that define the size of the blocks corresponding to the nodes of the QTBT structure 130 at the first and second levels. These parameters may include the CTU size (representing the size of the CTU 132 in the sample), the minimum quartic tree size (MinQTSize, representing the minimum allowed quartic tree leaf node size), the maximum binary tree size (MaxBTSize, representing the maximum allowed binary tree root node size), the maximum binary tree depth (MaxBTDepth, representing the maximum allowed binary tree depth), and the minimum binary tree size (MinBTSize, representing the minimum allowed binary tree leaf node size).

The root node of a QTBT structure corresponding to a CTU can have four child nodes at the first level of the QTBT structure, each of which can be partitioned according to a quartic tree. That is, a node at the first level is a leaf node (without child nodes) or has four child nodes. An example of QTBT structure 130 represents a node such as one containing a parent node and child nodes with solid lines for branching. If a node at the first level is not larger than the maximum allowed binary tree root node size (MaxBTSize), then it can be further partitioned by the corresponding binary tree. A binary tree split of a node can be repeated until the node produced by the split reaches the minimum allowed binary tree leaf node size (MinBTSize) or the maximum allowed binary tree depth (MaxBTDepth). An example of QTBT structure 130 represents a node such as one with dashed lines for branching. The binary tree leaf node is called a decoding unit (CU), which is used for prediction (e.g., intra-picture or inter-picture prediction) and transformation without any further partitioning. As discussed above, a CU can also be referred to as a "video block" or "block".

In an example of a QTBT segmentation structure, the CTU size is set to 128×128 (lightness sample and two corresponding 64×64 chroma samples), the MinQTSize is set to 16×16, the MaxBTSize is set to 64×64, the MinBTSize (for both width and height) is set to 4, and the MaxBTDepth is set to 4. Quarter tree segmentation is first applied to the CTU to generate quarter leaf nodes. Quarter leaf nodes can have sizes ranging from 16×16 (i.e., MinQTSize) to 128×128 (i.e., CTU size). If a leaf quarter node is 128×128, then the node will not be further split by the binary tree because its size exceeds MaxBTSize (i.e., 64×64 in this example). Otherwise, the leaf quarter node will be further segmented by the binary tree. Therefore, the quarter leaf node is also the root node of the binary tree and has a binary tree depth of 0. When the depth of a binary tree reaches MaxBTDepth (4 in this example), further splitting is not permitted. When a binary tree node has a width equal to MinBTSize (4 in this example), it means that further horizontal splitting is not permitted. Similarly, a binary tree node with a height equal to MinBTSize means that further vertical splitting is not permitted for that binary tree node. As mentioned above, the leaf nodes of the binary tree are called CUs and are further processed according to predictions and transformations without further splitting.

Video encoder 20 can use intra-frame prediction or inter-frame prediction to generate predictive blocks of the PU. If video encoder 20 uses intra-frame prediction to generate predictive blocks of the PU, then video encoder 20 can generate predictive blocks of the PU based on decoded samples of the image associated with the PU.

If the video encoder 20 uses inter-frame prediction to generate predictive blocks of the PU, then the video encoder 20 can generate predictive blocks of the PU based on decoded samples of one or more images different from the images associated with the PU. Inter-frame prediction can be unidirectional inter-frame prediction (i.e., one-way prediction) or bidirectional inter-frame prediction (i.e., two-way prediction). To perform unidirectional or bidirectional prediction, the video encoder 20 can generate a first reference image list (RefPicList0) and a second reference image list (RefPicList1) for the current slice.

Each of the reference image lists may contain one or more reference images. When using unidirectional prediction, the video encoder 20 may search for reference images in either or both of RefPicList0 and RefPicList1 to determine a reference position within the reference image. Furthermore, when using unidirectional prediction, the video encoder 20 may generate predictive sample blocks of the PU based at least partially on samples corresponding to the reference positions. Additionally, when using unidirectional prediction, the video encoder 20 may generate a single motion vector indicating the spatial displacement between the predicted block of the PU and the reference position. To indicate the spatial displacement between the predicted block of the PU and the reference position, the motion vector may include a horizontal component of the horizontal displacement between the predicted block of the PU and the reference position, and may include a vertical component of the vertical displacement between the predicted block of the PU and the reference position.

When using bidirectional predictive coding (PU), the video encoder 20 can determine a first reference position in a reference image in RefPicList0 and a second reference position in a reference image in RefPicList1. The video encoder 20 can then generate predictive blocks of the PU based at least in part on samples corresponding to the first and second reference positions. Furthermore, when using bidirectional predictive coding (PU), the video encoder 20 can generate a first motion indicating the spatial displacement between the sample block of the PU and the first reference position, and a second motion indicating the spatial displacement between the predictive block of the PU and the second reference position.

In some instances, JEM/VVC also provides an affine motion compensation mode, which can be viewed as an inter-frame prediction mode. In affine motion compensation mode, the video encoder 20 can determine two or more motion vectors representing non-translational motion (such as zooming in or out, rotation, perspective motion, or other irregular motion types).

After the video encoder 20 generates predictive luminance blocks, predictive Cb blocks, and predictive Cr blocks for one or more PUs of the CU, the video encoder 20 can generate luminance residual blocks for the CU. Each sample in the luminance residual block of the CU indicates the difference between a luminance sample in one of the predictive luminance blocks of the CU and a corresponding sample in the original luminance decoded block of the CU. Additionally, the video encoder 20 can generate Cb residual blocks for the CU. Each sample in the Cb residual block of the CU indicates the difference between a Cb sample in one of the predictive Cb blocks of the CU and a corresponding sample in the original Cb decoded block of the CU. The video encoder 20 can also generate Cr residual blocks for the CU. Each sample in the Cr residual block of the CU indicates the difference between a Cr sample in one of the predictive Cr blocks of the CU and a corresponding sample in the original Cr decoded block of the CU.

Furthermore, the video encoder 20 can use a quadtree partitioning method to decompose the luminance residual block, Cb residual block, and Cr residual block of the CU into one or more luminance transform blocks, Cb transform blocks, and Cr transform blocks. A transform block can be a rectangular block of samples on which the same transform is applied. A transform unit (TU) of the CU can include a transform block of luminance samples, two corresponding transform blocks of chrominance samples, and a syntax structure for transforming the transform block samples. In a monochrome image or an image with three separate color planes, a TU can include a single transform block and a syntax structure for transforming the transform block samples. Therefore, each TU of the CU can be associated with a luminance transform block, a Cb transform block, and a Cr transform block. The luminance transform block associated with a TU can be a sub-block of the luminance residual block of the CU. The Cb transform block can be a sub-block of the Cb residual block of the CU. The Cr transform block can be a sub-block of the Cr residual block of the CU.

The video encoder 20 can apply one or more transforms to the luminance transform block of the TU to generate a luminance coefficient block of the TU. The coefficient block can be a two-dimensional array of transform coefficients. The transform coefficients can be scalars. The video encoder 20 can apply one or more transforms to the Cb transform block of the TU to generate a Cb coefficient block of the TU. The video encoder 20 can apply one or more transforms to the Cr transform block of the TU to generate a Cr coefficient block of the TU.

After generating coefficient blocks (e.g., brightness coefficient blocks, Cb coefficient blocks, or Cr coefficient blocks), the video encoder 20 can quantize the coefficient blocks. Quantization typically refers to the process of quantizing transform coefficients to reduce the amount of data used to represent the transform coefficients as much as possible, thereby providing further compression. Furthermore, the video encoder 20 can inverse-quantize the transform coefficients and apply the inverse transform to the transform coefficients to reconstruct the transform blocks of the TU of the CU of the image. The video encoder 20 can use the reconstructed transform blocks of the TU of the CU and the predictive blocks of the PU of the CU to reconstruct the decoded blocks of the CU. By reconstructing the decoded blocks of each CU of the image, the video encoder 20 can reconstruct the image. The video encoder 20 can store the reconstructed image in a decoded picture buffer (DPB). The video encoder 20 can use the reconstructed image in the DPB for inter-frame prediction and intra-frame prediction.

Following the quantization coefficient block in video encoder 20, video encoder 20 may perform entropy encoding on the syntax elements indicating the quantized transform coefficients. For example, video encoder 20 may perform context-adaptive binary arithmetic decoding (CABAC) on the syntax elements indicating the quantized transform coefficients. Video encoder 20 may output the entropy-encoded syntax elements in the bitstream.

The video encoder 20 can output a bitstream containing a bit sequence that forms a representation of a decoded image and associated data. The bitstream may include a sequence of Network Abstraction Layer (NAL) units. Each NAL unit contains a NAL unit header and encapsulates a Raw Byte Sequence Payload (RBSP). The NAL unit header may include syntax elements indicating the NAL unit type code. The NAL unit type code, specified by the NAL unit header, indicates the type of the NAL unit. The RBSP may be a syntax structure containing an integer number of bytes encapsulated within the NAL units. In some cases, the RBSP contains zero bits.

Different types of NAL units can encapsulate different types of RBSPs. For example, a first type of NAL unit can encapsulate an RBSP of a Picture Parameter Set (PPS), a second type of NAL unit can encapsulate an RBSP of decoded slices, a third type of NAL unit can encapsulate an RBSP of Supplemental Enhancement Information (SEI), and so on. A PPS is a syntax structure that can contain syntax elements applicable to zero or more complete decoded pictures. NAL units that encapsulate RBSPs of video decoded data (as opposed to RBSPs of parameter sets and SEI messages) are called Video Decoding Layer (VCL) NAL units. NAL units that encapsulate decoded slices are referred to herein as decoded slice NAL units. RBSPs used for decoded slices can contain slice headers and slice data.

Video decoder 30 can receive a bitstream. Furthermore, video decoder 30 can parse the bitstream to decode syntax elements from it. Video decoder 30 can reconstruct images of video data, at least in part, based on the syntax elements decoded from the bitstream. The process of reconstructing the video data can be substantially the inverse of the process performed by video encoder 20. For example, video decoder 30 can use motion vectors of the PU to determine predictive blocks of the PU at the current CU. Video decoder 30 can use one or more motion vectors of the PU to generate predictive blocks of the PU.

Furthermore, the video decoder 30 can dequantize the coefficient block associated with the TU of the current CU. The video decoder 30 can perform an inverse transform on the coefficient block to reconstruct the transform block associated with the TU of the current CU. By adding samples of the predictive sample block of the PU of the current CU to the corresponding samples of the transform block of the TU of the current CU, the video decoder 30 can reconstruct the decoded block of the current CU. By reconstructing the decoded blocks of each CU of the image, the video decoder 30 can reconstruct the image. The video decoder 30 can store the decoded image in a decoded image buffer for output and/or use during the decoding of other images.

The next generation of video applications is expected to operate on video data representing captured scenes with HDR and/or WCG. The parameters utilized, dynamic range and color gamut, are two independent attributes of video content, and their specifications are defined by several international standards for digital television and multimedia services. For example, ITU-R Rec.BT.709, "Parameter values for the HDTV standards for production and international program exchange," defines parameters used for HDTV (High-Definition Television), such as standard dynamic range (SDR) and standard color gamut, and ITU-R Rec.BT.2020, " The "Parameter values for ultra-high definition television systems for production and international program exchange" specifies UHDTV (Ultra-High Definition Television) parameters such as HDR and WCG. Other standards development organization (SDO) documents also exist, specifying dynamic range and color gamut attributes for other systems; for example, the DCI-P3 color gamut is defined in SMPTE-231-2 (Institute of Moving Picture and Television Engineers), and some HDR parameters are defined in SMPTE-2084. A brief description of the dynamic range and color gamut of video data is provided below.

Dynamic range is typically defined as the ratio between the maximum and minimum luminance (e.g., illuminance) of a video signal. Dynamic range can also be measured in "f-stops," where one f-stop corresponds to doubling the signal's dynamic range. In the MPEG definition, HDR content is characterized by a luminance variation of more than 16 f-stops. In some terminology, the level between 10 and 16 f-stops is considered intermediate dynamic range, but in other definitions it is considered HDR. In some embodiments of the invention, HDR video content can be any video content with a higher dynamic range than conventionally used video content with a standard dynamic range (e.g., video content specified by ITU-R Rec.BT.709).

The human visual system (HVS) can perceive a much larger dynamic range than SDR and HDR content. However, the HVS includes an adaptation mechanism that narrows the HVS's dynamic range to what is known as the simultaneous range. The width of the simultaneous range can depend on current lighting conditions (e.g., current brightness). The observations of the dynamic range provided by the SDR of HDTV, the expected HDR of UHDTV, and the HVS dynamic range are shown in Figure 3, but the precise range can vary based on each individual and display.

Some instance video applications and services are regulated by ITU Rec.709 and provide SDR, which typically supports a range of luminance (e.g., illuminance) (often referred to as "nits") of approximately 0.1 to 100 candela per square meter, resulting in less than 10 f-stops. Some instance next-generation video services are expected to offer a dynamic range of up to 16 f-stops. Although detailed specifications for this content are currently under development, some initial parameters have been specified in SMPTE-2084 and ITU-R Rec.2020.

Besides HDR, another aspect of a more realistic video experience is the color dimension. The color dimension is typically defined by the color gamut. Figure 4 is a conceptual diagram illustrating the SDR color gamut (triangle 100 based on BT.709 color primaries) and the wider color gamut used in UHDTV (triangle 102 based on BT.2020 color primaries). Figure 3 also depicts the so-called spectral locus (bounded by tongue-shaped region 104), thus representing the boundaries of natural colors. As illustrated in Figure 3, moving from BT.709 (triangle 100) to BT.2020 (triangle 102), the color primaries are designed to provide UHDTV services with approximately more than 70% color. D65 specifies an example of white for the BT.709 and/or BT.2020 specifications.

Examples of color gamut specifications for the DCI-P3, BT.709, and BT.202 color spaces are shown in Table 1.

Table 1 - Color Gamut Parameters

As shown in Table 1, the color gamut can be defined by the X and Y values of the white point and by the X and Y values of the primary colors (e.g., red (R), green (G), and blue (B)). The X and Y values represent the chromaticity (X) and lightness (Y) of a color, as defined by the CIE 1931 color space. The CIE 1931 color space defines the connections between pure colors (e.g., in terms of wavelength) and how the human eye perceives such colors.

HDR/WCG video data is typically acquired and stored at extremely high precision per component (even floating-point) (in 4:4:4 chroma subsampling formats and extremely wide color spaces, such as CIE XYZ). This representation aims for high precision and is mathematically virtually lossless. However, this format used to store HDR/WCG video data can contain significant redundancy and may be suboptimal for compression purposes. Lower precision formats with HVS-based assumptions are commonly used in current state-of-the-art video applications.

An example of a video data format conversion process for compression purposes comprises three main processes, as shown in Figure 5. The technique described in Figure 5 can be performed by source device 12. Linear RGB data 110 may be HDR/WCG video data and may be stored in floating-point representation. The linear RGB data 110 can be compressed using a non-linear transfer function (TF) 112 for dynamic range compression. The transfer function 112 can use any number of non-linear transfer functions (e.g., PQ TF as defined in SMPTE-2084) to compress the linear RGB data 110. In some instances, color conversion process 114 converts the compressed data into a more compact or robust color space (e.g., YUV or YCrCb color space) suitable for compression by a hybrid video encoder. This data is then quantized using a floating-point to integer representation quantization unit 116 to produce converted HDR' data 118. In this example, the HDR' data 118 is represented as an integer. The HDR' data is now in a format suitable for compression by a hybrid video encoder (e.g., a video encoder 20 using HEVC technology). The order of processes depicted in Figure 5 is given as an example and may be changed in other applications. For instance, color conversion may precede the TF process. In some instances, additional processing, such as spatial subsampling, may be applied to the color components.

The inverse conversion at the decoder side is depicted in Figure 6. The technique of Figure 6 can be performed by the destination device 14. The converted HDR' data 120 can be obtained at the destination device 14 by decoding video data using a hybrid video decoder (e.g., a video decoder 30 applying HEVC technology). The HDR' data 120 can then be dequantized by the dequantization unit 122. The inverse color conversion process 124 can then be applied to the dequantized HDR' data. The inverse color conversion process 124 can be the reverse of the color conversion process 114. For example, the inverse color conversion process 124 can convert the HDR' data from YCrCb format back to RGB format. Next, the inverse transfer function 126 can be applied to the data to add back the dynamic range compressed by the transfer function 112, thereby reconstructing the linear RGB data 128.

The techniques depicted in Figure 5 will now be discussed in more detail. Mapping digital values presented in an image container to and from light energy may involve the use of a “transfer function.” Generally, a transfer function is applied to data (e.g., HDR/WCG video data) to compress the dynamic range of the data. This compression allows data to be represented with fewer bits. In one instance, the transfer function may be a one-dimensional (1D) nonlinear function and may reflect the inverse of the electro-optical transfer function (EOTF) of the end-user display, for example, as specified for SDR in ITU-R BT.1886 (also defined in Rec.709). In another instance, the transfer function may approximate the HVS perception of luminance changes, for example, the PQ transfer function specified for HDR in SMPTE-2084. The reverse process of OETF is the EOTF (electro-optical transfer function), which maps the code level back to illuminance. Figure 7 shows several examples of nonlinear transfer functions used to compress the dynamic range of certain color containers. The transfer function may also be applied individually to each R, G, and B component.

The reference EOTF specified in ITU-R Recommendation BT.1886 is defined by the following equation:

L＝a(max[V+b),0]) ^γ

in:

L: Screen illuminance in cd/m2

L _W : Illuminance of white screen

L _B : Illuminance of a black screen

V: Input video signal level (normalized, black at V=0, white at V=1). For content mastered according to ITU-R BT.709, the 10-bit digital code value "D" is mapped to the V value according to the following equation: V = (D-64)/876

γ: The exponent of the power function, γ = 2.404

a: Variables related to user gain (traditional "contrast" control)

a＝(L _W ^1/γ -L _B ^1/γ ) ^γ

b: Variables that increase the user's black level (traditional "brightness" control)

The variables a and b above can be derived by solving the following equations, such that V = 1 yields L = L _W , and such that V = 0 yields L = L _B :

L _B = a·b ^γ

L _W = a·(1+b) ^γ

To more efficiently support higher dynamic range, SMPTE has recently standardized a new transfer function called SMPTE ST-2084. The ST-2084 specification defines the EOTF application as follows. Applying TF to the standardized linear R, G, B values results in a non-linear representation of R'G'B'. ST-2084 defines normalization by NORM = 10000, which is associated with a peak luminance of 10000 nits (cd/m²).

οR'＝PQ_TF(max(0,min(R/NORM,1)))

οG'＝PQ_TF(max(0,min(G/NORM,1)))

οB'＝PQ_TF(max(0,min(B/NORM,1)))

in

Typically, EOTF is defined as a function with floating-point accuracy; therefore, if an inverse TF (so-called OETF) is applied, then error-free input is introduced to the signal with this non-linearity. The inverse TF (OETF) specified in ST-2084 is defined as the inversePQ function:

οR＝10000*inversePQ_TF(R')

οG＝10000*inversePQ_TF(G')

οB＝10000*inversePQ_TF(B')

in

It should be noted that EOTF and OETF are very active research and standardization entities, and the TF used in some video decoding systems may differ from ST-2084.

In the context of this invention, the terms "signal value" or "color value" can be used to describe the illuminance level corresponding to the value of a specific color component (such as R, G, B, or Y) of an image element. The signal value typically represents a linear illuminance level (illuminance value). The terms "code level" or "digital code value" can refer to a digital representation of the image signal value. Typically, this digital representation indicates a non-linear signal value. EOTF represents the relationship between the non-linear signal value provided to the display device (e.g., display device 32) and the linear color value generated by the display device.

RGB data is commonly used as the input color space because it is a data type typically generated by image capture sensors. However, the RGB color space has high redundancy in its components and is not optimal for compact representation. To achieve a more compact and robust representation, RGB components are often transformed (e.g., by performing color transformations) to a less correlated color space (e.g., YCbCr) that is more suitable for compression. The YCbCr color space separates the luminance (Y) and color information (CrCb) in the form of illumination (Y) from the different less correlated components. In this context, a robust representation can refer to a color space that is characterized by higher-order error resilience when compressed at a limited bit rate.

For modern video decoding systems, the commonly used color space is YCbCr, as specified in ITU-R BT.709. The YCbCr color space in the BT.709 standard specifies the following conversion process from R'G'B' to Y'CbCr (in non-constant illuminance representation):

a.Y'＝0.2126*R'+0.7152*G'+0.0722*B'

b.

c.

The above process can also be implemented using the following approximate transformation to avoid division of Cb and Cr components:

a.Y'＝0.212600*R'+0.715200*G'+0.072200*B'

b.Cb＝-0.114572*R'-0.385428*G'+0.500000*B'

c.Cr＝0.500000*R'-0.454153*G'-0.045847*B'

The ITU-R BT.2020 standard specifies the following conversion process from R'G'B' to Y'CbCr (non-constant illuminance representation):

a.Y'＝0.2627*R'+0.6780*G'+0.0593*B'

b.

c.

The above process can also be implemented using the following approximate transformation to avoid division of Cb and Cr components:

a.Y'＝0.262700*R'+0.678000*G'+0.059300*B'

b.Cb＝-0.139630*R'-0.360370*G'+0.500000*B'

c.Cr＝0.500000*R'-0.459786*G'-0.040214*B'

After color transformation, the input data in the target color space can still be represented at a high bit depth (e.g., floating-point accuracy). This high bit depth data can be converted to the target bit depth, for example, using a quantization process. Some studies have shown that 10- to 12-bit accuracy combined with PQ transfer is sufficient to provide HDR data with a 16f aperture and distortion below just distinguishable difference (JND). Generally, JND is the amount of something (e.g., video data) that must be changed to make the difference distinguishable (e.g., via HVS). Data represented at 10-bit accuracy can be further decoded by most current state-of-the-art video decoding solutions. This quantization is an element of lossy decoding and a source of inaccuracy introduced into the converted data.

An example of such quantization applied to codewords in the target color space (YCbCr in this example) is shown below. The input value YCbCr, expressed in floating-point accuracy, is converted into a signal with a fixed bit depth of BitDepthY for the Y value and a fixed bit depth of BitDepthC for the chromaticity values (Cb, Cr).

οD _Y′ =Clip1 _Y (Round((1＜＜(BitDepth _Y -8))*(219*Y′+16)))

οD _Cb ＝Clip1 _C (Round((1＜＜(BitDepth _C -8))*(224*Cb+128)))

οD _Cr ＝Clip1 _C (Round((1＜＜(BitDepth _C -8))*(224*Cr+128)))

in

Round(x)＝Sign(x)*Floor(Abs(x)+0.5)

If x < 0, then Sign(x) = -1; if x = 0, then Sign(x) = 0; if x > 0, then Sign(x) = 1.

Floor(x) is the largest integer less than or equal to x.

If x >= 0, then Abs(x) = x; if x < 0, then Abs(x) = -x

Clip1 _Y (x)＝Clip3(0,(1<<BitDepth _Y )-1,x)

Clip1 _C (x)＝Clip3(0,(1<<BitDepth _C )-1,x)

If z < x, then Clip3(x,y,z) = x; if z > y, then Clip3(x,y,z) = y; otherwise, Clip3(x,y,z) = z. The Rate Distortion Optimized Quantizer (RDOQ) will now be described. Most current state-of-the-art video decoding solutions (e.g., HEVC and the developing VVC) are based on so-called hybrid video decoding schemes, which essentially apply scalar quantization to transform coefficients generated from the residual signal through temporal or spatial prediction between the current decoded video signal and a reference image available at the decoder side. Scalar quantization is applied to the encoder side (e.g., video encoder 20) and inverse scalar dequantization is applied to the decoder side (e.g., video decoder 30). Lossy scalar quantization introduces distortion into the reconstructed signal and requires a certain number of bits to pass the quantized transform coefficients and the decoding mode description to the decoder side.

During the evolution of video compression technology, various methods have been developed aimed at improving the calculation of quantized coefficients. One such method is Rate Distortion Optimized Quantization (RDOQ), which is based on a rough estimate of the modified RD cost or the removal of selected transform coefficients or groups of transform coefficients. The goal of RDOQ is to find an ideal or optimal set of quantized transform coefficients that represent the residual data in the coded block. RDOQ calculates the image distortion in the coded block (introduced by the quantization of the transform coefficients) and the number of bits required to encode the corresponding quantized transform coefficients. Based on these two values, the encoder selects better coefficient values by calculating the RD cost.

The RDOQ in the encoder can comprise three stages: quantization of the transform coefficients, elimination of coefficient groups (CGs), and selection of the last non-zero coefficients. In the first stage, the video encoder generates transform coefficients using a uniform quantizer without zero-value regions, which causes the calculation of the level value for the current transform coefficient. After this, the video encoder considers two additional values for this quantized coefficient: level -1 and 0. For each of these three options {level, level -1, 0}, the video encoder calculates the RD cost of encoding the coefficients with the selected values and selects the coefficient with the lowest RD cost. Additionally, some RDOQ implementations may consider completely invalidating the transform coefficient groups, or reducing the size of the transformed coefficient groups represented by signals by decreasing the position of the last signaled coefficient in each of the groups. On the decoder side, inverse scalar quantization is applied to the quantized transform coefficients derived from the syntax elements of the bitstream.

Some existing transfer functions and color transformations used for video decoding can produce video data representations characterized by significant variations in the just-discriminate difference (JND) threshold within the dynamic range of the signal representation. That is, some ranges of codeword values for the luminance and/or chrominance components may have different JND thresholds compared to other ranges of codeword values for the luminance and/or chrominance components. For such representations, a quantization scheme that is uniform across the dynamic range of luminance values (e.g., uniform across all codeword values of luminance) will introduce quantization errors, resulting in different perceptual advantages within the signal segments (segments of the dynamic range). This effect on the signal can be interpreted as a processing system with non-uniform quantization that produces unequal signal-to-noise ratios within the processed data range.

An example of this representation is a video signal represented in the Non-Constant Illumination (NCL) YCbCr color space, where the color primaries are as defined in ITU-R Rec.BT.2020 and have the ST-2084 transfer function. As illustrated in Table 2, the NCLYCbCr color space assigns a significantly larger number of codewords to the low intensity values of the signal; for example, 30% of the codewords represent linear light samples <10 nits, while high intensity samples (high brightness) are represented with a smaller number of codewords; for example, 25% of the codewords are assigned to linear light in the range of 1000 to 10000 nits. Therefore, video decoding systems characterized by uniform quantization across the entire range of data (e.g., H.265/HEVC) will introduce more severe decoding artifacts to high intensity samples (bright areas of the signal), while the distortion introduced to low intensity samples (dark areas of the same signal) will be far less noticeable.

Table 2. Relationship between linear optical intensity and code value in SMPTE ST 2084 (bits = 10)

<![CDATA[Linear light intensity (cd/m²)]]> Full range SDI range Narrow range ~0.01 twenty one 25 83 ~0.1 64 67 119 ~1 153 156 195 ~10 307 308 327 ~100 520 520 509 ~1,000 769 767 723 ~4,000 923 920 855 ~10,000 1023 1019 940

In practice, this means that the design of video decoding systems or encoding algorithms can benefit from adjusting each chosen video data representation, i.e., each chosen transfer function and color space. The following methods have previously been proposed to address the problem of the aforementioned suboptimal perceptual quality codeword distribution.

In their September 2015 VCEG paper COM16-C 1027-E, "Dynamic Range Adjustment SEI to enable High Dynamic Range video coding with Backward-Compatible Capability," D. Rusanovskyy, A.K. Ramasubramonian, D. Bugdayci, S. Lee, J. Sole, and M. Karczewicz proposed applying codeword redistribution to video data prior to video decoding. The video data in the ST-2084/BT.2020 representation undergoes codeword redistribution before video compression. This redistribution, via dynamic range adjustment, introduces a linearization of the perceived distortion (signal-to-noise ratio) within the dynamic range of the data. This redistribution is intended to improve visual quality under bit-rate constraints. To compensate for redistribution and convert the data back to its original ST 2084/BT.2020 representation, a reverse process is applied to the data after video decoding.

One drawback of this method is that preprocessing and postprocessing are typically separated from the rate distortion optimization processes employed by encoders using current state-of-the-art techniques on a block-based basis. Therefore, the techniques described in the VCEG document COM16-C 1027-E do not utilize information available to the decoder, such as the target frame rate of quantization distortion introduced by the video codec's quantization scheme.

In the VCEG document COM16-C 1030-E, "Performance investigation of high dynamic range and wide color gamut video coding techniques," published in September 2015 by J. Zhao, S.-H. Kim, A. Segall, and K. Misra, a block-based quantization scheme was proposed to address the intensity-dependent spatial variation (block-based) quantization of visually perceived distortion between video decoding applied to Y ₂₀₂₀ (ST2084/BT2020) and Y ₇₀₉ (BT1886/BT 2020) representations, for aligning bit rate allocation. It was observed that to maintain the same level of quantization for the luminance component, the quantization phase difference of the signals in Y ₂₀₂₀ and Y ₇₀₉ depends on the luminance value, such that:

QP_Y ₂₀₂₀ ＝QP_Y ₇₀₉ -f(Y ₂₀₂₀ )

The function f(Y ₂₀₂₀ ) is considered to be linear with respect to the intensity values (luminance levels) of the video in Y ₂₀₂₀ , and the function can be approximated as:

f(Y ₂₀₂₀ )=max(0.03*Y ₂₀₂₀ -3,0)

The proposed spatial variation quantization scheme introduced during the encoding stage is believed to improve the visual perception signal-to-quantization noise ratio of decoded video signals in ST 2084/BT.2020 representation.

One drawback of this approach is the block-based granularity of QP tuning. Typically, the block size selected for compression at the encoder side is derived through a rate distortion optimization process and may not represent the dynamic range characteristics of the video signal; therefore, the selected QP setting will be suboptimal for the signal within the block. This problem may become even more significant for next-generation video decoding systems that tend to employ predictive and larger-dimensional transform block sizes. Another aspect of this design is to represent the QP tuning parameters as signals on the decoder side for the needs of inverse dequantization. Furthermore, spatial tuning of quantization parameters at the encoder side increases the complexity of coding optimization and can interfere with the rate control algorithm.

The paper "Intensity-dependent spatial quantization with application in HEVC" (Matteo Naaccari and Marta Marak, Proc. of IEEE, 2013 International Conference on Multimedia and Expo, July 2013) proposes an intensity-dependent spatial quantization (IDSQ) perceptual mechanism. IDSQ utilizes the intensity occlusion of the human visual system and perceptually adjusts the signal quantization at the block level. The authors propose employing in-loop pixel-domain scaling. The parameters for in-loop scaling of the current processed block are derived from the average of the luminance components in the predicted block. On the decoder side, inverse scaling is performed, and the decoder derives the scaling parameters from the predicted block available on the decoder side.

Similar to the techniques in the "Performance investigation of high dynamic range and wide color gamut video coding techniques," the block-based granularity of this method is limited by the suboptimal scaling parameters applied to all samples of the processed block, thus constraining its performance. Another aspect of the proposed solution is that the gradation value is derived from the predicted block and does not reflect signal fluctuations that may occur between the current codec block and the predicted block.

The paper "De-quantization and scaling for next-generation containers" (J. Zhao, A. Segall, S.-H. Kim, K. Misra (Sharp), JVET document B0054, January 2016) addresses the problem of non-uniform perceived distortion in ST.2084/BT.2020 representations. The authors propose an in-loop intensity-dependent block-based transform domain scaling. The parameters for in-loop scaling of the selected transform coefficients (AC coefficients) of the currently processed block are derived from the average value of the luminance components in the predicted block and the DC value derived for the current block. On the decoder side, inverse scaling is performed, and the decoder derives the AC coefficient scaling parameters from the predicted block available on the decoder side and from the quantized DC value represented by the signal to the decoder.

Similar to the techniques in "Performance investigation of high dynamic range and wide color gamut video coding techniques" and "Intensity-dependent spatial quantization with application in HEVC," the block-based granularity of this method limits its performance due to the suboptimal scaling parameters applied to all samples of the processed block. Another aspect of the proposed solution is that the gradation value is applied only to the AC transform coefficients. Therefore, the improvement in the signal-to-noise ratio does not affect the DC value, which degrades the scheme's performance. Furthermore, in some video decoding system designs, the quantized DC value may be unavailable when scaling the AC value, for example, in cases where a series of transform operations follow the quantization process. Another limitation of this proposal is that when the encoder selects transform-span or transform/quantization bypass mode for the current block, suboptimal scaling is not applied due to the exclusion of potential decoding gain for these two modes (therefore, at the decoder, scaling is not defined for transform-span and transform/quantization bypass modes).

U.S. Patent Application No. 15/595,793, filed May 15, 2017, describes in-loop sample processing for video signals with a non-uniformly distributed JD. This patent application describes the application of indexing and offsetting of signal samples represented in the pixel domain, residual domain, or transform domain. Several algorithms for deriving the indexing and offset are proposed.

This invention describes several video decoding and processing techniques applicable to video decoding loops in video decoding systems (e.g., during video encoding and/or decoding processes, but not during preprocessing or postprocessing). The techniques of this invention include encoder-side (e.g., video encoder 20) algorithms with content-adaptive spatial variation quantization to more efficiently compress HDR/WCG video signals without explicit signaling of quantization parameters (e.g., variations in quantization parameters represented by δQP syntax elements). The techniques of this invention also include decoder-side (e.g., video decoder 30) operations that improve the performance of video decoding tools that use quantization parameter information. Examples of such decoding tools may include deblocking filters, bidirectional filters, adaptive loop filters, or other video decoding tools that use quantization information as input.

The video encoder 20 and/or video decoder 30 may be configured to perform one or more of the following technologies independently or in any combination with other technologies.

In one embodiment of the invention, the video encoder 20 may be configured to perform a multi-stage quantization process for each video data block in a video data picture. The techniques described below can be applied to both the luminance and chrominance components of the video data. The video decoder may be configured to perform quantization using a base quantization parameter (QPb) value. That is, the QPb value is applied uniformly across all blocks. For a given base quantization parameter (QPb) value provided to the transform quantization of the sample s (Cb) to be applied to the decoded block Cb, the video encoder 20 may be further configured to utilize a content-dependent QP offset as a deviation from the QPb value. That is, for each video data block, or for a group of video data blocks, the video encoder 20 may further determine a QP offset based on the content of the blocks in the block group.

In this way, the video encoder 20 can consider the rate distortion optimization (RDO) selection of the quantization level LevelX, which is generated by practically different quantization parameters (QPe). In this invention, QPe can be referred to as the effective quantization parameter. QPe is the QP offset (δQP) plus the base QPb value. The video encoder 20 can derive the QPe of the current block Cb using the following equation:

QPe(Cb)＝QPb(Cb)+deltaQP(s(Cb)),with deltaQP>0 (1)

The variable δQP(Cb) is derived from local characteristics (e.g., statistics) of the decoded block Cb. For example, video encoder 20 can be configured to derive the δQP value of block Cb as a function of the average of the sample values (e.g., luminance or chrominance values) of block Cb. In other instances, video encoder 20 can determine the δQP value using other functions of the sample values of block Cb. For example, video encoder 20 can determine the δQP value using a second-order operation (e.g., difference) on the sample values of block Cb. As another example, video encoder 20 can determine the δQP value as a function of the sample values of block Cb and one or more sample values from neighboring blocks. As will be explained in more detail below, video encoder 20 can be configured to quantize the residual value of block Cb using both the QPb value and the QPe value. Therefore, the residual data r(Cb) derived for the current decoded block Cb is decoded using the quantization parameter QPb. However, the distortion introduced into the residual is first generated using the quantization parameter QPe, resulting in the transformed quantization coefficient tq(Cb). Since QPe can vary between blocks, the video encoder 20 can adjust the JND threshold, which exists in some color representations, and provide non-uniform quantization.

At video decoder 30, the quantized transform coefficients tq(Cb) undergo inverse quantization using the underlying quantization parameter QPb. Video decoder 30 can derive the underlying quantization parameter QPb from the syntax elements associated with the current block Cb. Video decoder 30 can receive syntax elements from the encoded video bitstream. Video decoder 30 can then perform one or more inverse transforms on the inverse-quantized transform coefficients to construct the decoded residual. Video decoder 30 can then perform a prediction process (e.g., inter-frame prediction or intra-frame prediction) to produce a decoded sample d(Cb) of the current block Cb.

It should be noted that when reconstructing the residual values of blocks, the video decoder 30 does not use the effective quantization parameter QPe. Therefore, the residuals still contain distortion introduced by the video encoder 20 when QPe was applied during encoding, thereby improving the non-uniform JND thresholding problem for certain color spaces, as discussed above. However, considering that the residual signal is characterized by distortion introduced by the quantization parameter QPe, which is greater than the QPb value conveyed in the bitstream and associated with the current Cb, other decoding tools (e.g., in-loop filtering, entropy decoding, etc.) that rely on the QP parameter provided by the bitstream to weaken their operation can be adjusted to improve their performance. This adjustment is made by providing the decoder under consideration with an estimate of the actual QPe applied to Cb by the video encoder 20. As will be explained in more detail below, the video decoder 30 can be configured to derive an estimate of the effective quantization parameter QPe from the statistics of the decoded sample d(Cb) and other parameters of the bitstream. In this way, bit overhead is saved when the block-by-block value of QPe is not represented as a signal in the bitstream.

The following sections provide non-limiting examples of implementing the technology of the present invention. First, an example of the structure of the encoder-side algorithm of the video encoder 20 will be described.

Figure 8 is a block diagram illustrating an example of a video encoder 20 on which the technology of the present invention can be implemented. As shown in Figure 8, the video encoder 20 receives the current video block of video data within a video frame to be encoded. According to the technology of the present invention, the video data received by the video encoder 20 may be HDR and/or WCG video data. In the example of Figure 8, the video encoder 20 includes a mode selection unit 40, a video data memory 41, a DPB 64, a summer 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. The mode selection unit 40 further includes a motion compensation unit 44, a motion estimation unit 42, an intra-frame prediction processing unit 46, and a segmentation unit 48. For video block reconstruction, the video encoder 20 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and a summer 62. A deblocking filter (not shown in Figure 8) may also be included to filter block boundaries to remove block artifacts from the reconstructed video. If necessary, the deblocking filter will typically filter the output of the summer 62. In addition to the deblocking filter, additional filters (inside or after the loop) can also be used. These filters are not shown for simplicity, but they can be used to filter the output of summer 50 (as in-loop filters) if needed.

Video data memory 41 can store video data to be encoded by components of video encoder 20. The video data stored in video data memory 41 can be obtained, for example, from video source 18. Decoded picture buffer 64 can be a reference picture memory that stores reference video data for use by video encoder 20, for example, when encoding video data in intra-frame or inter-frame decoding modes. Video data memory 41 and decoded picture buffer 64 can be formed of any of a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. Video data memory 41 and decoded picture buffer 64 can be provided by the same memory device or separate memory devices. In various instances, video data memory 41 can be on-chip along with other components of video encoder 20, or off-chip relative to those components.

During encoding processing, video encoder 20 receives video frames or slices to be decoded. The frames or slices can be divided into multiple video blocks. Motion estimation unit 42 and motion compensation unit 44 perform inter-frame predictive decoding of the received video blocks relative to one or more blocks in one or more reference frames to provide temporal prediction. Intra-frame prediction processing unit 46 may alternatively perform intra-frame predictive decoding of the received video blocks relative to one or more adjacent blocks in the same frame or slice as the block to be decoded to provide spatial prediction. Video encoder 20 may perform multiple decoding passes, for example, to select an appropriate decoding mode for each video data block.

Furthermore, segmentation unit 48 can segment video data blocks into sub-blocks based on an evaluation of previous segmentation schemes in previous decoding passes. For example, segmentation unit 48 can initially segment frames or slices into LCUs, and further segment each of the LCUs into sub-CUs based on bitrate-distortion analysis (e.g., rate-distortion optimization). Mode selection unit 40 can further generate a quadtree data structure indicating the segmentation of LCUs into sub-CUs. The leaf nodes of the quadtree, CUs, can contain one or more PUs and one or more TUs. In other instances, segmentation unit 48 can segment the input video data according to a QTBT segmentation structure.

The mode selection unit 40 may, for example, select a decoding mode (intra-frame or inter-frame) based on the error result, and provide the resulting intra-frame decoded block or inter-frame decoded block to the summer 50 to generate residual block data, and to the summer 62 to reconstruct the coded block for use as a reference frame. The mode selection unit 40 also provides syntax elements (such as motion vectors, intra-frame mode indicators, partition information, and other such syntax information) to the entropy coding unit 56.

Motion estimation unit 42 and motion compensation unit 44 can be highly integrated, but are described separately for conceptual purposes. Motion estimation performed by motion estimation unit 42 is a process of generating motion vectors that estimate the motion of video blocks. For example, a motion vector may indicate the displacement of a video block's PU within the current video frame or picture relative to a predictive block within a reference picture (or other decoded unit) relative to the current block being decoded within the current picture (or other decoded unit). The predictive block is a block found to closely match the block to be decoded in terms of pixel differences, which can be determined by sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. In some instances, video encoder 20 may calculate the value of the second-integer pixel position of the reference picture stored in decoded picture buffer 64. For example, video encoder 20 may interpolate the value of a quarter-pixel position, an eighth-pixel position, or other fractional pixel position of the reference picture. Therefore, motion estimation unit 42 can perform motion search with respect to full-pixel positions and fractional pixel positions and output motion vectors with fractional pixel accuracy.

The motion estimation unit 42 calculates the motion vector of the PU in the inter-frame decoded slice by comparing the position of the PU with the position of the predictive block of the reference image. The reference image can be selected from a first reference image list (list 0) or a second reference image list (list 1), each of which identifies one or more reference images stored in the decoded image buffer 64. The motion estimation unit 42 sends the calculated motion vector to the entropy coding unit 56 and the motion compensation unit 44.

Motion compensation performed by motion compensation unit 44 may involve extracting or generating predictive blocks based on motion vectors determined by motion estimation unit 42. Again, in some instances, motion estimation unit 42 and motion compensation unit 44 may be functionally integrated. After receiving the motion vector of the PU for the current video block, motion compensation unit 44 may locate the predictive block to which the motion vector points in one of the reference image lists. Summer 50 forms a residual video block by subtracting the pixel values of the predictive block from the pixel values of the decoded current video block, thus forming a pixel difference, as discussed below. Generally, motion estimation unit 42 performs motion estimation with respect to the luminance component, and motion compensation unit 44 uses motion vectors calculated based on the luminance component for both the chroma and luminance components. Mode selection unit 40 may also generate syntax elements associated with video blocks and video slices for use by video decoder 30 when decoding video slices.

As described above, instead of the inter-frame prediction performed by the motion estimation unit 42 and the motion compensation unit 44, the intra-frame prediction processing unit 46 may perform intra-frame prediction for the current block. Specifically, the intra-frame prediction processing unit 46 may determine an intra-frame prediction mode to use for encoding the current block. In some instances, the intra-frame prediction processing unit 46 may (e.g.) use various intra-frame prediction modes to encode the current block during individual encoding passes, and the intra-frame prediction processing unit 46 (or, in some instances, the mode selection unit 40) may select an appropriate intra-frame prediction mode from the tested modes for use.

For example, the intra-prediction processing unit 46 can use rate-distortion analysis for various tested intra-prediction modes to calculate rate-distortion values and select the intra-prediction mode with the best rate-distortion characteristics among the tested modes. Rate-distortion analysis typically determines the amount of distortion (or error) between the coded block and the original uncoded block (which is encoded to produce the coded block), as well as the bit rate (i.e., the number of bits) used to produce the coded block. The intra-prediction processing unit 46 can calculate the ratio from the distortion and rate of various coded blocks to determine which intra-prediction mode exhibits the best rate-distortion value for the block.

After selecting an intra-prediction mode for a block, the intra-prediction unit 46 may provide information indicating the selected intra-prediction mode for the block to the entropy coding unit 56. The entropy coding unit 56 may encode the information indicating the selected intra-prediction mode. The video encoder 20 may include in the transmitted bitstream: configuration data, which may include multiple intra-prediction mode index tables and multiple modified intra-prediction mode index tables (also called codeword maps); definitions of the coding contexts for various blocks; and indications of the most probable intra-prediction mode to be used for each of the contexts, the intra-prediction mode index tables, and the modified intra-prediction mode index tables.

Video encoder 20 forms a residual video block (e.g., r1(Cb) of the current block Cb) by subtracting the predicted data from mode selection unit 40 from the properly decoded original video block. Summer 50 represents one or more components that perform this subtraction operation. Transform processing unit 52 applies a transform (such as Discrete Cosine Transform (DCT) or a conceptually similar transform) to the residual block, thereby producing a video block that includes residual transform coefficient values. Transform processing unit 52 may perform other transforms conceptually similar to DCT. Wavelet transform, integer transform, subband transform, or other types of transforms may also be used. In any case, transform processing unit 52 applies a transform to the residual block, thereby producing a residual transform coefficient block. The transform can convert residual information from the pixel value domain to the transform domain, such as the frequency domain. Transform processing unit 52 may send the resulting transform coefficients tCb to quantization unit 54.

As described above, video encoder 20 can generate a residual signal r(Cb) of the current decoded block Cb from samples of the current decoded block s(Cb) and predicted samples p(Cb) (e.g., predicted samples from inter-frame prediction or intra-frame prediction). Video encoder 20 can perform one or more forward transforms on the residual r(Cb) to produce transform coefficients t(Cb). Video encoder 20 can then quantize the transform coefficients t(Cb) before entropy coding. Quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process can reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some instances, quantization unit 54 can then perform a scan of a matrix containing the quantized transform coefficients. Alternatively, entropy coding unit 56 can perform the scan.

According to the technology of the present invention, the quantization unit 54 can be configured to perform a multi-stage quantization process on the transform coefficients t(Cb). Figure 9 is a block diagram illustrating an example quantization unit of a video encoder in which the technology of the present invention can be implemented.

As shown in Figure 9, in the first stage, the QPe determination unit 202 can be configured to derive the quantization parameter offset (δQP(s(Cb))) of the current block Cb. In one instance, the QPe determination unit 202 can be configured to derive δQP(s(Cb)) from a lookup table (e.g., LUT_DQP 204). LUT_DQP 204 contains the δQP values and is accessed via an index derived from the average of s(Cb) samples (e.g., lightness or chroma samples) of block Cb. The following equation illustrates an example of deriving the quantization parameter offset:

δQP(s(Cb))=LUT_DQP(mean(s(Cb)) (2)

Where LUT_DQP is the lookup table for δQP(s(Cb)) and mean(s(Cb)) is the average of the sample values of block Cb.

In other instances, the QPe determination unit 202 may be configured to derive the value of δQP(s(Cb)) through a function of some other characteristic of the samples of the decoded block or the characteristics of the bitstream (e.g., a second-order function based on variance). The QPe determination unit 202 may be configured to determine the δQP value using an algorithm, a lookup table, or other means of explicitly deriving the δQP value. In some instances, the samples used to determine δQP() may include both luminance and chrominance samples, or more generally, samples containing one or more components of the decoded block.

QPe determination unit 202 can then derive the effective quantization parameter QPe using the variable δQP(Cb), as shown in equation (1) above. QPe determination unit 202 can then provide the QPe value to the first quantization unit 206 and the dequantization unit 208. In the second stage, the first quantization unit 206 performs pre-vectorization on the transform coefficients t(Cb) using the derived QPe value. Subsequently, the dequantization unit 208 dequantizes the quantized transform coefficients using the QPe value, and the inverse transform unit 210 performs an inverse transform (e.g., the inverse transform of the transform processing unit 52). This produces a residual block r2(Cb) with introduced distortion and QPE. The equations for the second stage of the process are shown below.

r2(Cb)＝InverseTrans(InverseQuant(QPe,ForwardQuant(QPe,t(Cb)))) (3)

InverseTrans is the inverse transformation process, InverseQuant is the inverse quantization process, and ForwardQuant is the forward vectorization process.

In the third stage, transform processing unit 212 performs one or more forward transforms on the residual r2(Cb) (e.g., the same as transform processing unit 52). Subsequently, second quantization unit 214 performs pre-vectorization on the transformed residual using the basic quantization parameter QPb. This produces quantized transform coefficients tq(Cb), as shown in the following equation:

tq(Cb)＝ForwardQuant(QPb,ForwardTrans(r2(Cb))) (4)

Where ForwardTrans represents the forward transformation process.

Returning to Figure 8, after quantization, entropy coding unit 56 performs entropy decoding on the quantized transform coefficients tq(Cb). For example, entropy coding unit 56 may perform context-adaptive variable-length decoding (CAVLC), context-adaptive binary arithmetic decoding (CABAC), syntax-based context-adaptive binary arithmetic decoding (SBAC), probabilistic interval partitioning entropy (PIPE) decoding, or another entropy decoding technique. In the case of context-based entropy decoding, the context may be based on neighboring blocks. After entropy decoding by entropy coding unit 56, the encoded bitstream can be transmitted to another device (e.g., video decoder 30) or archived for later transmission or retrieval.

Inverse quantization unit 58 and inverse transform unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct a residual block (e.g.,) in the pixel domain for later use as a reference block. Motion compensation unit 44 can calculate the reference block by adding the residual block to a predictive block of one of the frames in the decoded image buffer 64. Motion compensation unit 44 may also apply one or more interpolation filters to the reconstructed residual block to calculate sub-integer pixel values for use in motion estimation. Summer 62 adds the reconstructed residual block to the motion-compensated predictive block generated by motion compensation unit 44 to produce a reconstructed video block for storage in decoded image buffer 64. The reconstructed video block can be used as a reference block by motion estimation unit 42 and motion compensation unit 44 for inter-frame decoding of blocks in subsequent video frames.

Example embodiments of decoder-side processing will now be described. At the decoder side, some decoding tools rely on quantization parameters associated with the QP value used to decode the current block or group of blocks. Some non-limiting examples may include: deblocking filters, bidirectional filters, loop filters, interpolation filters, entropy codec initialization, or others.

Figure 10 is a block diagram illustrating an example of a video decoder 30 in which the technology of the present invention can be implemented. In the example of Figure 10, the video decoder 30 includes an entropy decoding unit 70, a video data memory 71, a motion compensation unit 72, an intra-frame prediction processing unit 74, an inverse quantization unit 76, an inverse transform processing unit 78, a decoded image buffer 82, a summer 80, a QPe estimation unit 84, a LUT_DQP 86, and a filter unit 88. In some examples, the video decoder 30 may perform a decoding pass that is generally the inverse of the encoding pass described with respect to the video encoder 20 (Figure 8). The motion compensation unit 72 may generate prediction data based on motion vectors received from the entropy decoding unit 70, while the intra-frame prediction processing unit 74 may generate prediction data based on intra-frame prediction mode indicators received from the entropy decoding unit 70.

Video data memory 71 may store video data, such as encoded video bitstreams, to be decoded by components of video decoder 30. The video data stored in video data memory 71 may be obtained via wired or wireless network communication of video data or by accessing physical data storage media, such as from computer-readable media 16 (e.g., from a local video source, such as a camera). Video data memory 71 may form a decoded picture buffer (CPB) storing encoded video data from the encoded video bitstream. Decoded picture buffer 82 may be a reference picture memory storing reference video data for use by video decoder 30, for example, when decoding video data in intra-frame or inter-frame decoding modes. Video data memory 71 and decoded picture buffer 82 may be formed from any of a variety of memory devices, such as DRAM, including SDRAM, MRAM, RRAM, or other types of memory devices. Video data memory 71 and decoded picture buffer 82 may be provided by the same memory device or separate memory devices. In various instances, video data memory 71 may be on-chip along with other components of video decoder 30, or off-chip relative to those components.

During the decoding process, the video decoder 30 receives from the video encoder 20 a encoded video bitstream representing video blocks of encoded video slices and associated syntax elements. The encoded video bitstream may have been encoded by the video encoder 20 using the multi-stage quantization process described above. The encoded video bitstream may also represent video data defined using HDR and/or WCG color formats. The entropy decoding unit 70 of the video decoder 30 performs entropy decoding on the bitstream to produce quantized coefficients, motion vectors or intra-frame prediction mode indicators, and other syntax elements. The entropy decoding unit 70 forwards the motion vectors and other syntax elements to the motion compensation unit 72. In some instances, the entropy decoding unit 70 may decode syntax elements indicating the underlying quantization parameter QPb used for the video data block to be decoded. The video decoder 30 may receive syntax elements at the video slice level and/or video block level.

When a video slice is decoded into an intra-frame decoded (I) slice, the intra-frame prediction processing unit 74 can generate prediction data for the video blocks of the current video slice based on the intra-frame prediction mode represented by the signal and data from the previously decoded blocks from the current frame or picture. When a video frame is decoded into an inter-frame decoded (i.e., B or P) slice, the motion compensation unit 72 generates predictive blocks for the video blocks of the current video slice based on motion vectors and other syntax elements received from the entropy decoding unit 70. The predictive blocks can be generated from one of the reference pictures in a reference picture list. The video decoder 30 can construct a reference picture list: list 0 and list 1, based on the reference pictures stored in the decoded picture buffer 82 using a preset construction technique. The motion compensation unit 72 determines the prediction information for the video blocks of the current video slice by analyzing the motion vectors and other syntax elements, and uses the prediction information to generate predictive blocks for the current video block for forward decoding. For example, motion compensation unit 72 uses some received syntax elements to determine the prediction mode (e.g., intra-frame prediction or inter-frame prediction) used to decode video blocks of a video slice, the inter-frame prediction slice type (e.g., B-slice or P-slice), the construction information of one or more reference picture lists of the slice, the motion vectors of each inter-frame coded video block of the slice, the inter-frame prediction state of each inter-frame decoded video block of the slice, and other information used to decode video blocks in the current video slice.

The motion compensation unit 72 can also perform interpolation based on interpolation filters. The motion compensation unit 72 can use interpolation filters, such as those used by the video encoder 20 during the encoding of video blocks, to calculate interpolated values for the second-integer pixels of the reference block. In this case, the motion compensation unit 72 can determine the interpolation filter used by the video encoder 20 from the received syntax elements and use the interpolation filter to generate a predictive block.

The dequantization unit 76 dequantizes (i.e., dequantizes) the quantized transform coefficients provided in the bitstream and decoded by the entropy decoding unit 70. The dequantization process may involve determining the degree of quantization using the underlying quantization parameter QPb determined by the video decoder 30 for each video block in the video slice, and similarly, determining the degree of dequantization to be applied. The inverse transform processing unit 78 applies an inverse transform (e.g., inverse DCT, inverse integer transform, or a conceptually similar inverse transform process) to the transform coefficients to produce a residual block in the pixel domain.

After the motion compensation unit 72 generates a predictive block for the current video block based on motion vectors and other syntax elements, the video decoder 30 forms a decoded video block by summing the residual block from the inverse transform processing unit 78 with the corresponding predictive block generated by the motion compensation unit 72. The summer 80 represents one or more components that perform this summation operation.

Filter unit 88 can be configured to apply one or more filtering operations to the decoded video data, which is then output and stored in decoded image buffer 82. Decoded video blocks from a given frame or image are then stored in decoded image buffer 82, which stores reference images for subsequent motion compensation. Decoded image buffer 82 also stores decoded video for later presentation on a display device (such as display device 32 of FIG. 1). Examples of filters applied by filter unit 88 include deblocking filters, bidirectional filters, adaptive loop filters, sample adaptive offset filters, and others. For example, a deblocking filter can be applied, if needed, to filter the decoded blocks to remove block artifacts. Other loop filters (within or after the decoding loop) can also be used to smooth pixel transitions or otherwise improve video quality. Decoded image buffer 82 also stores decoded video for later presentation on a display device (such as display device 32 of FIG. 1).

In some instances, the parameters of the filter applied by filter unit 88 may be based on quantization parameters. As described above, the video data received by video decoder 30 contains distortion introduced by video encoder 20 using an effective quantization parameter QPe, which is greater than the QPb value conveyed in the bitstream and associated with the current Cb. The filter applied by filter unit 88 may depend on the QP parameter provided by the bitstream to adjust performance. Therefore, video decoder 30 may be configured to derive an estimate of the actual QPe applied to Cb by video encoder 20. In this regard, video decoder 30 may include a QPe estimation unit 84 for deriving the QPe value.

For example, QPe estimation unit 84 can be configured to estimate the quantization parameter offset (δQP(s(Cb))) of the current block Cb. In one instance, QPe estimation unit 84 can be configured to estimate δQP(s(Cb)) from a lookup table (e.g., LUT_DQP 86). LUT_DQP 86 contains estimates of the δQP values and is accessed via an index derived from the average of decoded s(Cb) samples (e.g., lightness or chroma samples) of block Cb. The following equation illustrates an example of deriving the quantization parameter offset:

δQP(s(Cb))=LUT_DQP(mean(s(Cb)) (2)

Where LUT_DQP is the lookup table for δQP(s(Cb)) and mean(s(Cb)) is the average value of the decoded sample values of block Cb.

In other instances, the QPe estimation unit 84 may be configured to derive the value of δQP(s(Cb)) from some other feature of the samples of the decoded block or a function of the characteristics of the bitstream (e.g., a second-order function based on variance). The QPe estimation unit 84 may be configured to estimate the δQP value using an algorithm, a lookup table, or other means. In some instances, the samples used to determine δQP() may include both luminance and chrominance samples, or more generally, samples of one or more components of the decoded block. The QPe estimation unit 84 can then provide an estimate of QPe to the filter unit 88 for use by one or more decoding tools implemented by the filter unit 88.

In one instance, filter 88 can be configured to perform deblocking filtering. In a non-limiting example of deblocking implementation, the following describes the deblocking process as a variation of the HEVC specification for deblocking filtering. The introduced change is based on...

8.7.2.5.3 Decision process for the edges of lightness blocks

Variables QpQ and QpP are set to be equal to the values of decoding units Cbq and Cbp, which contain blocks containing samples q0,0 and p0,0 respectively. The following derivation follows.

The following is the exported variable qPL:

qPL＝((QpQ+Qpp+1)>>1)

8.7.2.5.5 Filtering process for chroma block edges

Variables QpQ and QpP are set to be equal to the values of the decoding unit, which contains decoding blocks containing samples q0,0 and p0,0 respectively. The following derivation follows.

If ChromaArrayType equals 1, then the variable QpC is determined based on the derived index qPi, as specified in Tables 8 to 10:

qPi＝((QpQ+QpP+1)>>1)+cQpPicOffset(

In the above examples, QpY and QPb are the same and QpY_EQ is the same as QPe.

In another instance, filter unit 88 may be configured to implement a bidirectional filter. The bidirectional filter modifies the sample based on a weighted average of the samples in its neighborhood, and the weights are derived based on the distance between the current sample and the neighboring samples and the difference between the sample values of the current sample and the neighboring samples.

Let x be the position of the current sample value filtered based on samples in its neighborhood N(x). For each sample d(y) belonging to N(x), let w(y,x) be the weight associated with the sample at position y to obtain the filtered version of the sample at position x. The filtered version D(x) of x is then obtained as...

D(x)＝∑ _y∈N(x) w(y,x)d(y) (8)

Weights are exported

w(y,x)＝f(y,x,d(y),d(x),QP(Cb)) (9)

Here, f() is a function that calculates weights based on sample positions and sample values. The QP used to decode the block containing the sample can also be other arguments derived from f(). In some instances, the QP value of the block containing x is used as an argument to f(). In this instance, the QP value used as another argument in f() is the derived QPd(Cb):

QPe(Cb)＝QP(Cb)+deltaQP(d(Cb)) (10)

Where QP(Cb) is the QP value (e.g., QPb) represented by the signal of the decoded block, and δQP(d(Cb)) is the QP value based on the characteristics of the decoded block, such as the mean. Therefore, the derived weights are as follows:

w(y,x)＝f(y,x,d(y),d(x),QPe(Cb)) (11)

In some instances, weighting functions are derived separately for luminance and chrominance. The QP associated with the chrominance decoding block can also have the effect of a chrominance offset, either derived or represented as a signal in the bit stream, and the derived δQP() can be a function of samples of one or more components.

In some instances, the QP values for other variables of f() can be obtained by considering the QPe() values derived from the decoded block for the sample at position x and the QPe() values derived from the decoded block for the sample at position y. For example, values derived from the two QPd() values (e.g., the average) can be chosen as variables of f().

In another embodiment of the invention, the video decoder 30 may be configured to use multiple LUT_DQPs. In some instances, two or more LUT_DQP tables may be used at the video decoder 30. The video decoder 30 may be configured to derive an index of a specific one of two or more lookup tables for use in deriving specific block edges. The video decoder 30 may be configured to derive the index from syntax elements, from decoding information of blocks in the same spatiotemporal neighborhood of the current sample, or from statistics of decoded image samples.

For example:

δQP(d(Cbq))＝LUT_DQP(d(Cbq),Idx1) (12)

δQP(d(Cbq))=LUT_DQP(d(Cbp),Idx2)

Idx1 and Idx2 are index selections from several LUT_DQP tables available at video decoder 30.

In another embodiment of the invention, the video encoder 20 and video decoder 30 may be configured to apply spatial variation quantization with finer block granularity. In some embodiments, the video encoder 20 may be configured to split the currently decoded block Cb into sub-partitions, each of which is processed independently according to Equations 2, 3, and 4 above. Once a reconstructed signal r2 is generated for each partition, it forms further processed r2(Cb) data as shown in Equation (5) above.

At video decoder 30, certain decoding tools (such as deblocking) are modified to reflect this segmentation, but this is not provided in CU segmentation. For example, deblocking is referred to as filtering the edges of these virtual blocks other than the currently specified TU and PU.

In some instances, finer-grained information about block segmentation can be represented as signals in the bitstream syntax elements (e.g., PPS, SPS, or slice headers) and provided to the decoder as side information.

In some instances, constraints on the maximum QP value containing δQP or chromaQP offset values can be removed or extended (e.g., implemented by a clipping process) to support a wider deviation of the QPe parameter from QPb utilizing a video decoding architecture similar to HEVC.

The technology described above in this invention offers the following advantages over other technologies. The technology described above in this invention avoids delta-QP signaling, thereby avoiding a bit rate reduction of a few percent compared to delta-QP-based methods supporting HDR/WCG video data.

Compared with the techniques in "De-quantization and scaling for next-generation containers" (J. Zhao, A. Segall, S.-H. Kim, K. Misra (Sharp), JVET document B0054, January 2016), the above-described techniques of the present invention allow for the same scaling of all transformation coefficients of t(Cb).

Compared to the technology in U.S. Patent Application No. 15/595,793, the above-described technology of the present invention can provide a higher accuracy estimate of local brightness because the decoded value provides a better estimate than the predicted sample.

The above-described technology of the present invention allows for finer granularity in δQP derivation and application without increasing the signaling overhead associated with δQP-based solutions.

Compared with the transformation scaling-based design in "De-quantization and scaling for next-generation containers" and U.S. Patent Application No. 15/595,793, the above-described technology of the present invention has a simpler implementation design.

Figure 11 is a flowchart illustrating the example encoding method. The video encoder 20, which includes a quantization unit 54, can be configured to perform the technique of Figure 11.

In one embodiment of the invention, the video encoder 20 may be configured to determine a base quantization parameter (1100) for a video data block and to determine a quantization parameter offset (1102) for the video data block based on statistics associated with it. The video encoder 20 may be further configured to add the quantization parameter offset to the base quantization parameter to establish an effective quantization parameter (1104), and to encode the video data block using the effective quantization parameter and the base quantization parameter (1106). In one embodiment, the base quantization parameter is the same for all video data blocks. In one embodiment, the sample values of the video data are defined using a high dynamic range video data color format.

In another embodiment of the invention, for encoding video data blocks, the video encoder 20 may be further configured to predict blocks to generate residual samples, transform the residual samples to establish transform coefficients, quantize the transform coefficients with effective quantization parameters, dequantize the quantized transform coefficients with effective quantization parameters to generate distortion transform coefficients, inverse transform the distortion transform coefficients to generate distortion residual samples, transform the distortion residual samples, and quantize the transformed distortion residual samples with basic quantization parameters.

In another embodiment of the invention, to determine the quantization parameter offset, the video encoder 20 may be further configured to determine the quantization parameter offset from a lookup table.

Figure 12 is a flowchart illustrating the example decoding method. The video decoder 30 (including an inverse quantization unit 76, a QPe estimation unit 84, and a filter unit 88) can be configured to perform the technique of Figure 12.

In one embodiment of the invention, a video decoder 30 may be configured to receive encoded blocks of video data, said encoded blocks of video data having been encoded using effective quantization parameters and basic quantization parameters, wherein said effective quantization parameters are a function of quantization parameter offsets added to the basic quantization parameters (1200). The video decoder 30 may be further configured to determine basic quantization parameters (1202) for encoding the encoded blocks of video data, and to decode the encoded blocks of video data using the basic quantization parameters to establish decoded blocks of video data (1204). The video decoder 30 may be further configured to determine an estimate of the quantization parameter offset of the decoded blocks of video data based on statistics associated with the decoded blocks of video data (1206), and to add the estimate of the quantization parameter offset to the basic quantization parameters to establish an estimate of the effective quantization parameters (1208). The video decoder 30 may be further configured to perform one or more filtering operations on the decoded blocks of video data according to the estimate of the effective quantization parameters (1210). In one embodiment, the basic quantization parameters are the same for all video data blocks. In another embodiment, the sample values of the video data are defined using a high dynamic range video data color format.

In another embodiment of the invention, to determine the underlying quantization parameters, the video decoder 30 may be further configured to receive an underlying quantization parameter syntax element in the encoded video bitstream, the value of which indicates the underlying quantization parameters.

In another embodiment of the invention, for decoding video data blocks, the video decoder 30 may be further configured to perform entropy decoding on the encoded blocks of video data to determine quantized transform coefficients, dequantize the quantized transform coefficients using basic quantization parameters to establish transform coefficients, perform inverse transform on the transform coefficients to establish residual values, and perform a prediction process on the residual values to establish decoded blocks of video data.

In another embodiment of the invention, to determine an estimate of the quantization parameter offset of a decoded block of video data, the video decoder 30 may be further configured to determine the average value of sample values of the decoded block of video data, and use the average value of sample values of the decoded block of video data to determine an estimate of the quantization parameter offset of the decoded block of video data.

In another embodiment of the invention, to determine an estimate of the quantization parameter offset, the video decoder 30 may be further configured to determine the estimate of the quantization parameter offset from a lookup table, wherein the average of the sample values is the input to the lookup table.

In another embodiment of the invention, the video decoder 30 may be further configured to determine a lookup table from a plurality of lookup tables.

In another embodiment of the invention, in order to perform one or more filtering operations on decoded blocks of video data, the video decoder 30 may be further configured to apply a deblocking filter to the decoded blocks of video data using effective quantization parameters.

In another embodiment of the invention, in order to perform one or more filtering operations on the decoded blocks of video data, the video decoder 30 may be further configured to apply a bidirectional filter to the decoded blocks of video data using effective quantization parameters.

Certain aspects of the invention have been described for illustrative purposes with reference to HEVC, its extensions, and example standards such as JEM and VVC. However, the techniques described herein can be used in other video decoding processes, including other standard or proprietary video decoding processes that have not yet been developed.

As described in this invention, a video decoder may refer to a video encoder or a video decoder. Similarly, a video decoding unit may refer to a video encoder or a video decoder. Likewise, if applicable, video decoding may refer to video encoding or video decoding.

It should be recognized that, depending on the instance, certain actions or events of any of the techniques described herein may be performed in different sequences, may be added, combined, or omitted entirely (e.g., not all described actions or events are necessary for practicing the techniques). Furthermore, in some instances, actions or events may be performed simultaneously rather than sequentially, for example, via multithreading, interrupt handling, or multiple processors.

In one or more instances, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored on or transmitted via a computer-readable medium as one or more instructions or code, and executed by a hardware-based processing unit. The computer-readable medium may comprise a computer-readable storage medium (which corresponds to a tangible medium such as a data storage medium) or a communication medium, which includes, for example, any medium that facilitates the transfer of a computer program from one place to another according to a communication protocol. In this way, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. The data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and/or data structures for implementing the techniques described herein. A computer program product may comprise a computer-readable medium.

By way of example, and not limitation, these computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, flash memory, or any other media that can be used to store desired program code in the form of instructions or data structures and that is accessible to a computer. Furthermore, any connection is properly referred to as computer-readable media. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of media. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but are in fact referring to non-transient tangible storage media. As used herein, disks and optical discs include optical discs (CDs), laser discs, optical discs, digital video discs (DVDs), floppy disks, and Blu-ray discs, wherein disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. The combination of the above should also be included in the scope of computer-readable media.

Instructions can be executed by one or more processors, such as one or more DSPs, general-purpose microprocessors, ASICs, FPGAs, or other equivalent integrated or discrete logic circuits. Therefore, as used herein, the term "processor" can refer to any of the above-described structures or any other structures suitable for implementing the techniques described herein. Furthermore, in some instances, the functionality described herein can be provided within dedicated hardware and/or software modules configured for encoding and decoding or incorporated into combined codecs. Moreover, the techniques can be fully implemented within one or more circuit or logic elements.

The technology of this invention can be implemented in a wide variety of devices or equipment, including wireless handheld devices, integrated circuits (ICs), or a set of ICs (e.g., chipsets). Various components, modules, or units are described in this invention to emphasize functional aspects of a device configured to perform the disclosed technology, but are not necessarily required to be implemented by different hardware units. Specifically, as described above, various units may be combined with suitable software and/or firmware in a codec hardware unit or provided as a collection of interoperable hardware units, which include one or more processors as described above.

Various examples have been described. These and other examples are within the scope of the following claims.

Claims (34)

1. A method for decoding video data, the method comprising: Receive the encoded blocks of the video data; Determine the basic quantization parameters for the coded block used to encode the video data; The encoded blocks of the video data are decoded using the basic quantization parameters to establish decoded blocks of the video data; Based on the content of the decoded block of the video data, an estimate of the quantization parameter offset for the decoded block of the video data is determined; The estimated quantization parameter offset is added to the base quantization parameter to establish an estimate of the effective quantization parameter; and Based on the estimate of the effective quantization parameters, one or more filtering operations are performed on the decoded block of the video data. 2. The method of claim 1, wherein the basic quantization parameters are the same for all of the blocks of the video data. 3. The method according to claim 1, wherein the sample values of the video data are defined by a high dynamic range video data color format. 4. The method according to claim 1, wherein determining the basic quantization parameters comprises: The basic quantization parameter syntax element is received in the encoded video bitstream, and the value of the basic quantization parameter syntax element indicates the basic quantization parameter. 5. The method of claim 1, wherein decoding the coded block of the video data comprises: Entropy decoding is performed on the coded blocks of the video data to determine the quantized transform coefficients; The quantized transform coefficients are dequantized using the basic quantization parameters to establish transform coefficients; Perform an inverse transformation on the transformation coefficients to establish residual values; and A prediction process is performed on the residual values to construct the decoded block of the video data. 6. The method of claim 1, wherein determining the estimate of the quantization parameter offset for the decoded block of the video data comprises: Determine the average value of the sample values of the decoded blocks of the video data; and The average value of the sample values of the decoded block of the video data is used to determine the estimate of the quantization parameter offset for the decoded block of the video data. 7. The method of claim 6, wherein determining the estimate of the quantization parameter offset comprises: The estimate of the quantization parameter offset is determined from a lookup table, wherein the average of the sample values is the input to the lookup table. 8. The method of claim 7, further comprising: The lookup table is determined from multiple lookup tables. 9. The method of claim 1, wherein performing the one or more filtering operations on the decoded block of the video data comprises: The deblocking filter is applied to the decoded block of video data using the effective quantization parameters. 10. The method of claim 1, wherein performing the one or more filtering operations on the decoded block of the video data comprises: The bidirectional filter is applied to the decoded block of video data using the effective quantization parameters. 11. A method for encoding video data, the method comprising: Determine the basic quantization parameters for the blocks of the video data; The quantization parameter offset for the block of video data is determined based on the content of the block. Add the quantization parameter offset to the base quantization parameter to establish an effective quantization parameter; and The block of the video data is encoded using the effective quantization parameters and the basic quantization parameters. 12. The method of claim 11, wherein the basic quantization parameters are the same for all of the blocks of the video data. 13. The method of claim 11, wherein the sample values of the video data are defined by a high dynamic range video data color format. 14. The method of claim 11, wherein the block encoding the video data comprises: Predict the block to generate residual samples; The residual samples are transformed to establish transformation coefficients; Quantize the transformation coefficients using the effective quantization parameters; The quantized transform coefficients are dequantized using the effective quantization parameters to generate distortion transform coefficients. The distortion transformation coefficients are inversely transformed to generate distorted residual samples; Transform the distorted residual sample; and The transformed distortion residual sample is quantized using the aforementioned basic quantization parameters. 15. The method of claim 11, wherein determining the quantization parameter offset comprises determining the quantization parameter offset from a lookup table. 16. An apparatus configured to decode video data, the apparatus comprising: A memory configured to store encoded blocks of the video data; and One or more processors communicating with the memory, the one or more processors being configured to: The encoded block that receives the video data; Determine the basic quantization parameters used to encode the coded blocks of the video data; The encoded blocks of the video data are decoded using the basic quantization parameters to establish decoded blocks of the video data; Based on the content of the decoded block of the video data, an estimate of the quantization parameter offset for the decoded block of the video data is determined; The estimated quantization parameter offset is added to the base quantization parameter to establish an estimate of the effective quantization parameter; and Based on the estimate of the effective quantization parameters, one or more filtering operations are performed on the decoded block of the video data. 17. The device of claim 16, wherein the basic quantization parameters are the same for all of the blocks of the video data. 18. The device of claim 16, wherein the sample values of the video data are defined by a high dynamic range video data color format. 19. The device of claim 16, wherein, for determining the fundamental quantization parameters, the one or more processors are further configured to... The basic quantization parameter syntax element is received in the encoded video bitstream, and the value of the basic quantization parameter syntax element indicates the basic quantization parameter. 20. The apparatus of claim 16, wherein, for decoding the block of video data, the one or more processors are further configured to: Entropy decoding is performed on the coded blocks of the video data to execute quantized transform coefficients; The quantized transform coefficients are dequantized using the basic quantization parameters to establish transform coefficients; Perform an inverse transformation on the transformation coefficients to establish residual values; and A prediction process is performed on the residual values to construct the decoded block of the video data. 21. The apparatus of claim 16, wherein, in order to determine the estimate of the quantization parameter offset for the decoded block of the video data, the one or more processors are further configured to: Determine the average value of the sample values of the decoded blocks of the video data; and The average value of the sample values of the decoded block of the video data is used to determine the estimate of the quantization parameter offset for the decoded block of the video data. 22. The device of claim 21, wherein, for determining the estimate of the quantization parameter offset, the one or more processors are further configured to: The estimate of the quantization parameter offset is determined from a lookup table, wherein the average of the sample values is the input to the lookup table. 23. The device of claim 21, wherein the one or more processors are further configured to: The lookup table is determined from multiple lookup tables. 24. The apparatus of claim 16, wherein, for performing the one or more filtering operations on the decoded block of the video data, the one or more processors are further configured to: The deblocking filter is applied to the decoded block of video data using the effective quantization parameters. 25. The apparatus of claim 16, wherein, for performing the one or more filtering operations on the decoded block of the video data, the one or more processors are further configured to: The bidirectional filter is applied to the decoded block of video data using the effective quantization parameters. 26. An apparatus configured to encode video data, the apparatus comprising: A memory, configured to store blocks of the video data; and One or more processors communicating with the memory, the one or more processors being configured to: Determine the underlying quantization parameters for the block of the video data; The quantization parameter offset for the block of video data is determined based on the content of the block. Add the quantization parameter offset to the base quantization parameter to establish an effective quantization parameter; and The block of the video data is encoded using the effective quantization parameters and the basic quantization parameters. 27. The device of claim 26, wherein the basic quantization parameters are the same for all of the blocks of the video data. 28. The device of claim 26, wherein the sample values of the video data are defined by a high dynamic range video data color format. 29. The apparatus of claim 26, wherein, for the block encoding the video data, the one or more processors are further configured to: Predict the block to generate residual samples; The residual samples are transformed to establish transformation coefficients; Quantize the transformation coefficients using the effective quantization parameters; The quantized transform coefficients are dequantized using the effective quantization parameters to generate distortion transform coefficients. The distortion transformation coefficients are inversely transformed to generate distorted residual samples; Transform the distorted residual sample; and The transformed distortion residual sample is quantized using the aforementioned basic quantization parameters. 30. The device of claim 26, wherein, in order to determine the quantization parameter offset, the one or more processors are further configured to determine the quantization parameter offset from a lookup table. 31. An apparatus configured to decode video data, the apparatus comprising: A means for receiving coded blocks of the video data; A means for determining the basic quantization parameters of the coded block used to encode the video data; A means for decoding the coded blocks of the video data using the basic quantization parameters to construct decoded blocks of the video data; A means for determining an estimate of the quantization parameter offset for the decoded block of the video data based on the content of the decoded block of the video data; A means for adding the estimate of the quantization parameter offset to the base quantization parameters to establish an estimate of the effective quantization parameters; and A means for performing one or more filtering operations on the decoded block of the video data based on the estimate of the effective quantization parameters. 32. An apparatus configured to encode video data, the apparatus comprising: A means for determining the basic quantization parameters of blocks of the video data; A means for determining a quantization parameter offset for a block of video data based on the content of the block; A means for adding the quantization parameter offset to the base quantization parameters to establish effective quantization parameters; and A means for encoding the block of video data using the effective quantization parameters and the basic quantization parameters. 33. A non-transitory computer-readable storage medium storing instructions, which, when executed, cause one or more processors to: Received encoded blocks of video data; Determine the basic quantization parameters for the coded block used to encode the video data; The encoded blocks of the video data are decoded using the basic quantization parameters to establish decoded blocks of the video data; Based on the content of the decoded block of the video data, an estimate of the quantization parameter offset for the decoded block of the video data is determined; The estimated quantization parameter offset is added to the base quantization parameter to establish an estimate of the effective quantization parameter; and Based on the estimate of the effective quantization parameters, one or more filtering operations are performed on the decoded block of the video data. 34. A non-transitory computer-readable storage medium storing instructions, which, when executed, cause one or more processors to: Determine the basic quantization parameters for the blocks of video data; The quantization parameter offset for the block of video data is determined based on the content of the block. Add the quantization parameter offset to the base quantization parameter to establish an effective quantization parameter; and The block of the video data is encoded using the effective quantization parameters and the basic quantization parameters.