HK1224117B - Method and device of decoding and encoding video data and computer-readable storage medium - Google Patents

Description

Method and apparatus for decoding and encoding video data and computer-readable storage medium

Technical Field

This disclosure relates to video coding, and more particularly to techniques for coding residual values for depth data in a three-dimensional (3D) video coding process.

Background Art

Digital video capabilities can be incorporated into a wide range of devices, including digital televisions, digital live broadcast systems, wireless broadcast systems, tablet computers, smartphones, personal digital assistants (PDAs), laptop or desktop computers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite radio telephones, video teleconferencing devices, set-top box devices, and the like. Digital video devices implement video compression techniques, such as those described in the standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264/MPEG-4 Part 10 Advanced Video Coding (AVC), the High Efficiency Video Coding (HEVC) standard, and extensions of such standards. Video devices can more efficiently transmit, receive, and store digital video information.

An encoder-decoder (CODEC) applies video compression techniques to perform spatial (intra-picture) prediction and/or temporal (inter-picture) prediction to reduce or remove redundancy inherent in video sequences. For block-based video coding, a video slice may be partitioned into video blocks, which may also be referred to as treeblocks, coding units (CUs), and/or coding nodes. Video blocks in an intra-coded (I) slice of a picture are encoded using spatial prediction with respect to reference samples in neighboring blocks in the same picture. Video blocks in an inter-coded (P or B) slice of a picture may use spatial prediction with respect to reference samples in neighboring blocks in the same picture or temporal prediction with respect to reference samples in other reference pictures. A picture may alternatively be referred to as a frame.

Spatial or temporal prediction results in a predictive block for the block to be coded. Residual data represents the pixel differences between the original block to be coded and the predictive block. An inter-coded block is encoded based on a motion vector pointing to a block of reference samples forming the prediction block and residual data indicating the difference between the coded block and the prediction block. An intra-coded block is encoded based on an intra-coding mode and the residual data. For further compression, the residual data can be transformed from the spatial domain to the transform domain, thereby generating residual transform coefficients, which can then be quantized. The quantized transform coefficients, initially arranged in a two-dimensional array, can be scanned to generate a one-dimensional vector of transform coefficients, and entropy coding can be applied to achieve even more compression.

A multi-view coding bitstream may be generated, for example, by encoding views from multiple perspectives. Multi-view coding may allow a decoder to select different views, or possibly reproduce multiple views. In addition, some three-dimensional (3D) video technologies and standards that have been developed or are under development utilize aspects of multi-view coding. For example, in some 3D video coding processes, different views may be used to transmit left-eye and right-eye views to support 3D video. Other 3D video coding processes may use multi-view plus depth coding. In a multi-view plus depth coding process, such as that defined by the 3D-HEVC extension of HEVC, a 3D video bitstream may contain multiple views that include not only texture view components but also depth view components. For example, a given view may include a texture view component and a depth view component. The texture view and depth view components may be used to construct 3D video data.

Summary of the Invention

In general, this disclosure describes techniques for signaling and processing information indicating the use of simplified depth coding (SDC) for depth intra prediction and depth inter prediction modes in a 3D video coding process, such as the process defined by the 3D-HEVC extension to HEVC. In some examples, this disclosure describes techniques for unifying the signaling of the use of SDC for depth intra prediction and depth inter prediction modes in 3D video coding, such as by using one syntax element to indicate the use of SDC for both depth intra and depth inter prediction.

The signaling of SDC can be unified so that a video encoder or video decoder (i.e., a video coder) uses the same syntax element instead of separate syntax elements to signal SDC for both deep intra prediction mode and deep inter prediction mode. For example, a single syntax element can be signaled at the CU level to indicate whether SDC is used for a CU, and whether the CU is intra predicted or inter predicted.

Furthermore, in some examples, the video coder may use the same syntax structure, the same type of syntax structure, or different instances of the same syntax structure for both deep intra prediction mode and deep inter prediction mode to signal and/or process residual values generated in SDC mode. In each case, the same syntax elements may be used to signal residual values, such as DC residual values. In some examples, the syntax elements used to signal SDC for deep inter prediction mode may be part of those used to signal SDC for deep intra prediction mode. In this case, common syntax elements may be shared by deep intra and deep inter SDC modes.

For shared syntax elements, context models, and binarization processes in the syntax structure for residual values generated by SDC, such as for a context-adaptive binary arithmetic coding (CABAC) entropy coding process, the relevant syntax elements for both deep intra prediction and deep inter prediction modes can be unified. For example, syntax elements for residual values generated by SDC can be coded using the same context model and/or binarization process for both deep intra prediction mode and deep inter prediction mode.

In some examples, the video coder may apply a constraint such that pulse code modulation (PCM) coding is disabled when SDC is enabled. Also, in some examples, the video coder may apply a constraint such that SDC applies to both deep intra prediction and deep inter prediction modes only for coding units with specific partition sizes.

Furthermore, in some examples, a single syntax element may be used to signal whether SDC is enabled or disabled for both deep intra prediction mode and deep inter prediction mode for CUs in an entire coded video sequence or slice. The syntax element may be presented, for example, in a video parameter set (VPS) extension, a sequence parameter set (SPS), a picture parameter set (PPS), a slice segment header, or another data structure used to signal video coding syntax information. As an example, if this syntax element indicates that SDC is enabled for a sequence, another syntax element may be signaled at the CU level for the CUs in the sequence to indicate whether SDC is used in each respective CU. If this syntax element indicates that SDC is disabled for the sequence, no syntax element needs to be signaled at the CU level for the CUs in the sequence because SDC is disabled. Alternatively, a syntax element may be provided to enable or disable the use of SDC for CUs in pictures of a slice.

In one example, the present invention provides a method for decoding video data, the method comprising: receiving a syntax element indicating whether a simplified depth coding (SDC) mode is used for both intra prediction and inter prediction of a depth coding unit (CU) of the video data; when the depth CU is intra predicted, performing intra prediction to generate a predicted depth CU; when the depth CU is inter predicted, performing inter prediction to generate the predicted depth CU; when the syntax element indicates that the SDC mode is used for each partition of a prediction unit (PU) of the depth CU, receiving information representing at least one DC residual value of the depth CU, wherein the at least one DC residual value represents a pixel difference between the partition of the PU of the depth CU and a corresponding partition of the predicted depth CU; and reconstructing the depth CU using the at least one DC residual value and the predicted depth CU.

In another example, the present invention provides a method for encoding video data, the method comprising: when a depth CU is intra-predicted, performing intra-prediction to generate a predicted depth CU; when the depth CU is inter-predicted, performing inter-prediction to generate the predicted depth CU; when a simplified depth coding (SDC) mode is used, for each partition of a prediction unit (PU) of the depth CU, generating information representing at least one DC residual value of the depth CU, wherein the at least one DC residual value represents a pixel difference between the partition of the PU of the depth CU and a corresponding partition of the predicted depth CU; when the SDC mode is used, generating a syntax element indicating that the SDC mode is used for both intra-prediction and inter-prediction of a depth coding unit (CU) of the video data; and encoding the depth CU in the video encoder based on the information representing the at least one DC residual value and the syntax element.

In another example, the present invention provides a video decoder comprising: a memory that stores video data; and one or more processors configured to: decode a syntax element indicating whether a simplified depth coding (SDC) mode is used for both intra-frame prediction and inter-frame prediction of a depth coding unit (CU) of the video data; when the depth CU is intra-frame predicted, perform intra-frame prediction to generate a predicted depth CU; when the depth CU is inter-frame predicted, perform inter-frame prediction to generate the predicted depth CU; and when the SDC mode is used, decode information representing at least one DC residual value of the prediction unit (PU) of the depth CU for each partition of the depth CU, wherein the at least one DC residual value represents a pixel difference between the partition of the PU of the depth CU and a corresponding partition of the predicted depth CU.

In another example, the present invention provides a method for decoding video data, the method comprising: when a depth coding unit (CU) of the video data is intra-predicted, performing intra-prediction to generate a predicted depth CU; when the depth CU is inter-predicted, performing inter-prediction to generate the predicted depth CU; when a simplified depth coding (SDC) mode is used, obtaining a syntax structure including one or more syntax elements indicating information of at least one DC residual value of a partition of a prediction unit (PU) representing the depth CU, wherein the syntax elements of the syntax structure are the same for the intra-prediction and the inter-prediction; and reconstructing the depth CU using the at least one DC residual value and the predicted depth CU.

In another example, the present invention provides a method for encoding video data, the method comprising: when a depth coding unit (CU) of the video data is intra-predicted, performing intra-prediction to generate a predicted depth CU; when the depth CU is inter-predicted, performing inter-prediction to generate the predicted depth CU; when a simplified depth coding (SDC) mode is used, decoding a syntax structure including one or more syntax elements indicating information of at least one DC residual value of a partition of a prediction unit (PU) representing the depth CU, wherein the syntax elements of the syntax structure are the same for the intra-prediction and the inter-prediction; and encoding the depth CU based on the information representing the at least one DC residual value.

In another example, the present invention provides a video decoder comprising: a memory that stores video data; when a depth coding unit (CU) of the video data is intra-predicted, performing intra-prediction to generate a predicted depth CU of the video data; when the depth CU is inter-predicted, performing inter-prediction to generate the predicted depth CU of the video data; and when a simplified depth coding (SDC) mode is used, decoding a syntax structure including one or more syntax elements indicating information of at least one DC residual value of a partition of a prediction unit (PU) representing the depth CU, wherein the syntax elements of the syntax structure are the same for the intra prediction mode and the inter prediction mode.

In another example, the present invention provides a method for decoding video data, the method comprising: receiving a first syntax element indicating whether a simplified depth coding (SDC) mode is enabled for depth coding units (CUs) of an entire sequence of coded video data; receiving a second syntax element indicating whether the SDC mode is used for both intra prediction and inter prediction of one of the depth CUs of the sequence; when the one of the depth CUs of the sequence is intra predicted, performing intra prediction to generate a predicted depth CU; when the one of the depth CUs of the sequence is inter predicted, performing inter prediction to generate the predicted depth CU; when the second syntax element indicates the use of the SDC mode, obtaining information representing at least one DC residual value of a partition of a prediction unit (PU) of the one of the depth CUs of the sequence; and reconstructing the partition of the PU of the one of the depth CUs of the sequence using the at least one DC residual value and the corresponding predicted depth CU.

In another example, the present invention provides a method for encoding video data, the method comprising: encoding a syntax element indicating whether a simplified depth coding (SDC) mode is enabled for depth coding units (CUs) of an entire sequence of coded video data; encoding a second syntax element indicating whether the SDC mode is used for both intra-prediction and inter-prediction of one of the depth CUs of the sequence; when one of the depth CUs of the sequence is intra-predicted, performing intra-prediction to generate a predicted depth CU; when one of the depth CUs of the sequence is inter-predicted, performing inter-prediction to generate the predicted depth CU; and when the second syntax element indicates that the SDC mode is used, encoding information representing at least one DC residual value of a partition of a prediction unit (PU) for the one of the depth CUs of the sequence.

In another example, the present invention provides a video decoder comprising: a memory that stores video data; and one or more processors configured to: decode a syntax element indicating whether a simplified depth coding (SDC) mode is enabled for depth coding units (CUs) of an entire code sequence of the video data; decode a second syntax element indicating whether the SDC mode is used for both intra-frame prediction and inter-frame prediction of one of the depth CUs of the sequence; when one of the depth CUs of the sequence is intra-frame predicted, perform intra-frame prediction to generate a predicted depth CU; when one of the depth CUs of the sequence is inter-frame predicted, perform inter-frame prediction to generate the predicted depth CU; and when the second syntax element indicates the use of the SDC mode, decode information representing at least one DC residual value of a partition of a prediction unit (PU) for the one of the depth CUs of the sequence.

The details of one or more aspects of the invention are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the techniques described in this disclosure will be apparent from the description and drawings, and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

1 is a block diagram illustrating an example video coding system that may utilize the techniques of this disclosure.

2 is a diagram illustrating intra prediction modes used in High Efficiency Video Coding (HEVC).

3 is a diagram illustrating an example of one Wedgelet partitioning mode for coding an 8x8 block of pixel samples.

4 is a diagram illustrating an example of one silhouette partition mode for coding an 8x8 block of pixel samples.

5 is a block diagram illustrating an example video encoder that may implement the techniques of this disclosure.

6 is a block diagram illustrating an example video decoder that may implement the techniques of this disclosure.

7 is a flow diagram illustrating encoding syntax elements to indicate SDC usage for both depth intra prediction and depth inter prediction modes for a CU.

8 is a flow diagram illustrating use of SDC to decode syntax elements to indicate both depth intra prediction and depth inter prediction modes for a CU.

9 is a flowchart illustrating decoding a syntax element to indicate whether SDC is enabled or disabled for depth CUs in an entire encoded video sequence.

10 is a flow diagram illustrating decoding a single syntax structure to obtain SDC residual data for PU partitions of a CU for intra prediction and inter prediction modes.

11 is a flow diagram illustrating the use of example constraints on SDC coding at the decoder side.

12 is a flow diagram illustrating the use of another example constraint on SDC coding at the decoder side.

DETAILED DESCRIPTION

This disclosure relates to depth coding modes in 3D video coding processes, such as the 3D-HEVC extension of HEVC. More specifically, this disclosure describes techniques in which the signaling of SDC applied to depth intra prediction and depth inter prediction modes is unified. In some cases, SDC may alternatively be referred to as simplified residual coding or slice-by-slice DC coding. Simplified depth coding, simplified residual coding, and slice-by-slice DC coding will be referred to herein as simplified depth coding (SDC). In any case, as described in this disclosure, SDC can be applied to depth intra and depth inter coding through a unified signaling technique.

In 3D-HEVC, both intra and inter SDC modes are used. However, in the current design of SDC, different syntax elements and context models are used to indicate the use of intra and inter SDC in 3D-HEVC, which makes the parsing process of decoding units more complicated. In some examples, the techniques described in this disclosure can partially address this issue by unifying the syntax and context for intra and inter SDC.

In SDC, in general, the video encoder encodes only one residual for each partition of a prediction unit (PU) of a depth coding unit (CU). Thus, in SDC, the video encoder may encode one residual value (e.g., a DC residual value) for a set of pixels associated with the partition of the PU. A PU may belong to an intra-coded depth CU or an inter-coded depth CU.

For each partition of a PU, instead of coding the difference for each pixel in the partition, video encoder 20 may determine the difference between the mean of the original signal (i.e., the mean of the pixels in the partition of one block to be coded) and the mean of the prediction signal (i.e., the mean of selected pixel samples in the corresponding partition of the predictive block), use this difference as the DC residual for all pixels in the PU, and signal this residual value to the video decoder.

Depth values can optionally be mapped to indices using a depth lookup table (DLT), which can be constructed by analyzing frames in the first intra period before encoding the full video sequence. If a DLT is used, the DLT can be transmitted from the video encoder to the video decoder, for example, in a sequence parameter set (SPS) or other syntax structure, and the decoded index value can be mapped back to the depth value by the video decoder based on the DLT. By using the DLT, further coding gains can be observed.

Intra- and inter-prediction mode signaling can be performed at the coding unit (CU) level. Similarly, the use of SDC mode can be signaled at the CU level. However, when a CU (intra- or inter-coded) is coded in SDC mode, it can be split into several prediction units (PUs). Each PU can be further split into several partitions. For SDC mode, a DC residual value can be signaled for each partition of each PU of the CU. That is, multiple DC residual values can be provided for multiple corresponding partitions of the PUs in the CU.

For signaling of the residual in SDC mode as described above, for each partition, the video encoder may signal in the encoded bitstream the difference between the representative value of the current partition and its predictor. In some examples, the difference may be signaled without transforming and quantizing the difference. Two different methods may be used to signal the residual, depending on whether DLT is used. When DLT is not used, the difference between the representative values of the current partition and its predictor is transmitted directly (e.g., as a DC residual value) with a sign. When DLT is used, rather than directly signaling the difference in depth values, the difference in DLT indices may be signaled, i.e., the difference between the DLT index of the representative value of the current partition and the DLT index of the predictor.

In existing designs for SDC, different syntax elements and context models may be used to indicate the use of SDC for depth intra prediction mode and depth inter prediction mode in 3D-HEVC. In 3D-HEVC, the use of different syntax elements to indicate the use of SDC for depth intra prediction mode and depth inter prediction mode makes the parsing process for coding units (CUs) more complex at the decoder side.

As discussed above, intra- and inter-prediction mode signaling may be performed for each depth CU, while also signaling an indication of the use of SDC mode at the CU level. According to various examples, this disclosure describes techniques for unifying the signaling of SDC for depth intra prediction and depth inter prediction modes in a 3D video coding process, such as 3D-HEVC. Rather than separately signaling the use of SDC for intra-predicted and inter-predicted CUs, a single SDC syntax element may be provided to indicate the use of SDC for a CU, regardless of whether the CU is intra- or inter-predicted. The techniques may be performed by a video encoder and/or a video decoder (i.e., a video decoder) to unify the signaling of SDC for depth intra prediction and depth inter prediction modes in 3D video coding. According to this disclosure, the signaling of SDC for depth intra and depth inter prediction modes in a 3D video coding process may be unified, which may be referred to herein as simplified depth coding, simplified residual coding, or slice-by-slice DC coding. Various aspects of the techniques for unifying SDC signaling are described below.

In some examples, in depth coding, the video coder may use only one syntax element (e.g., sdc_flag) to indicate the use of SDC for depth intra or depth inter mode. For example, the video encoder may use the same SDC syntax element (e.g., sdc_flag) to signal the use of SDC for both depth intra prediction and depth inter prediction modes. The video decoder may process the same syntax element (e.g., sdc_flag) to determine whether SDC will be used for depth intra prediction and depth inter prediction modes. If the syntax element indicates that SDC will be used, then SDC will be used for the CU for depth intra prediction or depth inter prediction. In this way, it is not necessary to signal different syntax elements to indicate the use of SDC for depth intra prediction and depth inter prediction, respectively. Instead, a single syntax element indicates that SDC use is used for the CU regardless of which mode (intra or inter). Signaling one syntax element for both intra and inter prediction can reduce signaling overhead, improve computational efficiency, and potentially reduce memory requirements.

For example, a video coder can use the same syntax element to signal the use of SDC for both deep intra prediction and deep inter prediction, rather than using different syntax elements for separately signaling the use of SDC for deep intra prediction and deep inter prediction. Thus, if a single syntax element such as sdc_flag is used, a separate syntax element such as inter_sdc_flag indicating that the current coding unit (CU) utilizes SDC for deep inter prediction mode can be removed. Similarly, the syntax element indicating that the current CU utilizes SDC for deep intra prediction mode can also be removed. Thus, a new syntax element, such as a flag such as sdc_flag, is introduced, regardless of whether the current CU is coded in deep intra prediction mode or deep inter prediction mode. When this flag is equal to 1, SDC is enabled for both deep intra prediction mode or deep inter prediction.

Furthermore, in some examples, in depth coding, when SDC is used to code the current depth CU, for each partition, the DC residual information representing the DC residual value for each partition of each PU of a depth inter-coded CU or a depth intra-coded CU may be unified in the sense that it may be presented by one SDC residual syntax structure, either directly or by using a DLT index value, wherein the SDC residual syntax structure includes information indicating the absolute value and sign of the DC residual value. For example, the video coder may use the same syntax structure or the same type of syntax structure to store and/or retrieve residual values generated in SDC mode for partitions of PUs in a CU coded in depth intra prediction or depth inter prediction mode. For example, for both depth intra prediction and depth inter prediction modes, the video encoder may generate a syntax structure containing residual values for partitions of PUs of the depth CU, and the video decoder may obtain the residual values for the partitions of PUs of the depth CU from this syntax structure. Thus, the syntax elements and syntax structures used to signal SDC may be unified for depth intra and inter coding.

Alternatively or additionally, if the depth CU is intra-coded and utilizes SDC, a flag (equal to 1) indicating whether the current CU contains any non-zero DC residual values in any of its partitions may be further signaled by the video encoder and processed by the video decoder. If this flag is 0, the video encoder does not signal a DC residual value, and the video decoder infers that the DC residual value is equal to zero for each partition.

As another alternative or in addition, a flag indicating whether the current CU contains any non-zero DC residual values in any of its PU partitions may be applied to both intra-coded and inter-coded CUs. Specifically, if the flag is equal to 1, the video decoder determines that non-zero DC residual values exist for both intra-coded and inter-coded CUs, as applicable. Thus, this flag may be used with the same syntax structure or the same type of syntax structure as described above for presenting DC residual values for partitions of PUs of an intra-coded CU or an inter-coded CU.

As another alternative or addition, this flag indicating whether the current CU contains any non-zero DC residual value in any of its partitions may not be used with the syntax structure used to present DC residual values for intra-coded CUs and inter-coded CUs. In this case, the same syntax structure or the same type of syntax structure may be used for intra-coded CUs and inter-coded CUs, as described above, but the DC residual value is signaled without the need to use a flag indicating whether the current CU contains any non-zero DC residual value in any of its partitions.

In some examples, because the syntax structure for the DC residual value of SDC in deep intra prediction and deep inter prediction modes shares the same syntax elements, the context model and binarization process used for entropy coding of related syntax elements (e.g., context-adaptive binary arithmetic coding (CABAC)) can also be unified. That is, the same context model can be used for syntax elements indicating that SDC uses or includes the DC residual value for both deep intra prediction and deep inter prediction modes of a CU. Therefore, using the same syntax elements and/or syntax structure for residual values generated by SDC for both deep intra and inter coding modes can facilitate reduced computational complexity in entropy coding, in particular. For example, syntax elements used to signal SDC can be coded using the same context model and/or binarization process for both deep intra prediction and deep inter prediction modes. Furthermore, the syntax structure carrying syntax elements indicating information representing residual values generated by SDC can also be entropy coded using the same context model and/or binarization process for intra and inter prediction. Using the same context model for depth intra prediction and depth inter prediction modes, such as in 3D-HEVC coding, may reduce the computational burden and memory requirements associated with entropy coding, especially at the decoder side.

As another example, even though SDC signaling for deep intra mode and deep inter mode may use the same syntax structure for the DC residual value, the video encoder and decoder may be configured to use different instances of the same syntax structure for deep intra coding and deep inter coding, respectively. Each instance of the syntax structure may include the same syntax elements, or the same type of syntax elements and the same structure, to signal SDC usage and DC residual values, but an instance of the syntax structure may signal the values of these syntax elements separately for deep intra prediction and deep inter prediction modes, e.g., one instance of the syntax structure for deep intra prediction and one instance of the syntax structure for deep inter prediction. Thus, in this example, the video encoder and/or video decoder may use the same context and/or binarization process for entropy coding (e.g., CABAC coding) for different instances of the same syntax structure, or may maintain the context and/or binarization process for entropy coding (e.g., CABAC coding) for different instances of the syntax structure separately for SDC usage in deep intra prediction and deep inter prediction modes. In some examples, SDC usage for CUs in intra or inter mode may be signaled by flag sdc_flag.In the syntax structure indicating the DC residual for each partition, in some examples, the DC residual value may be signaled by a magnitude indicated by depth_dc_abs and a sign indicated by depth_dc_sign_flag.

In some examples according to the present disclosure, a video decoder may be configured to apply a constraint such that pulse code modulation (PCM) coding is disabled when SDC is used. For example, a constraint may be applied at the decoder side such that, for example, pcm_flag will not be equal to 1 when sdc_flag is equal to 1 for a given depth CU. When SDC is enabled for both depth intra and depth inter prediction modes of a CU, PCM is not used for that CU. Thus, this constraint prevents the decoder from enabling PCM when the syntax element indicating the use of SDC (sdc_flag) is equal to 1.

Furthermore, in some examples, the video decoder may apply constraints to SDC such that SDC is used for both deep intra prediction and deep inter prediction modes only for coding units with a specific partition size or a specific range of partition sizes. For example, the video decoder may apply constraints to SDC at the decoder side such that both intra SDC mode and inter SDC mode are used only for CUs with a 2Nx2N partition size. As an alternative, the video decoder may apply constraints such that intra SDC mode is used for CUs with 2Nx2N and NxN partition sizes. Thus, in one example, SDC can be used for deep intra prediction and deep inter prediction only for a 2Nx2N partition size. In another example, SDC can be used for deep inter prediction only for a 2Nx2N partition size, but SDC can be used for deep intra prediction only for 2Nx2N and NxN partition sizes.

In some examples, a video encoder and/or video decoder may be configured to use a single syntax element, such as a one-bit flag or a multi-bit syntax element, to enable and disable SDC for both deep intra prediction and deep inter prediction modes during the decoding process, for example, for the entire coded video sequence. For example, this syntax element may be coded in a video parameter set (VPS) extension, a sequence parameter set (SPS), a picture parameter set (PPS), a slice segment header, or another data structure to indicate whether SDC is enabled for both deep intra prediction and deep inter prediction modes. In this example, the single syntax element indicates whether SDC is enabled for both deep intra and deep inter modes for the coded video sequence. In another example, as an alternative, the video encoder and/or video decoder may code one flag to indicate whether SDC is enabled for deep intra mode for the entire coded video sequence and, if so, additionally code another flag to indicate whether SDC is enabled for deep inter mode for the entire coded video sequence or the entire slice. In this example, one syntax element is used to enable or disable SDC for a coded video sequence. If SDC is enabled, the encoder may signal another syntax element (e.g., sdc_flag) to indicate the use of SDC at the CU level for CUs within the coded video sequence. If SDC is disabled, the encoder does not signal a syntax element indicating the use of SDC at the CU level, and the decoder does not receive this syntax element.

In some examples, a flag (e.g., SdcEnableFlag) may be provided to indicate whether SDC is enabled. The flag may have a value that varies with the corresponding value in the VPS, SPS, PPS, or slice header. Examples of VPS syntax elements that may partially determine the value of SdcEnableFlag include vps_inter_sdc_flag and vps_depth_modes_flag. A syntax element (sdc_flag) may then indicate whether SDC is used for a CU. Examples of these syntax elements are provided below. Similar syntax elements may be provided in the VPS, SPS, PPS, or slice segment header to enable and disable SDC for an entire coded video sequence, picture, or slice, as applicable.

1 is a block diagram illustrating an example video encoding and decoding system 10 that may be configured to utilize various techniques of this disclosure for signaling SDCs for depth intra prediction mode and depth inter prediction mode, as described in this disclosure. In some examples, video encoder 20 and/or video decoder 30 of system 10 may be configured to perform techniques for unifying the signaling of SDCs for depth intra prediction and depth inter prediction modes in a 3D video coding process, such as 3D-HEVC.

1 , system 10 includes a source device 12 that provides encoded video data to be decoded at a later time by a destination device 14. Specifically, source device 12 provides the video data to destination device 14 via a computer-readable medium 16. Source device 12 and destination device 14 may comprise any of a wide range of devices, including desktop computers, notebook (i.e., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, so-called "smart" pads, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, or the like. In some cases, source device 12 and destination device 14 may be equipped for wireless communication.

Destination device 14 may receive the encoded video data to be decoded via computer-readable medium 16. Computer-readable medium 16 may comprise any type of medium or device capable of moving the encoded video data from source device 12 to destination device 14. In one example, computer-readable medium 16 may comprise a communication medium, such as a transmission channel, to enable source device 12 to transmit encoded video data directly to destination device 14 in real-time.

The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to destination device 14. The communication medium may comprise any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet network, such as a local area network, a wide area network, or a global network, such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that can be used to facilitate communication from source device 12 to destination device 14.

In some examples, the encoded data may be output from output interface 22 to a computer-readable storage medium, such as a non-transitory computer-readable storage medium, i.e., a data storage device. Similarly, the encoded data may be accessed from a storage device by input interface 22. The storage device may include any of a variety of distributed or locally accessed non-transitory data storage media, such as a hard drive, a Blu-ray disc, a DVD, a CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In another example, the storage device may correspond to a file server or another intermediate storage device that can store the encoded video generated by source device 12.

Destination device 14 may access the stored video data from the storage device via streaming or downloading. The file server may be any type of server capable of storing and transmitting the encoded video data to destination device 14. Example file servers include a web server (e.g., for a website), an FTP server, a network attached storage (NAS) device, or a local disk drive. Destination device 14 may access the encoded video data through any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., DSL, cable modem, etc.), or a combination of both suitable for accessing encoded video data stored on a file server. The transmission of the encoded video data from the storage device may be a streaming transmission, a download transmission, or a combination thereof.

The techniques of this disclosure are not necessarily limited to wireless applications or settings. The techniques may be applied to video coding to support any of a variety of multimedia applications, such as over-the-air television broadcasts, cable television transmissions, satellite television transmissions, Internet streaming video transmissions (e.g., Dynamic Adaptive Streaming over HTTP (DASH)), digital video encoded onto data storage media, decoding of digital video stored on data storage media, or other applications. In some examples, system 10 may be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting, and/or video telephony.

1 , source device 12 includes a video source 18, a video encoder 20, and an output interface 22. Destination device 14 includes an input interface 28, a video decoder 30, and a display device 32. According to this disclosure, video encoder 20 of source device 12 may be configured to apply partition-based depth coding techniques with respect to non-rectangular partitions. In other examples, the source and destination devices may include other components or arrangements. For example, source device 12 may receive video data from an external video source 18, such as an external camera. Likewise, destination device 14 may interface with an external display device rather than including an integrated display device.

The illustrated system 10 of FIG. 1 is merely one example. The techniques for SDC for deep intra prediction mode may be performed by any digital video encoding and/or decoding device. Although the techniques of this disclosure are generally performed by video encoder 20 and/or video decoder 30, the techniques may also be performed by a video encoder/decoder (often referred to as a "codec"). Furthermore, the techniques of this disclosure may also be performed by a video preprocessor. Source device 12 and destination device 14 are merely examples of such coding devices in which source device 12 generates coded video data for transmission to destination device 14. In some examples, devices 12, 14 may operate in a substantially symmetrical manner, such that each of devices 12, 14 includes video encoding and decoding components. Thus, system 10 may support one-way or two-way video transmission between video devices 12, 14, for example, for video streaming, video playback, video broadcasting, or video telephony.

Video source 18 of source device 12 may include a video capture device, such as a video camera, a video archive containing previously captured video, and/or a video feed interface for receiving video from a video content provider. As another alternative, video source 18 may generate computer graphics-based data as the source video, or a combination of live video, archived video, and computer-generated video. In some cases, if video source 18 is a video camera, source device 12 and destination device 14 may form so-called smartphones, tablet computers, or video phones. However, as mentioned above, the techniques described in this disclosure may be generally applicable to video decoding and may be applied to wireless and/or wired applications. In each case, the captured, pre-captured, or computer-generated video may be encoded by video encoder 20. The encoded video information may then be output by output interface 22 to computer-readable medium 16.

Computer-readable medium 16 may include transient media, such as wireless broadcasts or wired network transmissions, or data storage media (i.e., non-transitory storage media). In some examples, a network server (not shown) may receive encoded video data from source device 12 and provide the encoded video data to destination device 14, for example, via network transmission. Similarly, a computing device at a media production facility (e.g., a disc stamping facility) may receive encoded video data from source device 12 and produce an optical disc containing the encoded video data. Thus, in various examples, computer-readable medium 16 may be understood to include one or more computer-readable media in various forms.

This disclosure may generally refer to video encoder 20 "signaling" certain information to another device, such as video decoder 30. However, it should be understood that video encoder 20 may signal information by associating certain syntax elements with various encoded portions of video data. That is, video encoder 20 may "signal" data by storing certain syntax elements in the headers or payloads of various encoded portions of video data. In some cases, such syntax elements may be encoded and stored (e.g., to computer-readable medium 16) prior to being received and decoded by video decoder 30. Thus, the term "signaling" may generally refer to the communication of syntax or other data used to decode compressed video data, whether such communication occurs in real or near real time or over a period of time, such as when syntax elements are stored on a medium at encoding time and can then be retrieved by a decoding device at any time after being stored on such medium.

Input interface 28 of destination device 14 receives information from computer-readable medium 16. The information of computer-readable medium 16 may include syntax information defined by video encoder 20, which is also used by video decoder 30, including syntax elements that describe characteristics and/or processing of blocks and other coded units (e.g., GOPs). Display device 32 displays the decoded video data to a user and may comprise any of a variety of display devices, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, a projection device, or another type of display device.

Although not shown in FIG1 , in some aspects, video encoder 20 and video decoder 30 may each be integrated with an audio encoder and decoder and may include appropriate multiplexer-demultiplexer units or other hardware and software to handle the encoding of both audio and video in a common data stream or in separate data streams. As an example, the multiplexer-demultiplexer units may conform to the ITU H.223 multiplexer protocol, or other protocols such as the User Datagram Protocol (UDP), if applicable.

Each of video encoder 20 and video decoder 30 may be implemented as any of a variety of suitable encoder or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic circuits, software, hardware, firmware, or any combination thereof. Each of video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined video encoder/decoder (CODEC). A device including video encoder 20 and/or video decoder 30 may comprise an integrated circuit, a microprocessor, and/or a wireless communication device (e.g., a cellular telephone).

Video encoder 20 and video decoder 30 may operate according to a video coding standard, such as the HEVC standard and, more particularly, the 3D-HEVC extension of the HEVC standard, as referenced in this disclosure. HEVC assumes several additional capabilities of video coding devices relative to devices configured to perform coding according to other procedures, such as ITU-T H.264/AVC. For example, while H.264 provides nine intra-prediction coding modes, HM may provide up to thirty-five intra-prediction coding modes.

In general, HEVC specifies that a video picture (or "frame") can be divided into a sequence of treeblocks or largest coding units (LCUs) that include both luma and chroma samples. Syntax data within the bitstream can define the size of the largest coding unit (LCU, which is the largest coding unit in terms of the number of pixels). A slice contains several consecutive treeblocks in coding order. A picture can be partitioned into one or more slices. Each treeblock can be split into coding units (CUs) according to a quadtree. In general, a quadtree data structure includes one node per CU, with one root node corresponding to the treeblock. If a CU is split into four sub-CUs, the node corresponding to the CU includes four leaf nodes, each of which corresponds to one of the sub-CUs.

Each node of the quadtree data structure may provide syntax data for the corresponding CU. For example, a node in the quadtree may include a split flag that indicates whether the CU corresponding to the node is split into sub-CUs. Syntax elements for a CU may be defined recursively and may depend on whether the CU is split into a number of sub-CUs. If a CU is not split further, it is referred to as a leaf-CU. The four sub-CUs of a leaf-CU may also be referred to as leaf-CUs, even when there is no explicit splitting of the original leaf-CU. For example, if a 16x16 sized CU is not split further, then the four 8x8 sub-CUs will also be referred to as leaf-CUs, even though the 16x16 CU is never split.

CUs in HEVC have similar purposes to macroblocks in the H.264 standard, except that CUs do not have size distinctions. For example, a treeblock can be split into four child nodes (also called sub-CUs), and each child node can, in turn, be a parent node and can be split into another four child nodes. The final unsplit child nodes (referred to as leaf nodes of the quadtree) comprise coding nodes, also referred to as leaf-CUs. Syntax data associated with the coded bitstream may define the maximum number of times a treeblock can be split, referred to as the maximum CU depth, and may also define the minimum size of a coding node. Thus, the bitstream may also define a smallest coding unit (SCU). This disclosure uses the term "block" to refer to any of a CU, PU, or TU in the context of HEVC, or similar data structures in the context of other standards (e.g., macroblocks and sub-blocks in H.264/AVC).

A CU includes a coding node and prediction units (PUs) and transform units (TUs) associated with the coding node. The size of the CU corresponds to the size of the coding node and must be square in shape. The size of a CU can range from 8×8 pixels up to a treeblock size with a maximum of 64×64 pixels or larger. Each CU can contain one or more PUs and one or more TUs. For example, the syntax data associated with a CU may describe the partitioning of the CU into one or more PUs. The partitioning mode may differ between whether the CU is skipped or encoded in direct mode, encoded in intra-prediction mode, or encoded in inter-prediction mode. In the case of depth coding as described in this disclosure, the PU may be partitioned into non-square shapes, or include partitions of non-rectangular shapes. For example, the syntax data associated with a CU may also describe the partitioning of the CU into one or more TUs according to a quadtree. A TU may be square or non-square (e.g., rectangular) in shape.

The HEVC standard allows for transforms based on TUs, which can be different for different CUs. The size of a TU is typically determined based on the size of the PU within a given CU defined for a partitioned LCU, but this may not always be the case. A TU is typically the same size as or smaller than a PU. In some examples, a quadtree structure called a "residual quadtree" (RQT) can be used to subdivide the residual samples corresponding to a CU into smaller units. The leaf nodes of the RQT can be called transform units (TUs). Pixel difference values associated with a TU can be transformed to produce transform coefficients, which can be quantized.

A leaf-CU may include one or more prediction units (PUs). Generally, a PU represents a spatial region corresponding to all or a portion of the corresponding CU and may include data used to retrieve reference samples for the PU. Reference samples may be pixels from a reference block. In some examples, reference samples may be obtained from the reference block or generated, for example, by interpolation or other techniques. The PU also includes data related to prediction. For example, when the PU is intra-mode encoded, data for the PU may be included in a residual quadtree (RQT), which may include data describing the intra-prediction mode used for the TU corresponding to the PU. As another example, when the PU is inter-mode encoded, the PU may include data defining one or more motion vectors for the PU. For example, the data defining the motion vector of the PU may describe the horizontal component of the motion vector, the vertical component of the motion vector, the resolution of the motion vector (e.g., quarter-pixel precision or eighth-pixel precision), the reference picture to which the motion vector points, and/or the reference picture list for the motion vector (e.g., List 0, List 1, or List C).

A leaf-CU with one or more PUs may also include one or more transform units (TUs). Transform units may be specified using an RQT (also known as a TU quadtree structure), as discussed above. For example, a split flag may indicate whether a leaf-CU is split into four transform units. Each transform unit may then be further split into more sub-TUs. When a TU is not further split, it may be referred to as a leaf-TU. In general, for intra coding, all leaf-TUs belonging to a leaf-CU share the same intra prediction mode. That is, the same intra prediction mode is generally applied to calculate prediction values for all TUs of a leaf-CU. For intra coding, video encoder 20 may use the intra prediction mode to calculate the residual value of each leaf-TU as the difference between the portion of the CU corresponding to the TU and the original block. A TU is not necessarily limited to the size of a PU. Thus, a TU may be larger or smaller than a PU. For intra coding, a PU may be collocated with a corresponding leaf-TU of the same CU. In some examples, the maximum size of a leaf-TU may correspond to the size of the corresponding leaf-CU.

Furthermore, the TUs of a leaf-CU may also be associated with corresponding quadtree data structures, referred to as residual quadtrees (RQTs). That is, a leaf-CU may include a quadtree that indicates how the leaf-CU is partitioned into TUs. The root node of a TU quadtree generally corresponds to a leaf-CU, while the root node of a CU quadtree generally corresponds to a treeblock (or LCU). TUs of an RQT that are not split are referred to as leaf-TUs. In general, unless otherwise noted, this disclosure uses the terms CU and TU to refer to leaf-CUs and leaf-TUs, respectively.

A video sequence typically includes a series of pictures. As described herein, the terms "picture" and "frame" may be used interchangeably. That is, a picture containing video data may be referred to as a video frame or simply a "frame." A group of pictures (GOP) generally includes a series of one or more video pictures. A GOP may include a header for the GOP, a header for one or more of the pictures, or syntax data elsewhere that describes the number of pictures included in the GOP. Each slice of a picture may include slice syntax data that describes the encoding mode for the corresponding slice. The video encoder 20 typically operates on video blocks within individual video slices in order to encode the video data. A video block may correspond to a decoding node within a CU. A video block may have a fixed or variable size and may differ in size according to a specified decoding standard.

As an example, HEVC supports prediction for various PU sizes. Assuming the size of a particular CU is 2Nx2N, HEVC supports intra-prediction for PU sizes of 2Nx2N or NxN, and inter-prediction for symmetric PU sizes of 2Nx2N, 2NxN, Nx2N, or NxN. A PU with a size of 2Nx2N represents an unpartitioned CU because it is the same size as the CU in which it resides. In other words, a 2Nx2N PU is the same size as its CU. HEVC also supports asymmetric partitioning for inter-prediction for PU sizes of 2NxnU, 2NxnD, nLx2N, and nRx2N. In asymmetric partitioning, one direction of the CU is not partitioned, but the other direction is partitioned into 25% and 75%. The portion of the CU corresponding to the 25% partition is indicated by an "n" followed by an indication of "up," "down," "left," or "right." Thus, for example, "2NxnU" refers to a 2Nx2N CU that is partitioned horizontally with a 2Nx0.5N PU on top and a 2Nx1.5N PU on the bottom.

In this disclosure, "NxN" and "N by N" may be used interchangeably to refer to the pixel dimensions of a video block in terms of vertical and horizontal dimensions, e.g., 16x16 pixels or 16 by 16 pixels. Generally speaking, a 16x16 block will have 16 pixels in the vertical direction (y=16) and 16 pixels in the horizontal direction (x=16). Similarly, an NxN block generally has N pixels in the vertical direction and N pixels in the horizontal direction, where N represents a non-negative integer value. The pixels in a block may be arranged in rows and columns. Furthermore, a block need not necessarily have the same number of pixels in the horizontal direction as in the vertical direction. For example, a block may comprise NxM pixels, where M is not necessarily equal to N.

After intra-predictive or inter-predictive coding of the PUs of a CU, video encoder 20 may calculate residual data for the TUs of the CU. The PU may include syntax data that describes a method or mode for generating predictive pixel data in the spatial domain (also called the pixel domain), and the TU may include coefficients in a transform domain where a transform (e.g., a discrete cosine transform (DCT), an integer transform, a wavelet transform, or a conceptually similar transform) is applied to the residual video data. The residual data may correspond to pixel differences between pixels of an unencoded picture and prediction values corresponding to the PU. Video encoder 20 may form the TUs including the residual data for the CU, and then transform the TUs to produce transform coefficients for the CU.

After performing any transforms used to generate transform coefficients, video encoder 20 may perform quantization of the transform coefficients. Quantization generally refers to the process of quantizing the transform coefficients to potentially reduce the amount of data used to represent the coefficients, thereby providing further compression. The quantization process may reduce the bit depth associated with some or all of the coefficients. For example, an n-bit value may be rounded down to an m-bit value during quantization, where n is greater than m.

After quantization, video encoder 20 may scan the transform coefficients, producing a one-dimensional vector from a two-dimensional matrix containing the quantized transform coefficients. The scan may be designed to place higher energy (and therefore lower frequency) coefficients at the front of the array and lower energy (and therefore higher frequency) coefficients at the back of the array.

In some examples, video encoder 20 may utilize a predefined scan order to scan the quantized transform coefficients to produce a serialized vector that can be entropy encoded. In other examples, video encoder 20 may perform adaptive scanning. After scanning the quantized transform coefficients to form a one-dimensional vector, video encoder 20 may encode the one-dimensional vector, for example, according to context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy encoding method. Video encoder 20 may also entropy encode syntax elements associated with the encoded video data for use by video decoder 30 when decoding the video data.

Video encoder 20 may further send syntax data, such as block-based syntax data, picture-based syntax data, and GOP-based syntax data, to video decoder 30, e.g., in a picture header, a block header, a slice header, or a GOP header. The GOP syntax data may describe a number of pictures in a corresponding GOP, and the picture syntax data may indicate a coding/prediction mode used to encode the corresponding picture.

Video encoder 20 and/or video decoder 30 may perform intra-picture prediction coding of depth data and inter-frame prediction coding of depth data. In addition, according to examples of this disclosure, video encoder 20 and/or video decoder 30 may use SDC to code residual data generated from depth intra-frame prediction coding of video data and/or depth inter-frame prediction coding of video data, e.g., according to any of a variety of examples, as will be described. More specifically, video encoder 20 and/or video decoder 30 may be configured according to the techniques of this disclosure to signal the use of SDC for depth intra-frame prediction and depth inter-frame prediction in a more unified manner in a 3D video coding process, such as 3D-HEVC.

This disclosure describes techniques for 3D video coding based on advanced codecs, such as the High Efficiency Video Coding (HEVC) codec. The 3D coding techniques described in this disclosure include depth coding techniques related to advanced inter-coding of depth views in multi-view plus depth video coding processes, such as the 3D-HEVC extension to HEVC currently under development. The following section reviews video coding standards and HEVC techniques relevant to this disclosure.

The most recent draft of the HEVC standard, JCTVC-L1003 (Benjamin Bloss, Wu-Kyung Han, Keens Ranier Ohm, Gary Sullivan, Yekui Wang, Thomas Weigand), “High Efficiency Video Coding (HEVC) Textual Specification Draft 10 (for FDIS and Last Call)” (Joint Collaboration Team on Video Coding (JCT-VC) of ITU-T SG16 WP 3 and ISO/IEC JTC 1/SC 29/WG11, 12th Meeting, Geneva, Switzerland, January 14-23, 2013 (“HEVC WD10”)) is incorporated herein by reference in its entirety and is available at the following link: http://phenix.it-sudparis.eu/jct/doc_end_user/documents/12_Geneva/wg11/JCTVC-L1003-v34.zip

A recent draft of 3D-HEVC is presented in JCTVC-F1001-v1 (Gerhard Technologies, Christoph Wegener, Ying Chen, and Xihong Ye) “3D-HEVC Draft Text 2” (Joint Collaborative Group on 3D Video Coding Extension Development of ITU-T SG16 WP 3 and ISO/IEC JTC 1/SC 29/WG11, 6th Meeting: Geneva, Switzerland, October 25-November 1, 2013 (“3D-HEVC”)), which is incorporated herein by reference in its entirety and is available at the following link: http://phenix.it-sudparis.eu/jct2/doc_end_user/documents/6_Geneva/wg11/JCT3V-F1001-v1.zip

In HEVC, assuming the size of a coding unit (CU) is 2Nx2N, video encoder 20 and video decoder 30 may support various prediction unit (PU) sizes of 2Nx2N or NxN for intra prediction, and symmetric PU sizes of 2Nx2N, 2NxN, Nx2N, NxN, or similar for inter prediction. The video encoder and video decoder may also support asymmetric partitioning of PU sizes of 2NxnU, 2NxnD, nLx2N, and nRx2N for inter prediction.

For depth coding as provided in 3D-HEVC, video encoders and video decoders can be configured to support a variety of different depth coding partitioning modes for intra prediction, including modes using non-rectangular partitions. Examples of depth coding with non-rectangular partitions include Wedgelet partition-based depth coding, contour partition-based depth coding, and region boundary chain partition-based coding. Techniques for partition-based intra coding of non-rectangular partitions, such as Wedgelet partitions or contour partitions (as examples), can be performed in conjunction with a simplified depth coding (SDC) mode for coding of residual information generated by intra prediction coding. Additionally, for conventional intra/inter modes in HEVC, the techniques can also be performed in conjunction with an SDC mode for coding of residual information generated from intra or inter prediction coding of depth data.

Video data coded using 3D video coding techniques can be reproduced and displayed to produce a three-dimensional effect. As an example, two images of different views (that is, corresponding to two camera perspectives with slightly different horizontal positions) can be displayed substantially simultaneously so that one image is seen by the viewer's left eye and the other image is seen by the viewer's right eye.

The 3D effect can be achieved using, for example, a stereoscopic display or an autostereoscopic display. A stereoscopic display can be used in conjunction with goggles that filter the two images accordingly. For example, passive glasses can filter the images using polarized lenses, differently colored lenses, or other optical filtering techniques to ensure that the appropriate eye sees the appropriate image. As another example, active glasses can rapidly block alternating lenses in coordination with a stereoscopic display that alternates between displaying a left-eye image and a right-eye image. An autostereoscopic display displays the two images in a manner that does not require glasses. For example, an autostereoscopic display can include mirrors or prisms configured to cause each image to be projected into the appropriate eye of the viewer.

The techniques of this disclosure relate to techniques for coding 3D video data by coding depth data to support 3D video. Generally, the term "texture" is used to describe the luma (i.e., brightness or "brightness") values of an image and the chroma (i.e., color or "chrominance") values of the image. In some examples, a texture image may include one set of luma data (Y) and two sets of chroma data for blue tones (Cb) and red tones (Cr). In certain chroma formats, such as 4:2:2 or 4:2:0, the chroma data is downsampled relative to the luma data. That is, the spatial resolution of a chroma pixel may be lower than the spatial resolution of the corresponding luma pixel, for example, half or one-quarter the luma resolution.

Depth data generally describes depth values for corresponding texture data. For example, a depth image may include a set of depth pixels (or depth values), each of which describes depth, for example, in a depth component of a view, for corresponding texture data in a texture component of the view. Each pixel may have one or more texture values (e.g., luminance and chrominance) and may also have one or more depth values. Texture pictures and depth maps may (but need not) have the same spatial resolution. For example, a depth map may include more or fewer pixels than a corresponding texture picture. Depth data may be used to determine horizontal disparity for corresponding texture data, and in some cases, vertical disparity may also be used.

A device receiving texture and depth data may display a first texture image for one view (e.g., a left-eye view) and modify the first texture image using the depth data to generate a second texture image for another view (e.g., a right-eye view) by offsetting pixel values of the first image by a horizontal disparity value determined based on the depth value. Generally speaking, horizontal disparity (or simply "disparity") describes the horizontal spatial offset of a pixel in the first view from a corresponding pixel in the right view, where the two pixels correspond to the same portion of the same object as represented in the two views.

In yet other examples, depth data can be defined for pixels in the z-dimension perpendicular to the image plane, such that the depth associated with a given pixel is defined relative to a zero-disparity plane defined for the image. This depth can be used to generate horizontal disparity for displaying pixels, such that the pixels appear differently to the left and right eyes depending on the z-dimension depth value of the pixels relative to the zero-disparity plane. The zero-disparity plane can change for different portions of a video sequence, and the amount of depth relative to the zero-disparity plane can also change.

Pixels located on the zero disparity plane can be similarly defined for the left and right eyes. Pixels located before the zero disparity plane can be displayed in different positions for the left and right eyes (e.g., with horizontal disparity) to create the perception that the pixels appear to be emerging from the image in the z-direction perpendicular to the image plane. Pixels located after the zero disparity plane can be displayed with a slight blur to slightly enhance the perception of depth, or can be displayed in different positions for the left and right eyes (e.g., with opposite horizontal disparity than pixels located before the zero disparity plane). Many other techniques can also be used to convey or define depth data for an image.

Two-dimensional video data is generally coded as a sequence of discrete pictures, each of which corresponds to a specific time instance. That is, each picture has an associated playback time relative to the playback times of other pictures in the sequence. These pictures can be considered texture pictures or texture images. In depth-based 3D video coding, each texture picture in the sequence can also correspond to a depth map. That is, the depth map corresponding to a texture picture describes the depth data of the corresponding texture picture. Multi-view video data may include data for various different views, where each view may include a respective sequence of texture components and corresponding depth components.

A picture generally corresponds to a specific time instance. Video data can be represented using a sequence of access units, where each access unit includes all data corresponding to a specific time instance. Thus, for example, for multi-view video data plus depth coding, the texture image from each view for a common time instance plus the depth map for each of the texture images may all be included within a particular access unit. Thus, an access unit may include multiple views, where each view may include data corresponding to a texture component of a texture image and data corresponding to a depth component of a depth map.

Each access unit may contain multiple view components or pictures. The view components of a particular view are associated with a unique view id or view order index, such that view components of different views are associated with different view ids or view order indices. View components may include texture view components and depth view components. Texture and depth view components in the same view may have different layer ids. Texture view components may be coded as one or more texture slices, while depth view components may be coded as one or more depth slices. Multi-view plus depth yields a variety of coding possibilities, such as intra-picture, inter-picture, intra-view, inter-view, motion prediction, and the like.

In this way, 3D video data can be represented using a multi-view video plus depth format, where the captured or generated views include texture components associated with corresponding depth maps. Furthermore, in 3D video coding, texture and depth maps can be coded and multiplexed into the 3D video bitstream. The depth map can be coded as a grayscale image, where the "luminance" samples (i.e., pixels) of the depth map represent depth values.

In general, a block of depth data (e.g., a block of samples of a depth map corresponding to a pixel) may be referred to as a depth block. A depth value may be referred to as a luma value associated with a depth sample. That is, a depth map can generally be considered a monochrome texture picture—in other words, a texture picture that includes luma values and no chroma values. In any case, conventional intra- and inter-coding methods can be applied to depth map coding.

Depth maps are typically characterized by sharp edges and constant regions, and edges in depth maps often exhibit strong correlation with corresponding texture data. Due to the different statistics and correlations between texture and corresponding depth, different coding schemes for depth maps have been designed based on 2D video codecs.

The following review of HEVC techniques is relevant to this disclosure. Examples of video coding standards include ITU-T H.261, ISO/IEC MPEG-1 Visual, ITU-T H.262 or ISO/IEC MPEG-2 Visual, ITU-T H.263, ISO/IEC MPEG-4 Visual, and ITU-T H.264 (also known as ISO/IEC MPEG-4 AVC), including its scalable video coding (SVC) and multi-view video coding (MVC) extensions. The latest joint draft of MVC is described in "Advanced Video Coding for Generic Audiovisual Services" (ITU-T Recommendation H.264) in March 2010.

In addition, there is a new and upcoming video coding standard, the aforementioned High Efficiency Video Coding (HEVC), which is being developed by the Joint Collaboration Team on Video Coding (JCT-VC) of the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG).

FIG2 is a diagram illustrating intra prediction modes used in HEVC. FIG2 generally illustrates the prediction directions associated with various directional intra prediction modes that can be used for intra coding in HEVC. In current HEVC, for the luma component of each prediction unit (PU), intra prediction methods are utilized with 33 angular prediction modes (indexed from 2 to 34), a DC mode (indexed with 1), and a planar mode (indexed with 0), as shown in FIG2.

In planar mode (indexed by 0), prediction is performed using a so-called "planar" function to determine a predictor value for each of the pixels within a block of video data (e.g., a PU). According to DC mode (indexed by 1), prediction is performed using an averaging of the pixel values within the block to determine a predictor value for each of the pixels within the block. According to directional prediction mode, prediction is performed based on reconstructed pixels of neighboring blocks along a particular direction (as indicated by the mode). In general, the tail end of an arrow shown in FIG2 represents the opposite one of the neighboring pixels from which a value is retrieved, while the head of the arrow represents the direction along which the retrieved value is propagated to form the predictive block.

For HEVC intra-prediction modes, video encoder 20 and/or video decoder 30 generates pixel-specific predictor values for each pixel in the PU using the various modes discussed above, for example, by using neighboring samples for PUs of modes 2 to 34. Video encoder 20 determines a residual value for the video block based on the difference between the actual depth values of the pixels of the block and the predictor value, and provides the residual value to video decoder 30. According to HEVC WD10, video encoder 20 transforms the residual value and quantizes the transform coefficients, and may also entropy encode the quantized transform coefficients. Video decoder 30 (e.g., after entropy decoding, inverse quantization, and inverse transformation) determines reconstructed values for the pixels of the block by adding the residual value to the predictor value. Further details regarding HEVC intra-prediction modes are specified in HEVC WD10.

In JCT-3V, two HEVC extensions are being developed: Multi-view extension (MV-HEVC) and 3D video extension (3D-HEVC). The latest version of the reference software "3D-HTM Version 8.2" for 3D-HEVC is incorporated herein by reference in its entirety and can be downloaded from the following link:

[3D-HTM version 8.2- dev0 ]: https://hevc.hhi.fraunhofer.de/svn/svn_3DVCSoftware/branches/HTM-8.2-dev0/

A new version of the reference software, 3D-HTM 9.0, will be presented on top of 3D-HTM version 8.2 and will be available shortly.

The latest working draft (Document No. F1001) is available from:

http://phenix.it-sudparis.eu/jct2/doc_end_user/documents/6_Geneva/ wg11/JCT3V-F1001-v1.zip

In 3D-HEVC, as defined in the 3D-HEVC draft referenced above, each access unit contains multiple pictures, and each of the pictures in each view has a unique view identification (ID) or view order index. However, depth pictures and texture pictures of the same view may have different layer ids.

Depth intra coding in 3D video coding will now be described. 3D video data is represented using a multi-view video plus depth format, where captured views (textures) are associated with corresponding depth maps. In 3D video coding, textures and depth maps are coded and multiplexed into the 3D video bitstream. Depth maps are coded as grayscale video, where luma samples represent depth values, and conventional intra- and inter-coding methods can be applied to depth map coding.

As discussed above, depth maps may be characterized by sharp edges and constant area.Due to the different statistics of depth map samples, different coding schemes are designed for depth maps based on 2D video codecs.

In 3D-HEVC, the same definition of intra prediction mode as in HEVC is used. That is, the intra mode used in 3D-HEVC includes the intra mode of HEVC. Also, in 3D-HEVC, the depth modeling mode (DMM) is introduced together with the HEVC intra prediction mode to decode the intra prediction unit of the depth slice. For better representation of sharp edges in the depth map, the current HTM (3D-HTM version 8.2) applies the DMM method for intra decoding of the depth map. There are two new intra modes in DMM. In both modes, the depth block is partitioned into two regions specified by the DMM mode, where each region is represented by a constant value.

The DMM mode can be explicitly signaled (DMM mode 1) or predicted by a co-located texture block (DMM mode 4). There are two types of partitioning models defined in the DMM, including Wedgelet partitioning and Contour partitioning. FIG3 is a diagram illustrating an example of a Wedgelet partitioning mode for coding a block of pixel samples. FIG4 is a diagram illustrating an example of a Contour partitioning mode for coding a block of pixel samples.

For Wedgelet partitioning, the depth block is partitioned into two regions by a straight line, as illustrated in Figure 3. For contour partitioning, the depth block can be partitioned into two irregular regions, as shown in Figure 4. Contour partitioning is more flexible than Wedgelet partitioning, but is difficult to explicitly signal. In DMM mode 4, the contour partitioning mode is implicitly derived using the reconstructed luma samples of the co-located texture block.

As an example, FIG3 provides an illustration of a Wedgelet pattern for an 8x8 block 40. For Wedgelet partitioning, a depth block (e.g., a PU) is partitioned by a line 46 into two regions 42 and 44, with a starting point 48 at (Xs, Ys) and an ending point 50 at (Xe, Ye), as illustrated in FIG3 . Regions 42 and 44 are also labeled P0 and P1, respectively. Each pattern in block 40 consists of an array of binary digits of size uB×vB, which indicate whether the corresponding sample belongs to region P0 or P1, where uB and vB represent the horizontal and vertical sizes of the current PU, respectively. Regions P0 and P1 are represented in FIG3 by white and shaded samples, respectively. The Wedgelet pattern is initialized at the beginning of both encoding and decoding.

As shown in the example of FIG4 , a depth block such as depth block 60 can be partitioned into three irregularly shaped regions 62, 64A, and 64B using contour segmentation, where region 62 is individually labeled P0 and regions 64A and 64B are collectively labeled P1. Although pixels in region 64A are not immediately adjacent to pixels in region 64B, regions 64A and 64B can be defined to form a single region for the purpose of predicting the PU of depth block 60. Contour segmentation can be more flexible than Wedgelet segmentation, but can be relatively more difficult to signal. In DMM mode 4, in the case of 3D-HEVC, the contour segmentation mode is implicitly derived using reconstructed luma samples of the co-located texture block.

3 and 4 , each individual square within depth blocks 40 and 60 represents a corresponding individual pixel of depth blocks 40 and 60, respectively. The numerical value within the square indicates whether the corresponding pixel belongs to region 42 (value “0” in the example of FIG. 3 ) or region 44 (value “1” in the example of FIG. 3 ). Shading is also used in FIG. 3 to indicate whether a pixel belongs to region 42 (white squares) or region 44 (grey shaded squares).

As discussed above, each mode (i.e., Wedgelet and Contour) may be defined by an array of binary digits of size uB×vB that mark whether the corresponding sample (i.e., pixel) belongs to region P0 or P1 (where P0 corresponds to region 42 in FIG. 3 and region 62 in FIG. 4 , and P1 corresponds to region 44 in FIG. 3 and regions 64A, 64B in FIG. 4 ), where uB and vB represent the horizontal and vertical sizes of the current PU, respectively. In the examples of FIG. 3 and FIG. 4 , the PUs correspond to blocks 40 and 60, respectively. A video coder, such as video encoder 20 and video decoder 30, may initialize a Wedgelet mode at the beginning of coding, e.g., at the beginning of encoding or at the beginning of decoding.

For HEVC intra prediction modes, a pixel-specific intra predictor value is generated for each pixel in the PU by using the PU's neighboring samples as specified in HEVC WD10, subclause 8.4.2.

For other depth intra modes, a partition-specific DC predictor is calculated for each partition within the PU using up to two neighboring samples of the PU. Assume that bPattern[x][y] is the partition pattern of the PU, where x=0..N-1, y=0..N-1 and N is the width of the PU. bPattern[x][y] indicates to which partition the pixel (x, y) belongs and bPattern[x][y] can be equal to 0 or 1. Assume that BitDepth is the bit depth of the depth samples and assume that RecSample[x][y] is the reconstructed neighboring samples of the PU, where x=-1 and y=0..N-1 (corresponding to the left neighboring pixels of the PU) or y=-1, x=0..N-1 (corresponding to the top neighboring pixels of the PU). The DC predictor for partition X, DCPred[X], where X=0 or 1, is then derived as follows:

Set bT = (bPattern[0][0]! = bPattern[N-1][0])? 1:0

Set bL = (bPattern[0][0]! = bPattern[0][N-1])? 1:0

If bT equals bL

-DCPred[X]＝(RecSample[-1][0]+RecSample[0][-1])>>1

-DCPred[1-X]=bL?(RecSample[-1][N-1]+RecSample[N-1][-1])>>1:2 ^{bit depth -1}

·otherwise

-DCPred[X]=bL? RecSample[(N-1)>>1][-1]:RecSample[-1][(N-1)>>1]

-DCPred[1-X]=bL? RecSample[-1][N-1]:RecSample[N-1][-1]

As discussed above, a depth lookup table (DLT) maps depth indices to depth values. The DLT can be constructed by analyzing frames within the first intra period before encoding the full video sequence. In the current design of 3D-HEVC, all valid depth values are sorted in ascending order and inserted into the DLT with increasing indices.

The DLT is an optional coding tool. In the current HTM, if more than half of the values from 0 to MAX_DEPTH_VALUE (e.g., 255 for 8-bit depth samples) appear in the original depth map during the analysis step, the encoder 20 will not use the DLT. Otherwise, the DLT will be coded in the sequence parameter set (SPS) and/or video parameter set (VPS). To code the DLT, the number of valid depth values is first coded using an Exponential-Golomb code. Subsequently, each valid depth value is also coded using an Exponential-Golomb code.

Encoder 20 reads a predefined number of frames from the input video sequence to be decoded and scans all samples for available depth map values.During this process, encoder 20 generates a mapping table that maps depth values to valid depth values based on the original uncompressed depth map.

The encoder 20 derives the depth lookup table Idx2Depth(.), the index lookup table Depth2Idx(.), the depth map table M(.), and the number of valid depth values d_valid using the following algorithm that analyzes the depth map D_t:

1. Initialization

For all depth values d, the Boolean vector B(d) = false

Index counter i = 0

2. Process each pixel position p in _Dt for multiple time instances t:

Set (B(D _t (p)) = true to mark a valid depth value

3. Count the number of true values in B(d) → d _{is valid}

4. For every d where B(d) == True:

Set Idx2Depth(i) = d

Set M(d) = d

Set Depth2Idx(d) = i

i＝i+1

5. For each d where B(d) == false:

Find d'=arg min|d-d'| and B(d') == true

Set M(d) = d'

Set Depth2Idx(d)=Depth2Idx(d').

The mapping from the index Idx back to the depth value d is as follows: d=Idx2Depth[Idx]. The mapping from the depth value d to the index Idx is as follows: Idx=Depth2Idx[d].

Intra SDC mode (which may also be referred to as simplified depth coding, simplified residual coding, or slice-by-slice DC coding) was introduced in 3D-HEVC along with HEVC intra prediction mode, DMM mode, and chain coding mode to code intra PUs of depth slices. In current 3D-HEVC, SDC is only applied to 2Nx2N PU partition sizes. Instead of coding quantized transform coefficients, SDC mode represents a depth block with the following two types of information:

1. Partition type of the current depth block, including:

a.DMM mode 1 (2 partitions)

b. Plane (1 partition)

2. For each partition, signal the residual value (in the pixel domain) in the bitstream.

Two sub-modes are defined in SDC, including SDC Mode 1 and SDC Mode 2, which correspond to the partition types of Planar and DMM Mode 1, respectively.

Simplified residual coding is used in SDC.In simplified residual coding, as described above, one DC residual value is signaled for each partition of the PU, and no transform or quantization is applied.

As discussed above, in order to signal the information representing the DC residual value of each partition, two methods may be applied:

1. Directly code the DC residual value of each partition calculated by subtracting the predictor denoted by Pred generated by neighboring samples from the DC value of the current partition in the current PU (ie, the average value, denoted by Aver).

2. When transmitting the DLT, instead of decoding the DC residual value, the index difference between Aver and Pred mapped from the index lookup table is decoded. The index difference is calculated by subtracting the index of Pred from the index of Aver. At the decoder side, the sum of the decoded index difference and the index of Pred is mapped back to the depth value based on the DLT.

In addition to inter-view sample prediction and inter-view motion prediction similar to texture coding, 3D-HEVC also adopts a new tool, namely simplified inter-mode depth coding (SIDC).

SIDC extends the basic idea of SDC to inter-mode depth coding. Therefore, SIDC may be referred to simply as "inter-SDC" in the following context. However, in many cases, throughout this disclosure, SDC will be generally used to refer to the application of SDC for intra- or inter-coding. Inter-SDC provides an alternative residual coding method by which the encoder 20 encodes only one DC residual value for a PU. The encoder 20 and/or video decoder 30 skip transforms and quantization, and the encoder does not need to generate an additional residual similar to a transform tree. Whether inter-SDC is used is signaled by the encoder 20 at the CU level in the general coding unit parameters and parsed by the decoder 30. For inter-SDC coded CUs, one DC residual value is signaled for each PU and used as the residual for all samples in the PU. For example, SDC signaling can be performed by the encoder 20 at the CU level and received by the decoder 30 together with deep intra- or inter-mode signaling. However, when a CU is coded in SDC mode, the CU may be split into several PUs, and each PU may be further split into partitions, such as Wedgelet partitions or outline partitions. The SDC residual data may include a DC residual value signaled for each partition within each PU. Thus, in SDC mode, each partition of a PU may have a DC residual value.

To reduce signaling bits for inter SDC mode, video encoder 20 and/or video decoder 30 may apply inter SDC only to non-skipped CUs. Furthermore, to avoid possible overlap between inter SDC mode and skip mode, video encoder 20 and/or video decoder 30 applies inter SDC mode only when the DC residual for each PU within the CU is non-zero. The DC residual for a PU may be calculated as the average of the differences between the original sample values and the predicted sample values for all samples within the partition of the PU. Because only the DC difference between the partitions of the original block and the predicted block is signaled, mean removal motion estimation is employed for deep inter mode coding to compensate for the AC difference.

FIG5 is a block diagram illustrating an example video encoder 20 that may be configured to implement the techniques of this disclosure. FIG5 is provided for purposes of explanation and should not be construed as limiting the techniques to those generally illustrated and described in this disclosure. For purposes of explanation, this disclosure describes video encoder 20 in the context of HEVC coding, and more specifically 3D-HEVC. However, the techniques of this disclosure may be applicable to other coding standards or methods.

In the example of FIG5 , video encoder 20 includes a prediction processing unit 100, a residual generation unit 102, a transform processing unit 104, a quantization unit 106, an inverse quantization unit 108, an inverse transform processing unit 110, a reconstruction unit 112, a filter unit 114, a decoded picture buffer 116, and an entropy encoding unit 118. Prediction processing unit 100 includes an inter-prediction processing unit 120 and an intra-prediction processing unit 126. Inter-prediction processing unit 120 includes a motion estimation (ME) unit 122 and a motion compensation (MC) unit 124. For ease of illustration, the components of prediction processing unit 100 are illustrated and described as performing both texture encoding and depth encoding. In some examples, texture and depth encoding may be performed by the same component of prediction processing unit 100 or by different components within prediction processing unit 100. For example, separate texture and depth encoders may be provided in some implementations. Furthermore, multiple texture and depth encoders may be provided to encode multiple views, such as for multi-view plus depth coding. In either case, prediction processing unit 100 may be configured to intra- or inter-encode texture data and depth data as part of a 3D coding process, such as a 3D-HEVC process. Thus, prediction processing unit 100 may operate generally in accordance with 3D-HEVC, subject to modifications and/or additions described in this disclosure, such as those related to signaling of SDC for intra- and inter-prediction modes. Prediction processing unit 100 may generate and encode residual data for intra- or inter-coded depth data using SDC or non-SDC residual coding techniques as described in this disclosure. When SDC is used, video encoder 20 may encode and signal syntax elements indicating the use of SDC for both depth intra- and depth inter-prediction modes. In other examples, video encoder 20 may include more, fewer, or different functional components.

Video encoder 20 may receive video data. Video encoder 20 may encode each CTU in a slice of a picture of the video data. Each of the CTUs may be associated with a luma coding tree block (CTB) of equal size in the picture and a corresponding CTB. As part of encoding the CTU, prediction processing unit 100 may perform quadtree partitioning to divide the CTBs of the CTU into progressively smaller blocks. These smaller blocks may be coding blocks of a CU. For example, prediction processing unit 100 may partition the CTB associated with the CTU into four equally sized sub-blocks, partition one or more of the sub-blocks into four equally sized sub-sub-blocks, and so on.

Video encoder 20 may encode a CU of a CTU to generate an encoded representation of the CU (i.e., a coded CU). As part of encoding a CU, prediction processing unit 100 may partition the coding blocks associated with the CU among one or more PUs of the CU. Thus, each PU may be associated with a luma prediction block and a corresponding chroma prediction block.

Video encoder 20 and video decoder 30 may support PUs of various sizes. As indicated above, the size of a CU may refer to the size of the luma coding block of the CU and the size of a PU may refer to the size of the luma prediction block of the PU. Assuming the size of a particular CU is 2Nx2N, video encoder 20 and video decoder 30 may support PU sizes of 2Nx2N or NxN for intra prediction, and symmetric PU sizes of 2Nx2N, 2NxN, Nx2N, NxN, or similar sizes for inter prediction. Video encoder 20 and video decoder 30 may also support asymmetric partitioning of PU sizes of 2NxnU, 2NxnD, nLx2N, and nRx2N for inter prediction. According to aspects of this disclosure, video encoder 20 and video decoder 30 also support non-rectangular partitions of PUs for depth inter coding.

The inter-prediction processing unit 120 may generate predictive data for each PU of a CU by performing inter-prediction on the PU. The predictive data for the PU may include the predictive sample block of the PU and the motion information for the PU. The inter-prediction processing unit 120 may perform different operations on the PU of a CU depending on whether the PU is in an I slice, a P slice, or a B slice. In an I slice, all PUs are intra-predicted. Therefore, if the PU is in an I slice, the inter-prediction processing unit 120 does not perform inter-prediction on the PU. Therefore, for a block encoded in I mode, the predicted block is formed using spatial prediction based on previously encoded neighboring blocks in the same frame.

If the PU is in a P slice, motion estimation (ME) unit 122 may search a reference picture list (e.g., "RefPicList0") for a reference region for the PU. The reference picture may be stored in decoded picture buffer 116. The reference region for the PU may be a region within a reference picture that contains sample blocks that most closely correspond to the sample blocks of the PU. Motion estimation (ME) unit 122 may generate a reference index that indicates the location in RefPicList0 of the reference picture containing the reference region for the PU. In addition, motion estimation (ME) unit 122 may generate an MV that indicates the spatial displacement between the coding block of the PU and a reference location associated with the reference region. For example, the MV may be a two-dimensional vector that provides an offset from coordinates in the current decoded picture to coordinates in the reference picture. Motion estimation (ME) unit 122 may output the reference index and the MV as the motion information for the PU. Based on actual or interpolated samples at the reference location indicated by the PU's motion vector, motion compensation (MC) unit 124 may generate a predictive sample block for the PU.

If the PU is in a B slice, motion estimation unit 122 may perform uni-directional prediction or bi-directional prediction on the PU. To perform uni-directional prediction on the PU, motion estimation unit 122 may search the reference picture of RefPicList0 or a second reference picture list ("RefPicList1") for the reference region of the PU. Motion estimation (ME) unit 122 may output the following as motion information for the PU: a reference index indicating the position of the reference picture containing the reference region in RefPicList0 or RefPicList1, an MV indicating the spatial displacement between the sample block of the PU and the reference position associated with the reference region, and one or more prediction direction indicators indicating whether the reference picture is in RefPicList0 or RefPicList1. Motion compensation (MC) unit 124 may generate a predictive sample block for the PU based at least in part on actual or interpolated samples at the reference region indicated by the motion vector of the PU.

To perform bidirectional inter prediction on a PU, motion estimation unit 122 may search for a reference region for the PU within the reference pictures in RefPicList0 and may also search for another reference region for the PU within the reference pictures in RefPicList1. Motion estimation (ME) unit 122 may generate a reference index that indicates a location in RefPicList0 and RefPicList1 of the reference pictures containing the reference region. Additionally, motion estimation (ME) unit 122 may generate an MV that indicates a spatial displacement between the reference location associated with the reference region and the sample block of the PU. The motion information of the PU may include the reference index and the MV for the PU. Motion compensation (MC) unit 124 may generate a predictive sample block for the PU based at least in part on actual or interpolated samples at the reference region indicated by the motion vector of the PU.

Intra-prediction processing unit 126 may generate predictive data for a PU by performing intra prediction on the PU. The predictive data for the PU may include a predictive sample block for the PU and various syntax elements. Intra-prediction processing unit 126 may perform intra prediction on PUs in I slices, P slices, and B slices.

To perform intra prediction on a PU, intra-prediction processing unit 126 may use multiple intra-prediction modes to generate multiple sets of predictive data for the PU, and then select one of the intra-prediction modes that produces acceptable or optimal coding performance, for example, using rate-distortion optimization techniques. To use an intra-prediction mode to generate a set of predictive data for a PU, intra-prediction processing unit 126 may extend samples from a sample block of a neighboring PU across the sample block of the PU in a direction associated with the intra-prediction mode. Assuming a left-to-right, top-to-bottom coding order for PUs, CUs, and CTUs, the neighboring PU may be above, to the right, to the left, or to the left of the PU. Intra-prediction processing unit 126 may use various numbers of intra-prediction modes, for example, 33 directional intra-prediction modes. In some examples, the number of intra-prediction modes may depend on the size of the region associated with the PU.

Prediction processing unit 100 may select predictive data for PUs of a CU from among the predictive data for the PUs generated by inter-prediction processing unit 120 or the predictive data for the PUs generated by intra-prediction processing unit 126. In some examples, prediction processing unit 100 selects the predictive data for the PUs of the CU based on rate/distortion metrics of the predictive data sets. The predictive sample block of the selected predictive data may be referred to herein as a selected predictive sample block.

Residual generation unit 102 may generate a luma, Cb, and Cr residual block for a CU based on the luma, Cb, and Cr coding blocks of the CU and the selected predictive luma, Cb, and Cr blocks of the PUs of the CU. For example, residual generation unit 102 may generate the residual block for the CU such that each sample in the residual block has a value equal to the difference between a sample in the coding block of the CU and a corresponding sample in a corresponding selected predictive sample block of the PUs of the CU.

Transform processing unit 104 may perform quadtree partitioning to partition the residual blocks associated with a CU into transform blocks associated with the TUs of the CU. Thus, a TU may be associated with a luma transform block and two chroma transform blocks. The size and position of the luma transform block and the chroma transform blocks of the TUs of the CU may or may not be based on the size and position of the prediction blocks of the PUs of the CU. A quadtree structure called a "residual quadtree" (RQT) may include a node associated with each of the regions. The TUs of a CU may correspond to leaf nodes of the RQT.

Transform processing unit 104 may generate a transform coefficient block for each TU of a CU by applying one or more transforms to the transform blocks of the TU. Transform processing unit 104 may apply various transforms to the transform blocks associated with the TU. For example, transform processing unit 104 may apply a discrete cosine transform (DCT), a directional transform, or a conceptually similar transform to the transform blocks. In some examples, transform processing unit 104 does not apply a transform to the transform blocks. In such examples, the transform blocks may be processed as transform coefficient blocks.

Quantization unit 106 may quantize the transform coefficients in a coefficient block. The quantization process may reduce the bit depth associated with some or all of the transform coefficients. For example, an n-bit transform coefficient may be rounded to an m-bit transform coefficient during quantization, where n is greater than m. Quantization unit 106 may quantize a coefficient block associated with a TU of a CU based on a quantization parameter (QP) value associated with the CU. Video encoder 20 may adjust the degree of quantization applied to a coefficient block associated with a CU by adjusting the QP value associated with the CU. Quantization may result in a loss of information, and thus, the quantized transform coefficients may have lower precision than the original transform coefficients.

Inverse quantization unit 108 and inverse transform processing unit 110 may apply inverse quantization and inverse transform, respectively, to the coefficient block to reconstruct a residual block from the coefficient block. Reconstruction unit 112 may add the reconstructed residual block to corresponding samples from one or more predictive sample blocks generated by prediction processing unit 100 to produce a reconstructed transform block associated with the TU. By reconstructing the transform blocks for each TU of a CU in this manner, video encoder 20 can reconstruct the coding blocks of the CU.

Filter unit 114 may perform one or more deblocking operations to reduce blocking artifacts in the coding blocks associated with the CU. Decoded picture buffer 116 may store the reconstructed coding blocks after filter unit 114 performs the one or more deblocking operations on the reconstructed coding blocks. Inter-prediction unit 120 may use a reference picture containing the reconstructed coding blocks to perform inter prediction on PUs of other pictures. Additionally, intra-prediction processing unit 126 may use the reconstructed coding blocks in decoded picture buffer 116 to perform intra prediction on other PUs in the same picture as the CU.

Entropy coding unit 118 may receive data from other functional components of video encoder 20. For example, entropy coding unit 118 may receive coefficient blocks from quantization unit 106 and may receive syntax elements from prediction processing unit 100. Entropy coding unit 118 may perform one or more entropy coding operations on the data to generate entropy-encoded data. For example, entropy coding unit 118 may perform a context-adaptive variable length coding (CAVLC) operation, a CABAC operation, a variable-to-variable (V2V) length coding operation, a syntax-based context-adaptive binary arithmetic coding (SBAC) operation, a probability interval partitioning entropy (PIPE) coding operation, an exponential Golomb coding operation, or another type of entropy coding operation on the data. Video encoder 20 may output a bitstream containing the entropy-encoded data generated by entropy coding unit 118. For example, the bitstream may include data representing an RQT for a CU.

Video encoder 20 is an example of a video encoder configured to perform any of the techniques for simplified residual coding for deep intra prediction or deep inter prediction as described herein. In accordance with one or more techniques of this disclosure, one or more units within video encoder 20 may perform one or more techniques described herein as part of a video encoding process. Similarly, video encoder 20 may perform a video decoding process using any of the techniques of this disclosure to regenerate video data used as a reference for subsequently coded video data. As discussed above, inverse quantization unit 108, inverse transform processing unit 110, and reconstruction unit 112, as well as other elements of video encoder 20, may be utilized in the video decoding process. Additional 3D processing components may also be included within video encoder 20.

For example, prediction processing unit 100, and more particularly, inter-prediction processing unit 120 and intra-prediction processing unit 126 may perform SDC mode for depth inter prediction and depth intra prediction encoding of a depth block, respectively, as described herein. Inter-prediction processing unit 120 and intra-prediction processing unit 126, when used, may each determine a DC residual value for a depth block (e.g., a PU) or each partition thereof. Prediction processing unit 120 may generate syntax elements and/or syntax structures indicating that SDC is used for the current PU for depth intra prediction and depth inter prediction in a unified manner, as described in various examples of this disclosure.

Intra-prediction processing unit 126 or inter-prediction processing unit 120 may provide the DC residual value of the depth block along with other syntax information to entropy encoding unit 118, e.g., as illustrated by the dashed line in FIG5 . Intra-prediction processing unit 126 or inter-prediction processing unit 120 may provide the DC residual value to entropy encoding unit 118 without the value being processed by transform processing unit 104 and quantization unit 106. In other examples, quantization unit 106 may quantize the DC residual value prior to entropy coding by entropy encoding unit 118. The syntax information may include various information, flags, or other syntax elements described herein in connection with the techniques of this disclosure for signaling.

For example, the syntax information may indicate, as an example, whether SDC mode is performed for a depth block, whether a partition-based (e.g., 3D-HEVC) or non-partition-based (HEVC) mode is used to determine a DC residual value, and which depth intra prediction mode is used to determine the DC residual value. In addition, the syntax information may indicate, in a uniform manner, whether SDC mode uses a CU, regardless of whether intra prediction or inter prediction is used for the CU, i.e., for depth intra prediction or depth inter prediction, as the case may be. According to the SDC mode, the video encoder 20 is configured to: determine at least one DC residual value for a depth block (e.g., a partition of a PU of a CU) based on the indicated one of the depth intra prediction mode or the depth inter prediction mode, wherein the DC residual value represents a residual value of a plurality of pixels of the depth block; and encode the DC residual value or information representing the DC residual value into the bitstream.

The video encoder 20 is an example of a video encoder that includes a memory for storing video data and one or more processors configured to perform a method, the method comprising: performing intra-frame prediction in the video encoder to generate a predicted depth CU when the depth CU is intra-frame predicted; performing inter-frame prediction in the video encoder to generate a predicted depth CU when the depth CU is inter-frame predicted; when using simplified depth coding (SDC) mode, for each partition of a prediction unit (PU) of the depth CU, generating in the video encoder information representing at least one DC residual value of the depth CU, wherein the at least one DC residual value represents a pixel difference between a partition of the PU of the depth CU and a corresponding partition of the predicted depth CU; when using the SDC mode, generating in a video decoder a syntax element indicating that the SDC mode is used for both intra-frame prediction and inter-frame prediction of a depth coding unit (CU) of video data; and encoding the depth CU in the video encoder based on the information representing the at least one DC residual value and the syntax element.

The video encoder 20 is also an example of a video encoder that includes a memory for storing video data and one or more processors configured to perform a method, the method comprising: performing intra-frame prediction in the video encoder to generate a predicted depth CU when a depth coding unit (CU) of the video data is intra-frame predicted; performing inter-frame prediction in the video encoder to generate a predicted depth CU when the depth CU is inter-frame predicted; decoding a syntax structure including one or more syntax elements indicating information of at least one DC residual value of a partition of a prediction unit (PU) representing the depth CU when a simplified depth coding (SDC) mode is used in video encoding, wherein the syntax elements of the syntax structure are the same for intra-frame prediction and inter-frame prediction; and encoding the depth CU in the video encoder based on the information representing the at least one DC residual value.

The video encoder 20 is also an example of a video encoder that includes a memory for storing video data and one or more processors configured to perform a method, the method comprising: encoding, in the video encoder, a syntax element indicating whether a simplified depth coding (SDC) mode is enabled for depth coding units (CUs) of an entire sequence of coded video data; encoding a second syntax element indicating whether the SDC mode is used for both intra-prediction and inter-prediction of one of the depth CUs of the sequence; when one of the depth CUs of the sequence is intra-predicted, performing intra-prediction in the video encoder to generate a predicted depth CU; when one of the depth CUs of the sequence is inter-predicted, performing inter-prediction in the video encoder to generate a predicted depth CU; and when the second syntax element indicates that the SDC mode is used, encoding information of at least one DC residual value for a partition of a prediction unit (PU) of the one of the depth CUs of the sequence.

FIG6 is a block diagram illustrating an example video decoder 30 configured to perform the techniques of this disclosure. FIG6 is provided for purposes of explanation and should not be construed as limiting the techniques to those generally illustrated and described in this disclosure. For purposes of explanation, this disclosure describes video decoder 30 in the context of HEVC coding, and more specifically, 3D-HEVC coding. However, the techniques of this disclosure may be applicable to other 3D video coding standards or methods.

In the example of FIG6 , video decoder 30 includes an entropy decoding unit 150, a prediction processing unit 152, an inverse quantization unit 154, an inverse transform processing unit 156, a reconstruction unit 158, a filter unit 160, and a decoded picture buffer 162. Prediction processing unit 152 includes a motion compensation (MC) unit 164 and an intra-prediction processing unit 166. For ease of illustration, the components of prediction processing unit 152 are illustrated and described as performing both texture decoding and depth decoding. In some examples, texture and depth decoding may be performed by the same component of prediction processing unit 152 or by different components within prediction processing unit 152. For example, in some implementations, separate texture and depth decoders may be provided. Furthermore, multiple texture and depth decoders may be provided to decode multiple views, such as for multi-view plus depth coding. In either case, prediction processing unit 152 may be configured to perform intra- or inter-decoding of texture data and depth data as part of a 3D coding process, such as a 3D-HEVC process. Thus, prediction processing unit 152 may generally operate in accordance with 3D-HEVC, subject to modifications and/or additions described in this disclosure, such as those related to signaling of SDC for intra- and inter-prediction modes. Prediction processing unit 152 may obtain residual data from an encoded video bitstream of intra-decoded or inter-decoded depth data using SDC or non-SDC residual coding techniques as described in this disclosure, and reconstruct a CU using the intra-predicted or inter-predicted depth data and the residual data. When SDC is used, video decoder 30 may decode syntax elements indicating the use of SDC for both depth intra- and depth inter-prediction modes. In other examples, video decoder 30 may include more, fewer, or different functional components.

Video decoder 30 may receive a bitstream. Entropy decoding unit 150 may parse the bitstream to decode syntax elements from the bitstream. Entropy decoding unit 150 may entropy decode the entropy-encoded syntax elements in the bitstream. Prediction processing unit 152, inverse quantization unit 154, inverse transform processing unit 156, reconstruction unit 158, and filter unit 160 may generate decoded video data based on the syntax elements extracted from the bitstream.

The bitstream may include a series of NAL units. The NAL units of the bitstream may include NAL units for coded slices. As part of encoding the bitstream, entropy decoding unit 150 may extract syntax elements from the coded slice NAL units and entropy decode the syntax elements. Each of the coded slices may include a slice header and slice data. The slice header may contain syntax elements related to the slice. The syntax elements in the slice header may include syntax elements that identify the PPS associated with the picture containing the slice. The PPS may reference the SPS, which in turn may reference the VPS. Entropy decoding unit 150 may also entropy decode other elements that may include syntax information, such as SEI messages.

The decoded syntax elements in any of the slice header, parameter set, or SEI message may include information described herein as signaling to perform SDC mode according to the example techniques described in this disclosure. For example, the decoded syntax elements may indicate whether SDC mode is performed for depth blocks used for both depth intra prediction and depth inter prediction modes in a unified manner as described in this disclosure. This syntax information may be provided to prediction processing unit 152 for use in decoding and reconstructing depth blocks according to the techniques described in this disclosure.

In general, in addition to decoding syntax elements from the bitstream, video decoder 30 may perform a reconstruction operation on a non-partitioned CU. To perform the reconstruction operation on a non-partitioned CU, video decoder 30 may perform the reconstruction operation on each TU of the CU. By performing the reconstruction operation on each TU of the CU, video decoder 30 may reconstruct the blocks of the CU.

As part of performing a reconstruction operation on a TU of a CU, inverse quantization unit 154 may inversely quantize (i.e., dequantize) a coefficient block associated with the TU. Inverse quantization unit 154 may use the QP value associated with the CU of the TU to determine the degree of quantization and (likely) the degree of inverse quantization that inverse quantization unit 154 will apply. That is, the compression ratio, i.e., the ratio of the number of bits used to represent the original sequence and the compressed sequence, may be controlled by adjusting the value of the QP used when quantizing the transform coefficients. The compression ratio may also depend on the entropy coding method employed.

After inverse quantization unit 154 inverse quantizes the coefficient block, inverse transform processing unit 156 may apply one or more inverse transforms to the coefficient block to generate a residual block associated with the TU. For example, inverse transform processing unit 156 may apply an inverse DCT, an inverse integer transform, an inverse Karhunen-Loeve transform (KLT), an inverse rotational transform, an inverse directional transform, or another inverse transform to the coefficient block.

If the PU is encoded using intra prediction, intra-prediction processing unit 166 may perform intra prediction to generate predictive blocks for the PU. Intra-prediction processing unit 166 may use an intra prediction mode to generate predictive luma blocks, Cb blocks, and Cr blocks for the PU based on the prediction blocks of spatially neighboring PUs. Intra-prediction processing unit 166 may determine the intra prediction mode for the PU based on one or more syntax elements decoded from the bitstream.

Prediction processing unit 152 may construct a first reference picture list (RefPicList0) and a second reference picture list (RefPicList1) based on syntax elements extracted from the bitstream. In addition, if the PU is encoded using inter prediction, entropy decoding unit 150 may extract motion information for the PU. Motion compensation (MC) unit 164 may determine one or more reference regions for the PU based on the motion information of the PU. Motion compensation (MC) unit 164 may generate a predictive luma block, a Cb block, and a Cr block for the PU based on sample blocks at one or more reference blocks for the PU.

6 . Reconstruction unit 158 may reconstruct the luma, Cb, and Cr coding blocks of the CU using the luma transform blocks, Cb transform blocks, and Cr transform blocks associated with the TUs of the CU and the predictive luma, Cb, and Cr blocks of the PUs of the CU (i.e., intra prediction data or inter prediction data), as appropriate. For example, reconstruction unit 158 may add samples of the luma, Cb, and Cr transform blocks to corresponding samples of the predictive luma, Cb, and Cr blocks to reconstruct the luma, Cb, and Cr coding blocks of the CU.

Filter unit 160 may perform a deblocking operation to reduce blocking artifacts associated with the luma, Cb, and Cr coding blocks of a CU. Video decoder 30 may store the luma, Cb, and Cr coding blocks of the CU in decoded picture buffer 162. Decoded picture buffer 162 may provide reference pictures for subsequent motion compensation, intra-prediction, and presentation on a display device, such as display device 32 of FIG. 1 . For example, video decoder 30 may perform intra-prediction or inter-prediction operations on PUs of other CUs based on the luma, Cb, and Cr blocks in decoded picture buffer 162. In this manner, video decoder 30 may extract transform coefficient levels for a large number of luma coefficient blocks from a bitstream, inverse quantize the transform coefficient levels, apply a transform to the transform coefficient levels to generate transform blocks, generate coded blocks based at least in part on the transform blocks, and output the coded blocks for display.

Video decoder 30 is an example of a video decoder configured to perform any of the techniques for unifying signaling and parsing syntax elements indicating the use of SDC for both depth intra prediction mode and depth inter prediction mode as described herein. In accordance with one or more techniques of this disclosure, one or more units within video decoder 30 may perform one or more techniques described herein as part of the video decoding process. Additional 3D coding components may also be included within video decoder 30.

For example, prediction processing unit 152, and more particularly intra-prediction processing unit 166 and motion compensation (MC) unit 164, may determine whether to perform SDC in depth intra prediction mode and depth inter prediction mode of a 3D video coding process such as 3D-HEVC, when applicable. Entropy decoding unit 150 may entropy decode one or more DC residual values of a depth block and syntax information described herein, such as whether to use depth intra prediction or depth inter prediction to encode the block and whether to perform SDC mode for depth intra prediction or depth inter prediction to encode the block. In this manner, decoder 30 determines whether to use SDC to decode the current CU (i.e., depth block) based on syntax elements such as sdc_flag, and may use the same syntax structure or different instances of the syntax structure to obtain residual values for use with an intra-predicted depth block or an inter-predicted depth block to reconstruct the CU.

Entropy decoding unit 150 may provide the DC residual value and syntax information for the block to prediction processing unit 152, as indicated by the dashed line in FIG6 . In this manner, the DC residual value need not first be provided to inverse quantization unit 154 and inverse transform processing unit 156 for inverse quantization and inverse transformation. In other examples, inverse quantization unit 154 may inverse quantize the DC residual value and provide the dequantized DC residual value to prediction processing unit 152.

Motion compensation (MC) unit 164 may determine a predictor value for the depth block based on the depth inter prediction mode as indicated by the syntax information, e.g., according to any of the techniques described herein. Motion compensation (MC) unit 164 may utilize a depth block reconstructed from a reference picture stored in decoded picture buffer 162 to determine the predictor value for the inter-predicted depth block. If SDC mode is indicated, i.e., for both depth intra prediction and inter prediction, motion compensation (MC) unit 164 applies SDC to reconstruct the depth block. Motion compensation (MC) unit 164 may reconstruct the depth block by summing the DC residual value with the predictor value. In some examples, motion compensation (MC) unit 164 may utilize reconstruction unit 158 for summing the residual and predictor value for the inter-predicted depth block. For example, entropy decoding unit 150 may provide the DC residual value to reconstruction unit 158, and motion compensation (MC) unit 164 may provide the predictor value to the reconstruction unit.

Intra-prediction processing unit 166 may determine a predictor value for the depth block based on the depth intra-prediction mode indicated by the syntax information, e.g., according to any of the techniques described herein. Intra-prediction processing unit 166 may utilize the reconstructed depth block stored in decoded picture buffer 162 to determine the predictor value. Intra-prediction processing unit 166 may reconstruct the depth block by summing the DC residual value and the predictor value, as described herein. In some examples, intra-prediction processing unit 166 may utilize reconstruction unit 158 for summing the residual and predictor value for the depth block. For example, entropy decoding unit 150 may provide the DC residual value to the reconstruction unit, and intra-prediction processing unit 166 may provide the predictor value to the reconstruction unit.

Video decoder 30 is an example of a video decoder that includes one or more processors configured to: decode a syntax element indicating whether simplified depth coding (SDC) mode is used for both intra prediction and inter prediction of a depth coding unit (CU) of video data; perform intra prediction to generate a predicted depth CU when the depth CU is intra predicted; perform inter prediction to generate a predicted depth CU when the depth CU is inter predicted; and generate at least one DC residual value for the depth CU when the syntax element indicates use of SDC mode, wherein the at least one DC residual value represents a pixel difference between the depth CU and the predicted depth CU. The one or more processors of video decoder 30 may be configured to decode the syntax element and reconstruct the depth CU using the at least one DC residual value and the predicted depth CU.

In addition, video decoder 30 is an example of a video decoder that includes one or more processors configured to: when a depth coding unit (CU) of the video data is intra-predicted, perform an intra-prediction mode to generate a predicted depth CU; when the depth CU is inter-predicted, perform an inter-prediction mode to generate a predicted depth CU; and when simplified depth coding (SDC) mode is used, decode a syntax structure including information representing at least one DC residual value of the depth CU, wherein syntax elements of the syntax structure are the same for intra-prediction mode and inter-prediction mode. Video decoder 30 may be further configured to decode the syntax structure and reconstruct the depth CU using the at least one DC residual value and the predicted depth CU.

In another example, video decoder 30 is an example of a video coder comprising one or more processors configured to: decode a syntax element indicating whether simplified depth coding (SDC) mode is used for both intra prediction and inter prediction of depth coding units (CUs) of an entire sequence of coded video data; perform intra prediction to generate a predicted depth CU when one of the depth CUs of the sequence is intra predicted; perform inter prediction to generate a predicted depth CU when one of the depth CUs of the sequence is inter predicted; and when the syntax element indicates use of SDC mode, decode information representing at least one DC residual value for each of the depth CUs of the sequence, wherein the at least one DC residual value represents a pixel difference between the respective depth CU and the respective predicted depth CU. Video decoder 30 may be configured to reconstruct the depth CU using the at least one DC residual value for each of the depth CUs and the respective predicted depth CU.

The syntax and semantics of the existing draft of 3D-HEVC will now be described.The following sets forth syntax elements indicating intra/inter SDC mode in 3D-HEVC, where references to sections and tables are made relative to 3D-HEVC.

H.7.3.2.1.2 Video parameter set extension 2 syntax

H.7.3.8.5 Decoding unit syntax

H.7.3.8.5.1 Depth mode parameter syntax

Semantics

vps_inter_sdc_flag[layerId] equal to 1 specifies that inter SDC coding is used for the layer with nuh_layer_id equal to layerId. vps_inter_sdc_flag[layerId] equal to 0 specifies that inter SDC coding is not used for the layer with nuh_layer_id equal to layerId. When not present, the value of vps_inter_sdc_flag[layerId] is inferred to be equal to 0.

inter_sdc_flag equal to 1 specifies that simplified depth coding of the residual block is used for the current coding unit. inter_sdc_flag equal to 0 specifies that simplified depth coding of the residual block is not used for the current coding unit. When not present, inter_sdc_flag is inferred to be equal to 0.

inter_sdc_resi_abs_minus1[x0][y0][i], inter_sdc_resi_sign_flag[x0][y0][i] are used to derive InterSdcResi[x0][y0][i] as follows:

InterSdcResi[x0][y0][i]＝(1-2*inter_sdc_resi_sign_flag[x0][y0][i])*

(inter_sdc_resi_abs_minus1[x0][y0][i]+1) (H-19)

H.7.4.9.5.1 Depth mode parameter semantics

The variable Log2MaxDmmCbSize is set equal to 5.

The variable depthIntraModeSet is exported as specified below:

- If log2CbSize is equal to 6, then set depthIntraModeSet equal to 0.

- Otherwise, if log2CbSize is equal to 3 and PartMode[xC][yC] is equal to PART_NxN, then depthIntraModeSet is set equal to 1.

Otherwise, set depthIntraModeSet equal to 2.

depth_intra_mode[x0][y0] specifies the depth intra mode of the current prediction unit. Table H-2 specifies the value of depthIntraModeMaxLen and the value of the variable DepthIntraMode depending on the variable depthIntraModeSet, and the associated name depending on depth_intra_mode and depthIntraModeSet.

The variable SdcFlag[x0][y0] is derived as specified below:

SdcFlag[x0][y0]＝

(DepthIntraMode[x0][y0]==INTRA_DEP_SDC_PLANAR)||(H-25)

(DepthIntraMode[x0][y0]==INTRA_DEP_SDC_DMM_WFULL)

The variable DmmFlag[x0][y0] is derived as specified below:

DmmFlag[x0][y0]＝

(DepthIntraMode[x0][y0]==INTRA_DEP_DMM_WFULL)||(H-26)

(DepthIntraMode[x0][y0]==INTRA_DEP_DMM_CPREDTEX)

Table H-2 - Specification of DepthIntraMode and associated names depending on depthIntraModeSet and depth_intra_mode, and specification of depthIntraModeMaxLen depending on depthIntraModeSet

wedge_full_tab_idx[x0][y0] specifies the index of the wedgelet mode in the corresponding mode list when DepthIntraMode[x0][y0] is equal to INTRA_DEP_DMM_WFULL.

depth_dc_flag[x0][y0] equal to 1 specifies that depth_dc_abs[x0][y0][i] and depth_dc_sign_flag[x0][y0][i] are present. depth_dc_flag[x0][y0] equal to 0 specifies that depth_dc_abs[x0][y0][i] and depth_dc_sign_flag[x0][y0][i] are not present.

depth_dc_abs[x0][y0][i], depth_dc_sign_flag[x0][y0][i] are used to derive DcOffset[x0][y0][i] as follows:

DcOffset[x0][y0][i]＝

(1-2*depth_dc_sign_flag[x0][y0][i])*

(depth_dc_abs[x0][y0][i]-dcNumSeg+2)(H-27)

Some recent developments related to SDC are now discussed. In JCT3V-F0126, "CE5-related: Generic SDC for all Intra modes in 3D-HEVC" by Liu et al. (6th Meeting of the Joint Collaboration Group on 3D Video Coding Extensions of ITU-T SG16WP 3 and ISO/IEC JTC 1/SC 29/WG11: Geneva, Switzerland, October 25–November 1, 2013), simplified residual coding was proposed for depth coding, which can be applied to additional depth intra prediction modes as well as the original HEVC intra prediction modes. In simplified residual coding, one DC residual value is signaled for each partition of the PU, i.e., a PU coded in HEVC intra prediction mode has one partition so that all pixels within the PU are in the same partition, and a PU coded in additional depth intra prediction mode has two partitions, transform and quantization are skipped, and no additional residual is generated, i.e., there is no transform tree in HEVC-based 3D codecs.

As discussed above, a problem with the current design for SDC is the use of different syntax elements and context models to indicate the use of intra or inter mode in 3D-HEVC, which makes the parsing process of the coding unit more complicated.

The techniques described in this disclosure may unify the signaling of slice-by-slice DC coding (SDC) for deep intra and inter prediction modes in 3D-HEVC. The various aspects discussed above are also summarized here as follows:

1. It is proposed in this disclosure that in some examples of depth coding, only one syntax element (eg, sdc_flag) is used to indicate the use of SDC for intra or inter mode.

a. Remove inter_sdc_flag indicating that the current inter CU utilizes SDC.

b. Also remove the indication of SDC mode for intra mode.

c. Regardless of whether the current decoding unit is coded using intra-frame or inter-frame prediction, a new flag, sdc_flag, is introduced. When this flag is 1, SDC is enabled.

2. The present invention proposes that in some examples of depth coding, when the current CU is coded as SDC, for each partition, the DC residual value for the inter-coded CU or the intra-coded CU is unified to exist through one syntax structure, which includes the absolute value of the DC residual value and its sign.

Alternatively or additionally, if the CU is intra-coded, a flag (equal to 1) indicating whether the current CU contains any non-zero DC residual value in any of its partitions may be further signaled. If this flag is 0, the DC residual value is not signaled and is inferred to be equal to zero for each partition.

b. Alternatively, the above flag applies to both intra and inter modes.

c. Alternatively, the above flag is not used for intra or inter mode.

3. Furthermore, it is proposed in the present invention that in some examples, the syntax structures of DC residual values for inter and intra modes share the same syntax elements, so the context model and binarization process for the relevant syntax elements are unified, ie the same context model.

a. Alternatively, in some instances, even though intra SDC mode and inter SDC mode may use the same syntax structure for the DC residual value, they use different instances of the syntax structure, so the context for elements related to the DC residual value is maintained separate for intra SDC and inter SDC modes.

4. It is proposed in this disclosure that in some examples, a decoder-side constraint may be applied, ie, when sdc_flag is equal to 1, pcm_flag shall not be equal to 1.

5. Additionally, it is proposed in this disclosure that in some examples, constraints on SDC on the decoder side may be applied, ie, both intra and inter SDC modes are only used for 2Nx2N partition size.

a. Alternatively, intra SDC mode is used for all partition sizes including 2Nx2N and NxN.

6. Additionally, this disclosure proposes that, in some examples, one syntax element be used for the entire coded video sequence to enable/disable both intra and inter SDC modes during decoding. If the one syntax element indicates that SDC is enabled for the sequence, the encoder signals another syntax element to indicate whether SDC is used at the CU level, e.g., as described with reference to the use of a unified sdc_flag element in item 1 above.

a. In some examples, this syntax element may be set in a video parameter set extension, a sequence parameter set, a picture parameter set, or a slice segment header, among others.

b. Alternatively, in some examples, one flag is used to indicate whether SDC for intra mode is enabled for the entire coded video sequence, and if so, another flag is used to indicate whether SDC for inter mode is enabled for the entire coded video sequence.

Various implementation examples are described below. According to the examples below, changes are made to the syntax elements and semantics of 3D-HEVC. In the syntax elements and semantics below, the relevant parts of 3D-HEVC have been changed to add the syntax elements and semantics proposed in the JCT3V-F0126 document. The changes proposed in the JCT3V-F0126 document are shown with additions indicated by italics and deletions shown by . According to this example of the present invention, additional changes to the syntax elements and semantics are shown by new additions presented in bold and new deletions marked by .

An example syntax table is presented below, followed by the relevant semantics, where section and table references refer to corresponding section and table references in 3D-HEVC.

The following example illustrates the use of the sdc_flag[x0][y0] syntax element to specify the use of SDC at the CU level, the use of the SdcEnableFlag syntax element to indicate whether SDC is enabled, and the use of the depth_dc_abs[x0][y0][i] and depth_dc_sign_flag[x0][y0][i] syntax elements to indicate the DC residual value and sign, respectively. The syntax elements are examples of the syntax elements described in this disclosure for signaling SDC use, SDC enable/disable status, and SDC residual value, respectively. In some examples, the SdcEnableFlag syntax element may be derived at least in part from one or more values stated in the VPS, such as vps_inter_sdc_flag[nuh_layer_id] and vps_depth_modes_flag[nuh_layer_id], and may be configured to enable or disable SDC for the entire coded video sequence.

H.7.3.8.5 Decoding unit syntax

H.7.3.8.5.1 Depth DC residual value syntax table

Alternatively, when both intra and inter SDC apply only to CUs with partition size equal to 2Nx2N, the following syntax may be used.

H.7.3.8.5.1 Depth DC residual value syntax table

The example semantics and decoding process will now be described with reference to 3D-HEVC. Again, the changes proposed in the JCT3V-F0126 document are shown with additions indicated by italics and deletions shown by . According to this example of the present invention, additional changes to syntax elements and semantics are shown by new additions presented in bold and new deletions marked with .

H.7.4.9.5 Decoding unit semantics

Export the variable SdcEnableFlag as follows:

SdcEnableFlag=

(vps_inter_sdc_flag[nuh_layer_id]&&PredMode[x0][y0]==MODE_INTER)||

(vps_depth_modes_flag[nuh_layer_id]

&&PredMode[x0][y0]==MODE_INTRA

&&PartMode[x0][y0]＝＝PART_2Nx2N)(H-16)

sdc_flag[x0][y0] equal to 1 specifies that slice-wise DC coding of the residual block is used for the current coding unit. sdc_flag[x0][y0] equal to 0 specifies that slice-wise DC coding of the residual block is not used for the current coding unit. When not present, sdc_flag[x0][y0] is inferred to be equal to 0.

hevc_intra_flag[x0][y0] equal to 1 specifies that intra mode with intraPredMode in the range of 0 to 34 is used for the current prediction unit. hevc_intra_flag[x0][y0] equal to 0 specifies that another intra mode is used for the current prediction unit. When not present, hevc_intra_flag[x0][y0] is inferred to be equal to 1.

The variable DmmFlag[x0][y0] is derived as specified below:

DmmFlag[x0][y0]＝! hevc_intra_flag[x0][y0] (H-25)

depth_intra_mode[x0][y0] specifies the depth intra mode for the current prediction unit. depth_intra_mode[x0][y0] equal to 0 specifies DepthIntraMode[x0][y0] equal to INTRA_DEP_DMM_WFULL, and depth_intra_mode[x0][y0] equal to 1 specifies DepthIntraMode[x0][y0] equal to INTRA_DEP_DMM_CPREDTEX. When not present, DepthIntraMode[x0][y0] is inferred to be equal to INTRA_DEP_NONE.

wedge_full_tab_idx[x0][y0] specifies the index of the wedgelet mode in the corresponding mode list when DepthIntraMode[x0][y0] is equal to INTRA_DEP_DMM_WFULL.

H.7.4.9.5.1 Deep DC residual semantics

depth_dc_flag[x0][y0] equal to 1 specifies that depth_dc_abs[x0][y0][i] and depth_dc_sign_flag[x0][y0][i] are present. depth_dc_flag[x0][y0] equal to 0 specifies that depth_dc_abs[x0][y0][i] and depth_dc_sign_flag[x0][y0][i] are not present. When not present, depth_dc_flag[x0][y0] is inferred to be equal to 1.

depth_dc_abs[x0][y0][i], depth_dc_sign_flag[x0][y0][i] are used to derive DcOffset[x0][y0][i] as follows:

DcOffset[x0][y0][i]＝

(1-2*depth_dc_sign_flag[x0][y0][i])*(depth_dc_abs[x0][y0][i]-dcNumSeg+2) (H-27)

H.8.4.2 Derivation process for luma intra prediction mode

The input to this process is the luma position (xPb, yPb) that specifies the top-left sample of the current luma prediction block relative to the top-left luma sample of the current picture.

In this process, the luma intra prediction mode IntraPredModeY[xPb][yPb] is derived.

Table H-3 specifies the values and associated names for intra prediction modes.

Table H-3 - Specification of intra prediction modes and associated names

Intra prediction mode Associated Name 0 INTRA_PLANAR 1 INTRA_DC 2..34 INTRA_ANGULAR2..INTRA_ANGULAR34 35 INTRA_DMM_WFULL 36 INTRA_DMM_CPREDTEX

IntraPredModeY[xPb][yPb] labeled 0..34 represents the direction of prediction as illustrated in FIG. 81 .

- If DepthIntraMode[xPb][yPb] is equal to INTRA_DEP_DMM_WFULL, then set IntraPredModeY[xPb][yPb] equal to INTRA_DMM_WFULL.

Otherwise, if DepthIntraMode[xPb][yPb] is equal to INTRA_DEP_DMM_CPREDTEX, then set IntraPredModeY[xPb][yPb] equal to INTRA_DMM_CPREDTEX.

Otherwise (DepthIntraMode[xPb][yPb] is equal to INTRA_DEP_NONE), derive IntraPredModeY[xPb][yPb] as follows:

…

H.8.4.4.2.1 General intra-frame sample prediction

The inputs to this process are:

- a sample position (xTbCmp, yTbCmp) which specifies the top left sample of the current transform block relative to the top left sample of the current picture,

- variable predModeIntra that specifies the intra prediction mode,

- variable nTbS specifying the transform block size,

- Variable cIdx specifies the color component of the current block.

The output of this process is the prediction samples predSamples[x][y], where x, y = 0..nTbS-1.

Set the variable bSamplePredFlag to 1.

Otherwise, if predModeIntra is equal to INTRA_DMM_CPREDTEX, then the corresponding intra prediction mode specified in subclause H.8.4.4.2.8 is called with position (xTbY, yTbY), with sample array p and transform block size nTbS and bSamplePredFlag as input and output is the prediction sample array predSamples.

H.8.4.4.2.8 Specification of intra prediction mode INTRA_DMM_CPREDTEX

The inputs to this process are:

-Specify the sample position (xTb, yTb) of the top left sample of the current block relative to the top left sample of the current picture,

- adjacent samples p[x][y], where x=-1, y=-1..nTbS*2-1 and x=0..nTbS*2-1, y=-1,

- variable nTbS specifying the transform block size,

- Specifies whether to generate prediction samples variable bSamplePredFlag,

The output of this process is:

-Prediction samples predSamples[x][y], where x, y = 0..nTbS-1.

-wedgewave pattern wedgePattern[x][y], where x,y=0..nT-1.

Specify the values of the derived prediction samples predSamples[x][y] by the following ordered steps, where x, y = 0..nTbS-1:

1. Set the variable recTextPic to the array of reconstructed luma picture samples equal to TexturePic.

2. Export the variable textThresh that specifies the threshold for segmentation of recTextPic as specified below.

-Set the variable sumTextPicVals equal to 0.

- For x = 0..nTbS-1, the following applies

- For y = 0..nTbS-1, the following applies

sumTextPicVals+＝recTextPic[xTb+x][yTb+y] (H-45)

-Set the variable textThresh equal to (sumTextPicVals>>(2*log2(nTbS)))

3. As specified below, export the variable wedgeletPattern[x][y] that specifies the binary partition pattern, where x, y

=0..nTbS-1.

- For x = 0..nTbS-1, the following applies

- For y = 0..nTbS-1, the following applies

wedgeletPattern[x][y]＝(recTextPic[xTb+x][yTb+y]>

textThresh) (H-46)

4. When bSamplePredFlag is equal to 1, the depth partition value derivation and assignment process as specified in subclause H.8.4.4.2.9 is called with as input the neighboring samples p[x][y], the binary pattern wedgeletPattern[x][y], the transform size nT, dcOffsetAvailFlag set equal to depth_dc_flag[xTb][yTb], and the DC offsets DcOffset[xTb][yTb][0] and DcOffset[xTb][yTb][1], and the output is assigned

Give predSamples[x][y].

H.8.4.4.3 Depth value reconstruction process

The inputs to this process are:

- specifies the luma position (xTb, yTb) of the top left luma sample of the current block relative to the top left luma sample of the current picture,

- variable nTbS specifying the transform block size,

-Prediction samples predSamples[x][y], where x, y = 0..nTbS-1

- Intra prediction mode predModeIntra,

The output of this process is:

- reconstructed depth value samples resSamples[x][y], where x, y = 0..nTbS-1.

Set the variable bSamplePredFlag to 0.

Depending on predModeIntra, the array wedgePattern[x][y] specifying the binary segmentation pattern is derived as follows, where x, y = 0..nTbS-1.

- If predModeIntra is equal to INTRA_DMM_WFULL, then the following applies.

wedgePattern=WedgePatternTable[Log2(nTbS)][wedge_full_tab_idx[xTb][yTb]]

Otherwise, if predModeIntra is equal to INTRA_DMM_CPREDTEX, then subclause H.8.4.4.2.8 is called with the position (xB, yB), the prediction samples predSamples, the transform block size nT and bSamplePredFlag as input, and the output is the Wedgelet pattern wedgePattern.

- Otherwise (predModeIntra is not equal to INTRA_DMM_WFULL and predModeIntra is not equal to INTRA_DMM_CPREDTEX), the following applies.

- For x,y = 0..nTbS-1, set wedgePattern[x][y] equal to 0.

Depending on dlt_flag[nuh_layer_id], reconstructed depth value samples resSamples[x][y] are derived as specified below:

- If dlt_flag[nuh_layer_id] is equal to 0, then the following applies:

- For x,y=0..nTbS-1, reconstructed depth value samples resSamples[x][y] are derived as specified below:

resSamples[x][y]＝predSamples[x][y]+

DcOffset[xTb][yTb][wedgePattern[x][y]] (H-59)

Otherwise (dlt_flag[nuh_layer_id] is equal to 1), the following applies:

-Export variables dcPred[0] and dcPred[1] as specified below:

- If predModeIntra is not equal to INTRA_DMM_WFULL and predModeIntra is not equal to INTRA_DMM_CPREDTEX, then the following applies:

dcPred[0]＝(predSamples[0][0]+predSamples[0][nTbS-1]+

predSamples[nTbS-1][0]

+predSamples[nTbS-1][nTbS-1]+2)>>2(H-61)

- Otherwise, if predModeIntra is equal to INTRA_DMM_WFULL, then the following applies.

dcPred[wedgePattern[0][0]]=predSamples[0][0] (H-62)

dcPred[wedgePattern[nTbS-1][0]]=predSamples[nTbS-1][0](H-63)

dcPred[wedgePattern[0][nTbS-1]]=predSamples[0][nTbS-1](H-64)

dcPred[wedgePattern[nTbS-1][nTbS-1]]=predSamples[nTbS-1][nTbS-1](H-65)

Otherwise, (intraPredMode equals INTRA_DMM_CPREDTEX), the following applies.

- For x,y=0..nTbS-1, reconstructed depth value samples resSamples[x][y] are derived as specified below:

dltIdxPred＝DepthValue2Idx[dcPred[wedgePattern[x][y]]] (H-66)

dltIdxResi＝DcOffset[xTb][yTb][wedgePattern[x][y]] (H-67)

resSamples[x][y]＝predSamples[x][y]+Idx2DepthValue[dltIdxPred+dltIdxResi]-dcPred[wedgePattern[x][y]](H-68)

H.8.5.4.1 General

…

- If sdc_flag is equal to 0, then the following applies, depending on the value of rqt_root_cbf, the following applies:

If rqt_root_cbf is equal to 0 or skip_flag[xCb][yCb] is equal to 1, then all samples of the (nCbS _L )x(nCbS _L ) array resSamples _L and all samples of the two (nCbS _C )x(nCbS _C ) arrays resSamples _Cb and resSamples _Cr are set equal to 0.

Otherwise (rqt_root_cbf is equal to 1), the following ordered steps apply:

The decoding process for the luma residual block as specified in subclause H.8.5.4.2 below is called with as input the luma position (xCb, yCb), the luma position (xB0, yB0) set equal to (0, ₀ ), the variable log2TrafoSize set equal to log2CbSize, the variable trafoDepth set equal to ₀ , the variable nCbS set equal to nCbSL, and the (nCbSL)x( _nCbSL ) array _resSamplesL , and the output is a modified version of the ( _nCbSL )x( _nCbSL ) array _resSamplesL .

The decoding process for chroma residual blocks as specified in subclause H.8.5.4.3 below is invoked with as input the luma position (xCb, yCb), the luma position (xB0, yB0) set equal to (0, ₀ ), the variable log2TrafoSize set equal to log2CbSize, the variable trafoDepth set equal to 0, the variable cIdx set equal to ₁ , the variable _nCbS set equal to nCbSC, and the ( _{nCbSC )} x(nCbSC) array resSamplesCb, and the output is a modified version of the (nCbSC)x( _nCbSC ) array _resSamplesCb .

The decoding process for chroma residual blocks as specified in subclause H.8.5.4.3 below is invoked with as input the luma position (xCb, yCb), the luma position (xB0, yB0) set equal to (0, ₀ ), the variable log2TrafoSize set equal to log2CbSize, the variable trafoDepth set equal to 0, the variable cIdx set equal to ₂ , the variable nCbS set equal to _nCbSC , and the ( _nCbSC )x(nCbSC) array _resSamplesCr , and the output is a modified version of the (nCbSC)x( _nCbSC ) array _resSamplesCr .

Otherwise (sdc_flag is equal to 1), the decoding process for simplified depth coded residual blocks as specified in the following subclause H.8.5.4.4 is called with as input the luma position (xCb, yCb), the luma position (xB0, yB0) set equal to (0, 0), the variable log2TrafoSize set equal to log2CbSize, the variable trafoDepth set equal to 0, the variable nCbS set equal to nCbS _L , and the nCbS array resSamples _L , and the output is a modified version of the (nCbS _L )x(nCbS _L ) array resSamples _L.

…

H.8.5.4.4 is used to simplify the decoding process of depth-coded residual blocks

For x in the range 0 to nCbS, the following applies:

- For y in the range 0 to nCbS, the following applies:

-Export variable i is specified as follows:

- If x is less than xOff and y is less than yOff, then set i equal to 0.

- Otherwise, if x is greater than or equal to xOff and y is less than yOff, then set i equal to 1.

Otherwise, if x is less than xOff and y is greater than or equal to yOff, then set i equal to 2.

- Otherwise, (x is greater than or equal to xOff and y is greater than or equal to yOff), then set i equal to 3.

- Set the value of resSamples[x][y] to be equal to DcOffset[xCb][yCb][interSdcResiIdx[i]]

Table H-10 - Association of ctxIdx and syntax elements for each initializationType in the initialization process

Table H-20 - Syntax elements and associated binarization

Table H-22 - Assignment of ctxInc to Syntax Elements by Context Coded Binary

According to another example, changes are made to the syntax elements and semantics of 3D-HEVC. In the following syntax elements and semantics, the relevant parts of 3D-HEVC have been changed to add the syntax elements and semantics proposed in this example of the present invention. The changes to the syntax elements and semantics according to this example of the present invention are shown by the newly added parts presented in bold and the newly deleted parts marked in bold.

The syntax table is presented below, followed by the relevant semantics.

H.7.3.8.5 General decoding unit syntax

H.7.3.8.5.1 Depth mode parameter syntax

H.7.3.8.5.2 Depth DC residual syntax table

Semantics and decoding process

H.7.4.9.5 Decoding unit semantics

The variable SdcEnableFlag is set equal to 0, and the following apply:

SdcEnableFlag=

(vps_inter_sdc_flag[nuh_layer_id]&&PredMode[x0][y0]==MODE_INTER)||

(vps_depth_modes_flag[nuh_layer_id]

&&PredMode[x0][y0]==MODE_INTRA

&&PartMode[x0][y0]＝＝PART_2Nx2N

&&(IntraPredModeY[x0][y0]＝＝INTRA_DMM_WFULL||

IntraPredModeY[x0][y0]＝＝INTRA_PLANAR)) (H-16)

sdc_flag[x0][y0] equal to 1 specifies that slice-wise DC coding of the residual block is used for the current coding unit. sdc_flag[x0][y0] equal to 0 specifies that slice-wise DC coding of the residual block is not used for the current coding unit. When not present, sdc_flag[x0][y0] is inferred to be equal to 0.

H.7.4.9.5.1 Depth mode parameter semantics

Set the variable Log2MaxDmmCbSize equal to 5.

The variable depthIntraModeSet is exported as specified below:

- If log2CbSize is equal to 6, then set depthIntraModeSet equal to 0.

- Otherwise, if log2CbSize is equal to 3 and PartMode[xC][yC] is equal to PART_NxN, then depthIntraModeSet is set equal to 1.

Otherwise, set depthIntraModeSet equal to 2.

depth_intra_mode[x0][y0] specifies the depth intra mode of the current prediction unit. Table H-2 specifies the value of depthIntraModeMaxLen and the value of the variable DepthIntraMode depending on the variable depthIntraModeSet, and the associated name depending on depth_intra_mode and depthIntraModeSet.

The variable DmmFlag[x0][y0] is derived as specified below:

DmmFlag[x0][y0]＝(DepthIntraMode[x0][y0]＝＝INTRA_DEP_DMM_WFULL)||(H-26)

(DepthIntraMode[x0][y0]==INTRA_DEP_DMM_CPREDTEX)

Table H-2 - Specification of DepthIntraMode and associated names depending on depthIntraModeSet and depth_intra_mode, and specification of depthIntraModeMaxLen depending on depthIntraModeSet

H.7.4.9.5.2 Deep DC residual semantics

depth_dc_flag[x0][y0] equal to 1 specifies that depth_dc_abs[x0][y0][i] and depth_dc_sign_flag[x0][y0][i] are present. depth_dc_flag[x0][y0] equal to 0 specifies that depth_dc_abs[x0][y0][i] and depth_dc_sign_flag[x0][y0][i] are not present. When not present, depth_dc_flag[x0][y0] is inferred to be equal to 1.

depth_dc_abs[x0][y0][i], depth_dc_sign_flag[x0][y0][i] are used to derive DcOffset[x0][y0][i] as follows:

DcOffset[x0][y0][i]＝

(1-2*depth_dc_sign_flag[x0][y0][i])*(depth_dc_abs[x0][y0][i]-dcNumSeg+2)(H-27)

H.8.5.4.1 General

…

- If sdc_flag is equal to 0, then the following applies, depending on the value of rqt_root_cbf, the following applies:

- If rqt_root_cbf is equal to 0 or skip_flag[xCb][yCb] is equal to 1, then all samples of the (nCbS _L )x(nCbS _L ) array resSamples _L and the two (nCbS _C )x(nCbS _C ) arrays

All samples in resSamples _Cb and resSamples _Cr are set equal to 0.

Otherwise (rqt_root_cbf is equal to 1), the following ordered steps apply:

The decoding process for the luma residual block as specified in subclause H.8.5.4.2 below is called with as input the luma position (xCb, yCb), the luma position (xB0, yB0) set equal to (0, ₀ ), the variable log2TrafoSize set equal to log2CbSize, the variable trafoDepth set equal to ₀ , the variable nCbS set equal to nCbSL, and the (nCbSL)x( _nCbSL ) array _resSamplesL , and the output is a modified version of the ( _nCbSL )x( _nCbSL ) array _resSamplesL .

The decoding process for chroma residual blocks as specified in subclause H.8.5.4.3 below is invoked with as input the luma position (xCb, yCb), the luma position (xB0, yB0) set equal to (0, ₀ ), the variable log2TrafoSize set equal to log2CbSize, the variable trafoDepth set equal to 0, the variable cIdx set equal to ₁ , the variable _nCbS set equal to nCbSC, and the ( _{nCbSC )} x(nCbSC) array resSamplesCb, and the output is a modified version of the (nCbSC)x( _nCbSC ) array _resSamplesCb .

The decoding process for chroma residual blocks as specified in subclause H.8.5.4.3 below is invoked with as input the luma position (xCb, yCb), the luma position (xB0, yB0) set equal to (0, ₀ ), the variable log2TrafoSize set equal to log2CbSize, the variable trafoDepth set equal to 0, the variable cIdx set equal to ₂ , the variable nCbS set equal to _nCbSC , and the ( _nCbSC )x(nCbSC) array _resSamplesCr , and the output is a modified version of the (nCbSC)x( _nCbSC ) array _resSamplesCr .

Otherwise (sdc_flag is equal to 1), the decoding process for simplified depth coded residual blocks as specified in the following subclause H.8.5.4.4 is called with as input the luma position (xCb, yCb), the luma position (xB0, yB0) set equal to (0, 0), the variable log2TrafoSize set equal to log2CbSize, the variable trafoDepth set equal to 0, the variable nCbS set equal to nCbS _L , and the nCbS array resSamples _L , and the output is a modified version of the (nCbS _L )x(nCbS _L ) array resSamples _L.

…

H.8.5.4.4 is used to simplify the decoding process of depth-coded residual blocks

For x in the range 0 to nCbS, the following applies:

- For y in the range 0 to nCbS, the following applies:

-Export variable i is specified as follows:

- If x is less than xOff and y is less than yOff, then set i equal to 0.

- Otherwise, if x is greater than or equal to xOff and y is less than yOff, then set i equal to 1.

Otherwise, if x is less than xOff and y is greater than or equal to yOff, then set i equal to 2.

- Otherwise, (x is greater than or equal to xOff and y is greater than or equal to yOff), then set i equal to 3.

- Set the value of resSamples[x][y] to be equal to DcOffset[xCb][yCb][interSdcResiIdx[i]]

Table H-10 - Association of ctxIdx and syntax elements for each initializationType in the initialization process

Table H-20 - Syntax elements and associated binarization

Table H-22 - Assignment of ctxInc to Syntax Elements by Context Coded Binary

7-12 are flowcharts illustrating various example operations performed by video encoder 20 and/or video decoder 20 in accordance with this disclosure. The flowcharts are provided for illustrative purposes and should not be considered limiting. The order of the various operations is presented for illustrative purposes and does not necessarily indicate that the operations must be performed in the order described. Furthermore, in many cases, the operations illustrated in FIGs. 7-12 may be practiced in various combinations with one another.

FIG7 is a flowchart illustrating encoding syntax elements to indicate SDC use for both depth intra prediction and depth inter prediction modes, for example, at the CU level. Video encoder 20 may be configured to perform the operations of FIG7 . For example, video encoder 30 may select SDC mode for coding a partition of a PU of a depth CU. In some examples, video encoder 30 may transmit syntax elements to enable or disable SDC mode for coding depth CUs in the entire encoded video sequence. Subsequently, in some examples, if SDC is enabled for a sequence, video encoder 30 may signal a syntax element for each depth CU in the sequence to indicate whether SDC is selected for the depth CU. Video encoder 30 selects either inter coding mode or intra coding mode for each CU ( 202 ) and then applies inter prediction for the depth CU ( 204 ) or intra prediction for the depth CU ( 206 ).

Various types of SDC intra prediction may be used. In some examples, for SDC, for a currently intra-coded CU, where the syntax element indicates whether SDC mode is used for the current depth CU, coding may include using samples from a texture component corresponding to a depth component of the depth CU when performing intra prediction to generate a predicted depth CU.

For depth coding, video encoder 30 may use a regular HEVC intra-prediction or inter-prediction mode or a DMM mode, such as a Wedgelet, Contour, or Plane partitioning mode. In either case, video encoder 20 may apply SDC to generate one or more DC residual values for one or more depth CU partitions (208). For example, with SDC, video encoder 20 may encode only one DC residual value for a partition associated with each PU of a depth CU. Thus, video encoder 20 signals one DC residual value for each PU and uses the one DC residual value as the residual for all samples in the PU. The PU may be an entire CU or an individual partition within the CU, such as a partition defined by Wedgelet or Contour partitioning.

7 , video encoder 20 may encode a syntax element indicating that SDC is used for a CU, whether for intra or inter mode, i.e., for both depth intra and depth inter prediction ( 210 ). Thus, rather than separately signaling SDC for depth intra and depth inter prediction independently, video encoder 20 generates a single syntax element for a CU that indicates that SDC applies to the CU, regardless of whether the CU is coded with depth intra or depth inter prediction. The syntax element may be an sdc_flag syntax element. The sdc_flag syntax element may be a one-bit flag. As an example, an sdc_flag value of 1 indicates that SDC is to be used for both depth intra and depth inter prediction, and an sdc_flag value of 0 indicates that SDC is not to be used for depth intra and depth inter prediction. Again, another syntax element may enable or disable SDC for the entire sequence. If SDC is enabled, the sdc_flag may be signaled at the CU level. If SDC is disabled for a sequence, then in some examples, sdc_flag is not signaled. Again, the enabling or disabling of SDC may be applied based on sequence, picture, or slice, while sdc_flag may be sent to indicate actual SDC usage at the CU level.

In some examples, video encoder 20 may signal the SDC syntax element sdc_flag in general coding unit parameters at the CU level. Thus, video encoder 20 may signal the SDC syntax element on a CU-by-CU basis. In other examples, as discussed above, video encoder 20 may signal another syntax element in a slice segment header that indicates that SDC is enabled (or disabled) for all depth CUs in a slice segment. In still other examples, the SDC syntax element generated by video encoder 20 may indicate that SDC is enabled (or disabled) for all depth CUs in an entire picture or an entire video sequence. For example, video encoder 20 may signal the enable/disable SDC syntax element in a video parameter set (VPS) extension (e.g., for 3D-HEVC), in a sequence parameter set (SPS), or in a picture parameter set (PPS). In some examples, one or more syntax elements signaled in a VPS, SPS, PPS, slice header, or the like may be used to calculate the value of an enable or disable flag that may be signaled separately, for example, at the CU level. Thus, a first SDC enable syntax element may be used to enable or disable SDC, for example, for all depth CUs in a slice, all depth CUs in a picture, or all depth CUs in the entire encoded video sequence, and a second SDC syntax element (e.g., sdc_flag) may be configured to indicate that SDC is actually used for both depth intra and depth inter prediction of individual depth CUs.

Video encoder 20 may also encode syntax structures with SDC residual data. For example, when the current depth CU is SDC coded, for each partition, the DC residual value for each partition for each PU of an inter-coded CU or an intra-coded CU may be presented in one syntax structure. The DC residual value may be a DC offset in the pixel value domain. This single syntax structure for a partition includes DC residual data for the partition, regardless of whether the partition is intra- or inter-predicted. For example, for intra- or inter-prediction, the syntax structure may include the absolute value of the DC residual value and its sign (positive or negative). Examples of DC residual syntax elements include depth_dc_abs, which indicates the residual, and depth_dc_sign_flag, which indicates the sign. For example, depth_dc_abs and depth_dc_sign_flag may be used to derive a DcOffset value. In this way, video encoder 20 may be configured to unify DC residual data for an inter-coded CU or an intra-coded CU to be presented in one syntax structure. In some examples, the DC residual data in the syntax structure may include the DC residual value and sign. In other examples, the DC residual data may be signaled using a DLT. In this case, the DC residual data may be signaled as, for example, the difference between the DLT index of the original DC value and the DLT index of the predicted DC value for a depth PU or partition.

As an alternative or in addition, video encoder 20 may be configured to signal and encode a flag for an intra-coded CU that indicates whether the CU contains any non-zero DC residual values in any of its partitions. If this flag is 0, video encoder 20 does not signal any DC residual values for the intra-coded CU, and decoder 30 infers that the DC residual value is equal to zero for each partition of the CU. As another alternative, a flag generated by video encoder 20 to indicate whether a CU contains any non-zero DC residual values in any of its partitions may be applied to both intra-coded and inter-coded CUs.

The syntax structure generated by video encoder 20 and used to signal the DC residual values for both intra-predicted and inter-predicted CUs may be a single syntax structure, as described above. That is, a single syntax structure may be generated for each depth CU, regardless of whether the CU is intra-predicted or inter-predicted. In this way, intra-prediction and inter-prediction modes may share the same syntax elements to signal the DC residual values for partitions of the depth CU. Video encoder 20 may entropy code the syntax elements of the syntax structure using the same context model and/or binarization process for the relevant syntax elements, regardless of whether the depth CU is intra-predicted or inter-predicted, thereby unifying the entropy coding process for intra-prediction and inter-prediction with SDC.

Alternatively, video encoder 20 may generate separate instances of a syntax structure, namely, one syntax structure including DC residual data for intra prediction and one syntax structure including DC residual data for inter prediction, where the instances of the syntax structure are identical or substantially identical in that the same syntax elements are included in each instance of the syntax structure. In this case, video encoder 30 may entropy code the separate instances of the syntax structure separately and use the same or separately maintained context, and possibly binarization, for the intra prediction instance of the syntax structure and the inter prediction instance of the syntax structure. Even if the context is maintained separately, intra SDC and inter SDC modes may still use the same syntax elements and the same syntax structure (but in different instances) to carry DC residual data for partitions of PUs of a depth CU.

Video encoder 214 encodes intra-prediction or inter-prediction information for prediction of one or more PUs associated with a depth CU (214). For example, video encoder 214 may encode an indication of the coding mode used for each PU and any other information used for inter- or intra-prediction of each PU on the decoder side. For example, for SDC intra-prediction, video encoder 20 may encode intra-coding mode information for HEVC intra-prediction mode, DMM mode, or other intra-modes. For example, for inter-prediction, video encoder 20 may encode motion information for the generation of inter-predicted depth PUs and/or partitions. Video encoder 214 may entropy code SDC syntax elements (i.e., indicating the use of SDC at the CU level), SDC syntax structures (i.e., indicating DC values for PU partitions), and intra- or inter-prediction information for use by decoder 30 on the decoder side to decode and reconstruct the depth CU as part of a 3D decoding process, such as a 3D-HEVC decoding process.

FIG8 is a flowchart illustrating the decoding of SDC syntax elements, such as the syntax elements described above with reference to FIG7 , to indicate the use of SDC for both depth intra prediction and depth inter prediction modes. Generally speaking, FIG8 describes the process illustrated in FIG7 from the decoder-side perspective of video decoder 30. Thus, the various operations and syntax details described with reference to FIG7 may be similarly applied to FIG8 , but from the perspective of video decoder 30. As shown in FIG8 , video decoder 30 may be configured to decode intra and/or inter prediction mode information for a depth CU to generate predicted depth information. Additionally, video decoder 30 may receive and decode an SDC syntax element (e.g., sdc_flag) that indicates whether SDC is to be used for a CU, i.e., whether depth intra prediction or depth inter prediction mode is to be used. In other words, the syntax element is used to signal SDC for both intra and inter CUs.

As discussed with reference to FIG7 , rather than independently receiving signaling of SDC for depth intra and depth inter prediction, video decoder 20 receives an SDC syntax element indicating that SDC applies to both depth intra and depth inter prediction. Again, the SDC syntax element may be an sdc_flag syntax element and may otherwise conform to the description of the syntax element described above with reference to FIG7 . An sdc_flag value of 1 may indicate that SDC is to be used for both depth intra and depth inter prediction of a CU, and an sdc_flag value of 0 may indicate that SDC is not to be used for depth intra and depth inter prediction of a CU.

Video decoder 30 may receive an SDC syntax element in a general coding unit parameter at the CU level. Thus, video decoder 30 may receive and decode the SDC syntax element on a CU-by-CU basis. In some examples, video decoder 30 may receive another SDC syntax element in a slice segment header indicating that SDC is enabled for all depth CUs in a slice segment. In still other examples, video decoder 30 may receive another syntax element indicating that SDC is enabled for intra and inter prediction of depth CUs in the entire coded video sequence or in a picture. For example, video decoder 30 may receive additional SDC syntax elements (indicating whether SDC is enabled or disabled) in a video parameter set (VPS) extension (e.g., for 3D-HEVC), in a sequence parameter set (SPS), or in a picture parameter set (PPS). Thus, video decoder 30 may receive a first SDC syntax element indicating that SDC is enabled for a CU in a sequence, picture, or slice, and a second SDC element (e.g., sdc_flag) at the CU level indicating that SDC is actually used for a particular CU.

If SDC is indicated as not used (e.g., sdc_flag = 0) for intra prediction and inter prediction (224), decoder 30 decodes the bitstream to obtain non-SDC residual data for reconstruction of the intra- or inter-predicted PU. If the syntax element indicates SDC use (e.g., sdc_flag = 1) for intra prediction and inter prediction (224), decoder 30 decodes the syntax structure to generate SDC residual data for the partition of the depth PU of the current CU. This may be repeated for multiple partitions of PUs in the CU. Decoder 30 may receive multiple syntax structures, where the syntax structures include SDC residual data for the respective depth PU partitions. As described with reference to FIG8, the syntax structure for each depth PU may be a single syntax structure, whether the depth PU is intra- or inter-coded, or separate instances of the same syntax structure for intra- and inter-coded depth PUs, respectively. When the current depth CU is SDC-coded, for each partition, the DC residual value for the inter-coded CU or the intra-coded CU may be presented in the syntax structure. Thus, this syntax structure for the partition includes DC residual data for the partition, regardless of whether the partition is intra- or inter-predicted.

For example, for intra prediction or inter prediction, the syntax structure decoded by video decoder 30 may include the absolute value of the DC residual value and its sign (positive or negative). Examples of the DC residual syntax element in the syntax structure include depth_dc_abs to indicate the residual and depth_dc_sign_flag to indicate the sign. In this way, by using the same syntax structure, or at least the same syntax elements, to convey DC residual data for both depth intra and depth inter coding, video decoder 30 may be configured to unify DC residual values for inter-coded CUs or intra-coded CUs. In some examples, the DC residual data in the syntax structure may include the DC residual value and the sign. In other examples, the DC residual data may be signaled using a DLT. In this case, the DC residual data may be signaled as, for example, the difference between the DLT index for the original DC value and the DLT index for the predicted DC value for the depth PU or partition.

In some examples, as described with reference to FIG7 , in the operation of FIG8 , video decoder 30 may be configured to signal and decode a flag for an intra-coded CU that indicates whether the CU contains any non-zero DC residual values in any of its partitions. If this flag is 0, video decoder 20 does not parse any DC residual values for the intra-coded CU and instead infers a DC residual value equal to zero for each partition of the CU. As another alternative, the flag decoded by video decoder 30 to indicate whether a CU contains any non-zero DC residual values in any of its partitions may be applied to both intra-coded and inter-coded CUs.

The syntax structure decoded by video decoder 30 may be a single syntax structure, as described above. That is, a single syntax structure may be generated for each depth CU, regardless of whether the CU is intra-predicted or inter-predicted. In this way, intra-prediction and inter-prediction modes may share the same syntax elements to signal the DC residual values of the partitions of the depth CU. Video decoder 20 may entropy decode the syntax structure using the same context model and at least one of the binarization process for the relevant syntax elements, regardless of whether the depth CU is intra-predicted or inter-predicted, thereby unifying the entropy decoding process for intra-prediction and inter-prediction with SDC. At least one of the context model and the binarization process used for the context-adaptive entropy coding process may be the same for the same syntax elements (e.g., depth_dc_abs and depth_dc_sign_flag) in the syntax structure.

Alternatively, in the operation of FIG8 , video decoder 30 may receive and decode separate instances of a syntax structure, namely, one syntax structure including DC residual data for intra prediction and one syntax structure including DC residual data for inter prediction, where the instances of the syntax structure are substantially identical in that the same syntax elements are included in each instance of the syntax structure, as described with reference to FIG7 . In this case, video decoder 30 may entropy decode the separate instances of the syntax structure separately but with the same syntax elements, using the same or separately maintained context for the intra and inter prediction instances of the syntax structure, and possibly using binarization. Again, the same syntax elements, depth_dc_abs to indicate the residual and depth_dc_sign_flag to indicate the sign, may be used, regardless of whether a single syntax structure or separate instances of the syntax structure are provided. In each case, the same syntax elements may be used to convey SDC residual data for both intra and inter prediction modes. In some examples, at least one of the context model and the binarization process for the context adaptive entropy coding process may be the same for the same syntax elements (e.g., depth_dc_abs and depth_dc_sign_flag) in different instances of the syntax structure. In other examples, one or both of the context model and the binarization process for the context adaptive entropy coding process may be different for the syntax elements (e.g., depth_dc_abs and depth_dc_sign_flag) in different instances of the syntax structure.

In the case of SDC coding or non-SDC coding, video decoder 30 determines whether depth intra mode or depth inter mode applies to the current CU (230). Video decoder 30 inter-predicts each depth PU (232) or intra-predicts each depth PU (234), as applicable, using mode information provided in the encoded video bitstream. The mode information may include intra-coding mode information, such as HEVC intra-prediction mode, DMM mode, or other intra-mode, for generating intra-predicted depth PUs and/or partitions, or inter-coding mode information and motion information for generating inter-predicted depth PUs and/or partitions. Using this information, video decoder 30 generates predictive samples for the depth PUs and/or partitions.

Video decoder 30 reconstructs the partitions of the PUs of the depth CU using the SDC residual data or non-SDC residual data and the predicted depth PU. For example, in the case of SDC coding, for a given PU, video decoder 30 applies the DC residual value signaled in the SDC syntax structure, or derives the DC residual value from the DLT based on the index value difference or other information provided in the SDC syntax structure. Video decoder 30 adds the DC residual value to the predicted samples of the predicted PU partition to reconstruct the original samples. In this way, video decoder 30 uses the SDC syntax elements and SDC syntax structure to unify SDC intra-frame and inter-frame decoding, rather than using separate syntax elements and syntax structures for intra-frame and inter-frame decoding, respectively.

FIG9 is a flowchart illustrating decoding a syntax element to indicate SDC usage for depth blocks in an entire encoded video sequence. In general, the operations illustrated in FIG9 from a decoder-side perspective represent possible features for use with the methods shown in FIG7 and 8. In the example of FIG9, video decoder 30 decodes an SDC syntax element indicating whether SDC is enabled or disabled for both depth intra prediction and depth inter prediction (240). Video decoder 30 may receive the SDC syntax element, for example, in a video parameter set (VPS) extension, a sequence parameter set (SPS), a picture parameter set (PPS), or a slice segment header. When received in a VPS or SPS, if the SDC syntax element indicates activation of SDC, video decoder 30 should consider SDC to be enabled for all intra- and inter-prediction modes for the entire video sequence and should receive another syntax element (e.g., sdc_flag) at the CU level indicating whether SDC is used for that CU. As an alternative, when an SDC syntax element is received in a PPS and indicates activation of SDC, video decoder 30 may receive additional SDC syntax elements (e.g., sdc_flag) at the CU level for all depth blocks of the corresponding picture. As another alternative, when an SDC syntax element is received in a slice segment header and indicates activation of SDC, video decoder 30 may receive additional SDC syntax elements (e.g., sdc_flag) at the CU level for all depth blocks of the corresponding slice segment.

9 , the video decoder interprets SDC as enabled for depth intra and depth inter coding modes for the entire coded video sequence and may parse the bitstream for CUs in the sequence to obtain the sdc_flag syntax element at the CU level. If SDC is not enabled (242), video decoder 30 applies regular non-SDC residual coding to depth blocks of the entire coded video sequence. In this way, SDC enablement or disablement may be signaled once for depth intra and inter modes and for the entire coded video sequence. Thus, one syntax element may be used to enable/disable both intra and inter SDC modes in the decoding process for the entire coded video sequence. Alternatively, SDC enablement/disablement may be signaled once for an entire picture or slice segment using a syntax element. In each case, the actual SDC usage for a CU may be signaled at the CU level in a syntax element in the general coding unit parameters.

In some examples, rather than indicating with one syntax element whether SDC is enabled for intra and inter modes for the entire coded video sequence or for a picture or slice segment, video encoder 20 may encode and video decoder 30 may decode a single (first) flag to indicate whether SDC is enabled for depth intra prediction mode. For example, this first flag may be used to indicate whether SDC is enabled for intra mode for the entire coded video sequence. In some examples, if the first flag is true, i.e., SDC is to be enabled for intra mode for the entire coded video sequence, video encoder 20 encodes and video decoder 30 decodes an additional (second) flag to indicate whether SDC is enabled for inter mode for the entire coded video sequence, i.e., whether SDC is also enabled for inter mode for the entire coded video sequence.

If the first flag is not true and SDC is not enabled for intra mode, video encoder 20 need not generate, and video decoder 30 need not parse, the second flag indicating whether SDC is enabled for inter mode. In some examples, the first flag to indicate enablement of SDC intra-frame and the second flag to indicate enablement of SDC inter-frame may be provided in the VPS or SPS for the entire coded video sequence. Alternatively, in some examples, the first flag to indicate enablement of SDC intra-frame and the second flag to indicate enablement of SDC inter-frame may be provided in the PPS or slice segment header. In each case, actual SDC usage for a CU may be indicated at the CU level, for example, using sdc_flag.

FIG10 is a flowchart illustrating decoding a single syntax structure to obtain SDC residual data for intra and inter prediction modes. In the example of FIG10 , video decoder 30 decodes a single SDC syntax structure for both depth intra and depth inter modes as described above with reference to FIG7 and 8. As shown in FIG10 , video decoder 30 decodes an SDC syntax element at the CU level, for example, for a depth CU, that indicates whether SDC is used for both depth intra and depth inter prediction. For the purposes of FIG10 , it is assumed that the syntax element indicates the use of SDC, in which case video decoder 30 further decodes a single SDC syntax structure that includes SDC residual data for one or more partitions of a PU of a given depth CU.

In the example of FIG10 , and as described above with reference to FIG7 and 8 , the single SDC syntax structure can be a single syntax structure sent for both depth intra or inter mode, or separate instances of the same syntax structure used for depth intra mode and depth inter mode, respectively. In each case, the syntax structure uses the same syntax elements for depth intra mode and depth inter mode. For example, the SDC syntax structure can include syntax elements indicating the DC residual value and sign, or a DLT index difference value used to derive the DC residual value. Examples of syntax elements for indicating DC residual data include depth_dc_abs to indicate the residual and depth_dc_sign_flag to indicate the sign.

Upon determining whether the depth CU is inter-coded or intra-coded (254), video decoder 30 inter-predicts (256) or intra-predicts (258) reference samples of the current depth PU. Video decoder 30 then reconstructs partitions of the PU of the depth CU using SDC residual data signaled in a single SDC syntax structure for depth intra or depth intra mode, or signaled in separate instances of the same SDC syntax structure for depth intra mode or depth inter mode, respectively. For example, video decoder 30 adds DC residual values to prediction samples of the intra- or inter-predicted PU to reconstruct the original samples.

Relevant syntax elements for both depth intra prediction and depth inter prediction modes may be unified if the syntax structure for the residual values generated by SDC shares the same syntax elements and/or the same syntax structure, context model, and binarization process, such as for a context-adaptive binary arithmetic coding (CABAC) entropy coding process performed by video encoder 20 and video decoder 30. For example, the syntax elements for the residual values generated by SDC may be entropy encoded by video encoder 20 and entropy decoded by video decoder 30 using the same context model for both depth intra prediction mode and depth inter prediction mode.

Again, as an alternative to using a single SDC syntax structure, separate instances of the same SDC syntax structure may be encoded by video encoder 20 and decoded by video decoder 30 to provide SDC residual data for depth intra and depth inter modes. A first instance of the SDC syntax structure may be coded (i.e., encoded or decoded) to provide SDC residual data for depth intra mode, and a second instance of the SDC syntax structure may be coded to provide SDC residual data for depth inter mode. Depending on which depth mode (intra or inter) is used for the depth CU, video decoder 30 decodes and retrieves syntax elements from the relevant instances of the SDC syntax structure (e.g., the first instance for intra and the second instance for inter), and reconstructs the depth CU using the DC residual data indicated by the syntax elements. In this example, the same syntax elements are used in the syntax structure for each instance of the syntax structure. Video encoder 20 and video encoder 30 may maintain separate contexts and/or binarizations for entropy coding the separate instances of the SDC syntax structure. Alternatively, separate instances may be entropy coded (eg, CABAC) using the same context and/or binarization.

FIG11 is a flowchart illustrating an example constraint on SDC coding used at the decoder side. As shown in FIG11 , video decoder 30 decodes an SDC syntax element, such as sdc_flag (262). When the SDC syntax element indicates, for example, that SDC is used for a CU (264), video decoder 30 applies a constraint such that, for example, pulse code modulation (PCM) is disabled for the current CU (266). When SDC coding is not used for a CU, PCM may remain enabled (267). In this way, video decoder 30 constrains its coding operations to not include PCM when SDC is signaled for a CU. With the constraint that PCM is disabled, in some examples, video decoder 30 does not need to parse PCM mode information.

FIG12 is a flowchart illustrating another example constraint for SDC coding used at the decoder side. As an example of an additional constraint, video decoder 30 may determine the partition size of the current CU (268), or the partition size of each CU to be coded in a slice, picture, or video sequence. In one example, if the partition size of the current CU is not equal to 2Nx2N, video decoder 30 disables SDC for the relevant CU (272). If the partition size is equal to 2Nx2N, video decoder 30 may use SDC (270). The use of SDC will depend on the value of sdc_flag for the CU. In this way, video decoder 30 applies a constraint so that SDC is only used for CUs with a partition size of 2Nx2N. As an alternative, video decoder 30 may apply a constraint so that SDC is only used for inter CUs with a partition size of 2Nx2N and intra CUs with a partition size of 2Nx2N or NxN. Video encoder 20 may be configured to apply SDC mode only for specific partition sizes as described above. However, by imposing a constraint on the decoder side, video encoder 20 may not have to signal which specific CU uses SDC mode prediction. Instead, video encoder 20 may simply generate an SDC syntax element to indicate whether SDC is used for both depth intra and depth inter modes, and then apply SDC to CUs that meet the partition size requirement. With this constraint, video decoder 30 applies non-SDC coding to CUs that do not meet the applicable partition size requirement.

The techniques described above may be performed by video encoder 20 (FIGS. 1 and 5) and/or video decoder 30 (FIGS. 1 and 6), both of which may generally be referred to as video coders. Additionally, video coding may generally refer to video encoding and/or video decoding, where applicable.

Although the techniques of this disclosure are generally described with respect to 3D-HEVC, they are not limited in this manner. The techniques described above may also be applicable to other current or future standards for video coding. For example, the techniques for depth coding may also be applicable to other current or future standards that require coding of depth components, such as for 3D video coding or other applications.

In one or more examples, the functions described herein may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to tangible media such as data storage media or communication media including, for example, any media that facilitates the transfer of a computer program from one location to another according to a communication protocol. In this manner, computer-readable media may generally correspond to (1) tangible computer-readable storage media, which is non-transitory, or (2) communication media, such as a signal or carrier wave. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and/or data structures for implementing the techniques described in this disclosure. A computer program product may include computer-readable media.

By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage, flash memory, or any other medium that can be used to store the desired program code in the form of instructions or data structures and that can be accessed by a computer. Furthermore, any connection can properly be referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but rather are directed to non-transitory tangible storage media. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the term "processor," as used herein, may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or incorporated into a combined codec. Furthermore, the techniques may be fully implemented in one or more circuits or logic elements.

The techniques of this disclosure can be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). The various components, modules, or units described herein are intended to emphasize functional aspects of a device configured to perform the disclosed techniques, but do not necessarily require implementation by different hardware units. In fact, as described above, the various units can be combined in a codec hardware unit in conjunction with appropriate software and/or firmware, or provided by a collection of interoperable hardware units, including one or more processors as described above. Various examples are described. These and other examples are within the scope of the following claims.

Claims (87)

1. A method for decoding video data, the method comprising: The received indication simplifies whether the SDC mode is used for both intra-frame prediction and inter-frame prediction syntax elements of the deep decoding unit (CU) of the video data. When the depth CU is predicted intra-frame, intra-frame prediction is performed to generate the predicted depth CU; When the depth CU is predicted by inter-frame prediction, inter-frame prediction is performed to generate the predicted depth CU; When the syntax element instructs the use of the SDC mode for each partition of the prediction unit PU of the depth CU, information representing at least one DC residual value of the depth CU is received, wherein the at least one DC residual value represents the pixel difference between the partition of the PU of the depth CU and the corresponding partition of the predicted depth CU; and The depth CU is reconstructed using the at least one DC residual value and the predicted depth CU. 2. The method of claim 1, wherein the syntax element includes a one-bit flag having a first value and a second value, the first value indicating that the SDC mode is used for both intra-frame prediction and inter-frame prediction of the depth CU, and the second value indicating that the SDC mode is not used for either intra-frame prediction or inter-frame prediction of the depth CU. 3. The method of claim 1, wherein the syntax element is a first syntax element, the method further comprising obtaining in a video decoder, when the first syntax element indicates the use of the SDC mode, a syntax structure comprising one or more second syntax elements including information indicating at least one DC residual value representing a partition of the PU of the depth CU, wherein the second syntax elements of the syntax structure are the same for the intra-frame prediction and the inter-frame prediction of the depth CU. 4. The method of claim 3, wherein obtaining the syntax structure comprises obtaining a single syntax structure for the intra-frame prediction and the inter-frame prediction, wherein the single syntax structure comprises a second syntax element indicating information representing at least one DC residual value of the intra-frame prediction and the inter-frame prediction of the depth CU. 5. The method of claim 3, wherein obtaining the syntax structure comprises obtaining different instances of the same syntax structure for the intra-frame prediction and the inter-frame prediction, each of the different instances of the syntax structure containing the same second syntax element. 6. The method of claim 5, wherein obtaining the grammatical structure comprises entropy decoding the different instances of the grammatical structure using a context-adaptive entropy decoding process, and wherein at least one of the context model used in the context-adaptive entropy decoding process and the binaryization process is identical for the same grammatical elements in the different instances of the grammatical structure. 7. The method of claim 1, further comprising, when the syntax element indicates the use of the SDC mode, receiving in the video decoder a flag indicating the presence of any non-zero DC residual value for any partition of the depth CU for at least one or the intra-frame prediction or the inter-frame prediction, the method further comprising, when the flag indicates the absence of any non-zero DC residual value for any partition of the depth CU, not receiving the information representing the at least one DC residual value of the depth CU, and inferring that the at least one DC residual value is zero. 8. The method of claim 1, further comprising disabling the use of pulse code modulation (PCM) decoding of the video decoder for the depth CU when the syntax element indicates the use of the SDC mode. 9. The method of claim 1, further comprising restricting the use of the SDC mode to a depth CU having a partition size of 2Nx2N. 10. The method of claim 1, further comprising restricting the use of the SDC mode to inter-frame prediction depth CUs having a 2Nx2N partition size and intra-frame prediction depth CUs having a 2Nx2N or NxN partition size. 11. The method of claim 1, further comprising using the SDC mode for reconstruction of each of a plurality of depth CUs in the entire decoded video sequence when the syntax element indicates the use of the SDC mode. 12. The method of claim 11, wherein obtaining the syntax element comprises obtaining the syntax element from one of a video parameter set (VPS), a sequence parameter set (SPS), or a picture parameter set (PPS). 13. The method of claim 1, wherein obtaining the syntax element comprises obtaining the syntax element in a slice fragment header associated with the depth CU. 14. The method of claim 1, wherein the depth CU is the current depth CU, and the syntax element indicates whether the SDC mode is used for the current depth CU, the method further comprising using the SDC mode for the reconstruction of the current depth CU when the syntax element indicates the use of the SDC mode. 15. The method of claim 1, wherein the depth CU is a current intra-decoded depth CU, and the syntax element indicates whether the SDC mode is used for the current depth CU, the method further comprising using samples from texture components corresponding to the depth components of the depth CU when performing the intra-prediction to generate the predicted depth CU. 16. A method for encoding video data, the method comprising: When the depth CU is predicted intra-frame, intra-frame prediction is performed to generate the predicted depth CU; When the depth CU is predicted by inter-frame prediction, inter-frame prediction is performed to generate the predicted depth CU; When using the Simplified Depth Decoding (SDC) mode, for each partition of the prediction unit (PU) of the depth CU, information representing at least one DC residual value of the depth CU is generated, wherein the at least one DC residual value represents the pixel difference between the partition of the PU of the depth CU and the corresponding partition of the predicted depth CU. When the SDC mode is used, syntax elements are generated that indicate whether the SDC mode is used for both intra-frame prediction and inter-frame prediction of the depth decoding unit (CU) of the video data; and The depth CU is encoded in the video encoder based on the information representing the at least one DC residual value and the syntax elements. 17. The method of claim 16, wherein the syntax element includes a one-bit flag having a first value and a second value, the first value indicating that the SDC mode is used for both intra-frame prediction and inter-frame prediction of the depth CU, and the second value indicating that the SDC mode is not used for either intra-frame prediction or inter-frame prediction of the depth CU. 18. The method of claim 16, wherein the syntax element is a first syntax element, the method further comprising, when using the SDC mode: The video encoder generates a syntax structure comprising one or more second syntax elements indicating information representing the at least one DC residual value of the depth CU, wherein the second syntax elements of the syntax structure are identical for the intra-frame prediction and the inter-frame prediction of the depth CU; and The syntax structure used for the depth CU is encoded. 19. The method of claim 18, wherein generating the syntax structure comprises generating a single syntax structure for the intra-frame prediction and the inter-frame prediction, wherein the single syntax structure comprises a second syntax element indicating information representing at least one DC residual value of the intra-frame prediction and the inter-frame prediction of the depth CU. 20. The method of claim 18, wherein generating the syntax structure comprises obtaining different instances of the syntax structure for the intra-frame prediction and the inter-frame prediction, each of the different instances of the syntax structure containing the same second syntax element. 21. The method of claim 20, wherein generating the grammatical structure includes entropy decoding the different instances of the grammatical structure using a context-adaptive entropy encoding process, and wherein the context model used for the context-adaptive entropy encoding process is the same for the different instances of the grammatical structure. 22. The method of claim 16, further comprising, when using the SDC mode, generating a flag indicating the presence of any non-zero DC residual value for any partition of the depth CU for at least one or the intra-frame prediction or the inter-frame prediction, the method further comprising not generating the information representing the at least one DC residual value of the depth CU when the flag indicates the absence of a non-zero DC residual value for any partition of the depth CU. 23. The method of claim 16, further comprising restricting the use of the SDC mode to a depth CU having a partition size of 2Nx2N. 24. The method of claim 16, further comprising restricting the use of the SDC mode to inter-frame prediction depths (CUs) having a partition size of 2Nx2N and intra-frame prediction depths (CUs) having a partition size of 2Nx2N or NxN. 25. The method of claim 16, further comprising encoding the syntax element to indicate the use of the SDC mode for each of the plurality of depth CUs in the entire decoded video sequence. 26. The method of claim 25, wherein encoding the syntax element comprises encoding the syntax element in one of a video parameter set (VPS), a sequence parameter set (SPS), or a picture parameter set (PPS). 27. The method of claim 16, wherein encoding the syntax element comprises encoding the syntax element in a slice header associated with the depth CU. 28. The method of claim 16, wherein the depth CU is a current depth CU, and the syntax element indicates whether the SDC mode is used for the current depth CU, the method further comprising encoding the syntax element used for the current depth CU. 29. The method of claim 16, wherein the depth CU is a current intra-decoded depth CU, and the syntax element indicates whether the SDC mode is used for the current depth CU, the method further comprising using samples from texture components corresponding to the depth components of the depth CU when performing the intra-prediction to generate the predicted depth CU. 30. A video decoder, comprising: The memory, which stores video data; and One or more processors, configured to: The syntax elements of both intra-frame prediction and inter-frame prediction of the deep decoding unit (CU) indicating whether the simplified deep decoding (SDC) mode is used for the video data are decoded. When the depth CU is predicted intra-frame, intra-frame prediction is performed to generate the predicted depth CU; When the depth CU is predicted via inter-frame prediction, inter-frame prediction is performed to generate the predicted depth CU; and When the SDC mode is used, for each partition of the prediction unit PU of the depth CU, information representing at least one DC residual value of the depth CU is decoded, wherein the at least one DC residual value represents the pixel difference between the partition of the PU of the depth CU and the corresponding partition of the predicted depth CU. 31. The video decoder of claim 30, wherein the video decoder includes a video decoder, and the one or more processors are configured to decode the syntax elements, decode information representing the at least one DC residual value, and reconstruct the depth CU using the at least one DC residual value and the predicted depth CU. 32. The video decoder of claim 30, wherein the video decoder includes a video encoder, and the one or more processors are configured to encode the information representing the at least one DC residual value and the syntax elements. 33. The video decoder of claim 30, wherein the syntax element includes a flag having a first value and a second value, the first value indicating that the SDC mode is used for both intra-frame prediction and inter-frame prediction of the depth CU, and the second value indicating that the SDC mode is not used for either intra-frame prediction or inter-frame prediction of the depth CU. 34. The video decoder of claim 30, wherein the syntax element is a first syntax element, and the one or more processors are configured to decode a syntax structure comprising one or more second syntax elements including information indicating the at least one DC residual value of the depth CU when the first syntax element indicates the use of the SDC mode, and wherein the second syntax element of the syntax structure is the same for the intra-frame prediction and the inter-frame prediction of the depth CU. 35. The video decoder of claim 34, wherein the syntax structure includes a single syntax structure for the intra-frame prediction and the inter-frame prediction, the single syntax structure including a second syntax element indicating information representing the information of at least one DC residual value of the intra-frame prediction and the inter-frame prediction of the depth CU. 36. The video decoder of claim 34, wherein the syntax structure includes different instances of the syntax structure for intra-frame prediction and inter-frame prediction, each of the different instances of the syntax structure containing the same second syntax element. 37. The video decoder of claim 36, wherein one or more processors are configured to perform entropy decoding on the different instances of the grammatical structure using a context-adaptive entropy decoding process, and wherein the context model used for the context-adaptive entropy decoding process is the same for the different instances of the grammatical structure. 38. The video decoder of claim 30, wherein the one or more processors are configured to decode a flag indicating the presence of any non-zero DC residual value for any partition of the depth CU when the SDC mode is used, and not to decode the information representing the at least one DC residual value of the depth CU when the flag indicates that no non-zero DC residual value exists for any partition of the depth CU. 39. The video decoder of claim 30, wherein the video decoder includes a video decoder, and wherein the one or more processors are configured to disable the use of pulse code modulation (PCM) decoding of the depth CU by the video decoder when the syntax element indicates the use of the SDC mode. 40. The video decoder of claim 30, wherein the video decoder includes a video decoder, and wherein the one or more processors are configured to restrict the use of the SDC mode to a depth CU having a partition size of 2Nx2N. 41. The video decoder of claim 30, wherein the video decoder includes a video decoder, wherein the one or more processors are configured to restrict the use of the SDC mode to inter-frame prediction depth CUs having a 2Nx2N partition size and intra-frame prediction depth CUs having a 2Nx2N or NxN partition size. 42. The video decoder of claim 30, wherein the video decoder includes a video decoder, and wherein the one or more processors are configured to use the SDC mode for reconstruction of each of a plurality of depth CUs in the entire decoded video sequence when the syntax element indicates the use of the SDC mode. 43. The video decoder of claim 30, wherein the one or more processors are configured to decode the syntax elements in one of a video parameter set (VPS), a sequence parameter set (SPS), or a picture parameter set (PPS). 44. The video decoder of claim 30, wherein the one or more processors are configured to decode the syntax elements in a slice header associated with the depth CU. 45. The video decoder of claim 30, wherein the video decoder includes a video decoder, wherein the depth CU is a current depth CU, and the syntax element indicates whether the SDC mode is used for the current depth CU, and the one or more processors are configured to use the SDC mode for reconstruction of the current depth CU when the syntax element indicates that the SDC mode is used. 46. A video decoder, comprising: A means for decoding the syntax elements of both intra-frame prediction and inter-frame prediction of a depth decoding unit (CU) that indicates whether the simplified depth decoding (SDC) mode is used for video data. A means for performing intra-frame prediction to generate a predicted depth CU when the depth CU is intra-predicted. A means for performing inter-frame prediction to generate the predicted depth CU when the depth CU is predicted by inter-frame prediction; and A means for decoding information representing at least one DC residual value of the depth CU for each partition of the prediction unit PU of the depth CU when using the SDC mode, wherein the at least one DC residual value represents the pixel difference between the partition of the PU of the depth CU and the corresponding partition of the predicted depth CU. 47. The video decoder of claim 46, wherein the video decoder includes a video decoder, and the means for decoding the syntax element includes means for decoding the syntax element, the decoder further including means for reconstructing the depth CU in the video decoder using the at least one DC residual value and the predicted depth CU. 48. A computer-readable storage medium comprising a program stored thereon, said program, when executed, causing one or more processors of a video decoder to perform the following operations: The syntax elements of both intra-frame prediction and inter-frame prediction of the deep decoding unit (CU) indicating whether the simplified deep decoding (SDC) mode is used for the video data are decoded. When the depth CU is predicted intra-frame, intra-frame prediction is performed to generate the predicted depth CU; When the depth CU is predicted via inter-frame prediction, inter-frame prediction is performed to generate the predicted depth CU; and When the SDC mode is used, for each partition of the prediction unit PU of the depth CU, information representing at least one DC residual value of the depth CU is decoded, wherein the at least one DC residual value represents the pixel difference between the partition of the PU of the depth CU and the corresponding partition of the predicted depth CU. 49. The computer-readable storage medium of claim 48, wherein the video decoder includes a video decoder, and the program causing the one or more processors to decode the syntax element includes a program causing the one or more processors to decode the syntax element, the computer-readable storage medium further including a program causing the one or more processors to reconstruct the depth CU using the at least one DC residual value and the predicted depth CU. 50. A method for decoding video data, the method comprising: When the depth decoding unit (CU) of the video data is intra-predicted, intra-prediction is performed to generate a predicted depth CU. When the depth CU is predicted by inter-frame prediction, inter-frame prediction is performed to generate the predicted depth CU; When using the Simplified Depth Decoding (SDC) mode, a syntax structure is obtained comprising one or more syntax elements including information indicating at least one DC residual value representing a partition of the prediction unit (PU) of the depth CU, wherein the syntax elements of the syntax structure are identical for both intra-frame prediction and inter-frame prediction; and The depth CU is reconstructed using the at least one DC residual value and the predicted depth CU. 51. The method of claim 50, wherein obtaining the syntax structure comprises obtaining a single syntax structure for the intra-frame prediction and the inter-frame prediction, wherein the single syntax structure comprises the syntax element indicating information representing at least one DC residual value of one of the intra-frame prediction and the inter-frame prediction of the depth CU. 52. The method of claim 50, wherein obtaining the syntax structure comprises obtaining different instances of the same syntax structure for the intra-frame prediction and the inter-frame prediction, each of the different instances of the same syntax structure containing the same syntax elements. 53. The method of claim 52, wherein obtaining the different instances of the same syntactic structure comprises entropy decoding the different instances of the syntactic structure using a context-adaptive entropy decoding process, and wherein the context model used for the context-adaptive entropy decoding process is the same for the different instances of the syntactic structure. 54. The method of claim 50, wherein the information indicated by the syntax element represents the absolute value of the DC residual value of the depth CU and the sign of the DC residual value. 55. A method for encoding video data, the method comprising: When the depth decoding unit (CU) of the video data is intra-predicted, intra-prediction is performed to generate a predicted depth CU. When the depth CU is predicted by inter-frame prediction, inter-frame prediction is performed to generate the predicted depth CU; When using the Simplified Depth Decoding (SDC) mode, a syntax structure comprising one or more syntax elements including information indicating at least one DC residual value representing a partition of the prediction unit (PU) of the depth CU is decoded, wherein the syntax elements of the syntax structure are identical for both intra-frame and inter-frame prediction; and The depth CU is encoded based on the information representing the at least one DC residual value. 56. The method of claim 55, wherein encoding the syntax structure comprises encoding a single syntax structure for the intra-frame prediction and the inter-frame prediction, wherein the single syntax structure includes the syntax element indicating information representing at least one DC residual value of one of the intra-frame prediction and the inter-frame prediction of the depth CU. 57. The method of claim 55, wherein encoding the syntax structure comprises encoding different instances of the same syntax structure for the intra-frame prediction and the inter-frame prediction, each of the different instances of the same syntax structure containing the same syntax elements. 58. The method of claim 57, wherein encoding the different instances of the same syntactic structure comprises entropy encoding the different instances of the syntactic structure using a context-adaptive entropy encoding process, and wherein the context model used for the context-adaptive entropy encoding process is the same for the different instances of the syntactic structure. 59. The method of claim 55, wherein the information indicated by the syntax element represents the absolute value of the DC residual value of the depth CU and the sign of the DC residual value. 60. A video decoder, comprising: The memory, which stores video data; and One or more processors, configured to: When the depth decoding unit (CU) of the video data is intra-predicted, intra-prediction is performed to generate the predicted depth CU of the video data. When the depth CU is predicted via inter-frame prediction, inter-frame prediction is performed to generate the predicted depth CU of the video data; and When using the Simplified Depth Decoding (SDC) mode, a syntax structure comprising one or more syntax elements including information indicating at least one DC residual value representing a partition of the prediction unit (PU) of the depth CU is decoded, wherein the syntax elements of the syntax structure are the same for the intra-frame prediction mode and the inter-frame prediction mode. 61. The video decoder of claim 60, wherein the video decoder includes a video decoder, and the one or more processors are configured to decode the syntax elements in the syntax structure to obtain the information representing the at least one DC residual value, and to reconstruct the depth CU using the at least one DC residual value and the predicted depth CU. 62. The video decoder of claim 60, wherein the video decoder includes a video encoder, and the one or more processors are configured to encode the syntax elements in the syntax structure to indicate the information representing the at least one DC residual value and the syntax elements. 63. The video decoder of claim 60, wherein the syntax structure includes a single syntax structure for the intra-frame prediction and the inter-frame prediction, wherein the single syntax structure includes the syntax element indicating information representing at least one DC residual value of the intra-frame prediction and the inter-frame prediction of the depth CU. 64. The video decoder of claim 60, wherein the syntax structure includes different instances of the same syntax structure for intra-frame prediction and inter-frame prediction, each of the different instances of the same syntax structure containing the same syntax elements. 65. The video decoder of claim 64, wherein the one or more processors are configured to perform entropy decoding on the different instances of the grammatical structure using a context-adaptive entropy decoding process, and wherein the context model used for the context-adaptive entropy decoding is the same for the different instances of the grammatical structure. 66. The video decoder of claim 60, wherein the information indicated by the syntax element represents the absolute value of the DC residual value of the depth CU and the sign of the DC residual value. 67. A video decoder, comprising: A means for performing intra-frame prediction to generate a predicted depth CU when the depth decoding unit (CU) of video data is intra-frame predicted. A means for performing inter-frame prediction to generate the predicted depth CU when the depth CU is predicted by inter-frame prediction; and A means for decoding a syntax structure comprising one or more syntax elements including information indicating at least one DC residual value representing a partition of a prediction unit PU of the depth CU when using a simplified depth decoding (SDC) mode, wherein the syntax elements of the syntax structure are the same for the intra-frame prediction mode and the inter-frame prediction mode. 68. The video decoder of claim 67, wherein the video decoder is a video decoder, the video decoder including means for reconstructing the depth CU using the at least one DC residual value and the predicted depth CU. 69. A computer-readable storage medium comprising a program stored thereon, said program, when executed, causing one or more processors of a video decoder to perform the following operations: When the depth decoding unit (CU) of the video data is intra-predicted, intra-prediction is performed to generate the predicted depth CU; When the depth CU is predicted via inter-frame prediction, inter-frame prediction is performed to generate the predicted depth CU; and A means for decoding a syntax structure comprising one or more syntax elements including information indicating at least one DC residual value representing a partition of a prediction unit PU of the depth CU when using a simplified depth decoding (SDC) mode, wherein the syntax elements of the syntax structure are the same for the intra-frame prediction mode and the inter-frame prediction mode. 70. The computer-readable storage medium of claim 69, wherein the video decoder is a video decoder, the video decoder including means for reconstructing the depth CU using the at least one DC residual value and the predicted depth CU. 71. A method for decoding video data, the method comprising: Receive a first syntax element, which indicates whether the Simplified Deep Decoding (SDC) mode is enabled for the entire sequence of Deep Decoding Units (CUs) of the decoded video data. Receive a second syntax element, the second syntax element indicating whether the SDC mode is used for both intra-frame prediction and inter-frame prediction of one of the depth CUs of the sequence; When one of the depths CU in the sequence is intra-predicted, intra-prediction is performed to produce a predicted depth CU; When one of the depths CU in the sequence is predicted by inter-frame prediction, inter-frame prediction is performed to generate the predicted depth CU; When the second syntax element indicates the use of the SDC mode, information is obtained representing at least one DC residual value of a partition of the prediction unit PU of one of the depth CUs for the sequence; and The partition of the PU of one of the depth CUs in the sequence is reconstructed using the at least one DC residual value and the corresponding predicted depth CU. 72. The method of claim 71, wherein receiving the first syntax element comprises receiving the first syntax element in one of a video parameter set (VPS), a sequence parameter set (SPS), or a picture parameter set (PPS). 73. The method of claim 71, wherein the DC residual value indicates the difference between the average depth value of pixels in the partition of the PU of one of the CUs of the sequence and the average depth value of pixels in the predicted depth partition. 74. The method of claim 71, wherein the first syntax element includes a one-bit flag having a first value and a second value, the first value indicating that the SDC mode is enabled and the second value indicating that the SDC mode is not enabled. 75. A method for encoding video data, the method comprising: Whether the Simplified Deep Decoding (SDC) mode encodes the enabled syntax elements of the Deep Decoding Units (CUs) for the entire sequence of decoded video data; The second syntax element, which indicates whether the SDC mode is used for intra-frame prediction and inter-frame prediction of one of the depth CUs in the sequence, is encoded. When one of the depths CU in the sequence is intra-predicted, intra-prediction is performed to produce a predicted depth CU; When one of the depths (CUs) of the sequence is predicted via inter-frame prediction, inter-frame prediction is performed to generate the predicted depth (CU); and When the second syntax element indicates the use of the SDC mode, information on at least one DC residual value of a partition representing the prediction unit PU of one of the depths CUs for the sequence is encoded. 76. The method of claim 75, wherein encoding the first syntax element comprises encoding the syntax element in one of a video parameter set (VPS), a sequence parameter set (SPS), or a picture parameter set (PPS). 77. The method of claim 75, wherein the DC residual value indicates the difference between the average depth value of pixels in the partition of the PU of one of the CUs of the sequence and the average depth value of pixels in the predicted depth partition. 78. The method of claim 75, wherein the first syntax element includes a one-bit flag having a first value and a second value, the first value indicating that the SDC mode is enabled and the second value indicating that the SDC mode is not enabled. 79. A video decoder, comprising: The memory, which stores video data; and One or more processors, configured to: The syntax elements of the Deep Decoding Unit (CU) that indicate whether the Simplified Deep Decoding (SDC) mode is used to decode the entire code sequence of the video data are decoded. Decode the second syntax element that indicates whether the SDC mode is used for intra-frame prediction and inter-frame prediction of one of the depth CUs in the sequence; When one of the depths CU in the sequence is intra-predicted, intra-prediction is performed to produce a predicted depth CU; When one of the depths (CUs) of the sequence is predicted via inter-frame prediction, inter-frame prediction is performed to generate the predicted depth (CU); and When the second syntax element indicates the use of the SDC mode, information representing at least one DC residual value of a partition of the prediction unit PU of one of the depths CUs for the sequence is decoded. 80. The video decoder of claim 79, wherein the video decoder is a video decoder, and the one or more processors are configured to reconstruct the partition of the PU of one of the depth CUs of the sequence using the at least one DC residual value and the corresponding predicted depth CU. 81. The video decoder of claim 79, wherein the one or more processors are configured to decode the syntax elements in one of a video parameter set (VPS), a sequence parameter set (SPS), or a picture parameter set (PPS). 82. The video decoder of claim 79, wherein the DC residual value indicates the difference between the average depth value of pixels in the partition of the prediction unit of the CU and the average depth value of pixels in the prediction depth partition. 83. The video decoder of claim 79, wherein the first syntax element includes a one-bit flag, and the partition of the PU of one of the depth CUs of the sequence is reconstructed in the video decoder using the at least one DC residual value and the corresponding predicted depth CU. 84. A video decoder, comprising: A means for decoding the syntax elements of a depth decoding unit CU that is enabled, which indicates whether the simplified depth decoding (SDC) mode is for the entire sequence of decoded video data. A means for decoding a second syntax element that indicates whether the SDC mode is used for one of the depth CUs in the sequence for both intra-frame prediction and inter-frame prediction. Means for performing intra-frame prediction in a video encoder to generate a predicted depth CU when one of the depth CUs of the sequence is intra-frame predicted. Means for performing inter-frame prediction in the video encoder to generate the predicted depth CU when one of the depth CUs of the sequence is inter-frame predicted. A means for decoding information representing at least one DC residual value of a partition of a prediction unit PU for one of the depths CUs in the sequence when the second syntax element indicates the use of the SDC mode. 85. The video decoder of claim 84, further comprising means for decoding the first syntax element in one of a video parameter set (VPS), a sequence parameter set (SPS), or a picture parameter set (PPS). 86. A computer-readable storage medium comprising a program stored thereon, said program, upon execution, immediately causing one or more processors of a video decoder to perform the following operations: The indicator specifies whether the Simplified Deep Decoding (SDC) mode decodes the syntax elements of the Deep Decoding Units (CUs) for the entire sequence of decoded video data that have been enabled. Decode the second syntax element that indicates whether the SDC mode is used for intra-frame prediction and inter-frame prediction of one of the depth CUs in the sequence; When one of the depths CU in the sequence is intra-predicted, intra-prediction is performed to produce a predicted depth CU; When one of the depths CU in the sequence is predicted by inter-frame prediction, inter-frame prediction is performed to generate the predicted depth CU; When the second syntax element indicates the use of the SDC mode, information representing at least one DC residual value of a partition of the prediction unit PU of one of the depths CUs for the sequence is decoded. 87. The computer-readable storage medium of claim 86, further comprising a program that causes the one or more processors to decode the first syntax element in one of a video parameter set (VPS), a sequence parameter set (SPS), or a picture parameter set (PPS).