HK1216571B - Methods, devices, and computer-readable storage mediums for video encoding and decoding - Google Patents

Description

Method, device and computer-readable storage medium for video encoding and decoding

This application claims the benefit of U.S. Provisional Application No. 61/872,540, filed August 30, 2013, and U.S. Provisional Application No. 61/913,031, filed December 6, 2013, each of which is incorporated herein by reference in its entirety.

Technical Field

The present invention relates to video encoding and decoding.

Background Art

Digital video capabilities can be incorporated into a wide variety of devices, including digital televisions, digital live broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite radio telephones (so-called "smartphones"), video teleconferencing devices, video streaming devices, and the like. Digital video devices implement video compression techniques, such as those described in the standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264/MPEG-4 Part 10 Advanced Video Coding (AVC), the High Efficiency Video Coding (HEVC) standard currently under development, and extensions of such standards. By implementing such video compression techniques, video devices can more efficiently transmit, receive, encode, decode, and/or store digital video information.

Video compression techniques perform spatial (intra-picture) prediction and/or temporal (inter-picture) prediction to reduce or remove redundancy inherent in video sequences. For block-based video coding, a video slice (i.e., a video frame or portion of a video frame) may be partitioned into video blocks. Video blocks in an intra-coded (I) slice of a picture are encoded using spatial prediction with respect to reference samples in neighboring blocks in the same picture. Video blocks in an inter-coded (P or B) slice of a picture may use spatial prediction with respect to reference samples in neighboring blocks in the same picture or temporal prediction with respect to reference samples in other reference pictures. Pictures may be referred to as frames, and reference pictures may be referred to as reference frames.

Spatial or temporal prediction results in a predictive block for the block to be coded. Residual data represents the pixel differences between the original block to be coded and the predictive block. An inter-coded block is encoded based on a motion vector pointing to a block of reference samples forming the predictive block and residual data indicating the difference between the coded block and the predictive block. An intra-coded block is encoded based on an intra-coding mode and the residual data. For further compression, the residual data can be transformed from the pixel domain to the transform domain, generating residual coefficients that can then be quantized. The quantized coefficients, initially arranged in a two-dimensional array, can be scanned to generate a one-dimensional vector of coefficients, and entropy coding can be applied to achieve even more compression.

A multi-view coding bitstream can be generated by, for example, encoding views from multiple viewpoints. Some three-dimensional (3D) video standards have been developed that utilize aspects of multi-view coding. For example, different views may transmit left-eye and right-eye views to support 3D video. Alternatively, some 3D video coding processes may apply so-called multi-view plus depth coding. In multi-view plus depth coding, a 3D video bitstream may contain not only texture view components but also depth view components. For example, each view may include one texture view component and one depth view component.

Summary of the Invention

In general, the present disclosure relates to three-dimensional (3D) video coding based on advanced codecs, including depth coding techniques. For example, some of the techniques of this disclosure relate to advanced motion prediction for 3-Dimensional High Efficiency Video Coding (3D-HEVC). In some examples, a video coder determines a candidate for inclusion in a candidate list for a current prediction unit (PU). The candidate is based on motion parameters of a plurality of sub-PUs of the current PU. When generating the candidates, the video coder may process the sub-PUs in a particular order (e.g., raster scan order). If a reference block corresponding to the sub-PU is not coded using motion compensated prediction, the video coder sets the motion parameters of the sub-PU to default motion parameters. For each respective sub-PU from the plurality of sub-PUs, if the reference block of the respective sub-PU is not coded using motion compensated prediction, then in response to a subsequent determination to code a reference block of any later sub-PU in the order using motion compensated prediction, the motion parameters of the respective sub-PU are not set.

In one example, the present disclosure describes a method for decoding multi-view video data, the method comprising: partitioning a current prediction unit (PU) into a plurality of sub-PUs, wherein the current PU is in a current picture; determining default motion parameters; processing the sub-PUs from the plurality of sub-PUs in a particular order, wherein, for each respective sub-PU from the plurality of sub-PUs, if a reference block of the respective sub-PU is not coded using motion compensated prediction, then, in response to a subsequent determination that a reference block of any later sub-PU in the order is coded using motion compensated prediction, the motion parameters of the respective sub-PU are not set, wherein the reference block of at least one of the sub-PUs is not coded using motion compensated prediction, and wherein processing the sub-PUs comprises, for each respective sub-PU from the plurality of sub-PUs, : determining a reference block for the corresponding sub-PU, wherein a reference picture includes the reference block for the corresponding sub-PU; if the reference block of the corresponding sub-PU is decoded using motion compensated prediction, setting the motion parameters of the corresponding sub-PU based on the motion parameters of the reference block of the corresponding sub-PU; and if the reference block of the corresponding sub-PU is not decoded using motion compensated prediction, setting the motion parameters of the corresponding sub-PU to the default motion parameters; and including a candidate in a candidate list for the current PU, wherein the candidate is based on the motion parameters of the multiple sub-PUs; obtaining a syntax element indicating a selected candidate in the candidate list from a bitstream; and using the motion parameters of the selected candidate to reconstruct a predictive block of the current PU.

In another example, the present disclosure describes a method of encoding video data, the method comprising: dividing a current prediction unit (PU) into a plurality of sub-PUs, wherein the current PU is in a current picture; determining default motion parameters; processing the sub-PUs from the plurality of sub-PUs in a particular order, wherein, for each respective sub-PU from the plurality of sub-PUs, if a reference block of the respective sub-PU is not coded using motion compensated prediction, then, in response to a subsequent determination that a reference block of any later sub-PU in the order is coded using motion compensated prediction, the motion parameters of the respective sub-PU are not set, wherein the reference block of at least one of the sub-PUs is not coded using motion compensated prediction, and wherein processing the sub-PUs comprises, for each respective sub-PU from the plurality of sub-PUs, For each corresponding sub-PU of a sub-PU: determining a reference block for the corresponding sub-PU, wherein a reference picture includes the reference block of the corresponding sub-PU; if the reference block of the corresponding sub-PU is decoded using motion compensated prediction, setting the motion parameters of the corresponding sub-PU based on the motion parameters of the reference block of the corresponding sub-PU; and if the reference block of the corresponding sub-PU is not decoded using motion compensated prediction, setting the motion parameters of the corresponding sub-PU to the default motion parameters; and including a candidate in a candidate list for the current PU, wherein the candidate is based on the motion parameters of the multiple sub-PUs; and signaling a syntax element indicating a selected candidate in the candidate list in a bitstream.

In another example, the disclosure describes a device for decoding video data, the device comprising: a memory for storing a decoded picture; and one or more processors configured to: divide a current prediction unit (PU) into a plurality of sub-PUs, wherein the current PU is in a current picture; determine default motion parameters; process the sub-PUs from the plurality of sub-PUs in a particular order, wherein, for each respective sub-PU from the plurality of sub-PUs, if a reference block of the respective sub-PU is not coded using motion compensated prediction, then, in response to a subsequent determination to use motion compensated prediction to decode a reference block of any later sub-PU in the order, the motion parameters of the respective sub-PU are not set, wherein the motion compensated prediction is not used to decode the reference block of the respective sub-PU. The method comprises: determining a reference block of at least one of the plurality of sub-PUs, wherein processing the sub-PU comprises, for each respective sub-PU from the plurality of sub-PUs: determining a reference block for the respective sub-PU, wherein a reference picture includes the reference block of the respective sub-PU; setting motion parameters of the respective sub-PU based on motion parameters of the reference block of the respective sub-PU if motion-compensated prediction is used to decode the reference block of the respective sub-PU; and setting the motion parameters of the respective sub-PU to the default motion parameters if motion-compensated prediction is not used to decode the reference block of the respective sub-PU; and including a candidate in a candidate list for the current PU, wherein the candidate is based on the motion parameters of the plurality of sub-PUs.

In another example, the disclosure describes a device for coding video data, the device comprising: means for partitioning a current prediction unit (PU) into a plurality of sub-PUs, wherein the current PU is in a current picture; means for determining default motion parameters; means for processing sub-PUs from the plurality of sub-PUs in a particular order, wherein, for each respective sub-PU from the plurality of sub-PUs, if a reference block of the respective sub-PU is not coded using motion compensated prediction, then, in response to a subsequent determination that a reference block of any later sub-PU in the order is coded using motion compensated prediction, the motion parameter of the respective sub-PU is not set, wherein the reference block of at least one of the sub-PUs is not coded using motion compensated prediction, and wherein the means for processing The device of the sub-PU includes, for each corresponding sub-PU from the multiple sub-PUs: a device for determining a reference block of the corresponding sub-PU, wherein a reference picture includes the reference block of the corresponding sub-PU; a device for setting the motion parameters of the corresponding sub-PU based on the motion parameters of the reference block of the corresponding sub-PU if the reference block of the corresponding sub-PU is decoded using motion compensated prediction; and a device for setting the motion parameters of the corresponding sub-PU to the default motion parameters if the reference block of the corresponding sub-PU is not decoded using motion compensated prediction; and a device for including a candidate in a candidate list of the current PU, wherein the candidate is based on the motion parameters of the multiple sub-PUs.

In another example, the present disclosure describes a non-transitory computer-readable data storage medium having instructions stored thereon that, when executed, cause a device to: divide a current prediction unit (PU) into a plurality of sub-PUs, wherein the current PU is in a current picture; determine default motion parameters; and process the sub-PUs from the plurality of sub-PUs in a particular order, wherein, for each respective sub-PU from the plurality of sub-PUs, if a reference block of the respective sub-PU is not coded using motion compensated prediction, then, in response to a subsequent determination that a reference block of any later sub-PU in the order is coded using motion compensated prediction, the motion parameters of the respective sub-PU are not set. the reference block of at least one of the plurality of sub-PUs, and wherein processing the sub-PU comprises, for each respective sub-PU from the plurality of sub-PUs: determining a reference block for the respective sub-PU, wherein a reference picture includes the reference block of the respective sub-PU; setting motion parameters of the respective sub-PU based on motion parameters of the reference block of the respective sub-PU if motion compensated prediction is used to decode the reference block of the respective sub-PU; and setting the motion parameters of the respective sub-PU to the default motion parameters if motion compensated prediction is not used to decode the reference block of the respective sub-PU; and including a candidate in a candidate list of the current PU, wherein the candidate is based on the motion parameters of the plurality of sub-PUs.

The details of one or more embodiments of the present invention are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description, drawings, and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

1 is a block diagram illustrating an example video coding system that may utilize the techniques described in this disclosure.

2 is a conceptual diagram illustrating example intra-prediction modes in High Efficiency Video Coding (HEVC).

3 is a conceptual diagram illustrating example spatially neighboring blocks relative to a current block.

4 is a conceptual diagram illustrating an example multi-view decoding order.

5 is a conceptual diagram illustrating an example prediction structure for multi-view coding.

6 is a conceptual diagram illustrating example temporal neighboring blocks in neighboring block-based disparity vector (NBDV) derivation.

7 is a conceptual diagram illustrating depth block derivation from a reference view to perform backward view synthesis prediction (BVSP).

8 is a conceptual diagram illustrating an example derivation of inter-view predicted motion vector candidates for merge/skip mode.

FIG. 9 is a table indicating example designations of l0CandIdx and l1CandIdx in 3D-HEVC.

10 is a conceptual diagram illustrating an example derivation of motion vector inheritance candidates for depth coding.

11 illustrates an example prediction structure for advanced residual prediction (ARP) in multi-view video coding.

12 is a conceptual diagram illustrating an example relationship among a current block, a reference block, and a motion compensated block.

13 is a conceptual diagram illustrating inter-view motion prediction for sub-prediction units (PUs).

14 is a block diagram illustrating an example video encoder that may implement the techniques described in this disclosure.

15 is a block diagram illustrating an example video decoder that may implement the techniques described in this disclosure.

16A is a flowchart illustrating example operation of a video encoder that encodes a coding unit (CU) using inter prediction, according to an example of this disclosure.

16B is a flowchart illustrating example operation of a video decoder that uses inter prediction to decode a CU, according to an example of this disclosure.

17 is a flowchart illustrating example operation of a video coder constructing a merge candidate list for a current PU in a current view component, according to examples of this disclosure.

FIG. 18 is a flow chart illustrating a continuation of the reference picture list construction operation of FIG. 17 according to an example of the present disclosure.

19 is a flowchart illustrating the operation of a video coder that determines an inter-view prediction motion vector candidate or a texture merge candidate, according to an example of this disclosure.

DETAILED DESCRIPTION

High-Efficiency Video Coding (HEVC) is a newly developed video coding standard. 3D-HEVC is an extension of HEVC for 3D video data. 3D-HEVC provides multiple views of the same scene from different viewpoints. Part of the 3D-HEVC standardization effort includes the standardization of a multi-view video codec based on HEVC. In 3D-HEVC, inter-view prediction is implemented based on reconstructed view components from different views.

In 3D-HEVC, inter-view motion prediction is similar to motion compensation used in standard HEVC and may utilize the same or similar syntax elements. Merge mode, skip mode, and advanced motion vector prediction (AMVP) mode are example types of motion prediction. When a video coder performs inter-view motion prediction on a prediction unit (PU), the video coder may use pictures in the same access unit as the PU but in different views as sources of motion information. In contrast, other motion compensation methods may only use pictures in different access units as reference pictures. Therefore, in 3D-HEVC, motion parameters for blocks in dependent views may be predicted or inferred based on already coded motion parameters in other views of the same access unit.

When a video coder performs motion prediction, the video coder may generate a candidate list (e.g., a merge candidate list or an AMVP candidate list) when signaling the motion information of the current PU using merge mode, skip mode, or AMVP mode. To implement inter-view motion prediction in 3D-HEVC, the video coder may include inter-view predicted motion vector candidates (IPMVC) in the merge candidate list and the AMVP candidate list. The video coder may use IPMVC in the same manner as other candidates in the candidate list. IPMVC may specify motion information of a PU (i.e., a reference PU) in an inter-view reference picture. The inter-view reference picture may be in the same access unit as the current PU but in a different view from the current PU.

In some examples, IPMVC may specify motion parameters (e.g., motion vectors, reference indices, etc.) for multiple sub-PUs of the current PU. In general, each sub-PU of a PU may be associated with a different, equally-sized sub-block of the PU's prediction block. For example, if the PU's luma prediction block is 32x32 and the sub-PU size is 4x4, the video coder may partition the PU into 64 sub-PUs associated with different 4x4 sub-blocks of the PU's luma prediction block. In this example, the sub-PUs may also be associated with corresponding sub-blocks of the PU's chroma prediction block. Thus, IPMVC may specify multiple motion parameter sets. In such examples, if IPMVC is the selected candidate in the candidate list, the video coder may determine the predictive block for the current PU based on the multiple motion parameter sets specified by the IPMVC.

To determine the IPMVC that specifies the motion parameters of a sub-PU of the current PU, the video coder may process each of the sub-PUs according to raster scan order. When the video coder processes a sub-PU (i.e., the current sub-PU), the video coder may determine a reference block corresponding to the sub-PU based on the disparity vector of the current PU. The reference block may be in the same time instance as the current picture but in a different view from the current picture. If the reference block corresponding to the current sub-PU is coded using motion compensated prediction (e.g., the reference block has one or more motion vectors, reference indices, etc.), the video coder may set the motion parameters of the current sub-PU to the motion parameters of the reference block corresponding to the sub-PU. Otherwise, if the reference block corresponding to the current sub-PU is not coded using motion compensated prediction (e.g., the reference block is coded using intra prediction), the video coder may identify the nearest sub-PU in raster scan order whose corresponding reference block is coded using motion compensated prediction. The video coder may then set the motion parameters of the current sub-PU to the motion parameters of the reference block corresponding to the identified sub-PU.

In some cases, the identified sub-PU appears later in the raster scan order of sub-PUs than the current sub-PU. Therefore, when determining the motion parameters of the current sub-PU, the video coder may scan forward to find a sub-PU whose corresponding reference block is coded using motion compensated prediction. Alternatively, the video coder may delay determining the motion parameters of the current sub-PU until the video coder encounters a PU whose corresponding reference block is coded using motion compensated prediction during the processing of the sub-PU. In either of these cases, additional complexity and coding delay are added.

According to one or more techniques of the present invention, a video decoder may partition a current PU into multiple sub-PUs. In addition, the video decoder may determine default motion parameters. In addition, the video decoder may process the sub-PUs from the multiple sub-PUs in a specific order. In some cases, the video decoder may determine the default motion parameters before processing any of the sub-PUs. For each corresponding PU of the current PU, the video decoder may determine a reference block for the corresponding sub-PU. If the reference block of the corresponding sub-PU is decoded using motion compensated prediction, the video decoder may set the motion parameters of the corresponding sub-PU based on the motion parameters of the reference block of the corresponding sub-PU. However, if the reference block of the corresponding sub-PU is not decoded using motion compensated prediction, the video decoder may set the motion parameters of the corresponding sub-PU to the default motion parameters.

According to one or more techniques of this disclosure, if the reference block of a corresponding sub-PU is not coded using motion compensated prediction, then in response to a subsequent determination that the reference block of any later sub-PU in a particular order is coded using motion compensated prediction, the motion parameters of the corresponding sub-PU are not set. Therefore, when the video coder processes a sub-PU, the video coder may not need to scan forward to find a sub-PU whose corresponding reference block is coded using motion compensated prediction or delay determining the motion parameters of the corresponding sub-PU until the video coder encounters a PU whose corresponding reference block is coded using motion compensated prediction during the processing of the sub-PU. Advantageously, this can reduce complexity and decoding latency.

FIG1 is a block diagram illustrating an example video coding system 10 that may utilize the techniques of this disclosure. As used herein, the term "video coder" generally refers to both video encoders and video decoders. In this disclosure, the terms "video coding" or "coding" may generally refer to video encoding or video decoding.

1 , video coding system 10 includes a source device 12 and a destination device 14. Source device 12 generates encoded video data. Thus, source device 12 may be referred to as a video encoding device or a video encoding apparatus. Destination device 14 may decode the encoded video data generated by source device 12. Thus, destination device 14 may be referred to as a video decoding device or a video decoding apparatus. Source device 12 and destination device 14 may be examples of video coding devices or video coding apparatuses.

Source device 12 and destination device 14 may comprise a wide range of devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called “smart” phones, televisions, cameras, display devices, digital media players, video game consoles, in-car computers, or the like.

Destination device 14 may receive encoded video data from source device 12 via channel 16. Channel 16 may include one or more media or devices capable of moving the encoded video data from source device 12 to destination device 14. In one example, channel 16 may include one or more communication media that enable source device 12 to transmit the encoded video data directly to destination device 14 in real time. In this example, source device 12 may modulate the encoded video data according to a communication standard (e.g., a wireless communication protocol) and may transmit the modulated video data to destination device 14. The one or more communication media may include wireless and/or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet network, such as a local area network, a wide area network, or a global network (e.g., the Internet). The one or more communication media may include routers, switches, base stations, or other equipment that facilitates communication from source device 12 to destination device 14.

In another example, channel 16 may include a storage medium that stores the encoded video data generated by source device 12. In this example, destination device 14 may access the storage medium, for example, via disk access or card access. The storage medium may include a variety of locally accessible data storage media such as Blu-ray discs, DVDs, CD-ROMs, flash memory, or other suitable digital storage media for storing encoded video data.

In another example, channel 16 may include a file server or another intermediate storage device that stores the encoded video data generated by source device 12. In this example, destination device 14 may access the encoded video data stored at the file server or other intermediate storage device via streaming or downloading. The file server may be a type of server capable of storing and transmitting the encoded video data to destination device 14. Example file servers include a network server (e.g., for a website), a File Transfer Protocol (FTP) server, a Network Attached Storage (NAS) device, and a local disk drive.

Destination device 14 may access the encoded video data through a standard data connection, such as an Internet connection. Example types of data connections may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both suitable for accessing encoded video data stored on a file server. Transmission of the encoded video data from the file server may be a streaming transmission, a download transmission, or a combination of both.

The techniques of this disclosure are not limited to wireless applications or settings. The techniques may be applied to video coding to support a variety of multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, streaming video transmission (e.g., via the Internet), encoding video data for storage on a data storage medium, decoding video data stored on a data storage medium, or other applications. In some examples, video coding system 10 may be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting, and/or video telephony.

FIG1 is merely an example, and the techniques of this disclosure may be applicable to video coding settings (e.g., video encoding or video decoding) that do not necessarily include any data communication between the encoding and decoding devices. In other examples, data may be retrieved from local memory, streamed over a network, or the like. A video encoding device may encode and store data to memory, and/or a video decoding device may retrieve and decode data from memory. In many examples, encoding and decoding are performed by devices that do not communicate with each other but merely encode and/or retrieve data from memory and decode data.

1 , source device 12 includes a video source 18, a video encoder 20, and an output interface 22. In some examples, output interface 22 may include a modulator/demodulator (modem) and/or a transmitter. Video source 18 may include a video capture device (e.g., a camera), a video archive containing previously captured video data, a video feed interface for receiving video data from a video content provider, and/or a computer graphics system for generating video data, or a combination of such sources of video data.

Video encoder 20 may encode video data from video source 18. In some examples, source device 12 transmits the encoded video data directly to destination device 14 via output interface 22. In other examples, the encoded video data may also be stored on a storage medium or a file server for later access by destination device 14 for decoding and/or playback.

In the example of FIG1 , destination device 14 includes an input interface 28, a video decoder 30, and a display device 32. In some examples, input interface 28 includes a receiver and/or a modem. Input interface 28 may receive encoded video data via channel 16. Video decoder 30 may decode the encoded video data. Display device 32 may display the decoded video data. Display device 32 may be integrated with destination device 14 or may be external to the destination device. Display device 32 may include a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

Video encoder 20 and video decoder 30 may each be implemented as any of a variety of suitable circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. When the techniques are implemented partially in software, the device may store the instructions for the software in a suitable non-transitory computer-readable storage medium and may execute the instructions in hardware using one or more processors to perform the techniques of this disclosure. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) may be considered to be one or more processors. Each of video encoder 20 and video decoder 30 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder/decoder (CODEC) in the respective device.

This disclosure may generally refer to video encoder 20 "signaling" certain information to another device (e.g., video decoder 30). The term "signaling" may generally refer to the communication of syntax elements and/or other data used to decode compressed video data. This communication may occur in real time or near real time. Alternatively, this communication may occur over a time span, such as when syntax elements are stored in an encoded bitstream to a computer-readable storage medium at encoding time, which may then be retrieved by a decoding device at any time after storage to such medium.

In some examples, video encoder 20 and video decoder 30 operate according to a video compression standard, such as ISO/IEC MPEG-4 Visual and ITU-T H.264 (also known as ISO/IEC MPEG-4 AVC), including its scalable video coding (SVC) extension, multi-view video coding (MVC) extension, and MVC-based 3DV extension. In some cases, any bitstream conforming to the MVC-based 3DV extension of H.264/AVC always contains a sub-bitstream compliant with the MVC extension of H.264/AVC. In addition, work is underway to create a three-dimensional video (3DV) coding extension to H.264/AVC, i.e., 3DV-based AVC. In other examples, video encoder 20 and video decoder 30 may operate according to ITU-T H.261, ISO/IEC MPEG-1 Visual, ITU-T H.262, or ISO/IEC MPEG-2 Visual and ITU-T H.264, ISO/IEC Visual.

In other examples, video encoder 20 and video decoder 30 may operate according to the High Efficiency Video Coding (HEVC) standard developed by the Joint Collaboration Team on Video Coding (JCT-VC) of the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Motion Picture Experts Group (MPEG). A draft of the HEVC standard, referred to as "HEVC Working Draft 10," is described in "High Efficiency Video Coding (HEVC) Textual Specification Draft 10 (for FDIS and Consent)" by Bloss et al. (Joint Collaboration Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 12th Meeting, Geneva, Switzerland, January 2013) (hereinafter referred to as "HEVC Working Draft 10" or the "HEVC Baseline Specification"). In addition, work is underway to create a scalable video coding extension to HEVC. The scalable video coding extension of HEVC may be referred to as SHEVC or SHVC.

Furthermore, the Joint Collaborative Team on 3D Video Coding (JCT-3V) of VCEG and MPEG is currently developing a multi-view coding extension for HEVC (i.e., MV-HEVC). "MV-HEVC Draft Text 4" by Teck et al. (Joint Collaborative Team on 3D Video Coding Extension Development of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 4th Meeting: Incheon, South Korea, April 2013) (hereinafter referred to as "MV-HEVC Test Model 4") is a draft for MV-HEVC. In MV-HEVC, only high-level syntax (HLS) changes may be present, so that modules at the CU or PU level in HEVC do not need to be redesigned. This allows modules configured for HEVC to be reused in MV-HEVC. In other words, MV-HEVC only provides high-level syntax changes and not low-level syntax changes, such as those at the CU/PU level.

In addition, VCEG and MPEG's JCT-3V are developing a 3DV standard based on HEVC, with parts of the standardization effort including the standardization of a multi-view video codec based on HEVC (MV-HEVC) and another 3D video coding part based on HEVC (3D-HEVC). For 3D-HEVC, new coding tools for both texture and depth views (including those at the CU and/or PU level) may be included and supported. As of December 17, 2013, software for 3D-HEVC (e.g., 3D-HTM) is available for download at the following link: [3D-HTM Version 7.0]: https://hevc.hhi.fraunhofer.de/svn/svn_3DVCSoftware/tags/HTM-7.0/.

A reference software description and a working draft of 3D-HEVC are available as follows: “3D-HEVC Test Model 4” by Gerhard Teck et al. (JCT3V-D1005_spec_v1, Joint Collaboration Group on Development of 3D Video Coding Extensions of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 4th meeting: Incheon, South Korea, April 2013) (hereinafter referred to as “3D-HEVC Test Model 4”), which as of December 17, 2013, was available for download from the following link: http://phenix.it-sudparis.eu/jct2/doc_end_user/documents/2_Shanghai/wg11/JCT3V-B1005-v1.zip. "3D-HEVC Draft Text 3" by Tektronix et al. (Joint Collaborative Group on Development of 3D Video Coding Extensions of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC29/WG 11, 3rd Meeting: Geneva, Switzerland, January 2013, Document No. JCT3V-C1005) (hereinafter referred to as "3D-HEVC Test Model 3") (available as of December 17, 2013 at http://phenix.it-sudparis.eu/jct2/doc_end_user/current_document.php?id=706) is another version of the reference software description for 3D-HEVC. 3D-HEVC is also described in "3D-HEVC Draft Text 2" by Tektronix et al. (Joint Collaborative Group on 3D Video Coding Extension Development of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 6th Meeting: Geneva, Switzerland, October 25-November 1, 2013, document No. JCT3V-F1001-v2) (hereinafter referred to as "3D-HEVC Draft Text 2"). Video encoder 20 and video decoder 30 may operate according to SHEVC, MV-HEVC, and/or 3D-HEVC.

In HEVC and other video coding specifications, a video sequence typically comprises a series of pictures. A picture may also be referred to as a "frame." A picture may comprise three sample arrays, denoted as _SL , _SCb , and _SCr . _SL is a two-dimensional array (i.e., a block) of luma samples. _SCb is a two-dimensional array of Cb chroma samples. _SCr is a two-dimensional array of Cr chroma samples. Chroma samples may also be referred to herein as "chroma" samples. In other cases, a picture may be monochrome and may comprise only a luma sample array.

To generate an encoded representation of a picture, video encoder 20 may generate a set of coding tree units (CTUs). Each of the CTUs may include a coding tree block of luma samples, two corresponding coding tree blocks of chroma samples, and syntax structures used to code the samples of the coding tree blocks. In monochrome pictures or pictures with three separate color planes, a CTU may include a single coding tree block and syntax structures used to code the samples of the coding tree block. A coding tree block may be an N×N block of samples. A CTU may also be referred to as a “tree block” or a “largest coding unit (LCU)”. A CTU of HEVC may be broadly similar to macroblocks of other standards, such as H.264/AVC. However, a CTU is not necessarily limited to a specific size and may include one or more coding units (CUs). A slice may include an integer number of CTUs ordered contiguously in raster scan order.

This disclosure may use the term "video unit," or "video block," or "block" to refer to one or more blocks of samples and syntax structures used to code samples of the one or more blocks of samples. Example types of video units may include CTUs, CUs, PUs, transform units (TUs), macroblocks, macroblock partitions, and the like. In some cases, discussion of PUs may be interchangeable with discussion of macroblocks or macroblock partitions.

To generate a coded CTU, video encoder 20 may recursively perform quadtree partitioning on the coding tree block of the CTU to divide the coding tree block into coding blocks, hence the name "coding tree unit". A coding block is an N×N block of samples. A CU may include a coding block of luma samples and two corresponding coding blocks of chroma samples of a picture having a luma sample array, a Cb sample array, and a Cr sample array, and syntax structures used to code the samples of the coding blocks. In a monochrome picture or a picture with three separate color planes, a CU may include a single coding block and syntax structures used to code the samples of the coding block.

Video encoder 20 may partition the coding blocks of a CU into one or more prediction blocks. A prediction block is a rectangular (i.e., square or non-square) block of samples to which the same prediction applies. A prediction unit (PU) of a CU may include a prediction block of luma samples, two corresponding prediction blocks of chroma samples, and syntax structures used to predict the prediction blocks. In monochrome pictures or pictures with three separate color planes, a PU may include a single prediction block and syntax structures used to predict the prediction blocks. Video encoder 20 may generate predictive luma, Cb, and Cr blocks for the luma, Cb, and Cr prediction blocks of each PU of a CU.

Video encoder 20 may use intra prediction or inter prediction to generate the predictive blocks for the PU. If video encoder 20 generates the predictive blocks for the PU using intra prediction, video encoder 20 may generate the predictive blocks for the PU based on decoded samples of the picture associated with the PU. In some versions of HEVC, for the luma component of each PU, an intra prediction method is utilized using 33 angular prediction modes (indexed from 2 to 34), a DC mode (indexed with 1), and a planar mode (indexed with 0), as shown in FIG2 . FIG2 is a conceptual diagram illustrating example intra prediction modes in HEVC.

If video encoder 20 uses inter-prediction to generate the predictive blocks of a PU, video encoder 20 may generate the predictive blocks of the PU based on decoded samples of one or more pictures other than the picture associated with the PU. Inter-prediction may be unidirectional inter-prediction (i.e., unidirectional prediction) or bidirectional inter-prediction (i.e., bidirectional prediction). To perform inter-prediction, video encoder 20 may generate a first reference picture list (RefPicList0) for the current slice and, in some cases, may also generate a second reference picture list (RefPicList1) for the current slice. Each of the reference picture lists may include one or more reference pictures. When using uni-directional prediction, video encoder 20 may search the reference pictures in either or both of RefPicList0 and RefPicList1 to determine a reference position within the reference pictures. Furthermore, when using uni-directional prediction, video encoder 20 may generate the predictive sample blocks of the PU based at least in part on samples corresponding to the reference position. Furthermore, when using uni-directional prediction, video encoder 20 may generate a single motion vector that indicates the spatial displacement between the prediction block of the PU and the reference position. To indicate the spatial displacement between the PU's prediction block and the reference location, the motion vector may include a horizontal component specifying the horizontal displacement between the PU's prediction block and the reference location and may include a vertical component specifying the vertical displacement between the PU's prediction block and the reference location.

When bi-prediction is used to encode a PU, video encoder 20 may determine a first reference location in a reference picture in RefPicList0 and a second reference location in a reference picture in RefPicList1. Video encoder 20 may then generate a predictive block for the PU based at least in part on samples corresponding to the first and second reference locations. Furthermore, when bi-prediction is used to encode a PU, video encoder 20 may generate a first motion vector indicating a spatial displacement between the sample block of the PU and the first reference location, and a second motion vector indicating a spatial displacement between the prediction block of the PU and the second reference location.

Typically, reference picture list construction for the first or second reference picture list (e.g., RefPicList0 or RefPicList1) of a B picture involves two steps: reference picture list initialization and reference picture list reordering (modification). Reference picture list initialization is a definitive mechanism that places reference pictures in the reference picture memory (also known as the decoded picture buffer (DPB)) into lists based on the order of their POC (Picture Order Count) values, which align with the display order of the pictures. The reference picture list reordering mechanism can modify the position of pictures placed in the lists during reference picture list initialization to any new position or place any reference picture in the reference picture memory at any position, even if the picture does not belong to the initialized list. Some pictures after reference picture list reordering (modification) may be placed at very far positions in the lists. However, if a picture's position exceeds the number of valid reference pictures for a list, it is not considered an entry in the final reference picture list. The number of valid reference pictures for each list can be signaled in the slice header. After the reference picture lists (ie, RefPicList0 and RefPicList1, if available) are constructed, reference indexes to the reference picture lists may be used to identify any reference pictures included in the reference picture lists.

After video encoder 20 generates predictive blocks (e.g., luma blocks, Cb blocks, and Cr blocks) for one or more PUs of a CU, video encoder 20 may generate one or more residual blocks for the CU. For example, video encoder 20 may generate a luma residual block for the CU. Each sample in the luma residual block of the CU indicates the difference between a luma sample in one of the CU's predictive luma blocks and a corresponding sample in the CU's original luma coding block. Additionally, video encoder 20 may generate a Cb residual block for the CU. Each sample in the Cb residual block of the CU may indicate the difference between a Cb sample in one of the CU's predictive Cb blocks and a corresponding sample in the CU's original Cb coding block. Video encoder 20 may also generate a Cr residual block for the CU. Each sample in the Cr residual block of the CU may indicate the difference between a Cr sample in one of the CU's predictive Cr blocks and a corresponding sample in the CU's original Cr coding block.

Furthermore, video encoder 20 may use quadtree partitioning to decompose a residual block (e.g., luma, Cb, and Cr residual blocks) of a CU into one or more transform blocks (e.g., luma, Cb, and Cr transform blocks). A transform block is a rectangular (e.g., square or non-square) block of samples to which the same transform is applied. A transform unit (TU) of a CU may include a transform block of luma samples, two corresponding transform blocks of chroma samples, and syntax structures used to transform the transform block samples. Thus, each TU of a CU may be associated with a luma transform block, a Cb transform block, and a Cr transform block. The luma transform block associated with a TU may be a sub-block of the CU's luma residual block. The Cb transform block may be a sub-block of the CU's Cb residual block. The Cr transform block may be a sub-block of the CU's Cr residual block. In monochrome pictures or pictures with three separate color planes, a TU may include a single transform block and syntax structures used to transform the samples of the transform block.

Video encoder 20 may apply one or more transforms to a transform block of a TU to generate a coefficient block for the TU. For example, video encoder 20 may apply one or more transforms to a luma transform block of a TU to generate a luma coefficient block for the TU. A coefficient block may be a two-dimensional array of transform coefficients. A transform coefficient may be a scalar. Video encoder 20 may apply one or more transforms to a Cb transform block of a TU to generate a Cb coefficient block for the TU. Video encoder 20 may apply one or more transforms to a Cr transform block of a TU to generate a Cr coefficient block for the TU.

After generating a coefficient block (e.g., a luma coefficient block, a Cb coefficient block, or a Cr coefficient block), video encoder 20 may quantize the coefficient block. Quantization generally refers to the process of quantizing transform coefficients to potentially reduce the amount of data used to represent the transform coefficients, thereby providing further compression. After video encoder 20 quantizes the coefficient block, video encoder 20 may entropy encode syntax elements indicating the quantized transform coefficients. For example, video encoder 20 may perform context-adaptive binary arithmetic coding (CABAC) on the syntax elements indicating the quantized transform coefficients.

Video encoder 20 may output a bitstream that includes a sequence of bits that form a representation of a coded picture and associated data. The bitstream may include a sequence of network abstraction layer (NAL) units. A NAL unit is a syntax structure that contains an indication of the type of data in the NAL unit and bytes containing the data in the form of a raw byte sequence payload (RBSP) interspersed with emulation prevention bits as needed. Each NAL unit includes a NAL unit header and encapsulates the RBSP. The NAL unit header may include a syntax element that indicates a NAL unit type code. The NAL unit type code, specified by the NAL unit header of a NAL unit, indicates the type of the NAL unit. The RBSP may be a syntax structure that contains an integer number of bytes encapsulated within the NAL unit. In some cases, the RBSP includes zero bits.

Different types of NAL units may encapsulate different types of RBSPs. For example, different types of NAL units may encapsulate different RBSPs for video parameter sets (VPSs), sequence parameter sets (SPSs), picture parameter sets (PPSs), coded slices, supplemental enhancement information (SEI), and so on. NAL units that encapsulate RBSPs for video coding data (as opposed to RBSPs for parameter sets and SEI messages) may be referred to as video coding layer (VCL) NAL units.

In HEVC, the SPS may contain information applicable to all slices of a coded video sequence (CVS). A CVS may include a sequence of pictures. In HEVC, a CVS may start with an instantaneous decoding refresh (IDR) picture, a broken link access (BLA) picture, or a clean random access (CRA) picture that is the first picture in the bitstream, and include all subsequent pictures that are not IDR or BLA pictures. That is, in HEVC, a CVS may include a sequence of access units that, in decoding order, may consist of: a CRA access unit, an IDR access unit, or a BLA access unit that is the first access unit in the bitstream, followed by zero or more non-IDR and non-BLA access units, including all subsequent access units up to but not including any subsequent IDR or BLA access unit. In HEVC, an access unit may be a set of NAL units that are consecutive in decoding order and contain exactly one coded picture. In addition to the coded slice NAL units of a coded picture, an access unit may also contain other NAL units that do not contain slices of a coded picture. In some examples, decoding an access unit always produces a decoded picture.

A VPS is a syntax structure that includes syntax elements that apply to zero or more entire CVSs. An SPS is also a syntax structure that includes syntax elements that apply to zero or more entire CVSs. An SPS may include syntax elements that identify the VPS that is in effect when the SPS is in effect. Thus, syntax elements of a VPS may be more generally applicable than syntax elements of an SPS. A PPS is a syntax structure that includes syntax elements that apply to zero or more coded pictures. A PPS may include syntax elements that identify the SPS that is in effect when the PPS is in effect. A slice header for a slice may include syntax elements that indicate the PPS that is in effect when the slice is being coded.

Video decoder 30 may receive a bitstream generated by video encoder 20. Additionally, video decoder 30 may parse the bitstream to obtain syntax elements from the bitstream. Video decoder 30 may reconstruct a picture of the video data based, at least in part, on the syntax elements obtained from the bitstream. The process of reconstructing the video data may generally be the inverse of the process performed by video encoder 20. For example, video decoder 30 may determine a predictive block for a PU of a current CU using the motion vector of the PU. Additionally, video decoder 30 may inverse quantize a coefficient block associated with a TU of the current CU. Video decoder 30 may perform an inverse transform on the coefficient block to reconstruct a transform block associated with the TU of the current CU. Video decoder 30 may reconstruct the coding blocks of the current CU by adding samples of the predictive blocks of the PUs of the current CU to corresponding samples of the transform blocks of the TUs of the current CU. By reconstructing the coding blocks of each CU of the picture, video decoder 30 may reconstruct the picture.

In some examples, video encoder 20 may signal the motion information of a PU using merge mode or advanced motion vector prediction (AMVP) mode. For example, in HEVC, there are two modes for motion parameter prediction, one is merge mode and the other is AMVP. Motion prediction may include determining the motion information of a block (e.g., a PU) based on the motion information of one or more other blocks. The motion information of a PU (also referred to herein as motion parameters) may include a motion vector of the PU and a reference index of the PU.

When video encoder 20 uses merge mode to signal the motion information of a current PU, video encoder 20 generates a merge candidate list. In other words, video encoder 20 may perform a motion vector predictor list construction process. The merge candidate list includes a set of merge candidates that indicate the motion information of PUs that are spatially or temporally neighboring the current PU. That is, in merge mode, a candidate list of motion parameters (e.g., reference index, motion vector, etc.) is constructed, where the candidates may come from spatially and temporally neighboring blocks.

Furthermore, in merge mode, video encoder 20 may select a merge candidate from a merge candidate list and may use the motion information indicated by the selected merge candidate as the motion information for the current PU. Video encoder 20 may signal the position of the selected merge candidate in the merge candidate list. For example, video encoder 20 may signal the selected motion vector parameters by transmitting an index to the candidate list. Video decoder 30 may obtain an index to the candidate list (i.e., a candidate list index) from the bitstream. Alternatively, video decoder 30 may generate the same merge candidate list and may determine the selected merge candidate based on the indication of the position of the selected merge candidate. Video decoder 30 may then use the motion information of the selected merge candidate to generate a predictive block for the current PU. That is, video decoder 30 may determine the selected candidate in the candidate list based at least in part on the candidate list index, where the selected candidate specifies the motion vector for the current PU. In this manner, at the decoder side, once the index is decoded, all motion parameters for the corresponding block pointed to by the index may be inherited by the current PU.

Skip mode is similar to merge mode. In skip mode, video encoder 20 and video decoder 30 generate and use a merge candidate list in the same manner as video encoder 20 and video decoder 30 use a merge candidate list in merge mode. However, when video encoder 20 signals the motion information of the current PU using skip mode, video encoder 20 does not signal any residual data for the current PU. Therefore, video decoder 30 can determine the predictive block of the PU based on the reference block indicated by the motion information of the selected candidate in the merge candidate list without using residual data.

AMVP mode is similar to merge mode in that video encoder 20 may generate a candidate list and may select a candidate from the candidate list. However, when video encoder 20 signals the RefPicListX motion information of the current PU using AMVP mode, video encoder 20 may signal the RefPicListX motion vector difference (MVD) of the current PU and the RefPicListX reference index of the current PU in addition to signaling the RefPicListX MVP flag of the current PU. The RefPicListX MVP flag of the current PU may indicate the position of the selected AMVP candidate in the AMVP candidate list. The RefPicListX MVD of the current PU may indicate the difference between the RefPicListX motion vector of the current PU and the motion vector of the selected AMVP candidate. In this way, video encoder 20 can signal the RefPicListX motion information of the current PU by signaling the RefPicListX motion vector predictor (MVP) flag, the RefPicListX reference index value, and the RefPicListX MVD. In other words, the data representing the motion vector of the current PU in the bitstream may include data representing the reference index, an index into the candidate list, and the MVD.

In addition, when the motion information of the current PU is signaled using the AMVP mode, the video decoder 30 may obtain the MVD and MVP flag of the current PU from the bitstream. The video decoder 30 may generate the same AMVP candidate list and may determine the selected AMVP candidate based on the MVP flag. The video decoder 30 may recover the motion vector of the current PU by adding the MVD to the motion vector indicated by the selected AMVP candidate. That is, the video decoder 30 may determine the motion vector of the current PU based on the motion vector indicated by the selected AMVP candidate and the MVD. The video decoder 30 may then use the recovered motion vector(s) of the current PU to generate the predictive block of the current PU.

When video decoder 30 generates an AMVP candidate list for a current PU, video decoder 30 may derive one or more AMVP candidates based on motion information of PUs that cover positions that are spatially adjacent to the current PU (i.e., spatially adjacent PUs). FIG3 is a conceptual diagram illustrating example spatially adjacent PUs relative to current block 40. In the example of FIG3, spatially adjacent PUs may be PUs that cover positions indicated as _A0 , _A1 , _B0 , _B1 , and _B2 . When the prediction block of a PU includes a position, the PU may cover that position.

Candidates in a merge candidate list or an AMVP candidate list that are based on motion information of a PU that is temporally adjacent to the current PU (i.e., a PU at a different time instance than the current PU) may be referred to as temporal motion vector predictors. The use of temporal motion vector prediction may be referred to as temporal motion vector prediction (TMVP). TMVP may be used to improve HEVC coding efficiency, and unlike other coding tools, TMVP may require access to motion vectors for frames in a decoded picture buffer (more specifically, in a reference picture list).

TMVP may be enabled or disabled on a per-CVS basis, a per-slice basis, or another basis. A syntax element in the SPS (e.g., sps_temporal_mvp_enable_flag) may indicate whether use of TMVP is enabled for a CVS. Furthermore, when TMVP is enabled for a CVS, TMVP may be enabled or disabled for a specific slice within the CVS. For example, a syntax element in a slice header (e.g., slice_temporal_mvp_enable_flag) may indicate whether TMVP is enabled for a slice. Thus, in an inter-predicted slice, when TMVP is enabled for the entire CVS (e.g., sps_temporal_mvp_enable_flag in the SPS is set to 1), a slice_temporal_mvp_enable_flag is signaled in the slice header to indicate whether TMVP is enabled for the current slice.

To determine the temporal motion vector predictor, the video coder may first identify a reference picture that includes a PU that is co-located with the current PU. In other words, the video coder may identify a so-called "co-located picture." If the current slice of the current picture is a B slice (i.e., a slice that allows inclusion of bi-directionally inter-predicted PUs), video encoder 20 may signal in the slice header a syntax element (e.g., collocated_from_l0_flag) that indicates whether the co-located picture is from RefPicList0 or RefPicList1. In other words, when TMVP is enabled for the current slice and the current slice is a B slice (e.g., a slice that allows inclusion of bi-directionally inter-predicted PUs), video encoder 20 may signal in the slice header a syntax element (e.g., collocated_from_l0_flag) that indicates whether the co-located picture is in RefPicList0 or RefPicList1.

A syntax element in the slice header (e.g., collocated_ref_idx) may indicate a collocated picture in the identified reference picture list. Thus, after video decoder 30 identifies a reference picture list that includes a collocated picture, video decoder 30 may use the collocated_ref_idx signaled in the slice header to identify the collocated picture in the identified reference picture list. The video coder may identify a collocated PU by examining the collocated picture. The temporal motion vector predictor may indicate motion information for the PU to the bottom right of the collocated PU, or motion information for the center PU of the collocated PU.

When the motion vector identified by the above process (i.e., the motion vector of the temporal motion vector predictor) is used to generate a motion candidate for merge mode or AMVP mode, the video coder may scale the motion vector based on the temporal position (reflected by the POC value). For example, when the difference between the POC values of the current picture and the reference picture is greater than when the difference between the POC values of the current picture and the reference picture is smaller, the video coder may increase the magnitude of the motion vector by the excess amount.

The target reference index of all possible reference picture lists for temporal merging candidates derived from the temporal motion vector predictor may be always set to 0. However, for AMVP, the target reference index of all possible reference pictures may be set equal to the decoded reference index. In other words, the target reference index of all possible reference picture lists for temporal merging candidates derived from TMVP is always set to 0, while for AMVP, the temporal merging candidates may be set equal to the decoded reference index. In HEVC, the SPS may include a flag (e.g., sps_temporal_mvp_enable_flag) and when sps_temporal_mvp_enable_flag is equal to 1, the slice header may include a flag (e.g., pic_temporal_mvp_enable_flag). When both pic_temporal_mvp_enable_flag and temporal_id are equal to 0 for a particular picture, motion vectors from pictures that precede the particular picture in decoding order are not used as temporal motion vector predictors to decode the particular picture or pictures that follow the particular picture in decoding order.

The techniques of this disclosure are potentially applicable to multi-view coding and/or 3DV standards and specifications, including MV-HEVC and 3D-HEVC. In multi-view coding, such as that defined in MV-HEVC and 3D-HEVC, there may be multiple views of the same scene from different viewpoints. In the context of multi-view coding, the term "access unit" may be used to refer to a set of pictures corresponding to the same time instance. In some cases, in the context of multi-view coding, an access unit may be a set of NAL units that are associated with each other according to a specified classification rule, are consecutive in decoding order, and contain the VCL NAL units and their associated non-VCL NAL units of all coded pictures associated with the same output time. Thus, video data can be conceptualized as a series of access units occurring over time.

In 3DV coding, such as that defined in 3D-HEVC, a "view component" may be a coded representation of a view in a single access unit. A view component may contain a depth view component and a texture view component. A depth view component may be a coded representation of the depth of a view in a single access unit. A texture view component may be a coded representation of the texture of a view in a single access unit. In this disclosure, a "view" may refer to a sequence of view components associated with the same view identifier.

Texture view components and depth view components within a view's picture set may be considered to correspond to each other. For example, a texture view component within a view's picture set may be considered to correspond to a depth view component within the view's picture set, and vice versa (i.e., a depth view component corresponds to its texture view component in the set, and vice versa). As used in this disclosure, a texture view component corresponding to a depth view component may be considered to be part of the same view of a single access unit, as may the texture view component and the depth view component.

The texture view component contains the actual image content being displayed. For example, a texture view component may include luma (Y) and chroma (Cb and Cr) components. A depth view component may indicate the relative depth of pixels in its corresponding texture view component. As an example, a depth view component is a grayscale image containing only luma values. In other words, a depth view component may not convey any image content, but rather provides a measurement of the relative depth of pixels in the texture view component.

For example, a pure white pixel in a depth view component indicates that its corresponding pixel in the corresponding texture view component is closer to the viewer's perspective, and a pure black pixel in a depth view component indicates that its corresponding pixel in the corresponding texture view component is farther from the viewer's perspective. Various shades of gray between black and white indicate different depth levels. For example, a dark gray pixel in a depth view component indicates that its corresponding pixel in the texture view component is farther away than a light gray pixel in the depth view component. Because only grayscale is needed to identify the depth of a pixel, the depth view component does not need to include chroma components, as the color values of the depth view component may not serve any purpose.

The use of only luma values (eg, intensity values) to identify depth view components is provided for illustration purposes and should not be considered limiting. In other examples, any technique may be utilized to indicate the relative depth of pixels in a texture view component.

In multi-view coding, a view may be referred to as a "base view" if a video decoder (e.g., video decoder 30) can decode pictures in the view without referencing pictures in any other view. When coding a picture in one of the non-base views, the video coder (e.g., video encoder 20 or video decoder 30) may add the picture to a reference picture list (e.g., RefPicList0 or RefPicList1) if the picture is in a different view but within the same time instance (i.e., access unit) as the picture currently being coded by the video coder. Like other inter-prediction reference pictures, the video coder may insert an inter-view prediction reference picture at any position in the reference picture list.

Multi-view coding supports inter-view prediction. Inter-view prediction is similar to inter-frame prediction used in H.264/AVC, HEVC, or other video coding specifications, and can use the same syntax elements. However, when the video coder performs inter-view prediction on the current block (e.g., macroblock, CU, or PU), the video encoder 20 can use pictures in the same access unit as the current block but in different views as reference pictures. In other words, in multi-view coding, inter-view prediction is performed among pictures captured in different views of the same access unit (i.e., within the same time instance) to remove the correlation between views. In contrast, conventional inter-frame prediction only uses pictures in different access units as reference pictures.

FIG4 is a conceptual diagram illustrating an example multi-view decoding order. The multi-view decoding order may be a bitstream order. In the example of FIG4, each square corresponds to a view component. The columns of squares correspond to access units. Each access unit may be defined as a coded picture containing all views of a time instance. The rows of squares correspond to views. In the example of FIG4, the access units are labeled T0 ... T8 and the views are labeled S0 ... S7. Because each view component of an access unit is decoded before any view component of the next access unit, the decoding order of FIG4 may be referred to as time-first decoding. The decoding order of the access units may not be the same as the output or display order of the views.

FIG5 is a conceptual diagram illustrating an example prediction structure for multi-view coding. The multi-view prediction structure of FIG5 includes temporal and inter-view prediction. In the example of FIG5, each square corresponds to a view component. In the example of FIG5, the access units are labeled T0...T11 and the views are labeled S0...S7. The squares labeled "I" are intra-predicted view components. The squares labeled "P" are unidirectional inter-predicted view components. The squares labeled "B" and "b" are bidirectional inter-predicted view components. The square labeled "b" can use the square labeled "B" as a reference picture. An arrow pointing from a first square to a second square indicates that the first square can be used as a reference picture for the second square in inter-prediction. As indicated by the vertical arrows in FIG5, view components in different views of the same access unit can be used as reference pictures. Using one view component of an access unit as a reference picture for another view component of the same access unit is referred to as inter-view prediction.

In multi-view coding, such as the MVC extension of H.264/AVC, inter-view prediction is supported by disparity motion compensation, which uses the syntax of H.264/AVC motion compensation but allows pictures in different views to be used as reference pictures. Coding of two views can also be supported by the MVC extension of H.264/AVC. One of the advantages of the MVC extension of H.264/AVC is that an MVC encoder can treat more than two views as 3D video input, and an MVC decoder can decode this multi-view representation. Therefore, any display with an MVC decoder can expect 3D video content with more than two views.

In the context of multi-view video coding, such as that defined in MV-HEVC and 3D-HEVC, there are two types of motion vectors. One type of motion vector is a normal motion vector that points to a temporal reference picture. The inter-frame prediction type corresponding to the normal temporal motion vector may be referred to as "motion compensated prediction" or "MCP". When an inter-view prediction reference picture is used for motion compensation, the corresponding motion vector is referred to as a "disparity motion vector". In other words, the disparity motion vector points to a picture in a different view (i.e., an inter-view reference picture). The inter-frame prediction type corresponding to the disparity motion vector may be referred to as "disparity compensated prediction" or "DCP".

3D-HEVC can improve coding efficiency using inter-view motion prediction and inter-view residual prediction. Specifically, to further improve coding efficiency, two new techniques, inter-view motion prediction and inter-view residual prediction, have been adopted in the reference software. In inter-view motion prediction, the video coder can determine (i.e., predict) the motion information of the current PU based on the motion information of PUs in views other than the current PU. In inter-view residual prediction, the video coder can determine the residual block of the current CU based on residual data in views other than the current CU.

To enable inter-view motion prediction and inter-view residual prediction, a video coder may determine a disparity vector for a block (e.g., a PU, CU, etc.). In other words, to enable these two coding tools, the first step is to derive a disparity vector. Generally speaking, a disparity vector is used as an estimator of the displacement between two views. The video coder may use the disparity vector of a block to locate a reference block in another view for inter-view motion or residual prediction, or the video coder may convert the disparity vector into a disparity motion vector for inter-view motion prediction. That is, the disparity vector may be used to locate a corresponding block in another view for inter-view motion/residual prediction, or may be converted into a disparity motion vector for inter-view motion prediction.

In some examples, the video coder may use a neighboring block-based disparity vector (NBDV) derivation method to derive the disparity vector of a PU (i.e., the current PU). For example, to derive the disparity vector of the current PU, a process called NBDV derivation may be used in the 3D-HEVC test model (i.e., 3D-HTM).

The NBDV derivation process uses disparity motion vectors from spatially and temporally neighboring blocks to derive the disparity vector for the current block. Because neighboring blocks (e.g., blocks that are spatially or temporally neighboring the current block) are likely to share nearly identical motion and disparity information in video coding, the current block can use the motion vector information in the neighboring blocks as a predictor for the disparity vector for the current block. Therefore, the NBDV derivation process uses neighboring disparity information for estimating disparity vectors in different views.

During the NBDV derivation process, the video coder may check the motion vectors of spatially and temporally neighboring PUs in a fixed order. When the video coder checks the motion vectors of spatially or temporally neighboring PUs, it may determine whether the motion vector is a disparity motion vector. The disparity motion vector of a PU of a picture is a motion vector that points to a location within an inter-view reference picture of the picture. An inter-view reference picture of a picture may be a picture in the same access unit as the picture but in a different view. When the video coder identifies a disparity motion vector or an implicit disparity vector (IDV), the video coder may terminate the checking process. The IDV may be the disparity vector of a spatially or temporally neighboring PU coded using inter-view prediction. The IDV may be generated when the PU utilizes inter-view motion vector prediction (i.e., a candidate for AMVP or merge mode is derived from a reference block in another view using the disparity vector). The IDV may be stored in the PU for disparity vector derivation purposes. Furthermore, when the video coder identifies a disparity motion vector or IDV, it may return the identified disparity motion vector or IDV.

A simplified version of the IDV and NBDV derivation processes is included in Sung et al., "3D-CE5.h: Simplified Disparity Vector Derivation for HEVC-Based 3D Video Coding," document JCTV3-A0126. The use of IDV in the NBDV derivation process is further simplified in Kang et al., "3D-CE5.h Related: Disparity Vector Derivation Improvements," document JCT3V-B0047, by removing the IDV stored in the decoded picture buffer and also providing improved coding gain through random access point (RAP) picture selection. The video coder can convert the returned disparity motion vector or IDV into a disparity vector and use the disparity vector for inter-view motion prediction and inter-view residual prediction.

In some designs of 3D-HEVC, when the video coder performs the NBDV derivation process, the video coder checks the disparity motion vectors in the temporally neighboring blocks, the disparity motion vectors in the spatially neighboring blocks, and then the IDV in order. Once the video coder finds the disparity motion vector for the current block, the video coder may terminate the NBDV derivation process. Thus, once the disparity motion vector or IDV is identified, the checking process is terminated and the identified disparity motion vector is returned and converted into a disparity vector to be used for inter-view motion prediction and inter-view residual prediction. When the video coder is unable to determine the disparity vector for the current block by performing the NBDV derivation process (i.e., when no disparity motion vector or IDV is found during the NBDV derivation process), the video coder may mark the NBDV as unavailable.

In some examples, if the video coder is unable to derive the disparity vector for the current PU by performing the NBDV derivation process (i.e., if no disparity vector is found), the video coder may use a zero disparity vector as the disparity vector for the current PU. A zero disparity vector is a disparity vector in which both the horizontal and vertical components are equal to 0. Therefore, even when the NBDV derivation process returns an unusable result, other coding processes of the video coder that require a disparity vector may use a zero disparity vector for the current block.

In some examples, if the video coder is unable to derive the disparity vector of the current PU by performing the NBDV derivation process, the video coder may disable inter-view residual prediction for the current PU. However, regardless of whether the video coder is able to derive the disparity vector of the current PU by performing the NBDV derivation process, the video coder may use inter-view motion prediction for the current PU. That is, if no disparity vector is found after checking all predefined neighboring blocks, a zero disparity vector may be used for inter-view motion prediction, while inter-view residual prediction for the corresponding PU may be disabled.

As mentioned above, as part of the process of determining the disparity vector for the current PU, the video coder may check spatially neighboring PUs. In some examples, the video coder checks the following spatially neighboring blocks: a lower left spatially neighboring block, a left spatially neighboring block, a top right spatially neighboring block, an upper spatially neighboring block, and a top left spatially neighboring block. For example, in some versions of the NBDV derivation process, five spatially neighboring blocks are used for disparity vector derivation. The five spatially neighboring blocks may cover positions A ₀ , A ₁ , B ₀ , B ₁ , and B ₂ , respectively, as indicated in FIG3 . The video coder may check the five spatially neighboring blocks in the order of A ₁ , B ₁ , B ₀ , A ₀ , and B ₂ . The same five spatially neighboring blocks may be used for the merge mode of HEVC. Therefore, in some examples, no additional memory access is required. If one of the spatially neighboring blocks has a disparity motion vector, the video coder may terminate the checking process, and the video coder may use the disparity motion vector as the final disparity vector for the current PU. In other words, if one of them uses a disparity motion vector, the checking process is terminated and the corresponding disparity motion vector will be used as the final disparity vector.

Furthermore, as mentioned above, as part of the process of determining the disparity vector for the current PU, the video coder may check temporally neighboring PUs. To check temporally neighboring blocks (e.g., PUs), the candidate picture list construction process may be performed first. In some examples, the video coder may check up to two reference pictures from the current view to find the disparity motion vector. The first reference picture may be a co-located picture. Therefore, the co-located picture (i.e., the co-located reference picture) may be inserted first into the candidate picture list. The second reference picture may be a random access picture, or the reference picture with the smallest POC value difference and the smallest temporal identifier. In other words, up to two reference pictures from the current view, the co-located picture and the random access picture, or the reference picture with the smallest POC difference and the smallest temporal identifier, are considered for temporal block checking. The video coder may first check the random access picture and then the co-located picture.

For each candidate picture (i.e., random access pictures and co-located pictures), the video coder may check two blocks. Specifically, the video coder may check a center block (CR) and a bottom right block (BR). FIG6 is a conceptual diagram illustrating an example temporal neighboring block in the NBDV derivation process. The center block may be the center 4×4 block of the co-located region of the current PU. The bottom right block may be the bottom right 4×4 block of the co-located region of the current PU. Thus, for each candidate picture, two blocks are checked in order, CR and BR for the first non-base view, or BR, CR for the second non-base view. If one of the PUs covering the CR or BR has a disparity motion vector, the video coder may terminate the checking process and may use the disparity motion vector as the final disparity vector for the current PU. In this example, the decoding of the picture associated with the first non-base view may depend on the decoding of the picture associated with the base view, but not on the decoding of the pictures associated with the other views. Furthermore, in this example, decoding of pictures associated with the second non-base view may depend on decoding of pictures associated with the base view and, in some cases, the first non-base view, rather than pictures associated with other views, if any.

In the example of FIG6 , block 42 indicates the co-located region of the current PU. Furthermore, in the example of FIG6 , the block labeled "Pos. A" corresponds to the center block. The block labeled "Pos. B" corresponds to the lower-right block. As indicated in the example of FIG6 , the center block may be located directly below and to the right of the center point of the co-located region.

When the video coder checks a neighboring PU (i.e., a spatial or temporal neighboring PU), the video coder may first check whether the neighboring PU has a disparity motion vector. If none of the neighboring PUs has a disparity motion vector, the video coder may determine whether any of the spatially neighboring PUs has an IDV. In other words, first check whether a disparity motion vector is used for all spatial/temporal neighboring blocks and then check the IDV. First check the spatial neighboring blocks and then check the temporal neighboring blocks. When checking neighboring blocks to find the IDV, the video coder may check the spatially neighboring PUs in the order of A ₀ , A ₁ , B ₀ , B ₁ , and B _2. If one of the spatially neighboring PUs has an IDV and the IDV is coded as a merge/skip mode, the video coder may terminate the checking process and may use the IDV as the final disparity vector for the current PU. In other words, check the five spatially neighboring blocks in the order of A ₀ , A ₁ , B ₀ , B ₁ , and B ₂ . If one of the neighboring blocks uses IDV and it can be coded as skip/merge mode, the checking process is terminated and the corresponding IDV can be used as the final disparity vector.

As indicated above, the disparity vector of the current block may indicate a position in a reference picture (i.e., a reference view component) in a reference view. In some 3D-HEVC designs, the video coder is allowed to access depth information of the reference view. In some such 3D-HEVC designs, when the video coder derives the disparity vector of the current block using the NBDV derivation process, the video coder may apply an optimization process to further optimize the disparity vector of the current block. The video coder may optimize the disparity vector of the current block based on the depth map of the reference picture. In other words, the disparity vector generated from the NBDV scheme may be further optimized using information in the coded depth map. That is, the accuracy of the disparity vector may be enhanced by utilizing the information coded in the base view depth map. This optimization process may be referred to herein as NBDV optimization ("NBDV-R"), an NBDV optimization process, or depth-oriented NBDV (Do-NBDV).

When the NBDV derivation process returns a usable disparity vector (e.g., when the NBDV derivation process returns a variable indicating that the NBDV derivation process is able to derive the disparity vector of the current block based on the disparity motion vector or IDV of the neighboring block), the video coder can further optimize the disparity vector by retrieving depth data from the depth map of the reference view. In some examples, the optimization process includes the following steps:

1. Use the disparity vector of the current block to locate the block in the depth map of the reference view. In other words, locate the corresponding depth block in the previously coded reference depth view (e.g., the base view) through the derived disparity vector. In this example, the size of the corresponding block in depth can be the same as the size of the current PU (i.e., the size of the prediction block of the current PU).

2. Calculate the disparity vector from the maximum of the four corner depth values of the co-located depth block. This value is set equal to the horizontal component of the disparity vector, while the vertical component of the disparity vector is set to 0.

In some examples, when the NBDV derivation process does not return a usable disparity vector (e.g., when the NBDV derivation process returns a variable indicating that the NBDV derivation process is unable to derive the disparity vector of the current block based on the disparity motion vector or IDV of the neighboring block), the video coder does not perform the NBDV optimization process and the video coder may use a zero disparity vector as the disparity vector of the current block. In other words, when the NBDV derivation process does not provide a usable disparity vector and thus the result of the NBDV derivation process is unusable, the NBDV-R process described above is skipped and a zero disparity vector is directly returned.

In some proposals for 3D-HEVC, the video coder uses the optimized disparity vector of the current block for inter-view motion prediction while the video coder uses the unoptimized disparity vector of the current block for inter-view residual prediction. For example, the video coder may use an NBDV derivation process to derive the unoptimized disparity vector of the current block. The video coder may then apply the NBDV optimization process to derive the optimized disparity vector of the current block. The video coder may use the optimized disparity vector of the current block to determine motion information for the current block. Furthermore, the video coder may use the unoptimized disparity vector of the current block to determine the residual block of the current block.

In this way, this new disparity vector is called "depth-oriented neighbor-based disparity vector (DoNBDV)". This newly derived disparity vector from the DoNBDV scheme is then used to replace the disparity vector from the NBDV scheme for inter-view candidate derivation for AMVP and merge mode. The video decoder can use the unoptimized disparity vector for inter-view residual prediction.

The video coder may use a similar optimization process to optimize the disparity motion vector for use in backward view synthesis prediction (BVSP). In this way, depth may be used to optimize the disparity vector or the disparity motion vector to be used for BVSP. If the optimized disparity vector is coded in BVSP mode, the optimized disparity vector may be stored as the motion vector of one PU.

The video coder can perform Backward Warping (BVSP) to synthesize view components. The BVSP method was proposed in Tian et al., "CE1.h: Backward View Synthesis Prediction Using Neighboring Blocks," document JCT3V-C0152 (hereinafter referred to as "JCT3V-C0152"), and was adopted at the third JCT-3V meeting. BVSP is conceptually similar to block-based VSP in 3D-AVC. In other words, the basic idea behind Backward Warping (BVSP) is the same as block-based VSP in 3D-AVC. Both BVSP and block-based VSP in 3D-AVC use backward warping and block-based VSP to avoid transmitting motion vector differences and use more accurate motion vectors. However, implementation details may differ due to different platforms.

In some versions of 3D-HEVC, texture-first coding is applied. In texture-first coding, the video coder codes (e.g., encodes or decodes) a texture view component before coding the corresponding depth view component (i.e., a depth view component having the same POC value and view identifier as the texture view component). As a result, the non-base view depth view component is not available for coding the corresponding non-base view texture view component. In other words, when the video coder codes the non-base texture view component, the corresponding non-base depth view component is not available. Therefore, depth information can be estimated and used to perform BVSP.

To estimate the depth information of a block, it is proposed to first derive a disparity vector from neighboring blocks and then use the derived disparity vector to obtain a depth block from a reference view. In 3D-HEVC Test Model 5.1 (i.e., HTM 5.1 Test Model), there is a process for deriving a disparity vector predictor called the NBDV derivation process. Let (dv _x , dv _y ) denote the disparity vector identified from the NBDV derivation process, and the current block position be (block _x , block _y ). The video coder may extract a depth block at (block _x + dv _x , block _y + dv _y ) in the depth image of the reference view. The extracted depth block may have the same size as the current PU. The video coder may then use the extracted depth block to perform inverse warping of the current PU. FIG7 is a conceptual diagram illustrating the derivation of a depth block from a reference view to perform BVSP. FIG7 illustrates the three steps of how a depth block from a reference view is located and then used for BVSP prediction.

If BVSP is enabled in a sequence, the NBDV derivation process for inter-view motion prediction may be changed and the differences are shown in bold in the following paragraphs:

• For each of the temporal neighboring blocks, if the temporal neighboring block uses a disparity motion vector, the disparity motion vector is returned as the disparity vector and further optimized with the methods described elsewhere in this disclosure.

• For each of the spatially neighboring blocks, the following applies:

o For each reference picture list 0 or reference picture list 1, the following applies:

■ If the spatial neighboring block uses a disparity motion vector, the disparity motion vector is returned as the disparity vector and further optimized using the methods described elsewhere in this disclosure.

■ Otherwise, if the spatial neighboring block uses BVSP mode, the associated motion vector may be returned as the disparity vector.The disparity vector may be further optimized in a manner similar to that described elsewhere in this disclosure.

However, the maximum depth value may be selected from all pixels of the corresponding depth block instead of the four corner pixels.

For each of the spatially neighboring blocks, if the spatially neighboring block uses IDV, then the IDV is returned as the disparity vector.The video coder may further optimize the disparity vector using one or more of the methods described elsewhere in this disclosure.

The video coder may process the BVSP mode described above as a special inter-coded mode and the video coder may maintain a flag indicating the use of BVSP mode for each PU. The video coder may add a new merge candidate (BVSP merge candidate) to the merge candidate list instead of signaling the flag in the bitstream and the flag depends on whether the decoded merge candidate index corresponds to a BVSP merge candidate. In some examples, the BVSP merge candidate is defined as follows:

● Reference picture index for each reference picture list: -1

● Motion vector for each reference picture list: optimized disparity vector

In some examples, the insertion position of a BVSP merge candidate depends on spatial neighboring blocks. For example, if five spatial neighboring blocks ( _A0 , _A1 , _B0 , _B1 , or _B2 ) are coded in BVSP mode, i.e., the maintained flag of the neighboring blocks is equal to 1, then the video coder may treat the BVSP merge candidate as the corresponding spatial merge candidate and may insert the BVSP merge candidate into the merge candidate list. The video coder may insert the BVSP merge candidate into the merge candidate list only once. Otherwise, in this example (e.g., when none of the five spatial neighboring blocks are coded in BVSP mode), the video coder may insert the BVSP merge candidate into the merge candidate list immediately before any temporal merge candidate. During the combined bi-predictive merge candidate derivation process, the video coder may check additional conditions to avoid including BVSP merge candidates.

For each BVSP-coded PU, the video coder may further partition the BVSP into several sub-regions of size equal to K×K (where K may be 4 or 2). The size of the BVSP-coded PU may be represented by N×M. For each sub-region, the video coder may derive a separate disparity motion vector. Furthermore, the video coder may predict each sub-region from one block located by the derived disparity motion vector in the inter-view reference picture. In other words, the size of the motion compensation unit for the BVSP-coded PU is set to K×K. In some test conditions, K is set to 4.

Regarding BVSP, the video coder may perform the following disparity motion vector derivation process. For each sub-region (4×4 block) within a PU coded in BVSP mode, the video coder may first locate the corresponding 4×4 depth block in the reference depth view using the optimized disparity vector mentioned above. Second, the video coder may select the maximum value of the sixteen depth pixels in the corresponding depth block. Third, the video coder may convert the maximum value into the horizontal component of the disparity motion vector. The video coder may set the vertical component of the disparity motion vector to 0.

Based on the disparity vector derived from the DoNBDV technique, the video coder can add new motion vector candidates, namely, inter-view predicted motion vector candidates (IPMVC), if available, to AMVP and skip/merge modes. IPMVC, if available, is a temporal motion vector. Since skip mode has the same motion vector derivation process as merge mode, the techniques described in this document can be applied to both merge and skip modes.

For merge/skip mode, the IPMVC can be derived through the following steps. First, the video coder can use the disparity vector to locate the corresponding block of the current block in the reference view of the same access unit (e.g., PU, CU, etc.). Second, if the corresponding block is not intra-coded and inter-view predicted and the reference picture of the corresponding block has a POC value equal to an entry in the same reference picture list as the current block, the video coder can convert the reference index of the corresponding block based on the POC value. In addition, the video coder can derive the IPMVC to specify the prediction direction of the corresponding block, the motion vector of the corresponding block, and the converted reference index.

Section H.8.5.2.1.10 of 3D-HEVC Test Model 4 describes the derivation process of temporal inter-view motion vector candidates. IPMVC may be referred to as a temporal inter-view motion vector candidate because it indicates a position in a temporal reference picture. As described in section H.8.5.2.1.10 of 3D-HEVC Test Model 4, the reference layer luma position (xRef, yRef) is derived as follows:

xRef＝Clip3(0,PicWidthInSamples _L -1,xP+((nPSW-1)>>1)+

((mvDisp[0]+2)>>2))(H-124)

yRef＝Clip3(0,PicHeightInSamples _L -1,yP+((nPSH-1)>>1)+

((mvDisp[1]+2)>>2))(H-125)

In equations H-124 and H-125 above, (xP, yP) represents the coordinates of the top-left luma sample of the current PU relative to the top-left luma sample of the current picture, nPSW and nPSH represent the width and height of the current prediction unit, respectively, refViewIdx represents the reference view order index, and mvDisp represents the disparity vector. The corresponding block is set to cover the PU with luma position (xRef, yRef) in the view component with ViewIdx equal to refViewIdx. In equations H-124 and H-125 above and other equations in this disclosure, the Clip3 function may be defined as:

FIG8 is a conceptual diagram illustrating an example derivation of an IPMVC for merge/skip mode. In other words, FIG8 shows an example of the derivation process for inter-view predicted motion vector candidates. In the example of FIG8 , current PU 50 occurs in view V1 at time instance T1. Reference PU 52 for current PU 50 occurs in a different view than current PU 50 (i.e., view V0), but at the same time instance as current PU (i.e., time instance T1). In the example of FIG8 , reference PU 52 is bidirectionally inter-predicted. Reference PU 52 then has a first motion vector 54 and a second motion vector 56. Motion vector 54 indicates a location in reference picture 58. Reference picture 58 occurs in view V0 and time instance T0. Motion vector 56 indicates a location in reference picture 60. Reference picture 60 occurs in view V0 and time instance T3.

The video coder may generate an IPMVC for inclusion in the merge candidate list of current PU 50 based on the motion information of reference PU 52. The IPMVC may have first motion vector 62 and second motion vector 64. Motion vector 62 matches motion vector 54, and motion vector 64 matches motion vector 56. The video coder generates the IPMVC such that a first reference index of the IPMVC indicates a position in RefPicList0 of current PU 50 of a reference picture (i.e., reference picture 66) that appears in the same time instance as reference picture 58 (i.e., time instance T0). In the example of FIG. 8 , reference picture 66 appears in the first position (i.e., Ref0) in RefPicList0 of current PU 50. Furthermore, the video coder generates the IPMVC such that a second reference index of the IPMVC indicates a position in RefPicList1 of current PU 50 of a reference picture (i.e., reference picture 68) that appears in the same time instance as reference picture 60. Therefore, in the example of FIG. 8 , the RefPicList0 reference index of the IPMVC may be equal to 0. 8 , reference picture 70 appears in the first position (i.e., Ref0) in RefPicList1 of current PU 50, and reference picture 68 appears in the second position (i.e., Ref1) in RefPicList1 of current PU 50. Therefore, the RefPicList1 reference index of the IPMVC may be equal to 1.

In addition to generating an IPMVC and including the IPMVC in the merge candidate list, the video coder may convert the disparity vector of the current PU into an inter-view disparity motion vector (IDMVC) and may include the IDMVC in the merge candidate list for the current PU. In other words, the disparity vector may be converted into an IDMVC, which is added to the merge candidate list at a position different from the IPMVC or to the AMVP candidate list at the same position as the IPMVC when the IDMVC is available. The IPMVC or IDMVC is referred to as an 'inter-view candidate' in this context. In other words, the term "inter-view candidate" may be used to refer to the IPMVC or IDMVC. In some examples, in merge/skip mode, the video coder always inserts the IPMVC (if available) into the merge candidate list before all spatial and temporal merge candidates. In addition, the video coder may insert the IDMVC before the spatial merge candidate derived from _A0 .

As indicated above, the video decoder can derive the disparity vector using the DoNBDV method. With the disparity vector, the merge candidate list construction process in 3D-HEVC can be defined as follows:

1.IPMVC Insertion

The IPMVC is derived by the procedure described above. If the IPMVC is available, it is inserted into the merge list.

2.3D-HEVC Derivation Process for Spatial Merging Candidates and IDMVC Insertion

The motion information of spatially neighboring PUs is checked in the following order: A ₁ , B ₁ , B ₀ , A ₀ or B ₂ . Constrained reduction is performed by the following procedure:

If _A1 and IPMVC have the same motion vector and the same reference index, do not insert _A1 into the candidate list; otherwise insert _A1 into the list.

If _B1 and _A1 /IPMVC have the same motion vector and the same reference index, do not insert _B1 into the candidate list; otherwise insert _B1 into the list.

If _B0 is available, add _B0 to the candidate list. Derive IDMVC by the procedure described above. If IDMVC is available and IDMVC is different from the candidates derived from _A1 and _B1 , insert IDMVC into the candidate list.

If BVSP is enabled for the entire picture or the current slice, insert the BVSP merge candidate into the merge candidate list.

- If A ₀ is available, add A ₀ to the candidate list.

- If B ₂ is available, add B ₂ to the candidate list.

3. Derivation process for temporal merging candidates

Similar to the temporal merge candidate derivation process in HEVC where the motion information of the co-located PU is utilized, however, the target reference picture index of the temporal merge candidate may actually be changed to be fixed to 0. When the target reference index equal to 0 corresponds to a temporal reference picture (in the same view) and the motion vector of the co-located PU points to an inter-view reference picture, the target reference index may be changed to another index corresponding to the first entry of the inter-view reference picture in the reference picture list. Conversely, when the target reference index equal to 0 corresponds to an inter-view reference picture and the motion vector of the co-located PU points to a temporal reference picture, the target reference index may be changed to another index corresponding to the first entry of the temporal reference picture in the reference picture list.

4. Derivation Process for Combining Bidirectional Predictive Merging Candidates in 3D-HEVC

If the total number of candidates derived from the two steps above is less than the maximum number of candidates, the same process as defined in HEVC is performed, except for the designation of l0CandIdx and l1CandIdx. FIG9 is a table indicating an example designation of l0CandIdx and l1CandIdx in 3D-HEVC. The table in FIG9 defines the relationship between combIdx, l0CandIdx, and l1CandIdx. Section 8.5.3.2.3 of HEVC Working Draft 10 defines the example use of l0CandIdx and l1CandIdx when deriving combined bi-predictive merging candidates.

5. Derivation process for zero motion vector merge candidates

- Perform the same procedures as defined in HEVC.

In some versions of the reference software for 3D-HEVC, the total number of candidates in a merge (e.g., MRG) list is at most six, and five_minus_max_num_merge_cand is signaled in the slice header to specify the maximum number of merge candidates subtracted from 6. five_minus_max_num_merge_cand is in the range of 0 to 5, inclusive. The five_minus_max_num_merge_cand syntax element may specify the maximum number of merge MVP candidates supported in a slice subtracted from 5. The maximum number of merge motion vector prediction (MVP) candidates, MaxNumMergeCand, may be calculated as MaxNumMergeCand=5−five_minus_max_num_merge_cand+iv_mv_pred_flag[nuh_layer_id]. The value of five_minus_max_num_merge_cand may be limited such that MaxNumMergeCand is in the range of 0 to (5+iv_mv_pred_flag[nuh_layer_id]), inclusive.

As indicated above, section 8.5.3.2.3 of HEVC Working Draft 10 defines example uses of l0CandIdx and l1CandIdx in deriving combined bi-predictive merge candidates. Section 8.5.3.2.3 of HEVC Working Draft 10 is reproduced below.

Derivation process for combining bidirectional predictive merge candidates

The inputs to this process are:

-Merge candidate list mergeCandList,

- the reference indices refldxLON and refldxL1N for each candidate N in mergeCandList,

-mergeCandList each candidate N prediction list using flags predFlagL0N and predFlagL1N,

-mergeCandList for each candidate N's motion vectors mvL0N and mvL1N,

-The number of elements numCurrMergeCand in mergeCandList,

- The number of elements numOrigMergeCand in mergeCandList after the spatial and temporal merge candidate derivation process.

The output of this process is:

-Merge candidate list mergeCandList,

-The number of elements numCurrMergeCand in mergeCandList,

- the reference index refIdxL0combCand _k and refIdxL1combCand _k of each new candidate combCand _k added to mergeCandList during this call of this process,

- The prediction list of each new candidate combCand _k added to mergeCandList during the call of this process uses the flags predFlagL0combCand _k and predFlagL1combCand _k ,

- The motion vectors mvL0combCand _k and mvL1combCand _k for each new candidate combCand _k added to mergeCandList during the call of this process.

When numOrigMergeCand is greater than 1 and less than MaxNumMergeCand, the variable numInputMergeCand is set equal to numCurrMergeCand, the variable combIdx is set equal to 0, the variable combStop is set equal to FALSE, and the following steps are repeated until combStop is equal to TRUE:

1. Export variables l0CandIdx and l1CandIdx using combIdx as specified in Table 8-6.

2. The following assignment is made, where in the merge candidate list mergeCandList, l0Cand is the candidate at position l0CandIdx, and l1Cand is the candidate at position l1CandIdx:

-l0Cand=mergeCandList[l0CandIdx]

-l1Cand=mergeCandList[l1CandIdx]

3. When all of the following conditions are true:

-predFlagL0l0Cand==1

-predFlagL1l1Cand==1

-(DiffPicOrderCnt(RefPicList0[refIdxL0l0Cand],

RefPicList1[refIdxL1l1Cand])! =0)||

(mvL0l0Cand!=mvL1l1Cand)

Add candidate combCand _k (where k is equal to (numCurrMergeCand - numInputMergeCand)) to the end of mergeCandList, i.e., set mergeCandList[numCurrMergeCand] equal to combCand _k and derive the reference index, prediction list utilization flag and motion vector of combCand _k as follows and increment numCurrMergeCand by 1:

refIdxL0combCand _k =refIdxL0l0Cand (8-113)

refIdxL1combCand _k =refIdxL1l1Cand (8-114)

predFlagL0combCand _k = 1 (8-115)

predFlagL1combCand _k = 1 (8-116)

mvL0combCand _k [0]＝mvL0l0Cand[0] (8-117)

mvL0combCand _k [1]＝mvL0l0Cand[1] (8-118)

mvL1combCand _k [0]＝mvL1l1Cand[0] (8-119)

mvL1combCand _k [1]＝mvL1l1Cand[1] (8-120)

numCurrMergeCand＝numCurrMergeCand+1 (8-121)

4. The variable combIdx is incremented by 1.

5. When combIdx is equal to (numOrigMergeCand*(numOrigMergeCand-1)) or numCurrMergeCand is equal to MaxNumMergeCand, set combStop equal to TRUE.

Motion vector inheritance (MVI) exploits the similarity in motion characteristics between a texture image and its associated depth image. Specifically, a video coder may include MVI candidates in a merge candidate list. For a given PU in a depth image, an MVI candidate reuses the motion vector and reference index of the already coded corresponding texture block (if available). FIG10 is a conceptual diagram illustrating an example derivation of motion vector inheritance candidates for depth coding. FIG10 shows an example of the derivation process for MVI candidates, where the corresponding texture block is selected as a 4×4 block located to the lower right of the center of the current PU.

In some examples, motion vectors with integer precision are used for depth coding while motion vectors with quarter precision are used for texture coding.Thus, the motion vectors of the corresponding texture blocks may be scaled before being used as MVI candidates.

In case of MVI candidate generation, the merge candidate list for the depth view can be constructed as follows:

1.MVI Insertion

The MVI is derived through the procedure described above. If the MVI is available, the video coder may insert the MVI into the merge list.

2.3D-HEVC Derivation Process for Spatial Merging Candidates and IDMVC Insertion

The motion information of spatially neighboring PUs is checked in the following order: A ₁ , B ₁ , B ₀ , A ₀ or B ₂ . The video coder may perform constrained downscaling by the following procedure:

If _A1 and MVI have the same motion vector and the same reference index, the video coder does not insert _A1 into the candidate list.

If B ₁ and A ₁ /MVI have the same motion vector and the same reference index, the video coder does not insert B ₁ into the candidate list.

If _B0 is available, the video coder adds _B0 to the candidate list.

- If A ₀ is available, the video coder adds A ₀ to the candidate list.

If B ₂ is available, the video coder adds B ₂ to the candidate list.

3. Derivation process for temporal merging candidates

Similar to the temporal merge candidate derivation process in HEVC where the motion information of a co-located PU is utilized, however, instead of fixing the target reference picture index to 0, the target reference picture index of the temporal merge candidate may be changed as explained elsewhere in this disclosure with respect to the merge candidate list construction for texture coding in 3D-HEVC.

4. Derivation Process for Combining Bidirectional Predictive Merging Candidates in 3D-HEVC

If the total number of candidates derived from the above two steps is less than the maximum number of candidates, the video coder may perform the same process as defined in HEVC except for the specification of l0CandIdx and l1CandIdx. The relationship among combIdx, l0CandIdx, and l1CandIdx is defined in the table of FIG.

6. Derivation process for zero motion vector merge candidates

- Perform the same procedures as defined in HEVC.

As indicated above, 3D-HEVC provides inter-view residual prediction. Advanced residual prediction (ARP) is a form of inter-view residual prediction. The application of ARP to CUs with a partitioning mode equal to Part_2Nx2N was adopted in the 4th JCT3V meeting as proposed in Zhang et al., "CE4: Advanced residual prediction for multi-view coding" (Joint Collaboration Group on 3D Video Coding Extensions of ITU-T SG 16 WP 3 and ISO/IEC JTC1/SC 29/WG 11, 4th meeting: Incheon, South Korea, April 20-26, 2013, document JCT3V-D0177, available from http://phenix.it-sudparis.eu/jct3v/doc_end_user/documents/4_Incheon/wg11/JCT3V-D0177-v2.zip as of December 17, 2013) (hereinafter referred to as JCT3V-D0177).

FIG11 illustrates an example prediction structure of ARP in multi-view video coding.As shown in FIG11 , a video coder may call the following blocks in current block prediction.

1. Current Block: Curr

2. Reference block in the reference/base view derived from the disparity vector (DV): Base.

3. A block in the same view as the block Curr derived from the (temporal) motion vector (denoted as TMV) of the current block: CurrTRef.

4. A block in the same view as the block Base derived from the temporal motion vector (TMV) of the current block: BaseTRef. This block is identified by the vector of TMV+DV compared to the current block.

The residual predictor is expressed as BaseTRef-Base, where a subtraction operation is applied to each pixel of the represented pixel array. The video decoder can multiply the weighting factor w by the residual predictor. Therefore, the final predictor for the current block can be expressed as: CurrTRef+w*(BaseTRef-Base).

Both the above description and FIG11 are based on the assumption that unidirectional prediction is used. When extended to the case of bidirectional prediction, the above steps are applied to each reference picture list. When an inter-view reference picture (in a different view) is used for the current block in a reference picture list, the residual prediction process is disabled.

The main procedures of the proposed ARP at the decoder side can be described as follows. First, the video decoder can obtain the disparity vector pointing to the target reference view as specified in 3D-HEVC Working Draft 4. Then, in the picture of the reference view within the same access unit as the current picture, the video decoder can use the disparity vector to locate the corresponding block. Next, the video decoder can reuse the motion information of the current block to derive the motion information of the reference block. The video decoder can then apply motion compensation to the corresponding block based on the same motion vector of the current block and the derived reference picture in the reference view of the reference block to derive the residual block. Figure 12 shows the relationship among the current block, the corresponding block, and the motion compensated block. In other words, Figure 12 is a conceptual diagram illustrating an example relationship among the current block, the reference block, and the motion compensated block. A reference picture in the reference view ( _V0 ) with the same POC (Picture Order Count) value as the reference picture of the current view ( _Vm ) is selected as the reference picture of the corresponding block. Next, the video decoder can apply a weighting factor to the residual block to determine a weighted residual block. The video coder may add the values of the weighted residual block to the prediction samples.

Three weighting factors are used in ARP: 0, 0.5, and 1. Video encoder 20 may select the weighting factor that results in the minimum rate-distortion cost for the current CU as the final weighting factor. Video encoder 20 may signal the corresponding weighting factor index (0, 1, and 2 corresponding to weighting factors of 0, 1, and 0.5, respectively) in the bitstream at the CU level. All PU predictions in a CU may share the same weighting factor. When the weighting factor is equal to 0, the video coder does not use ARP for the current CU.

In Zhang et al., "3D-CE4: Advanced residual prediction for multiview coding" (Joint Collaborative Group on 3D Video Coding Extension Development of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 3rd Meeting: Geneva, Switzerland, January 17-23, 2013, document JCT3V-C0049, available from http://phenix.int-evry.fr/jct3v/doc_end_user/documents/3_Geneva/wg11/JCT3V-C0049-v2.zip as of August 30, 2013) (hereinafter referred to as JCT3V-C0049), the reference pictures of a PU coded with a non-zero weighting factor may differ between blocks. Therefore, the video coder may need to access different pictures from the reference view to generate the motion compensation block (i.e., BaseTRef in the example of FIG. 11 ) for the corresponding block. When the weighting factor is not equal to 0, the video coder may scale the decoded motion vector of the current PU toward a fixed picture before performing motion compensation for the residual generation process. In JCT3V-D0177, if each reference picture list is from the same view, a fixed picture is defined as the first reference picture in each reference picture list. When the decoded motion vector does not point to a fixed picture, the video coder may first scale the decoded motion vector and then use the scaled motion vector to identify CurrTRef and BaseTRef. This reference picture used for ARP may be referred to as a target ARP reference picture.

In JCT3V-C0049, the video coder may apply a bilinear filter during the interpolation process of the corresponding block and the prediction block of the corresponding block. However, for the prediction block of the current PU in a non-base view, the video coder may apply a conventional 8/4 tap filter. When applying ARP, JCT3V-D0177 always proposes to utilize bilinearity regardless of whether the block is in a base view or a non-base view.

In ARP, the video coder can identify the reference view by the view order index returned from the NBDV derivation process. In some designs of ARP, when a PU's reference pictures in a reference picture list are from a different view than the current view, ARP is disabled for this reference picture list.

In U.S. Provisional Patent Application No. 61/840,400, filed on June 28, 2013, and U.S. Provisional Patent Application No. 61/847,942, filed on July 18, 2013 (each of which is incorporated by reference in its entirety), when coding a depth picture, a disparity vector is converted from an estimated depth value of neighboring samples of the current block. Furthermore, more merge candidates can be derived, for example, by accessing a reference block of a base view identified by the disparity vector.

In 3D-HEVC, the video coder can identify the reference 4×4 block in two steps. The first step is to identify the pixel using the disparity motion vector. The second step is to obtain the 4×4 block (using the unique motion information set corresponding to RefPicList0 or RefPicList1, respectively) and use the motion information to create a merge candidate.

A pixel (xRef, yRef) in the reference view can be identified as follows:

xRef＝Clip3(0,PicWidthInSamples _L -1,xP+((nPSW-1)>>1)+

((mvDisp[0]+2)>>2))(H-124)

yRef＝Clip3(0,PicHeightInSamples _L -1,yP+((nPSH-1)>>1)+

((mvDisp[1]+2)>>2))(H-125)

Where (xP, yP) are the coordinates of the top left sample of the current PU, mvDisp is the disparity vector and nPSWxnPSH is the size of the current PU, and _{PicWidthInSamplesL} and _{PicHeightInSamplesL} define the resolution of the picture in the reference view (same as the current view).

An et al., “3D-CE3.h related: Inter-view motion prediction at the sub-PU level” (Joint Collaboration Group on 3D Video Coding Extensions of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 5th Meeting: Vienna, Austria, July 27–August 2, 2013, document JCT3V-E0184 (hereinafter “JCT3V-E0184”), available as of December 17, 2013 at http://phenix.it-sudparis.eu/jct2/doc_end_user/documents/5_Vienna/wg11/JCT3V-E0184-v2.zip) proposes a sub-PU level inter-view motion prediction method for temporal inter-view merging candidates (i.e., candidates derived from reference blocks in a reference view). The basic concepts of inter-view motion prediction are described elsewhere in this disclosure. In the basic concept of inter-view motion prediction, only the motion information of the reference block is used for the current PU in the dependent view. However, the current PU may correspond to a reference area in the reference view (having the same size as the current PU identified by the disparity vector of the current PU), and the reference area may have sufficient motion information. Therefore, as shown in FIG13 , a sub-PU-level inter-view motion prediction (SPIVMP) method is proposed. In other words, FIG13 is a conceptual diagram illustrating an example of sub-PU inter-view motion prediction.

Temporal inter-view merge candidates may be derived as follows: In the derivation process of temporal inter-view merge candidates, the assigned sub-PU size may be expressed as N×N. Different sub-PU block sizes may be applied, such as 4×4, 8×8, and 16×16.

In the derivation process of temporal inter-view merge candidates, the video coder may first divide the current PU into multiple sub-PUs, each of which has a smaller size than the current PU. The size of the current PU may be represented by nPSW×nPSH. The size of the sub-PU may be represented by nPSWsub×nPSHSub. nPSWsub and nPSHsub may be related to nPSW and nSPH as shown in the following equations.

nPSWsub＝min(N,nPSW)

nPSHSub＝min(N,nPSH)

Additionally, for each reference picture list, the video coder may set the default motion vector tmvLX to (0, 0) and may set the reference index refLX to -1 (where X is 0 or 1).

Furthermore, when determining a temporal inter-view merge candidate, the video coder may apply the following actions for each sub-PU in raster scan order. First, the video coder may add the disparity vector to the middle position of the current sub-PU (whose top-left sample position is (xPSub, yPSub)) to obtain a reference sample position (xRefSub, yRefSub). The video coder may determine (xRefSub, yRefSub) using the following equation:

xRefSub＝Clip3(0,PicWidthInSamplesL-1,xPSub+nPSWsub/2+

((mvDisp[0]+2)>>2))

yRefSub＝Clip3(0,PicHeightInSamplesL-1,yPSub+nPSHSub/2+

((mvDisp[1]+2)>>2))

The video coder may use the block in the reference view that overlays (xRefSub, yRefSub) as a reference block for the current sub-PU.

If the identified reference block is coded using temporal motion vectors and if both refL0 and refL1 are equal to -1 and the current sub-PU is not the first in raster scan order, the motion information of the reference block is inherited by all previous sub-PUs. Furthermore, if the identified reference block is coded using temporal motion vectors, the associated motion parameters may be used as the motion parameters of the current sub-PU. Additionally, if the identified reference block is coded using temporal motion vectors, the video coder may update tmvLX and refLX to the motion information of the current sub-PU. Otherwise, if the reference block is intra-coded, the video coder may set the motion information of the current sub-PU to tmvLX and refLX.

The sub-PU motion prediction method proposed in JCT3V-E0184 has one or more problems. For example, when the corresponding block of a sub-PU in the reference view is intra-coded (i.e., its motion information is not available), the motion information of the most recent sub-PU in raster scan order is copied to the current sub-PU. Therefore, if the corresponding blocks of the first N sub-PUs in raster scan order are intra-coded and the corresponding block of the (N+1)th sub-PU is inter-coded, the relevant motion information set to the (N+1)th sub-PU is copied to the first N sub-PUs, which causes additional complexity and decoding delay.

One or more embodiments of the present disclosure relate to inter-view motion prediction. For example, one or more embodiments of the present disclosure may be applicable in the context of when a merge index indicates inter-view motion prediction.

For example, in one example, when the video coder uses inter-view motion prediction in a sub-PU manner, if the motion information of the current sub-PU is not available, the video coder may copy the motion information from a default motion vector and a reference index. For example, if the motion information of the current sub-PU is not available, the video coder may copy the motion information of the current sub-PU from a default motion vector and a default reference index. In this example, the default motion parameters are the same for each sub-PU in the plurality of sub-PUs, regardless of whether there is a subsequent sub-PU with a reference block coded using motion compensated prediction.

In some examples, a video coder (e.g., video encoder 20 or video decoder 30) may divide a current PU into a plurality of sub-PUs. The current PU is in a current picture. In addition, the video coder may determine default motion parameters. The default motion parameters may include one or more default motion vectors and one or more default reference indexes. In addition, the video coder may process the sub-PUs from the plurality of sub-PUs in a particular order. For each respective sub-PU from the plurality of sub-PUs, the video coder may determine a reference block for the respective sub-PU.

In some examples, the reference picture may be in a different view than the current picture, and the video coder may determine the reference sample location in the reference picture based on the disparity vector of the current PU. In such examples, the reference block of the corresponding sub-PU may cover the reference sample location. In other examples, the current picture is a depth view component and the reference picture is a texture view component in the same view and access unit as the current picture. In such examples, the video coder may determine that the reference block of the corresponding sub-PU is a PU of the reference picture that is co-located with the corresponding sub-PU.

Furthermore, for each corresponding sub-PU from the plurality of sub-PUs (or a subset of the plurality of sub-PUs), if the reference block of the corresponding sub-PU is coded using motion compensated prediction, the video coder may set the motion parameters of the corresponding sub-PU based on the motion parameters of the reference block of the corresponding sub-PU. On the other hand, if the reference block of the corresponding sub-PU is not coded using motion compensated prediction, the video coder may set the motion parameters of the corresponding sub-PU to default motion parameters.

According to one or more examples of the present disclosure, if the reference block of a corresponding sub-PU is not coded using motion compensated prediction, then in response to a subsequent determination that the reference block of any later sub-PU in the coding order is coded using motion compensated prediction, the motion parameters of the corresponding sub-PU are not set. Therefore, in the event that the reference block of at least one of the sub-PUs is not coded using motion compensated prediction, the video coder may not need to scan forward to find a sub-PU whose corresponding reference block is coded using motion compensated prediction. Similarly, the video coder may not need to delay determining the motion parameters of the corresponding sub-PU until the video coder encounters a PU whose corresponding reference block is coded using motion compensated prediction during processing of the sub-PU. Advantageously, this can reduce complexity and coding latency.

The video coder may include a candidate in a candidate list for the current PU, where the candidate is based on motion parameters of multiple sub-PUs. In some examples, the candidate list is a merge candidate list. Furthermore, if the video coder is a video encoder (e.g., video encoder 20), the video encoder may signal a syntax element (e.g., merge_idx) in the bitstream indicating the selected candidate in the candidate list. If the video coder is a video decoder (e.g., video decoder 30), the video decoder may obtain the syntax element (e.g., merge_idx) indicating the selected candidate in the candidate list from the bitstream. The video decoder may use the motion parameters of the selected candidate to reconstruct the predictive block of the current PU.

At least some of the techniques described in this disclosure may be implemented alone or in combination with each other.

FIG14 is a block diagram illustrating an example video encoder 20 that may implement the techniques of this disclosure. FIG14 is provided for explanation purposes and should not be construed as limiting the techniques to those generally illustrated and described in this disclosure. For explanation purposes, this disclosure describes video encoder 20 in the context of HEVC coding. However, the techniques of this disclosure may be applicable to other coding standards or methods.

14 , video encoder 20 includes a prediction processing unit 100, a residual generation unit 102, a transform processing unit 104, a quantization unit 106, an inverse quantization unit 108, an inverse transform processing unit 110, a reconstruction unit 112, a filter unit 114, a decoded picture buffer 116, and an entropy encoding unit 118. Prediction processing unit 100 includes an inter-prediction processing unit 120 and an intra-prediction processing unit 126. Inter-prediction processing unit 120 includes a motion estimation unit 122 and a motion compensation unit 124. In other examples, video encoder 20 may include more, fewer, or different functional components.

Video encoder 20 may receive video data. Video encoder 20 may encode each CTU in a slice of a picture of the video data. Each of the CTUs may be associated with an equally sized luma coding tree block (CTB) and a corresponding CTB of the picture. As part of encoding the CTU, prediction processing unit 100 may perform quadtree partitioning to divide the CTBs of the CTU into progressively smaller blocks. The smaller blocks may be coding blocks of a CU. For example, prediction processing unit 100 may partition the CTB associated with the CTU into four equally sized sub-blocks, partition one or more of the sub-blocks into four equally sized sub-sub-blocks, and so on.

Video encoder 20 may encode a CU of a CTU to generate an encoded representation of the CU (i.e., a coded CU). As part of encoding a CU, prediction processing unit 100 may partition the coding blocks associated with the CU among one or more PUs of the CU. Thus, each PU may be associated with a luma prediction block and a corresponding chroma prediction block. Video encoder 20 and video decoder 30 may support PUs of various sizes. As indicated above, the size of a CU may refer to the size of the luma coding block of the CU and the size of a PU may refer to the size of the luma prediction block of the PU. Assuming the size of a particular CU is 2N×2N, video encoder 20 and video decoder 30 may support PU sizes of 2N×2N or N×N for intra-prediction, and symmetric PU sizes of 2N×2N, 2N×N, N×2N, N×N, or similar sizes for inter-prediction. Video encoder 20 and video decoder 30 may also support asymmetric partitioning of PU sizes of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter-prediction.

Inter-prediction processing unit 120 may generate predictive data for each PU of a CU by performing inter prediction on the PU. The predictive data for the PU may include the predictive block of the PU and the motion information of the PU. Inter-prediction processing unit 120 may perform different operations on a PU of a CU depending on whether the PU is in an I slice, a P slice, or a B slice. In an I slice, all PUs are intra predicted. Therefore, if the PU is in an I slice, inter-prediction processing unit 120 does not perform inter prediction on the PU.

If the PU is in a P slice, motion estimation unit 122 may search a reference picture list (e.g., "RefPicList0") for the PU's reference region. The PU's reference region may be a region within a reference picture that contains samples that most closely correspond to the PU's prediction block. Motion estimation unit 122 may generate a reference index that indicates the position of the reference picture in RefPicList0 that contains the PU's reference region. In addition, motion estimation unit 122 may generate a motion vector that indicates the spatial displacement between the PU's coding block and a reference position associated with the reference region. For example, a motion vector may be a two-dimensional vector that provides an offset from coordinates in the current picture to coordinates in a reference picture. Motion estimation unit 122 may output the reference index and motion vector as the motion information for the PU. Motion compensation unit 124 may generate the PU's predictive block based on actual or interpolated samples at the reference position indicated by the PU's motion vector.

If the PU is in a B slice, motion estimation unit 122 may perform uni-prediction or bi-prediction on the PU. To perform uni-prediction on the PU, motion estimation unit 122 may search the reference picture of RefPicList0 or a second reference picture list ("RefPicList1") for the reference region of the PU. Motion estimation unit 122 may output, as the motion information of the PU, a reference index indicating a position in RefPicList0 or RefPicList1 of the reference picture containing the reference region, a motion vector indicating a spatial displacement between the prediction block of the PU and the reference position associated with the reference region, and one or more prediction direction indicators indicating whether the reference picture is in RefPicList0 or RefPicList1. Motion compensation unit 124 may generate a predictive block for the PU based at least in part on actual or interpolated samples at the reference position indicated by the motion vector of the PU.

To perform bidirectional inter prediction for a PU, motion estimation unit 122 may search a reference picture in RefPicList0 for the PU's reference region and may also search a reference picture in RefPicList1 for another reference region of the PU. Motion estimation unit 122 may generate a reference index that indicates the position of the reference picture containing the reference region in RefPicList0 and RefPicList1. In addition, motion estimation unit 122 may generate a motion vector that indicates the spatial displacement between the reference location associated with the reference region and the prediction block of the PU. The motion information of the PU may include the reference index and motion vector of the PU. Motion compensation unit 124 may generate a predictive block for the PU based at least in part on actual or interpolated samples at the reference location indicated by the motion vector of the PU.

In some examples, motion estimation unit 122 may generate a merge candidate list for a PU. As part of generating the merge candidate list, motion estimation unit 122 may determine IPMVC and/or texture merge candidates. When determining the IPMVC and/or texture merge candidates, motion estimation unit 122 may partition the PU into sub-PUs and process the sub-PUs according to a specific order to determine motion parameters for the sub-PUs. According to one or more techniques of this disclosure, if a reference block of a corresponding sub-PU is not coded using motion compensated prediction, motion estimation unit 122 does not set the motion parameters for the corresponding sub-PU in response to a subsequent determination to use motion compensated prediction to code the reference blocks of any later sub-PU in the specific order. In fact, if the reference block of the corresponding sub-PU is not coded using motion compensated prediction, motion estimation unit 122 may set the motion parameters of the corresponding sub-PU to default motion parameters. If an IPMVC or texture merge candidate is the selected merge candidate in the merge candidate list, motion compensation unit 124 may determine the predictive blocks for the corresponding PU based on the motion parameters specified by the IPMVC or texture merge candidate.

Intra-prediction processing unit 126 may generate predictive data for a PU by performing intra prediction on the PU. The predictive data for the PU may include the predictive blocks and various syntax elements for the PU. Intra-prediction processing unit 126 may perform intra prediction on PUs in I slices, P slices, and B slices.

To perform intra prediction on a PU, intra-prediction processing unit 126 may use multiple intra-prediction modes to generate multiple sets of predictive blocks for the PU. When performing intra prediction using a particular intra-prediction mode, intra-prediction processing unit 126 may use a particular set of samples from neighboring blocks to generate the predictive blocks for the PU. Assuming a left-to-right, top-to-bottom coding order for the PU, CU, and CTU, the neighboring blocks may be above, to the right, to the left, or to the left of the prediction block of the PU. Intra-prediction processing unit 126 may use various numbers of intra-prediction modes, such as 33 directional intra-prediction modes. In some examples, the number of intra-prediction modes may depend on the size of the prediction block of the PU.

Prediction processing unit 100 may select predictive data for PUs of a CU from among the predictive data for the PUs generated by inter-prediction processing unit 120 or the predictive data for the PUs generated by intra-prediction processing unit 126. In some examples, prediction processing unit 100 selects predictive data for the PUs of the CU based on rate/distortion metrics of the predictive data sets. The predictive block of the selected predictive data may be referred to herein as a selected predictive block.

Residual generation unit 102 may generate a luma, Cb, and Cr residual block for a CU based on the luma, Cb, and Cr coding blocks of the CU and the selected predictive luma, Cb, and Cr blocks of the PUs of the CU. For example, residual generation unit 102 may generate the residual block for the CU such that each sample in the residual block has a value equal to the difference between a sample in the coding block of the CU and a corresponding sample in the corresponding selected predictive sample block of the PUs of the CU.

Transform processing unit 104 may perform quadtree partitioning to partition the residual blocks of a CU into transform blocks associated with the TUs of the CU. Thus, a TU may be associated with a luma transform block and two corresponding chroma transform blocks. The sizes and positions of the luma transform blocks and chroma transform blocks of a TU of a CU may or may not be based on the sizes and positions of the prediction blocks of the PUs of the CU.

Transform processing unit 104 may generate a transform coefficient block for each TU of a CU by applying one or more transforms to the transform blocks of the TU. Transform processing unit 104 may apply various transforms to the transform blocks associated with the TU. For example, transform processing unit 104 may apply a discrete cosine transform (DCT), a directional transform, or a conceptually similar transform to the transform blocks. In some examples, transform processing unit 104 does not apply a transform to the transform blocks. In such examples, the transform blocks may be processed as transform coefficient blocks.

Quantization unit 106 may quantize the transform coefficients in a coefficient block. The quantization process may reduce the bit depth associated with some or all of the transform coefficients. For example, an n-bit transform coefficient may be rounded to an m-bit transform coefficient during quantization, where n is greater than m. Quantization unit 106 may quantize a coefficient block associated with a TU of a CU based on a quantization parameter (QP) value associated with the CU. Video encoder 20 may adjust the degree of quantization applied to the coefficient block associated with the CU by adjusting the QP value associated with the CU. Quantization may introduce information loss, and thus, the quantized transform coefficients may have lower precision than the original transform coefficients.

Inverse quantization unit 108 and inverse transform processing unit 110 may apply inverse quantization and inverse transform, respectively, to the coefficient block to reconstruct a residual block from the coefficient block. Reconstruction unit 112 may add the reconstructed residual block to corresponding samples from one or more predictive blocks generated by prediction processing unit 100 to produce a reconstructed transform block associated with the TU. By reconstructing the transform blocks for each TU of a CU in this manner, video encoder 20 can reconstruct the coding blocks of the CU.

Filter unit 114 may perform one or more deblocking operations to reduce blocking artifacts in the coding blocks associated with the CU. Decoded picture buffer 116 may store the reconstructed coding blocks after filter unit 114 performs the one or more deblocking operations on the reconstructed coding blocks. Inter-prediction processing unit 120 may use a reference picture containing the reconstructed coding blocks to perform inter prediction on PUs of other pictures. Additionally, intra-prediction processing unit 126 may use the reconstructed coding blocks in decoded picture buffer 116 to perform intra prediction on other PUs in the same picture as the CU.

Entropy encoding unit 118 may receive data from other functional components of video encoder 20. For example, entropy encoding unit 118 may receive coefficient blocks from quantization unit 106 and may receive syntax elements from prediction processing unit 100. Entropy encoding unit 118 may perform one or more entropy encoding operations on the data to generate entropy-encoded data. For example, entropy encoding unit 118 may perform a context-adaptive variable length coding (CAVLC) operation, a CABAC operation, a variable-to-variable (V2V) length coding operation, a syntax-based context-adaptive binary arithmetic coding (SBAC) operation, a probability interval partitioning entropy (PIPE) coding operation, an exponential Golomb coding operation, or another type of entropy encoding operation on the data. Video encoder 20 may output a bitstream including the entropy-encoded data generated by entropy encoding unit 118.

FIG15 is a block diagram illustrating an example video decoder 30 that may implement the techniques of this disclosure. FIG15 is provided for explanation purposes and does not limit the techniques to those generally illustrated and described in this disclosure. For explanation purposes, this disclosure describes video decoder 30 in the context of HEVC coding. However, the techniques of this disclosure may be applicable to other coding standards or methods.

15 , video decoder 30 includes entropy decoding unit 150, prediction processing unit 152, inverse quantization unit 154, inverse transform processing unit 156, reconstruction unit 158, filter unit 160, and decoded picture buffer 162. Prediction processing unit 152 includes motion compensation unit 164 and intra-prediction processing unit 166. In other examples, video decoder 30 may include more, fewer, or different functional components.

Coded picture buffer (CPB) 151 may receive and store encoded video data (e.g., NAL units) of a bitstream. Entropy decoding unit 150 may receive NAL units from CPB 151 and parse the NAL units to obtain syntax elements from the bitstream. Entropy decoding unit 150 may entropy decode the entropy-encoded syntax elements in the NAL units. Prediction processing unit 152, inverse quantization unit 154, inverse transform processing unit 156, reconstruction unit 158, and filter unit 160 may generate decoded video data based on the syntax elements extracted from the bitstream.

The NAL units of the bitstream may include coded slice NAL units. As part of decoding the bitstream, entropy decoding unit 150 may extract syntax elements from the coded slice NAL units and entropy decode the syntax elements. Each of the coded slices may include a slice header and slice data. The slice header may contain syntax elements related to the slice.

In addition to obtaining syntax elements from the bitstream, video decoder 30 may perform a decoding operation on a CU. By performing a decoding operation on a CU, video decoder 30 may reconstruct the coding blocks of the CU.

As part of performing a decoding operation on a CU, inverse quantization unit 154 may inverse quantize (i.e., dequantize) coefficient blocks associated with the TUs of the CU. Inverse quantization unit 154 may use the QP value associated with the CU of the TU to determine the degree of quantization and, similarly, the degree of inverse quantization that inverse quantization unit 154 will apply. That is, the compression ratio, i.e., the ratio of the number of bits used to represent the original sequence and the compressed sequence, may be controlled by adjusting the value of the QP used when quantizing transform coefficients. The compression ratio may also depend on the entropy coding method utilized.

After inverse quantization unit 154 inverse quantizes the coefficient block, inverse transform processing unit 156 may apply one or more inverse transforms to the coefficient block to generate a residual block associated with the TU. For example, inverse transform processing unit 156 may apply an inverse DCT, an inverse integer transform, an inverse Karhunen-Loeve transform (KLT), an inverse rotational transform, an inverse directional transform, or another inverse transform to the coefficient block.

If the PU is encoded using intra prediction, intra-prediction processing unit 166 may perform intra prediction to generate predictive blocks for the PU. Intra-prediction processing unit 166 may use an intra prediction mode to generate predictive luma blocks, Cb blocks, and Cr blocks for the PU based on prediction blocks of spatially-neighboring PUs. Intra-prediction processing unit 166 may determine the intra prediction mode for the PU based on one or more syntax elements decoded from the bitstream.

Prediction processing unit 152 may construct a first reference picture list (RefPicList0) and a second reference picture list (RefPicList1) based on syntax elements extracted from the bitstream. Furthermore, if the PU is encoded using inter-frame prediction, entropy decoding unit 150 may obtain the motion information of the PU. Motion compensation unit 164 may determine one or more reference regions for the PU based on the motion information of the PU. Motion compensation unit 164 may generate predictive luma blocks, Cb blocks, and Cr blocks for the PU based on samples at one or more reference blocks of the PU.

In some examples, motion compensation unit 164 may generate a merge candidate list for a PU. As part of generating the merge candidate list, motion compensation unit 164 may determine IPMVC and/or texture merge candidates. When determining the IPMVC and/or texture merge candidates, motion compensation unit 164 may partition the PU into sub-PUs and process the sub-PUs according to a specific order to determine motion parameters for each of the sub-PUs. According to one or more techniques of this disclosure, if a reference block of a corresponding sub-PU is not coded using motion compensated prediction, motion compensation unit 164 does not set the motion parameters of the corresponding sub-PU in response to a subsequent determination to use motion compensated prediction to code the reference blocks of any later sub-PU in the specific order. In fact, if the reference block of the corresponding sub-PU is not coded using motion compensated prediction, motion compensation unit 164 may set the motion parameters of the corresponding sub-PU to default motion parameters. If an IPMVC or texture merge candidate is the selected merge candidate in the merge candidate list, motion compensation unit 164 may determine the predictive block of the corresponding PU based on the motion parameters specified by the IPMVC or texture merge candidate.

Reconstruction unit 158 may use residual values (i.e., intra prediction data or inter prediction data, as applicable) from the luma, Cb, and Cr transform blocks associated with the TUs of the CU and the predictive luma, Cb, and Cr blocks of the PUs of the CU to reconstruct the luma, Cb, and Cr coding blocks of the CU. For example, reconstruction unit 158 may add samples of the luma, Cb, and Cr transform blocks to corresponding samples of the predictive luma, Cb, and Cr blocks to reconstruct the luma, Cb, and Cr coding blocks of the CU.

Filter unit 160 may perform a deblocking operation to reduce blocking artifacts associated with the luma, Cb, and Cr coding blocks of a CU. Video decoder 30 may store the luma, Cb, and Cr coding blocks of the CU in decoded picture buffer 162. Decoded picture buffer 162 may provide reference pictures for subsequent motion compensation, intra prediction, and presentation on a display device (e.g., display device 32 of FIG. 1 ). For example, video decoder 30 may perform intra prediction or inter prediction operations on PUs of other CUs based on the luma, Cb, and Cr blocks in decoded picture buffer 162. In this manner, video decoder 30 may extract transform coefficient levels for a large number of luma coefficient blocks from the bitstream, inverse quantize the transform coefficient levels, apply a transform to the transform coefficient levels to generate transform blocks, generate coded blocks based at least in part on the transform blocks, and output the coded blocks for display.

The following sections provide example decoding process changes for 3D-HEVC (which is publicly available). In the derivation process for sub-PU temporal inter-view motion vector candidates, the video coder may first generate a PU-level inter-view predicted motion vector candidate. If the center reference sub-PU (i.e., the center sub-PU in an inter-view reference block) is coded in inter-prediction mode and its reference picture in reference picture list X has the same POC value as an entry in reference picture list X of the current slice (for X=0 or 1), its motion vector and reference picture are used as the PU-level predicted motion vector candidate. Otherwise, the video coder may use zero motion with a reference picture index equal to 0 for reference picture list 0 and reference picture list 1 (if the current slice is a B slice) as the PU-level predicted motion vector candidate. The video coder may then use the PU-level predicted motion vector candidate as the motion of a sub-PU whose corresponding reference block is coded in intra-prediction mode or in inter-prediction mode, but whose reference picture is not included in the reference picture list of the current slice.

Examples of this disclosure may modify the sub-PU level inter-view motion prediction process (or the derivation process for sub-prediction block temporal inter-view motion vector candidates) defined in 3D-HEVC Draft Text 2 (i.e., document JCT3V-F1001v2). According to one or more examples of this disclosure, text added to 3D-HEVC Draft Text 2 is underlined and text deleted from 3D-HEVC Draft Text 2 is italicized and enclosed in double square brackets.

Decoding process

H.8.5.3.2.16 Derivation process for sub-prediction block temporal inter-view motion vector candidates

When iv_mv_pred_flag[nuh_layer_id] is equal to 0, this process is not called.

The inputs to this process are:

- the luma position (xPb, yPb) of the top-left luma sample of the current prediction unit relative to the top-left luma sample of the current picture,

- variables nPbW and nPbH specifying the width and height of the current prediction unit respectively,

- Reference view index refViewIdx.

- disparity vector mvDisp,

The output of this process is:

- a flag availableFlagLXInterView specifying whether a temporal inter-view motion vector candidate is available (where X is in the range of 0 to 1 inclusive),

- Temporal inter-view motion vector candidate mvLXInterView (where X is in the range of 0 to 1 inclusive).

- a reference index refIdxLXInterView (where X is in the range of 0 to 1, inclusive) of a reference picture in the reference picture list RefPicListLX,

For X in the range 0 to 1 (inclusive), the following applies:

-Set the flag availableFlagLXInterView to 0.

-Set the motion vector mvLXInterView equal to (0,0).

- Set the reference index refldxLXInterView equal to -1.

The variables nSbW and nSbH are derived as:

nSbW＝Min(nPbW,SubPbSize[nuh_layer_id])(H-173)

nSbH＝Min(nPbH,SubPbSize[nuh_layer_id])(H-174)

The variable ivRefPic is set equal to the picture in the current access unit with ViewIdx equal to refViewIdx

The following applies to derive the variables flag centerPredFlagLX, motion vector centerMvLX and reference index centerRefIdxLX.

-The variable centerAvailableFlag is set equal to 0.

- For X in the range 0 to 1 (inclusive), the following applies:

- Set the flag centerPredFlagLX to 0.

- Set the motion vector centerMvLX equal to (0,0).

- Set the reference index centerRefIdxLX equal to -1.

- Derive the reference layer luminance position (xRef, yRef) by

xRef＝Clip3(0,PicWidthInSamplesL-1,

xPb+(nPbW/nSbW/2)*nSbW+nSbW/2+((mvDisp[0]+2)>>2))

(H-175)

yRef＝Clip3(0,PicHeightInSamplesL-1,

yPb+(nPbH/nSbH/2)*nSbH+nSbH/2+((mvDisp[1]+2)>>2))

(H-176)

The variable ivRefPb specifies the luma prediction block covering the position given by (xRef, yRef) inside the inter-view reference picture specified by ivRefPic .

- Relative to the top-left luma sample of the inter-view reference picture specified by ivRefPic, the luma position (xIvRefPb, yIvRefPb) is set equal to the top-left sample of the inter-view reference luma prediction block specified by ivRefPb.

When ivRefPb is not coded in intra prediction mode, for X in the range of 0 to 1 (inclusive), the following applies:

- When X is equal to 0 or the current slice is a B slice, for Y in the range of X to (1-X), inclusive, the following applies:

- The variables refPicListLYIvRef, predFlagLYIvRef, mvLYIvRef, and refIdxLYIvRef are set equal to RefPicListLY, PredFlagLY, MvLY, and RefIdxLY of the picture ivRefPic, respectively.

- When predFlagLYIvRef[xIvRefPb][yIvRefPb] is equal to 1, for each i from 0 to num_ref_idx_lX_active_minus1 (inclusive), the following applies:

-When PicOrderCnt(refPicListLYIvRef[refIdxLYIvRef[xIvRefPb][yIvRefPb]]) is equal to PicOrderCnt(RefPicListLX[i]) and centerPredFlagLX is equal to 0, the following applies.

centerMvLX＝mvLYIvRef[xIvRefPb][yIvRefPb](H-177)

centerRefIdxLX＝i(H-178)

centerPredFlagLX=1(H-179)

centerAvailableFlag=1(H-180)

If centerAvailableFlag is equal to 0 and ivRefPic is not an I-slice, then for X in the range of 0 to 1 (inclusive), the following applies:

centerMvLX＝(0,0)(H-181)

centerRefIdxLX=0(H-182)

centerPredFlagLX=1(H-183)

For X in the range 0 to 1 (inclusive), the following applies:

-Set the flag availableFlagLXInterView equal to centerPredFlagLX.

- Set the motion vector mvLXInterView equal to centerMvLX.

- Set the reference index refldxLXInterView equal to centerRefldxLX.

For yBlk in the range of 0 to (nPbH/nSbH-1), inclusive, and for xBlk in the range of 0 to (nPbW/nSbW-1), inclusive, the following applies:

-Set the variable curAvailableFlag to 0.

- For X in the range 0 to 1 (inclusive), the following applies:

- Set the flag spPredFlagL1[xBlk][yBlk] to 0.

- Set the motion vector spMvLX equal to (0,0).

- Set the reference index spRefIdxLX[xBlk][yBlk] equal to -1.

- Derive the reference layer luminance position (xRef, yRef) by

xRef＝Clip3(0,PicWidthInSamplesL-1,xPb+xBlk*nSbW+nSbW/2+((mvDisp[0]+2)>>2))(H-184[[175]])

- The variable ivRefPb specifies the luma prediction block covering the position given by (xRef, yRef) inside the inter-view reference picture specified by ivRefPic.

- Relative to the top-left luma sample of the inter-view reference picture specified by ivRefPic, the luma position (xIvRefPb, yIvRefPb) is set equal to the top-left sample of the inter-view reference luma prediction block specified by ivRefPb.

When ivRefPb is not coded in intra prediction mode, for X in the range of 0 to 1 (inclusive), the following applies:

- When X is equal to 0 or the current slice is a B slice, for Y in the range of X to (1-X), inclusive, the following applies:

- The variables refPicListLYIvRef, predFlagLYIvRef[x][y], mvLYIvRef[x][y], and refIdxLYIvRef[x][y] are set equal to RefPicListLY, PredFlagLY[x][y], MvLY[x][y], and RefIdxLY[x][y] of the picture ivRefPic, respectively.

- When predFlagLYIvRef[xIvRefPb][yIvRefPb] is equal to 1, for each i from 0 to num_ref_idx_1X_active_minus1 (inclusive), the following applies:

-When PicOrderCnt(refPicListLYIvRef[refIdxLYIvRef[xIvRefPb][yIvRefPb]]) is equal to PicOrderCnt(RefPicListLX[i]) and spPredFlagLX[xBlk][yBlk] is equal to 0, the following applies.

spMvLX[xBlk][yBlk]=mvLYIvRef[xIvRefPb][yIvRefPb](H-186[[177]])

spRefIdxLX[xBlk][yBlk]=i(H-187[[178]])

spPredFlagLX[xBlk][yBlk]＝1(H-188[[179]])

curAvailableFlag=1(H-189[[180]])

-[[Depending on curAvailableFlag, the following applies:

If curAvailableFlag is equal to 1, the following ordered steps apply:

1. When lastAvailableFlag is equal to 0, the following applies:

- For X in the range 0 to 1 (inclusive), the following applies:

mvLXInterView=spMvLX[xBlk][yBlk](H-181)

refIdxLXInterView=spRefIdxLX[xBlk][yBlk](H-182)

availableFlagLXInterView=spPredFlag[xBlk][yBlk](H-183)

- When curSubBlockIdx is greater than 0, for k in the range of 0 to (curSubBlockIdx-1), inclusive, the following applies:

- Export the variables i and k as specified below:

i＝k%(nPSW/nSbW)(H-184)

j＝k/(nPSW/nSbW)(H-185)

- For X in the range 0 to 1 (inclusive), the following applies:

spMvLX[i][j]＝spMvLX[xBlk][yBlk](H-186)

spRefIdxLX[i][j]＝spRefIdxLX[xBlk][yBlk](H-187)

spPredFlagLX[i][j]＝spPredFlagLX[xBlk][yBlk](H-188)

2. Set the variable lastAvailableFlag to 1.

3. Set the variables xLastAvail and yLastAvail to xBlk and yBlk respectively.

-[[else (]]If curAvailableFlag is equal to 0[[), when lastAvailable flag is equal to 1,]]for X in the range of 0 to 1 (inclusive), the following applies:

[[spMvLX[xBlk][yBlk]＝spMvLX[xLastAvail][yLastAvail](H-189)

spRefIdxLX[xBlk][yBlk]＝spRefIdxLX[xLastAvail][yLastAvail](H-190)

spPredFlagLX[xBlk][yBlk]=spPredFlagLX[xLastAvail][yLastAvail]

(H-191)

spMvLX[xBlk][yBlk]=centerMvLX(H-190)

spRefIdxLX[xBlk][yBlk]=centerRefIdxLX(H-191)

spPredFlagLX[xBlk][yBlk]=centerPredFlagLX(H-192)

-[[Set the variable curSubBlockIdx equal to curSubBlockIdx+1. ]]

For the derivation process of variables to be used later in the decoding process, the following assignments are made, for x = 0..nPbW-1 and for y = 0..nPbH-1:

- For X in the range 0 to 1 (inclusive), the following applies:

- Export the variables SubPbPredFlagLX, SubPbMvLX, and SubPbRefIdxLX as specified below:

SubPbPredFlagLX[xPb+x][yPb+y]＝spPredFlagLX[x/nSbW][y/nSbW](H-193[[192]])

SubPbMvLX[xPb+x][yPb+y]＝spMvLX[x/nSbW][y/nSbW](H-194[[193]])

SubPbRefIdxLX[xPb+x][yPb+y]＝spRefIdxLX[x/nSbW][y/nSbW](H-195[[194]])

- Invoke the derivation process for chroma motion vectors in subclause 8.5.3.2.9 with SubPbMvLX[xPb+x][yPb+y] as input and output as SubPbMvCLX[xPb+x][yPb+y].

FIG16A is a flowchart illustrating an example operation of video encoder 20 that encodes a CU using inter prediction according to an example of this disclosure. In the example of FIG16A , video encoder 20 may generate a merge candidate list for a current PU of the current CU (200). According to one or more examples of this disclosure, video encoder 20 may generate the merge candidate list such that the merge candidate list includes temporal inter-view merge candidates based on motion information of sub-PUs of the current PU. In some examples, the current PU may be a depth PU, and video encoder 20 may generate the merge candidate list such that the merge candidate list includes texture merge candidates based on motion information of sub-PUs of the current depth PU. Furthermore, in some examples, video encoder 20 may perform the operations of FIG17 to generate the merge candidate list for the current PU.

After generating a merge candidate list for the current PU, video encoder 20 may select a merge candidate from the merge candidate list (202). In some examples, video encoder 20 may select the merge candidate based on a rate/distortion analysis. Furthermore, video encoder 20 may use motion information (e.g., a motion vector and a reference index) of the selected merge candidate to determine a predictive block for the current PU (204). Video encoder 20 may signal a merge candidate index indicating a position of the selected merge candidate within the merge candidate list (206).

If the selected merge candidate is an IPMVC or MVI candidate constructed using sub-PUs (i.e., a texture merge candidate), as described in the examples of this disclosure, the IPMVC or MVI candidate may specify a separate set of motion parameters (e.g., a set of one or more motion vectors and a set of one or more reference indices) for each sub-PU of the current PU. When video encoder 20 is determining the predictive block of the current PU, video encoder 20 may use the motion parameters of the sub-PUs of the current PU to determine the predictive block of the sub-PU. Video encoder 20 may determine the predictive block of the current PU by assembling the predictive blocks of the sub-PUs of the current PU.

Video encoder 20 may determine whether there are any remaining PUs in the current CU (208). If there are one or more remaining PUs in the current CU ("YES" of 208), video encoder 20 may repeat actions 200-208 with another PU of the current CU as the current PU. In this manner, video encoder 20 may repeat actions 200-208 for each PU of the current CU.

When there are no remaining PUs for the current CU ("No" of 208), video encoder 20 may determine residual data for the current CU (210). In some examples, each sample of the residual data may indicate a difference between a sample in a coding block of the current CU and a corresponding sample in a predictive block of a PU of the current CU. In other examples, video encoder 20 may use ARP to determine the residual data for the current CU. Video encoder 20 may signal the residual data in the bitstream (212). For example, video encoder 20 may signal the residual data in the bitstream by applying one or more transforms to the residual data to generate coefficient blocks, quantizing the coefficients, entropy encoding syntax elements indicating the quantized coefficients, and including the entropy-encoded syntax elements in the bitstream.

FIG16B is a flowchart illustrating an example operation of video decoder 30 for decoding a CU using inter-frame prediction according to an example of this disclosure. In the example of FIG16B , video decoder 30 may generate a merge candidate list for a current PU of the current CU (220). According to one or more examples of this disclosure, video decoder 30 may generate the merge candidate list such that the merge candidate list includes temporal inter-view merge candidates based on motion information of sub-PUs of the current PU. In some examples, the current PU may be a depth PU, and video decoder 30 may generate the merge candidate list such that the merge candidate list includes texture merge candidates based on motion information of sub-PUs of the current depth PU. Furthermore, in some examples, video decoder 30 may perform the operations of FIG17 to generate the merge candidate list for the current PU.

After generating the merge candidate list for the current PU, video decoder 30 may determine a selected merge candidate from the merge candidate list (222). In some examples, video decoder 30 may determine the selected merge candidate based on a merge candidate index signaled in the bitstream. Furthermore, video decoder 30 may use motion parameters (e.g., motion vector and reference index) of the selected merge candidate to determine the predictive blocks for the current PU (224). For example, video decoder 30 may use the motion parameters of the selected merge candidate to determine the luma predictive blocks, Cb predictive blocks, and Cr predictive blocks for the current PU.

If the selected merge candidate is an IPMVC or MVI candidate constructed using sub-PUs, as described in the examples of this disclosure, the IPMVC or MVI candidate may specify a separate set of motion parameters (e.g., a set of one or more motion vectors and a set of one or more reference indices) for each sub-PU of the current PU. When video decoder 30 is determining the predictive block of the current PU, video decoder 30 may use the motion parameters of the sub-PUs of the current PU to determine the predictive blocks of the sub-PUs. Video decoder 30 may determine the predictive block of the current PU by assembling the predictive blocks of the sub-PUs of the current PU.

Video decoder 30 may then determine whether there are any remaining PUs in the current CU (226). If there are one or more remaining PUs in the current CU ("YES" of 226), video decoder 30 may repeat actions 220-226 by using another PU of the current CU as the current PU. In this manner, video decoder 30 may repeat actions 220-226 for each PU of the current CU.

When there are no remaining PUs for the current CU ("NO" of 226), video decoder 30 may determine residual data for the current CU (228). In some examples, video decoder 30 may determine the residual data in parallel with determining the motion parameters of the PUs of the current CU. In some examples, video decoder 30 may use ARP to determine the residual data for the current CU. In addition, video decoder 30 may reconstruct the coding blocks of the current CU based on the predictive blocks of the PUs of the current CU and the residual data of the current CU (230).

FIG17 is a flowchart illustrating example operation of a video coder to construct a merge candidate list for a current PU in a current view component according to an example of this disclosure. In the example of FIG17 , a video coder (e.g., video encoder 20 or video decoder 30) may determine spatial merge candidates (250). The spatial merge candidates may include merge candidates that specify motion parameters of PUs covering positions _A0 , _A1 , _B0 , _B1 , and _B2 in FIG3 . In some examples, the video coder may determine the spatial merge candidates by performing the operations described in subclause G.8.5.2.1.2 of MV-HEVC Test Model 4. Furthermore, in the example of FIG17 , the video coder may determine temporal merge candidates (252). The temporal merge candidates may specify motion parameters of a PU of a reference view component in a time instance different from the current view component. In some examples, the video coder may determine the temporal merge candidates by performing the operations described in subclause H.8.5.2.1.7 of 3D-HEVC Test Model 4.

In addition, the video coder may determine the IPMVC and the IDMVC (254). According to an example of the present disclosure, the video coder may generate the IPMVC using a sub-PU level inter-view motion prediction technique. The IPMVC may then specify motion parameters for each sub-PU of the current PU. In some examples, the video coder may perform the operations of FIG. 19 to determine the IPMVC. The IDMVC may specify a disparity vector for the current PU. In some examples, the video coder determines the IPMVC and the IDMVC only when an inter-view motion prediction flag (e.g., iv_mv_pred_flag) of the current layer indicates that inter-view motion prediction is enabled for the current layer. The current layer may be the layer to which the current view component belongs.

17 , the video coder may determine a VSP merge candidate ( 256 ). In some examples, the video coder may determine the VSP merge candidate by performing operations described in subclause H.8.5.2.1.12 of 3D-HEVC Test Model 4. In some examples, the video coder determines the VSP merge candidate only when the view synthesis prediction flag of the current layer indicates that view synthesis prediction is enabled for the current layer.

In addition, the video coder may determine whether the current view component is a depth view component (258). In response to determining that the current view component is a depth view component ("YES" of 258), the video coder may determine a texture merge candidate (260). The texture merge candidate may specify motion information of one or more PUs in the texture view component corresponding to the current (depth) view component. According to one or more examples of the present disclosure, the video coder may generate the texture merge candidate using a sub-PU level motion prediction technique. Thereafter, the texture merge candidate may specify motion parameters for each sub-PU of the current PU. In some examples, the video coder may perform the operations of FIG. 19 to determine the texture merge candidate. The video coder may then determine whether a texture merge component is available (262). In response to determining that a texture merge component is available ("YES" of 262), the video coder may insert the texture merge candidate into the merge candidate list (264).

In response to determining that the current picture is not a depth picture ("No" of 258), in response to determining that a texture merge candidate is not available ("No" of 262), or after inserting a texture merge candidate into the merge candidate list, the video coder may determine whether an IPMVC is available (266). When the video coder is unable to determine the IPMVC (e.g., when the current PU is in the base view), the IPMVC may not be available. In response to determining that the IPMVC is available ("Yes" of 268), the video coder may insert the IPMVC into the merge candidate list (268).

In response to determining that the IPMVC is not available ("No" of 266), or after inserting the IPMVC into the merge candidate list, the video coder may determine whether a spatial merge candidate at position _A1 (i.e., _A1 spatial merge candidate) is available (270). A spatial merge candidate, such as the A1 spatial merge candidate, may be unavailable when the PU or PUs covering the position associated with the spatial merge candidate (e.g., position _A0 , _A1 , _B0 , _B1 , or _B2 ) are coded using intra prediction and are outside the current slice or picture boundary. In response to determining that the _A1 spatial merge candidate is available ("Yes" of 270), the video coder may determine whether the motion vector and reference index of _{the A1} spatial merge candidate match the representative motion vector and representative reference index of the IPMVC (270). In response to determining that the motion vector and reference index of the A ₁ spatial merge candidate do not match the representative motion vector and representative reference index of the IPMVC (“NO” of 272 ), the video coder may insert the A ₁ spatial merge candidate into the merge candidate list ( 274 ).

As indicated above, the video coder may use sub-PU level motion prediction techniques to generate IPMVC and/or texture merge candidates. The IPMVC and/or texture merge candidates may then specify multiple motion vectors and multiple reference indices. Thus, the video coder may determine whether the motion vector of the _A1 spatial merge candidate matches the representative motion vector of the IPMVC and/or texture merge candidate, and whether the reference index of the _A1 spatial merge candidate matches the representative reference index of the IPMVC and/or texture merge candidate. The representative motion vector and representative reference index of the IPMVC may be referred to herein as a “PU-level IPMVC”. The representative motion vector and representative reference index of the texture merge candidate may be referred to herein as a “PU-level motion parameter inheritance (MPI) candidate”. The video coder may determine the PU-level IPMVC and PU-level MPI candidates in different ways. Examples of how the video coder may determine the PU-level IPMVC and PU-level MPI candidates are described elsewhere in this disclosure.

In response to determining that the _A1 spatial merge candidate is not available ("No" of 270), in response to determining that the motion vector and reference index of the _A1 spatial merge candidate match the representative motion vector and representative reference index of the IPMVC ("Yes" of 272), or after inserting the _A1 spatial merge candidate into the merge candidate list, the video coder may determine whether a spatial merge candidate at position _B1 (i.e., the _B1 spatial merge candidate) is available (276). In response to determining that the _B1 spatial merge candidate is available ("Yes" of 276), the video coder may determine whether the motion vector and reference index of the _B1 spatial merge candidate match the representative motion vector and representative reference index of the IPMVC (278). In response to determining that the motion vector and reference index of the _B1 spatial merge candidate do not match the representative motion vector and representative reference index of the IPMVC ("No" of 278), the video coder may include the _B1 spatial merge candidate in the merge candidate list (280).

In response to determining that the _B1 spatial motion vector is not available ("No" of 276), in response to determining that the motion vector and reference index of the _B1 spatial motion vector match the representative motion vector and representative reference index of the IPMVC ("Yes" of 278), or after inserting the _B1 spatial merge candidate into the merge candidate list, the video coder may determine whether a spatial merge candidate at position _B0 (i.e., the _B0 spatial merge candidate) is available (282). In response to determining that the _B0 spatial merge candidate is available ("Yes" of 282), the video coder may insert the _B0 spatial merge candidate into the merge candidate list (284).

As indicated above, the video coder may determine the representative motion vector and representative reference index of the IPMVC in different ways. In one example, the video coder may determine the center sub-PU from among the sub-PUs of the current PU. In this example, the center sub-PU is the sub-PU that is closest to the center pixel of the luma prediction block of the current PU. Because the height and/or width of the prediction block may be an even number of samples, the "center" pixel of the prediction block may be the pixel adjacent to the true center of the prediction block. Furthermore, in this example, the video coder may then determine the inter-view reference block of the center sub-PU by adding the disparity vector of the current PU to the coordinates of the center of the luma prediction block of the center sub-PU. If the inter-view reference block of the center sub-PU is coded using motion compensated prediction (i.e., the inter-view reference block of the center sub-PU has one or more motion vectors and reference indices), the video coder may set the motion information of the PU-level IPMVC to the motion information of the inter-view reference block of the center sub-PU. Therefore, the PU-level IPMVC provided by the center sub-PU can be used to refine this sub-PU candidate and other candidates (such as spatial neighboring candidates _A1 and _B1 ). For example, if a regular candidate (such as _A1 spatial merge candidate or _B1 spatial merge candidate) is equal to the candidate generated by the center sub-PU, the other regular candidate is not added to the merge candidate list.

In response to determining that the _B0 spatial merge candidate is not available ("No" of 282) or after inserting the _B0 spatial merge candidate into the merge candidate list, the video coder may determine whether IDMVC is available and whether the motion vector and reference index of the IDMVC are different from the motion vectors and reference indexes of the _A1 spatial merge candidate and the _B1 spatial merge candidate (286). In response to determining that the IDMVC is available and the motion vector and reference index of the IDMVC are different from the motion vectors and reference indexes of the _A1 spatial merge candidate and the _B1 spatial merge candidate ("Yes" of 286), the video coder may insert the IDMVC into the merge candidate list (288).

In response to determining that IDMVC is not available or the motion vector and reference index of IDMVC are not different from the motion vector and reference index of the _A1 spatial merge candidate or the _B1 spatial merge candidate ("No" of 286) or after inserting the IDMVC into the merge candidate list, the video decoder may perform the portion of the reference picture list construction operation shown in Figure 18 (denoted as A in Figure 17).

FIG18 is a flowchart illustrating a continuation of the reference picture list construction operation of FIG17 according to an example of this disclosure. In the example of FIG18 , the video coder may determine whether a VSP merge candidate is available ( 300 ). In response to determining that a VSP merge candidate is available (“YES” of 300 ), the video coder may insert the VSP merge candidate into the merge candidate list ( 302 ).

In response to determining that the VSP merge candidate is not available ("No" of 300) or after inserting the VSP merge candidate into the merge candidate list, the video coder may determine whether a spatial merge candidate at position _A0 (i.e., _A0 spatial merge candidate) is available (304). In response to determining that the _A0 spatial merge candidate is available ("Yes" of 304), the video coder may insert the _A0 spatial merge candidate into the merge candidate list (306).

Furthermore, in response to determining that the _A0 spatial merge candidate is not available ("No" of 306), or after inserting the _A0 spatial merge candidate into the merge candidate list, the video coder may determine whether a spatial merge candidate at position _B2 (i.e., a _B2 spatial merge candidate) is available (308). In response to determining that the _B2 spatial merge candidate is available ("Yes" of 308), the video coder may insert the _B2 spatial merge candidate into the merge candidate list (310).

In response to determining that the _B2 spatial merge candidate is not available ("No" of 308), or after inserting the _B2 spatial merge candidate into the merge candidate list, the video coder may determine whether a temporal merge candidate is available (312). In response to determining that the temporal merge candidate is available ("Yes" of 312), the video coder may insert the temporal merge candidate into the merge candidate list (314).

Furthermore, in response to determining that the temporal merge candidate is not available ("No" of 312) or after inserting the temporal merge candidate into the merge candidate list, the video coder may determine whether the current slice is a B slice (316). In response to determining that the current slice is a B slice ("Yes" of 316), the video coder may derive a combined bi-predictive merge candidate (318). In some examples, the video coder may derive the combined bi-predictive merge candidate by performing the operations described in subclause H.8.5.2.1.3 of 3D-HEVC Test Model 4.

In response to determining that the current slice is not a B slice ("NO" of 316), or after deriving the combined bi-predictive merge candidate, the video coder may derive a zero motion vector merge candidate (320). The zero motion vector merge candidate may specify a motion vector having a horizontal component and a vertical component equal to zero. In some examples, the video coder may derive the zero motion vector candidate by performing the operations described in subclause 8.5.2.1.4 of 3D-HEVC Test Model 4.

FIG19 is a flowchart illustrating the operation of a video coder to determine an IPMVC or texture merge candidate according to an example of this disclosure. In the example of FIG19 , a video coder (e.g., video encoder 20 or video decoder 30) may divide a current PU into a plurality of sub-PUs (348). In different examples, the block size of each of the sub-PUs may be 4×4, 8×8, 16×16, or another size.

19 , the video coder may set a default motion vector and a default reference index (350). In different examples, the video coder may set the default motion vector and the default reference index differently. In some examples, the default motion parameters (i.e., the default motion vector and the default reference index) are equal to the PU-level motion vector candidate. Furthermore, in some examples, the video coder may determine the default motion information differently depending on whether the video coder is determining an IPMVC or texture merge candidate.

In some examples where the video coder is determining the IPMVC, the video coder may derive the PU-level IPMVC from the center position of the corresponding region of the current PU, as defined in 3D-HEVC Test Model 4. Furthermore, in this example, the video coder may set the default motion vector and reference index equal to the PU-level IPMVC. For example, the video coder may set the default motion vector and default reference index to the PU-level IPMVC. In this example, the video coder may derive the PU-level IPMVC from the center position of the corresponding region of the current PU.

In another example where the video coder is determining the IPMVC, the video coder may set the default motion parameters to the motion parameters contained by the inter-view reference block covering the pixel at coordinates (xRef, yRef) of the reference picture in the reference view. The video coder may determine the coordinates (xRef, yRef) as follows:

xRef＝Clip3(0,PicWidthInSamplesL-1,xP+((nPSW[[-1]])>>1)+((mvDisp[0]+2)>>2))

yRef＝Clip3(0,PicHeightInSamplesL-1,yP+((nPSH[[-1]])>>1)+((mvDisp[1]+2)>>2))

In the above equations, (xP, yP) indicates the coordinates of the top-left sample of the current PU, mvDisp is the disparity vector, nPSWxnPSH is the size of the current PU, and PicWidthInSamplesL and PicHeightInSamplesL define the resolution of the picture in the reference view (same as the current view). In the above equations, italic text in double square brackets indicates text deleted from equations H-124 and H-125 in section H.8.5.2.1.10 of 3D-HEVC Test Model 4.

As discussed above, section H.8.5.2.1.10 of 3D-HEVC Test Model 4 describes the derivation process for temporal inter-view motion vector candidates. Furthermore, as discussed above, equations H-124 and H-125 are used in section H.8.5.2.1.10 of 3D-HEVC Test Model 4 to determine the luma position of a reference block in a reference picture. Unlike equations H-124 and H-125 in 3D-HEVC Test Model 4, the equations of this example do not subtract 1 from nPSW and nPSH. Consequently, xRef and yRef indicate the coordinates of the pixel directly below and to the right of the true center of the prediction block of the current PU. Because the width and height of the prediction block of the current PU, which are sample values, may be even numbers, there may not be a sample value at the true center of the prediction block of the current PU. Compared to the case where xRef and yRef indicate the coordinates of the pixel directly above and to the left of the true center of the prediction block of the current PU, coding gain may be achieved when xRef and yRef indicate the coordinates of the pixel directly below and to the right of the true center of the prediction block of the current PU. In other examples, the video coder may use other blocks covering different pixels (xRef, yRef) to derive the default motion vector and reference index.

In another example of how a video coder may set default motion parameters when the video coder is determining IPMVC, before setting the motion parameters of the sub-PUs of the current PU, the video coder may select the sub-PU that is closest to the center pixel of the luma prediction block of the current PU from among all the sub-PUs of the current PU. The video coder may then determine a reference block in a reference view component for the selected sub-PU. In other words, the video coder may determine an inter-view reference block for the selected sub-PU. When decoding the inter-view reference block of the selected sub-PU using motion compensated prediction, the video coder may use the inter-view reference block of the selected sub-PU to derive a default motion vector and reference index. In other words, the video coder may set the default motion parameters to the motion parameters of the sub-PU of the reference block that is closest to the center pixel of the luma prediction block of the current PU.

In this way, the video coder can determine a reference block in the reference picture that has the same size as the prediction block of the current PU. In addition, the video coder can determine the sub-PU closest to the center pixel of the reference block from among the sub-PUs of the reference block. The video coder can derive default motion parameters from the motion parameters of the determined sub-PUs of the reference block.

The video coder may determine the sub-PU that is closest to the center pixel of the reference block in different ways. For example, assuming the sub-PU size is ^{2U x2U} , the video coder may select the sub-PU with the following coordinates relative to the upper left sample of the luma prediction block of the current PU: (((nPSW>>(u+1))-1)<<u, (((nPSH>>(u+1))-1)<<u). In other words, the sub-PU that is closest to the center pixel of the reference block includes the pixel with the following coordinates relative to the upper left sample of the reference block: (((nPSW>>(u+1))-1)<<u, (((nPSH>>(u+1))-1)<<u). Alternatively, the video coder may select the sub-PU with the following relative coordinates relative to the upper left sample of the luma prediction block of the current PU: ((nPSW>>(u+ 1))<<u,(nPSH>>(u+1))<<u). In other words, the sub-PU closest to the center pixel of the reference block includes the pixel with the following coordinates relative to the top-left sample of the reference block: ((nPSW>>(u+1))<<u,(nPSH>>(u+1))<<u). In these equations, nPSW and nPSH are the width and height of the luma prediction block of the current PU, respectively. Therefore, in one example, the video decoder may determine the sub-PU closest to the center pixel of the luma prediction block of the current PU from among multiple sub-PUs of the current PU. In this example, the video decoder may derive default motion parameters from the inter-view reference block of the determined sub-PU.

In other examples where the video coder is determining the IPMVC, the default motion vector is a zero motion vector. Furthermore, in some examples, the default reference index is equal to the first available temporal reference picture in the current reference picture list (i.e., a reference picture at a different time instance than the current picture) or the default reference index may be equal to 0. In other words, the default motion parameters may include a default motion vector and a default reference index. The video coder may set the default motion vector to the zero motion vector and may set the default reference index to 0 or to the first available temporal reference picture in the current reference picture list.

For example, if the current slice is a P slice, the default reference index may indicate the first available temporal reference picture in RefPicList0 of the current picture (i.e., the temporal reference picture with the smallest reference index in RefPicList0 of the current picture). Furthermore, if the current slice is a B slice and inter-frame prediction from RefPicList0 is enabled but inter-frame prediction from RefPicList1 of the current picture is not enabled, the default reference index may indicate the first available temporal reference picture in RefPicList0 of the current picture. If the current slice is a B slice and inter-frame prediction from RefPicList1 of the current picture is enabled but inter-frame prediction from RefPicList0 of the current picture is not enabled, the default reference index may indicate the first available temporal reference picture in RefPicList1 of the current picture (i.e., the temporal reference picture with the smallest reference index in RefPicList1 of the current picture). If the current slice is a B slice and inter-frame prediction from RefPicList0 of the current picture and RefPicList1 of the current picture is enabled, the default RefPicList0 reference index may indicate the first available temporal reference picture in RefPicList0 of the current picture and the default RefPicList1 reference index may indicate the first available temporal reference picture in RefPicList1 of the current picture.

Furthermore, in some of the examples provided above for determining default motion parameters when the video coder is determining IPMVC, the video coder may set the default motion parameters to the motion parameters of the sub-PU closest to the center pixel of the luma prediction block of the current PU. However, in these and other examples, the default motion parameters may remain unavailable. For example, if the inter-view reference block corresponding to the sub-PU closest to the center pixel of the luma prediction block of the current PU is intra-predicted, the default motion parameters may remain unavailable. Thereafter, in some examples, when the default motion parameters are unavailable and the inter-view reference block of the first sub-PU is coded using motion-compensated prediction (i.e., the inter-view reference block of the first sub-PU has available motion information), the video coder may set the default motion parameters to the motion parameters of the first sub-PU. In this example, the first sub-PU may be the first sub-PU of the current PU in raster scan order of the sub-PUs of the current PU. Thus, when determining default motion parameters, in response to determining that the first sub-PU in the raster scan order of multiple sub-PUs has available motion parameters, the video decoder may set the default motion parameters to the available motion parameters of the first sub-PU in the raster scan order of the multiple sub-PUs.

Otherwise, when the default motion information is not available (for example, when the motion parameters of the inter-view reference block of the first sub-PU are not available), if the first sub-PU of the current sub-PU row has available motion parameters, the video coder may set the default motion information to the motion parameters of the first sub-PU of the current sub-PU row. When the default motion parameters are still not available (for example, when the inter-view reference block of the first sub-PU of the current sub-PU row is not available), the video coder may set the default motion vector to a zero motion vector and may set the default reference index to be equal to the first available temporal reference picture in the current reference picture list. In this way, when the video coder is determining the default motion parameters, in response to determining that the first sub-PU of the sub-PU row including the corresponding sub-PU has available motion parameters, the video coder may set the default motion parameters to the available motion parameters of the first sub-PU of the sub-PU row including the corresponding sub-PU.

Furthermore, as described above with respect to the example of FIG. 17 , the video coder may use sub-PU-level motion prediction techniques to determine texture merge candidates. In such examples, the current PU may be referred to herein as the “current depth PU.” The video coder may perform the operations of FIG. 19 to determine texture merge candidates. Thereafter, when the video coder is determining texture merge candidates, the video coder may divide the current depth PU into several sub-PUs, and each sub-PU uses the motion information of the collocated texture block for motion compensation. Furthermore, when the video coder is determining texture merge candidates, if the sub-PU's corresponding texture block is intra-coded or is not included in the reference picture list of the current depth PU in a picture in the same access unit as the corresponding texture block, the video coder may assign a default motion vector and reference index to the sub-PU. Therefore, in general, when the collocated texture block is not intra-coded and the reference picture used by the collocated texture block is in the reference picture list of the current depth picture, the video coder may determine that the collocated texture block has available motion information. In contrast, when the co-located texture block is intra-coded or the co-located texture block uses a reference picture that is not in the reference picture list of the current depth picture, the motion parameters of the co-located texture block may not be available.

As indicated above, the video coder may determine default motion information differently depending on whether the video coder is determining an IPMVC or texture merge candidate. For example, when the video coder is determining a texture merge candidate, the video coder may determine a default motion vector and a default reference index according to one of the following examples or other examples. In one example, a co-located texture block may be co-located with the current depth PU and may have the same size as the current depth PU. In this example, the video coder sets the default motion vector and the default reference index to the motion information of the block covering the center pixel of the co-located texture block.

Thus, in some examples where the current picture is a depth view component and the reference picture is a texture view component in the same view and access unit as the current picture, the video coder may set the default motion parameters to the motion parameters associated with a block of pixels covering a reference block that is in the reference picture, co-located with the current PU, and has the same size as the current PU. In such examples, the pixel may be the center pixel of the reference block or another pixel of the reference block.

In another example where the video coder is determining a texture merge candidate, the co-located texture block may have the same size as the current depth PU. In this example, the video coder may set a default motion vector and a default reference index to the motion information of a block (e.g., a PU) covering any given pixel within the co-located texture block.

In another example where the video coder is determining a texture merge candidate, the video coder may first select a center sub-PU of the current depth PU. Among all sub-PUs of the current depth PU, the center sub-PU may be located closest to (or may include) the center pixel of the prediction block of the current depth PU. The video coder may then use the texture block co-located with the center sub-PU to derive a default motion vector and reference index. Assuming a sub-PU size of ^{2U x2U} , the video coder may determine the center sub-PU as the sub-PU with the following coordinates relative to the top-left sample of the prediction block of the current depth PU (and subsequently the top-left sample of the co-located texture block): (((nPSW >> (u+1))-1) <<u, (((nPSH >> (u+1))-1) <<u). Alternatively, the video coder may determine the relative coordinates of the center sub-PU as: ((nPSW >> (u+1)) <<u, (nPSH >> (u+1)) <<u). In these equations, nPSW and nPSH are the width and height of the prediction block of the current depth PU, respectively.

Thus, in one example, the video coder may determine the sub-PU closest to the center of the prediction block of the current PU from among the multiple sub-PUs of the current PU. In this example, the video coder may derive default motion parameters from the co-located texture block of the determined sub-PU.

In some instances where the video coder is determining a texture merge candidate and default motion information is not available (e.g., when the motion parameters of the collocated texture block of the center sub-PU are not available), the video coder may determine whether the collocated texture block of the first sub-PU of the current depth PU has available motion information. The first sub-PU of the current depth PU may be the first sub-PU of the current depth PU in the raster scan order of the sub-PUs of the current depth PU. If the motion parameters of the collocated texture block of the first sub-PU of the current depth PU are available, the video coder may set the default motion parameters to the motion parameters of the first sub-PU of the current depth PU.

Furthermore, in some examples where the video coder is determining a texture merge candidate, when default motion information is unavailable (e.g., when motion parameters of a co-located texture block of the first sub-PU are unavailable), if the first sub-PU of the current sub-PU row has available motion information, the video coder sets the default motion information to the motion information of the first sub-PU of the current sub-PU row. Furthermore, when default motion information is unavailable (e.g., when motion information of the first sub-PU of the current sub-PU row is unavailable), the default motion vector is a zero motion vector, and the default reference index is equal to the first available temporal reference picture in the current reference picture list or 0.

In some examples where the video coder is determining a texture merge candidate, the default motion vector is a zero motion vector, and the default reference index is equal to the first available temporal reference picture in the current reference picture list or 0.

Regardless of whether the video coder is determining an IPMVC or texture merge candidate, the video coder can set default motion information for the entire current PU. Therefore, the video coder does not need to store more motion vectors in the current PU for predicting spatial neighboring blocks, temporal neighboring blocks (when the picture containing this PU is used as a co-located picture during TMVP), or deblocking.

Furthermore, the video coder may determine PU-level motion vector candidates (352). For example, the video coder may determine a PU-level IPMVC or PU-level motion parameter inheritance (MPI) candidate (i.e., a PU-level texture merge candidate) depending on whether the video coder is determining an IPMVC or texture merge candidate. The video coder may determine whether to include one or more spatial merge candidates in the candidate list based on the PU-level motion vector candidate. In some examples, the PU-level motion vector candidate specifies motion parameters that are the same as the default motion parameters.

In some examples where the video coder is determining the IPMVC, the video coder may derive the PU-level IPMVC from the center position of the corresponding region of the current PU, as defined in 3D-HEVC Test Model 4. As described in the example of FIG 17 , the video coder may use the representative motion vector and the representative reference index of the IPMVC (i.e., the PU-level IPMVC) to determine whether to include the _A1 spatial merge candidate and the _B1 spatial merge candidate in the merge candidate list.

In another example where the video coder is determining the IPMVC, the video coder may determine a reference block in an inter-view reference picture based on the disparity vector of the current PU. The video coder may then determine a sub-PU that covers a center pixel of the reference block (i.e., a sub-PU that is closest to the center pixel of the reference block). In this example, the video coder may determine motion parameters of the determined sub-PU for the reference block specified by the PU-level IPMVC. As indicated elsewhere in this disclosure, the video coder may determine the sub-PU that is closest to the center pixel of the reference block in different ways. For example, assuming a sub-PU size of 2 ^U x 2 ^U , the sub-PU closest to the center pixel of the reference block includes the pixel with the following coordinates relative to the top left sample of the reference block: (((nPSW >> (u+1))-1) <<u, (((nPSH >> (u+1))-1) <<u). Alternatively, the sub-PU closest to the center pixel of the reference block includes the pixel with the following coordinates relative to the top left sample of the reference block: ((nPSW >> (u+1)) <<u, (nPSH >> (u+1)) <<u). In these equations, nPSW and nPSH are the width and height of the luma prediction block of the current PU, respectively. In this example, the video coding The video coder can use the determined motion parameters of the sub-PU as the PU-level IPMVC. The PU-level IPMVC can specify the representative motion vector and representative reference index of the IPMVC. In this way, the video coder can use the motion parameters of the sub-PU closest to the center pixel of the reference block to determine the PU-level IPMVC. In other words, the video coder can derive the PU-level IPMVC from the center position of the corresponding region of the current PU and determine whether to include the spatial merge candidate in the candidate list based on the PU-level IPMVC. The motion parameters used from the sub-PU can be the same as the motion parameters used by the video coder to establish the IPMVC.

In some examples where the video coder is determining texture merge candidates, the motion information used from the center sub-PU for default motion parameters may be the same as the motion information used to establish the PU-level motion parameter inheritance (MPI) candidate. The video coder may determine whether to include a particular spatial merge candidate in the merge candidate list based on the PU-level MPI candidate. For example, if the _A1 spatial merge candidate has the same motion vector and the same reference index as the PU-level MPI candidate, the video coder does not insert the _A1 spatial merge candidate into the merge candidate list. Similarly, if the _B1 spatial merge candidate has the same motion vector and the same reference index as the _A1 spatial merge candidate or the PU-level MPI candidate, the video coder does not insert _B1 into the merge candidate list.

In the example of FIG19 , the video coder may determine a reference sample location in a reference picture for a current sub-PU of the current PU (354). The reference picture may be in a different view than the picture containing the current PU (i.e., the current picture). In some examples, the video coder may determine the reference location by adding the disparity vector of the current PU to the coordinates of the center pixel of the current sub-PU. In other examples (e.g., when the current PU is a depth PU), the reference sample location may be co-located with samples of the prediction block of the current depth PU.

In addition, the video coder may determine a reference block for the current sub-PU (356). The reference block may be a PU of a reference picture and may cover the determined reference sample location. Next, the video coder may determine whether the reference block was coded using motion compensated prediction (358). For example, if the reference block was coded using intra prediction, the video coder may determine that the reference block was not coded using motion compensated prediction. If the reference block was coded using motion compensated prediction, the reference block has one or more motion vectors.

In response to determining that the reference block is coded using motion compensated prediction ("YES" of 358), the video coder may set the motion parameters of the current sub-PU based on the motion parameters of the reference block (360). For example, the video coder may set the RefPicList0 motion vector of the current sub-PU to the RefPicList0 motion vector of the reference block, may set the RefPicList0 reference index of the current sub-PU to the RefPicList0 reference index of the reference block, may set the RefPicList1 motion vector of the current sub-PU to the RefPicList1 motion vector of the reference block, and may set the RefPicList1 reference index of the current sub-PU to the RefPicList1 reference index of the reference block.

On the other hand, in response to determining that the reference block is not coded using motion compensated prediction ("No" of 358), the video coder may set the motion parameters of the current sub-PU to default motion parameters (362). Therefore, in the example of FIG19, when the reference block of the current sub-PU is not coded using motion compensated prediction, the video coder does not set the motion parameters of the current sub-PU to the motion parameters of the closest sub-PU having a reference block coded using motion compensated prediction. In fact, the video coder may directly set the motion parameters of the current sub-PU to the default motion parameters. This operation can simplify and speed up the decoding process.

After setting the motion parameters of the current sub-PU, the video coder may determine whether the current PU has any additional sub-PUs (364). In response to determining that the current PU has one or more additional sub-PUs ("YES" of 364), the video coder may perform actions 354 to 364 by taking another of the sub-PUs of the current PU as the current sub-PU. In this manner, the video coder may set the motion parameters of each of the sub-PUs of the current PU. On the other hand, in response to determining that there are no additional sub-PUs of the current PU ("NO" of 366), the video coder may include a candidate (e.g., IPMVC) in the merge candidate list for the current PU (366). The candidate may specify the motion parameters of each of the sub-PUs of the current PU.

In one or more examples, the described functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to tangible media (e.g., data storage media), or communication media that includes any media that facilitates the transfer of a computer program from one place to another (e.g., according to a communication protocol). In this manner, computer-readable media may generally correspond to (1) tangible computer-readable storage media, which is non-transitory, or (2) communication media, such as a signal or carrier wave. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and/or data structures for implementing the techniques described in this disclosure. A computer program product may include computer-readable media.

By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Moreover, any connection is properly referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but rather refer to non-transitory, tangible storage media. As used herein, disk and optical disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disks typically reproduce data magnetically, while optical discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the term "processor," as used herein, may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or incorporated into a combined codec. Moreover, the techniques may be fully implemented in one or more circuits or logic elements.

The techniques of this disclosure can be implemented in a wide variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs), or collections of ICs (e.g., chipsets). The various components, modules, or units described in this disclosure are intended to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require implementation by distinct hardware units. In fact, as described above, the various units may be combined in a codec hardware unit in conjunction with appropriate software and/or firmware, or provided by a set of interoperable hardware units, including one or more processors as described above.

Various embodiments have been described. These and other embodiments are within the scope of the following claims.

Claims (43)

1. A method for decoding multi-view video data, the method comprising: The current prediction unit (PU) is divided into multiple sub-PUs, wherein the current PU is in the current image; Determine the default motion parameters; Process the sub-PUs from the plurality of sub-PUs in a specific order. Wherein, for each corresponding subPU from the plurality of subPUs, if motion compensation prediction decoding of the reference block of the corresponding subPU is not used, then in response to subsequently determining the reference block of any later subPU in the order of motion compensation prediction decoding, the motion parameters of the corresponding subPU are not set, wherein motion compensation prediction decoding of the reference block of at least one of the subPUs is not used, and wherein processing the subPU includes, for each corresponding subPU from the plurality of subPUs: Determine the reference block of the corresponding sub-PU, wherein the reference image contains the reference block of the corresponding sub-PU; If motion compensation predictive decoding is used for the reference block of the corresponding sub-PU, then the motion parameters of the corresponding sub-PU are set based on the motion parameters of the reference block of the corresponding sub-PU; and If motion compensation prediction decoding of the reference block of the corresponding sub-PU is not used, then the motion parameters of the corresponding sub-PU are set to the default motion parameters; and The candidate is included in the candidate list of the current PU, wherein the candidate is based on the motion parameters of the plurality of subPUs; Obtain the syntax element indicating the selected candidate from the candidate list from the bit stream; and The motion parameters of the selected candidates are used to reconstruct the predictive block of the current PU. 2. The method according to claim 1, wherein the reference image is in a view different from the current image; and Determining the reference block of the corresponding sub-PU includes determining the location of a reference sample in the reference image based on the disparity vector of the current PU, wherein the reference block of the corresponding sub-PU covers the location of the reference sample. 3. The method according to claim 1, wherein: The current image is a depth view component, and the reference image is a texture view component in the same view and access unit as the current image; and Determining the reference block of the corresponding sub-PU includes determining that the reference block of the corresponding sub-PU is the PU of the reference image co-located with the corresponding sub-PU. 4. The method according to claim 1, wherein: The current image is a depth view component, and the reference image is a texture view component in the same view and access unit as the current image; and Determining the default motion parameters includes setting the default motion parameters to motion parameters associated with a block of pixels covering a reference block, which is co-located with the current PU in the reference image and has the same size as the current PU. 5. The method of claim 4, wherein the pixel is the center pixel of the reference block. 6. The method of claim 1, further comprising determining whether to include spatial merging candidates in the candidate list based on PU-level motion vector candidates, wherein the PU-level motion vector candidates specify motion parameters that are the same as the default motion parameters. 7. The method according to claim 1, wherein: The default motion parameters include default motion vectors and default reference indices, and Determining the default motion parameters includes: Set the default motion vector to zero; and Set the default reference index to 0 or the first available time reference image in the current reference image list. 8. The method of claim 1, wherein determining the default motion parameter includes, in response to determining that a first subPU in the raster scan sequence of the plurality of subPUs has available motion parameters, setting the default motion parameter to the available motion parameter of the first subPU in the raster scan sequence of the plurality of subPUs. 9. The method of claim 1, wherein determining the default motion parameter includes, in response to determining that a first subPU comprising the subPU row containing the corresponding subPU has available motion parameters, setting the default motion parameter to the available motion parameters of the first subPU comprising the subPU row containing the corresponding subPU. 10. The method of claim 1, wherein the default motion parameters are the same for each of the plurality of sub-PUs. 11. The method of claim 1, wherein determining the default motion parameters comprises: A reference block is determined in the reference image, the reference block having the same size as the prediction block of the current PU; From the sub-PUs of the reference block, determine the sub-PU closest to the center pixel of the reference block; and The default motion parameters are derived from the motion parameters of the determined sub-PU in the reference block. 12. The method of claim 11, wherein each of the sub-PUs of the reference block is 2 ^U x 2 ^U , and the sub-PU closest to the center pixel of the reference block comprises pixels with the following coordinates relative to the upper left sample of the reference block: (((nPSW>>(u+1))-1)<<u, (((nPSH>>(u+1))-1)<<u), where nPSW represents the width of the reference block in pixels and nPSH represents the height of the reference block in pixels. 13. The method of claim 11, wherein each of the sub-PUs of the reference block is 2 ^U x 2 ^U , and the sub-PU closest to the center pixel of the reference block comprises pixels having the following coordinates relative to the upper left sample of the reference block: ((nPSW>>(u+1))<<u,(nPSH>>(u+1))<<u), where nPSW represents the width of the reference block in pixels and nPSH represents the height of the reference block in pixels. 14. A method for encoding video data, the method comprising: The current prediction unit (PU) is divided into multiple sub-PUs, wherein the current PU is in the current image; Determine the default motion parameters; Process the sub-PUs from the plurality of sub-PUs in a specific order. Wherein, for each corresponding subPU from the plurality of subPUs, if motion compensation prediction decoding of the reference block of the corresponding subPU is not used, then in response to subsequently determining the reference block of any later subPU in the order of motion compensation prediction decoding, the motion parameters of the corresponding subPU are not set, wherein motion compensation prediction decoding of the reference block of at least one of the subPUs is not used, and wherein processing the subPU includes, for each corresponding subPU from the plurality of subPUs: Determine the reference block of the corresponding sub-PU, wherein the reference image contains the reference block of the corresponding sub-PU; If motion compensation predictive decoding is used for the reference block of the corresponding sub-PU, then the motion parameters of the corresponding sub-PU are set based on the motion parameters of the reference block of the corresponding sub-PU; and If motion compensation prediction decoding of the reference block of the corresponding sub-PU is not used, then the motion parameters of the corresponding sub-PU are set to the default motion parameters; and The candidates are included in the candidate list of the current PU, wherein the candidates are based on the motion parameters of the plurality of sub-PUs; and The syntax element of the selected candidate in the candidate list is signaled in the bit stream. 15. The method of claim 14, wherein: The reference image is in a view different from the current image; and Determining the reference block of the corresponding sub-PU includes determining the location of a reference sample in the reference image based on the disparity vector of the current PU, wherein the reference block of the corresponding sub-PU covers the location of the reference sample. 16. The method of claim 14, wherein: The current image is a depth view component, and the reference image is a texture view component in the same view and access unit as the current image; and Determining the reference block of the corresponding sub-PU includes determining that the reference block of the corresponding sub-PU is the PU of the reference image co-located with the corresponding sub-PU. 17. The method of claim 14, wherein: The current image is a depth view component, and the reference image is a texture view component in the same view and access unit as the current image; and Determining the default motion parameters includes setting the default motion parameters to motion parameters associated with a block of pixels covering a reference block, which is co-located with the current PU in the reference image and has the same size as the current PU. 18. The method of claim 17, wherein the pixel is the center pixel of the reference block. 19. The method of claim 14, further comprising determining whether to include a spatial merging candidate in the candidate list based on PU-level motion vector candidates, wherein the PU-level motion vector candidates specify motion parameters identical to the default motion parameters. 20. The method of claim 14, wherein: The default motion parameters include default motion vectors and default reference indices, and Determining the default motion parameters includes: Set the default motion vector to zero; and Set the default reference index to 0 or the first available time reference image in the current reference image list. 21. The method of claim 14, wherein determining the default motion parameter includes, in response to determining that a first subPU in the raster scan sequence of the plurality of subPUs has available motion parameters, setting the default motion parameter to the available motion parameters of the first subPU in the raster scan sequence of the plurality of subPUs. 22. The method of claim 14, wherein determining the default motion parameter includes, in response to determining that a first subPU comprising the subPU row containing the corresponding subPU has available motion parameters, setting the default motion parameter to the available motion parameters of the first subPU comprising the subPU row containing the corresponding subPU. 23. The method of claim 14, wherein the default motion parameters are the same for each of the plurality of sub-PUs. 24. The method of claim 14, wherein determining the default motion parameters comprises: A reference block is determined in the reference image, the reference block having the same size as the prediction block of the current PU; From the sub-PUs of the reference block, determine the sub-PU closest to the center pixel of the reference block; and The default motion parameters are derived from the motion parameters of the determined sub-PU in the reference block. 25. The method of claim 24, wherein each of the sub-PUs of the reference block is 2 ^U x 2 ^U , and the sub-PU closest to the center pixel of the reference block comprises pixels with the following coordinates relative to the upper left sample of the reference block: (((nPSW>>(u+1))-1)<<u, (((nPSH>>(u+1))-1)<<u), where nPSW represents the width of the reference block in pixels and nPSH represents the height of the reference block in pixels. 26. The method of claim 24, wherein each of the sub-PUs of the reference block is 2 ^U x 2 ^U , and the sub-PU closest to the center pixel of the reference block comprises pixels having the following coordinates relative to the upper left sample of the reference block: ((nPSW>>(u+1))<<u,(nPSH>>(u+1))<<u), where nPSW represents the width of the reference block in pixels and nPSH represents the height of the reference block in pixels. 27. An apparatus for decoding video data, the apparatus comprising: A memory, used to store decoded images; and One or more processors, configured to: The current prediction unit (PU) is divided into multiple sub-PUs, wherein the current PU is in the current image; Determine the default motion parameters; Process the sub-PUs from the plurality of sub-PUs in a specific order. Wherein, for each corresponding subPU from the plurality of subPUs, if motion compensation prediction decoding of the reference block of the corresponding subPU is not used, then in response to subsequently determining the reference block of any later subPU in the order of motion compensation prediction decoding, the motion parameters of the corresponding subPU are not set, wherein motion compensation prediction decoding of the reference block of at least one of the subPUs is not used, and wherein processing the subPU includes, for each corresponding subPU from the plurality of subPUs: Determine the reference block of the corresponding sub-PU, wherein the reference image contains the reference block of the corresponding sub-PU; If motion compensation predictive decoding is used for the reference block of the corresponding sub-PU, then the motion parameters of the corresponding sub-PU are set based on the motion parameters of the reference block of the corresponding sub-PU; and If motion compensation prediction decoding of the reference block of the corresponding sub-PU is not used, then the motion parameters of the corresponding sub-PU are set to the default motion parameters; and The candidates are included in the candidate list of the current PU, wherein the candidates are based on the motion parameters of the plurality of subPUs. 28. The apparatus according to claim 27, wherein: The reference image is in a view different from the current image; and The one or more processors are configured to determine the location of a reference sample in the reference image based on the disparity vector of the current PU, wherein the reference block of the corresponding sub-PU covers the location of the reference sample. 29. The apparatus according to claim 27, wherein: The current image is a depth view component, and the reference image is a texture view component in the same view and access unit as the current image; and The one or more processors are configured to determine that the reference block of the corresponding sub-PU is the PU of the reference image co-located with the corresponding sub-PU. 30. The apparatus according to claim 29, wherein: The current image is a depth view component, and the reference image is a texture view component in the same view and access unit as the current image; and The one or more processors are configured to set the default motion parameters as motion parameters associated with a block of pixels covering a reference block in the reference image, co-located with the current PU, and having the same size as the current PU. 31. The apparatus of claim 30, wherein the pixel is the center pixel of the reference block. 32. The apparatus of claim 27, wherein the one or more processors are configured to determine whether to include a spatial merging candidate in the candidate list based on PU-level motion vector candidates, wherein the PU-level motion vector candidates specify motion parameters that are the same as the default motion parameters. 33. The apparatus according to claim 27, wherein: The default motion parameters include default motion vectors and default reference indices, and The one or more processors are configured to determine the default motion parameters at least in part by: Set the default motion vector to zero; and Set the default reference index to 0 or the first available time reference image in the current reference image list. 34. The apparatus of claim 27, wherein the one or more processors are configured such that, in response to determining that a first subPU in the raster scan sequence of the plurality of subPUs has available motion parameters, the one or more processors set the default motion parameters to the available motion parameters of the first subPU in the raster scan sequence of the plurality of subPUs. 35. The apparatus of claim 27, wherein the one or more processors are configured such that, in response to determining that a first subPU comprising the corresponding subPU row has available motion parameters, the one or more processors set the default motion parameters to the available motion parameters of the first subPU comprising the corresponding subPU row. 36. The apparatus of claim 27, wherein the default motion parameters are the same for each of the plurality of sub-PUs. 37. The apparatus of claim 27, wherein the one or more processors are configured to: A reference block is determined in the reference image, the reference block having the same size as the prediction block of the current PU; From the sub-PUs of the reference block, determine the sub-PU closest to the center pixel of the reference block; and The default motion parameters are derived from the motion parameters of the determined sub-PU in the reference block. 38. The apparatus of claim 37, wherein each of the sub-PUs of the reference block is 2 ^U x 2 ^U , and the sub-PU closest to the center pixel of the reference block comprises pixels having the following coordinates relative to the upper left sample of the reference block: (((nPSW>>(u+1))-1)<<u, (((nPSH>>(u+1))-1)<<u), where nPSW represents the width of the reference block in pixels and nPSH represents the height of the reference block in pixels. 39. The apparatus of claim 37, wherein each of the sub-PUs of the reference block is 2 ^U x 2 ^U , and the sub-PU closest to the center pixel of the reference block comprises pixels having the following coordinates relative to the upper left sample of the reference block: ((nPSW>>(u+1))<<u,(nPSH>>(u+1))<<u), where nPSW represents the width of the reference block in pixels and nPSH represents the height of the reference block in pixels. 40. The apparatus of claim 27, wherein the one or more processors are further configured to: Obtain the syntax element indicating the selected candidate from the candidate list from the bit stream; and The motion parameters of the selected candidates are used to reconstruct the predictive block of the current PU. 41. The apparatus of claim 27, wherein the one or more processors are configured to signal in the bit stream the syntax element indicating the selected candidate in the candidate list. 42. An apparatus for decoding video data, the apparatus comprising: A means for dividing a current prediction unit (PU) into multiple sub-PUs, wherein the current PU is in the current image; A device for determining default motion parameters; A means for processing sub-PUs from the plurality of sub-PUs in a specific order, wherein for each corresponding sub-PU from the plurality of sub-PUs, if a reference block of the corresponding sub-PU is not used for motion compensation prediction decoding, then in response to subsequently determining a reference block for any later sub-PU in the order to be motion compensation prediction decoded, motion parameters of the corresponding sub-PU are not set, wherein a reference block for at least one of the sub-PUs is not used for motion compensation prediction decoding, and wherein the means for processing the sub-PUs includes, for each corresponding sub-PU from the plurality of sub-PUs: A means for determining a reference block of the corresponding sub-PU, wherein the reference image includes the reference block of the corresponding sub-PU; An apparatus for setting motion parameters of a corresponding sub-PU based on motion parameters of the reference block of the corresponding sub-PU if motion compensation predictive decoding of the reference block is used; and A means for setting the motion parameters of the corresponding sub-PU to the default motion parameters if the reference block of the corresponding sub-PU is not used for motion compensation predictive decoding; and A means for including candidates in the candidate list of the current PU, wherein the candidates are based on the motion parameters of the plurality of subPUs. 43. A non-transitory computer-readable data storage medium having instructions stored thereon, the instructions causing an apparatus to: The current prediction unit (PU) is divided into multiple sub-PUs, wherein the current PU is in the current image; Determine the default motion parameters; Process the sub-PUs from the plurality of sub-PUs in a specific order. Wherein, for each corresponding subPU from the plurality of subPUs, if motion compensation prediction decoding of the reference block of the corresponding subPU is not used, then in response to subsequently determining the reference block of any later subPU in the order of motion compensation prediction decoding, the motion parameters of the corresponding subPU are not set, wherein motion compensation prediction decoding of the reference block of at least one of the subPUs is not used, and wherein processing the subPU includes, for each corresponding subPU from the plurality of subPUs: Determine the reference block of the corresponding sub-PU, wherein the reference image contains the reference block of the corresponding sub-PU; If motion compensation predictive decoding is used for the reference block of the corresponding sub-PU, then the motion parameters of the corresponding sub-PU are set based on the motion parameters of the reference block of the corresponding sub-PU; and If motion compensation prediction decoding of the reference block of the corresponding sub-PU is not used, then the motion parameters of the corresponding sub-PU are set to the default motion parameters; and The candidates are included in the candidate list of the current PU, wherein the candidates are based on the motion parameters of the plurality of subPUs.