HK1211764B - Depth map delivery formats for multi-view auto-stereoscopic displays - Google Patents

Description

Depth map transport format for multi-view autostereoscopic displays

Cross reference to related applications

Priority of U.S. provisional application No.61/808422, filed on 4/2013, the entire contents of which are incorporated herein by reference.

Technical Field

The present invention relates generally to images. More particularly, embodiments of the invention relate to a format for the transmission of depth maps for multi-view autostereoscopic displays.

Background

Three-dimensional (3D) video systems, whether in movie theaters or at home, are of great interest in enhancing user experience. These systems use stereoscopic or autostereoscopic rendering methods, including: anaglyph, linear polarization, circular polarization, shutter glasses, and spectral separation.

Most 3D displays available on the market today are stereoscopic Televisions (TVs), requiring the user to wear shutter 3D glasses to experience the 3D effect. The transfer of 3D content to these displays requires the delivery of two separate views: a left view and a right view. Large-scale use of autostereoscopic (glasses-free) displays is coming to the fore. These displays provide a certain amount of motion parallax (parallax); a viewer may move his/her head slightly around to view objects from very different angles.

Conventional stereoscopic displays provide a single 3D view; however, autostereoscopic displays (also referred to as multi-view displays) provide multiple views, such as 5 views, 9 views, 28 views, etc., based on the design of the display. When conventional stereoscopic content is provided to an autostereoscopic display, the display extracts a depth map and creates or presents multiple views based on the depth map. As used herein, the term "depth map" refers to an image or other bitstream containing information related to the distance from a viewpoint to the surface of a scene object. As explained in more detail below, the depth map can be easily converted to a disparity map (disparity map), and in the context of this document, the terms depth map and disparity map are the same and interchangeable.

The depth map information may also be used to customize a 3D experience for different display types with different resolutions and display sizes (e.g., 1080p displays or 4K displays). There have been a number of studies showing that the amount of depth designed for 3D cinema is not suitable for smaller mobile devices and vice versa. The depth map may be used to re-render the view to change the perceived depth and make additional adjustments. As understood by the inventors herein, improved techniques for delivering depth maps with content are desired to improve the user experience with autostereoscopic and stereoscopic displays. It is further appreciated that these improved techniques are preferably backward compatible with existing single view (2D) and 3D systems.

The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Accordingly, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. Similarly, unless otherwise indicated, it should not be assumed based on this section that any prior art has recognized problems determined with respect to one or more methods.

Drawings

Embodiments of the invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which:

FIG. 1 illustrates an example system for transmitting 3D video and associated depth data in accordance with an embodiment of the present invention;

fig. 2 shows an example of an asymmetric single layer depth map transport format.

Fig. 3 illustrates an example of an asymmetric depth map transmission format using a rotated depth map according to an embodiment of the present invention.

Fig. 4A and 4B illustrate examples of a depth map transmission format according to an embodiment of the present invention.

Fig. 5 illustrates an example process for generating image data and depth map data segments using both depth map rotation and depth map segments in accordance with an embodiment of the present invention.

Detailed Description

Transport formats for depth maps for stereoscopic and autostereoscopic (multi-view) displays are described herein. These formats support a variety of video delivery scenarios, including traditional cable, satellite or over the air (over the air) broadcast and over-the-top (over-the-top) delivery. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It may be evident, however, that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are not shown in detail to avoid unnecessarily obscuring the present invention.

SUMMARY

Example embodiments described herein relate to a transmission format of depth map information for a multi-view display. Given a 3D input picture and corresponding input depth map data, a multiplexed asymmetric output image frame combining image data sections and depth map data sections, wherein the image data sections have different sizes than the depth map data sections, may be created. The image data section contains one or more of the input views of the 3D input, while the depth map section contains at least a portion of the input depth map data rotated with respect to the orientation of the image data in the multiplexed output image frame.

In some embodiments, the depth map data may also be segmented into one or more depth map segments, which may be rearranged before being multiplexed into the depth map portion of the output image frame.

Example transport formats for 3D

Fig. 1 illustrates an example system for encoding and decoding 3D data and associated depth data according to an embodiment of this disclosure. As shown in fig. 1, the left and right views (105-1,105-2) of the input stereo signal (105) and associated depth data Z (107) are first formatted and multiplexed by the image data and depth formatter (110) into a 3D + Z signal (112) according to a method that will be described herein.

In one embodiment of the invention, the depth data (107) represents a disparity between the left view and the right view. As used herein, the term "disparity" refers to the difference in lateral distance between the positions of objects in the left and right views of a stereoscopic image. In stereoscopic video imaging, disparity typically represents the horizontal displacement (e.g., to the left or right) of an image feature in one view (e.g., a left image) when viewed in another view (e.g., a right image). For example, at a horizontal position h in the left image_LAnd is located at a horizontal position h in the right image_RCan be represented as having h_L-h_RThe disparity of the pixels. Although alternative representations may be used, howeverDisparity data may also be represented as depth or "input Z" data, typically represented as [0,255 [ ]]8 bits of gray scale data in the range.

Depending on the coding format, the 3D + Z signal (112) may contain one or more layers, such as a base layer and one or more enhancement layers. The 3D + Z signal (112) is encoded by an encoder (120) to generate an encoded bitstream (122). The encoder (120) may be any of the known video encoders, such as those specified by the ISO/IEC MPEG-2, MPEG-4 part 2, MPEG-4, part 10(H.264/AVC) or HEVC standards, or other encoders (such as VP8, VC-1), and so forth. Prior to storage or transmission, the encoded bitstream (122) may be multiplexed with additional auxiliary data or metadata (not shown) to assist a suitable decoder in decoding and demultiplexing the stereoscopic image data and their corresponding depth data.

In a receiver such as a set-top box, television, etc., a decoder (130) (e.g., an MPEG-2 or h.264 decoder) may decode a bitstream encoded by the encoder (120) and generate a decoded 3D + Z signal that is a closer approximation of the transmitted 3D + Z signal (112) for lossy compression. The demultiplexer (140) extracts the depth map data (147) and one or more image views (142) and may pass them to subsequent processing, such as processing related to display management and display. Legacy receivers may ignore the depth data and the second view, thereby displaying only a single view as a conventional 2D image; yet other decoders may use all available information to regenerate one or more views of the 3D signal (105).

Fig. 2 shows an example of a single layer 3D + Z signaling format (200). The formatted 3D + Z signal (212) comprises a luminance or luminance component (212-Y) and a corresponding chrominance component (212-UV). For example, in some embodiments, the signal (212) may be encoded in a 4:2:0YUV format. In certain other embodiments, the signal (212) may be encoded in a 4:2:0YCbCr format. As shown in fig. 2, the chrominance components of the signal (212) may have a lower pixel resolution than the luminance components; however, all methods described herein are applicable to color formats where chroma may have the same resolution as the luma component (e.g., 4:4:4YCbCr or 4:4:4 YUV).

The transport format (200) may use asymmetric spatial multiplexing; that is, in a multiplexed picture combining image data (212-Y1) (e.g., left view (L)105-1) and associated depth data (212-YZ) (e.g., Z107), the resolution of the image data (e.g., left view (L)) and its associated depth map (e.g., Z107)_L) Are different.

In one embodiment, given a multiplexed input frame (e.g., 112) with a pixel resolution h × w (e.g., h 1080 and w 1920), a subsampled left view (L) may be allocated more pixels than its associated depth map. Thus, given a scale a, where 1> a ≧ 1/2, the original left view picture can be scaled (e.g., subsampled) to a size h × aw, while the depth map can be scaled to a size h × (1-a) w. This approach may result in a 3D picture that is sharper than symmetric left and right view pictures (e.g., when a ═ 1/2).

Optionally, additional depth data (e.g. Z)_L’And Z_L”) May also be embedded in the corresponding chroma component (e.g., 212-UV) of the encoded frame.

In an alternative embodiment, the size of the picture frame containing the image and depth map sections (e.g., 212-Y) may be larger than the active image size of the view of the input image; thus, scaling of the image data to fit within the image section may not be required. For example, given 1080 x 1920 image data (e.g., L), (212-Y) may have a width w greater than 1920 and a height h greater than 1080. The transport frame may also be padded with dummy data to conform to the size implementation of the encoded unit (e.g., macroblock) used by the encoder (120). For example, in an example embodiment, 8 pixel lines may be added to the 1080 high bits such that the total number of lines 1088 is a multiple of 16.

In an alternative embodiment, the multiplexed image frame (e.g., 112) may be divided in the vertical direction. Thus, the image data section (212-YI) may have a resolution of ha × w and the depth map data section (212-YZ) may have a resolution of h (1-a) × w.

In some embodiments, the image data section may be multiplexed to the right of the depth map data section.

In some embodiments, the image data section may be multiplexed to the bottom of the depth map data section.

In an embodiment, backward compatibility may be achieved by defining active regions of a picture (e.g., h × aw) using cut rectangle and aspect ratio syntax parameters (aspect ratio syntax parameters) in the encoded bitstream, similar to that defined in the AVC/H.264 or HEVC video coding standards_L). A 3D capable receiver may decode the entire picture, use the cropping parameters to determine the picture region and the depth map region, and then use the depth map information to render multiple views. The 3D receiver may use the received cropping and aspect ratio parameters to scale the 2D picture and depth as needed. Auxiliary data (or metadata) containing information about the picture layout on a per picture basis may also be transmitted.

In an example embodiment, if a — 2/3, given an input signal of 1080 × 1920, image data (e.g., L) may be scaled down in the horizontal dimension and encoded using a resolution of 1080 × 1280 pixels, while the depth component (e.g., Z) is encoded_L) In some embodiments, depth components may be scaled down and encoded at both horizontal and vertical resolutions at alternative resolutions that are less than the resolution of the available regions in the depth map segment, e.g., 540 × 640 or 360 ×The performance of (2) is shown in fig. 3.

Fig. 3 shows an example of a depth map transmission format according to an embodiment of the present invention. For simplicity, only the luminance component of the signal (112) is shown; however, similar pixel assignment may also be performed for the chroma components. As shown in FIG. 3, the image (212-Y) includes a scaled representation of the luminance component of one or more of the image views (e.g., L214) and a depth map of interest (e.g., Z)_L218) By way of example, for an input of 1080 × 1920, in one embodiment, the luminance of an image view may be scaled to a resolution of 1080 × 1280 (corresponding to a scaling of 2/3 in the horizontal direction), while the initial depth map may be scaled to a resolution of 540 × 960 (corresponding to a half-scaling in the horizontal and vertical directions). as shown in fig. 3, a 90 degree rotation of the depth map allows for a higher horizontal resolution depth map to be transmitted than the conventional format shown in fig. 2, which results in overall better and more accurate 3D picture quality at the receiver.

As shown in FIG. 3, due to scaling and rotation, certain pixel regions of the image frame (212-Y) may be unused. The region (e.g. 216) may be set to a fixed fill value (e.g. 128) or the image and depth data regions may be scaled appropriately so that their sum fills the entire region. For example, in an embodiment, the image region may have a 1080 × 1380 pixel resolution, while the depth map data region may have a 540 × 960 pixel resolution. In a preferred embodiment, to improve coding efficiency, the dimensions of the image and the depth map view size may be selected as a multiple of the coding unit size (e.g., 16 × 16) used by the encoder (120). Alternatively, the frame size of the transport format (212-Y) may be adjusted by adding dummy data to achieve a correspondence with the size of the encoded units (e.g., macroblocks) used by the encoder (120). For example, in an example embodiment, 8 pixel lines may be added to the 1080 high bits such that the total number of lines 1088 is a multiple of 16.

As shown in the example embodiments in FIGS. 4A and 4B, the rotated depth map may be used for a variety of other depth data transfersThe format shown in FIG. 4A is similar to the format shown in FIG. 3 except that the image portion (410) includes both a left view (L) and a right view (R) that are subsampled in the horizontal direction and stacked side-by-side_L) Is stored rotated in depth map data section (418) of 1080 × 480.

As shown in FIG. 4B, in another embodiment, the image portion (412) includes both a left view (L) and a right view (R) that are sub-sampled in the vertical and horizontal directions and stacked one above the other, hi an example embodiment, using 1080 × 1920 frames, each view may be stored with a resolution of 540 × 1440, while a depth map of 480 × 960 (e.g., Z_L) Is stored rotated in depth map data section (418) of 1080 × 480.

In fig. 5, a depth map data transfer format (520) shows another example of 3D + Z data transfer according to an embodiment. This format combines the rotation and optional segments of the depth map data. As an example, consider that initial image data (I) (510) and depth data (Z) (512) of 1080 × 1920 resolution would be multiplexed into a single 1080 × 1920 output frame (520). Like before, for simplicity, only multiplexing for luminance components is shown; however, a similar operation may also be performed on the chrominance components.

Using the principles of asymmetric spatial scaling, both image data and depth data may be scaled to produce scaled image data I_S(514) And scaled depth map data Z_S(516) For example, image I may be scaled in the horizontal direction at 3/4 to produce image I of 1080 × 1440_S(514) Whereas the depth data Z (512) may be scaled by a factor of 2 in both directions to produce a scaled depth map Z of 540 × 960_S(516). To make Z_SAdapted to depth map section (520-Z), (e.g. 1080 × (1920-_S(516) May be divided into two or more segments (e.g., 480 × 960ZA and 60 × 480ZB and ZC)The depth map section (520-Z) of the multiplexed image (520) may include: a rotated ZA depth segment stacked on top of the ZB and ZC depth segments.

The scaling of the depth data Z (512) may take into account the size of the assigned depth map section (520-Z) and any subsequent rotations and segmentation of the depth map section.

In some embodiments, the size of the image portion section (520-I) of the output multiplexed picture (520) may be equal to or larger than the size of the active input image data I (510), so the scaling of the image data (510) may be skipped.

In some embodiments, the image data section and the depth data section may be multiplexed by being vertically stacked one on top of the other.

The position, size and orientation of the depth segment in the output image (520) may be informed from the encoder to the decoder using auxiliary data or metadata. In the receiver, after decoding and demultiplexing the image data and depth data regions, the receiver may use the metadata to demultiplex the depth data segments (518) and reconstruct the unified depth map region (516), which may be scaled as needed to view the image data (510).

Example computer System implementation

Embodiments of the invention may be implemented using a computer system, a system configured as electronic circuits and components, an Integrated Circuit (IC) device such as a microcontroller, a Field Programmable Gate Array (FPGA) or another configurable or Programmable Logic Device (PLD), a discrete-time or Digital Signal Processor (DSP), an application specific IC (asic), and/or an apparatus comprising one or more such systems, devices, or components. The computer and/or IC may execute, control, or execute instructions related to encoding and decoding depth map transmission formats, such as those described herein. The computer and/or IC may calculate any of a variety of parameters or values related to encoding and decoding a depth map transport format as described herein. Image and video dynamic range extension embodiments may be implemented as hardware, software, firmware, and various combinations thereof.

Certain implementations of the invention include a computer processor executing software instructions that cause the processor to perform the method of the invention. For example, one or more processors in a display, encoder, set-top box, transcoder, etc. may implement the methods for encoding and decoding the depth map transport format described above by executing software instructions in a program memory accessible to the processor. The present invention may also be provided as a program product. The program product may comprise any medium carrying a set of computer-readable signals comprising instructions which, when executed by a data processor, cause the data processor to carry out the method of the invention. The program product according to the invention may be in any of a number of non-transitory forms. The program product may include, for example, physical media such as magnetic data storage media (including floppy disks, hard drives), optical data storage media including CD ROMs, DVDs, electronic data storage media including ROMs, flash RAMs, etc. The computer readable signal on the program product may optionally be compressed or encrypted.

When a component (e.g., a software module, processor, assembly, device, circuit, etc.) is referred to above, unless otherwise indicated, reference to the component (including a reference to a "means") should be interpreted as including as equivalents of the component and any component which performs the function of the described component (e.g., that is functionally equivalent), including components which are not structurally equivalent to the disclosed structure which performs the function in the illustrated exemplary embodiments of the invention.

Equivalents, extensions, alternatives, and others

Example embodiments related to encoding and decoding depth map transport formats are thus described. In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details that may vary from implementation to implementation. Thus, the claims (including any subsequent amendments) that follow, when granted hereon are the only and exclusive indications of what is the invention as claimed and what applicants intend. Any definitions expressly set forth herein for terms contained in such claims shall govern the meaning of such terms as used in the claims. Hence, no limitation, element, property, feature, advantage or attribute that is not expressly recited in a claim should limit the scope of such claim in any way. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

Claims (41)

1. A method for communicating 3D depth data, the method comprising:

accessing an input stereoscopic picture and input depth data associated with the input stereoscopic picture;

generating a segment of image data in a first orientation in response to an input stereoscopic picture;

generating a section of depth map data, wherein the section of depth map data has a different size than the section of image data and comprises two or more depth map segments generated in response to the input depth data, wherein generating the section of depth map data comprises:

scaling the input depth data to produce a scaled depth map;

segmenting the scaled depth map to produce two or more depth map segments;

rotating at least one of the depth segments relative to the first orientation to produce a rotated depth segment; and

multiplexing the at least one rotated depth segment and one or more of the remaining depth segments to form a depth map data section;

multiplexing the image data segment and the depth map data segment to form a multiplexed output picture; and

the multiplexed output pictures are encoded using an encoder to produce an encoded bitstream to be transmitted to a decoder.

2. The method of claim 1, wherein the image data segment comprises pixel data based on a first view of the input stereoscopic picture or a second view of the input stereoscopic picture.

3. The method of claim 1, wherein the image data segment comprises pixel data based on both a first view of the input stereoscopic picture and a second view of the input stereoscopic picture.

4. The method of claim 2, wherein the image data section is generated by up-down sampling a first view or a second view of the input stereoscopic picture in a horizontal direction or a vertical direction.

5. The method of claim 2, wherein the image data section is generated by up-down sampling either the first view or the second view of the input stereoscopic picture in both a vertical direction and a horizontal direction.

6. The method of claim 1, wherein generating the depth map data section comprises scaling the input depth data taking into account the size of the assigned depth map section (520-Z) and any subsequent rotations and segmentation of the depth map segments.

7. The method of claim 1, wherein the one or more remaining depth segments are not rotated.

8. The method of claim 3, wherein the image data section is generated by sampling at least one of the image views up and down in a horizontal direction or a vertical direction.

9. The method of claim 3, wherein the image data section is generated by up-down sampling at least one of the image views in both a horizontal direction and a vertical direction.

10. The method of any one of claims 1 to 5, wherein the first orientation is a horizontal orientation.

11. The method of any one of claims 1 to 5, wherein the at least one depth segment is rotated by 90 degrees.

12. The method of any of claims 1 to 5, wherein the image data section and the depth map data section are multiplexed as a side-by-side picture.

13. The method of any of claims 1 to 5, wherein the image data section and the depth map data section are multiplexed into a top-bottom picture.

14. The method of claim 3, wherein generating the image data segment further comprises:

downsampling the first view to create a downsampled first view;

downsampling the second view to create a downsampled second view; and

multiplexing the downsampled first view and the downsampled second view by stacking the downsampled first view and the downsampled second view in a side-by-side format or in a top-down format to produce the section of image data.

15. The method of claim 1, wherein the scaled depth map is segmented in either a horizontal direction or a vertical direction.

16. The method of claim 14 or 15, wherein multiplexing the rotated depth segment and one or more of the remaining depth segments to form a depth map data section comprises stacking the rotated depth segment and another depth segment vertically or horizontally.

17. A method for decoding 3D depth map data transmitted from an encoder, the method comprising:

decoding the encoded bitstream to produce a section of depth map data and a section of image data in a first orientation, wherein the section of depth map data comprises two or more depth map segments, at least one of the depth map segments being rotated with respect to the first orientation;

rotating the at least one rotated depth segment to produce at least one second depth segment having an orientation matching the first orientation; and

generating a decoded output signal at least in response to the section of image data, the at least one second depth segment and one or more of the remaining depth segments.

18. The method of claim 17, wherein the image data section comprises a scaled representation of a first view or a second view of the transmitted 3D signal.

19. The method of claim 17, wherein the image data section comprises a scaled representation of both the first view and the second view of the transmitted 3D signal.

20. The method of claim 17, wherein generating a decoded output signal comprises rendering a multi-view using the image data section, the at least one second depth segment, and the one or more of remaining depth segments.

21. An apparatus comprising a processor, the apparatus configured to perform the method of any of claims 1 to 20.

22. An apparatus for transmitting 3D depth data, the apparatus comprising:

means for accessing an input stereoscopic picture and input depth data associated with the input stereoscopic picture;

means for generating a segment of image data in a first orientation in response to an input stereoscopic picture;

means for generating a section of depth map data, wherein the section of depth map data has a different size than a section of image data and comprises two or more depth segments generated in response to the input depth data, wherein the means for generating a section of depth map data comprises:

means for scaling the input depth data to generate a scaled depth map;

means for segmenting the scaled depth map to produce two or more depth map segments;

means for rotating at least one of the depth segments relative to the first orientation to produce a rotated depth segment; and

means for multiplexing the at least one rotated depth segment and one or more of the remaining depth segments to form a depth map data section;

means for multiplexing the image data segment and the depth map data segment to form a multiplexed output picture; and

means for encoding the multiplexed output pictures using an encoder to produce an encoded bitstream to be transmitted to a decoder.

23. The apparatus of claim 22, wherein the image data segment comprises pixel data based on a first view of the input stereoscopic picture or a second view of the input stereoscopic picture.

24. The apparatus of claim 22, wherein the image data segment comprises pixel data based on both a first view of the input stereoscopic picture and a second view of the input stereoscopic picture.

25. The apparatus of claim 23, wherein the image data section is generated by up-down sampling a first view or a second view of the input stereoscopic picture in a horizontal direction or a vertical direction.

26. The apparatus of claim 23, wherein the image data section is generated by up-down sampling a first view or a second view of the input stereoscopic picture in both a vertical direction and a horizontal direction.

27. Apparatus as claimed in claim 22, wherein the means for generating the depth map data sections comprises means for scaling the input depth data taking into account the size of the assigned depth map section (520-Z) and any subsequent rotation and segmentation of the depth map sections.

28. The apparatus of claim 22, wherein the one or more remaining depth segments are not rotated.

29. The apparatus of claim 24, wherein the image data section is generated by sampling at least one of the image views up and down in a horizontal direction or a vertical direction.

30. The apparatus of claim 24, wherein the image data section is generated by up-down sampling at least one of the image views in both a horizontal direction and a vertical direction.

31. The apparatus of any one of claims 22 to 26, wherein the first orientation is a horizontal orientation.

32. The apparatus of any one of claims 22 to 26, wherein the at least one depth segment is rotated by 90 degrees.

33. The apparatus of any of claims 22 to 26, wherein the image data section and the depth map data section are multiplexed as a side-by-side picture.

34. The apparatus of any of claims 22 to 26, wherein the image data section and the depth map data section are multiplexed into a top-bottom picture.

35. The apparatus of claim 24, wherein the means for generating the image data segment further comprises:

means for downsampling the first view to create a downsampled first view;

means for downsampling the second view to create a downsampled second view; and

means for multiplexing the downsampled first view and the downsampled second view by stacking the downsampled first view and the downsampled second view in a side-by-side format or in a top-down format to produce the section of image data.

36. The apparatus of claim 22, wherein the scaled depth map is segmented in either a horizontal direction or a vertical direction.

37. The apparatus of claim 35 or 36, wherein the means for multiplexing the rotated depth segment and one or more of the remaining depth segments to form a depth map data section comprises means for stacking the rotated depth segment and another depth segment vertically or horizontally.

38. An apparatus for decoding 3D depth map data transmitted from an encoder, the apparatus comprising:

means for decoding an encoded bitstream to generate a section of depth map data and a section of image data in a first orientation, wherein the section of depth map data comprises two or more depth map segments, at least one of the depth map segments being rotated with respect to the first orientation;

means for rotating the at least one rotated depth segment to generate at least one second depth segment having an orientation matching the first orientation; and

means for generating a decoded output signal at least in response to the section of image data, the at least one second depth segment and one or more of the remaining depth segments.

39. The apparatus of claim 38, wherein the image data section comprises a scaled representation of a first view or a second view of the transmitted 3D signal.

40. The apparatus of claim 38, wherein the image data section comprises a scaled representation of both the first view and the second view of the transmitted 3D signal.

41. The apparatus of claim 38, wherein the means for generating a decoded output signal comprises means for rendering a multi-view using the image data segment, the at least one second depth segment, and the one or more of the remaining depth segments.