HK1126337A - Channel switch frame - Google Patents

Abstract

Methods and apparatus to process multimedia data enabling faster channel acquisitions, improved error recovery and improved efficiency. An encoder device encodes a first portion of multimedia data using inter-coding to generate a first version, and encodes the first portion of multimedia data using intra-coding to generate a second version. A decoder device receives a first version of a first portion of multimedia data, wherein the first version is inter-coded, receives a second version of the first portion of multimedia data, wherein the second version is intra-coded, and selectively decodes the first and second received versions.

Description

Channel switch frame

Claiming priority under 35U.S.C. § 119

This patent application claims priority to four U.S. provisional applications having the following serial numbers: 60/721,345 entitled "method of video compression" filed on 27/9/2005; 60/760,484 entitled "method for fast channel switching" filed on 20.1.2006; 60/789,442 entitled "channel switch frame" filed on 4/2006; and ____ entitled "channel switch frame" filed on 20.9.2006 (attorney docket No. 050843P 3). Each of the above provisional patent applications is assigned to the assignee of the present invention and is expressly incorporated herein by reference.

Technical Field

The present invention is directed to multimedia signal processing, and more particularly, to video encoding and decoding.

Background

Multimedia processing systems, such as video encoders, may encode multimedia data using encoding methods based on international standards, such as the Moving Picture Experts Group (MPEG) -1, MPEG-2, and MPEG-4 standards, the International Telecommunication Union (ITU) -T H.263 standard, and the ITU-T H.264 standard, and their corresponding versions ISO/IEC MPEG-4 part 10 (i.e., Advanced Video Coding (AVC)). These encoding methods are generally directed to compressing multimedia data for transmission and/or storage. Compression is broadly the process of removing redundancy from data.

A video signal may be described in terms of a series of pictures including a frame (a full picture) or a field (e.g., an interlaced video stream including fields with alternating odd or even rows of pictures). As used herein, the term "frame" refers to a picture, frame, or field. Video encoding methods compress a video signal by using lossless or lossy compression algorithms to compress each frame. Frame intra-coding (referred to herein as intra-coding) refers to encoding a frame using only one frame. Frame inter-coding (referred to herein as inter-coding) refers to encoding a frame based on other "reference" frames. For example, video signals often exhibit temporal redundancy, where frames that are close to each other in the temporal sequence of frames have portions that at least match each other or at least partially match each other.

A multimedia processor, such as a video encoder, may encode a frame by dividing the frame into blocks or "macroblocks" (e.g., 16 x 16 pixels). The encoder may further divide each macroblock into sub-blocks. Each subblock may further comprise additional subblocks. For example, the sub-blocks of a macroblock may include 16 x 8 and 8 x 16 sub-blocks. A sub-block of an 8 x 16 sub-block may comprise an 8 x 8 sub-block, and so on. As used herein, the term "block" refers to a macroblock or a subblock.

The encoder exploits this temporal redundancy using an algorithm based on inter-coding motion compensation. The motion compensation algorithm identifies portions of the reference frame that at least partially match a block. The block may be shifted in the frame relative to the matching portion of the reference frame. This shift is characterized by a motion vector. Any differences between the block and the partially matched portion of the reference frame may be characterized in terms of residuals. An encoder may encode a frame as data that includes one or more motion vectors and residuals used to specifically partition the frame. The particular partition of a block used to encode a frame may be selected by approximately minimizing a cost function (e.g., balancing the encoding size with distortion of the frame content resulting from encoding).

Inter-frame coding enables more compression efficiency than intra-frame coding. Inter-coding, however, can create problems when reference data (e.g., a reference frame or a reference field) is lost due to channel errors, etc. In addition to reference data loss due to errors, reference data may also be unavailable due to initial acquisition or re-acquisition of the video signal at inter-coded frames. In these situations, decoding of inter-coded data may not be possible or may result in poor errors and error propagation. These situations can result in loss of synchronization of the video streams. Independently decodable intra-coded frames are the most common form of frame that enables resynchronization of a video signal. The MPEG-x and h.26x standards use pictures called groups of pictures (GOPs) that contain an intra-coded frame (also called I-frame) and temporally predicted P-frames or bi-directionally predicted B-frames that reference the I-frame and/or other P-and/or B-frames within the GOP. Longer GOPs are required for increased compression, but shorter GOPs allow faster acquisition and resynchronization. Increasing the number of I-frames will allow faster acquisition and resynchronization, but at the cost of reduced compression. What is needed is a method that enables fast acquisition and resynchronization of video streams while maintaining compression efficiency.

Disclosure of Invention

The systems, methods, and devices of the present disclosure each have several aspects, no single one of which is solely responsible for its desirable attributes. Without limiting the scope of the invention as expressed by the claims which follow, its more prominent features will now be discussed briefly. After considering this discussion, and particularly after reading the section entitled "embodiments" one will understand how the sample features of this invention provide advantages that include faster channel acquisition, improved error recovery, and improved efficiency.

A method of processing multimedia data is provided. The method includes encoding a first portion of multimedia data using inter-coding to produce a first version; and encoding the first portion of multimedia data using intra-coding to produce a second version.

A multimedia data processor is provided. The processor includes a configuration to encode a first portion of multimedia data using inter-coding to produce a first version and to encode the first portion of multimedia data using intra-coding to produce a second version.

An apparatus for processing multimedia data is provided. The apparatus comprises: a first encoder to encode a first portion of multimedia data using inter-coding to produce a first version; and a second encoder to encode the first portion of multimedia data using intra-coding to produce a second version.

A machine-readable medium is provided. The machine-readable medium comprises instructions that when executed cause a machine to: encoding a first portion of multimedia data using inter-coding to produce a first version; and encoding the first portion of multimedia data using intra-coding to produce a second version.

A method of processing multimedia data is provided. The method includes receiving a first version of a first portion of multimedia data (wherein the first version is inter-coded); receiving a second version of the first portion of multimedia data, wherein the second version is intra-coded, and selectively decoding the first and second received versions.

A multimedia data processor is provided. The processor comprises a configuration to receive a first version of a first portion of multimedia data (wherein the first version is inter-coded); receiving a second version of the first portion of multimedia data, wherein the second version is intra-coded, and selectively decoding the first and second received versions.

An apparatus for processing multimedia data is provided. The apparatus comprises: a receiver to receive a first version of a first portion of multimedia data (wherein the first version is inter-coded) and to receive a second version of the first portion of multimedia data (wherein the second version is intra-coded); and a decoder to selectively decode the first and second received versions.

A machine-readable medium is provided. The machine-readable medium comprises instructions that when executed cause a machine to: receiving a first version of a first portion of multimedia data (wherein the first version is inter-coded); receiving a second version of the first portion of multimedia data, wherein the second version is intra-coded, and selectively decoding the first and second received versions.

A method for transmitting digital video data is provided. The method includes transmitting inter-coded digital video frames for each of a plurality of digital video channels and transmitting channel switch frames for each of the channels, wherein at least one of the channel switch frames includes intra-coded digital video data corresponding to one of the inter-coded frames of the respective channel.

A method for receiving digital video data is provided. The method includes obtaining an inter-coded frame comprising inter-coded digital video data associated with a first channel; a request to switch to a second channel is received and a channel switch frame is obtained in response to the request, the channel switch frame including intra-coded digital video data corresponding to one of inter-coded frames of the second channel.

Drawings

Fig. 1 is a block diagram illustrating a multimedia communication system according to one aspect.

FIG. 2A is a block diagram illustrating an embodiment of an encoder device that may be used in a system such as illustrated in FIG. 1.

FIG. 2B is a block diagram illustrating an embodiment of a decoder device that may be used in a system such as illustrated in FIG. 1.

Fig. 3A is a flow diagram illustrating an example of a method of encoding a portion of a video stream in a system such as illustrated in fig. 1.

Fig. 3B is a flow chart illustrating in more detail an example of step 202 of the method illustrated in fig. 3A.

Fig. 3C is a flow chart illustrating in more detail another example of step 202 of the method illustrated in fig. 3A.

Fig. 3D is a flow chart illustrating in more detail another example of step 202 of the method illustrated in fig. 3A.

Fig. 4 illustrates a portion of a video stream encoded using the method illustrated in fig. 3A and 3B.

Fig. 5 illustrates portions of a video stream encoded using the method illustrated in fig. 3A and 3C.

Fig. 6 illustrates portions of a video stream encoded using the method illustrated in fig. 3A and 3D.

Fig. 7A is a flow diagram illustrating one example of a method of decoding a portion of a video stream in a system such as illustrated in fig. 1.

FIG. 7B is a flow chart illustrating in more detail an example of a portion of the method illustrated in FIG. 7A.

FIG. 7C is a flow chart illustrating in more detail another example of a portion of the method illustrated in FIG. 7A.

FIG. 7D is a flow chart illustrating in more detail another example of a portion of the method illustrated in FIG. 7A.

Fig. 8A and 8B illustrate portions of a scalable video stream decoded using the method illustrated in fig. 7.

FIG. 9 is a block diagram illustrating another embodiment of an encoder device that may be used in a system such as illustrated in FIG. 1.

FIG. 10 is a block diagram illustrating another embodiment of a decoder device that may be used in a system such as illustrated in FIG. 1.

FIG. 11 is a block diagram illustrating another embodiment of an encoder device that may be used in a system such as illustrated in FIG. 1.

FIG. 12 is a block diagram illustrating another embodiment of a decoder device that may be used in a system such as illustrated in FIG. 1.

Fig. 13 is a block diagram illustrating an example of an apparatus for transmitting digital video data that may be used in a system such as illustrated in fig. 1.

Fig. 14 is a block diagram illustrating an example of an apparatus for receiving digital video data that may be used in a system such as illustrated in fig. 1.

Detailed Description

The following detailed description is directed to certain sample embodiments of the invention. However, the invention can be embodied in many different ways as defined and covered by the claims. In this description, reference is made to the drawings wherein like parts are designated with like numerals throughout.

A video signal may be characterized according to a series of pictures, frames, or fields. As used herein, the term "frame" is a broad term that may encompass a frame of a progressive (progressive) video signal or a field of an interlaced video signal.

Embodiments include systems and methods for improving processing in encoders and decoders in multimedia transmission systems. The multimedia data may include one or more of motion video, audio, still images, or any other suitable type of audiovisual data. Embodiments include apparatuses and methods of encoding video data. For example, one aspect includes a method of encoding first and second versions of a frame of video data, where one of the versions enables acquisition and/or resynchronization of a video stream. In particular, it has been found that, according to one aspect, a first version of inter-coded video data and a second version of intra-coded video data allow a decoding device to selectively decode the second version enabling acquisition (if appropriate). Other embodiments include apparatuses and methods of decoding video data encoded according to the above embodiments. For example, it has been found that, according to one aspect, selectively decoding an intra-coded version when reference data (on which the inter-coded version depends) is not available allows a decoding device to acquire and/or resynchronize a video stream. In particular, it has been found that user experience can be improved by providing an increased number of acquisition frames in a bit rate efficient manner according to one aspect without significantly affecting the quality perceived by the user.

Multimedia distribution system

Fig. 1 is a block diagram illustrating a multimedia communication system 100 according to one aspect. The system 100 includes an encoder device 110 in communication with a decoder device 150 via a network 140. In one example, the encoder device receives a multimedia signal from an external source 102 and encodes the signal for transmission over the network 140.

In this example, the encoder device 110 includes a processor 112 coupled to a memory 114 and a transceiver 116. The processor 112 encodes data from a multimedia data source and provides it to the transceiver 116 for communication over the network 140.

In this example, the decoder device 150 includes a processor 152 coupled to a memory 154 and a transceiver 156. The processor 152 may comprise one or more general purpose processors and/or digital signal processors. The memory 154 may include one or more solid state or disk-based storage devices. The transceiver 156 is configured to receive multimedia data over the network 140 and provide it to the processor 152 for decoding. In one example, the transceiver 156 includes a wireless transceiver. Network 140 may include one or more wired or wireless communication systems including one or more of an ethernet, telephone (e.g., POT), cable, power line, and fiber optic system, and/or a wireless system including one or more of a code division multiple access (CDMA or CDMA2000) communication system, a Frequency Division Multiple Access (FDMA) system, an orthogonal frequency division multiple access (OFDM) system, a Time Division Multiple Access (TDMA) system (e.g., GSM/GPRS (general packet radio service)/EDGE (enhanced data GSM environment), TETRA (terrestrial trunked radio) mobile phone system), a Wideband Code Division Multiple Access (WCDMA) system, a high data rate (1xEV-DO or 1xEV-DO gold medal multicast) system, an IEEE 802.11 system, a medflo system, a DMB system, or a DVB-H system.

FIG. 2A is a block diagram illustrating an embodiment of an encoder device 110 that may be used in a system, such as the system 100 illustrated in FIG. 1. In this embodiment, encoder 110 includes an inter-coding encoder element 118, an intra-coding encoder element 120, a reference data generator element 122, and a transmitter element 124. The inter-coded encoder 118 encodes inter-coded portions of temporal prediction (e.g., using motion compensated prediction) of video with reference to other portions of the video data located in other temporal frames. The intra-coded encoder 120 encodes independently decodable intra-coded portions of video without reference to other temporally located video data. In some embodiments, intra-coding encoder 120 may use spatial prediction to exploit redundancy in other video data located in the same temporal frame.

In one aspect, reference data generator 122 generates data indicating where intra-coded and inter-coded video data generated by encoders 120 and 118, respectively, are located. For example, the reference data may comprise identifiers of sub-blocks and/or macroblocks used by the decoder to locate a position within the frame. The reference data may also include a frame number used to locate a frame in the sequence of video frames.

Transmitter 124 transmits inter-coded data, intra-coded data, and (in some embodiments) reference data over a network, such as network 140 of fig. 1. Data may be transmitted over one or more communication links. The term communication link is used in a generic sense and may include any communication channel including, but not limited to, wired or wireless networks, virtual channels, and the like. In some embodiments, intra-coded data is transmitted over a base layer communication link and inter-coded data is transmitted over an enhancement layer communication link. In some embodiments, intra-coded data and inter-coded data are transmitted over the same communication link. In some embodiments, one or more of inter-coded data, intra-coded data, and reference data may be transmitted over a sideband communication link. For example, a sideband communication link such as a Supplemental Enhancement Information (SEI) message of H.264 or a user _ data message of MPEG-2 may be used. In some embodiments, one or more of intra-coded data, inter-coded data, and reference data are transmitted on a virtual channel. A virtual channel may include data packets that contain an identifiable packet header that identifies the data packet as if the data packet belongs to the virtual channel. Other forms of identifying virtual channels are known in the art, such as frequency division, time division, code spreading, and the like.

In some embodiments, one or more of the elements of encoder 110 of fig. 2A may be rearranged and/or combined. The elements may be implemented by hardware, software, firmware, middleware, microcode, or any combination thereof. Details of the actions performed by the elements of the encoder 110 will be discussed below with reference to the method illustrated in fig. 3.

Fig. 2B is a block diagram illustrating an embodiment of a decoder device 150 that may be used in a system, such as the system 100 illustrated in fig. 1. In this embodiment, decoder 150 includes a receiver element 158, a selective decoder element 160, a reference data determiner element 162, and one or more reference data availability detectors (e.g., a channel switch detector element 164 and an error detector element 166).

The receiver 158 receives encoded video data (e.g., data encoded by the encoder 110 of fig. 1 and 2A). The receiver 158 may receive the encoded data over a wired or wireless network, such as the network 140 of fig. 1. Data may be received over one or more communication links. In some embodiments, intra-coded data is received over a base layer communication link and inter-coded data is received over an enhancement layer communication link. In some embodiments, intra-coded data and inter-coded data are received over the same communication link. In some embodiments, one or more of inter-coded data, intra-coded data, and reference data may be received over a sideband communication link. For example, a sideband communication link such as a Supplemental Enhancement Information (SEI) message of H.264 or a user _ data message of MPEG-2 may be used. In some embodiments, one or more of intra-coded data, inter-coded data, and reference data are received on a virtual channel. A virtual channel may include data packets that contain an identifiable packet header that identifies the data packet as if the data packet belongs to the virtual channel. Other forms of identifying virtual channels are known in the art.

The selective decoder 160 decodes the received inter-coded and intra-coded video data. In some embodiments, the received data includes an inter-coded version of a portion of video data and an intra-coded version of the portion of video data. Inter-coded data may be decoded after decoding reference data on which prediction of inter-coded data depends. For example, data encoded using motion compensated prediction includes a motion vector and a frame identifier that identify the location of the reference data. The selective decoder 160 may decode an inter-coded version if the portion of the frame identified by the motion vector and frame identifier of the inter-coded version is available (e.g., has been decoded). However, if reference data is not available, the selective decoder 160 may decode the intra-coded version.

In one aspect, the reference data determiner 162 identifies received reference data that indicates where intra-coded and inter-coded video data in the received encoded video data is located. For example, the reference data may comprise identifiers of sub-blocks and/or macroblocks used by the selective decoder 160 to locate a position within the frame. The reference data may also include a frame number used to locate a frame in the sequence of video frames. Using this received reference data enables the decoder to determine whether reference data (on which inter-coded data depends) is available.

Reference data availability may be affected by a user switching a channel of a multi-channel communication system. For example, multiple video broadcasts may be used by the receiver 158. If the user commands the receiver 158 to change to a different broadcast channel, the reference data for the inter-coded data on the new channel may not be immediately available. The channel switch detector 164 detects whether a channel switch command has been issued and signals the selective decoder 160. The selective decoder 160 may then use the information obtained from the reference data determiner to identify whether reference data of the inter-coded version is unavailable, and then identify the location of the most recent intra-coded version and selectively decode the identified intra-coded version.

Reference data availability may also be affected by errors in the received video data. Error detector 166 may utilize error detection techniques (e.g., forward error correction) to identify uncorrectable errors in the bit stream. If there are uncorrectable errors in the reference data (on which the inter-coded version depends), the error detector 166 may signal to the selective decoder 160 which video data is affected by the error. The selective decoder 160 may then determine whether to decode an inter-coded version (e.g., if reference data is available) or to decode an intra-coded version (e.g., if reference data is not available).

In some embodiments, one or more of the elements of decoder 150 of fig. 2B may be rearranged and/or combined. The elements may be implemented by hardware, software, firmware, middleware, microcode, or any combination thereof. Details of the actions performed by the elements of decoder 110 will be discussed below with reference to the method illustrated in fig. 7.

Switching frame coding method

Fig. 3A is a flow diagram illustrating an example of a method of encoding a portion of a video stream in a system such as illustrated in fig. 1. In this example, the method generates multiple versions of a portion of multimedia data, where one of the versions allows at least partial acquisition and/or resynchronization of the portion of multimedia data. The version that allows partial retrieval is independently decodable, such as the intra-coded data discussed above. Predictive inter-coding may be used for one of the versions to take advantage of the increased compression efficiency of inter-coding. In one aspect, the method of fig. 3A provides an efficient method of providing random access points in a video bitstream. More frequent random access points allow for faster acquisition of the channel when the channel is initiated, e.g., when a user commands a receiving device to switch channels. More frequent random access points also allow for faster resynchronization if synchronization of the video bitstream is lost due to errors or unreliable channel conditions.

Referring to fig. 3A, the method 200 begins at step 202 where an encoding device generates first and second versions of a portion of multimedia data in step 202, where one of the versions enables at least partial acquisition. For purposes of illustration only, the second version will be assumed to enable partial fetching. In this example, partial acquisition refers to the ability to independently decode a second version of the portion of the frame without reference to other multimedia data (e.g., via an intra-coded frame or intra-coded blocks in the frame). If the portion of the frame represented by the second version is the entire frame, then full acquisition of the frame will be possible. Partial acquisition may refer to the ability to decode the data portion of a frame without reference to other previously decoded or to-be-decoded multimedia. For example, the intra-coding discussed above is independently decodable and thereby enables acquisition. The portion of multimedia data may be an entire frame or a field of a frame. In one example, the encoding device 110 of fig. 1 and/or 2A generates the first and second versions at step 202.

The method 200 continues at optional step 204, where the encoding device generates information relating to the first and second versions. In one aspect, the generated information includes data indicating that a second version of redundant multimedia data is available and data indicating where the portions of the multimedia data represented by the first and second versions are located. For example, the reference data may comprise identifiers of sub-blocks and/or macroblocks used by the decoder to locate a position within the frame. The reference data may also include a frame number used to locate a frame in the sequence of video frames. In one example, the reference data generator 122 in FIG. 2A generates information at step 204. In some embodiments, the action of optional step 204 in this example may be omitted. For example, a predetermined frame number and/or a predetermined sequence of macroblocks or subblocks may be periodically generated such that the decoder device knows in advance that the first and second versions are to be received for a certain frame portion at a certain time of the bitstream.

Moving to optional step 206, the encoding device transmits the first and second versions and, optionally, the reference data information generated in step 204. Data may be transmitted over one or more communication links. In some embodiments, the second version of the encoded data is transmitted over a base layer communication link (in this example the version enables acquisition), and the first version of the encoded data is transmitted over an enhancement layer communication link. In some embodiments, the first and second versions are transmitted over the same communication link. In some embodiments, one or more of the first version, the second version, and the reference data may be transmitted over a sideband communication link. In some embodiments, one or more of the first version, the second version, and the reference data are transmitted over a virtual channel. A virtual channel may include data packets that contain an identifiable packet header that identifies the data packet as if the data packet belongs to the virtual channel. Other forms of identifying virtual channels are known in the art. The transmitter 124 of fig. 2A may perform the transmitting action at optional step 206.

It is to be recognized that depending on the embodiment, certain acts or events of any of the methods described herein can be performed in a different order or can be added, merged, or left out all together (e.g., not all described acts or events apply to the performance of the methods). Further, in some embodiments, acts or events may be performed concurrently (e.g., via multi-threaded processing, interrupt processing, or multiple processors) rather than sequentially.

Fig. 3B is a flow chart illustrating in more detail an example of step 202 of the method illustrated in fig. 3A. In particular, step 202 of FIG. 3A is shown to include two steps 210 and 212. At step 210, a first version of the multimedia is generated by an encoder device using inter-coding. Inter-coding may be motion compensated prediction. The inter-frame encoding encoder 118 in fig. 2A may perform inter-frame encoding at step 210. At step 212, a second version of the multimedia is generated by the encoder device using intra-coding. The second version may be independently decoded thereby enabling acquisition and/or resynchronization. The intra-coding encoder 120 of fig. 2A may perform intra-coding at step 212.

Fig. 4 illustrates a portion of a video stream encoded using the method illustrated in fig. 3A and, in particular, fig. 3B. The example illustrates a portion of a source video sequence 400 that includes source frames labeled S1-S7. The source sequence 400 represents multimedia data being compressed by an encoder device performing the method 200 of fig. 3B. The example also contains an encoding sequence 410 that includes encoding frames labeled E1-E7. An encoded frame represents a compressed version of an individual source frame encoded by various inter-frame coding and/or intra-frame coding techniques. For example, the encoded frame E1 may be an intra-coded version of the source frame S1, such as may occur at the beginning of a video sequence or at a scene change. The other encoded frames in the encoded sequence 410 contain inter-coded portions that are predicted from the other frames in the sequence 410, as illustrated by arrows 412, 414, and 416. For example, frame E5, which represents source frame S5, is depicted as a P frame that includes motion vectors that point to the portion of encoded frame E3 illustrated by arrow 416. Frame E4, representing source frame S4, is illustrated as a B frame including motion vectors pointing to the portions of encoded frames E3 and E5 illustrated by arrows 414 and 412, respectively. It should be noted that a frame may contain more than one type of inter-coded and/or intra-coded portion. For example, the encoded frame E5 depicted as a P frame, indicated by arrow 416, may also contain portions that are intra-coded portions predicted from another frame and portions predicted from multiple frames (e.g., bi-directionally predicted portions).

The encoded frame sequence 410 may typically be decoded upon receipt of all reference frames upon which inter-coded portions depend. Errors introduced during transmission over the network may hinder or make impossible the decoding of inter-coded portions. Moreover, at any point (except for the encoded frame E1 of the intra-coded frame in this example), initial acquisition of sequence 410 (e.g., due to channel switching) may result in erroneous decoded multimedia data. To enable resynchronization of the encoded sequence 410 (e.g., due to erroneous reference data and/or initial acquisition), a second version of the source frame S3 is encoded (frame 420). In this example, frame 420 is the second version of the multimedia data encoded at step 212 using the method of fig. 3B, while encoded frame E3 is the first version of the multimedia data encoded at step 210. In this example, frame 420 is fully intra-coded and thus can be fully independently decoded. The decoder device may determine whether reference data is available for decoding the encoded frame E3, and if so, may discard the frame 420. However, if reference data is lost (upon which frame E3 depends), frame 420 may be decoded instead of encoded frame E3. This can be done for resynchronization of existing video sequences corrupted by erroneously received reference data, or for acquisition of a new channel. Without loss of generality, frames such as the frame representing the second version 420 (encoded in step 212 in this example) will be referred to herein as switch frames, with the understanding that switch frames may be used for purposes other than channel switching (e.g., stopping error propagation, enabling resynchronization, or allowing fast forward or reverse).

If the switch frame 420 is decoded instead of frame E3, a subsequent frame in the encoded sequence 410 that is dependent on frame E3 may be decoded with reference to frame 420. For example, the encoded frame E5 is a P frame that depends on frame E3, as indicated by arrow 416. Frame E5 may be decoded by reference to decoded switch frame 420, as illustrated by dashed arrow 422. Similarly, B frames E4 may be decoded by referencing switch frames 420 (as indicated by dashed arrow 424) and by referencing encoded frames E5 (as indicated by arrow 412). After decoding the frames in the encoded sequence 410 that depend on frame E3 (which, in this example, are decoded with reference to switch frame 420), the remaining frames in the sequence 410 may be decoded in the normal manner. The switch frame is encoded such that subsequent frames do not include references to any frames prior to the switch frame.

The encoding frame E3 is arbitrarily selected for explaining the example illustrated in fig. 4. Various methods of selecting switching frame positions in a video sequence may be used. Details of these methods will be discussed below.

The example illustrated in fig. 4 depicts the second version of the multimedia data encoded at step 212 of fig. 3B as comprising an entire frame. It should be noted, however, that the method 200 may be utilized in other examples, where only a portion of a frame is encoded into the first version and the second version. In these examples, the second version enables acquisition of a portion of the frame that includes the second version. The examples discussed below with reference to fig. 3C, 3D, 5, and 6 will illustrate this.

Fig. 3C is a flow chart illustrating in more detail another example of step 202 of the method illustrated in fig. 3A. In particular, step 202 of FIG. 3A is shown to include two steps 214 and 216. In step 214, a first version of the multimedia is generated by the encoder device using inter-coding for a first portion of the frame and using intra-coding for a second portion of the frame. Inter-coding may be motion compensated prediction. Inter-coding encoder 118 in fig. 2A may perform inter-coding at step 214, while intra-coding encoder 120 may perform intra-coding at step 214.

At step 216, a second version of the multimedia data is generated by the encoder device using intra-coding for the first portion. Accordingly, the second version of the first portion of the frame may be independently decoded, thereby enabling acquisition and/or resynchronization of at least the first portion of the switch frame. The intra-coding encoder 120 of fig. 2A may perform intra-coding at step 216. The encoder device may choose not to encode the one or more second portions of the frame because the intra-coded version is already available to the decoder in the first version encoded at step 214. However, the encoder may choose to encode the one or more second portions, such as to maintain conformance with a standard in which each region of the frame needs to be encoded. In some embodiments, the encoder may choose to encode the second portion of the frame using the smallest number of bits possible. This can be achieved by roughly encoding using quantization methods as are available, thus using the least number of bits to represent intra-coded coefficients. Other methods of encoding the second portion known to those skilled in the art may also be used. A detailed example of using the method of fig. 3C will now be discussed.

Fig. 5 illustrates a portion of a video stream encoded using the method illustrated in fig. 3A and, in particular, fig. 3C. The example illustrates a portion of a source video sequence 500 that includes source frames labeled S1-S7. The source sequence 500 represents the multimedia data being compressed by the encoder device performing the method 200 of fig. 3C. The example also contains an encoding sequence 510 that includes encoded frames labeled E1-E7. As discussed above with reference to fig. 4, the encoded sequence 510 includes I frames, P frames, and B frames. P-frames and B-frames are predicted from portions of other frames in the sequence 510, as indicated by arrows 512, 514, and 516. In this example, frame E3 represents the first version of the multimedia data encoded at step 214. Frame E3 includes an inter-coded first portion of the frame depicted by the portion labeled 526. Frame E3 also contains two intra-coded portions labeled 528.

The encoded frame sequence 510 may typically be decoded when all reference frames are received, depending in part on which inter-coded frames depend. Errors introduced during transmission over the network or during switching to a new video channel may hinder or make impossible the decoding of inter-coded portions. The switch frame 520 is used to enable resynchronization of the encoded sequence 510. The switch frame 520 is a second version of the multimedia data encoded at step 216 using the method of fig. 3C. In this example, the switch frame 520 includes one or more intra-coded portions 530 that can be independently decoded. The decoder device may determine whether reference data is available for decoding the encoded frame E3, and may discard the frame 520 if reference data is available. However, if reference data (upon which frame E3 depends) is lost, the intra-coded portion 530 of the switch frame 520 may be decoded instead of the corresponding inter-coded portion 526 of the encoded frame E3. The intra-coded portion 528 of the encoded frame E3 may also be decoded independently and combined with the decoded portion 530 of the switch frame 520 to form a complete decoded frame.

Frames subsequent to the frame formed by combining intra-coded portions 528 and 530 of frame E3 in encoded sequence 410 may be decoded with reference to combined portions 528 and 530. For example, the encoded frame E5 is a P frame that depends on frame E3, as indicated by arrow 416. Frame E5 may be decoded by reference to decoding switch portion 530 (as indicated by arrow 522) and by reference to decoding portion 528 from frame E3 (as indicated by arrow 516). Similarly, the B frame E4 may be decoded by referring to the decoding switch portion 530 (as indicated by arrow 524) and by referring to the decoding portion 528 from frame E3 (as indicated by arrow 514), and also by referring to the already decoded frame E5 (as indicated by arrow 512). After decoding the frames in the encoded sequence 510 that depend on frame E3 (which in this example are decoded with reference to switching portion 530 and/or intra-coded portion 528), the remaining frames in the sequence 510 may be decoded in the normal manner. The switch frame is encoded such that subsequent frames do not include references to any frames prior to the switch frame.

The switch frame 520 also contains a portion 532 corresponding to the portion 528 in the encoded frame E3. The portion 532 may be left unencoded at step 216 of generating the second version of the multimedia data. However, this would not be consistent with standards such as h.26x and MPEGx. In one example, the intra-coded portion 528 of frame E3 may be coarsely encoded while the parallel portion 532 of switch frame 520 may be finely encoded. In this way, the encoded frame E3 may still be encoded using a smaller number of bits. In this example, the coarsely encoded portion 528 in frame E3 may be replaced with the finely encoded portion 532. In another example, intra-coded portion 528 may be finely encoded while parallel portion 532 of switch frame 520 may be coarsely encoded. In this way, the switch frame 520 may be encoded using a smaller number of bits. In this example, the coarsely encoded portion 532 in the switch frame 520 may be replaced with the finely encoded portion 528 of the encoded frame E3. In both examples, in one aspect, the coarsely encoded portion may be intra-or inter-encoded using the fewest number of bits to represent all image regions and setting all coefficient values to zero. This will reduce the number of bits required and still allow both the switch frame 520 and the encoded frame E3 to conform to the standard.

In order for the intra-coded portion 528 of this example to be truly independently decodable, the portion should be intra-coded using a manner that is not dependent on the inter-coded portion 526 in frame E3. For example, some intra-coding utilizes spatial prediction that depends on neighboring blocks of pixels. If inter-coded portion 526 is dependent on another frame (which may not be available due to errors or due to channel switching conditions), portion 526 may not be decodable, and thus, any intra-coded portion 528 that is dependent on portion 526 will also not be properly decodable.

Fig. 3D is a flow chart illustrating in more detail another example of step 202 of the method illustrated in fig. 3A. In particular, step 202 of FIG. 3A is shown to include two steps 218 and 220. At step 218, a first version of the multimedia data is generated by the encoder device using inter-coding for the first portion of the first frame. Inter-coding may be motion compensated predictive. The inter-frame encoder 118 in fig. 2A may perform inter-frame encoding at step 218. The one or more second portions of the first frame may be encoded by inter-coding and/or intra-coding, but in this example, the type of encoding of the second portions of the first frame is not critical. Also at step 218, the one or more second portions are intra-coded in one or more subsequent frames in the sequence of video frames. Intra-coding encoder 120 may perform intra-coding at step 218.

At step 220, a second version of the multimedia data is generated by the encoder device using intra-coding for the first portion of the switch frame. Accordingly, the second version of the first portion of the switch frame may be independently decoded, thereby enabling acquisition and/or resynchronization of at least the first portion. The intra-coding encoder 120 of fig. 2A may perform intra-coding at step 220. The encoder device may choose not to encode the one or more second portions of the switch frame because the intra-coded version is available to the decoder in a subsequent frame to the first version encoded at step 214. However, the encoder may choose to encode the one or more second portions, such as to maintain conformance with a standard in which each region of the frame needs to be encoded. In some embodiments, the encoder may choose to encode the one or more second portions of the switch frame using the smallest number of bits possible. This can be achieved by roughly encoding using quantization methods as are available, thus using the least number of bits to represent intra-coded coefficients. Other methods of encoding the second portion known to those skilled in the art may also be used. A detailed example of using the method of fig. 3D will now be discussed.

Fig. 6 illustrates a portion of a video stream encoded using the method illustrated in fig. 3A and, in particular, fig. 3D. The example illustrates a portion of a source video sequence 600 including source frames labeled S1-S7. Source sequence 600 represents multimedia data being compressed by an encoder device performing method 200 of fig. 3D. The example also contains an encoding sequence 610 that includes the encoded frames labeled E1-E7. As discussed above with reference to fig. 4, the encoded sequence 610 includes I frames, P frames, and B frames. P-frames and B-frames are predicted from portions of other frames in the sequence 610, as indicated by arrows 612, 614, and 616. In this example, frame E3 represents the first version of the multimedia data encoded at step 218. Frame E3 contains the first version of the inter-coded first portion encoded at step 218, depicted by the portion labeled 626. Frame E3 also contains two second portions labeled 628A and 629A. For this example, second portions 628A and 629A are inter-coded.

The encoded frame sequence 610 also contains frames E5 and E6, in this example, frames E5 and E6 include intra-coded portions 628B and 629B, respectively. Intra-coded portions 628B and 629B represent the second portion that was intra-coded at step 218. The portions 628B and 629B are co-located in their respective regions of the frame with the portions 628A and 628B in frame E3.

The encoded frame sequence 610 may typically be decoded upon receipt of all reference frames upon which inter-coding depends in part. Errors introduced during transmission over the network or during switching to a new video channel may hinder or make impossible the decoding of inter-coded portions. The switch frame 620 may be used to enable resynchronization of the encoded sequence 610. The switch frame 620 contains the second version of the multimedia data encoded at step 220 using the method of fig. 3D. In this example, the first version in the switch frame 620 includes one or more intra-coded portions 630 that can be independently decoded. Portion 630 is co-located in a region of the frame with inter-coded portion 626 in frame E3. The decoder device may determine whether reference data is available for decoding the encoded frame E3, and may discard the frame 620 if reference data is available. However, if reference data (upon which frame E3 depends) is lost, the intra-coded portion 630 of the switch frame 620 may be decoded instead of the corresponding inter-coded portion 626 of the encoded frame E3. If the reference data (upon which the inter-coded portions 628A and 629A of the encoded frame E3 depend) is also not available to the decoder device, then the inter-coded portions may not be decoded. In this case, the decoding of the regions of the frame corresponding to portions 628A and 629A may be delayed or concealed (e.g., replaced) using the intra-coded regions 628B and 629B of later received frames E5 and E6 (which may be independently decoded and, in this example, combined with the decoded portion 630 of the switch frame 620 to form a complete decoded frame). In another example, the encoded portion 632 of the switch frame 620 may be decoded and displayed using the intra-encoded portion 630 of the switch frame 620.

The inter-coded portions of the frames following frame E3 and switch frame 620 may be decoded with reference to portions 630 and 632. The inter-coded portion of the reference portion 632 may exhibit some errors because the portion 632 may not have the same picture quality as the portions 628A and 629A that the portion 632 replaced. However, because portions 628B and 629B are in a commonly located portion of frames E5 and E6, the degraded image will be displayed for a short duration, only for as long as the 3 frames to be displayed. If the frame rate is thirty frames per second, then 3 frames take only one tenth of a second to be displayed. After decoding the frames in the encoded sequence 610 that depend on frame E3 (where, in this example, they are decoded with reference to switch frame 620), the remaining frames in the sequence 610 may be decoded in the normal manner. In another example where a channel switch condition has occurred and the switch frame 620 is the first frame to be displayed, the display of frames may be delayed until all intra-coded portions 630, 628B and 629B are available. In this case, the inter-coded portions of frames E4 and E5 may be decoded with reference to switch frame 620, and may then be used as reference data for other frames in encoded sequence 610, but are not necessarily displayed. The display of the first acquired frame may be delayed until frame E6 is decoded.

As discussed above with reference to portion 532 of fig. 5, the portion 632 may be left unencoded at step 220 of generating the second version of the multimedia data. However, this would not be consistent with standards such as h.26x and MPEGx. In one example, the portions 628A and 629A may be coarsely encoded while the parallel portion 632 of the switch frame 520 may be finely encoded. In this way, the encoded frame E3 may still be encoded using a smaller number of bits. In this example, the coarsely encoded portions 628A and 629A in frame E3 may be replaced with finely encoded portions 632. In another example, the intra-coded portions 628B and 629B may be finely encoded while the parallel portion 632 of the switch frame 620 may be coarsely encoded. In this way, the switch frame 620 may be encoded using a smaller number of bits. In this example, the coarsely encoded portion 632 in the switch frame 620 may be replaced with the finely encoded portion 628B of the encoded frame E5 and the finely encoded portion 629B of the encoded frame E6, respectively. In both examples, in one aspect, the coarsely encoded portion may be intra-or inter-encoded using the fewest number of bits to represent all image regions and setting all coefficient values to zero. This will reduce the number of bits required and still allow the switch frame 620 and the encoded frame E3 to conform to the standard. In the case where the inter-coded portion 632 in the switch frame 620 is used, some form of spatial error concealment may be used by the decoder device to conceal the inter-coded portion if it is not decodable without reference data. To prevent propagation of errors introduced due to inter-coded portions at frame E5 that refer to lower quality regions of the switch frame 620, the intra-coded portions 628B and 629B of this example, for example, may be intra-coded in a manner that is independent of the inter-coded portions of their respective frames E5 and E6.

The encoding methods of fig. 3A, 3B, 3C, and 3D are examples of methods used to encode channel switch frames that may be used to stop error propagation and enable efficient channel switching. Frequent occurrences of channel switch frames are desirable to stop error propagation and enable fast channel switching or acquisition. It is known that a channel switch delay of two seconds or less improves the overall user experience.

In one example, a channel switch frame is encoded at least once per second and is associated with a group of frames called superframes, where a superframe lasts one second. If the frame rate is 30 frames per second, a superframe will include 30 frames. However, a superframe may also include any number of frames and/or any length of time. For example, a superframe may include N frames, where N is an integer in the range of about 1 to about 100 (or more). In one aspect, the handover frame is located at the beginning of a superframe. In another aspect, if the encoder determines that a frame within a superframe is to be intra-coded (e.g., in the case of a scene change), such a frame may be designated as a switch frame. The encoder may restrict the superframe to have only one switch frame or may encode multiple switch frames in one superframe. To stop the switching frame from error propagation, the switching frame may be encoded as an IDR (instantaneous decoding update) picture, where no subsequent frame in the sequence references a frame in the sequence that precedes the switching frame. Accordingly, frames following the switch frame can be independently decoded from frames preceding the switch frame.

In another example, where the encoder determines that I frames are not needed within a superframe (e.g., due to a scene change), an additional or redundant frame is encoded as a switch frame. This redundant switch frame may be the first frame in a superframe. In some examples, the switch frame may be located at a position in the sequence of frames where a P-frame or B-frame contains a larger percentage of intra-coded portions (e.g., using the methods depicted in fig. 3C and 3D).

In another example, the switch frame is co-located with the first P-frame of the superframe. In one aspect, a first inter-coded P-frame and/or B-frame of a superframe may be encoded with reference to a frame in another superframe while all other inter-coded frames in the superframe are encoded so as not to reference data in other superframes. This approach enables high operating efficiency in decoding a video sequence regardless of whether there is an effective channel switch and/or regardless of whether there is an error in the previous superframe. When there is no channel switch event or detected error, it is often advantageous to decode and display only the first version of the inter-frame coding and discard the switch frame. Displaying an intra-coded frame every second may result in a pulsating effect of the displayed video. Details of this encoding method and related decoding methods, which relate to an example of scalable coding using a base layer and an enhancement layer, will be discussed below with reference to fig. 8. The decoding method associated with encoding the switch frame discussed above will now be discussed.

Switching frame decoding method

Fig. 7A is a flow diagram illustrating one example of a method 300 of decoding a portion of a video stream in a system such as illustrated in fig. 1. A decoder device, such as decoder device 150 in fig. 1 and 2B, may perform all or part of method 300. In step 302, a decoder device receives a first version of a first portion of multimedia data. In this example, the first version is an inter-coded version, such as produced by inter-coded encoder element 118 of encoder device 110 in fig. 2A. The inter-coded first version may comprise an entire frame of multimedia data or a partial frame such as a plurality of macroblocks and/or sub-blocks.

Proceeding to step 304, the decoder device receives a second version of the first portion of the multimedia data. In this example, the second version is an intra-coded version of the first portion of the multimedia data, such as generated by intra-coded encoder element 120 illustrated in fig. 2A. The second version may comprise an entire frame of multimedia data or a partial frame such as a plurality of macroblocks and/or sub-blocks. The second version is a switch frame as encoded by a method such as the exemplary method illustrated in fig. 3A, 3B, 3C, and 3D. Receiver element 158 of decoder device 150 illustrated in fig. 2B may perform the receiving acts of steps 302 and 304.

After receiving the first inter-coded version and the second intra-coded version of the first portion of the multimedia data, the decoder device selectively decodes the first and/or second version at step 306. In the normal decoding mode, the decoder device may successfully decode the first inter-coded version received at step 302. In this case, the second version received at step 304 may be discarded. However, in some cases, the decoder may not be able to decode some or all of the first version of the inter-coded data (e.g., after a user's channel switch request), and the decoder device may selectively decode some or all of the second version of the intra-coded multimedia data. The selective decoder element 160 of fig. 2B may be used to perform the selective decoding action of step 306.

FIG. 7B is a flow chart illustrating in more detail an example of a portion of the method illustrated in FIG. 7A. Specifically, an example of a method of performing the selective decoding action of step 306 is shown. The decoder device determines whether reference data (upon which the first inter-coded version depends) is available at decision step 308. If reference data is available, the method proceeds to step 310 and the decoder device decodes a first inter-coded version of the multimedia data. If the inter-coded first version is successfully decoded, the decoder device may discard the second version of the intra-coded data. If reference data is not available, such as when a user requests a channel switch, the method proceeds to step 312 and the decoder device decodes the second intra-coded version.

In one aspect, the decoder device may operate in a channel switch mode, where the decoder device performs initial acquisition of the video stream being received. In this mode, the decoder device may selectively decode the second version of the first portion of the multimedia data at step 312. Since the second version is intra-coded data, it can be decoded by the decoder device without any reference data from other frames. The first version of inter-coded data may not be decodable in this channel switch condition because it may depend on the reference multimedia data that is not available for decoding. The channel switch detector 164 of the decoder device 150 in fig. 2B may detect the channel switch command and thus cause the decoder device to look for the next switch frame.

In another aspect, the decoder device may use the error detector 166 shown in fig. 2B to detect errors in the reference data. The error detector 166 is configured to perform one or more of various error detection and/or correction schemes. The encoded data may include various schemes to provide error resilience. Error-prone channels, such as wireless and/or wired networks, may introduce errors into the bit stream received by the decoder. These error resilience schemes may include one or more error control coding schemes, interleaving schemes, and other schemes known to those skilled in the art. Error detector 166 includes corresponding error decoding components that enable detection and correction of errors. Some errors introduced via the network may not be correctable by error detector 166. For those errors that cannot be corrected, the decoder device determines which inter-coded multimedia data depends on the erroneous data and determines that the reference data is not available at decision step 308. The decoder device then looks for the next intra-coded version of the multimedia data in order to stop the propagation of errors caused by erroneous data.

The decoder device may determine whether reference data is available at the frame, slice, macroblock, and/or sub-block layers at decision step 308. For example, the decoder device may determine whether all reference data is available for the entire frame and decide to decode the second version if all reference data is not available. Similar decoding decisions may be made for individual slices (groups of one or more macroblocks), for individual macroblocks, and/or for individual sub-blocks. The reference data determiner element 162 of the decoder device 150 in fig. 2B may perform the action at decision step 308.

Fig. 7C is a flow chart illustrating in more detail another example of a portion of the method 300 illustrated in fig. 7A. The exemplary method 300 illustrated in fig. 7C may be used to decode a portion of video data that is encoded as using the method shown in fig. 3D and as graphically depicted in fig. 6. Referring to fig. 6 and 7C, the method 300 begins at step 302, where a decoder device receives a first inter-coded version of a first portion of a first frame of multimedia data, such as portion 626 of encoded frame E3. In this example, the portion 626 is located in the first frame of a superframe of multimedia data. However, this is an optional feature and the frame may be in any portion of the superframe, or generally in any location of the video sequence. Proceeding to step 304, the decoder device receives a second intra-coded version of a first portion of the first frame, such as portion 630 of switch frame 620 shown in fig. 6. In one aspect, the first portion of the first frame excludes one or more second portions of the first frame, such as portions 628A, 628B in the encoded frame E3 and portion 632 in the switch frame 620. As discussed above with reference to fig. 6, these second portions may be intra-coded or inter-coded portions in the encoded frame E3 and intra-coded portions in the switch frame 620.

Continuing to step 314, the decoder device receives an intra-coded version of one or more second portions in one or more other frames of the super-frame, such as portion 628B in frame E5 and portion 629B in frame E6, respectively. These intra-coded portions are coded so as to not depend on any of the inter-coded portions of the frame in which they are located, as discussed above. The independently decodable second intra-coded portion is used to limit error propagation and/or to initially acquire and/or reacquire the encoded video sequence 610 shown in fig. 6. Receiver element 158 shown in fig. 2B may perform the receiving acts of steps 302, 304, and 314.

After receiving the first and second versions of the first portion and the intra-coded version of the second portion, the process 300 continues at a selective decoding step 306. In the exemplary method 300 shown in fig. 7C, the selective decoding step 306 includes a decision step 308 and decoding steps 310 and 316. Decision step 308 functions in the same manner as in fig. 7B. If reference data (upon which the inter-coded first version of the first portion depends) is available, then the inter-coded first portion 626 is decoded at step 310. If some or all of the reference data is not available due to errors and/or initial acquisition of the encoded frame sequence 610, the decoder device decodes the second intra-coded version of the first portion (portion 630 of fig. 6) and decodes the intra-coded second portion 628B in frame E5 and the intra-coded second portion 629B in frame E6, respectively. In one aspect, the decoder also receives portions 632 of the switch frame 620 and decodes these portions as well. The intra-coded portion 630 of the switch frame 620 combines with the intra-coded portions 628B and 629B of frames E5 and E6 to form a complete intra-coded picture for the entire frame. Other portions of other frames in the encoded sequence 610 that depend on the encoded frame E3 are then decoded with reference to the decoded portion of the switch frame 620, as discussed above (see frames E4 and E5 and prediction arrows 622 and 624 in fig. 6). Thus, in this example, the entire frame is intra-coded at least once in the switch frame 620 and/or frames E5 and E6. This serves to enable acquisition of the encoded sequence 610 and limits error propagation. The reference data determiner element 162 in fig. 2B may determine whether reference data is available at decision step 308. The selective decoder element 160 may perform decoding actions at steps 310 and 316.

FIG. 7D is a flow chart illustrating in more detail another example of a portion of the method 300 illustrated in FIG. 7A. The exemplary method 300 illustrated in fig. 7D may be used to decode a portion of video data that is encoded as using the method shown in fig. 3C and as graphically depicted in fig. 5. Referring to fig. 5 and 7D, the method 300 begins at step 302, where a decoder device receives a first version of a first frame E3 of a superframe. The first version of the first frame E3 includes an inter-coded first portion 526 and one or more intra-coded second portions 528. In this example, portion 526 and portion 528 are in the first frame of a superframe of multimedia data. However, this is an optional feature and the frame may be in any portion of the superframe, or generally in any location of the video sequence. Proceeding to step 304, the decoder device receives a second intra-coded version of a first portion of the first frame, such as portion 530 of switch frame 520 shown in fig. 5. In one aspect, the second version of the first portion of the first frame excludes one or more second portions that are intra-coded in other frames, such as portions 628B, 629B in encoded frames E5 and E6 and portion 632 in switch frame 620. These second portions may also be intra-coded in the switch frame 620, as discussed above with reference to fig. 6.

After receiving the first and second versions of the first portion and the intra-coded version of the second portion, the process 300 continues at a selective decoding step 306. In the exemplary method 300 shown in fig. 7D, the selective decoding step 306 includes a decision step 308 and decoding steps 310 and 318. Decision step 308 functions in substantially the same manner as in fig. 7B and 7C. If reference data is available (the first version of the inter-coding of the first portion depends on the reference data), then the inter-coded first portion 526 is decoded at step 310. If some or all of the reference data is not available due to errors or initial acquisition of the encoded frame sequence 510, the decoder device decodes the second intra-coded version of the first portion (see portion 530 of fig. 5) and decodes the intra-coded second portion 528 in frame E3. In one aspect, the decoder also receives portions 532 of the switch frame 520 and decodes these portions or discards them. In one example, the portion 532 of the switch frame 520 is encoded using a minimum number of bits to maintain compliance with the standard as discussed above. The intra-coded portion 530 of the switch frame 520 is combined with the intra-coded portion 528 of the frame E3 to form a complete intra-coded image of the entire frame. Other portions of other frames in the encoded sequence 510 that depend on the encoded frame E3 are then decoded with reference to the decoded portion 530 of the switch frame 520 (and possibly the portion 532 if it is decoded) and the decoded portion 528 of the frame E3, as discussed above (see frames E4 and E5 and prediction arrows 522, 524, 514, and 516 in fig. 6). Thus, in this example, the entire frame is intra-coded at least once in switch frame 620 and/or frame E3. This serves to enable acquisition of the encoded sequence 510 and limits error propagation. The reference data determiner element 162 in fig. 2B may determine whether reference data is available at decision step 308. The selective decoder element 160 may perform decoding actions at steps 310 and 318.

In some examples, the decoder device receives a first inter-coded version (and optionally any intra-coded portions illustrated in fig. 5 and 6) on a first communication link and a second intra-coded switch frame version on a second communication link. One or both of these communication links may comprise a virtual channel, such as defined by packet header information or the like. In some examples, the first inter-coded version and the second intra-coded version are received over the same communication link. Examples of a pair of virtual channels that may be used to transmit and/or receive the first inter-coded and second intra-coded versions are a base layer and an enhancement layer of a scalable communication system.

Method for switching frames using base layer and enhancement layer

In some examples, a multimedia bitstream (e.g., a video bitstream) targeted for a single application may be encoded (e.g., using scalable encoding) into two or more separate layers (e.g., a base layer and an enhancement layer). These layers may then be used to provide scalability capabilities, e.g., temporal and/or SNR (signal-to-noise ratio) scalability capabilities. One example of scalable coding partitions an intra-coded picture (e.g., an I-frame) and a different inter-coded picture (e.g., a P-frame or a B-frame using, for example, motion compensated prediction) into different layers in the bitstream. I frames may be encoded in the base layer and P and/or B frames may be encoded in the enhancement layer. Scalable coding is useful in dynamic channels, where a scalable bitstream can be adapted to match fluctuations in network bandwidth. In error-prone channels, scalable coding may add robustness via unequal error protection of the base and enhancement layers. Better error protection can be applied to more important layers. Scalable coding may also be used to provide switching frames to limit error propagation and enable efficient channel acquisition.

FIG. 8A illustrates an example of a portion of the method illustrated in FIG. 7. The multimedia bitstream 800 includes a base layer 802 and an enhancement layer 804. The depicted portion of the multimedia bitstream 800 includes P frames and B frames (e.g., predicted using uni-directional and/or bi-directional motion compensated prediction). In one aspect of this example, the base layer data contains video at a certain level of quality for P-frames, B-frames, and I-frames (not shown), while the enhancement layer data contains improvements to the base layer data. P-frames and B-frames are predicted from other frames in the video sequence (e.g., B-frames are predicted from a previous frame and a subsequent frame and P-frames are predicted from a previous frame). The portion of the video bitstream 800 is on a boundary 806 that separates the first superframe SF i (see 808) from the second superframe SF i +1 (see 810). In this example, the first frame of the second superframe 810 is a B frame B1 and the second frame is a P frame P1. The B frame B1 may be assumed to be predicted from the last frame of the first superframe 808 and the P frame P1 in the second superframe.

The multimedia bitstream 800 also includes a switch frame 812 that is intra-coded as an I-frame. However, the switch frame 812 may be any of the switch frames, as discussed above with reference to fig. 3, 4, 5, and 6. In some examples, the switch frame 812 is transmitted and received on a different channel (real or virtual) than the base layer and enhancement layer data. The switch frame 812 may be transmitted and received in a sideband message. For example, switching frame 812 may be transmitted and received in one or more Supplemental Enhancement Information (SEI) messages of H.264 or a user data message of MPEG-2.

When the decoder receiving the multimedia bitstream 800 has successfully decoded the first superframe 808, B-frames and P-frames of the base layer 802 and the enhancement layer 804 may be normally decoded with reference to frames predicted therefrom in the encoding process. The switch frame 812 may be dropped if reference data for P and B frames is available. However, if the decoder has received erroneous data in the first superframe 808 or a channel switch request during the first superframe 808, the decoder may determine that it should use the switch frame 812. In this case, B frames B1 and P frames P1 in the base layer and enhancement layer are dropped or partially dropped, as discussed above with reference to fig. 5, 6, and 7. The switch frame is then decoded and a subsequent P frame P2 is decoded with reference to switch frame 812, as indicated by dashed arrow 814. B-frame B2 may also be decoded with reference to, for example, switch frame 812 and P-frame P2.

In one aspect, as discussed above, reference frames for the inter-coded portion of the superframe may be restricted. For example, the first B frame B1 and the first P frame P1 of the second superframe 810 may be predicted from a frame in another superframe (e.g., the first superframe 808). Because the switch frame 812 may be used in place of P1, and because frame B1 may be dropped (assuming no frames after B1 are predicted from B1, which is a common rule for B frames), encoding frames B1 and P1 by predicting from the first superframe 808 will not cause error propagation. However, in this aspect, the other inter-coded portions of the second superframe are restricted from being predicted from another superframe or any other frame preceding the switch frame 812, as indicated by the cross-headed arrows 816 and 818. Limiting the reference frames may affect the coding efficiency of frames B2, P2, and other subsequent frames in superframe 810. For example, frames B2 and P2 may use only P1 as a forward reference, while P3 and B3 may use P1 and P2 as forward references, thus giving a higher probability of finding a better matching block of multimedia data. The further away from the superframe boundary 806, the less impact will be. Thus, P2 and B2 would lose the maximum coding efficiency in this example.

In an aspect, to reduce the rate consumption of switch frames, the switch frames 812 may be encoded at a lower quality level than the enhancement layer frames 804. For example, if the average QP (quantization parameter) for the enhancement layer is QPenh, the QP for the channel switch frame may be QPenh +6, which is a lower quality QP. The lower quality switching frames may be imperceptible to the user due to their short duration. The embodiment shown in fig. 8A is not limited to the scalable approach shown, but may also be applied to a single layer scheme.

FIG. 8B illustrates another example of a portion of the method illustrated in FIG. 7. The multimedia bitstream 850 includes a base layer 852 and an enhancement layer 854. In this example, the first frame of the enhancement layer in the second superframe 856 is an intra-coded switch frame 858. In this example, the base layer P frames are encoded at a lower quality than the enhancement layer P frames. In the exemplary video sequence 850, base layer quality I and P frames may be decoded, or enhancement layer quality P frames may be decoded. For example, the decoder device may decode the lower quality base layer 852 to conserve battery power and may decode the higher quality enhancement layer 854 if a user requires higher quality. Typically, enhancement layer P frames have a higher quality than base layer P and I frames.

In this example, the switch frame 858 is used when the channel switch condition is valid and the decoder is receiving data in the first superframe 866, or when the first superframe 866 is received with erroneous reference data (the first P frame of the second superframe 856 depends on the reference data). If one of these occurs, the I frame 858 is decoded and the enhancement layer P frame 860 is discarded. In one aspect, as discussed above, the second and subsequent P frames in second superframe 850 may be encoded without reference to frames in another superframe (e.g., first superframe 866 in this example), as indicated by the cross-over arrows 862 and 864.

If switch frame 858 is not needed (e.g., all reference data is available when, for example, a channel switch is not requested), switch frame 858 can be dropped and the enhancement layer quality P-frame can be decoded in the normal manner. Alternatively, if only base layer quality is to be decoded, enhancement layer P frames may be dropped and base layer switch frame 858 and subsequent P frames may be decoded.

It should be noted that the position of the switch frame may be at other points in the superframe or video frame sequence (in general) than the beginning illustrated in the examples of fig. 8A and 8B. In some embodiments, the switch frame may be encoded at fixed time intervals. The fixed time interval may be in the range of about 0.5 seconds to about 2.0 seconds. In other embodiments, the switch frames may be at irregular time intervals in the video sequence. For example, the encoding process may choose to switch the position of the frame to conform to an inter-coded frame that has a higher percentage of the intra-coded blocks described above.

The examples depicted in fig. 8A and 8B limit the allowed prediction frames for frames following the first P and/or B frame in the superframe. In other examples, the motion vectors of the inter-coded portion of the frame may be scaled from the original reference frame in another superframe to correspond to the switch frame or other frame in the current superframe. An example of motion vector scaling will now be discussed.

Referring to fig. 8B, assume that the second P frame 868 in the second superframe 856 is predicted from the last frame 870 of the first superframe 866. If the channel switch frame 858 is decoded due to erroneous reference data or a channel switch mode, the motion vectors of the P-frame 868 may be scaled to correspond to the temporal location of the switch frame 858. Note that there will be some errors due to the non-linear motion of various image objects. The scaling of the motion vectors is characterized by the following:

MV_f_x＝MV_x*N/M；(1)

MV_f_y＝MV_y*N/M (2)

where MV _ f _ x and MV _ f _ y are the x and y components of the new motion vector pointing to the switch frame 858, MV _ x and MV _ y are the x and y components of the original motion vector pointing to the frame 870, N is the distance from the frame 868 to the switch frame 858, and M is the distance from the frame 868 to the original reference frame 870. In this example, N is 1 and M is 2, resulting in MV _ f _ x and MV _ f _ y vector components that are half of the original motion vector components MV _ x and MV _ y. Thus, frame 868 may be estimated from switch frame 858. Other forms of motion vector scaling using various distances N and M will be apparent to those skilled in the art.

FIG. 9 is a block diagram illustrating another embodiment of an encoder device that may be used in a system such as illustrated in FIG. 1. The exemplary encoder device 110 of fig. 9 includes: a first encoder device for encoding a first portion of multimedia data using inter-coding to produce a first version; and second encoder means for encoding the first portion of the multimedia data using intra-coding to produce the second version. The first encoder device may comprise the inter-coded decoder 905 of fig. 9. The second encoder device may comprise the intra-coding encoder 910 of fig. 9.

FIG. 10 is a block diagram illustrating another embodiment of a decoder device that may be used in a system such as illustrated in FIG. 1. The exemplary decoder device 150 of fig. 10 comprises means for receiving a first version of a first portion of multimedia data, wherein the first version is inter-coded, and receiving a second version of the first portion of multimedia data, wherein the second version is intra-coded. The decoder means 150 further comprises means for selectively decoding the first and second received versions. The means for receiving may comprise the receiver 1005 of fig. 10. The means for selectively decoding may include the selective decoder 1010 of fig. 10.

FIG. 11 is a block diagram illustrating another embodiment of an encoder device that may be used in a system such as illustrated in FIG. 1. The exemplary encoder device 110 of fig. 11 includes: a first encoder device for encoding a first portion of multimedia data using inter-coding to produce a first version; and second encoder means for encoding the first portion of the multimedia data using intra-coding to produce the second version. The first encoder device may comprise the module 1105 of fig. 11 that encodes using inter-coding. The second encoder device may comprise module 1110 of fig. 11 that encodes using intra-coding.

FIG. 12 is a block diagram illustrating another embodiment of a decoder device that may be used in a system such as illustrated in FIG. 1. The exemplary decoder device 150 of fig. 12 comprises means for receiving a first version of a first portion of multimedia data, wherein the first version is inter-coded, and receiving a second version of the first portion of multimedia data, wherein the second version is intra-coded. The decoder means 150 further comprises means for selectively decoding the first and second received versions. The means for receiving may comprise the module for receiving 1205 of fig. 12. The means for selectively decoding may include the module for selectively decoding 1210 of fig. 12.

Fig. 13 is a block diagram illustrating an example of an apparatus for transmitting digital video data that may be used in a system such as illustrated in fig. 1. The exemplary apparatus 1300 of fig. 13 comprises means for transmitting inter-coded digital video frames for each of a plurality of digital video channels and means for transmitting channel switch frames for each of the channels, wherein at least one of the channel switch frames includes intra-coded digital video data corresponding to one of the inter-coded frames of the respective channel. The means for transmitting inter-coded digital video frames may comprise a module for transmitting inter-coded digital video frames 1305. The means for transmitting the channel switch frame may include a module 1310 for transmitting the channel switch frame.

Fig. 14 is a block diagram illustrating an embodiment of an apparatus for receiving digital video data that may be used in a system such as illustrated in fig. 1. The exemplary apparatus 1400 of fig. 14 comprises means for obtaining an inter-coded frame comprising inter-coded digital video data associated with a first channel, means for receiving a request to switch to a second channel. The apparatus 1400 further comprises means for obtaining (in response to the request) a channel switch frame that includes intra-coded digital video data corresponding to one of the inter-coded frames of the second channel. The means for obtaining an inter-coded frame may comprise a module 1405 for obtaining an inter-coded frame. The means for receiving the request may include a module 1410 for receiving the request. The means for acquiring the channel switch frame may include a module 1415 for acquiring the channel switch frame.

Those of ordinary skill in the art would understand that information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

Those of ordinary skill would further appreciate that the various illustrative logical blocks, modules, and algorithm steps described in connection with the examples disclosed herein may be implemented as electronic hardware, firmware, computer software, middleware, microcode, or any combination thereof. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed methods.

The various illustrative logical blocks, components, modules, and circuits described in connection with the examples disclosed herein may be implemented or performed with a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

The steps of a method or algorithm described in connection with the examples disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an Application Specific Integrated Circuit (ASIC). The ASIC may reside in a wireless modem. In the alternative, the processor and the storage medium may reside as discrete components in a wireless modem.

The previous description of the disclosed examples is provided to enable any person skilled in the art to make or use the disclosed methods and apparatus. Various modifications to these examples will be readily apparent to those skilled in the art, and the principles defined herein may be applied to other examples and additional elements may be added.

Thus, a method and apparatus to perform temporal error concealment of erroneous bi-directionally predicted multimedia data has been described.

Claims (182)

1. A method of processing multimedia data, comprising:

encoding a first portion of multimedia data using inter-coding to produce a first version; and

encoding the first portion of the multimedia data using intra-coding to produce a second version.

2. The method of claim 1, further comprising generating information related to the encoded first and second versions.

3. The method of claim 2, wherein the generated information includes reference information indicating that the second intra-coded version is redundant.

4. The method of claim 1, wherein the first portion comprises a first frame of a superframe of the multimedia data.

5. The method of claim 4, further comprising encoding a second frame located after the first frame, wherein the second frame is encoded without reference to data in any previous superframe.

6. The method of claim 1, wherein the first portion comprises a portion of an entire frame that excludes one or more second portions and is located in a first frame of a superframe of the multimedia data, the method further comprising:

one or more of the second portions are encoded by intra-coding, wherein the one or more second portions are located in one or more second frames of the superframe.

7. The method of claim 1, wherein the first portion comprises a portion of an entire frame that excludes one or more second portions and is located in a first frame of a superframe of the multimedia data, the method further comprising:

encoding one or more of the second portions of the first frame by intra-coding for inclusion within the first version.

8. The method of claim 2, further comprising:

transmitting the first version;

transmitting the second version; and

and transmitting the information.

9. The method of claim 8, wherein transmitting the first version comprises transmitting on a first communication link and transmitting the second version comprises transmitting on a second communication link.

10. The method of claim 8, wherein the transmission of the first and second versions occurs over one communication link.

11. The method of claim 9, wherein transmitting on the first communication link or transmitting on the second communication link comprises transmitting on a virtual channel.

12. The method of claim 4, wherein generating the information indicating that the second version is redundant information comprises generating information identifying a frame sequence number of a frame containing the first portion.

13. The method of claim 1, wherein the first version is associated with a first level of quality and the second version is associated with a second level of quality that is lower than or equal to the first level of quality.

14. A multimedia data processor comprising a configuration to:

encoding a first portion of multimedia data using inter-coding to produce a first version; and

encoding the first portion of the multimedia data using intra-coding to produce a second version.

15. The processor of claim 14, wherein the processor is further configured to generate information related to the encoded first and second versions.

16. The processor of claim 15, wherein the generated information includes reference information indicating that the second intra-coded version is redundant.

17. The processor of claim 14, wherein the first portion comprises a first frame of a superframe of the multimedia data.

18. The processor of claim 17, wherein the processor is further configured to encode a second frame that follows the first frame, wherein the second frame is encoded without reference to data in any previous superframe.

19. The processor of claim 14, wherein the first portion comprises a portion of an entire frame that excludes one or more second portions and is located in a first frame of a superframe of the multimedia data, and the processor is further configured to encode one or more of the second portions by intra-coding, and further wherein the one or more second portions are located in one or more second frames of the superframe.

20. The processor of claim 14, wherein the first portion comprises a portion of an entire frame that excludes one or more second portions and is located in a first frame of a superframe of the multimedia data, and the processor is further configured to encode one or more of the second portions of the first frame using intra-coding for inclusion within the first version.

21. The processor of claim 15, wherein the processor is further configured to:

transmitting the first version;

transmitting the second version; and

and transmitting the information.

22. The processor of claim 21, wherein the processor is further configured to transmit the first version over a first communication link and the second version over a second communication link.

23. The processor of claim 21, wherein the first and second versions are transmitted over one communication link.

24. The processor of claim 22, wherein the first communication link or the second communication link comprises a virtual channel.

25. The processor of claim 17, wherein the information indicating that the second version is redundant information comprises information identifying a frame sequence number of a frame containing the first portion.

26. The processor of claim 14, wherein the first version is associated with a first quality level and the second version is associated with a second quality level that is lower than or equal to the first quality level.

27. An apparatus for processing multimedia data, comprising:

a first encoder to encode a first portion of multimedia data using inter-coding to produce a first version; and

a second encoder to encode the first portion of multimedia data using intra-coding to generate a second version.

28. The apparatus of claim 27, further comprising a generator to generate information related to the encoded first and second versions.

29. The apparatus of claim 28, wherein the generated information comprises reference information indicating that the second intra-coded version is redundant.

30. The apparatus of claim 27, wherein the first portion of multimedia data comprises a first frame of a superframe of the multimedia data.

31. The apparatus of claim 30, wherein the first encoder encodes multimedia data of a second frame that follows the first frame, wherein the second frame of multimedia data is encoded without reference to data in any previous superframe.

32. The apparatus of claim 27, wherein the first portion comprises a portion of an entire frame that excludes one or more second portions and is located in a first frame of a superframe of the multimedia data, and the second encoder encodes one or more of the second portions using intra-coding, and further wherein the one or more second portions are located in one or more second frames of the superframe.

33. The apparatus of claim 27, wherein the first portion comprises a portion of an entire frame that excludes one or more second portions and is located in a first frame of a superframe of the multimedia data, and the second encoder encodes one or more of the second portions of the first frame using intra-coding for inclusion within the first version.

34. The apparatus of claim 28, further comprising a transmitter to transmit the first version, the second version, and the generated information.

35. The apparatus of claim 34, wherein the transmitter transmits the first version over a first communication link and transmits the second version over a second communication link.

36. The apparatus of claim 34, wherein the first and second versions are transmitted over one communication link.

37. The apparatus of claim 35, wherein the first communication link or the second communication link comprises a virtual channel.

38. The apparatus of claim 30, wherein the information indicating that the second version is redundant information comprises information identifying a frame sequence number of a frame containing the first portion.

39. The apparatus of claim 27, wherein the first version is associated with a first level of quality and the second version is associated with a second level of quality that is lower than or equal to the first level of quality.

40. An apparatus for processing multimedia data, comprising:

a first encoder device for encoding a first portion of multimedia data using inter-coding to produce a first version; and

second encoder means for encoding the first portion of multimedia data using intra-coding to produce a second version.

41. The apparatus of claim 40, further comprising means for generating information related to the encoded first and second versions.

42. The apparatus of claim 41, wherein the information comprises reference information indicating that the second intra-coded version is redundant.

43. The apparatus of claim 40, wherein the first portion comprises a first frame of a superframe of the multimedia data.

44. The apparatus of claim 43, wherein the first encoder means encodes a second frame that follows the first frame, wherein the second frame is encoded without reference to data in any previous superframe.

45. The apparatus of claim 40, wherein the first portion comprises a portion of an entire frame that excludes one or more second portions and is located in one frame of a superframe of the multimedia data, and the second encoder device encodes one or more of the second portions using intra-coding, wherein the one or more second portions are located in one or more second frames of the superframe.

46. The apparatus of claim 40, wherein the first portion comprises a portion of an entire frame that excludes one or more second portions and is located in a first frame of a superframe of the multimedia data, and the second encoder means encodes one or more of the second portions of the first frame using intra-coding for inclusion within the first version.

47. The apparatus of claim 41, further comprising means for transmitting the first version, the second version, and the related information.

48. The apparatus of claim 47, wherein the means for transmitting transmits the first version over a first communication link and transmits the second version over a second communication link.

49. The apparatus of claim 47, wherein the means for transmitting transmits the first and second versions over one communication link.

50. The apparatus of claim 48, wherein the first communication link or the second communication link comprises a virtual channel.

51. The apparatus of claim 41, wherein the means for generating generates information identifying a frame sequence number of a frame containing the first portion.

52. The apparatus of claim 40, wherein the first version is associated with a first level of quality and the second version is associated with a second level of quality that is lower than or equal to the first level of quality.

53. A machine-readable medium comprising instructions that upon execution cause a machine to:

encoding a first portion of multimedia data using inter-coding to produce a first version; and

the first portion of multimedia data is encoded using intra-coding to produce a second version.

54. The machine-readable medium of claim 53, further comprising instructions that upon execution cause a machine to generate information relating to the encoded first and second versions.

55. The machine-readable medium of claim 54, wherein the generated information includes reference information indicating that the second intra-coded version is redundant.

56. The machine-readable medium of claim 53, wherein the first portion comprises a first frame of a superframe of the multimedia data.

57. The machine-readable medium of claim 56, further comprising instructions that upon execution cause a machine to encode a second frame that follows the first frame, wherein the second frame is encoded without reference to data in any previous superframe.

58. The machine-readable medium of claim 53, wherein the first portion comprises a portion of an entire frame that excludes one or more second portions and is located in a first frame of a superframe of the multimedia data, and further comprising instructions that when executed cause a machine to:

one or more of the second portions are encoded by intra-coding, wherein the one or more second portions are located in one or more second frames of the superframe.

59. The machine-readable medium of claim 53, wherein the first portion comprises a portion of an entire frame that excludes one or more second portions and is located in a first frame of a superframe of the multimedia data, and further comprising instructions that when executed cause a machine to:

encoding one or more of the second portions of the first frame by intra-coding for inclusion within the first version.

60. The machine-readable medium of claim 54, further comprising instructions that when executed cause a machine to:

transmitting the first version;

transmitting the second version; and

and transmitting the information.

61. The machine-readable medium of claim 60, further comprising instructions that when executed cause a machine to:

transmitting the first version over a first communication link; and

transmitting the second version over a second communication link.

62. The machine-readable medium of claim 60, wherein the transmission of the first and second versions occurs over one communication link.

63. The machine-readable medium of claim 61, wherein the transmission on the first communication link or the transmission on the second communication link comprises transmitting on a virtual channel.

64. The machine-readable medium of claim 56, wherein the information indicating that the second version is redundancy information comprises information identifying a frame sequence number of a frame containing the first portion.

65. The machine-readable medium of claim 53, wherein the first version is associated with a first level of quality and the second version is associated with a second level of quality that is lower than or equal to the first level of quality.

66. A method of processing multimedia data, comprising:

receiving a first version of a first portion of multimedia data, wherein the first version is inter-coded;

receiving a second version of the first portion of multimedia data, wherein the second version is intra-coded; and

selectively decoding the first and second received versions.

67. The method of claim 66, wherein selectively decoding further comprises:

determining whether reference data is available, the reference data being associated with the first inter-coded version; and

decoding the second intra-coded version based on the determination.

68. The method of claim 66, wherein selectively decoding further comprises:

determining whether reference data is available, the reference data being associated with the first inter-coded version; and

decoding the second intra-coded version in response to determining that the reference data is not available.

69. The method of claim 68, wherein determining that reference data is unavailable comprises detecting a channel switch command, and determining that the reference data is unavailable in response to detecting the channel switch command.

70. The method of claim 68, wherein determining that reference data is unavailable comprises detecting an error in the reference data, and determining that the reference data is unavailable in response to detecting the error, the first inter-coded version being dependent on the reference data.

71. The method of claim 66, wherein the first portion comprises a first frame of a superframe of the multimedia data.

72. The method of claim 71, further comprising receiving a second frame of the superframe, wherein the second frame is subsequent to the first frame and is encoded without reference to data in any previous superframe.

73. The method of claim 66, wherein the first portion is part of an entire frame that excludes one or more second portions and is located in a first frame of a superframe of the multimedia data, and receiving the first version further comprises receiving one or more of the second portions that are intra-coded, wherein the one or more intra-coded second portions are located in one or more second frames of the superframe.

74. The method of claim 73, wherein selectively decoding further comprises:

decoding the intra-coded first portion of the second version of the multimedia data;

decoding the one or more intra-coded second portions of the first version of the multimedia data; and

the decoded first portion and the one or more decoded second portions are combined for display in a single frame.

75. The method of claim 66, wherein the first portion is a portion of an entire frame that excludes one or more second portions and is located in a first frame of a superframe of the multimedia data, and receiving the first version further comprises receiving one or more of the second portions of the first frame that are intra-coded.

76. The method of claim 75, wherein selectively decoding further comprises:

decoding the intra-coded first portion of the second version of the multimedia data;

decoding the one or more intra-coded second portions of the first version of the multimedia data; and

the decoded first portion and the one or more decoded second portions are combined for display in a single frame.

77. The method of claim 66, further comprising receiving information related to the received first and second versions of the multimedia data.

78. The method of claim 66, wherein receiving the first version comprises receiving over a first communication link, and wherein receiving the second version comprises receiving over a second communication link.

79. The method of claim 78, wherein receiving on the first communication link or receiving on the second communication link comprises receiving a virtual channel.

80. The method of claim 66, wherein the first and second versions are received over a single communication link.

81. A multimedia data processor comprising a configuration to:

receiving a first version of a first portion of multimedia data, wherein the first version is inter-coded;

receiving a second version of the first portion of multimedia data, wherein the second version is intra-coded; and

selectively decoding the first and second received versions.

82. The processor of claim 81, wherein the processor is further configured to:

determining whether reference data is available, the reference data being associated with the first inter-coded version; and selectively decoding the second intra-coded version based on the determination.

83. The processor of claim 81, wherein the processor is further configured to:

determining whether reference data is available, the reference data being associated with the first inter-coded version; and

selectively decode the second intra-coded version in response to determining that the reference data is not available.

84. The processor of claim 83, wherein the processor is further configured to:

detecting a channel switching command; and

determining that the reference data is unavailable in response to detecting the channel switch command.

85. The processor of claim 83, wherein the processor is further configured to:

detecting an error in the reference data, the first inter-coded version being dependent on the reference data; and

determining that the reference data is unavailable in response to detecting the error.

86. The processor of claim 81, wherein the first portion comprises a first frame of a superframe of the multimedia data.

87. The processor of claim 86, wherein the processor is further configured to receive a second frame of the superframe, wherein the second frame follows the first frame and is encoded without reference to data in any preceding superframe.

88. The processor of claim 81, wherein the first portion is part of an entire frame that excludes one or more second portions and is located in a first frame of a superframe of the multimedia data, and the processor is further configured to receive one or more of the second portions that are intra-coded, further wherein the one or more intra-coded second portions are located in one or more second frames of the superframe.

89. The processor of claim 88, wherein the processor is further configured to:

selectively decode the intra-coded first portion of the second version of the multimedia data;

selectively decode the one or more intra-coded second portions of the first version of the multimedia data; and

the decoded first portion and the one or more decoded second portions are combined for display in a single frame.

90. The processor of claim 81, wherein the first portion is a portion of an entire frame that excludes one or more second portions and is located in a first frame of a superframe of the multimedia data, and the processor is further configured to receive one or more of the second portions of the first frame that are intra-coded.

91. The processor of claim 90, wherein the processor is further configured to:

selectively decode the intra-coded first portion of the second version of the multimedia data;

selectively decode the one or more intra-coded second portions of the first version of the multimedia data; and

the decoded first portion and the one or more decoded second portions are combined for display in a single frame.

92. The processor of claim 81, wherein the processor is further configured to receive information relating to the received first and second versions of the multimedia data.

93. The processor of claim 81, wherein the processor is further configured to:

receiving the first version over a first communication link; and

receiving the second version includes receiving over a second communication link.

94. The processor of claim 93, wherein the first communication link or the second communication link comprises a virtual channel.

95. The processor of claim 81, wherein the processor is configured to receive the first and second versions over a single communication link.

96. An apparatus for processing multimedia data, comprising:

a receiver to receive a first version of a first portion of multimedia data and to receive a second version of the first portion of multimedia data, wherein the first version is inter-coded, and wherein the second version is intra-coded; and

a decoder to selectively decode the first and second received versions.

97. The apparatus of claim 96, further comprising:

a determiner to determine whether reference data is available, the reference data being associated with the first inter-coded version; and is

Wherein the decoder selectively decodes the second intra-coded version based on the determination.

98. The apparatus of claim 96, further comprising:

a determiner to determine whether reference data is available, the reference data being associated with the first inter-coded version; and is

Wherein the decoder selectively decodes the second intra-coded version in response to determining that the reference data is not available.

99. The apparatus of claim 98, further comprising:

a detector to detect a channel switch command; and is

Wherein the determiner determines that the reference data is unavailable in response to detecting the channel switch command.

100. The apparatus of claim 98, further comprising:

a detector to detect an error in the reference data, the first inter-coded version being dependent on the reference data; and is

Wherein the determiner determines that the reference data is unavailable in response to detecting the error.

101. The apparatus of claim 96, wherein the first portion comprises a first frame of a superframe of the multimedia data.

102. The apparatus of claim 101, wherein the receiver receives a second frame of the superframe, wherein the second frame is subsequent to the first frame and is encoded without reference to data in any preceding superframe.

103. The apparatus of claim 101, wherein the first portion is part of an entire frame that excludes one or more second portions and is located in a first frame of a superframe of the multimedia data, and the receiver receives one or more of the second portions that are intra-coded, and further wherein the one or more intra-coded second portions are located in one or more second frames of the superframe.

104. The apparatus of claim 103, wherein the decoder is configured to:

selectively decode the intra-coded first portion of the second version of the multimedia data;

selectively decode the one or more intra-coded second portions of the first version of the multimedia data; and

the decoded first portion and the one or more decoded second portions are combined for display in a single frame.

105. The apparatus of claim 96, wherein the first portion is a portion of an entire frame that excludes one or more second portions and is located in a first frame of a superframe of the multimedia data, and the receiver receives one or more of the second portions of the first frame that are intra-coded.

106. The apparatus of claim 105, wherein the decoder is configured to:

selectively decode the intra-coded first portion of the second version of the multimedia data;

selectively decode the one or more intra-coded second portions of the first version of the multimedia data; and

the decoded first portion and the one or more decoded second portions are combined for display in a single frame.

107. The apparatus of claim 96, wherein the receiver receives information related to the received first and second versions of the multimedia data.

108. The apparatus of claim 96, wherein the receiver is configured to:

receiving the first version over a first communication link; and

receiving the second version includes receiving over a second communication link.

109. The apparatus of claim 108, wherein the first communication link or the second communication link comprises a virtual channel.

110. The apparatus of claim 96, wherein the receiver receives the first and second versions over a single communication link.

111. An apparatus for processing multimedia data, comprising:

means for receiving a first version of a first portion of multimedia data and receiving a second version of the first portion of multimedia data, wherein the first version is inter-coded, and wherein the second version is intra-coded; and

means for selectively decoding the first and second received versions.

112. The apparatus of claim 111, further comprising:

means for determining whether reference data is available, the reference data being associated with the first inter-coded version; and is

Wherein the means for selectively decoding decodes the second intra-coded version based on the determination.

113. The apparatus of claim 111, further comprising:

means for determining whether reference data is available, the reference data being associated with the first inter-coded version; and is

Wherein the means for selectively decoding decodes the second intra-coded version in response to determining that the reference data is unavailable.

114. The apparatus of claim 113, further comprising:

means for detecting a channel switch command; and is

Wherein the means for determining determines that the reference data is unavailable in response to detecting the channel switch command.

115. The apparatus of claim 113, further comprising:

means for detecting an error in the reference data, the first inter-coded version being dependent on the reference data; and is

Wherein the means for determining determines that the reference data is unavailable in response to detecting the error.

116. The apparatus of claim 111, wherein the first portion comprises a first frame of a superframe of the multimedia data.

117. The apparatus of claim 116, wherein the means for receiving receives a second frame of the superframe, wherein the second frame is subsequent to the first frame and is encoded without reference to data in any preceding superframe.

118. The apparatus of claim 111, wherein the first portion is part of an entire frame that excludes one or more second portions and is located in a first frame of a superframe of the multimedia data, and the means for receiving receives one or more of the second portions that are intra-coded, wherein the one or more intra-coded second portions are located in one or more second frames of the superframe.

119. The apparatus of claim 118, wherein the means for selectively decoding decodes the intra-coded first portion of the second version of the multimedia data and decodes the one or more intra-coded second portions of the first version of the multimedia data, and combines the decoded first portion and the one or more decoded second portions for display in a single frame.

120. The apparatus of claim 111, wherein the first portion is a portion of an entire frame that excludes one or more second portions and is located in a first frame of a superframe of the multimedia data, and the means for receiving receives one or more of the second portions of the first frame that are intra-coded.

121. The apparatus of claim 120, wherein the means for selectively decoding decodes the intra-coded first portion of the second version of the multimedia data, decodes the one or more intra-coded second portions of the first version of the multimedia data, and combines the decoded first portion and the one or more decoded second portions for display in a single frame.

122. The apparatus of claim 111, wherein the means for receiving receives information related to the first and second versions of the multimedia data.

123. The apparatus of claim 111, wherein the means for receiving receives the first version over a first communication link and receives the second version over a second communication link.

124. The apparatus of claim 123, wherein the first communication link or the second communication link comprises a virtual channel.

125. The apparatus of claim 111, wherein the means for receiving receives the first and second versions over a single communication link.

126. A machine-readable medium comprising instructions that upon execution cause a machine to:

receiving a first version of a first portion of multimedia data, wherein the first version is inter-coded;

receiving a second version of the first portion of multimedia data, wherein the second version is intra-coded; and

selectively decoding the first and second received versions.

127. The machine-readable medium of claim 126, further comprising instructions that when executed cause a machine to:

determining whether reference data is available, the reference data being associated with the first inter-coded version; and

selectively decoding the second intra-coded version based on the determination.

128. The machine-readable medium of claim 126, further comprising instructions that when executed cause a machine to:

determining whether reference data is available, the reference data being associated with the first inter-coded version; and

selectively decode the second intra-coded version in response to determining that the reference data is not available.

129. The machine-readable medium of claim 128, further comprising instructions that when executed cause a machine to:

detecting a channel switching command; and

determining that the reference data is unavailable in response to detecting the channel switch command.

130. The machine-readable medium of claim 128, further comprising instructions that when executed cause a machine to:

detecting an error in the reference data, the first inter-coded version being dependent on the reference data; and determining that the reference data is unavailable in response to detecting the error.

131. The machine-readable medium of claim 126, wherein the first portion comprises a first frame of a superframe of the multimedia data.

132. The machine-readable medium of claim 131, further comprising instructions that upon execution cause a machine to receive a second frame of the superframe, wherein the second frame follows the first frame and is encoded without reference to data in any preceding superframe.

133. The machine-readable medium of claim 126, wherein the first portion is part of an entire frame that excludes one or more second portions and is located in a first frame of a superframe of the multimedia data, further comprising instructions that upon execution cause a machine to receive one or more of the second portions that are intra-coded, further wherein the one or more intra-coded second portions are located in one or more second frames of the superframe.

134. The machine-readable medium of claim 133, further comprising instructions that when executed cause a machine to:

selectively decode the intra-coded first portion of the second version of the multimedia data;

selectively decode the one or more intra-coded second portions of the first version of the multimedia data; and

the decoded first portion and the one or more decoded second portions are combined for display in a single frame.

135. The machine-readable medium of claim 126, wherein the first portion is a portion of an entire frame that excludes one or more second portions and is located in a first frame of a superframe of the multimedia data, further comprising instructions that when executed cause a machine to receive one or more of the second portions of the first frame that are intra-encoded.

136. The machine-readable medium of claim 135, further comprising instructions that when executed cause a machine to:

selectively decode the intra-coded first portion of the second version of the multimedia data;

selectively decode the one or more intra-coded second portions of the first version of the multimedia data; and

the decoded first portion and the one or more decoded second portions are combined for display in a single frame.

137. The machine-readable medium of claim 126, further comprising instructions that upon execution cause a machine to receive information relating to the received first and second versions of the multimedia data.

138. The machine-readable medium of claim 126, further comprising instructions that when executed cause a machine to:

receiving the first version over a first communication link; and

receiving the second version includes receiving over a second communication link.

139. The machine-readable medium of claim 138, wherein the first communication link or the second communication link comprises a virtual channel.

140. The machine-readable medium of claim 126, further comprising instructions that upon execution cause a machine to receive the first and second versions over a single communication link.

141. A method for transmitting digital video data, the method comprising:

transmitting an inter-coded digital video frame for each of a plurality of digital video channels; and

transmitting channel switch frames for each of the channels, wherein at least one of the channel switch frames includes intra-coded digital video data corresponding to one of the inter-coded frames of the respective channel.

142. The method of claim 141, wherein at least one of the inter-coded frames following one of the channel switch frames is encoded to exclude backward references to any of the inter-coded frames preceding the respective channel switch frame.

143. The method of claim 142, wherein at least one of the inter-coded frames following one of the channel switch frames references a frame that coincides with the respective channel switch frame.

144. The method of claim 143, wherein the frame coinciding with the individual channel switch frame is the channel switch frame for a requested channel if a channel switch is requested or one of the inter-coded frames for a current channel if a channel switch is not requested.

145. The method of claim 141, further comprising periodically transmitting the channel switch frame at least once per second.

146. The method of claim 141, wherein at least one of the channel switch frames is embedded within a management frame.

147. The method of claim 146, wherein the management frame comprises a Supplemental Enhancement Information (SEI) frame.

148. The method of claim 141, wherein the digital video data in the channel switch frame is encoded at a reduced bit rate relative to other intra-coded frames.

149. The method of claim 141, wherein transmitting inter-coded frames comprises transmitting predictive (P) frames, and wherein the channel switch frame comprises an intra (I) frame that contains intra-coded content corresponding to one of the P frames.

150. An apparatus for transmitting digital video data, comprising:

means for transmitting an inter-coded digital video frame for each of a plurality of digital video channels; and

means for transmitting channel switch frames for each of the channels, wherein at least one of the channel switch frames includes intra-coded digital video data corresponding to one of the inter-coded frames of the respective channel.

151. The apparatus of claim 150, wherein at least one of the inter-coded frames following one of the channel switch frames is encoded to exclude backward references to any of the inter-coded frames preceding the respective channel switch frame.

152. The apparatus of claim 151, wherein at least one of the inter-coded frames following one of the channel switch frames refers to a frame that coincides with the respective channel switch frame.

153. The apparatus of claim 152, wherein the frame coinciding with the individual channel switch frame is the channel switch frame for a requested channel if a channel switch is requested or one of the inter-coded frames for a current channel if a channel switch is not requested.

154. The apparatus of claim 150, further comprising means for periodically transmitting the channel switch frame at least once per second.

155. The apparatus of claim 150, wherein at least one of the channel switch frames is embedded within a management frame.

156. The apparatus of claim 155, wherein the management frame comprises a Supplemental Enhancement Information (SEI) frame.

157. The device of claim 150, wherein the digital video data in the channel switch frame is encoded at a reduced bit rate relative to other intra-coded frames.

158. The apparatus of claim 150, wherein the means for transmitting inter-coded frames further transmits predicted (P) frames, and wherein the channel switch frame comprises an intra (I) frame containing intra-coded content corresponding to one of the P frames.

159. An apparatus for transmitting digital video data, comprising:

a first transmitter configured to transmit an inter-coded digital video frame for each of a plurality of digital video channels; and

a second transmitter configured to transmit channel switch frames for each of the channels, wherein at least one of the channel switch frames includes intra-coded digital video data corresponding to one of the inter-coded frames of the respective channel.

160. A processor for transmitting digital video data, the processor configured to:

transmitting an inter-coded digital video frame for each of a plurality of digital video channels; and

transmitting channel switch frames for each of the channels, wherein at least one of the channel switch frames includes intra-coded digital video data corresponding to one of the inter-coded frames of the respective channel.

161. A machine-readable medium comprising instructions for transmitting digital video data, wherein the instructions, when executed, cause a machine to:

transmitting an inter-coded digital video frame for each of a plurality of digital video channels; and

transmitting channel switch frames for each of the channels, wherein at least one of the channel switch frames includes intra-coded digital video data corresponding to one of the inter-coded frames of the respective channel.

162. A method for receiving digital video data, the method comprising:

obtaining an inter-coded frame comprising inter-coded digital video data associated with a first channel;

receiving a request to switch to a second channel; and

in response to the request, obtaining a channel switch frame that includes intra-coded digital video data corresponding to one of the inter-coded frames of the second channel.

163. The method of claim 162, wherein at least one of the inter-coded frames after the channel switch frame does not include a backward reference to any of the inter-coded frames before the individual channel switch frame.

164. The method of claim 163, wherein at least one of the inter-coded frames following one of the channel switch frames only references frames that coincide with the respective channel switch frame.

165. The method of claim 164, wherein the frame coinciding with the individual channel switch frame is the channel switch frame for the second channel if a request to switch to the second channel is requested or one of the inter-coded frames for the first channel if a request to switch to the second channel is not requested.

166. The method of claim 162, wherein acquiring a channel switch frame comprises acquiring one of a plurality of channel switch frames corresponding to different channels.

167. The method of claim 162, wherein the channel switch frame is embedded within a management frame.

168. The method of claim 167, wherein the management frame comprises a Supplemental Enhancement Information (SEI) frame.

169. The method of claim 162, wherein the digital video data in the channel switch frame is encoded at a reduced bit rate relative to other intra-coded frames.

170. The method of claim 162, wherein the inter-coded frames comprise predictive (P) frames, and wherein the channel switch frame comprises an intra (I) frame that contains intra-coded digital video data corresponding to one of the P frames.

171. An apparatus for receiving digital video data, comprising:

means for obtaining an inter-coded frame comprising inter-coded digital video data associated with a first channel;

means for receiving a request to switch to a second channel; and

means for obtaining a channel switch frame in response to the request, the channel switch frame including intra-coded digital video data corresponding to one of the inter-coded frames of the second channel.

172. The apparatus of claim 171, wherein at least one of the inter-coded frames following the channel switch frame does not include a backward reference to any of the inter-coded frames preceding the individual channel switch frame.

173. The apparatus of claim 172, wherein at least one of the inter-coded frames following one of the channel switch frames only references frames that coincide with the respective channel switch frame.

174. The apparatus of claim 173, wherein the frame conforming to the individual channel switch frame is one of the channel switch frame for the second channel if a request to switch to the second channel is requested, or the inter-coded frame for the first channel if a request to switch to the second channel is not requested.

175. The apparatus of claim 171, wherein the means for acquiring the channel switch frame further acquires one of a plurality of channel switch frames corresponding to different channels.

176. The apparatus of claim 171, wherein the channel switch frame is embedded within a management frame.

177. The apparatus of claim 176, wherein the management frame comprises a Supplemental Enhancement Information (SEI) frame.

178. The device of claim 171, wherein the digital video data in the channel switch frame is encoded at a reduced bit rate relative to other intra-coded frames.

179. The apparatus of claim 171, wherein the inter-coded frames comprise predictive (P) frames, and wherein the channel switch frame comprises an intra (I) frame that contains intra-coded digital video data corresponding to one of the P frames.

180. An apparatus for receiving digital video data, comprising:

a first acquisition module configured to acquire an inter-coded frame comprising inter-coded digital video data associated with a first channel;

a receiver configured to receive a request to switch to a second channel; and

a second acquisition module configured to acquire, in response to the request, a channel switch frame that includes intra-coded digital video data corresponding to one of the inter-coded frames of the second channel.

181. A processor for receiving digital video data, the processor configured to:

obtaining an inter-coded frame comprising inter-coded digital video data associated with a first channel;

receiving a request to switch to a second channel; and

in response to the request, obtaining a channel switch frame that includes intra-coded digital video data corresponding to one of the inter-coded frames of the second channel.

182. A machine-readable medium comprising instructions for receiving digital video data, wherein the instructions, when executed, cause a machine to:

obtaining an inter-coded frame comprising inter-coded digital video data associated with a first channel;

receiving a request to switch to a second channel; and

in response to the request, obtaining a channel switch frame that includes intra-coded digital video data corresponding to one of the inter-coded frames of the second channel.