HK1248042B - Electronic devices for sending a message and buffering a bitstream - Google Patents

Description

Electronic devices for sending messages and buffering bit streams

This application is a divisional application of Chinese national application No. 201380019557.3 filed on October 11, 2014, entitled “Electronic device for sending messages and buffering bit streams”.

Technical Field

The present disclosure relates generally to electronic devices and more particularly to electronic devices for sending messages and buffering bitstreams.

Background Art

To meet consumer demand and improve portability and convenience, electronic devices are becoming smaller and more powerful. Consumers have become dependent on electronic devices and expect increased functionality. Some examples of electronic devices include desktop computers, laptop computers, cellular phones, smartphones, media players, integrated circuits, etc.

Some electronic devices are used to process and display digital media. For example, portable electronic devices now allow consumers to use digital media almost anywhere. In addition, some electronic devices can provide downloading or streaming of digital media content for consumers to use and enjoy.

The increasing popularity of digital media presents several problems. For example, efficiently representing high-quality digital media for storage, transmission, and fast playback presents several challenges. As can be seen from this discussion, systems and methods for efficiently representing digital media with improved performance would be advantageous.

Summary of the Invention

Technical issues

It would be desirable to provide more efficient techniques for representing digital media.

Solution

One aspect of the present invention provides an electronic device for sending a message, comprising:

processor;

a memory in electronic communication with the processor;

Instructions stored in the memory, the instructions being executable to:

(a) determining whether the first picture is a random access picture;

(b) If the first picture is a random access picture, determine whether there is a leading picture;

(c) if a preceding picture exists, generating a message including a flag indicating whether an initial coded picture buffer (CPB) removal delay parameter for starting random access to the picture exists and the initial CPB removal delay parameter for starting random access to the picture; and

(d) sending the message.

Another aspect of the present invention provides an electronic device for buffering a bit stream, comprising:

processor;

a memory in electronic communication with the processor;

Instructions stored in the memory, the instructions being executable to:

(a) receiving a message;

(b) determining whether the first access unit is an access unit indicating a random access picture, whether there is no preceding picture, and whether a flag indicates that an initial coded picture buffer (CPB) removal delay parameter for starting random access pictures is present in the message;

(c) removing the first access unit; and

(d) Decode the first access unit.

Another aspect of the present invention provides a method for sending a message through an electronic device, comprising:

determining whether the first picture is a random access picture;

If the first picture is a random access picture, determining whether there is a preceding picture;

If a preceding picture exists, generating a message including a flag indicating whether an initial coded picture buffer (CPB) removal delay parameter for starting random access to the picture exists and the initial CPB removal delay parameter for starting random access to the picture; and

The message is sent.

Another aspect of the present invention provides a method for buffering a bitstream by an electronic device, comprising:

Receive messages;

determining whether the first access unit is an access unit indicating a random access picture, whether there is no preceding picture, and whether the flag indicates that there is an initial coded picture buffer (CPB) removal delay parameter for starting a random access picture;

removing the first access unit; and

A first access unit is decoded.

Advantageous Effects of the Invention

The above and other objects, features and advantages of the present invention will be more readily understood when the following detailed description of the present invention is considered in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

1 is a block diagram illustrating an example of one or more electronic devices in which systems and methods for sending messages and buffering bitstreams may be implemented;

FIG2 is a flow chart illustrating one configuration of a method for sending a message;

FIG3 is a flow chart showing a more specific configuration of a method for sending a message;

FIG4 is a flow chart illustrating one configuration of a method for buffering a bitstream;

FIG5 is a flow chart showing a more specific configuration of a method for buffering a bit stream;

FIG6 is a block diagram showing one configuration of an encoder on an electronic device;

FIG7 is a block diagram showing one configuration of a decoder on an electronic device;

FIG8 illustrates various components that may be used in a transmitting electronic device;

FIG9 is a block diagram illustrating various components that may be used in a receiving electronic device;

FIG10 is a block diagram illustrating one configuration of an electronic device in which systems and methods for sending messages may be implemented; and

11 is a block diagram illustrating one configuration of an electronic device in which systems and methods for buffering a bitstream may be implemented.

DETAILED DESCRIPTION

An electronic device for sending a message is described. The electronic device includes a processor and instructions stored in a memory, the memory being in electronic communication with the processor. The electronic device determines whether a first picture is a clean random access (CRA) picture. If the first picture is a CRA picture, the electronic device further determines whether a preceding picture exists. If a preceding picture exists, the electronic device additionally generates a message including a CAR preceding picture discard flag and an initial CRA coded picture buffer (CPB) removal delay parameter. The electronic device also sends the message. The message may be a buffering period supplemental enhancement information (SEI) message. The initial CRA CPB removal delay parameter may be referred to as an initial CPB removal delay parameter for starting random access pictures. The CRA preceding picture discard flag may be referred to as a flag indicating whether the initial CPB removal delay parameter for starting random access pictures exists in the message.

The initial CRA CPB removal delay parameter may be initial_cra_cpb_removal_delay[SchedSelIdx]. SchedSelIdx may be an index variable.

If a preceding picture exists, the electronic device may further generate an initial CRA CPB removal delay offset parameter. The initial CRA CPB removal delay offset parameter may be initial_cra_cpb_removal_delay_offset[SchedSelIdx]. SchedSelIdx may be an index variable.

Determining whether the first picture is a CRA picture may include determining whether the first picture includes only I slices. Determining whether a preceding picture exists may include determining whether the picture follows the CRA picture in decoding order and precedes the CRA picture in output order.

An electronic device for buffering a bitstream is also described. The electronic device includes a processor and instructions stored in a memory, the memory being in electronic communication with the processor. The electronic device receives a message. The electronic device also determines whether a first access unit is a clean random access (CRA) access unit, whether a preceding picture does not exist, and whether a CRA preceding picture discard flag indicates discard. The electronic device also removes the first access unit. Additionally, the electronic device decodes the first access unit. The message may be a buffering period supplemental enhancement information (SEI) message. Receiving the message may include receiving a CRA discard flag, an initial CRA coded picture buffer (CPB) removal delay parameter, and an initial CRA CPB removal delay offset parameter.

If the first access unit is a CRA access unit, if there is no preceding picture and if the CRA preceding picture discard flag indicates discard, then the initial removal delay variable may also be set to an initial CRA coded picture buffer (CPB) removal delay parameter. Removal of the first access unit may be based on the initial CRA CPB removal delay parameter.

The initial CRA CPB removal delay parameter may be initial_cra_cpb_removal_delay[SchedSelIdx]. SchedSelIdx may be an index variable.

If the first access unit is a CRA access unit, if there is no preceding picture and if the CRA preceding picture discard flag indicates discard, the electronic device may further determine a removal time based on the CRA CPB removal delay parameter. Determining whether the first access unit is a CRA access unit may include determining whether all network access layer (NAL) unit types corresponding to the first access unit indicate coded slices of a CRA picture. If the first access unit is a CRA access unit, if there is no preceding picture and if the CRA preceding picture discard flag indicates discard, the electronic device may determine a bitstream arrival time based on an initial CRA coded picture buffer (CPB) removal delay parameter.

A method for sending a message via an electronic device is also described. The method includes determining whether a first picture is a clean random access (CRA) picture. The method also includes, if the first picture is a CRA picture, determining whether a preceding picture exists. The method also includes, if the preceding picture exists, generating a message including a CRA discard flag and an initial CRA coded picture buffer (CPB) removal delay parameter. Additionally, the method includes sending the message.

A method for buffering a bitstream by an electronic device is also described. The method includes receiving a message. The method also includes determining whether a first access unit is a clean random access (CRA) access unit, whether a preceding picture is not present, and whether a CRA preceding picture discard flag indicates discard. The method also includes removing the first access unit. Additionally, the method includes decoding the first access unit.

The systems and methods disclosed herein describe electronic devices for sending messages and buffering bitstreams. For example, the systems and methods disclosed herein describe buffering a bitstream that begins with a clean random access (CRA) picture. In some configurations, the systems and methods disclosed herein may describe buffering a hypothetical reference decoder (HRD) for a bitstream that begins with a CRA picture. For example, the systems and methods disclosed herein describe modifications to the buffering period supplemental enhancement information (SEI) message and HRD for a bitstream that begins with a CRA picture when preceding pictures are present. The systems and methods disclosed herein (e.g., HRD modifications) may provide the following advantages: reduced initial buffering latency when playback begins with a CRA picture at a random access point. A random access point may be any point in a data stream (e.g., a bitstream) where decoding the bitstream does not require access to any point in the bitstream before the random access point in order to decode the current picture and all pictures following the current picture in output order. Furthermore, the systems and methods disclosed herein may provide the following advantages: the HRD will not underflow when one or more preceding pictures following a CRA picture are discarded.

It should be noted that although the term "hypothetical" is used with respect to HRD, HRD can be physically implemented. For example, "HRD" can be used to describe the implementation of an actual encoder. In some configurations, HRD can be implemented to determine whether a bitstream conforms to the High Efficiency Video Coding (HEVC) specification. For example, HRD can be used to determine whether Type I bitstreams and Type II bitstreams conform to the HEVC specification. Type I bitstreams may contain only Video Coding Layer (VCL), Network Access Layer (NAL) units, and filler data NAL units. Type II bitstreams may contain additional NAL units and syntax elements.

The Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-H0496 proposes a bitstream that starts with a CRA picture. This functionality has been incorporated into the High Efficiency Video Coding (HEVC) group draft (JCTVC-H1003).

Table (1) below gives an example of the modified syntax and semantics according to the system and method disclosed herein. Modifications made according to the system and method disclosed herein are indicated in bold.

Table 1

The following illustrates examples related to buffering period SEI message semantics according to the systems and methods disclosed herein. Specifically, the following illustrates additional details regarding the semantics of the modified syntax elements. 'cra_leadingpict_discard_flag' equal to 1 indicates the presence of 'initial_cra_cpb_removal_delay[SchedSelIdx]' and 'initial_cra_cpb_removal_delay_offset[SchedSelIdx]' syntax elements. 'cra_leadingpict_discard_flag' equal to 0 indicates the absence of 'initial_cra_cpb_removal_delay[SchedSelIdx]' and 'initial_cra_cpb_removal_delay_offset[SchedSelIdx]' syntax elements. This flag may not be equal to 1 when the associated or corresponding picture is not a CRA picture.

When an access unit containing a preceding picture associated with the starting CRA picture is discarded, initial_cra_cpb_removal_delay[SchedSelIdx] specifies the delay for the SchedSelIdx-th coded picture buffer (CPB) between the arrival time of the first bit of coded data associated with the access unit at the CPB and the removal time of coded data associated with the same access unit from the CPB, for the first buffering period after HRD initialization with the starting CRA picture, where the buffering period SEI message is associated. The bit length of the syntax element is initial_cpb_removal_delay_length_minus1+1. In some configurations, the unit of this syntax element is a 90 kilohertz (kHz) clock.

'initial_cra_cpb_removal_delay[SchedSelIdx]' may not be equal to 0. In addition, it may not exceed 90000*(CpbSize[SchedSelIdx]/**BitRate[SchedSelIdx]), where 90000*(CpbSize[SchedSelIdx]/**BitRate[SchedSelIdx]) is the time equivalent of the CPB size in units of a 90 kHz clock.

'initial_cra_cpb_removal_delay_offset[SchedSelIdx]' may be used together with cpb_removal_delay for the SchedSelIdx-th CPB to specify, for a bitstream starting with a CRA picture, the initial delivery time of coded access units to the CPB when access units containing preceding pictures associated with the starting CRA picture are discarded. In some configurations, the unit of 'initial_cra_cpb_removal_delay_offset[SchedSelIdx]' is a 90 kHz clock. The 'initial_cra_cpb_removal_delay_offset[SchedSelIdx]' syntax element may be a fixed-length encoding with a bit length of initial_cpb_removal_delay_length_minus1+1. This syntax element may not be used by the decoder and may only be used by the delivery scheduler (e.g., the Hypothetical Stream Scheduler (HSS) specified in Annex C of JCTVC-H1003). Over the entire coded video sequence, for each SchedSelIdx value, the sum of 'initial_cra_cpb_removal_delay[SchedSelIdx]' and 'initial_cra_cpb_removal_delay_offset[SchedSelIdx]' may be constant. SchedSelIdx may be an index variable. PayloadSize may be referred to as bytes of data.

seq_parameter_set_id specifies the sequence parameter set that contains the sequence HRD attributes. The value of seq_parameter_set_id can be equal to the value of seq_parameter_set_id in the picture parameter set referenced by the primary coded picture associated with the buffering period SEI message. The value of seq_parameter_set_id can range from 0 to 31, inclusive.

initial_cpb_removal_delay[SchedSelIdx] specifies the delay for the SchedSelIdx-th CPB between the arrival of the first bit of coded data associated with an access unit at the CPB and the removal of coded data associated with the same access unit from the CPB, for the first buffering period after HRD initialization, where the buffering period SEI message is associated. The bit length of this syntax element is initial_cpb_removal_delay_length_minus1+1. The units of this syntax element are 90 kilohertz (kHz) clocks.

initial_cpb_removal_delay[SchedSelIdx] may not be equal to 0 and may not exceed 90000*(CpbSize[SchedSelIdx]/BitRate[SchedSelIdx]), where 90000*(CpbSize[SchedSelIdx]/BitRate[SchedSelIdx]) is the time equivalent of the CPB size in units of a 90kHz clock.

The initial_cpb_removal_delay_offset[SchedSelIdx] may be used with cpb_removal_delay for the SchedSelIdx-th CPB to specify the initial delivery time of a coded access unit to the CPB. The unit of initial_cpb_removal_delay_offset[SchedSelIdx] is a 90 kHz clock. The initial_cpb_removal_delay_offset[SchedSelIdx] syntax element is a fixed-length encoding with a bit length of initial_cpb_removal_delay_length_minus1+1. This syntax element may not be used by the decoder and may only be used for the Transport Scheduler (HSS) specified in Annex C of the HEVC specification. Over the entire coded video sequence, for each SchedSelIdx value, the sum of initial_cpb_removal_delay[SchedSelIdx] and initial_cpb_removal_delay_offset[SchedSelIdx] may be constant.

cpb_cnt_minus1+1 specifies the number of alternative CPB specifications in the bitstream. The value of cpb_cnt_minus1 can range from 0 to 31, inclusive. When low_delay_hrd_flag is equal to 1, cpb_cnt_minus1 can be equal to 0. When cpb_cnt_minus1 is not present, it can be considered to be equal to 0.

nal_hrd_parameters_present_flag is equal to 1 to indicate the presence of NAL HRD parameters (relevant for Type II bitstream conformance). nal_hrd_parameters_present_flag is equal to 0 to indicate the absence of NAL HRD parameters. It should be noted that when nal_hrd_parameters_present_flag is equal to 0, bitstream conformance cannot be verified by means not specified in the HEVC specification without providing NAL HRD parameters (including NAL sequence HRD parameter information and all buffering period and picture timing SEI messages).

When nal_hrd_parameters_present_flag is equal to 1, NAL HRD parameters (e.g., according to subclauses E.1.2 and E.2.2 of the HEVC specification) immediately follow this flag. The variable NalHrdBpPresentFlag can be obtained as follows. The value of NalHrdBpPresentFlag may be set to 1 by some means not specified in the HEVC specification if any of the following is true: nal_hrd_parameters_present_flag is present in the bitstream and is equal to 1, or the need for the presence of a buffering period for NAL HRD operation to be present in the bitstream within a buffering period SEI message is determined by the application. Otherwise, the value of NalHrdBpPresentFlag may be set to 0.

vcl_hrd_parameters_present_flag equal to 1 indicates the presence of VCL HRD parameters (relevant for all bitstream conformance). vcl_hrd_parameters_present_flag equal to 0 indicates the absence of VCL HRD parameters. It should be noted that when vcl_hrd_parameters_present_flag is equal to 0, bitstream conformance cannot be verified by some means not specified in the HEVC specification without providing VCL HRD parameters and all buffering period and picture timing SEI messages. When vcl_hrd_parameters_present_flag is equal to 1, VCL HRD parameters (e.g., according to subclauses E.1.2 and E.2.2 of the HEVC specification) immediately follow this flag.

The variable VclHrdBpPresentFlag may be obtained as follows. The value of VclHrdBpPresentFlag may be set to 1 by some means not specified in the HEVC specification if any of the following are true: vcl_hrd_parameters_present_flag is present in the bitstream and is equal to 1, or the application determines the need for a buffering period for VCL HRD operations to be present in the bitstream within a buffering period SEI message. Otherwise, the value of VclHrdBpPresentFlag may be set to 0.

The following illustrates examples related to the operation of a CPB according to the systems and methods disclosed herein. The specifications shown below may be applied individually to each CPB parameter set present and to both Type I and Type II conformance according to the HEVC specification.

The timing of bitstream arrival (e.g., one or more bitstream arrival times) can be determined as follows. The HRD can be initialized at any buffering period SEI message. Before initialization, the CPB can be empty. It should be noted that after initialization, the HRD may not be initialized again by a subsequent buffering period SEI message.

The access unit associated with the buffering period SEI message that initializes the CPB may be referred to as access unit 0. All other access units may be referred to as access unit n, where n is incremented by 1 for the next access unit in decoding order.

If the first access unit is a CRA access unit, there is no leading picture and cra_leadingpict_discard_flag is equal to 1, use_initial_cpb_removal_delay[SchedSelIdx] may be set to the value of initial_cra_cpb_removal_delay[SchedSelIdx]. Otherwise, the value of use_initial_cpb_removal_delay[SchedSelIdx] may be set to the value of initial_cpb_removal_delay[SchedSelIdx].

The time when the first bit of access unit n begins to enter the CPB can be called the initial arrival time _tai (n). The initial arrival time of an access unit can be obtained as follows. If the access unit is access unit 0, then _tai (n) = 0. Otherwise (for example, the access unit is access unit n, where n>0), the following method can be applied. If cbr_flag[SchedSelIdx] is equal to 1, then the initial arrival time of access unit n is equal to the final arrival time of access unit n-1 (the final arrival time can be obtained as follows) (for example, _tai (n) = _taf (n-1)).

Otherwise, if cbr_flag[SchedSelIdx] is equal to 0 and access unit n is not the first access unit of the subsequent buffering period, the initial arrival time of access unit n can be obtained by _tai (n)=Max( _taf (n-1), tai _,earliest (n)). In some configurations, tai _,earliest (n) can be given as follows: tai _,earliest (n) = t _r,n (n) - (use_initial_cpb_removal_delay[SchedSelIdx] + use_initial_cpb_removal_delay_offset[SchedSelIdx])/90000, where t _r,n (n) is the nominal removal time of access unit n from the CPB (e.g., as specified in subclause C.2.2 of JCTVC-H1003), and use_initial_cpb_removal_delay[SchedSelIdx] and use_initial_cpb_removal_delay_offset[SchedSelIdx] are specified based on the previous buffering period SEI message as described above.

Otherwise (e.g., cbr_flag[SchedSelIdx] is equal to 0 and the subsequent access unit n is the first access unit of the subsequent buffering period), the initial arrival time of access unit n can be obtained by _tai (n)=t _r,n (n)-(use_initial_cpb_removal_delay[SchedSelIdx]/90000). In this case, use_initial_cpb_removal_delay[SchedSelIdx] can be specified based on the buffering period SEI message associated with access unit n as described above.

The final arrival time of access unit n can be obtained by _taf (n) = _tai (n) + b(n) / BitRate[SchedSelIdx]. In this case, b(n) can be the bit size of access unit n, counting the bits of the type I bitstream for type I conformance or the bits of the type II bitstream for type II conformance.

The values of SchedSelIdx, BitRate[SchedSelIdx], and CpbSize[SchedSelIdx] may be constrained as follows. If access unit n and access unit n-1 are part of different coded video sequences, and the contents of the active sequence parameter sets for the two coded video sequences are different, then the HSS may select a value of SchedSelIdx, SchedSelIdx 1, from the values of SchedSelIdx provided for the coded video sequence containing access unit n, resulting in a BitRate[SchedSelIdx1] or CpbSize[SchedSelIdx1] for the second of the two coded video sequences (containing access unit n-1) that is different from a BitRate[SchedSelIdx0] or CpbSize[SchedSelIdx0] for the value of SchedSelIdx, SchedSelIdx0, used for the coded video sequence containing access unit n-1. Otherwise, the HSS may continue to operate with the previous values of SchedSelIdx, BitRate[SchedSelIdx], and CpbSize[SchedSelIdx].

When the HSS selects a value for BitRate[SchedSelIdx] or CpbSize[SchedSelIdx] that is different from the previous access unit, the following method can be applied. The variable BitRate[SchedSelIdx] can be effective at time t _ai (n). The variable CpbSize[SchedSelIdx] can be effective as follows. If the new value of CpbSize[SchedSelIdx] exceeds the old CPB size, it can be effective at time t _ai (n). Otherwise, the new value of CpbSize[SchedSelIdx] can be effective at time t _r (n).

In some configurations, the timing of coded picture removal can be implemented as follows: It can be assumed that when the HRD is initialized, the nominal CPB removal time and the CPB removal time of the coded picture are determined (e.g., calculated) immediately after the previous coded picture is removed from the CPB or are determined (e.g., calculated) for access unit 0.

If the first access unit is a CRA access unit, a leading picture exists, and cra_leadingpict_discard_flag is equal to 1, then the nominal removal time of the access unit from the CPB may be specified as t _r,n (0) = initial_cra_cpb_removal_delay[SchedSelIdx] / 90000 for access unit 0. Otherwise, the nominal removal time of the access unit from the CPB may be specified as t _r,n (0) = initial_cpb_removal_delay[SchedSelIdx] / 90000 for access unit 0.

For the first access unit of a buffering period that does not initialize HRD, the nominal removal time of the access unit from the CPB can be specified as t _r,n (n) = t _r,n (n _b ) + t _c * cpb_removal_delay(n). In this case, t _r,n (n _b ) is the nominal removal time of the first picture of the previous buffering period, and cpb_removal_delay(n) is specified in the picture timing SEI message associated with access unit n.

When access unit n is the first access unit of a buffering period, _nb may be set equal to n at the removal time tr _,n (n) of access unit n. The nominal removal time tr _,n (n) of access unit n that is not the first access unit of a buffering period may be expressed as _tr,n (n)t _r,n ( _nb )+ _tc *cpb_removal_delay(n).

The removal time of access unit n may be specified as follows. If low_delay_hrd_flag is equal to 0 or t _r,n (n) >= t _af (n), then the removal time of access unit n may be specified as t _r (n) = t _r,n (n). Otherwise (e.g., low_delay_hrd_flag is equal to 1 and t _r,n (n) < t _af (n)), the removal time of access unit n may be specified as t _r (n) = t _r,n (n) + t _c * Ceil((t _af (n) - _{t r,n} (n)) / t _c ). Note that the latter case indicates that the size b(n) of access unit n may be sufficiently large to prevent removal at the nominal removal time.

In some configurations, bitstream conformance is required where for each test, all of the following conditions shall be met. For each access unit n associated with a buffering period SEI message, where n>0, delta tg _,90 (n) is specified as delta _tg,90 (n)=90000*(t _r,n (n)-t _af (n-1)) and the value of use_initial_cpb_removal_delay[SchedSelIdx] shall be constrained as shown below. If cbr_flag[SchedSelIdx] is equal to 0, then use_initial_cpb_removal_delay[SchedSelIdx]<=Ceil(delta tg _,90 (n)). Otherwise (cbr_flag[SchedSelIdx] is equal to 1), Floor(delta t _{g, 90} (n)) <= use_initial_cpb_removal_delay[SchedSelIdx] <= Ceil(delta t _{g, 90} (n)). It should be noted that the exact number of bits in the CPB at the removal time of each picture may depend on which buffering period SEI message is selected to initialize the HRD. The encoder can take this into account so that when initializing the HRD at any buffering period SEI message, no matter which buffering period SEI message is selected to initialize the HRD, all specified constraints are met.

As described above, the systems and methods disclosed herein provide syntax and semantics for modifying the buffering period SEI message for a bitstream starting with a CRA picture when a preceding picture is present. In some configurations, the systems and methods disclosed herein may be applied to the HEVC specification.

For convenience, several definitions are provided below that may be applied to the systems and methods described herein. A random access point may be any point in a data stream (e.g., a bitstream) where decoding the bitstream does not require access to any point in the bitstream before the random access point in order to decode the current picture and all pictures following the current picture in output order.

A buffering period can be defined as the set of access units between two instances of a buffering period SEI message in decoding order. Supplemental enhancement information (SEI) may contain information that is not necessary for decoding samples of coded pictures from VCL NAL units. SEI messages may assist in processes related to decoding, display, or other purposes. However, the decoding process may not require SEI messages to construct luma or chroma samples. A conforming decoder is not required to process this information in order to conform to the output order of the HEVC specification (e.g., Annex C of the HEVC specification includes a conforming specification). Some SEI message information may be required to check bitstream consistency and for output timing decoder consistency.

The buffering period SEI message may be an SEI message associated with the buffering period. It may define the syntax and semantics that define the timing of bitstream arrival and coded picture removal.

A picture parameter set (PPS) is a syntax structure containing syntax elements that apply to zero or more complete coded pictures identified by the pic_parameter_set_id syntax element present in each slice header. The pic_parameter_set_id element may identify the picture parameter set referenced in the slice header. The value of pic_parameter_set_id may range from 0 to 255, inclusive.

The coded picture buffer (CPB) may be a first-in, first-out buffer containing access units in the decoding order specified in the hypothetical reference decoder (HRD). An access unit may be a collection of network access layer (NAL) units that are consecutive in decoding order and contain exactly one coded picture. In addition to coded slice NAL units for a coded picture, an access unit may also include other NAL units that do not contain slices of a coded picture. Decoding an access unit always results in a decoded picture. A NAL unit may be a syntactic structure containing an indication of the type of data that follows and containing the data in the form of a raw byte sequence payload (interspersed with race protection bytes as necessary).

Various configurations will now be described with reference to the accompanying drawings, where like reference numerals may indicate functionally similar elements. The systems and methods, as generally described and illustrated herein in the accompanying drawings, may be arranged and designed in a variety of different configurations. Therefore, the following more detailed description of several configurations shown in the accompanying drawings is not intended to limit the scope of the claims but is merely representative of the systems and methods.

FIG1 is a block diagram illustrating an example of one or more electronic devices 102 in which systems and methods for sending messages and buffering bitstreams may be implemented. In this example, electronic device A 102 a and electronic device B 102 b are shown. However, it should be noted that in some configurations, one or more features and functions described with respect to electronic device A 102 a and electronic device B 102 b may be incorporated into a single electronic device.

Electronic device A 102a includes an encoder 104. The encoder 104 includes a message generation module 108. Each element included in electronic device A 102a (eg, encoder 104 and message generation module 108) may be implemented in hardware, software, or a combination thereof.

Electronic device A 102a may obtain one or more input pictures 106. In some configurations, the input pictures 106 may be captured on electronic device A 102a using an image sensor, may be retrieved from memory, and/or may be received from another electronic device.

The encoder 104 may encode an input picture 106 to generate encoded data. For example, the encoder 104 may encode a series of input pictures 106 (e.g., a video). In one configuration, the encoder 104 may be a HEVC encoder. The encoded data may be digital data (e.g., a portion of a bitstream 114). The encoder 104 may generate overhead signaling based on the input signal.

The message generation module 108 may generate one or more messages. For example, the message generation module 108 may generate one or more SEI messages or other messages. The electronic device 102 (e.g., the encoder 104) may determine whether a first picture (e.g., a coded picture) is a CRA picture, and if the first picture is a CRA picture, determine whether a leading picture exists. If the first picture is a CRA picture and a leading picture exists, the message generation module 108 may generate a message (e.g., a buffering period SEI message or other message) including one or more CRA leading picture discard flags (e.g., cra_leadingpict_discard_flag), an initial CRA CPB removal delay parameter (e.g., initial_cra_cpb_removal_delay[SchedSelIdx]), and an initial CRA CPB removal delay offset parameter (e.g., initial_cra_cpb_removal_delay_offset[SchedSelIdx]). For example, the message generation module 108 may perform one or more processes described below in conjunction with FIG. 2 and FIG. 3 . In some configurations, electronic device A 102a can send a message to electronic device B 102b as part of the bitstream 114. In some configurations, electronic device A 102a can send the message to electronic device B 102b via a separate transmission 110. For example, the separate transmission may not be part of the bitstream 114. For example, some out-of-band mechanism may be used to send the buffering period SEI message or other messages. It should be noted that in some configurations, the other message may include one or more features of the buffering period SEI message described above. Furthermore, similar to the SEI message described above, other messages may be used in one or more aspects.

The encoder 104 (e.g., and the message generation module 108) can generate a bitstream 114. The bitstream 114 can include coded picture data based on the input picture 106. In some configurations, the bitstream 114 can also include overhead data, such as a buffering period SEI message or other information, a slice header, a PPS, etc. When encoding additional input pictures 106, the bitstream 114 can include one or more coded pictures. For example, the bitstream 114 can include one or more coded pictures with corresponding overhead data (e.g., a buffering period SEI message or other information).

The bitstream 114 can be provided to a decoder 112. In one example, a wired or wireless link can be used to send the bitstream 114 to an electronic device B 102b. In some cases, this can be accomplished via a network (e.g., the Internet or a local area network (LAN)). As shown in Figure 1, the decoder 112 can be implemented on an electronic device B 102b separately from the encoder 104 of the electronic device A 102a. However, it should be noted that in some configurations, the encoder 104 and the decoder 112 can be implemented on the same electronic device. In the embodiment where the encoder 104 and the decoder 112 are implemented on the same electronic device, for example, the bitstream 114 can be provided to the decoder 112 via a bus or the bitstream can be stored in a memory for the decoder 112 to retrieve.

The decoder 112 may be implemented in hardware, software, or a combination thereof. In one configuration, the decoder 112 may be an HEVC decoder. The decoder 112 may receive (e.g., obtain) a bitstream 114. The decoder 112 may generate one or more decoded pictures 118 based on the bitstream 114. The decoded pictures 118 may be displayed, played back, stored in memory, and/or transmitted to another device, etc.

The decoder 112 may include a CPB 120. The CPB 120 may temporarily store coded pictures. For example, the CPB 120 may store coded pictures until a removal time. According to the systems and methods disclosed herein, one or more of an arrival time (e.g., an arrival time of a bitstream) and a removal time (e.g., a removal time of a coded picture) may be determined by the decoder 112 based on a message (e.g., a buffering period SEI message or other message).

The decoder 112 may receive a message (e.g., a buffering period SEI message or other message). The decoder may also determine whether the first access unit is a CRA access unit, whether a preceding picture does not exist, and whether a CRA preceding picture discard flag indicates discard. If the first access unit is a CRA access unit, if a preceding picture does not exist, and if the CRA preceding picture discard flag indicates discard, the decoder may set the initial removal delay variable to the initial CRA CPB removal delay parameter and may remove the first access unit (e.g., a coded picture) from the CPB based on the CRA CPB removal delay parameter. For example, the CPB 120 may perform one or more of the processes described below in conjunction with FIG. 4 and FIG. 5 .

The aforementioned HRD may be an example of the decoder 112 shown in Figure 1. Thus, in some configurations, the electronic device 102 may operate according to the aforementioned HRD and CPB.

It should be noted that one or more elements or components included in the electronic device 102 can be implemented in hardware. For example, one or more elements or components can be implemented as chips, circuits, or hardware components. It should also be noted that one or more functions or methods described herein can be implemented in hardware and/or performed using hardware. For example, one or more methods described herein can be implemented in and/or using a chipset, an application specific integrated circuit (ASIC), a large-scale integrated circuit (LSI), or an integrated circuit.

FIG2 is a flow chart illustrating one configuration of a method 200 for sending a message. An electronic device 102 (e.g., electronic device A 102a) may determine 202 whether a first picture is a CRA picture. For example, the encoder 104 may encode the input picture 106 as a CRA picture. In some configurations, a CRA picture may be a coded picture that uses only intra prediction for decoding. For example, a CRA picture may include one or more (e.g., only) I slices, where each slice has a nal_unit_type (e.g., network abstraction layer unit type) equal to 4. All coded pictures that follow the CRA picture in decoding order and output order may not use inter prediction based on any picture that precedes the CRA picture in decoding order or output order. Furthermore, any picture that precedes the CRA picture in decoding order may also precede the CRA picture in output order. Therefore, if the encoder 104 encodes the input picture 106 as a CRA picture (e.g., if the coded picture includes only I slices), the electronic device 102 may determine 202 that the first picture is a CRA picture. Otherwise, the electronic device 102 may determine 202 that the first picture is not a CRA picture.

It should be noted that CRA pictures may appear at random access points in the bitstream 114. A random access point may be any point in a data stream (e.g., a bitstream) where decoding the bitstream does not require access to any point in the bitstream before the random access point in order to decode the current picture and all pictures that follow the current picture in output order.

If the first picture is a CRA picture, the electronic device 102 may determine 204 whether a preceding picture exists. A preceding picture may be a picture that follows the CRA picture in decoding order and precedes the CRA picture in output order. For example, if the encoder 104 specifies a picture that follows the CRA picture in decoding order and precedes the CRA picture in output order (e.g., an order output from the decoder 112), then a preceding picture exists.

Determining 204 whether a preceding picture exists can be accomplished according to one or more methods. In one method, if the first picture is a CRA picture and there is another picture designated (e.g., by the encoder 104) to follow the CRA picture in decoding order and precede the CRA picture in output order, the electronic device 102 determines that a preceding picture exists. In some configurations, the electronic device 102 may read data corresponding to the CRA picture and one or more other pictures to determine whether a preceding picture exists. For example, the electronic device 102 may read data specifying the decoding order and output order of the CRA picture and one or more other pictures. For example, the output order may be determined using a picture order count (POC). The decoding order may be determined based on the order in which syntax elements appear in the bitstream 114. In some configurations, if decoded pictures are to be output from a decoded picture buffer, the output order may be defined as the order in which the decoded pictures are output from the decoded picture buffer. The output order of pictures may be specified by the POC value, regardless of whether the pictures are to be output. In some configurations, the decoding order may be defined as the order in which syntax elements are processed during the decoding process. If the condition that a picture is designated as following the CRA picture in decoding order and preceding the CRA picture in output order is met, the electronic device 102 may determine 204 that a preceding picture exists.

If a leading picture exists, the electronic device 102 may generate 206 a message (e.g., a buffering period SEI message or other message) that includes a CRA leading picture discard flag and an initial CRA CPB removal delay parameter. For example, the electronic device 102 may generate 206 a message including cra_leadingpict_discard_flag and initial_cra_cpb_removal_delay[SchedSelIdx]. In some configurations, the message may also include a CRA CPB removal delay offset parameter. For example, the electronic device 102 may generate 206 a buffering period SEI message as shown in Table 1 above.

The electronic device 102 may transmit 208 the message (e.g., a buffering period SEI message or other message). For example, the electronic device 102 may transmit the message via one or more of a wireless transmission, a wired transmission, a device bus, a network, etc. For example, electronic device A 102a may transmit the message to electronic device B 102b. For example, the message may be part of the bitstream 114. In some configurations, electronic device A 102a may transmit 208 the message to electronic device B 102b in a separate transmission 110 (i.e., not part of the bitstream 114). For example, the message may be transmitted using an out-of-band mechanism.

FIG3 is a flow chart illustrating a more specific configuration of a method 300 for sending a message. An electronic device 102 (eg, electronic device A 102a) may determine 302 whether a first picture is a CRA picture. This may be implemented as described above in conjunction with FIG2 .

If the first picture is a CRA picture, the electronic device 102 may determine 304 whether there is a preceding picture. This may be achieved as described above in conjunction with FIG.

If a leading picture exists, the electronic device 102 may generate 306 a buffering period SEI message that includes a CRA leading picture discard flag, an initial CRA CPB removal delay parameter, and an initial CRA CPB removal delay offset parameter. For example, the electronic device 102 may generate 306 a buffering period SEI message that includes cra_leadingpict_discard_flag, initial_cra_cpb_removal_delay[SchedSelIdx], and initial_cra_cpb_removal_delay_offset[SchedSelIdx]. For example, the electronic device 102 may generate 306 a buffering period SEI message as shown in Table 1 above.

The electronic device 102 may transmit 308 the buffering period SEI message. For example, the electronic device 102 may transmit the buffering period SEI message via one or more of wireless transmission, wired transmission, a device bus, a network, etc. For example, electronic device A 102a may transmit the buffering period SEI message to electronic device B 102b. For example, the buffering period SEI message may be part of the bitstream 114.

FIG4 is a flow chart illustrating one configuration of a method 400 for buffering a bitstream. An electronic device 102 (e.g., electronic device B 102b) may receive 402 a message (e.g., a buffering period SEI message or other message). For example, electronic device 102 may receive 402 the message via one or more of a wireless transmission, a wired transmission, a device bus, a network, etc. For example, electronic device B 102b may receive the message from electronic device A 102a. For example, the message may be part of the bitstream 114. In another example, electronic device B 102b may receive the message from electronic device A 102a in a separate transmission 110 (i.e., not part of the bitstream 114). For example, the buffering period SEI message may be received using an out-of-band mechanism. In some configurations, the message may include one or more of a CRA preceding picture discard flag, an initial CRA CPB removal delay parameter, and an initial CRA CPB removal delay offset parameter. Thus, receiving 402 the message may include receiving one or more of the CRA preceding picture discard flag, the initial CRA CPB removal delay parameter, and the initial CRA CPB removal delay offset parameter.

The electronic device 102 may determine 404 whether the first access unit is a CRA access unit, whether one or more preceding pictures are not present, and whether a CRA preceding picture discard flag indicates discard. An access unit is a set of one or more NAL units that may contain a coded picture. The bitstream 114 may include one or more access units. In some configurations, the electronic device 102 may determine 404 whether the first access unit is a CRA access unit based on a NAL unit type parameter (e.g., nal_unit_type) corresponding to the first access unit. For example, nal_unit_type 4 may indicate a coded slice of a CRA picture. In this case, if one or more (e.g., all) NAL unit types (e.g., nal_unit_type) corresponding to the first access unit indicate a coded slice of a CRA picture (e.g., equal to 4), the electronic device may determine that the access unit is a CRA access unit. Otherwise, the electronic device 102 may determine that the first access unit is not a CRA access unit.

The electronic device 102 may determine whether any preceding pictures exist. For example, the electronic device 102 may determine whether any preceding pictures are included in a portion of the bitstream 114. As described above, a preceding picture may be a picture that follows a CRA picture in decoding order and precedes the CRA picture in output order. For example, a preceding picture exists if a picture is designated by the encoder 104 as following a CRA picture in decoding order and preceding the CRA picture in output order (e.g., the order output from the decoder 112).

Determining whether a preceding picture exists can be implemented according to one or more methods. In one method, if a picture (a picture in the first access unit) is a CRA picture and there is another picture designated (e.g., by the encoder 104) as following the CRA picture in decoding order and preceding the CRA picture in output order, the electronic device 102 determines the presence of a preceding picture. In some configurations, the electronic device 102 may read data corresponding to the CRA picture and one or more other pictures to determine whether a preceding picture exists. For example, the electronic device 102 may read data specifying the decoding order and output order of the CRA picture and one or more other pictures. In one example, the electronic device 102 may receive an indicator in the bitstream 114. In another example, the electronic device 102 may determine the presence of a preceding picture by determining a Point of View (POC) value for the picture and comparing it with the POC value of a corresponding CRA picture. If the conditions for designating a picture as following the CRA picture in decoding order and preceding the CRA picture in output order are met, the electronic device 102 may determine the presence of a preceding picture. However, if one or more criteria are not met, the electronic device 102 may determine that a preceding picture does not exist.

The electronic device 102 may determine whether the CRA leading picture discard flag indicates discard. For example, the electronic device 102 may read the message to determine whether the CRA leading picture discard flag in the SEI message indicates discard. For example, if the value of cra_leadingpict_discard_flag is 1, the electronic device 102 may determine that the CRA leading picture discard flag indicates discard. However, if the value of cra_leadingpict_discard_flag is 0, the electronic device 102 may determine that the CRA leading picture discard flag does not indicate discard.

If the first access unit is a CRA access unit, if there is no preceding picture and if the CRA preceding picture discard flag indicates discard, the electronic device 102 may set the initial removal delay variable to the initial CRA CPB removal delay parameter 406. In this case, for example, the electronic device 102 may set use_initial_cpb_removal_delay[SchedSelIdx] to the value of initial_cra_cpb_removal_delay[SchedSelIdx].

In some configurations, the electronic device 102 (e.g., the decoder 112) may determine the bitstream arrival time based on one or more of the initial CRA CPB removal delay parameter and the initial CRA CPB removal delay offset parameter. For example, if the first access unit is a CRA access unit, if there is no preceding picture, and if the CRA picture discard flag indicates discard, the bitstream arrival time may be determined as follows.

As described above, for an access unit subsequent to the access unit that initializes the CPB 120 (e.g., n>0), if cbr_flag[SchedSelIdx] is 0 and the access unit (e.g., n) is not the first access unit of a subsequent buffering period, the electronic device 102 may determine the bitstream arrival time t _ai (n) based on t _ai,earliest (n). t _ai,earliest (n) may be determined based on an initial removal delay variable (e.g., use_initial_cpb_removal_delay[SchedSelIdx]) and an initial removal delay offset variable (e.g., use_initial_cpb_removal_delay_offset). As described above, the initial removal delay variable may be set 406 to the initial CRA CPB removal delay parameter (e.g., initial_cra_cpb_removal_delay[SchedSelIdx]). Thus, determining the bitstream arrival time may be based on the initial CRA CPB removal delay parameter. In some configurations, an initial removal delay offset variable (eg, use_initial_cpb_removal_delay_offset[SchedSelIdx]) may also be set to the initial CRA CPB removal delay offset parameter (eg, initial_cra_cpb_removal_delay_offset[SchedSelIdx]). Thus, additionally, determining the bitstream arrival time may be based on the initial CRA CPB removal delay offset parameter.

As described above, for an access unit subsequent to the access unit that initialized the CPB 120 (e.g., n>0), if cbr_flag[SchedSelIdx] is 0 and the access unit (e.g., n) is the first access unit of a subsequent buffering period, the electronic device 102 determines the bitstream arrival time t _ai (n) based on the initial removal delay variable (e.g., use_initial_cpb_removal_delay[SchedSelIdx]). As described above, the initial removal delay variable can be set 406 to the initial CRA CPB removal delay parameter (e.g., initial_cra_cpb_removal_delay[SchedSelIdx]). Therefore, determining the bitstream arrival time can be based on the initial CRA CPB removal delay parameter.

If the first access unit is a CRA access unit, if there is no preceding picture and if the CRA preceding picture discard flag indicates discard, the electronic device 102 may remove 408 the first access unit based on the initial CRA CPB removal delay parameter. For example, the electronic device 102 may determine a removal time for the first access unit based on the initial CRA CPB removal delay parameter (e.g., initial_cra_cpb_removal_delay[SchedSelIdx]). When the removal time is reached, the electronic device 102 may remove the first access unit from the CPB, for example.

The electronic device 102 may decode 410 the first access unit. For example, the electronic device 102 may decode 410 the coded picture included in the first access unit. In some configurations, upon removal, the electronic device 102 may remove 408 the first access unit and decode 410 it. For example, at the CPB removal time, the data associated with each access unit may be removed and immediately decoded by an immediate decoding process. It should be noted that the term "immediately" and its variations do not necessarily indicate the absence of any time period. For example, "immediate" processing may occupy a period of time.

If the first access unit is not a CRA access unit, if the first access unit is a CRA access unit and there is a preceding picture, or if the CRA preceding picture discard flag does not indicate discard, the electronic device 102 may set the initial removal delay variable to the initial CPB removal delay parameter 412. In this case, for example, the electronic device 102 may set use_initial_cpb_removal_delay[SchedSelIdx] to the value of initial_cpb_removal_delay[SchedSelIdx].

If the first access unit is not a CRA access unit, if the first access unit is a CRA access unit and a preceding picture exists, or if the CRA preceding picture discard flag does not indicate discard, the electronic device 102 may remove 414 the first access unit based on an initial CPB removal delay parameter. For example, the electronic device 102 may determine a removal time for the first access unit based on an initial CPB removal delay parameter (e.g., initial_cpb_removal_delay[SchedSelIdx]). When the removal time is reached, for example, the electronic device 102 may remove the first access unit from the CPB.

The electronic device 102 may decode the first access unit 416. For example, the electronic device 102 may decode the coded picture included in the first access unit 416. For example, at the CPB removal time, the data associated with each access unit may be removed and decoded immediately by the immediate decoding process.

5 is a flow chart illustrating a more specific configuration of a method 500 for buffering a bitstream. An electronic device 102 (eg, electronic device B 102b) may receive 502 a buffering period SEI message. For example, this may be implemented as described above in conjunction with FIG.

The electronic device 102 may determine 504 whether the first access unit is a CRA access unit, whether one or more preceding pictures are absent, and whether a CRA preceding picture discard flag indicates discard. This may be implemented as described above in conjunction with FIG.

If the first access unit is a CRA access unit, if there is no preceding picture and if the CRA preceding picture discard flag indicates discard, the electronic device 102 may set the initial removal delay variable to the initial CRA CPB removal delay parameter 506. In this case, for example, the electronic device 102 may set use_initial_cpb_removal_delay[SchedSelIdx] to the value of initial_cra_cpb_removal_delay[SchedSelIdx].

In some configurations, the electronic device 102 may determine the bitstream arrival time based on one or more of the initial CRA CPB removal delay parameter and the initial CRA CPB removal delay offset parameter. For example, this may be implemented as described above in conjunction with FIG.

If the first access unit is a CRA access unit, if there is no preceding picture and if the CRA preceding picture discard flag indicates discard, the electronic device 102 may determine 508 a removal time (e.g., t _r,n ) based on the initial CRA CPB removal delay parameter. This may be implemented as described above in conjunction with initial_cra_cpb_removal_delay[SchedSelIdx]/90000.

If the first access unit is a CRA access unit, if there is no preceding picture and if the CRA preceding picture discard flag indicates discard, the electronic device 102 may remove 510 the first access unit based on the initial CRA CPB removal delay parameter. For example, as described above, the electronic device 102 may determine 508 a removal time (e.g., t _{r, n} ) of the first access unit based on the initial CRA CPB removal delay parameter (e.g., initial_cra_cpb_removal_delay[SchedSelIdx]). When the removal time (e.g., t r, _n ) is reached, the electronic device 102 may remove 510 the first access unit from the CPB, for example.

The electronic device 102 may decode the first access unit 512. For example, the electronic device 102 may decode the coded picture included in the first access unit 512. For example, at the CPB removal time, the data associated with each access unit may be removed and decoded immediately by the immediate decoding process.

If the first access unit is not a CRA access unit, if the first access unit is a CRA access unit and there is a preceding picture, or if the CRA preceding picture discard flag does not indicate discard, the electronic device 102 may set the initial removal delay variable to the initial CPB removal delay parameter 514. In this case, for example, the electronic device 102 may set use_initial_cpb_removal_delay[SchedSelIdx] to the value of initial_cpb_removal_delay[SchedSelIdx].

If the first access unit is not a CRA access unit, if the first access unit is a CRA access unit and there is a preceding picture, or if the CRA preceding picture discard flag does not indicate discard, the electronic device 102 may determine 516 a removal time (e.g., t _r,n ) based on the initial CPB removal delay parameter. This may be implemented as described above in conjunction with initial_cpb_removal_delay[SchedSelIdx]/90000.

If the first access unit is not a CRA access unit, if the first access unit is a CRA access unit and there is a preceding picture, or if the CRA preceding picture discard flag does not indicate discard, the electronic device 102 may remove 518 the first access unit based on the initial CPB removal delay parameter. For example, the electronic device 102 may determine 516 a removal time (e.g., t _{r, n} ) of the first access unit based on the initial CPB removal delay parameter (e.g., initial_cpb_removal_delay[SchedSelIdx]). When the removal time (e.g., t _{r, n} ) is reached, the electronic device 102 may remove 518 the first access unit from the CPB, for example.

The electronic device 102 may decode the first access unit 520. For example, the electronic device 102 may decode the coded picture included in the first access unit 520. For example, at the CPB removal time, the data associated with each access unit may be removed and decoded immediately by the immediate decoding process.

FIG6 is a block diagram illustrating one configuration of an encoder 604 on an electronic device 602. It should be noted that one or more elements shown as included within the electronic device 602 may be implemented in hardware, software, or a combination thereof. For example, the electronic device 602 includes an encoder 604, which may be implemented in hardware, software, or a combination thereof. For example, the encoder 604 may be implemented as a circuit, an integrated circuit, an application-specific integrated circuit (ASIC), a processor in electronic communication with a memory having executable instructions, firmware, a field-programmable gate array (FPGA), or the like, or a combination thereof. In some configurations, the encoder 604 may be an HEVC encoder.

The electronic device 602 may include a source 622. The source 622 may provide pictures or image data (e.g., video) as one or more input pictures 606 to the encoder 604. Examples of the source 622 may include an image sensor, a memory, a communication interface, a network interface, a wireless receiver, a port, etc.

One or more input pictures 606 may be provided to the intra prediction module and reconstruction buffer 624. The input pictures 606 may also be provided to the motion estimation and motion compensation module 646 and the subtraction module 628.

The intra prediction module and reconstruction buffer 624 may generate intra mode information 640 and an intra signal 626 based on one or more input pictures 606 and the reconstruction data 660. The motion estimation and motion compensation module 646 may generate inter mode information 648 and an inter signal 644 based on one or more input pictures 606 and a reference picture buffer 676 signal 678. In some configurations, the reference picture buffer 676 may include data from one or more reference pictures in the reference picture buffer 676.

The encoder 604 can select between an intra-frame signal 626 and an inter-frame signal 644 depending on the mode. In intra-frame coding mode, the intra-frame signal 626 can be used to exploit spatial characteristics. In inter-frame coding mode, the inter-frame signal 644 can be used to exploit temporal characteristics. When in intra-frame coding mode, the intra-frame signal 626 can be provided to a subtraction module 628, and intra-frame mode information 640 can be provided to an entropy coding module 642. When in inter-frame coding mode, the inter-frame signal 644 can be provided to the subtraction module 628, and inter-frame mode information 648 can be provided to the entropy coding module 642.

At a subtraction module 628, the intra signal 626 or the inter signal 644 (depending on the mode) is subtracted from the input picture 606 to produce a prediction residual 630. The prediction residual 630 is provided to a transform module 632. The transform module 632 may compress the prediction residual 630 to produce a transformed signal 634, which is provided to a quantization module 636. The quantization module 636 quantizes the transformed signal 634 to produce transformed and quantized coefficients (TQC) 638.

The TQC 638 is provided to an entropy coding module 642 and an inverse quantization module 650. The inverse quantization module 650 performs inverse quantization on the TQC 638 to produce an inverse quantized signal 652, which is provided to an inverse transform module 654. The inverse transform module 654 decompresses the inverse quantized signal 652 to produce a compressed signal 656, which is provided to a reconstruction module 658.

The reconstruction module 658 can generate reconstructed data 660 based on the decompressed signal 656. For example, the reconstruction module 658 can reconstruct (modify) a picture. The reconstructed data 660 can be provided to a deblocking filter 662 and an intra prediction module and reconstruction buffer 624. The deblocking filter 662 can generate a filtered signal 664 based on the reconstructed data 660.

The filtered signal 664 may be provided to a sample adaptive offset (SAO) module 666. The SAO module 666 may generate SAO information 668, which is provided to the entropy coding module 642, and an SAO signal 670, which is provided to an adaptive loop filter (ALF) 672. The ALF 672 generates an ALF signal 674, which is provided to a reference picture buffer 676. The ALF signal 674 may include data from one or more pictures that may be used as reference pictures.

The entropy coding module 642 may encode the TQC 638 to generate a bitstream A 614a (e.g., encoded picture data). For example, the entropy coding module 642 may encode the TQC 638 using context-adaptive variable length coding (CAVLC) or context-adaptive binary arithmetic coding (CABAC). Specifically, the entropy coding module 642 may encode the TQC 638 based on one or more of the intra mode information 640, the inter mode information 648, and the SAO information 668. The bitstream A 614a (e.g., encoded picture data) may be provided to the message generation module 608. The message generation module 608 may be configured similarly to the message generation module 108 described in conjunction with FIG. Additionally or alternatively, the message generation module 608 may perform one or more of the processes described in conjunction with FIG. 2 and FIG. 3.

For example, if the first picture (e.g., the first picture in bitstream A 614a) is a CRA picture and if a preceding picture exists, the message generation module 608 may generate a message (e.g., a buffering period SEI message or other message) including one or more of a CRA preceding picture discard flag, an initial CRA CPB removal delay parameter, and an initial CRA CPB removal delay offset parameter. In some configurations, the message may be inserted into bitstream A 614a to generate bitstream B 614b. Thus, the message may be generated after all of bitstream A 614a has been generated (e.g., after a substantial portion of bitstream B 614b has been generated). In other configurations, the message may not be inserted into bitstream A 614 (in which case bitstream B 614b may be identical to bitstream A 614a) but may be provided in a separate transmission 610.

In some configurations, the electronic device 602 transmits a bitstream 614 to another electronic device. For example, the bitstream 614 can be provided to a communication interface, a network interface, a wireless transmitter, a port, etc. For example, the bitstream 614 can be transmitted to another electronic device via a LAN, the Internet, a cellular phone base station, etc. Additionally or alternatively, the bitstream 614 can be stored in a memory or other component of the electronic device 602.

Figure 7 is a block diagram illustrating one configuration of a decoder 712 on an electronic device 702. The decoder 712 may be included in the electronic device 702. For example, the decoder 712 may be an HEVC decoder. The decoder 712 and one or more elements shown as included in the decoder 712 may be implemented in hardware, software, or a combination of both. The decoder 712 may receive a bitstream 714 (e.g., one or more coded pictures and overhead data included in the bitstream 714) for decoding. In some configurations, the received bitstream 714 may include received overhead data, such as messages (e.g., a buffering period SEI message or other messages), slice headers, PPS, etc. In some configurations, the decoder 712 may also receive a separate transmission 710. The separate transmission 710 may include messages (e.g., a buffering period SEI message or other messages). For example, the buffering period SEI message or other messages may be received in the separate transmission 710 (rather than in the bitstream 714). However, it should be noted that the separate transmission 710 is optional and may not be used in some configurations.

The decoder 712 includes a CPB 720. The CPB 720 can be configured similarly to the CPB 120 described above in conjunction with FIG. 1 . Additionally or alternatively, the decoder 712 can perform one or more of the processes described in conjunction with FIG. 4 and FIG. 5 . For example, the decoder 712 can receive a message (e.g., a buffering period SEI message or other message). Additionally, if the first access unit is a CRA access unit, if no preceding picture exists and if the CRA preceding picture discard flag indicates discard, the decoder 712 can perform one or more of the following operations: setting an initial removal delay variable to an initial CRA CPB removal delay parameter, and removing the first access unit based on the initial CRA CPB removal delay parameter. Note that one or more access units can be included in the bitstream and can include one or more of coded picture data and overhead data.

The coded picture buffer (CPB) 720 may provide the coded picture data to the entropy decoding module 701. The coded picture data may be entropy decoded by the entropy decoding module 701 to generate a motion information signal 703 and quantized, scaled and/or transformed coefficients 705.

At the motion compensation module 780, the motion information signal 703 may be combined with a portion of the reference frame signal 798 from the frame memory 709, which may generate an inter-frame prediction signal 782. The quantized, descaled, and/or transformed coefficients 705 may be inversely quantized, scaled, and inversely transformed by the inverse module 707, thereby generating a decoded residual signal 784. The decoded residual signal 784 may be added to the prediction signal 792 to generate a combined signal 786. The prediction signal 792 may be a signal selected from the inter-frame prediction signal 782 generated by the motion compensation module 780 or the intra-frame prediction signal 790 generated by the intra-frame prediction module 788. In some configurations, this signal selection may be based on (e.g., controlled by) the bitstream 714.

The intra prediction signal 790 may be predicted based on previously decoded information from the merge signal 786 (e.g., in the current frame). The merge signal 786 may also be filtered by a deblocking filter 794. The resulting filtered signal 796 may be written to the frame memory 709. The resulting filtered signal 796 may include a decoded picture. The frame memory 709 may provide a decoded picture 718.

FIG8 illustrates various components that may be used in a transmitting electronic device 802. One or more of the electronic devices 102, 602, and 702 described herein may be implemented according to the transmitting electronic device 802 shown in FIG8.

The transmitting electronic device 802 includes a processor 817 that controls the operation of the electronic device 802. The processor 817 may also be referred to as a CPU. The memory 811 provides instructions 813a (e.g., executable instructions) and data 815a to the processor 817. The memory 811 may include read-only memory (ROM), random access memory (RAM), or any other type of device that can store information. A portion of the memory 811 may also include non-volatile random access memory (NVRAM). The memory 811 may be in electronic communication with the processor 817.

Instructions 813b and data 815b may reside in processor 817. Instructions 813b and/or data 815b loaded into processor 817 may also include instructions 813a and/or data 815a loaded from memory 811 for execution or processing by processor 817. Instructions 813b may be executed by processor 817 to implement the systems and methods described herein. For example, instructions 813b may be executable to perform one or more of methods 200, 300, 400, and 500 described above.

The transmitting electronic device 802 may include one or more communication interfaces 819 for communicating with other electronic devices (e.g., receiving electronic devices). The communication interface 819 may be based on wired communication technology, wireless communication technology, or both. Examples of the communication interface 819 include a serial port, a parallel port, a universal serial bus (USB), an Ethernet adapter, an IEEE 1394 bus interface, a small computer system interface (SCSI) bus interface, an infrared (IR) communication port, a Bluetooth wireless communication adapter, a wireless transceiver according to the Third Generation Partnership Project (3GPP) specification, and the like.

The transmitting electronic device 802 may include one or more output devices 823 and one or more input devices 821. Examples of output devices 823 include speakers, printers, and the like. One output device that may be included in the electronic device 802 is a display device 825. The display device 825 used with the configurations disclosed herein may utilize any suitable image projection technology, such as a cathode ray tube (CRT), liquid crystal display (LCD), light emitting diode (LED), gas plasma, electroluminescence, and the like. A display controller 827 may be provided for converting data stored in the memory 811 into text, graphics, and/or moving images (as appropriate) displayed on the display 825. Examples of input devices 821 include a keyboard, mouse, microphone, remote control device, buttons, joystick, trackball, touchpad, touch screen, laser pointer, and the like.

The various components of the transmitting electronic device 802 are coupled together by a bus system 829. In addition to the data bus, the bus system 829 may also include a power bus, a control signal bus, and a status signal bus. However, for clarity, FIG8 illustrates the various buses as the bus system 829. The transmitting electronic device 802 shown in FIG8 is a functional block diagram rather than a list of specific components.

9 is a block diagram illustrating various components that may be used in a receiving electronic device 902. One or more of the electronic devices 102, 602, 702 described herein may be implemented according to the receiving electronic device 902 shown in FIG.

The receiving electronic device 902 includes a processor 917 that controls the operation of the electronic device 902. The processor 917 may also be referred to as a CPU. The memory 911 provides instructions 913a (e.g., executable instructions) and data 915a to the processor 917. The memory 911 may include read-only memory (ROM), random access memory (RAM), or any other type of device capable of storing information. A portion of the memory 911 may also include non-volatile random access memory (NVRAM). The memory 911 may be in electronic communication with the processor 917.

Instructions 913b and data 915b may also reside in the processor 917. The instructions 913b and/or data 915b loaded into the processor 917 may also include instructions 913a and/or data 915a loaded from the memory 911 for execution or processing by the processor 917. The instructions 913b may be executed by the processor 917 to implement the systems and methods described herein. For example, the instructions 913b may be executable to perform one or more of the methods 200, 300, 400, and 500 described above.

The receiving electronic device 902 may include one or more communication interfaces 919 for communicating with other electronic devices (e.g., the sending electronic device). The communication interface 919 may be based on a wired communication technology, a wireless communication technology, or both. Examples of the communication interface 919 include a serial port, a parallel port, a universal serial bus (USB), an Ethernet adapter, an IEEE 1394 bus interface, a small computer system interface (SCSI) bus interface, an infrared (IR) communication port, a Bluetooth wireless communication adapter, a wireless transceiver according to the Third Generation Partnership Project (3GPP) specification, and the like.

The transmitting electronic device 902 may include one or more output devices 923 and one or more input devices 921. Examples of output devices 923 include speakers, printers, and the like. One output device that may be included in the electronic device 902 is a display device 925. The display device 925 used with the configurations disclosed herein may use any suitable image projection technology, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a light emitting diode (LED), gas plasma, electroluminescence, and the like. A display controller 927 may be provided for converting data stored in the memory 911 into text, graphics, and/or moving images (as appropriate) for display on the display 925. Examples of input devices 921 include a keyboard, a mouse, a microphone, a remote control device, buttons, a joystick, a trackball, a touchpad, a touch screen, a laser pointer, and the like.

The various components of the transmitting electronic device 902 are coupled together by a bus system 929. In addition to the data bus, the bus system 929 may also include a power bus, a control signal bus, and a status signal bus. However, for clarity, FIG9 illustrates the various buses as the bus system 929. The receiving electronic device 902 shown in FIG9 is a functional block diagram rather than a list of specific components.

FIG10 is a block diagram illustrating one configuration of an electronic device 1002 in which the system and method for sending a message can be implemented. Electronic device 1002 includes an encoding device 1031 and a transmitting device 1033. Encoding device 1031 and transmitting device 1033 can be configured to perform one or more of the functions described above in conjunction with one or more of FIG1 , FIG2 , FIG3 , FIG6 , and FIG8 . For example, encoding device 1031 and transmitting device 1033 can generate a bitstream 1014. FIG8 illustrates an example of an actual device structure of FIG10 . Various other structures can be implemented to achieve one or more of the functions of FIG1 , FIG2 , FIG3 , FIG6 , and FIG8 . For example, a DSP can be implemented using software.

Figure 11 is a block diagram illustrating one configuration of an electronic device 1102 in which the system and method for buffering a bitstream 1114 can be implemented. Electronic device 1102 includes a receiving device 1135 and a decoding device 1137. Receiving device 1135 and decoding device 1137 can be configured to perform one or more of the functions described above in conjunction with one or more of Figures 1, 4, 5, 7, and 9. For example, receiving device 1135 and decoding device 1137 can receive bitstream 1114. Figure 9 above illustrates an example of a practical device structure for Figure 11. Various other structures can be implemented to achieve one or more of the functions of Figures 1, 4, 5, 7, and 9. For example, a DSP can be implemented using software.

The term "computer-readable medium" refers to any available medium that can be accessed by a computer or processor. As used herein, the term "computer-readable medium" may refer to a non-transitory and tangible computer and/or processor-readable medium. For example, and not limitation, computer-readable or processor-readable media may include: RAM, ROM, EEPROM, CD-COM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices; or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures that can be accessed by a computer or processor. As used herein, magnetic disks and optical disks include: compact disks (CDs), laser disks, optical disks, digital video disks (DVDs), floppy disks, and Blu-ray (registered trademark) disks, where magnetic disks typically reproduce data magnetically, while optical disks reproduce data optically with lasers.

It should be noted that one or more of the methods described herein may be implemented in hardware and/or performed using hardware. For example, one or more of the methods or solutions described herein may be implemented in and/or using a chip, an ASIC, a large-scale integrated circuit (LSI), or an integrated circuit.

Each method disclosed herein includes one or more steps or actions for implementing the method. The method steps and/or actions may be interchanged with other steps and/or actions, and/or combined into a single step, without departing from the scope of the claims. In other words, unless a specific order of steps or actions is required for the proper operation of the method, the order and/or use of the specific steps and/or actions may be modified without departing from the scope of the claims.

It is to be understood that the claims are not limited to the precise configuration and components described above. Various modifications, changes and variations may be made in the arrangement, operation and details of the systems, methods and apparatus described herein without departing from the scope of the claims.

Claims (5)

1. A method for buffering a bit stream, comprising: Decoding is performed using the bitstream generated through the following encoding steps: encoding a first initial encoded picture buffer (CPB) removal delay parameter, a flag indicating the presence of a second initial CPB removal delay parameter, and the second initial CPB removal delay parameter when the flag indicates its presence. in When the set of conditions is met, the initial removal delay variable is set to the second initial CPB removal delay parameter; when the set of conditions is not met, the initial removal delay variable is set to the first initial CPB removal delay parameter. The set of conditions includes the following items: 1) The first access unit includes only I-stripes. 2) No preceding image exists, and 3) The mark is equal to 1. 2. The method according to claim 1, In checking bitstream consistency and output timing decoder consistency: When the condition set is met, the initial removal delay variable is set to the second initial CPB removal delay parameter, and when the condition set is not met, the initial removal delay variable is set to the first initial CPB removal delay parameter. 3. An electronic device for buffering bit streams, comprising: The decoding circuit is configured to decode the bitstream generated by using the following encoding circuit. The encoding circuit is configured to encode a first initial encoded image buffer (CPB) removal delay parameter, a flag indicating the presence of a second initial CPB removal delay parameter, and the second initial CPB removal delay parameter when the flag indicates its presence. in, When the set of conditions is met, the initial removal delay variable is set to the second initial CPB removal delay parameter; when the set of conditions is not met, the initial removal delay variable is set to the first initial CPB removal delay parameter. The set of conditions includes the following items: 1) The first access unit includes only I-stripes. 2) No preceding image exists, and 3) The mark is equal to 1. 4. The electronic device according to claim 3, In checking bitstream consistency and output timing decoder consistency: When the condition set is met, the initial removal delay variable is set to the second initial CPB removal delay parameter, and when the condition set is not met, the initial removal delay variable is set to the first CPB removal delay parameter. 5. An electronic device for buffering a bit stream consisting of multiple blocks, comprising: Memory; and A processor, wherein the processor is configured to perform the following steps: Decoding is performed using a bitstream generated through the following encoding steps: encoding a first initial CPB removal delay parameter, a flag indicating the presence of a second initial CPB removal delay parameter, and the second initial CPB removal delay parameter when the flag indicates its presence. in, When the set of conditions is met, the initial removal delay variable is set to the second initial CPB removal delay parameter; when the set of conditions is not met, the initial removal delay variable is set to the first initial CPB removal delay parameter. The set of conditions includes the following items: 1) The first access unit includes only I-stripes. 2) No preceding image exists, and 3) The mark is equal to 1.