HK1245491B - Computer-readable medium - Google Patents

Description

Computer readable media

This application is a divisional application of the PCT invention patent application with application number 201280023547.2, application date March 19, 2012, and name “Audio encoder and decoder with flexible configuration function”.

Technical Field

The present invention relates to audio coding, in particular to high-quality and low-bit-rate coding, such as is known from the so-called USAC coding (USAC=Unified Speech and Audio Coding).

Background Art

The USAC codec is defined in ISO/IEC CD 23003-1. This standard, entitled "Information Technology - Moving Picture Experts Group (MPEG) Audio Techniques - Part 3: Unified Speech and Audio Coding", details the functional blocks of the reference model for the call for recommendations on unified speech and audio coding.

Figures 10a and 10b show block diagrams of the encoder and decoder. The block diagrams of the USAC encoder and decoder reflect the structure of MPEG-D USAC coding. The general structure can be described as follows: First, there is a common pre-/post-processing including an MPEG Surround (MPEGS) functional unit and an enhanced SBR (eSBR) unit. The MPEGS functional unit handles stereo or multi-channel processing, and the eSBR handles the parametric representation of higher audio frequencies in the input signal. Then, there are two branches, one branch including an improved Advanced Audio Coding (AAC) tool path and the other branch including a path based on linear predictive coding (LP or LPC domain), which in turn features a frequency domain representation or a time domain representation of the LPC residual. All transmission spectra for both AAC and LPC are represented in the modified discrete cosine transform (MDCT) domain after quantization and arithmetic coding. The time domain representation uses the Algebraic Coded Excited Linear Prediction (ACELP) excitation coding scheme.

The basic structure of MPEG-D USAC is shown in Figures 10a and 10b. The data flow in the figure is from left to right and from top to bottom. The decoder's function is to find the description of the quantized audio spectrum or time domain representation in the bitstream payload and decode the quantized values and other reconstruction information.

In the case of transmitted spectral information, the decoder will reconstruct the quantized spectrum, process the reconstructed spectrum through any tools available in the bitstream payload to arrive at the actual signal spectrum as described by the input bitstream payload, and finally convert the frequency domain spectrum to the time domain. After the initial reconstruction and scaling of the spectral reconstruction, there are optional tools that modify one or more of the spectra in the spectrum to provide more efficient encoding.

In case a time domain signal representation is transmitted, the decoder shall reconstruct the quantized time signal, processing the reconstructed time signal by any tools available in the bitstream payload to arrive at the actual time domain signal as described by the input bitstream payload.

For each optional tool that operates on signal data, the option "through" is retained, and in all cases where processing is omitted, the spectrum or time samples at its input are passed directly through the tool without modification.

In the case where the bitstream changes its signal representation from time domain to frequency domain representation or from LP domain to non-LP domain, or from frequency domain representation to time domain representation or from non-LP domain to LP domain, the decoder will use appropriate conversion overlap-add windowing method to facilitate the transition from one domain to another.

After the transition process, eSBR and MPEGS processing are applied to both encoding paths in the same way.

The input to the Bitstream Payload Demultiplexer tool is the MPEG-D USAC bitstream payload. The Demultiplexer separates the bitstream payload into parts for each tool and provides each tool with the bitstream payload information relevant to that tool.

The output from the Bitstream Payload Demultiplexer tool is:

●Depends on the core encoding type of the current frame, which is:

○ The quantized and noiselessly encoded spectrum represented by

○ Scale factor information

○ Spectral lines of arithmetic coding

• or: Linear Prediction (LP) parameters together with an excitation signal represented by either:

o quantized and arithmetically coded spectral lines (transform coded excitation, TCX) or

○ACELP coded time domain excitation

●Spectrum noise filling information (optional)

●M/S decision information (optional)

Temporal Noise Shaping (TNS) information (optional)

●Filter bank control information

●Time expansion (TW) control information (optional)

Enhanced Spectrum Bandwidth Replication (eSBR) control information (optional)

MPEG Surround (MPEGS) control information

The Scale Factor Noiseless Decoding Tool takes information from the bitstream payload demultiplexer, parses this information and decodes the Huffman and DPCM coded scale factors.

The input to the scale factor noiseless decoding tool is:

● Scale factor information for noiselessly encoded spectrum

The output of the scale factor noiseless decoding tool is:

• The decoded integer representation of the scale factor.

The Spectral Noiseless Decoding Tool takes information from the Bitstream Payload Demultiplexer, parses the information, decodes the arithmetic coded data and reconstructs the quantized spectrum. The input to the Noiseless Decoding Tool is:

●Noiselessly coded spectrum

The output of the noiseless decoding tool is:

●Quantized value of the spectrum.

The inverse quantizer tool takes the quantized values of the spectrum and transforms the integer values into an unscaled reconstructed spectrum. The quantizer is a companding quantizer, and its scaling factor depends on the selected core coding mode.

The input to the Inverse Quantizer tool is:

●Quantization value for the spectrum

The output of the Inverse Quantizer tool is:

●Unscaled inverse quantized spectrum

The noise filling tool is used to fill spectral gaps in the decoded spectrum, which occur when spectral values are quantized to zero, eg due to strict constraints on bit requirements in the encoder.The use of the noise filling tool is optional.

The input to the Noise Fill tool is:

●Unscaled inverse quantized spectrum

●Noise filling parameters

● The decoded integer representation of the scale factor

The output of the Noise Fill tool is:

● Unscaled inverse quantized spectral values for spectral lines that were previously quantized to zero

● Modified integer representation of the scale factor

The rescaling tool converts the integer representation of the scale factors into real values and multiplies the unscaled inverse quantized spectrum by the associated scale factors.

The inputs to the Scale Factor tool are:

● The decoded integer representation of the scale factor

●Unscaled inverse quantized spectrum

The output from the Scale Factor tool is:

●Scaled inverse quantized spectrum

For an overview of M/S tools, refer to ISO/IEC 14496-3:2009, 4.1.1.2.

For an overview of temporal noise shaping (TNS) tools, refer to ISO/IEC 14496-3:2009, 4.1.1.2.

The filter bank/block switching tool applies the inverse of the frequency mapping implemented in the encoder. The inverse modified discrete cosine transform (IMDCT) is used for the filter bank tool. The IMDCT can be configured to support 120, 128, 240, 256, 480, 512, 960 or 1024 spectral coefficients.

The input to the Filter Bank tool is:

●(Inverse quantization) spectrum

●Filter bank control information

The output from the filterbank tool is:

●Reconstruct audio signal in time domain

When time warping mode is enabled, a time- warped filterbank/block switching tool replaces the normal filterbank/block switching tool. The filterbank is the same as the normal filterbank (IMDCT), except that the windowed time domain samples are mapped from the warped time domain to the linear time domain via time-shifted resampling.

The input to the Time Warping Filterbank tool is:

●Inverse quantized spectrum

●Filter bank control information

Time warping control information

The output from the filterbank tool is:

●Linear time domain reconstruction of audio signals.

The enhanced SBR (eSBR) tool regenerates the high-frequency band of an audio signal. It is based on replicating the harmonic sequence truncated during encoding. It adjusts the spectral envelope of the generated high-frequency band and applies inverse filtering, as well as adding noise and sinusoidal components to recreate the spectral characteristics of the original signal.

The input to the eSBR tool is:

●Quantified envelope data

Comprehensive control data

●Time domain signal from frequency domain core decoder or ACELP/TCX core decoder

The output of the eSBR tool is:

●Time domain signal, or

• A QMF domain representation of the signal, for example in case of using MPEG Surround tools.

MPEG Surround (MPEGS) tools generate multiple signals from one or more input signals by applying a complex upmixing procedure to the input signals controlled by appropriate spatial parameters. In the context of USAC, MPEGS is used to encode multi-channel signals by transmitting parametric side information along with the transmitted downmix signal.

The input to the MPEGS tool is:

● Downmixed time domain signal, or

●QMF domain representation of the downmix signal from the eSBR tool

The output of the MPEGS tool is:

●Multi-channel time domain signal

The Signal Classifier tool analyzes the original input signal and generates control information based on it that triggers the selection of different coding modes. The analysis of the input signal is implementation-dependent and will attempt to select the best core coding mode for a given input signal frame. The output of the Signal Classifier can also (optionally) be used to influence the behavior of other tools (such as MPEG Surround, Enhanced SBR, Time Warping Filter Bank, and others).

The input to the Signal Classifier tool is:

●Original unmodified input signal

● Additional implementation-dependent parameters

The output of the Signal Classifier tool is:

• A control signal that controls the selection of the core codec (frequency domain coding without LP filtering, frequency domain coding with LP filtering, or time domain coding with LP filtering).

The ACELP tool provides a way to efficiently represent the time domain excitation signal by combining a long-term predictor (adaptive codeword) with a pulse-like sequence (innovative codeword).The reconstructed excitation is sent through an LP synthesis filter to form the time domain signal.

The input to the ACELP tool is:

● Adaptability and Innovation Codebook Index

●Adaptability and innovative code gain

●Other control data

●Inverse quantized and interpolated LPC filter coefficients

The output of the ACELP tool is:

●Audio signal reconstructed in time domain

The MDCT-based TCX decoding tool is used to transform the weighted LP residual representation from the MDCT domain back to the time domain signal and output a time domain signal including the weighted LP synthesis filtered IMDCT can be configured to support 256, 512 or 1024 spectral coefficients.

The input to the TCX tool is:

●(Inverse quantization) MDCT spectrum

●Inverse quantized and interpolated LPC filter coefficients

The output of the TCX tool is:

●Reconstruct audio signal in time domain

The techniques disclosed in ISO/IEC CD 23003-3 (which is incorporated herein by reference) allow for the definition of, for example, a channel element as a single channel element containing only payload for a single channel, or a channel element as a channel pair element including payload for two channels, or a channel element as an LFE (Low Frequency Enhancement) channel element including payload for the LFE channel.

A five-channel multi-channel audio signal can, for example, be represented by the following channel elements: a single channel element comprising a center channel; a first channel pair element comprising a left channel and a right channel; and a second channel pair element comprising a left surround channel (Ls) and a right surround channel (Rs). These different channel elements, collectively representing the multi-channel audio signal, are fed into a decoder and processed using the same decoder configuration. According to the prior art, the decoder applies the decoder configuration, transmitted in a USAC-specific configuration element, to all channel elements. Therefore, it is not possible to optimally select an element of the configuration that is valid for all channel elements for each channel element, but rather to set it simultaneously for all channel elements. However, it has been found that the channel elements used to describe a direct five-channel multi-channel signal are very different from one another. The center channel, as a single channel element, has significantly different characteristics from the channel pair elements describing the left/right channels and the left/right surround channels. Furthermore, the characteristics of the two channel pair elements are also significantly different because the surround channels contain information that is significantly different from that contained in the left and right channels.

The selection of configuration data for each channel element necessitates a compromise, such that a configuration may be selected that is not optimal for all channel elements, but represents a compromise between all channel elements. Alternatively, a configuration that is optimal for one channel element may be selected, but this inevitably results in a situation where the configuration is suboptimal for other channel elements. This, however, results in an increased bitrate for the channel elements with the suboptimal configuration, or alternatively or additionally, a reduction in audio quality for those channel elements that do not have the optimal configuration settings.

Summary of the Invention

It is therefore an object of the present invention to provide an improved audio encoding/decoding concept.

This object is achieved by an audio decoder according to claim 1, a method of audio decoding according to claim 14, an audio encoder according to claim 15, a method of audio encoding according to claim 16, a computer program according to claim 17 and an encoded audio signal according to claim 18.

The present invention is based on the discovery that an improved audio encoding/decoding concept is achieved when transmitting decoder configuration data for individual channel elements. According to the present invention, the encoded audio signal thus comprises a first channel element and a second channel element in a payload section of the data stream; and first decoder configuration data for the first channel element and second decoder configuration data for the second channel element in a configuration section of the data stream. Thus, the payload section of the data stream, in which the payload data for the channel elements are located, is separated from the configuration data section of the data stream, in which the configuration data for the channel elements are located. Preferably, the configuration section is a continuous portion of a serial bit stream, wherein all bits belonging to the payload section or continuous portion of the bit stream are configuration data. Preferably, the configuration data section is followed by a payload section of the data stream, in which the payload data for the channel elements are located. The audio decoder of the present invention comprises a data stream reader for reading the configuration data for each channel element in the configuration section and for reading the payload data for each channel element in the payload section. In addition, the audio decoder includes a configurable decoder for decoding multiple channel elements and a configuration controller for configuring the configurable decoder, so that when decoding a first channel element, the configurable decoder is configured according to first decoder configuration data, and when decoding a second channel element, the configurable decoder is configured according to second decoder configuration data.

Thus, it is ensured that an optimal configuration can be selected for each channel element. This allows the different characteristics of the different channel elements to be optimally taken into account.

The audio encoder according to the present invention is arranged for encoding a multi-channel audio signal having, for example, at least two, three or preferably more than three channels. The audio encoder comprises: a configuration processor for generating first configuration data for a first channel element and second configuration data for a second channel element; and a configurable encoder for encoding the multi-channel audio signal using the first configuration data and the second configuration data, respectively, to obtain the first channel element and the second channel element. In addition, the audio encoder comprises a data stream generator for generating a data stream representing the encoded audio signal, the data stream having: a configuration section having the first configuration data and the second configuration data; and a payload section comprising the first channel element and the second channel element.

The encoder and decoder in this case now determine the respective preferred optimal configuration data for each channel element.

This ensures that the configurable decoder for each channel element is configured such that for each channel element the best choice with respect to audio quality and bitrate is obtained and no compromises need to be made.

BRIEF DESCRIPTION OF THE DRAWINGS

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings, in which:

Figure 1 is a block diagram of a decoder;

FIG2 is a block diagram of an encoder;

Figures 3a and 3b show tables summarizing the channel configurations for different loudspeaker setups;

Figures 4a and 4b identify and graphically illustrate different speaker setups;

5a to 5d show different aspects of an encoded audio signal having a configuration section and a payload section;

Figure 6a shows the syntax of the UsacConfig element;

Figure 6b shows the syntax of the UsacChannelConfig element;

FIG6 c shows the syntax of UsacDecoderConfig;

Figure 6d shows the syntax of UsacSingleChannelElementConfig;

Figure 6e shows the syntax of UsacChannelPairElementConfig;

Figure 6f shows the syntax of UsacLfeElementConfig;

Figure 6g shows the syntax of UsacCoreConfig;

Figure 6h shows the syntax of SbrConfig;

FIG6i shows the syntax of SbrDfltHeader;

Figure 6j shows the syntax of Mps212Config;

Figure 6k shows the syntax of UsacExtElementConfig;

FIG61 shows the syntax of UsacConfigExtension;

Figure 6m shows the syntax of escapedValue;

FIG7 shows different alternatives for identifying and configuring the different encoder/decoder tools of channel elements, respectively;

FIG8 shows a preferred embodiment of a decoder implementation with parallel operating decoder instances for generating a 5.1 multi-channel audio signal;

FIG9 shows a preferred implementation of the decoder of FIG1 in the form of a flow chart;

Figure 10a shows a block diagram of a USAC encoder; and

FIG10 b shows a block diagram of a USAC decoder.

DETAILED DESCRIPTION

High-level information about the contained audio content (e.g., sampling rate, exact channel configuration) is present in the audio bitstream. This makes the bitstream more self-contained and makes transmission of the configuration and payload easier when embedded in a transport scheme that may not have a means to explicitly transmit this information.

The configuration structure contains an index (coreSbrFrameLengthIndex) of the combined frame length and spectral bandwidth replication (SBR) sampling rate ratio. This ensures that both values are transmitted efficiently and that meaningless combinations of frame length and SBR ratio cannot be communicated. The latter simplifies decoder implementation.

The configuration can be extended by means of a dedicated configuration extension mechanism. This will prevent large and inefficient transmission of configuration extensions as known from MPEG-4 AudioSpecificConfig().

This configuration allows for free communication of the speaker positions associated with each transmitted audio channel.Communication of common channel to speaker mappings can be efficiently communicated by means of the channelConfigurationIndex.

The configuration of each channel element is contained in a separate structure, allowing each channel element to be configured independently.

The SBR configuration data ("SBR header") is divided into SbrInfo() and SbrHeader(). For SbrHeader(), a default version (SbrDfltHeader()) is defined that can be efficiently referenced in the bitstream. This reduces the bit requirements at locations where the SBR configuration data needs to be retransmitted.

Configuration changes that are more commonly applied to SBR can be efficiently communicated with the help of the SbrInfo() syntax element.

The configurations for Parametric Bandwidth Extension (SBR) and Parametric Stereo Coding tools (MPS212, also known as MPEG Surround 2-1-2) are tightly integrated into the USAC configuration structure, which means that the two technologies are actually better implemented in the standard.

The syntax features an extension mechanism that allows the transmission of existing and future extensions to the codec.

Extensions can be placed next to channel elements in any order (ie interleaved). This allows extensions to be read before or after the specific channel element to which the extension is applied.

A default length can be defined for syntax extensions, which makes the transmission of constant-length extensions very efficient since the length of the extension payload does not need to be transmitted every time.

The common case of conveying values with the help of escape mechanisms to extend the range of values, if necessary, is modularized into a dedicated real syntax element (escapedValue()) that is flexible enough to cover all expected escape value complexes and bitfield extensions.

Bitstream Configuration

UsacConfig() (Figure 6a)

UsacConfig() has been extended to include information about the contained audio content and everything needed for a complete decoder setup. Top-level information about the audio (sampling rate, channel configuration, output frame length) is gathered at the beginning for easy access from higher (application) layers.

channelConfigurationIndex, UsacChannelConfig() (Figure 6b)

Such an element gives information about the contained bitstream elements and their mapping to loudspeakers.channelConfigurationIndex allows an easy and convenient way of conveying which one of a range of predefined mono, stereo or multi-channel configurations is considered to be actually relevant.

For more elaborate configurations not covered by channelConfigurationIndex, UsacChannelConfig() allows free assignment of elements to speaker positions from a list of 32 speaker positions covering all currently known speaker positions in all known speaker setups for home or cinema sound reproduction.

This list of speaker positions is a superset of the list featured in the MPEG Surround standard (see Table 1 and Figure 1 of ISO/IEC 23003-1). Four additional speaker positions have been added to cover the recently introduced 22.2 speaker setup (see Figures 3a, 3b, 4a, and 4b).

UsacDecoderConfig() (Figure 6c)

This element is placed at a key position in the decoder configuration so that it contains all the additional information the decoder needs to interpret the bitstream.

Specifically, the structure of a bitstream is defined herein by explicitly stating the number of elements in the bitstream and their order.

Then, looping over all elements allows configuration of all elements of all types (single, pair, lfe, extended).

UsacConfigExtension() (Figure 6l)

To allow for future extensions, the configuration features a robust mechanism for extending the configuration for not yet existing configuration extensions of USAC.

UsacSingleChannelElementConfig() (Figure 6d)

This element configuration contains all the information needed to configure the decoder to decode a single pass. This is basically the information related to the core encoder, and if SBR is used, the information related to SBR.

UsacChannelPairElementConfig() (Figure 6e)

Similar to the above, this element configuration contains all the information needed to configure the decoder to decode a channel pair. In addition to the core configuration and SBR configuration mentioned above, it also includes stereo-specific configurations, such as the exact type of stereo encoding applied (with or without MPS212, residual, etc.). Note that this element covers the full range of stereo encoding options available in USAC.

UsacLfeElementConfig() (Figure 6f)

Because the LFE element has a static configuration, the LFE element configuration does not contain configuration data.

UsacExtElementConfig() (Figure 6k)

This element configuration can be used to configure any type of existing or future extension to the codec. Each extension element type has its own dedicated ID value. A length field is included to facilitate skipping configuration extensions unknown to the decoder. The optional definition of a default payload length further improves the coding efficiency of extension payloads present in the actual bitstream.

Known extensions that are foreseen for combination with USAC include: MPEG Surround, SAOC and some kind of FIL element known from MPEG-4 AAC.

UsacCoreConfig() (Figure 6g)

This element contains configuration data that affects the core encoder settings. Currently, these are switches for the Time Warp tool and the Noise Fill tool.

SbrConfig() (Figure 6h)

To reduce the bit overhead caused by frequent retransmissions of sbr_header(), the default values for elements of sbr_header() that are usually kept constant are now carried in the configuration element SbrDfltHeader(). In addition, static SBR configuration elements are also carried in SbrConfig(). These static bits include flags for enabling or disabling specific features of Enhanced SBR, such as harmonic transposition or inter-time envelope shaping (inter-TES).

SbrDfltHeader() (Figure 6i)

This element carries the sbr_header() element which normally remains constant. Elements affecting things like amplitude resolution, crossbands, spectrum pre-flattening are now carried in SbrInfo() which allows them to be effectively changed in real time.

Mps212Config() (Figure 6j)

Similar to the SBR configuration above, all the settings parameters for the MPEG Surround 2-1-2 tool are gathered in this configuration. All elements from SpatialSpecificConfig() that are not relevant to the context or redundant are removed.

Bitstream Payload

UsacFrame()

It is the outermost wrapper around the USAC bitstream payload and represents a USAC access unit. It contains a loop through all contained channel elements and extension elements as conveyed in the config section. This makes the bitstream format significantly more flexible in what it can contain and is future-proof for any future extensions.

UsacSingleChannelElement()

This element contains all the data required to decode the mono stream. The content is divided into a core encoder-related part and an eSBR-related part. The eSBR-related part is now significantly more tightly connected to the core, which also significantly better reflects the order in which the decoder needs the data.

UsacChannelPairElement()

This element covers data for all possible ways of encoding a stereo pair. Specifically, it covers all styles of unified stereo coding, from traditional M/S based coding to fully parametric stereo coding with the help of MPEG Surround 2-1-2. stereoConfigIndex indicates the actual style used. Appropriate eSBR data and MPEG Surround 2-1-2 data are sent in this element. UsacLfeElement()

Only the previous lfe_channel_element() has been renamed to follow a consistent naming scheme. UsacExtElement()

Extension elements are carefully designed to maximize flexibility while maximizing efficiency, even for extensions with small (or often no) payloads. Extension payload lengths are communicated to naive decoders to skip them. User-defined extensions can be communicated using a reserved range of extension types. Extensions can be freely placed in element order. A range of extension elements are considered, including a mechanism for writing padding bytes.

UsacCoreCoderData()

This new element summarizes all information affecting the core encoder and therefore also includes fd_channel_stream() and lpd_channel_stream().

StereoCoreToolInfo()

To ease the readability of the syntax, all stereo related information is captured in this element. It handles the numerous dependencies of the bits in the stereo coding mode.

UsacSbrData()

The CRC functionality element and legacy description elements for Scalable Audio Coding are removed from what used to be the sbr_extension_data() element. To reduce the overhead caused by frequent retransmission of SBR information and header data, their presence can be explicitly communicated.

SbrInfo()

SBR configuration data is frequently modified in real time. This includes elements that previously required the transmission of a complete sbr_header() to control things like amplitude resolution, crossbands, and spectral pre-flattening (see 6.3, "Efficiency" in [N11660]).

SbrHeader()

To maintain SBR's ability to change the values in sbr_header() in real time, SbrHeader() can now be carried within UsacSbrData() in cases where other values than those sent in SbrDfltHeader() should be used. The bs_header_extra mechanism is maintained to keep overhead as low as possible for the most common cases.

sbr_data()

Furthermore, the remainder of the SBR scalable coding is removed since it is not applicable in the USAC context. Depending on the number of channels, sbr_data() contains either a sbr_single_channel_element() or a sbr_channel_pair_element(). usacSamplingFrequencyIndex

This table is a superset of the table used in MPEG-4 to convey the sampling frequency of the audio codec. This table has been further extended to also cover the sampling rates currently used in the USAC mode of operation. Some multiples of the sampling frequency are also included.

channelConfigurationIndex

This table is a superset of the table used in MPEG-4 to communicate channelConfiguration. This table is further extended to allow communication of common and foreseen future speaker setups. Indexes in this table are communicated using 5 bits to allow for future extensions.

usacElementType

There are only four element types. One for each of the four basic bitstream elements: UsacSingleChannelElement(), UsacChannelPairElement(), UsacLfeElement(), and UsacExtElement(). These elements provide the required top-level structure while maintaining all the required flexibility.

usacExtElementType

Within UsacExtElement(), this element allows a plethora of extensions to be communicated. To be future-proof, the bitfield has been chosen to be large enough to allow for all conceivable extensions. Of the currently known extensions, a few are recommended for consideration: filler elements, MPEG surround, and SAOC.

usacConfigExtType

It may be necessary to extend the configuration at some point, so this can be handled by UsacConfigExtension(), which will then allow a type to be assigned to each new configuration. Currently the only type that can be conveyed is the population mechanism to use for that configuration.

coreSbrFrameLengthIndex

This table conveys several configuration aspects of the decoder. Specifically, these are the output frame length, the SBR ratio, and the resulting core encoder frame length (ccfl). It also indicates the number of synthesis bands used in the SBR and QMF analysis.

stereoConfigIndex

This table determines the internal structure of UsacChannelPairElement(). This table indicates the use of mono or stereo core, the use of MPS 212, whether stereo SBR is applied, and whether residual coding is applied in MPS 212.

By moving most of the eSBR header fields to a default header that can be referenced via the default header tag, the bit requirements for sending eSBR control data have been significantly reduced. The aforementioned sbr_header() bitfields, considered the most likely to change in real-world systems, have instead been outsourced to the sbrInfo() element, now consisting of only four elements covering a maximum of 8 bits. This saves 10 bits compared to the sbr_header() element, which consisted of at least 18 bits.

It is difficult to assess the impact of this change on the overall bitrate, as the overall bitrate depends strongly on the transmission rate of the eSBR control data in sbrInfo(). However, for the common use case of changing the sbr interleaving in the bitstream, the bit savings can be as high as 22 bits each time an sbrInfo() is sent instead of a full transmitted sbr_header().

The output of the USAC decoder can be further processed by MPEG Surround (MPS) (ISO/IEC 23003-1) or SAOC (ISO/IEC 23003-2). If the SBR tool in USAC is valid, the USAC decoder can usually be effectively combined with the subsequent MPS/SAOC decoder by connecting the USAC decoder and the subsequent MPS/SAOC decoder in the QMF domain in the same way as described for HE-AAC in ISO/IEC 23003-14.4. If the connection in the QMF domain is not feasible, they need to be connected in the time domain.

If MPS/SAOC side information is embedded into the USAC bitstream using the usacExtElement mechanism (where usacExtElementType is ID_EXT_ELE_MPEGS or ID_EXT_ELE_SAOC), the time alignment between the USAC data and the MPS/SAOC data presents the most efficient connection between the USAC decoder and the MPS/SAOC decoder. If the SBR tool in USAC is valid and if MPS/SAOC uses a 64-band QMF domain representation (see ISO/IEC 23003-16.6.3), the most efficient connection is in the QMF domain. Otherwise, the most efficient connection is in the time domain. This corresponds to the time alignment of the combination of MPS and HE-AAC as defined in ISO/IEC 23003-14.4, 4.5, and 7.2.1.

The additional delay introduced by adding MPS decoding after USAC decoding is given by ISO/IEC 23003-14.5 and depends on whether HQ MPS or LP MPS is used, and whether MPS is connected to USAC in the QMF domain or in the time domain.

ISO/IEC 23003-14.4 describes the interface between the USAC system and the MPEG system. Each access unit passed from the system interface to the audio decoder will result in a corresponding combination unit, or combiner, being passed from the audio decoder to the system interface. This will include a start condition and a shutdown condition, i.e., when an access unit is the first or last in a finite sequence of access units.

For audio combination units, ISO/IEC 14496-17.1.3.5 Combination Timestamp (CTS) specifies the combination time applied to the nth audio sample within the combination unit. For USAC, the value of n is always 1. Note that this applies to the output of the USAC decoder itself. In the case where the USAC decoder is combined with an MPS decoder, for example, the combination unit delivered at the output of the MPS decoder needs to be taken into account.

Features of USAC bitstream payload syntax

Table - Syntax of UseFrame()

Table - Syntax of UseSingleChannelElement()

Table - Syntax of UseChannelPairElement()

Table - Syntax of UseLfeElement()

Table - Syntax of UseExtElement()

Characteristics of the syntax of the attachment payload element

Table - Syntax of UseCoreCoderData()

Table - Syntax of StereoCoreToolInfo()

Table - Syntax of fd_channel_stream()

Table - Syntax of lpd_channel_stream()

Table - Syntax of fac_data()

Features of the Enhanced SBR Payload Syntax

Table - Syntax of UsacSbrData()

Table - Syntax of SbrInfo

Table - Syntax of SbrHeader

Table - Syntax of sbr_data()

Table - Syntax of sbr_envelope()

Table - Syntax of FramingInfo()

A short description of the data element

UsacConfig()

This element contains information about the contained audio content and everything needed for a complete decoder setup.

UsacChannelConfig()

This element gives information about the contained bitstream elements and their mapping to speakers.

UsacDecoderConfig()

This element contains all the additional information needed by the decoder to interpret the bitstream. Specifically, the SBR resampling rate is conveyed here, and the structure of the bitstream is defined here by explicitly stating the number of elements in the bitstream and their order.

UsacConfigExtension()

Configuration extension mechanism to extend the configuration for future configuration extensions of USAC.

UsacSingleChannelElementConfig()

It contains all the information needed to configure the decoder to decode a single pass. This is basically the core encoder related information, and if SBR is used, SBR related information.

UsacChannelPairElementConfig()

Similar to the above, this element configuration contains all the information needed to configure the decoder to decode a channel pair. In addition to the core configuration and SBR configuration mentioned above, it also includes stereo-specific configurations, such as the exact type of stereo encoding applied (with or without MPS212, residual, etc.). This element covers the full range of stereo encoding options currently available in USAC.

UsacLfeElementConfig()

Because the LFE element has a static configuration, the LFE element configuration does not contain configuration data.

UsacExtElementConfig()

This element configuration can be used to configure any type of existing or future extensions to the codec. Each extension element type has its own dedicated type value. A length field is included to enable skipping of configuration extensions that are unknown to the decoder.

UsacCoreConfig()

It contains configuration data that affects the core encoder settings.

SbrConfig()

It contains default values for configuration elements used for eSBR that are generally kept constant. In addition, static SBR configuration elements are also carried in SbrConfig(). These static bits include flags for enabling or disabling specific features of enhanced SBR (such as harmonic transposition or inter-TES).

SbrDfltHeader()

This element carries the default version of the elements of SbrHeader(), which can be referenced if no differences between these elements are expected.

Mps212Config()

All settings for the MPEG Surround 2-1-2 tool are grouped together in this configuration.

escapedValue()

This element implements a general method for transmitting integer values using different numbers of bits. It features a two-stage escape mechanism that allows the range of representable values to be extended by continuously transmitting additional bits.

usacSamplingFrequencyIndex

The index determines the sampling frequency of the decoded audio signal. The values of usacSamplingFrequencyIndex and their associated sampling frequencies are described in Table C.

Table C - usacSamplingFrequencyIndex values and meanings

usacSamplingFrequency

In the case where usacSamplingFrequencyIndex is equal to zero, the output sampling frequency of the decoder is encoded as an unsigned integer value.

channelConfigurationIndex

This index determines the channel configuration. If channelConfigurationIndex > 0, then the index explicitly defines the number of channels, channel elements, and associated loudspeaker mappings according to Table Y. The names of the loudspeaker positions, the abbreviations used, and the general positions of the available loudspeakers can be derived from Figures 3a, 3b, and 4a and 4b.

bsOutputChannelPos

The index describes the loudspeaker positions associated with a given channel according to Figures 4a and 4b. Figure 4b shows the loudspeaker positions in the listener's 3D environment. To facilitate understanding of the loudspeaker positions, Figure 4a also includes the loudspeaker positions according to IEC 100/1706/CDV, which are listed here for the convenience of the interested reader.

Table - depends on the coreCoderFrameLength, sbrRatio, outputFrameLength and numSlots values of coreSbrFrameLengthIndex

usacConfigExtEnsionPresent

It indicates the presence of an extension to the configuration.

numOutChannels

If the value of channelConfigurationIndex indicates that none of the predefined channel configurations are being used, then this element determines the number of audio channels that a particular speaker position will be associated with.

numElements

This field contains the number of elements that will follow the loop through the element type of UsacDecoderConfig().

usacElementType[elemIdx]

This defines the USAC channel element type for the element at position elemIdx in the bitstream. There are four element types, one for each of the four basic bitstream elements: UsacSingleChannelElement(), UsacChannelPairElement(), UsacLfeElement(), and UsacExtElement(). These elements provide the required top-level structure while maintaining all the required flexibility. The meaning of usacElementType is defined in Table A.

Table A - Values of usacElementType

usacElementType value ID_USAC_SCE 0 ID_USAC_CPE 1 ID_USAC_LFE 2 ID_USAC_EXT 3

stereoConfigIndex

This element specifies the internal structure of UsacChannelPairElement(). It indicates the use of mono or stereo core, the use of MPS 212, whether stereo SBR is applied, and whether residual coding is applied in MPS 212, according to Table ZZ. This element also defines the values of the auxiliary elements bsStereoSbr and bsResidualCoding.

Table ZZ - Values of stereoConfigIndex and their meanings and implicit assignment of bsStereoSbr and bsResidualCoding

tw_mdct

This flag communicates the use of time-warped MDCT in this stream.

noiseFilling

This flag communicates the use of noise filling of spectral holes in the FD core encoder.

harmonicSBR

This flag communicates the use of harmonic repair in SBR.

bs_interTes

This flag communicates the use of the inter-TES tool in SBR.

dflt_start_freq

This is the default value for the bitstream element bs_start_freq, which applies if the flag sbrUseDfltHeader indicates that the default value for the SbrHeader() element is to be assumed.

dflt_stop_freq

This is the default value for the bitstream element bs_stop_freq, which applies if the flag sbrUseDfltHeader indicates that the default value for the SbrHeader() element is to be assumed.

dflt_header_extra1

This is the default value for the bitstream element bs_header_extra1, which applies if the flag sbrUseDfltHeader indicates that the default value for the SbrHeader() element is to be assumed.

dflt_header_extra2

This is the default value for the bitstream element bs_header_extra2, which applies if the flag sbrUseDfltHeader indicates that the default value for the SbrHeader() element is to be assumed.

dflt_freq_scale

This is the default value for the bitstream element bs_freq_scale, which applies if the flag sbrUseDfltHeader indicates that the default value for the SbrHeader() element is to be assumed.

dflt_alter_scale

This is the default value for the bitstream element bs_alter_scale, which applies if the flag sbrUseDfltHeader indicates that the default value for the SbrHeader() element is to be assumed.

dflt_noise_bands

This is the default value for the bitstream element bs_noise_bands, which applies if the flag sbrUseDfltHeader indicates that the default value for the SbrHeader() element is to be assumed.

dflt_limiter_bands

This is the default value for the bitstream element bs_limiter_bands, which applies if the flag sbrUseDfltHeader indicates that the default value for the SbrHeader() element is to be assumed.

dflt_limiter_gains

This is the default value for the bitstream element bs_limiter_gains, which applies if the flag sbrUseDfltHeader indicates that the default value for the SbrHeader() element is to be assumed.

dflt_interpol_freq

This is the default value for the bitstream element bs_interpol_freq, which applies if the flag sbrUseDfltHeader indicates that the default value for the SbrHeader() element is to be assumed.

dflt_smoothing_mode

This is the default value for the bitstream element bs_smoothing_mode, which applies if the flag sbrUseDfltHeader indicates that the default value for the SbrHeader() element is to be assumed.

usacExtElementType

This element allows the communication of a bitstream extension type. The meaning of usacExtElementType is defined in Table B.

Table B - Values of usacExtElementType

usacExtElementConfigLength

It conveys the length of the extended configuration in bytes (octets).

usacExtElementDefaultLengthPresent

This flag communicates whether usacExtElementDefaultLength is passed in UsacExtElementConfig().

usacExtElementDefaultLength

This conveys the default length of the extension element in bytes. Whenever an extension element in a given access unit deviates from this value, an additional length needs to be transmitted in the bitstream. If the element is not explicitly transmitted (usacExtElementDefaultLengthPresent == 0), the value of usacExtElementDefaultLength will be set to zero.

usacExtElementPayloadFrag

This flag indicates whether the payload of this extension element can be fragmented and sent as several segments in consecutive USAC frames.

numConfigExtensions

If extensions to the configuration are present in UsacConfig(), this value indicates the number of configuration extensions conveyed.

confExtIdx

Configure the extended index.

usacConfigExtType

This element allows the communication of a configuration extension type. The meaning of usacConfigExtType is defined in Table D.

Table D - usacConfigExtType values

usacConfigExtType value ID_CONFIG_EXT_FILL 0 /*Reserved for ISO use*/ 1-127 /*Reserved for use outside the ISO range*/ 128 and higher

usacConfigExtLength

It conveys the length of the configuration extension in bytes (octets).

bsPseudoLr

This flag communicates that reverse medial/lateral rotation should be applied to the core signal prior to Mps212 processing.

Table-bsPseudoLr

bsPseudoLr meaning 0 Core decoder output is DMX/RES 1 Core decoder output is Pseudo L/R

bsStereoSbr

This flag is used to communicate the use of stereo SBR in conjunction with MPEG surround decoding.

Table -bsStereoSbr

bsStereoSbr meaning 0 Mono SBR 1 Stereo SBR

bsResidualCoding

It indicates whether residual coding is applied according to the following table: The BsResidualCoding value is defined by stereoConfigIndex (see X).

Table -bsResidualCoding

bsResidualCoding meaning 0 No residual coding, the core encoder is mono 1 Residual coding, core encoder is stereo

sbrRatioIndx

It represents the ratio between the core sampling rate and the sampling rate after eSBR processing. At the same time, it represents the number of synthesis bands and QMF analysis used in SBR according to the following table.

Table - Definition of sbrRatioIndex

elemIdx

The index of the element present in UsacDecoderConfig() and UsacFrame().

UsacConfig()

UsacConfig() contains information about the output sampling frequency and channel configuration. This information will be the same as that conveyed outside this element as in MPEG-4AudioSpecificConfig().

Usac output sampling frequency

If the sampling rate is not one of the ratios listed in the right column of Table 1, then the sampling frequency dependency table (code table, scale factor band table, etc.) must be obtained to parse the bitstream payload. Since a given sampling frequency is associated with only one sampling frequency table, and since maximum flexibility is desired within the range of possible sampling frequencies, the following table will be used to associate the implicit sampling frequency with the desired sampling frequency dependency table.

Table 1 - Sampling frequency mapping

Frequency range (Hz) Usage table for sampling frequency (Hz) f>=92017 96000 92017>f>＝75132 88200 75132>f>＝55426 64000 55426>f>＝46009 48000 46009>f>＝37566 44100 37566>f>＝27713 32000 27713>f>＝23004 24000 23004>f>＝18783 22050 18783>f>＝13856 16000 13856>f>＝11502 12000 11502>f>＝9391 11025 9391>f 8000

UsacChannelConfig()

The channel configuration table covers the most commonly used loudspeaker positions. For further flexibility, channels can be mapped to a total selection of 32 loudspeaker positions found in modern loudspeaker setups for various applications (see Figures 3a, 3b).

For each channel included in the bitstream, UsacChannelConfig() specifies the associated speaker position to which that particular channel will be mapped. The speaker positions indexed by bsOutputChannelPos are listed in Figure 4a. In the case of a multi-channel element, the index i of bsOutputChannelPos[i] indicates the position at which the channel appears in the bitstream. Figure Y gives an overview of the speaker positions with respect to the listener.

More precisely, channels are numbered in the order in which they appear in the bitstream, starting with 0 (zero). In the ordinary case of UsacSingleChannelElement() or UsacLfeElement(), a channel number is assigned to the channel and the channel count value is incremented by 1. In the case of UsacChannelPairElement(), the first channel in the element (with index ch == 0) is numbered 1, while the second channel in the same element (with index ch == 1) receives the next higher number and the channel count value is incremented by 2.

It follows that numOutChannels will be equal to or less than the cumulative sum of all channels contained in the bitstream. The cumulative sum of all channels is equal to the number of all UsacSingleChannelElement() plus the number of all UsacLfeElement() plus twice the number of all UsacChannelPairElement().

All entries in the array bsOutputChannelPos will be separated from each other to avoid double allocation of speaker positions in the bitstream.

In the specific case where channelConfigurationIndex is 0 and numOutChannels is less than the cumulative sum of all channels contained in the bitstream, then the handling of non-allocated channels is outside the scope of this specification. Information about this can be conveyed, for example, by appropriate means of higher application layers or by specially designed (private) extension payloads.

UsacDecoderConfig()

UsacDecoderConfig() contains all the additional information needed by the decoder to interpret the bitstream. First, the value of sbrRatioIndex determines the ratio between the core encoder frame length (ccfl) and the output frame length. sbrRatioIndex then loops through all channel elements in the bitstream. For each iteration, the element type is conveyed in usacElementType[], followed by its corresponding configuration structure. The order in which the elements appear in UsacDecoderConfig() will be the same as the order in which the corresponding payload appears in UsacFrame().

Each instance of an element can be configured independently. When reading each channel element in UsacFrame(), for each element, the corresponding configuration of the instance will be used, i.e., with the same elemIdx.

UsacSingleChannelElementConfig()

UsacSingleChannelElementConfig() contains all the information needed to configure the decoder to decode a single channel. If SBR is actually used, only the SBR configuration data is transmitted.

UsacChannelPairElementConfig()

UsacChannelPairElementConfig() contains configuration data related to the core encoder and SBR configuration data depending on the use of SBR. The exact type of stereo encoding algorithm is indicated by stereoConfigIndex. In USAC, channel pairs can be encoded in various ways. These ways are:

1. Extending the stereo core encoder pair using the conventional joint stereo coding technique by composite prediction possibilities in the MDCT domain.

2. The mono core encoder channel is combined with MPS212 based MPEG Surround for full parametric stereo coding. Mono SBR processing is applied to the core signal.

3. A stereo core encoder pair is combined with an MPS212 based on MPEG Surround, where the first core encoder channel carries the downmix signal and the second channel carries the residual signal. The residual can be band-limited to achieve partial residual coding. Mono SBR processing is applied to the downmix signal only before MPS212 processing.

4. A stereo core encoder pair is combined with MPS212 based on MPEG Surround, where the first core encoder channel carries the downmix signal and the second channel carries the residual signal. The residual can be band-limited to achieve partial residual coding. Stereo SBR is applied to the reconstructed stereo signal after MPS212 processing.

After the core encoder, options 3 and 4 can be further combined with pseudo LR channel rotation.

UsacLfeElementConfig()

Since the LFE channel does not allow the use of time-warped MDCT and noise filling, the usual core encoder flags for these tools do not need to be transmitted. Instead, they will be set to zero.

Furthermore, SBR is not allowed in the LFE context and therefore, no SBR configuration data is transmitted.

UsacCoreConfig()

UsacCoreConfig() contains only flags to enable or disable the use of time-warped MDCT and spectral noise filling at the global bitstream level. If tw_mdct is set to zero, no time warping is applied. If noiseFilling is set to zero, no spectral noise filling is applied.

SbrConfig()

The SbrConfig() bitstream element serves the purpose of communicating the exact eSBR setup parameters. On the one hand, SbrConfig() communicates the general deployment of the eSBR tool. On the other hand, SbrConfig() contains a default version of SbrHeader(), namely SbrDfltHeader(). If a different SbrHeader() is not transmitted in the bitstream, the value of this default header will be assumed. The background of this mechanism is that usually only one set of SbrHeader() values is applied in a bitstream. The transmission of SbrDfltHeader() then allows to refer to this set of default values very efficiently by using only one bit in the bitstream. By allowing in-band transmission of a new SbrHeader in the bitstream itself, the possibility of changing the SbrHeader values in real time is still maintained.

SbrDfltHeader()

SbrDfltHeader() can be called the basic SbrHeader() template and should contain the values for the eSBR configuration that is primarily used. In the bitstream, this configuration can be referenced by setting the sbrUseDfltHeader() flag. The structure of SbrDfltHeader() is the same as that of SbrHeader(). In order to be able to distinguish the values of SbrDfltHeader() and SbrHeader(), the bit fields in SbrDfltHeader() are prefixed with "dflt_" instead of "bs_". If the use of SbrDfltHeader() is indicated, the SbrHeader() bit fields will take the values of the corresponding SbrDfltHeader(), i.e.

bs_start_freq=dflt_start_freq;

bs_stop_freq=dflt_stop_freq;

wait

(Continue with all elements in SbrHeader(), such as:

bs_xxx_yyy=dflt_xxx_yyy;

Mps212Config()

Mps212Config() is similar to and largely derived from SpatialSpecificConfig() of MPEG Surround. However, its extent is reduced to only contain information related to mono to stereo upmixing in the USAC context. Therefore, MPS212 only configures one OTT box.

UsacExtElementConfig()

UsacExtElementConfig() is a general container for configuration data for USAC extension elements. Each USAC extension has a unique type identifier, usacExtElementType, which is defined in Figure 6k. For each UsacExtElementConfig(), the length of the included extension configuration is transmitted in the variable usacExtElementConfigLength, allowing the decoder to safely skip extension elements with unknown usacExtElementType.

For USAC extensions that typically have a constant payload length, UsacExtElementConfig() allows the transmission of usacExtElementDefaultLength. Defining a default payload length in the configuration allows for highly efficient communication of usacExtElementPayloadLength within UsacExtElement() where bit consumption needs to be kept low.

In the case of USAC extensions where larger amounts of data are accumulated and transmitted not on a per-frame basis but only every other frame or even more sparsely, this data can be transmitted in fragments or segments spread across several USAC frames. This can help maintain more even bit storage. The use of this mechanism is signaled by the usacExtElementPayloadFrag flag. The fragment mechanism is further described in the description of usacExtElement in 6.2.X.

UsacConfigExtension()

UsacConfigExtension() is a general container for UsacConfig() extensions. It provides a convenient way to modify or extend information that is switched when the decoder is initialized or set up. The presence of a configuration extension is indicated by usacConfigExtensionPresent. If the configuration extension is present (usacConfigExtensionPresent == 1), the exact number of these extensions follows the bitfield numConfigExtensions. Each configuration extension has a unique type identifier, usacConfigExtType. For each UsacConfigExtension, the length of the contained configuration extension is transmitted in the variable usacConfigExtLength, and allows the configuration bitstream parser to safely skip configuration extensions whose usacConfigExtType is unknown.

Top-level payload for audio object type USAC

Terms and Definitions

UsacFrame()

This data block contains the audio data, related information, and other data for the time period of a USAC frame. As conveyed in UsacDecoderConfig(), UsacFrame() contains numElements elements. These elements can contain audio data for one or two channels, audio data for low-frequency enhancement, or extended payload.

UsacSingleChannelElement()

Abbreviated SCE. A syntax element that contains the bitstream of coded data for a single audio channel. single_channel_element() essentially includes UsacCoreCoderData(), which contains data for the FD or LPD core coder. If SBR is enabled, UsacSingleChannelElement also contains SBR data.

UsacChannelPairElement()

Abbreviated CPE. A syntax element containing a bitstream payload for a pair of channels. A channel pair can be implemented by transmitting two discrete channels or a single discrete channel and the associated MPLS 212 payload. This is communicated via the stereoConfigIndex parameter. If SBR is enabled, the UsacChannelPairElement also contains SBR data.

UsacLfeElement()

Abbreviation for LFE. Contains the syntax element for the low sampling frequency enhancement channel. LFE is always encoded using the fd_channel_stream() element.

UsacExtElement()

The syntax element that contains the extension payload. The length of the extension element is communicated as a default length in the configuration (USACExtElementConfig()) or in UsacExtElement() itself. If present, the extension payload is of type usacExtElementType, as communicated in the configuration.

usacIndependencyFlag

It indicates whether the current UsacFrame() can be fully decoded without knowing information from the previous frame according to the table below.

Table - Meaning of usacIndependencyFlag

Note: Please refer to X.Y for suggestions regarding usacIndependencyFlag.

usacExtElementUseDefaultLength

It indicates whether the length of the extension element corresponds to usacExtElementDefaultLength defined in UsacExtElementConfig().

usacExtElementPayloadLength

It shall contain the length of the extension element in bytes. This value shall only be conveyed explicitly in the bitstream if the extension element length in the current access unit deviates from the default value usacExtElementDefaultLength.

usacExtElementStart

It indicates whether the current usacExtElementSegmentData starts a data block.

usacExtElementStop

It indicates whether the current usacExtElementSegmentData ends the data block.

usacExtElementSegmentData

The concatenation of all usacExtElementSegmentData from UsacExtElement() of consecutive USAC frames, starting with UsacExtElement() with usacExtElementStart == 1 up to and including UsacExtElement() with usacExtElementStop == 1, forms a data block. In case the complete data block is contained in one UsacExtElement(), both usacExtElementStart and usacExtElementStop will be set to 1. The data block is interpreted as a byte-aligned extended payload depending on the usacExtElementType according to the following table:

Table - Explanation of data blocks decoded for USAC extended payload

fill_byte

An octet that can be used to fill the bits of the bitstream with bits that do not carry information. The exact bit pattern used for fill_byte should be '10100101'.

Auxiliary elements

nrCoreCoderChannels

In the context of a channel-pair element, this variable represents the number of core encoder channels that form the basis for stereo encoding. Depending on the value of stereoConfigIndex, this value will be 1 or 2.

nrSbrChannels

In the context of a channel pair element, this variable indicates the number of channels to which SBR processing is applied. Depending on the value of stereoConfigIndex, this value will be 1 or 2.

Ancillary payloads for USAC

Terms and Definitions

UsacCoreCoderData()

This data block contains the core encoder audio data. For FD or LPD mode, the payload element contains data for one or two core encoder channels. The specific mode is conveyed per channel at the beginning of the element.

StereoCoreToolInfo()

All stereo related information is captured in this element. It handles many dependencies of the bit fields in stereo coding mode.

Auxiliary elements

commonCoreMode

In CPE, this flag indicates whether the two encoded core encoder passes use the same mode.

Mps212Data()

This data block contains the payload for the Mps212 stereo module. The existence of this data depends on stereoConfigIndex.

common_window

It indicates whether channel 0 and channel 1 of the CPE use the same window parameters.

common_tw

This indicates whether channels 0 and 1 of the CPE use the same parameters for the time-warped MDCT.

Decoding of UsacFrame()

A UsacFrame() forms an access unit of the USAC bitstream. Each UsacFrame decodes into 768, 1024, 2048 or 4096 output samples, depending on the outputFrameLength determined from the table.

The first bit in UsacFrame() is usacIndependencyFlag, which determines whether a given frame can be decoded without any knowledge of the previous frame. If usacIndependencyFlag is set to 0, there may be a dependency on a previous frame in the payload of the current frame.

UsacFrame() further consists of one or more syntax elements that will appear in the bitstream in the same order as their corresponding configuration elements in UsacDecoderConfig(). The position of each element in the series of all elements is indexed by elemIdx. For each element, the corresponding configuration of this instance (as transmitted in UsacDecoderConfig()) will be used, i.e., with the same elemIdx.

These syntax elements are of one of the four types listed in the table. The type of each element is determined by usacElementType. Multiple elements of the same type may exist. Elements that appear at the same position elemIdx in different frames belong to the same stream.

Table - Simple Examples of Possible Bitstream Payloads

If these bitstream payloads are transported over a constant rate channel, they may include an extended payload element with usacExtElementType of ID_EXT_ELE_FILL to adjust the instantaneous bit rate. In this case, an example of a coded stereo signal is:

Table - Example of a simple stereo bitstream with an extended payload to write padding bits

Decoding of UsacSingleChannelElement()

The simple structure of UsacSingleChannelElement() consists of an instance of UsacCoreCoderData() with nrCoreCoderChannels set to 1. The following UsacSbrData() element with nrSbrChannels also set to 1, depending on the sbrRatioIndex of this element.

Decoding of UsacExtElement()

The UsacExtElement() structure in the bitstream can be decoded or skipped by a USAC decoder. Each extension is identified by a usacExtElementType transmitted in the UsacExtElementConfig() associated with the UsacExtElement(). For each usacExtElementType, a specific decoder can exist.

If the decoder for the extension is capable of being used with the USAC decoder, then the payload of the extension is forwarded to the extension decoder right after the UsacExtElement() has been parsed by the USAC decoder.

If none of the decoders for the extension can be used with a USAC decoder, a minimal structure is provided within the bitstream so that the extension can be ignored by the USAC decoder.

The length of the extension element is specified by a default length in octets, which can be communicated within the corresponding UsacExtElementConfig() and can be declared invalid in UsacExtElement(); or by utilizing the syntax element escapedValue(), the length of the extension element is specified by length information explicitly provided in UsacExtElement(), which is one or three octets long.

Extension payloads that span one or more UsacFrame()s can be fragmented and have their payload distributed across several UsacFrame()s. In this case, the usacExtElementPayloadFrag flag is set to 1, and the decoder must collect all fragments from a UsacFrame() with usacExtElementStart set to 1 up to and including a UsacFrame() with usacExtElementStop set to 1. When usacExtElementStop is set to 1, the extension is considered complete and is passed to the extension decoder.

Note that this specification does not provide integrity protection for fragmented extension payloads, and other means should be used to ensure the integrity of the extension payload.

Note that all extended payload data is assumed to be byte aligned.

Each UsacExtElement() shall comply with the requirements imposed by the use of usacIndependencyFlag. More specifically, if usacIndependencyFlag is set (==1), then UsacExtElement() will be able to decode without knowledge of the previous frame (and any extension payload that may be contained therein).

Decoding process

The stereoConfigIndex transmitted in UsacChannelPairElementConfig() determines the exact type of stereo encoding applied in a given CPE. Depending on the type of stereo encoding, one or two core encoder channels are actually transmitted in the bitstream, and the variable nrCoreCoderChannels must be set accordingly. The syntax element UsacCoreCoderData() then provides data for one or two core encoder channels.

Similarly, depending on the type of stereo coding and the use of eSBR (i.e. if sbrRatioIndex>0), there may be data available for one or two channels. The value of nrSbrChannels needs to be set accordingly, and the syntax element UsacSbrData() provides eSBR data for one or two channels.

Finally, Mps212Data() is transmitted depending on the value of stereoConfigIndex.

Low frequency enhancement (LFE) channel element, UsacLfeElement()

Introduction

In order to maintain a regular structure in the decoder, UsacLfeElement() is defined as a standard fd_channel_stream(0,0,0,0,x) element, i.e. it is equal to UsacCoreCoderData() using the frequency domain encoder. Thus, decoding can be performed using the standard procedure for decoding UsacCoreCoderData()-elements.

However, to accommodate higher bitrate and hardware-efficient implementations of the LFE decoder, several restrictions are imposed on the options for encoding this element:

The window_sequence field is always set to 0 (ONLY_LONG_SEQUENCE)

● Only the lowest 24 spectral coefficients of any LFE can be non-zero

● Temporal noise shaping is not used, i.e. tns_data_present is set to 0

Time bending does not work

No noise filling

UsacCoreCoderData()

UsacCoreCoderData() contains all the information needed to decode one or two core encoder channels.

The decoding order is:

●Get core_mode[] for each channel

In case of two core encoder channels (nrChannels == 2), parse StereoCoreToolInfo() and determine all stereo related parameters

● Depending on the core_modes communicated, lpd_channel_stream() or fd_channel_stream() is transmitted for each channel

As can be seen from the above list, decoding of one core encoder channel (nrChannels == 1) results in obtaining the core_mode bit, which is followed by one lpd_channel_stream or fd_channel_stream, depending on the core_mode.

In the case of two core encoder channels, some communication redundancy between channels can be exploited, especially when both channels have core_mode set to 0. See 6.2.X (Decoding of StereoCoreToolInfo()) for details.

StereoCoreToolInfo()

StereoCoreToolInfo() allows efficient encoding of parameters whose values can be shared across the CPE's core encoder channels when encoding two channels in FD mode (core_mode[0,1] == 0). Specifically, the following data elements are shared when the appropriate flags in the bitstream are set to 1.

Table - Bitstream elements shared across channels of a core encoder channel pair

If the appropriate flag is not set, the data elements are transmitted separately for each core encoder channel in StereoCoreToolInfo() (max_sfb, max_sfbl) or in fd_channlel_stream() following StereoCoreToolInfo() in a UsacCoreCoderData() element.

In case common_window == 1, StereoCoreToolInfo() also contains information about M/S stereo coding in the MDCT domain and complex prediction data (see 7.7.2).

UsacSbrData()

This data block contains the payload of the SBR bandwidth extension for one or two channels. The presence of this data depends on the sbrRatioIndex.

SbrInfo()

This element contains SBR control parameters that do not require a decoder reset when changed.

SbrHeader()

This element contains SBR header data with SBR configuration parameters, which generally do not change over the duration of the bitstream.

SBR payload for USAC

In USAC, the SBR payload is transmitted in UsacSbrData() as the integer part of each single channel element or channel pair element. UsacSbrData() immediately follows UsacCoreCoderData(). There is no SBR payload for the LFE channel.

numSlots

The number of time slots in the Mps212Data frame.

FIG1 shows an audio decoder for decoding an encoded audio signal provided at an input 10. An encoded audio signal is provided on input line 10, for example, as a data stream or, more illustratively, as a serial data stream. The encoded audio signal comprises a first channel element and a second channel element in a payload section of the data stream, and comprises first decoder configuration data for the first channel element and second decoder configuration data for the second channel element in a configuration section of the data stream. Typically, the first decoder configuration data will differ from the second decoder configuration data, as the first channel element will typically also differ from the second channel element.

A data stream or encoded audio signal is input to a data stream reader 12 for reading configuration data for each channel element and forwarding this configuration data to a configuration controller 14 via connection line 13. Furthermore, the data stream reader is arranged to read payload data for each channel element in the payload section, and this payload data, including a first channel element and a second channel element, is provided to a configurable decoder 16 via connection line 15. Configurable decoder 16 is arranged to decode a plurality of channel elements and output data for each channel element, as represented by output lines 18a, 18b. Specifically, when decoding a first channel element, configurable decoder 16 is configured according to first decoder configuration data, while when decoding a second channel element, configurable decoder 16 is configured according to second decoder configuration data. This is represented by connection lines 17a, 17b, where connection line 17a transmits the first decoder configuration data from configuration controller 14 to the configurable decoder, and connection line 17b transmits the second decoder configuration data from the configuration controller to the configurable decoder. The configuration controller is to be implemented in any manner such that the configurable decoder operates according to the decoder configuration communicated in the corresponding decoder configuration data or on the corresponding lines 17a, 17b. Thus, the configuration controller 14 can be implemented as an interface between the data stream reader 12, which actually obtains the configuration data from the data stream, and the configurable decoder 16, which is configured by the configuration data actually read.

FIG2 illustrates a corresponding audio encoder for encoding a multi-channel input audio signal provided at an input 20. The input 20 is shown as comprising three different lines 20a, 20b, and 20c, wherein line 20a carries, for example, a center channel audio signal, line 20b carries a left channel audio signal, and line 20c carries a right channel audio signal. All three channel signals are input to a configuration processor 22 and a configurable encoder 24. The configuration processor is adapted to generate first configuration data on line 21a and second configuration data on line 21b for a first channel element, e.g., including only the center channel so that the first channel element is a single channel element, and for a second channel element, e.g., a channel pair element carrying a left channel and a right channel. The configurable encoder 24 is adapted to use the first configuration data 21a and the second configuration data 21b to encode the multi-channel audio signal 20 to obtain a first channel element 23a and a second channel element 23b. The audio encoder further comprises a data stream generator 26 which receives the first and second configuration data at input lines 25a and 25b and further receives the first and second channel elements 23a and 23b. The data stream generator 26 is adapted to generate a data stream 27 representing the encoded audio signal, the data stream having a configuration section comprising the first and second configuration data and a payload section comprising the first and second channel elements.

In this document, it is outlined that the first configuration data and the second configuration data can be the same as or different from the first decoder configuration data or the second decoder configuration data. In the event that the first configuration data and the second configuration data are different from the first decoder configuration data or the second decoder configuration data, the configuration controller 14 is configured to convert the configuration data in the data stream into corresponding decoder-directed data by applying, for example, a unique function or a lookup table, if the configuration data is encoder-directed data. However, preferably, the configuration data written to the data stream is already decoder configuration data, so that the configurable encoder 24 or configuration processor 22 has, for example, the following functionality: the functionality is used to derive encoder configuration data from the calculated decoder configuration data, or the functionality is used to calculate or determine decoder configuration data from the calculated encoder configuration data by applying a unique function or a lookup table or other prior knowledge.

FIG5 a shows a schematic diagram of an encoded audio signal input to the data stream reader 12 of FIG1 or output by the data stream generator 26 of FIG2 . The data stream includes a configuration section 50 and a payload section 52. FIG5 b shows a more detailed implementation of the configuration section 50 in FIG5 a. The data stream shown in FIG5 b - which is typically a serial data stream carrying bits one by one - includes general configuration data related to the higher layers of the transport structure (such as the MPEG-4 file format) at its first end 50 a. Alternatively or in addition, the configuration data 50 a (which may or may not be present) includes further general configuration data contained in the UsacChannelConfig shown at 50 b.

Typically, configuration data 50a may also include data from the UsacConfig shown in Figure 6a, and item 50b includes elements implemented and shown in the UsacChannelConfig of Figure 6b. Specifically, the same configuration for all channel elements may, for example, include the output channel representations shown and described in the context of Figures 3a, 3b and 4a, 4b.

The configuration section 50 of the bitstream is then followed by a UsacDecoderConfig element, which in this example is formed by first configuration data 50c, second configuration data 50d and third configuration data 50e. The first configuration data 50c is for the first channel element, the second configuration data 50d is for the second channel element and the third configuration data 50e is for the third channel element.

Specifically, each configuration data for a channel element as shown in FIG5b includes an identifier element type index idx used with respect to its syntax in FIG6c. The element type index idx having two bits is then followed by bits describing the channel element configuration data as follows: the channel element configuration data is found in FIG6c and further described in FIG6d for a single channel element, FIG6e for a channel pair element, FIG6f for an LFE element, and FIG6k for an extension element, all of which are channel elements that may be typically included in a USAC bitstream.

FIG5c illustrates a UASC frame included in a payload segment 52 of the bitstream shown in FIG5a. When the configuration segment in FIG5b forms the configuration segment 50 of FIG5a, i.e., when the payload segment includes three channel elements, then payload segment 52 is implemented as shown in FIG5c, i.e., the payload data for the first channel element 52a is followed by the payload data for the second channel element represented by 52b, and the payload data for the second channel element represented by 52b is followed by the payload data for the third channel element 52c. Thus, according to the present invention, the configuration segment and the payload segment are organized in such a way that the order of the configuration data relative to the channel elements is the same as the order of the payload data relative to the channel elements in the payload segment. Therefore, when the order in the UsacDecoderConfig element is configuration data for the first channel element, configuration data for the second channel element, and configuration data for the third channel element, then the order in the payload segment is the same, that is, in the serial data or bit stream there is payload data for the first channel element, followed by payload data for the second channel element, and then followed by payload data for the third channel element.

The parallel structure in the configuration and payload sections is advantageous because it allows for easy organization of which configuration data belongs to which channel element, communicating with very low overhead. In the prior art, no ordering is required because there is no individual configuration data for each channel element. However, according to the present invention, individual configuration data for each channel element is introduced to ensure that the best configuration data for each channel element can be optimally selected.

Typically, a USAC frame includes data for a period of 20 to 40 milliseconds. When considering longer data streams, as shown in FIG5 d, there is a configuration section 60a, followed by payload sections or frames 62a, 62b, 62c, ... 62e, and then a configuration section 62d is included in the bit stream.

The order of the configuration data in the configuration section (as discussed with respect to FIG5 b and FIG5 c ) is the same as the order of the channel element payload data in each of frames 62 a to 62 e. Therefore, the order of the payload data for the individual channel elements is also exactly the same in each of frames 62 a to 62 e.

Typically, when the encoded signal is a single file stored on a hard disk, for example, then at the beginning of the entire track (e.g., a track of approximately 10 or 20 minutes), a single configuration section 50 is sufficient. This single configuration section is then followed by a high number of individual frames, and the configuration is valid for each frame, the order of the channel element data (configuration or payload) being the same in each frame and configuration section.

However, when the coded audio signal is a data stream, configuration segments must be introduced between the individual frames to provide access points so that the decoder can start decoding even when an earlier configuration segment has been transmitted but not received by the decoder because the decoder has not yet been switched on to receive the actual data stream. The number of frames n between different configuration segments can be chosen arbitrarily, but if one wants to achieve one access point per second, the number of frames between two configuration segments will be between 25 and 50.

Subsequently, FIG. 7 shows a direct example for encoding and decoding a 5.1 multi-channel signal.

Preferably, four channel elements are used, where the first channel element is a single channel element comprising a center channel, the second channel element is a channel pair element CPE1 comprising a left channel and a right channel, and the third channel element is a second channel pair element CPE2 comprising a left surround channel and a right surround channel. Finally, the fourth channel element is an LFE channel element. In an embodiment, for example, the configuration data for the single channel element may cause the noise filling tool to be enabled, while for the second channel pair element comprising the surround channels, for example, the noise filling tool is disabled and a low-quality parametric stereo encoding procedure is applied. However, the low-bitrate stereo encoding procedure results in a low bitrate, but quality loss is not a problem due to the fact that the channel pair element has surround channels.

On the other hand, the left and right channels contain a large amount of information, so the MPS 212 configuration conveys a high-quality stereo encoding process. M/S stereo encoding is advantageous in that it provides high quality, but suffers from a very high bit rate. Therefore, M/S stereo encoding is preferred for CPE1 but not for CPE2. Furthermore, the noise filling feature can be turned on or off depending on the implementation, but is preferably turned on because it places a high emphasis on good, high-quality representation of the left and right channels, and noise filling is also turned on for the center channel.

However, when the core bandwidth of the channel element C is, for example, very low and the number of consecutive lines quantized to zero in the center channel is also low, then it may also be advantageous to switch off noise filling for individual channel elements of the center channel due to the fact that noise filling does not provide an additional quality gain and given that the quality is not or only slightly improved, the bits required for transmitting the side information of the noise filling tool can be saved.

Typically, the tools conveyed in the configuration section for a channel element are those mentioned in, for example, Figures 6d, 6e, 6f, 6g, 6h, 6i, and 6j, and additionally include elements for the extended element configurations in Figures 6k, 6l, and 6m. As shown in Figure 6e, the MPS 2121 configuration for each channel element can be different.

MPEG Surround uses a compact parameter representation of human auditory cues for spatial perception to allow for bitrate-efficient representation of multichannel signals. In addition to CLD and ICC parameters, IPD parameters can be transmitted. For efficient representation of phase information, OPD parameters are estimated from the given CLD and IPD parameters. The IPD and OPD parameters are used to synthesize phase differences to further refine the stereo image.

In addition to the parametric mode, residual coding can be used, where the residual has limited bandwidth or full bandwidth. In this procedure, two output signals are generated by mixing a mono input signal and a residual signal using the CLD, ICC, and IPD parameters. In addition, all parameters mentioned in FIG6j can be selected separately for each channel element. The individual parameters are described in detail, for example, in ISO/IEC CD 23003-3, dated September 24, 2010, which is incorporated herein by reference.

In addition, as shown in Figures 6f and 6g, core features (such as time warping features and noise filling features) can be turned on or off for each channel element respectively. The time warping tool described under the term "time warping filter bank and block switching" in the above reference replaces the standard filter bank and block switching. In addition to IMDCT, this tool includes time-domain to time-domain mapping from an arbitrarily spaced grid to a normal linearly spaced time grid, as well as corresponding adaptation of the window shape.

In addition, as shown in Figure 7, the noise filling tool can be turned on or off for each channel element separately. In low bit rate encoding, noise filling can be used for two purposes. The process quantization of spectral values in low bit rate audio encoding may result in a very sparse spectrum after inverse quantization because many spectral lines may have been quantized to zero. The sparse spectrum will cause the decoded signal to sound sharp or unstable (screeching). By replacing the zero lines with "small" values in the decoder, these very obvious artifacts can be masked or reduced without adding obvious new noise artifacts.

If noise-like signal portions are present in the original spectrum, a perceptually equivalent representation of these noise signal portions can be reproduced in the decoder based on only a small amount of parametric information, such as the energy of the noise signal portions. Fewer bits are required to transmit the parametric information than to transmit the encoded waveform. Specifically, the data elements that need to be transmitted are the noise offset element, which is an additional offset value that modifies the scale factor of the frequency bands quantized to zero, and the noise level, which is an integer representing the quantization noise to be added to each spectral line quantized to zero.

As shown in Figure 7 and Figures 6f and 6g, this feature can be turned on or off for each channel element separately.

Additionally, there are SBR features that can now be communicated for each channel element separately.

As shown in Figure 6h, these SBR elements include turning on/off different tools in SBR. The first tool to be turned on or off for each channel element is Harmonic SBR. When Harmonic SBR is turned on, Harmonic SBR tones are performed, while when Harmonic SBR is turned off, tones with continuous lines known from MPEG-4 (High Efficiency) are used.

Furthermore, a PVC or "Prediction Vector Coding" decoding process can be applied. In order to improve the subjective quality of the eSBR tool, especially for speech content at low bit rates, Prediction Vector Coding (PVC) is added to the eSBR tool. Typically, for speech signals, there is a fairly high correlation between the spectral envelopes of the low and high frequency bands. In the PVC scheme, the spectral envelope of the high frequency band is predicted from the spectral envelope of the low frequency band, where the coefficient matrix used for the prediction is encoded with the help of vector quantization. The HF envelope adjuster is modified to process the envelope generated by the PVC decoder.

Thus, the PVC tool may be particularly useful for single channel elements such as speech in the center channel; however, the PVC tool is not useful for, for example, the surround channels of CPE2 or the left and right channels of CPE1.

In addition, the inter-time envelope shaping feature (inter-TES) can be turned on or off for each channel element separately. Following the envelope adjuster, the inter-subband sample time envelope shaping (inter-TES) processes the QMF subband samples. This module shapes the time envelope of the higher frequency bandwidth with a finer time granularity than the time granularity of the envelope adjuster. Inter-Tes shapes the time envelope in the QMF subband samples by applying a gain factor to each QMF subband sample in the SBR envelope. Inter-Tes consists of three modules, namely the time envelope calculator between lower frequency subband samples, the time envelope adjuster between subband samples, and the time envelope shaper between subband samples. Due to the fact that the tool requires additional bits, there will be channel elements for which the additional bit consumption is not adjusted in view of the quality gain, and channel elements for which the additional bit consumption is adjusted in view of the quality gain. Therefore, according to the present invention, activation/deactivation of the tool is used for each channel element.

In addition, FIG6i shows the syntax of the SBR default header, and all SBR parameters of the SBR default header mentioned in FIG6i can be selected differently for each channel element. For example, this is related to actually setting the start frequency or stop frequency of the crossover frequency, where the crossover frequency is the frequency at which the signal reconstruction changes from the mode to the parameter mode. Other features (such as frequency resolution and noise band resolution, etc.) can also be used to selectively set for each channel element.

Therefore, as shown in Figure 7, configuration data is preferably set for stereo features, core encoder features and SBR features respectively. The respective settings of the element refer not only to the SBR parameters in the SBR default header shown in Figure 6i, but also apply to all parameters in SbrConfig shown in Figure 6h.

Next, the implementation of the decoder of FIG. 1 is explained with reference to FIG. 8 .

In particular, the functionality of the data stream reader 12 and the configuration controller 14 is similar to that described in the context of Figure 1. However, the configurable decoder 16 is now implemented, for example, for individual decoder instances, wherein each decoder instance has an input for configuration data C provided by the configuration controller 14 and an input for data D for receiving the corresponding channel element from the data stream reader 12.

In particular, the functionality of Figure 8 is such that for each individual channel element, an individual decoder instance is provided.Thus, a first decoder instance is configured by the first configuration data as a single channel element, for example for the center channel.

Furthermore, the second decoder instance is configured according to second decoder configuration data for the left and right channels of the channel pair element. Furthermore, the third decoder instance 16 c is configured for a further channel pair element comprising a left surround channel and a right surround channel. Finally, the fourth decoder instance is configured for the LFE channel. Thus, the first decoder instance provides a single channel C as an output. However, the second decoder instance 16 b and the third decoder instance 16 c each provide two output channels, namely, the left and right channels on the one hand, and the left surround channel and the right surround channel on the other hand. Finally, the fourth decoder instance 16 d provides the LFE channel as an output. All six channels of the multichannel signal are forwarded via the decoder instances to the output interface 19 and then ultimately sent for, for example, storage or for playback in, for example, a 5.1 speaker setup. Clearly, different decoder instances and a different number of decoder instances are required when the speaker setup is different.

FIG. 9 shows a preferred implementation of a method for decoding an encoded audio signal according to an embodiment of the present invention.

In step 90, the data stream reader 12 begins reading the configuration section 50 of FIG. 5a. Then, as indicated in step 92, a channel element is identified based on the channel element identifier in the corresponding configuration data block 50c. In step 94, the configuration data for the identified channel element is read and used to actually configure the decoder, or to be stored for later use in configuring the decoder when processing the channel element. This is shown in step 94.

In step 96, the element type identifier of the second configuration data in portion 50d of Figure 5b is used to identify the next channel element. This is shown in step 96 of Figure 9. Then, in step 98, the configuration data is read and used to actually configure the decoder or decoder instance, or the configuration data is read to alternatively store the configuration data when the payload for that channel element is to be decoded.

Then, in step 100 , the entire configuration data is looped through, ie the identification of channel elements and the reading of configuration data for the channel elements are continued until all configuration data have been read.

Then, in steps 102, 104, 106, the payload data for each channel element is read and finally decoded in step 108 using the configuration data C, where the payload data is represented by D. The result of step 108 is the data output by, for example, blocks 16a to 16d, which can then be sent directly to the loudspeaker, or the data can be synchronized, amplified, further processed or digital/analog converted to ultimately be sent to the corresponding loudspeaker.

Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of a corresponding method, where a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or a description of an item or feature of a corresponding device.

Depending on certain implementation requirements, embodiments of the present invention can be implemented in hardware or software. The implementation can be performed using a digital storage medium such as a floppy disk, a digital versatile disk (DVD), a compact disk (CD), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory, on which electronically readable control signals are stored, which cooperate with a programmable computer system (or are capable of cooperating with a programmable computer system) to cause the execution of the various methods.

Some embodiments according to the invention comprise a non-transitory data carrier having electronically readable control signals, which cooperate with a programmable computer system, such that one of the methods described herein is performed.

The encoded audio signal may be transmitted via a wired or wireless transmission medium, or may be stored on a machine-readable carrier or a non-transitory storage medium.

Generally, embodiments of the present invention can be implemented as a computer program product with a program code, which, when the computer program product runs on a computer, is operative to perform one of the methods described. The program code can, for example, be stored on a machine-readable carrier.

Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

In other words, an embodiment of the method according to the invention is, therefore, a computer program having a program code for performing one of the methods described herein, when it runs on a computer.

A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein.

Therefore, a further embodiment of the inventive method is a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or signal sequence can, for example, be configured to be transmitted via a data communication connection, such as via the Internet.

A further embodiment comprises a processing means, such as a computer or a programmable logic device, which may be configured to or adapted to perform one of the methods described herein.

A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

In some embodiments, a programmable logic device (e.g., a field programmable gate array) can be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array can collaborate with a microprocessor to perform one of the methods described herein. Typically, the method is preferably performed by any hardware device.

The above embodiments are merely illustrative of the principles of the present invention. It is to be understood that modifications and variations of the arrangements and details described herein will be apparent to those skilled in the art. Accordingly, it is intended that the present invention be limited only by the scope of the pending patent claims and not by the specific details presented by the description and illustration of the embodiments herein.

The present disclosure also includes the following technical solutions.

Solution 1. An audio decoder for decoding an encoded audio signal, the encoded audio signal comprising: a first channel element and a second channel element in a payload section of a data stream; and first decoder configuration data for the first channel element and second decoder configuration data for the second channel element in a configuration section of the data stream, the audio decoder comprising:

a data stream reader for reading the configuration data for each channel element in the configuration section and for reading the payload data for each channel element in the payload section;

a configurable decoder configured to decode the plurality of channel elements; and

A configuration controller is configured to configure the configurable decoder so that the configurable decoder is configured according to the first decoder configuration data when decoding the first channel element, and is configured according to the second decoder configuration data when decoding the second channel element.

Solution 2. The audio decoder according to Solution 1,

wherein the first channel element is a single channel element including payload data of the first output channel, and

The second channel element is a channel pair element including payload data of the second output channel and the third output channel,

wherein the configurable decoder is arranged to generate a single output channel when decoding the first channel element and to generate two output channels when decoding the second channel element, and

The audio decoder is configured to output the first output channel, the second output channel, and the third output channel for simultaneous output via three different audio output channels.

Solution 3. The audio decoder according to solution 1 or 2,

The first channel is a center channel, and the second channel and the third channel are a left channel and a right channel, or a left surround channel and a right surround channel.

Solution 4. The audio decoder according to Solution 1,

wherein the first channel element is a first channel pair element including data of a first output channel and a second output channel, and wherein the second channel element is a second channel pair element including payload data of a third output channel and a fourth output channel,

wherein the configurable decoder is configured to generate a first output channel and a second output channel when decoding the first channel element, and to generate a third output channel and a fourth output channel when decoding the second channel element, and

Wherein the audio decoder is configured to output the first output channel, the second output channel, the third output channel and the fourth output channel for simultaneous output lines for different audio output channels.

Solution 5. The audio decoder according to Solution 4,

The first channel is a left channel, the second channel is a right channel, the third channel is a left surround channel, and the fourth channel is a right surround channel.

Solution 6. The audio decoder according to one of the preceding solutions,

wherein the encoded audio signal further comprises a general configuration segment in the configuration segment of the data stream, the general configuration segment having information for the first channel element and the second channel element, and wherein the configuration controller is arranged to configure the configurable decoder for the first channel element and the second channel element with the configuration information from the general configuration segment.

Solution 7. The audio decoder according to one of the preceding solutions,

wherein the first configuration section is different from the second configuration section, and

Wherein the configuration controller is arranged to configure the configurable decoder to decode the second channel element differently than the configuration used when decoding the first channel element.

Solution 8. The audio decoder according to one of the preceding solutions,

wherein the first decoder configuration data and the second decoder configuration data include information on a stereo decoding tool, a core decoding tool, or a spectrum bandwidth replication decoding tool, and

The configurable decoder includes the spectrum bandwidth replication decoding tool, the core decoding tool, and the stereo decoding tool.

Solution 9. The audio decoder according to one of the preceding solutions,

wherein the payload section comprises a sequence of frames, each frame comprising the first channel element and the second channel element, and

wherein the first decoder configuration data for the first channel element and the second decoder configuration data for the second channel element are associated with the frame sequence,

The configuration controller is configured to configure the configurable decoder for each frame in the frame sequence so that the first channel element in each frame is decoded using the first decoder configuration data, and the second channel element in each frame is decoded using the second decoder configuration data.

Solution 10. The audio decoder according to one of the preceding solutions,

wherein the data stream is a serial data stream, and the configuration section sequentially includes decoder configuration data for a plurality of channel elements, and

The payload section includes the payload data of the multiple channel elements in the same order.

Solution 11. The audio decoder according to one of the preceding solutions,

wherein the configuration segment comprises a first channel element identifier followed by the first decoder configuration data, and a second channel element identifier followed by the second decoder configuration data, wherein the data stream reader is arranged to loop the following processing for all elements: sequentially passing through the first channel element identifier and sequentially reading the first decoder configuration data for the channel element, and sequentially passing through the second channel element identifier and sequentially reading the second decoder configuration data.

Solution 12. The audio decoder according to one of the preceding solutions,

wherein the configurable decoder comprises a plurality of parallel decoder instances,

wherein the configuration controller is arranged to configure the first decoder instance using the first decoder configuration data and to configure the second decoder instance using the second decoder configuration data, and

Wherein the data stream reader is arranged to forward the payload data of the first channel element to the first decoder instance, and to forward the payload data of the second channel element to the second decoder instance.

Solution 13. The audio decoder according to Solution 12,

The payload section includes a payload frame sequence, and

wherein the data stream reader is configured to forward data from each channel element of a currently processed frame only to the corresponding decoder instance configured by the configuration data for that channel element.

Embodiment 14. A method for decoding an encoded audio signal, the encoded audio signal comprising: a first channel element and a second channel element in a payload section of a data stream; and first decoder configuration data for the first channel element and second decoder configuration data for the second channel element in a configuration section of the data stream, the method comprising:

reading the configuration data for each channel element in the configuration section, and reading the payload data for each channel element in the payload section;

decoding, by a configurable decoder, the plurality of channel elements; and

The configurable decoder is configured such that when decoding the first channel element, the configurable decoder is configured according to the first decoder configuration data, and when decoding the second channel element, the configurable decoder is configured according to the second decoder configuration data.

Solution 15. An audio encoder for encoding a multi-channel audio signal, comprising:

a configuration processor configured to generate first configuration data for a first channel element and second configuration data for a second channel element;

a configurable encoder configured to encode the multi-channel audio signal using the first configuration data and the second configuration data to obtain the first channel element and the second channel element; and

A data stream generator is configured to generate a data stream representing an encoded audio signal, the data stream having a configuration section and a payload section, the configuration section having the first configuration data and the second configuration data, the payload section including the first channel element and the second channel element.

Solution 16. A method for encoding a multi-channel audio signal, comprising:

generating first configuration data for a first channel element and second configuration data for a second channel element;

Encoding the multi-channel audio signal by a configurable encoder using the first configuration data and the second configuration data to obtain the first channel element and the second channel element; and

A data stream representing an encoded audio signal is generated, the data stream having a configuration section having the first configuration data and the second configuration data and a payload section including the first channel element and the second channel element.

Solution 17. A computer program, which, when executed on a computer, performs the method according to Solution 14 or Solution 16.

Solution 18. An encoded audio signal comprising:

a configuration section having first decoder configuration data for a first channel element and second decoder configuration data for a second channel element, a channel element being an encoded representation of a single channel or two channels of a multi-channel audio signal; and

A payload section includes payload data of the first channel element and the second channel element.

Claims (2)

1. A computer-readable medium having a computer program recorded thereon, the computer program executing, when run on a computer, a method for decoding an encoded audio signal (10), the encoded audio signal (10) comprising: first payload data (52a) for a first channel element (23a) and second payload data (52b) for a second channel element (23b) in a payload segment (52) of a data stream; and first decoder configuration data (50c) for the first channel element (23a) and second decoder configuration data (50d) for the second channel element (23b) in a configuration segment (50) of the data stream, the method comprising: Read the first decoder configuration data (50c) for the first channel element (23a) and the second decoder configuration data (50d) for the second channel element (23b) from the configuration segment. Read the first payload data (52a) for the first channel element (23a) and the second payload data (52b) for the second channel element (23b) from the payload segment (52); Configurable decoding of the first payload data (52a) for the first channel element (23a) and the second payload data (52b) for the second channel element (23b); and The configurable decoder is configured such that when decoding the first payload data (52a) for the first channel element (23a), the configurable decoder is configured according to the first decoder configuration data (50c), and when decoding the second payload data (52b) for the second channel element (23b), the configurable decoder is configured according to the second decoder configuration data (50d). 2. A computer-readable medium having a computer program recorded thereon, the computer program executing, when run on a computer, a method for encoding a multi-channel audio signal (20), the method comprising: Generate first configuration data (50c) for the first channel element (23a); Generate second configuration data (50d) for the second channel element (23b); The multi-channel audio signal (20) is configurably encoded using the first configuration data (50c) to obtain first payload data (52a) for the first channel element (23a); The multi-channel audio signal is configurably encoded using the second configuration data (50d) to obtain second payload data (52b) for the second channel element (23b); and A data stream (27) representing an encoded multi-channel audio signal (27) is generated. The data stream (27) has a configuration segment (50) and a payload segment (52). The configuration segment (50) has first configuration data (50c) and second configuration data (50d). The payload segment (52) includes first payload data (52a) for the first channel element (23a) and second payload data (52b) for the second channel element (23b).