HK40008788B - Method for decoding audio scene, audio decoder and medium - Google Patents

Description

Methods for decoding audio scenes, audio decoders, and media

This application is a divisional application of the invention patent application filed on May 23, 2014, with application number "201480030011.2" and invention title "Encoding of Audio Scenes".

Cross-references to related applications

This application claims priority to U.S. Provisional Patent Application No. 61/827,246, filed May 24, 2013, which is incorporated herein by reference in its entirety.

Technical Field

The inventions disclosed herein generally relate to the field of audio encoding and decoding. In particular, the invention relates to the encoding and decoding of audio scenes including audio objects.

Background Technology

Audio coding systems exist for parametric spatial audio coding. For example, MPEG Surround describes a system for parametric spatial coding of multichannel audio. MPEG SAOC (Spatial Audio Object Coding) describes a system for parametric coding of audio objects.

On the encoder side, these systems typically downmix the channel/object into a downmix, usually a mono (one channel) or stereo (two channels) downmix, and extract side information describing the properties of the channel/object through factors such as level difference and cross-correlation. The downmix and side information are then encoded and sent to the decoder side. On the decoder side, the channel/object is reconstructed, or approximately estimated, based on the downmix, under the control of parameters of the side information.

The drawback of these systems is that reconstruction is often mathematically complex and frequently relies on assumptions about the properties of audio content not explicitly described by parameters sent as side information. Such assumptions could be, for example, that channels/objects are considered uncorrelated unless cross-correlation parameters are sent; or that channel/object downmixing is generated in a specific manner. Furthermore, the mathematical complexity and the need for additional assumptions increase significantly as the number of channels in the downmix increases.

Furthermore, the necessary assumptions are inherently reflected in the algorithmic details of the processing applied on the decoder side. This means that a considerable amount of intelligence must be included on the decoder side. This is a drawback because when the decoder is placed in consumer devices that are, for example, difficult or even impossible to upgrade, it is difficult to upgrade and improve the algorithm.

Summary of the Invention

A first aspect of the present invention relates to a method for decoding an audio scene represented by N audio signals, the method comprising: receiving a bitstream including M downmixed signals and matrix elements of a reconstruction matrix; generating a reconstruction matrix using the matrix elements; and reconstructing the N audio signals from the M downmixed signals using the reconstruction matrix, wherein the N audio signals are approximated as a linear combination of the M downmixed signals, and the matrix elements of the reconstruction matrix are used as coefficients in the linear combination, wherein M is less than N and M is equal to or greater than 1.

A second aspect of the invention relates to an audio decoder for decoding an audio scene represented by N audio signals, the audio decoder comprising: a receiver receiving a bitstream including M downmixed signals and matrix elements of a reconstruction matrix; a reconstruction matrix generator receiving matrix elements from the receiver and generating a reconstruction matrix based on the matrix elements; and a reconstructor receiving the reconstruction matrix from the reconstruction matrix generator and reconstructing the N audio signals from the M downmixed signals using the reconstruction matrix, wherein the N audio signals are approximated as a linear combination of the M downmixed signals, the matrix elements of the reconstruction matrix serving as coefficients in the linear combination, wherein M is less than N and M is equal to or greater than 1.

A third aspect of the invention relates to a non-transitory computer-readable medium comprising instructions that, when executed by a processor of an information processing system, cause the information processing system to perform a method according to an embodiment.

Attached Figure Description

In the following description, exemplary embodiments will be illustrated with reference to the accompanying drawings and in more detail, wherein:

Figure 1 is a schematic diagram of an audio encoding/decoding system according to an example embodiment;

Figure 2 is a schematic diagram of an audio encoding/decoding system with a legacy decoder according to an example embodiment;

Figure 3 is a schematic diagram of the encoding side of an audio encoding/decoding system according to an example embodiment;

Figure 4 is a flowchart of the encoding method according to an example embodiment;

Figure 5 is a schematic diagram of an encoder according to an example embodiment;

Figure 6 is a schematic diagram of the decoder side of an audio encoding/decoding system according to an example embodiment;

Figure 7 is a flowchart of a decoding method according to an example embodiment;

Figure 8 is a schematic diagram of the decoder side of an audio encoding/decoding system according to an example embodiment; and

Figure 9 is a schematic diagram of time-frequency transformation performed on the decoder side of an audio encoding/decoding system according to an example embodiment.

All accompanying drawings are schematic and generally only show parts necessary to illustrate the invention, while other parts may be omitted or only implied. Unless otherwise stated, the same reference numerals in different drawings indicate the same parts.

Detailed Implementation

In light of the above, the aim is to provide encoders and decoders, as well as related methods for providing less complex and more flexible reconstruction of audio objects.

I. Overview – Encoder

According to the first aspect, exemplary embodiments have proposed an encoding method, an encoder, and a computer program product for encoding. The proposed method, encoder, and computer program product can generally have the same features and advantages.

According to an example embodiment, a method is provided for encoding a time-frequency block of an audio scene comprising at least N audio objects. The method includes: receiving the N audio objects; generating M downmixing signals based on the at least N audio objects; generating a reconstruction matrix using matrix elements, the reconstruction matrix enabling the reconstruction of the at least N audio objects from the M downmixing signals; and generating a bitstream comprising at least some of the matrix elements from the M downmixing signals and the reconstruction matrix.

The number of audio objects N can be equal to or greater than 1. The number of downmixed signals M can be equal to or greater than 1.

This method generates a bitstream comprising at least some matrix elements of a reconstruction matrix serving as side information, and M downmixing signals. By including individual matrix elements of the reconstruction matrix in the bitstream, very little intelligence is required on the decoder side. For example, complex calculations of the reconstruction matrix based on transmitted object parameters and additional assumptions are not required on the decoder side. Therefore, the mathematical complexity on the decoder side is significantly reduced. Furthermore, because the complexity of this method does not depend on the number of downmixing signals used, it increases flexibility regarding the number of downmixing signals compared to existing methods.

As used in this article, an audio scene generally refers to a three-dimensional audio environment that includes audio units that can be presented for playback on an audio system and are associated with positions in three-dimensional space.

As used in this article, an audio object refers to a unit of audio scene. An audio object typically includes an audio signal as well as additional information such as the object's position in three-dimensional space. This additional information is usually used to optimally represent the audio object on a given playback system.

As used herein, a downmix signal refers to a signal that is a combination of at least N audio objects. Other signals of an audio scene, such as bed channels (described below), can also be combined into a downmix signal. For example, M downmix signals may correspond to the presentation of an audio scene for a given speaker configuration, such as a standard 5.1 configuration. The number of downmix signals represented by M in this document is typically (but not necessarily) less than the sum of the number of audio objects and bed channels, which explains why M downmix signals are called a downmix.

Audio encoding/decoding systems typically divide the time-frequency space into time-frequency blocks, for example, by applying a suitable filter bank to the input audio signal. A time-frequency block generally means a portion of the time-frequency space corresponding to a time interval and frequency sub-bands. The time interval may typically correspond to the duration of a time frame used in the audio encoding/decoding system. The frequency sub-bands may typically correspond to one or more adjacent frequency sub-bands defined by the filter bank used in the encoding/decoding system. In the case where the frequency sub-bands correspond to several adjacent frequency sub-bands defined by the filter bank, this allows for non-uniform frequency sub-bands during the decoding of the audio signal; for example, wider frequency sub-bands may be used for higher frequencies of the audio signal. In the case of a wideband in which the audio encoding/decoding system operates over the entire frequency range, the frequency sub-bands of the time-frequency block may correspond to the entire frequency range. The above method discloses encoding steps for encoding an audio scene during one such time-frequency block. However, it should be understood that the method can be repeated for each time-frequency block of the audio encoding/decoding system. Furthermore, it should be understood that several time-frequency blocks can be encoded simultaneously. Typically, adjacent time-frequency blocks may slightly overlap in time and/or frequency. For example, temporal overlap can be equivalent to the elements of the reconstruction matrix being linearly interpolated from one time interval to the next. However, the present disclosure is intended for other components of the encoding/decoding system, while any temporal and/or frequency overlap between adjacent time-frequency blocks is left to those skilled in the art to implement.

According to the example embodiment, M downmixed signals are arranged in a first field of the bitstream using a first format, and matrix elements are arranged in a second field of the bitstream using a second format. This allows decoders that only support the first format to decode and play back the M downmixed signals in the first field and discard the matrix elements in the second sub-field. The advantage of this is that the M downmixed signals in the bitstream are backward compatible with legacy decoders not used for audio object reconstruction. In other words, legacy decoders can still decode and play back the M downmixed signals of the bitstream, for example, by mapping each downmixed signal to the decoder's channel output.

According to an example embodiment, the method may further include the step of: receiving position data corresponding to each of N audio objects, wherein M downmixed signals are generated based on the position data. The position data typically associates each audio object with a position in three-dimensional space. The position of the audio object may change over time. By using the position data when downmixing the audio objects, the audio objects are mixed into the M downmixed signals in such a way that, for example, if the M downmixed signals are listened to on a system with M output channels, the audio objects sound as if they are approximately located at their respective positions. This is advantageous, for example, when the M downmixed signals need to be backward compatible with legacy decoders.

According to the example embodiment, the matrix elements of the reconstruction matrix are time-varying and frequency-varying. In other words, the matrix elements of the reconstruction matrix can be different for different time-frequency blocks. In this way, excellent flexibility in the reconstruction of audio objects is achieved.

According to the example embodiment, the audio scene also includes multiple bed channels. This is common, for example, in cinema audio applications where the audio content includes bed channels in addition to audio objects. In this case, M downmixed signals can be generated based on at least N audio objects and multiple bed channels. A bed channel generally means an audio signal corresponding to a fixed position in three-dimensional space. For example, a bed channel may correspond to one of the output channels of an audio encoding/decoding system. Thus, a bed channel can be interpreted as having a corresponding position in three-dimensional space that is the same as the position of one of the output speakers of the audio encoding/decoding system. Therefore, a bed channel can be associated with a label that only indicates the position of the corresponding output speaker.

When the audio scene includes a bed channel, the reconstruction matrix can include matrix elements that enable the reconstruction of the bed channel based on M downmixed signals.

In some cases, an audio scene can include a large number of objects. To reduce the complexity and data volume required to represent an audio scene, the audio scene can be simplified by reducing the number of audio objects. Therefore, if the audio scene initially includes K audio objects, where K > N, the method may further include the step of receiving K audio objects and reducing the K audio objects to N audio objects by clustering the K audio objects into N clusters and representing each cluster with one audio object.

To simplify the scenario, the method may further include the step of receiving position data corresponding to each of the K audio objects, wherein the K objects are clustered into N clusters based on the positional distances between the K objects given by the position data of the K audio objects. For example, audio objects that are close to each other in three-dimensional space can be clustered together.

As described above, the example embodiments of this method are flexible in terms of the number of downmixing signals used. Specifically, the method can be advantageously used when there are more than two downmixing signals, i.e., when M is greater than two. For example, five or seven downmixing signals corresponding to conventional 5.1 or 7.1 audio settings can be used. This is advantageous because, unlike prior art systems, the mathematical complexity of the proposed coding principle remains the same regardless of the number of downmixing signals used.

To further improve the reconstruction of N audio objects, the method may further include: forming L auxiliary signals based on the N audio objects; including matrix elements in a reconstruction matrix that enables the reconstruction of at least N audio objects based on M downmixing signals and L auxiliary signals; and including the L auxiliary signals in a bitstream. Thus, the auxiliary signals act as aids, which can, for example, capture aspects of audio objects that are difficult to reconstruct from the downmixing signals. The auxiliary signals may also be based on the audio bed channel. The number of auxiliary signals may be equal to or greater than one.

According to one example embodiment, auxiliary signals may correspond to particularly important audio objects, such as audio objects representing dialogue. Therefore, at least one of the L auxiliary signals may be identical to one of the N audio objects. This allows for higher quality rendering of important objects compared to situations where reconstruction must be performed based solely on M downmix channels. In practice, the audio content provider may have prioritized and/or labeled some audio objects as preferably included individually as auxiliary objects. Furthermore, this makes modifications/processing of these objects prior to rendering less prone to artifacts. As a trade-off between bitrate and quality, a mixture of two or more audio objects may also be sent as auxiliary signals. In other words, at least one of the L auxiliary signals may be formed as a combination of at least two audio objects from the N audio objects.

According to one example embodiment, the auxiliary signal represents the signal dimension of an audio object lost during the generation of M downmixes. This loss is due, for example, because the number of independent objects is often greater than the number of downmix channels, or because the positions of the two objects are associated with each other, causing them to be mixed into the same downmix. An example of the latter case is when two objects are separated only in the vertical direction but share the same position when projected onto a horizontal plane. This means that the two objects would typically be presented as the same downmix channel in a standard 5.1 surround speaker setup, where all speakers are on the same horizontal plane. Specifically, the M downmixes span a hyperplane in the signal space. By forming a linear combination of the M downmixes, only audio signals located in the hyperplane can be reconstructed. To improve the reconstruction, auxiliary signals not located in the hyperplane can be included, thereby enabling the reconstruction of signals not located in the hyperplane as well. In other words, according to the example embodiment, at least one of the multiple auxiliary signals is not located in the hyperplane spanned by the M downmixes. For example, at least one of the multiple auxiliary signals may be orthogonal to the hyperplane spanned by the M downmixes.

According to an example embodiment, a computer-readable medium is provided that includes computer code instructions adapted to perform any of the methods of the first aspect when run on a processing device.

According to an example embodiment, an encoder is provided for encoding a time-frequency block of an audio scene comprising at least N audio objects. The encoder includes: a receiving unit configured to receive the N audio objects; a downmixing generation unit configured to receive the N audio objects from the receiving unit and generate M downmixing signals based on the at least N audio objects; an analysis unit configured to generate a reconstruction matrix using matrix elements, the reconstruction matrix enabling the reconstruction of at least N audio objects based on the M downmixing signals; and a bitstream generation unit configured to receive the M downmixing signals from the downmixing generation unit and the reconstruction matrix from the analysis unit, and generate a bitstream comprising at least some of the matrix elements of the M downmixing signals and the reconstruction matrix.

II. Overview – Decoder

According to the second aspect, exemplary embodiments provide a decoding method, a decoding apparatus, and a computer program product for decoding. The proposed method, apparatus, and computer program product can generally have the same features and advantages.

The advantages associated with the features and settings presented in the overview of the encoder above can generally be effective for the corresponding features and settings of the decoder.

According to an example embodiment, a method is provided for decoding a time-frequency block of an audio scene comprising at least N audio objects. The method includes the steps of: receiving a bitstream comprising at least some matrix elements of a matrix including M downmixed signals and a reconstruction matrix; generating a reconstruction matrix using the matrix elements; and reconstructing the N audio objects based on the M downmixed signals using the reconstruction matrix.

According to the example embodiment, M downmixed signals are arranged in a first field of the bitstream using a first format, and matrix elements are arranged in a second sub-segment of the bitstream using a second format, thereby allowing a decoder that only supports the first format to decode and play back the M downmixed signals in the first field and discard the matrix elements in the second sub-segment.

According to the example embodiment, the matrix elements of the reconstructed matrix are time-varying and frequency-varying.

According to an example embodiment, the audio scene also includes multiple audio bed channels, and the method further includes reconstructing the audio bed channels based on M downmixed signals using a reconstruction matrix.

According to the example embodiment, the number M of downmixed signals is greater than 2.

According to an example embodiment, the method further includes: receiving L auxiliary signals formed by N audio objects; reconstructing the N audio objects using a reconstruction matrix based on M downmixing signals and L auxiliary signals, wherein the reconstruction matrix includes matrix elements that enable the reconstruction of at least N audio objects based on the M downmixing signals and L auxiliary signals.

According to the example embodiment, at least one of the L auxiliary signals is the same as one of the N audio objects.

According to the example embodiment, at least one of the L auxiliary signals is a combination of N audio objects.

According to the example embodiment, M downmixed signals span a hyperplane, and at least one of the plurality of auxiliary signals is not located in the hyperplane spanned by the M downmixed signals.

According to an example embodiment, at least one of a plurality of auxiliary signals not located in the hyperplane is orthogonal to the hyperplane spanned by M downmixing signals.

As mentioned above, audio encoding/decoding systems typically operate in the frequency domain. Therefore, audio encoding/decoding systems use filter banks to perform time-frequency transformations of the audio signals. Different types of time-frequency transformations can be used. For example, the M downmixed signals can be represented with respect to a first frequency domain, and the reconstruction matrix can be represented with respect to a second frequency domain. To reduce the computational burden on the decoder, it is advantageous to choose the first and second frequency domains intelligently. For example, the first and second frequency domains can be chosen to be the same frequency domain, such as the improved discrete cosine transform (MDCF) domain. In this way, it is avoided in the decoder to transform the M downmixed signals from the first frequency domain to the time domain and then to the second frequency domain. Alternatively, the first and second frequency domains can be chosen such that the transformation from the first to the second frequency domain can be performed jointly, so that it is not necessary to go through the time domain between the first and second frequency domains.

The method may also include receiving position data corresponding to N audio objects, and using the position data to represent the N audio objects to create at least one output audio channel. In this way, the reconstructed N audio objects are mapped onto the output channel of the audio encoder/decoder system based on their positions in three-dimensional space.

The rendering is preferably performed in the frequency domain. To reduce the computational burden on the decoder, the frequency domain for rendering is preferably selected intelligently with respect to the frequency domain of the reconstructed audio object. For example, if the reconstruction matrix is represented with respect to a second frequency domain corresponding to a second filter bank, and rendering is performed in a third frequency domain corresponding to a third filter bank, then the second and third filter banks are preferably selected to be at least partially the same filter banks. For example, the second and third filter banks may include a quadrature mirror filter (QMF) domain. Alternatively, the second and third frequency domains may include an MDCT filter bank. According to an example embodiment, the third filter bank may consist of a series of filter banks, such as a QMF filter bank followed by a Nyquist filter bank. If so, at least one of the filter banks in the sequence (the first filter bank in the sequence) is the same as the second filter bank. In this way, it can be said that the second and third filter banks are at least partially the same filter banks.

According to an example embodiment, a computer-readable medium is provided that includes computer code instructions adapted to perform any of the methods of the second aspect when run on a device with processing capabilities.

According to an example embodiment, a decoder is provided for decoding a time-frequency block of an audio scene comprising at least N audio objects. The decoder includes: a receiving component configured to receive a bitstream comprising at least some matrix elements of a matrix including M downmixed signals and a reconstruction matrix; a reconstruction matrix generating component configured to receive matrix elements from the receiving component and generate a reconstruction matrix based on the matrix elements; and a reconstruction component configured to receive the reconstruction matrix from the reconstruction matrix generating component and reconstruct the N audio objects using the reconstruction matrix based on the M downmixed signals.

III. Example Implementation

Figure 1 illustrates an encoding/decoding system 100 for encoding/decoding audio scene 102. The encoding/decoding system 100 includes an encoder 108, a bitstream generation unit 110, a bitstream decoding unit 118, a decoder 120, and a renderer 122.

Audio scene 102 is represented by one or more audio objects 106a (such as N audio objects), i.e., audio signals. Audio scene 102 may also include one or more sound bed channels 106b, i.e., signals directly corresponding to one of the output channels of the renderer 122. Audio scene 102 is also represented by metadata including position information 104. Position information 104 is used, for example, by the renderer 122 when rendering audio scene 102. Position information 104 can associate the audio objects 106a, and possibly sound bed channels 106b, with spatial positions in three-dimensional space as a function of time. Metadata may also include other types of data useful for rendering audio scene 102.

The encoding portion of system 100 includes an encoder 108 and a bitstream generation unit 110. Encoder 108 receives an audio object 106a, a bed channel 106b (if present), and metadata including position information 104. Based on this, encoder 108 generates one or more downmix signals 112, such as M downmix signals. For example, downmix signals 112 may correspond to the channels of a 5.1 audio system [Lf Rf Cf Ls Rs LFE]. ("L" represents left, "R" represents right, "C" represents center, "f" represents front, "s" represents surround, and "LFE" represents low-frequency effects).

The encoder 108 also generates side information. The side information includes a reconstruction matrix. The reconstruction matrix includes matrix elements 114 that enable the reconstruction of at least audio object 106a based on the downmix signal 112. The reconstruction matrix can also enable the reconstruction of the audio bed channel 106b.

Encoder 108 transmits M downmixed signals 112 and at least some matrix elements from matrix elements 114 to bitstream generation unit 110. Bitstream generation unit 110 generates a bitstream 116 comprising the M downmixed signals 112 and at least some matrix elements from matrix elements 114 by performing quantization and encoding. Bitstream generation unit 110 also receives metadata including position information 104 to be included in bitstream 116.

The system's decoding section includes a bitstream decoding unit 118 and a decoder 120. The bitstream decoding unit 118 receives a bitstream 116 and performs decoding and dequantization to extract M downmixed signals 112 and side information including at least some matrix elements 114 of the reconstruction matrix. The M downmixed signals 112 and matrix elements 114 are then input to the decoder 120, which generates a reconstruction 106' based on the downmixed signals 112 and matrix elements 114, consisting of N audio objects 106a and possibly a bed channel 106b. Therefore, the reconstruction 106' of the N audio objects is an approximation of the N audio objects 106a and possibly the bed channel 106b.

For example, if the downmixer 112 corresponds to a 5.1 channel configuration [Lf Rf Cf Ls Rs LFE], then the decoder 120 can reconstruct object 106' using only the full-band channel [Lf Rf Cf Ls Rs], thus ignoring the LFE. This also applies to other channel configurations. The LFE channel of the downmixer 112 (essentially unmodified) can be sent to the renderer 122.

The reconstructed audio object 106' and position information 104 are then input to the presenter 122. Based on the reconstructed audio object 106' and position information 104, the presenter 122 presents an output signal 124 with a format suitable for playback on the desired speaker or headphone configuration. Typical output formats are a standard 5.1 surround setup (3 front speakers, 2 surround speakers, and 1 low-frequency effect (LFE) speaker) or a 7.1+4 setup (3 front speakers, 4 surround speakers, 1 LFE speaker, and 4 overhead speakers).

In some embodiments, the original audio scene may include a large number of audio objects. Processing a large number of audio objects comes at the cost of high computational complexity. Furthermore, the amount of side information (position information 104 and reconstruction matrix elements 114) to be embedded in the bitstream 116 depends on the number of audio objects. Typically, the amount of side information increases linearly with the number of audio objects. Therefore, it is advantageous to reduce the number of audio objects before encoding to save computational complexity and/or to reduce the bit rate required to encode the audio scene. To this end, the audio encoder/decoder system 100 may also include a scene simplification module (not shown) disposed upstream of the encoder 108. The scene simplification module takes the original audio objects, and possibly also bed channels, as input, and performs processing to output audio objects 106a. The scene simplification module reduces the number of original audio objects, for example K, to a more suitable number N of audio objects 106a by performing clustering. More specifically, the scene simplification module organizes K original audio objects, and possibly also bed channels, into N clusters. Typically, clustering is defined based on the spatial proximity of the K original audio objects/bed channels in the audio scene. To determine spatial proximity, the scene simplification module can take the location information of the original audio objects/soundbed channels as input. Once the scene simplification module has formed N clusters, it continues to execute to represent each cluster with an audio object. For example, the audio object representing a cluster can be formed as the sum of the audio objects/soundbed channels that form part of the cluster. More specifically, audio content of the audio objects/soundbed channels can be added to generate the audio content of the representative audio object. Furthermore, the locations of the audio objects/soundbed channels in the clusters can be averaged to give the location of the representative audio object. The scene simplification module includes the location of the representative audio object in location data 104. Additionally, the scene simplification module outputs the representative audio object constituting the N audio objects 106a in Figure 1.

The M downmixed signals 112 can be arranged in the first field of the bitstream 116 using the first format. The matrix element 114 can be arranged in the second field of the bitstream 116 using the second format. In this way, a decoder that only supports the first format can decode and play back the M downmixed signals 112 in the first field and discard the matrix element 114 in the second field.

The audio encoder/decoder system 100 of Figure 1 supports a first format and a second format. More specifically, the decoder 120 is configured to decode both the first and second formats, meaning it is capable of reconstructing object 106' based on M downmixed signals 112 and matrix elements 114.

Figure 2 illustrates an audio encoder/decoder system 200. The encoding sections 108 and 110 of system 200 correspond to the encoding section of Figure 1. However, the decoding section of audio encoder/decoder system 200 differs from the decoding section of audio encoder/decoder system 100 of Figure 1. Audio encoder/decoder system 200 includes a legacy decoder 230 that supports a first format but not a second format. Therefore, the legacy decoder 230 of audio encoder/decoder system 200 cannot reconstruct audio object/sound bed channels 106a to 106b. However, because the legacy decoder 230 supports the first format, it can still decode M downmixed signals 112 to generate output 224, which is a channel-based representation suitable for direct playback via appropriate multi-channel speaker setups, such as 5.1 representation. This property of the downmixed signals is called backward compatibility, meaning that a legacy decoder that does not support the second format, i.e., cannot decode the side information including matrix element 114, can still decode and play back M downmixed signals 112.

The operation of the encoder side of the audio encoding/decoding system 100 will now be described in more detail with reference to the flowcharts in Figures 3 and 4.

Figure 4 shows the encoder 108 and bitstream generation unit 110 of Figure 1 in more detail. The encoder 108 has a receiving unit (not shown), a downmixing generation unit 318, and an analysis unit 328.

In step E02, the receiving component of encoder 108 receives N audio objects 106a and a sound bed channel 106b (if present). Encoder 108 may also receive position data 104. Using vector notation, the N audio objects can be represented by vector S = [S1S2...SN] ^T , and the sound bed channel is represented by vector B. The N audio objects and the sound bed channel can ^be represented together by vector A = [ ^BTST ] ^T .

In step E04, the downmixing generation unit 318 generates M downmixing signals 112 based on N audio objects 106a and audio bed channels 106b (if present). Using vector notation, the M downmixing signals can be represented by a vector D = [D1 D2 ... DM] ^T comprising the M downmixing signals. Generally, downmixing of multiple signals is a combination of signals, such as a linear combination of signals. For example, the M downmixing signals may correspond to a specific speaker configuration, such as the speaker configuration [Lf Rf Cf LsRs LFE] in a 5.1 speaker configuration.

When generating M downmix signals, the downmix generation unit 318 can use the position information 104 to combine these objects into different downmix signals based on their positions in three-dimensional space. This is particularly relevant when the M downmix signals themselves correspond to a specific speaker configuration, as in the example above. For example, the downmix generation unit 318 can derive a representation matrix Pd (corresponding to the representation matrix applied in the renderer 122 of Figure 1) based on the position information, and use this representation matrix to generate downmixes according to D = pd * [B ^T S ^T ] ^T.

N audio objects 106a and audio bed channels 106b (if present) are also input to the analysis unit 328. The analysis unit 328 typically operates on time-frequency blocks of the input audio signals 106a and 106b. For this purpose, the N audio objects 106a and audio bed channels 106b can be fed through a filter bank, i.e., a QMF bank, that performs time-to-frequency transformation on the input audio signals 106a and 106b. Specifically, the filter bank 338 is associated with multiple frequency sub-bands. The frequency resolution of the time-frequency block corresponds to one or more of these frequency sub-bands. The frequency resolution of the time-frequency block can be non-uniform, i.e., it can vary with frequency. For example, a low frequency resolution can be used for high frequencies, meaning that a time-frequency block in the high-frequency range can correspond to several frequency sub-bands defined by the filter bank 338.

In step E06, the analysis unit 328 generates a reconstruction matrix, denoted herein by R1. The generated reconstruction matrix consists of multiple matrix elements. The reconstructed matrix R1 enables the reconstruction (approximately) of N audio objects 106a, and possibly also the audio bed channel 106b, in the decoder based on M downmixed signals 112.

Analysis unit 328 can employ different methods to generate the reconstruction matrix. For example, a minimum mean square error (MMSE) prediction method can be used, taking N audio objects 106a/bed channels 106b and M downmix signals 112 as inputs. This method can be described as aiming to derive a reconstruction matrix that minimizes the mean square error of the reconstructed audio objects/bed channels. Specifically, this method uses candidate reconstruction matrices to reconstruct the N audio objects/bed channels and compares the audio objects/bed channels with the input audio objects 106a/bed channels 106b regarding the mean square error. The candidate reconstruction matrix that minimizes the mean square error is selected as the reconstruction matrix, and its matrix element 114 is the output of analysis unit 328.

The MMSE method requires estimation of the correlation and covariance matrices for N audio objects 106a/bed channels 106b and M downmix signals 112. According to the method described above, these correlation and covariance matrices are measured based on the N audio objects 106a/bed channels 106b and the M downmix signals 112. In an alternative model-based method, the analysis unit 328 takes position data 104 instead of the M downmix signals 112 as input. By making certain assumptions, such as assuming the N audio objects are uncorrelated, and using these assumptions in conjunction with the downmixing rules applied in the downmixing generation unit 318, the analysis unit 328 can calculate the required correlation and covariance matrices for performing the MMSE method described above.

The elements 114 of the reconstructed matrix and the M downmixed signals 112 are then input to the bitstream generation unit 110. In step E108, the bitstream generation unit 110 quantizes and encodes the M downmixed signals 112 and at least some of the matrix elements 114 of the reconstructed matrix, and arranges them in the bitstream 116. Specifically, the bitstream generation unit 110 can arrange the M downmixed signals 112 in the first field of the bitstream 116 using a first format. Furthermore, the bitstream generation unit 110 can arrange the matrix elements 114 in the second field of the bitstream 116 using a second format. As described above with reference to FIG2, this allows legacy decoders that only support the first format to decode and play back the M downmixed signals 112 and discard the matrix elements 114 in the second field.

Figure 5 illustrates an alternative embodiment of encoder 108. Compared to the encoder shown in Figure 3, encoder 508 of Figure 5 further enables one or more auxiliary signals to be included in bitstream 116. For this purpose, encoder 508 includes an auxiliary signal generation unit 548. The auxiliary signal generation unit 548 receives audio objects 106a/sound bed channels 106b and generates one or more auxiliary signals 512 based on the audio objects 106a/sound bed channels 106b. The auxiliary signal generation unit 548 may, for example, generate auxiliary signals 512 as a combination of audio objects 106a/sound bed channels 106b. The auxiliary signals are represented by a vector C = [CI C2...CL] ^T , and can be generated as C = Q*[B ^T S ^T ] ^T , where Q is a matrix that can be time-varying and frequency-varying. This includes the case where the auxiliary signal is equal to one or more audio objects and the case where the auxiliary signal is a linear combination of audio objects. For example, the auxiliary signal may represent a particularly important object, such as dialogue.

The auxiliary signal 512 serves to improve the reconstruction of the audio object 106a/sound bed channel 106b in the decoder. More specifically, on the decoder side, the audio object 106a/sound bed channel 106b can be reconstructed based on M downmix signals 112 and L auxiliary signals 512. Therefore, the reconstruction matrix will include matrix elements 114 capable of reconstructing the audio object/sound bed channel based on the M downmix signals 112 and L auxiliary signals.

Therefore, L auxiliary signals 512 can be input to the analysis unit 328, so that the L auxiliary signals 512 are taken into account when generating the reconstruction matrix. The analysis unit 328 can also send control signals to the auxiliary signal generation unit 548. For example, the analysis unit 328 can control which audio objects/soundbed channels are included in the auxiliary signals and how they are included. In particular, the analysis unit 328 can control the selection of the Q matrix. This control can be based, for example, on the MMSE method described above, so that the auxiliary signals can be selected so that the reconstructed audio objects/soundbed channels are as close as possible to the audio objects 106a/soundbed channels 106b.

The operation of the decoder side of the audio encoding/decoding system 100 will now be described in more detail with reference to the flowcharts in Figures 6 and 7.

Figure 6 illustrates in more detail the bitstream decoding unit 118 and decoder 120 of Figure 1. Decoder 120 includes a reconstruction matrix generation unit 622 and a reconstruction unit 624.

In step D02, the bitstream decoding unit 118 receives the bitstream 116. The bitstream decoding unit 118 decodes and dequantizes the information in the bitstream 116 to extract M downmixed signals 112 and at least some matrix elements 114 in the reconstruction matrix.

The reconstruction matrix generation unit 622 receives matrix element 114 and continues in step D04 to generate reconstruction matrix 614. The reconstruction matrix generation unit 622 generates reconstruction matrix 614 by arranging matrix element 114 in appropriate positions within the matrix. If not all matrix elements of the reconstruction matrix are received, the reconstruction matrix generation unit 622 may, for example, insert zeros to replace the missing elements.

The reconstruction matrix 614 and M downmixed signals are then input to the reconstruction unit 624. The reconstruction unit 624 then reconstructs the N audio objects in step D06, and, if possible, reconstructs the bed channel. In other words, the reconstruction unit 624 generates an approximation 106' of the N audio objects 106a/bed channel 106b.

For example, the M downmix signals may correspond to a specific speaker configuration, such as the speaker configuration [Lf Rf Cf Ls Rs LFE] in a 5.1 speaker configuration. In this case, the reconstruction unit 624 can make the reconstruction of object 106' based solely on the downmix signals of the full-band channels corresponding to the speaker configuration. As explained above, the band-limited signal (low-frequency LFE signal) can be sent to the presenter substantially unmodified.

The reconstruction unit 624 typically operates in the frequency domain. More precisely, the reconstruction unit 624 operates on individual time-frequency blocks of the input signal. Therefore, before being input to the reconstruction unit 624, the M downmixed signals 112 typically undergo a time-to-frequency transformation 623. The time-to-frequency transformation 623 is typically the same as or similar to the transformation 338 applied on the encoder side. For example, the time-to-frequency transformation 623 could be a QMF transformation.

To reconstruct the audio object/sound bed channel 106', reconstruction component 624 applies matrix operations. More specifically, using the previously introduced notation, reconstruction component 624 can generate an approximation A' of the audio object/sound bed channel as A' = R1 * D. The reconstruction matrix R1 can vary according to time and frequency. Therefore, the reconstruction matrix can differ between different time-frequency blocks processed by reconstruction component 624.

Before being output from decoder 120, the reconstructed audio object/sound bed channel 106' is typically transformed back to the time domain 625.

Figure 8 illustrates the case when bitstream 116 additionally includes auxiliary signals. Compared to the embodiment of Figure 7, bitstream decoding unit 118 now additionally decodes one or more auxiliary signals 512 from bitstream 116. The auxiliary signals 512 are input to reconstruction unit 624, where they are included in the reconstruction of the audio object/soundbed channel. More specifically, reconstruction unit 624 generates the audio object/soundbed channel by applying matrix operation A' = R1 * [D ^T C ^T ] ^T.

Figure 9 illustrates the different time-frequency transformations used on the decoder side of the audio encoding/decoding system 100 of Figure 1. Bitstream decoding unit 118 receives bitstream 116. Decoding and dequantization unit 918 decodes and dequantizes bitstream 116 to extract position information 104, M downmixed signals 112, and matrix elements 114 of the reconstruction matrix.

In this stage, M downmixed signals 112 are typically represented in a first frequency domain, which corresponds to a first set of time-frequency filter banks, denoted herein by T/ _FC and F/ _TC , for transformations from the time domain to the first frequency domain and from the first frequency domain to the time domain, respectively. Typically, the filter banks corresponding to the first frequency domain can implement overlapping window transforms, such as MDCT and inverse MDCT. The bitstream decoding unit 118 may include a transformation unit 901 that transforms the M downmixed signals 112 to the time domain using the filter bank F/ _TC .

Decoder 120, particularly reconstruction unit 624, typically processes signals with respect to the second frequency domain. The second frequency domain corresponds herein to a second set of time-frequency filter banks, denoted herein by T/F _U and F/T _U , for transformations from the time domain to the second frequency domain and from the second frequency domain to the time domain, respectively. Therefore, decoder 120 may include a transformation unit 903 that transforms the M downmixed signals 112 represented in the time domain to the second frequency domain using the filter bank T/F _U. When reconstruction unit 624 has reconstructed object 106' based on the M downmixed signals by performing processing in the second frequency domain, transformation unit 905 can transform the reconstructed object 106' back to the time domain using the filter bank F/T _U.

Presenter 122 typically processes signals with respect to the third frequency domain. The third frequency domain corresponds to a third set of time-frequency filter banks, denoted herein by T/ _FR and F/ _TR , used for transformations from the time domain to the third frequency domain and from the third frequency domain to the time domain, respectively. Therefore, presenter 122 may include a transformation unit 907 that transforms the reconstructed audio object 106' from the time domain to the third frequency domain using the filter bank T/ _FR. When presenter 122 has already presented the output channel 124 via presentation unit 922, the output channel can be transformed to the time domain by transformation unit 909 using the filter bank F/ _TR .

As is evident from the above description, the decoder side of an audio encoding/decoding system includes many time-frequency transformation steps. However, if the first, second, and third frequency domains are chosen in a certain way, some steps in the time-frequency transformation process become redundant.

For example, some choices in the first, second, and third frequency domains can be the same, or they can be implemented together as a direct transition from one frequency domain to another without passing through the time domain in between. An example of the latter is the case where the second and third frequency domains differ only in that the transform unit 907 in the renderer 122 uses a Nyquist filter bank in addition to the shared QMF filter bank of both transform units 905 and 907 to improve frequency resolution at low frequencies. In this case, transform units 905 and 907 can be implemented together as a Nyquist filter bank, thus saving computational complexity.

In another example, the second and third frequency domains are identical. For instance, both the second and third frequency domains could be QMF frequency domains. In this case, transform components 905 and 907 are redundant and can be removed, thus saving computational complexity.

According to another example, the first frequency domain and the second frequency domain can be the same. For example, both the first frequency domain and the second frequency domain can be the MDCT domain. In this case, the first transformation component 901 and the second transformation component 903 can be eliminated, thereby saving computational complexity.

Equivalents, extensions, alternatives, and others

Other embodiments of this disclosure will become apparent to those skilled in the art upon studying the above description. While embodiments and examples are disclosed in this specification and accompanying drawings, this disclosure is not limited to these specific examples. Many modifications and variations may be made without departing from the scope of this disclosure as defined by the appended claims. Any reference numerals appearing in the claims are not to be construed as limiting their scope.

Furthermore, based on a study of the accompanying drawings, the disclosure, and the appended claims, those skilled in the art can understand and implement variations of the disclosed embodiments in practicing this disclosure. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" does not exclude a plural form. The fact that certain measures are cited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to generate profit.

The systems and methods disclosed above can be implemented as software, firmware, hardware, or a combination thereof. In a hardware implementation, the task division among the functional units mentioned above does not necessarily correspond to the division of physical units; rather, a physical component may have multiple functions, and several physical components may jointly perform a task. Some or all of the components may be implemented as software executed by a digital signal processor or microprocessor, or may be implemented as hardware or application-specific integrated circuits. Such software may be distributed on a computer-readable medium that may include computer storage media (or non-transitory media) and communication media (or transient media). As is well known to those skilled in the art, the term computer storage medium includes volatile and non-volatile media, removable and non-removable media, implemented in any way or by any technique for storing information such as computer-readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage devices, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is well known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves, or other transmission mechanisms, and include any information transmission medium.

This disclosure also includes the following schemes.

(1) A method for encoding a time-frequency block of an audio scene comprising at least N audio objects, the method comprising:

Receive the N audio objects;

Generate M downmixed signals based on at least the N audio objects;

A reconstruction matrix is generated using matrix elements, the reconstruction matrix enabling the reconstruction of at least the N audio objects based on the M downmixed signals; and

Generate a bitstream, the bitstream comprising at least some of the matrix elements of the M downmixed signals and the matrix elements of the reconstruction matrix.

(2) The method according to scheme (1) wherein the M downmixed signals are arranged in a first field of the bitstream using a first format and the matrix elements are arranged in a second field of the bitstream using a second format, thereby allowing a decoder that only supports the first format to decode and replay the M downmixed signals in the first field and discard the matrix elements in the second field.

(3) The method according to any of the above schemes further includes the step of: receiving position data corresponding to each of the N audio objects, wherein the M downmixing signals are generated based on the position data.

(4) The method according to any of the foregoing schemes, wherein the matrix elements of the reconstructed matrix are time-varying and frequency-varying.

(5) The method according to any of the foregoing schemes, wherein the audio scene further includes multiple audio bed channels, wherein the M downmix signals are generated based on at least the N audio objects and the multiple audio bed channels.

(6) The method according to scheme (5), wherein the reconstruction matrix includes matrix elements that enable the reconstruction of the sound bed channel based on the M downmixed signals.

(7) The method according to any of the above schemes, wherein the audio scene initially includes K audio objects, where K>N, and the method further includes the steps of: receiving the K audio objects and reducing the K audio objects to the N audio objects by clustering the K audio objects into N clusters and representing each cluster with an audio object.

(8) The method according to scheme (7) further includes the step of: receiving location data corresponding to each of the K audio objects, wherein the K objects are clustered into N clusters based on the location distance between the K objects given by the location data of the K audio objects.

(9) The method according to any of the foregoing schemes, wherein the number M of the downmixed signals is greater than 2.

(10) The method according to any of the foregoing schemes further includes:

The N audio objects form L auxiliary signals;

This will enable the matrix elements that allow the reconstruction of at least the N audio objects based on the M downmixed signals and the L auxiliary signals to be included in the reconstruction matrix; and

The L auxiliary signals are included in the bit stream.

(11) The method according to scheme (10), wherein at least one of the L auxiliary signals is the same as one of the N audio objects.

(12) The method according to any of the schemes (10) to (11), wherein at least one of the L auxiliary signals is formed as a combination of at least two audio objects among the N audio objects.

(13) The method according to any of the schemes (10) to (12), wherein the M downmixing signals cross a hyperplane, and wherein at least one of the plurality of auxiliary signals is not located in the hyperplane crossed by the M downmixing signals.

(14) The method according to scheme (13), wherein at least one of the plurality of auxiliary signals is orthogonal to the hyperplane spanned by the M downmixing signals.

(15) A computer-readable medium comprising computer code instructions adapted to perform the method according to any one of schemes (1) to (14) when operated on a processing device.

(16) An encoder for encoding a time-frequency block of an audio scene comprising at least N audio objects, the encoder comprising:

A receiving component, configured to receive the N audio objects;

A downmixing generation unit is configured to receive the N audio objects from the receiving unit and to generate M downmixing signals based on at least the N audio objects;

Analysis components are configured to generate a reconstruction matrix using matrix elements, the reconstruction matrix enabling the reconstruction of at least the N audio objects based on the M downmixed signals; and

A bitstream generation unit is configured to receive the M downmixed signals from the downmixing generation unit and the reconstruction matrix from the analysis unit, and to generate a bitstream comprising at least some of the matrix elements of the M downmixed signals and the reconstruction matrix.

(17) A method for decoding a time-frequency block of an audio scene comprising at least N audio objects, the method comprising the steps of:

Receive a bit stream comprising at least some matrix elements of M downmixed signals and a reconstruction matrix;

The reconstructed matrix is generated using the matrix elements; and

The reconstruction matrix is used to reconstruct the N audio objects based on the M downmixed signals.

(18) The method according to scheme (17) wherein the M downmixed signals are arranged in a first field of the bitstream using a first format and the matrix elements are arranged in a second field of the bitstream using a second format, thereby allowing a decoder that only supports the first format to decode and replay the M downmixed signals in the first field and discard the matrix elements in the second field.

(19) The method according to any of the schemes (17) to (18), wherein the matrix elements of the reconstructed matrix are time-varying and frequency-varying.

(20) The method according to any one of the schemes (17) to (19), wherein the audio scene further includes a plurality of audio bed channels, and the method further includes reconstructing the audio bed channels according to the M downmixing signals using the reconstruction matrix.

(21) The method according to any of the schemes (17) to (20), wherein the number of downmixed signals M is greater than 2.

(22) The method according to any one of schemes (17) to (21) further includes:

Receive L auxiliary signals formed by the N audio objects;

The reconstruction matrix is used to reconstruct the N audio objects based on the M downmixed signals and the L auxiliary signals, wherein the reconstruction matrix includes matrix elements that enable the reconstruction of at least the N audio objects based on the M downmixed signals and the L auxiliary signals.

(23) The method according to scheme (22) wherein at least one of the L auxiliary signals is the same as one of the N audio objects.

(24) The method according to any of the schemes (22) to (23), wherein at least one of the L auxiliary signals is a combination of the N audio objects.

(25) The method according to any of the schemes (22) to (24), wherein the M downmixing signals cross a hyperplane, and wherein at least one of the plurality of auxiliary signals is not located in the hyperplane crossed by the M downmixing signals.

(26) The method according to scheme (25), wherein at least one of the plurality of auxiliary signals not located in the hyperplane is orthogonal to the hyperplane spanned by the M downmixing signals.

(27) The method according to any of the schemes (17) to (26), wherein the M downmixed signals are represented with respect to the first frequency domain, and wherein the reconstruction matrix is represented with respect to the second frequency domain, wherein the first frequency domain and the second frequency domain are the same frequency domain.

(28) The method according to scheme (27), wherein the first frequency domain and the second frequency domain are improved discrete cosine transform (MDCT) domains.

(29) The method according to any one of schemes (17) to (28) further includes: receiving position data corresponding to the N audio objects, and

The location data is used to render the N audio objects to create at least one output audio channel.

(30) The method according to scheme (29), wherein the reconstruction matrix is represented with respect to a second frequency domain corresponding to the second filter group, and the presentation is performed in a third frequency domain corresponding to the third filter group, wherein the second filter group and the third filter group are at least partially the same filter group.

(31) The method according to scheme (30), wherein the second filter bank and the third filter bank include a quadrature mirror filter (QMF) filter bank.

(32) A computer-readable medium comprising computer code instructions adapted to perform the method according to any one of schemes 17 to 31 when operated on a processing device.

(33) A decoder for decoding a time-frequency block of an audio scene comprising at least N audio objects, the decoder comprising:

A receiving unit configured to receive a bit stream comprising at least some matrix elements of a matrix including M downmixed signals and a reconstruction matrix;

A reconstruction matrix generation component is configured to receive the matrix elements from the receiving component and generate the reconstruction matrix based on the matrix elements; and

A reconstruction component is configured to receive the reconstruction matrix from the reconstruction matrix generation component and to reconstruct the N audio objects based on the M downmixed signals using the reconstruction matrix.

Claims (20)

1. A method for decoding an audio scene represented by N audio signals, the method comprising: Receive a bitstream comprising M downmixed signals and matrix elements of a reconstruction matrix, wherein at least some of the matrix elements are zero values; Use the matrix elements to generate a reconstructed matrix; and The reconstruction matrix is used to reconstruct the N audio signals from the M downmixed signals, wherein the N audio signals are approximated as a linear combination of the M downmixed signals, and the matrix elements of the reconstruction matrix serve as coefficients in the linear combination. Where M is less than N, and M is equal to or greater than 1. 2. The method according to claim 1, further comprising: receiving L auxiliary signals in the bitstream, and reconstructing the N audio signals using the reconstruction matrix based on the M downmixed signals and the L auxiliary signals. 3. The method according to claim 1, wherein at least some of the M downmixed signals are formed by two or more of the N audio signals. 4. The method of claim 1, wherein at least some of the N audio signals are presented to generate a three-dimensional audio environment. 5. The method of claim 1, wherein the audio scene includes a three-dimensional audio environment, the three-dimensional audio environment including audio units associated with positions in three-dimensional space that can be presented for playback on an audio system. 6. The method of claim 1, wherein the M downmixed signals are arranged in a first field of the bitstream using a first format, and the matrix elements are arranged in a second field of the bitstream using a second format. 7. The method of claim 1, wherein the linear combination is formed by multiplying the matrix of the M downmixed signals with the reconstruction matrix. 8. The method of claim 1, further comprising: receiving L auxiliary signals, wherein the linear combination is formed by multiplying the matrix of the M downmixed signals and the L auxiliary signals with the reconstruction matrix. 9. The method of claim 1, wherein the M downmixed signals are decoded before the reconstruction is performed. 10. The method of claim 1, further comprising: receiving one or more audio bed channels in the bitstream, and reconstructing the N audio signals using the reconstruction matrix based on the M downmixed signals and the audio bed channels. 11. The method of claim 10, further comprising: receiving L auxiliary signals in the bitstream, and reconstructing the N audio signals using the reconstruction matrix based on the M downmixing signals, the L auxiliary signals, and the one or more audio bed channels. 12. The method of claim 11, wherein the one or more audio channels represent audio units having fixed positions in the audio scene. 13. A non-transitory computer-readable medium comprising instructions that, when executed by a processor of an information processing system, cause the information processing system to perform the method according to claim 1. 14. An audio decoder for decoding an audio scene represented by N audio signals, the audio decoder comprising: A receiver that receives a bitstream comprising M downmixed signals and matrix elements of a reconstructed matrix, wherein at least some of the matrix elements are zero values; A reconstruction matrix generator that receives the matrix elements from the receiver and generates the reconstruction matrix based on the matrix elements; and A reconstructor receives the reconstruction matrix from the reconstruction matrix generator and uses the reconstruction matrix to reconstruct the N audio signals based on the M downmixed signals, wherein the N audio signals are approximated as a linear combination of the M downmixed signals, and the matrix elements of the reconstruction matrix serve as coefficients in the linear combination. Where M is less than N, and M is equal to or greater than 1. 15. The audio decoder of claim 14, wherein the receiver further receives L auxiliary signals in the bitstream, and the reconstructor reconstructs the N audio signals using the reconstruction matrix based on the M downmixed signals and the L auxiliary signals. 16. The audio decoder of claim 14, wherein at least some of the M downmixed signals are formed by two or more of the N audio signals. 17. The audio decoder of claim 14, wherein the M downmixed signals are arranged in a first field of the bitstream using a first format, and the matrix elements are arranged in a second field of the bitstream using a second format. 18. The audio decoder of claim 14, wherein the audio decoder is configured to present at least some of the N audio signals to generate a three-dimensional audio environment. 19. The audio decoder of claim 14, wherein the linear combination is formed by multiplying the matrix of the M downmixed signals with the reconstruction matrix. 20. The audio decoder of claim 14, wherein the audio decoder is configured to decode the M downmixed signals prior to performing the reconstruction.