HK1241131B - Method, apparatus and computer readable medium for decoding hoa audio signals - Google Patents

Description

Method, apparatus, and computer-readable medium for decoding HOA audio signals

This application is a divisional application based on the patent application with application number 201380036698.6, application date July 16, 2013, and invention name “Method and device for encoding multi-channel HOA audio signals for noise reduction and method and device for decoding multi-channel HOA audio signals for noise reduction”.

Technical Field

The present invention relates to a method and apparatus for encoding a multi-channel higher-order Ambisonics audio signal for noise reduction, and a method and apparatus for decoding a multi-channel higher-order Ambisonics audio signal for noise reduction.

Background Art

Higher Order Ambisonics (HOA) is a multi-channel sound field representation [4], and an HOA signal is a multi-channel audio signal. Playing back certain multi-channel audio signal representations, in particular HOA representations, on a specific loudspeaker arrangement requires special rendering, which usually involves matrixing operations. After decoding, the Ambisonics signal is "matrixed", i.e., mapped to new audio signals corresponding to, for example, the actual spatial positions of the loudspeakers. Typically, there is a high cross-correlation between the individual channels.

The problem is that the coding noise increases after the matrixing operation. The reason seems to be unknown in the prior art. This effect also occurs when the HOA signal is transformed into the spatial domain, for example by the Discrete Spherical Harmonics Transform (DSHT), before being compressed by the perceptual coder.

A common approach for compression of high-order Ambisonics audio signal representations is to apply independent perceptual coders to the individual Ambisonics coefficient channels [7]. Specifically, the perceptual coder only considers the noise masking effect that occurs in each individual mono channel signal for coding. However, this effect is typically non-linear. If such mono channels are matrixed into a new signal, noise unmasking may occur. This effect also occurs when the high-order Ambisonics signal is transformed into the spatial domain by the discrete spherical harmonics transform before compression with the perceptual coder [8].

The transmission or storage of such a multi-channel audio signal representation generally requires appropriate multi-channel compression techniques. Typically, channel-independent perceptual decoding is performed before finally matrixing the I decoded signals into J new signals. The term matrixing means adding or mixing the decoded signals in a weighted manner to arrange all signals and all new signals in a vector according to the following:

The term "matrixization" comes from the fact that mathematically, the matrix is obtained from

Here, A represents a mixing matrix consisting of mixing weights. The terms "mixing" and "matrix" are used synonymously herein. Mixing/matrixization serves the purpose of rendering the audio signal for any particular loudspeaker setup. The specific individual loudspeaker setup to which the matrix depends, and therefore the matrix used for matrixing during operation, is generally unknown during the perceptual coding stage.

Summary of the Invention

The present invention provides for encoding and/or decoding of multi-channel high-order Ambisonics audio signals in order to obtain improved noise reduction. In particular, the present invention provides a way to de-mask 3D audio ratio compression suppression coding noise.

The present invention describes a technique for an adaptive discrete spherical harmonic transform (aDSHT) that minimizes (undesirable) noise demasking effects. Furthermore, it is described how the aDSHT can be integrated into a compression coder architecture. The described technique is particularly advantageous, at least for HOA signals. One advantage of the present invention is that the amount of side information to be transmitted is reduced. In principle, only the rotation axis and the rotation angle need to be transmitted. The DSHT sampling grid can be signaled indirectly via the number of channels transmitted. Compared to other methods, such as the Karhunen Loève transform (KLT), which require the transmission of more than half of the correlation matrix, the amount of this side information is very small.

According to one embodiment of the present invention, a method for encoding a multi-channel HOA audio signal for noise reduction includes the following steps: decorrelating the channels using an inverse adaptive DSHT, the inverse adaptive DSHT including a rotation operation and an inverse DSHT (iDSHT), the rotation operation rotating the spatial sampling grid of the iDSHT; perceptually encoding each decorrelated channel; encoding rotation information, the rotation information including parameters defining the rotation operation; and transmitting or storing the perceptually encoded audio channels and the encoded rotation information. The step of decorrelating the channels using the inverse adaptive DSHT is, in principle, a spatial encoding step.

According to one embodiment of the present invention, a method for decoding an encoded multi-channel HOA audio signal with reduced noise comprises the following steps: receiving the encoded multi-channel HOA audio signal and channel rotation information; decompressing the received data, wherein perceptual decoding is used; spatially decoding each channel using adaptive DSHT (aDSHT), correlating the perceptually decoded and spatially decoded channels, wherein a rotation of the spatial sampling grid of the aDSHT according to the rotation information is performed; and matrixing the correlated perceptually decoded and spatially decoded channels, wherein a reproducible audio signal mapped to a speaker position is obtained.

Disclosed is a device for encoding a multi-channel HOA audio signal. Disclosed is a device for decoding a multi-channel HOA audio signal.

In one aspect, the computer-readable medium has executable instructions to cause a computer to perform a method for encoding comprising the steps disclosed above, or to perform a method for decoding comprising the steps disclosed above. Advantageous embodiments of the invention are disclosed in the dependent claims, the following description and the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the present invention are described with reference to the accompanying drawings, in which:

FIG1 shows a known encoder and decoder for rate compression of a block of M coefficients;

FIG2 shows a known encoder and decoder for transforming an HOA signal into the spatial domain using conventional DSHT (Discrete Spherical Harmonic Transform) and conventional inverse DSHT;

FIG3 shows an encoder and decoder for transforming an HOA signal into the spatial domain using adaptive DSHT and adaptive inverse DSHT;

Figure 4 shows a test signal;

FIG5 shows an example of spherical sampling positions of a codebook used in encoder and decoder building blocks;

Figure 6 shows the signal adaptive DSHT building blocks (pE and pD);

FIG7 shows a first embodiment of the present invention;

FIG8 shows a flowchart of the encoding process and the decoding process; and

FIG9 shows a second embodiment of the present invention.

DETAILED DESCRIPTION

Figure 2 shows a known system that transforms an HOA signal into the spatial domain using inverse DSHT. The signal undergoes transformation using iDSHT 21, rate compression E1/decompression D1, and retransformation to the coefficient domain S24 using DSHT 24. In contrast, Figure 3 shows a system according to one embodiment of the present invention: the DSHT processing block of the known solution is replaced with processing blocks 31 and 34 that control inverse adaptive DSHT and adaptive DSHT, respectively. Side information SI is transmitted within the bitstream bs. The system includes elements of an apparatus for encoding multi-channel HOA audio signals and elements of an apparatus for decoding multi-channel HOA audio signals.

In one embodiment, an apparatus ENC for encoding a multi-channel HOA audio signal for noise reduction includes a decorrelator 31 for decorrelating channel B using an inverse adaptive DSHT (iaDSHT), which includes a rotation operation unit 311 and an inverse DSHT (iDSHT) 310. The rotation operation unit rotates the spatial sampling grid of the iDSHT. The decorrelator 31 provides decorrelated channels _Wsd and side information SI including rotation information. Furthermore, the apparatus includes a perceptual encoder 32 for perceptually encoding each decorrelated channel _Wsd and a side information encoder for encoding the rotation information. The rotation information includes parameters defining the rotation operation. The perceptual encoder 32 provides the perceptually encoded audio channel and the encoded rotation information, thereby reducing the data rate. Finally, the apparatus for encoding includes an interface device 320 for creating a bitstream bs from the perceptually encoded audio channel and the encoded side information, and for transmitting or storing the bitstream bs.

The apparatus DEC for decoding a multi-channel HOA audio signal with reduced noise includes an interface 330 for receiving an encoded multi-channel HOA audio signal and channel rotation information; and a decompression module 33 for decompressing the received data, including a perceptual decoder for perceptually decoding each channel. The decompression module 33 provides recovered perceptually decoded channels _W'sd and recovered side information SI'. Furthermore, the apparatus for decoding includes a correlator 34 for correlating the perceptually decoded channels _W'sd using adaptive DSHT (aDSHT), wherein DSHT and a rotation of the DSHT spatial sampling grid based on the rotation information are performed; and a mixer MX for matrixing the correlated perceptually decoded channels, thereby obtaining a reproducible audio signal mapped to speaker positions. At least aDSHT can be performed in a DSHT unit 340 within the correlator 34. In one embodiment, the rotation of the spatial sampling grid is performed in a grid rotation unit 341, which essentially recalculates the original DSHT sampling points. In another embodiment, the rotation is performed within the DSHT unit 340.

The mathematical model for defining and describing demasking is given below. Assume that a given discrete-time multichannel signal comprises I channels x _i (m), i = 1, ..., I, where m represents the time sample index. The individual signals can be real or complex values. Consider a frame of M samples starting with time sample index m _START + 1:, where the individual signals are assumed to be stationary. The corresponding samples are arranged in the matrix according to the following formula:

X: =[x(m _START +1),...,x(m _START +M)] (1)

in

x(l):=[x ₁ (m),...,x _I (m)] ^T (2)

where (·) ^T denotes the transpose. The corresponding empirical correlation matrix is given by:

∑ _X ：＝XX ^H (3)

where (·) ^H represents the joint complex conjugate and transpose.

Now assume that the multi-channel signal frame has been encoded, thereby introducing coding error noise during reconstruction. Therefore, the matrix of reconstructed frame samples represented by is composed of the real sample matrix X and the coding noise component E according to the following formula:

in

E:=[e(m _START +1),...,e(m _START +L)] (5)

and

e(m):=[e ₁ (m),...,e _I (m)] ^T (6)

Since each channel is assumed to have been independently coded, it can be assumed that the coded noise signals _ei (m) are independent of each other for i=1, .., I. Using this property and the assumption that the noise signal is zero-mean, the empirical correlation matrix of the noise signal is given by the following diagonal matrix:

Here, represents a diagonal matrix with the empirical noise signal power on its diagonal

Another basic assumption is that the encoding is performed so that a predefined signal-to-noise ratio (SNR) is satisfied for each channel. Without loss of generality, it is assumed that the predefined SNR is equal for each channel, that is:

in

From now on, consider matrixing the reconstructed signal into J new signals y _j (m), j=1, ..., J. Without introducing any coding error, the sample matrix of the matrixed signal can be expressed as:

Y＝A X (11)

where denotes the mixing matrix, and where

Y: =[y(m _START +1),...,y(m _START +M)] (12)

in

y(m):=[y ₁ (m),...,y _J (m)] ^T (13)

However, due to coding noise, the sample matrix of the matrixed signal is given by:

Where N is a matrix containing samples of the matrixed noise signal. It can be expressed as:

N＝AE (15)

N＝[n(m _START +1) ... n(m _START +M) (16)

in

n(m):=[n ₁ (m) ... n _J (m)] ^T (17)

is the vector of all matrixed noise signals at time sample index m.

Using equation (11), the empirical correlation matrix of the matrixed noise-free signal can be formulated as:

∑ _Y =A∑ _X A ^H (18)

Therefore, the empirical power of the j-th matrixed noise-free signal, which is the j-th element on the diagonal of ∑ _Y, can be written as:

where _aj is the jth column of ^AH according to the following formula:

A ^H =[a ₁ ,...,a _J ] (20)

Similarly, using equation (15), the empirical correlation matrix of the matrixed noise signal can be written as:

∑ _N =A∑ _E A ^H (21)

The empirical power of the jth matrixed noise signal, which is the jth element on the diagonal of ∑ _N, is given by:

Therefore, for the empirical SNR of the matrixed signal defined by,

It can be reformulated using equations (19) and (22) as:

By decomposing ∑ _X into its diagonal and off-diagonal components as follows:

as well as

And by using the following properties from assumptions (7) and (9) and constant SNR on all channels:

Finally we obtain the desired expression for the empirical SNR of the matrixed signal:

As can be seen from this expression, the SNR is obtained from the predefined SNR (SNR _x ) by multiplying the terms that depend on the diagonal and off-diagonal components of the signal correlation matrix Σ _X. Specifically, if the signals x _i (m) are uncorrelated with each other, so that Σ _X,NG becomes a zero matrix, then the empirical SNR of the matrixed signal is equal to the predefined SNR, that is:

For all j = 1, ..., J, if ∑ _X,NG = 0 _{I × I} (30)

Where 0 _I×I represents a zero matrix with I rows and I columns. That is, if the signals x _i (m) are correlated, the empirical SNR of the matrixed signal may deviate from the predefined SNR. In the worst case, it may be much lower than the SNR _x . This phenomenon is referred to herein as noise unmasking during matrixing.

The following section gives a brief introduction to Higher Order Ambisonics (HOA) and defines the signals to be processed (data rate compression).

High-order Ambisonics (HOA) is based on the description of the sound field in a compact region of interest that is assumed to be free of sound sources. In this case, the spatiotemporal behavior of the sound pressure p(t, x) at time t and at position x = [r, θ, φ] ^T (in spherical coordinates) within the region of interest is physically determined entirely by homogeneous wave equations. It can be shown that the Fourier transform of the sound pressure with respect to time, i.e.,

where ω represents the angular frequency (and corresponds to),

It can be expanded into spherical harmonic series (SHs) according to [10]:

In equation (32), _cs represents the speed of sound, and represents the angular wave number. In addition, _jn (·) indicates the nth-order spherical Bessel function of the first kind, representing the nth-order mth spherical harmonic (SH). Complete information about the sound field is actually contained in the sound field coefficients.

It should be noted that SHs are generally complex-valued functions. However, through appropriate linear combinations of them, real-valued functions can be obtained, and expansion can be performed on these functions.

In conjunction with the pressure field description in equation (32), the source field can be defined as:

The source field or amplitude density [9] D( _kcs , Ω) depends on the angular wave number and angular direction Ω = [θ, φ] ^T. The source field can include far field/near field, discrete/continuous sources [1]. The source field coefficient is related to the acoustic field coefficient according to the following formula [1]:

where is the spherical Hankel function of the second kind and _rs is the source distance from the origin.

The signal in the HOA domain can be represented in the frequency domain or time domain as the inverse Fourier transform of the source field or sound field coefficients. The following description will assume the use of a time domain representation of a finite number of source field coefficients:

The finite number: The infinite series in (33) is truncated at n = N. The truncation corresponds to the spatial bandwidth limitation. The number of coefficients (or HOA channels) is given by:

O _3D = (N+1) ² for 3D (36)

Or for a 2D-only description, this is given by O _2D = 2N + 1. The coefficients contain the audio information for one time sample m for later reproduction by the loudspeaker. They can be stored or transmitted and are therefore the subject of data rate compression. A single time sample m of the coefficients can be represented by a vector b(m) with O _3D elements:

And a block of M time samples is represented by the matrix B:

B: =[b(m _START +1), b(m _START +2), .., b(m _START +M)] (38)

A two-dimensional representation of the sound field can be obtained by expanding the circular harmonics. This can be seen as a special case of the general description above using different weightings of fixed tilt coefficients and reducing the set to O _2D coefficients (m=±n). Therefore, all the following considerations also apply to the 2D representation, and the term sphere needs to be replaced by the term circle.

The following describes the transformation from the HOA coefficient domain to the channel-based spatial domain, and vice versa. Equation (33) can be rewritten using the time domain HOA coefficients for l discrete spatial sample locations Ω _l = [θ _l , φ _l ] ^T on the unit sphere:

Assuming L _sd =(N+1) ² spherical sample positions Ω _l , this can be rewritten in vector notation for the HOA data block B:

W＝Ψ _i B (36)

where W = [w(M _START +1), w(M _START +2), .. , W(m _START +M)] and represents a single time sample of the L _sd multichannel signal, the matrix where the vector If the spherical sample positions are chosen very regularly, then there exists a matrix Ψ _f where:

Ψ _f Ψ _i ＝I， (37)

where I is the identity matrix of O _3D ×O _3D . Then, the corresponding transformation to equation (36) can be defined as:

B＝Ψ _f W (38)

Equation (38) transforms the L _sd spherical signals into the coefficient domain and can be rewritten as the forward transform:

B＝DSHT{W), (39)

Where DSHT{} denotes discrete spherical harmonic transform. The corresponding inverse transform transforms the O _3D coefficient signal into the spatial domain to form L _sd channel-based signals, and equation (36) becomes:

W＝iDSHT{B} (40)

Here, this definition of the discrete spherical harmonic transform is sufficient for considerations of data rate compression for HOA data, since we start with a given coefficient B and focus only on the case where B = DSHT{iDSHT{B}}. A more rigorous definition of the discrete spherical harmonic transform is given in [2]. Appropriate spherical sample positions for the DSHT and the process of deriving such positions can be reviewed in [3], [4], [6], [5]. An example of a sampling grid is shown in FIG5.

5 shows examples of spherical sampling positions of the codebook used in the encoder and decoder building blocks pE, pD, i.e., for L _Sd = 4 in FIG. 5 a), for L _Sd = 9 in FIG. 5 b), for L _Sd = 16 in FIG. 5 c), and for L _Sd = 25 in FIG. 5 d).

The following describes ratio compression and noise demasking of higher order Ambisonics coefficient data. First, a test signal is defined to emphasize some of the properties used below.

A single far-field source located in the direction is represented by a vector of M discrete-time samples g = [g(m), ..., g(M)] ^T and can be represented by a block of HOA coefficients by encoding:

B _g =yg ^T (45)

where the matrix _Bg is similar to equation (38) and the code vectors consist of the conjugated complex spherical harmonics evaluated in the direction (this conjugation is invalid if real-valued SH is used). The test signal can be viewed as the simplest case of an HOA signal. More complex signals consist of the superposition of many such signals.

Considering the direct compression of the HOA channel, the following shows why noise demasking occurs when the HOA coefficient channel is compressed. The direct compression and decompression of the O _3D coefficient channel of the actual HOA data block B will introduce coding noise E similar to equation (4):

Assuming the constants in equation (9), the signal needs to be rendered in order to reproduce it on the loudspeaker. The process can be described as:

where the decoding matrix is (and A ^H = [a ₁ , ..., a _L ]), and the matrix holds M time samples of the L loudspeaker signals. This is similar to (14). Applying all the above considerations, the SNR of loudspeaker channel l can be described (similar to equation (29)):

where is the oth diagonal element and ∑ _B,NG holds:

∑ _B =BB ^H (49)

The off-diagonal elements of .

The decoding matrix A should not be affected (since it should be able to decode for any loudspeaker layout), so the matrix ∑ _B needs to be diagonal to obtain (B = B _g ), ∑ _B = yg ^H gy ^H = c yy ^H and becomes non-diagonal with a constant scalar value c = g ^T g via equations (45) and (49). Compared to , the signal-to-noise ratio at the loudspeaker channel is reduced. However, since the sound source signal g and the loudspeaker layout are usually unknown at the encoding stage, direct lossy compression of the coefficient channel may lead to uncontrollable demasking effects, especially for low data rates.

The following describes why noise demasking occurs when compressing HOA coefficients in the spatial domain after using DSHT.

The current block B of HOA coefficient data is transformed into the spatial domain using the spherical harmonic transform given in equation (36) before compression:

W _sd ＝Ψ _i B (50)

where the inverse transform matrix Ψ _i is associated with L _Sd ≥ O _3D spatial sample positions, and the spatial signal matrix compresses and decompresses these, and adds quantization noise (similar to equation (4)):

where the coding noise component E is according to equation (5). Again, a constant SNR for all spatial channels, namely SNB _Sd , is assumed. This signal is transformed into the coefficient domain equation (42) using the transformation matrix _Ψf , which has _the property (41): _ΨfΨi =I. The new block of coefficients becomes:

The signal is rendered as L loudspeaker signals by applying the decoding matrix _AD : This can be rewritten using (52) and A = _AD Ψ _f :

Here, A becomes a mixing matrix with . Equation (53) should be viewed as analogous to Equation (14). Applying all the above considerations again, the SNR of loudspeaker channel l can be described (similar to Equation (29)) as:

where is the l-th diagonal element and holds:

The off-diagonal elements of .

Since it never affects _AD (as it should be present for any loudspeaker layout), and therefore never has any effect on A, it needs to become close to diagonal to maintain the desired SNR: using a simple test signal from equation (45) (B = _Bg ), this becomes:

where c = g ^T g is constant. Using a fixed spherical harmonic transform (Ψ _i , Ψ _f fixed), it can only become diagonal in very rare cases and worse, as described above, the term depends on the coefficient signal space characteristics. Therefore, low-rate lossy compression of HOA coefficients in the spherical domain may lead to a reduction in SNR and uncontrollable demasking effects.

The basic idea of the present invention is to minimize noise demasking by using an adaptive DSHT (aDSHT), which consists of a rotation of the spatial sampling grid of the DSHT related to the spatial characteristics of the HOA input signal and the DSHT itself.

Next, we describe a signal-adaptive DSHT (aDSHT) with a number of spherical positions L _Sd that matches the number of HOA coefficients O _3D . First, we select a default spherical sample grid as in conventional non-adaptive DSHT. For a block of M time samples, we rotate the spherical sample grid so that the term

The logarithm of , where is the absolute value of the elements of (with matrix row index l and column index j), and is the diagonal element of . This is equivalent to minimizing the term of equation (54)

Intuitively, as shown in Figure 4, this process corresponds to a rotation of the spherical sampling grid of the DSHT in such a way that a single spatial sample position matches the strongest source direction. Using a simple test signal from equation (45) (B = _Bg ), it can be shown that the term W _Sd of equation (55) becomes a vector (where all elements except one are close to zero). Therefore, it becomes close to a diagonal and the desired SNR can be maintained.

Figure 4 shows the test signal B _g transformed into the spatial domain. In Figure 4a), the default sampling grid is used, and in Figure 4b), a rotated grid using aDSHT is used. The values (in dB) associated with the spatial channels are shown by the color/grayscale changes of the Voronoi cells around the corresponding sample positions. Each cell of the spatial structure represents a sampling point, and the brightness/darkness of the cell represents the signal strength. As can be seen in Figure 4b), the strongest source direction is found, and the sampling grid is rotated so that one of the sides (i.e., a single spatial sample position) matches the strongest source direction. This side is illustrated as white (corresponding to a strong source direction), while the other sides are dark (corresponding to a weak source direction). In Figure 4a), that is, before the rotation, no side matches the strongest source direction, and several sides are darker/lighter gray, which means that a considerable (but not maximal) intensity audio signal is received at the corresponding sampling point.

The main building blocks of aDSHT used within the compression encoder and decoder are described below.

Figure 6 shows details of the encoder and decoder processing building blocks pE and pD. Both modules share the same codebook for the spherical sampling position grid that underlies the DSHT. Initially, a base grid with L _Sd = O _3D positions is selected in module pE according to the common codebook using the number of coefficients O _3D . L _Sd must be transferred to block pD for initialization, selecting the same base sampling position grid as indicated in Figure 3. The base sampling grid is described by the matrix Ω _l = [θ l , φ l ] T , where Ω l = [θ _l , φ _l ] ^T defines the positions on the unit sphere. As mentioned above, Figure 5 shows an example of a base grid.

The input to the rotation discovery block (building block "find best rotation") 320 is the coefficient matrix B. This building block is responsible for rotating the base sampling grid so that the value of equation (57) is minimized. The rotation is represented by an "axis-angle" representation, and the compressed axis _ψrot and the rotation angle associated with the rotation are output to the building block as side information SI. The rotation axis _ψrot can be described by a unit vector from the origin to a position on the unit sphere. In spherical coordinates, this can be combined by two angles: _ψrot = [ _θaxis , _φaxis ] ^T , with an implicit associated radius that does not need to be transmitted. The three angles _θaxis , _φaxis , are quantized and entropy coded by a special escape pattern that signals the reuse of previously used values to create the side information SI.

The building block "Build Ψ _i " 330 decodes the rotation axis and angle into known and applies the rotation to the base sampling grid to obtain a rotated grid whose output is the iDSHT matrix derived from the vector

In the building block "iDSHT" 310, the actual block B of HOA coefficient data is transformed into the spatial domain by _WSd = _ΨiB .

The building block "Build Ψ _f " 350 of the decoding processing block pD receives and decodes the rotation axis and angle into known and applies the rotation to the base sampling grid to obtain a rotated grid by using the vector iDSHT matrix and calculating the DSHT matrix Ψ _f =Ψ _i ⁻¹ on the decoding side.

In the building block "DSHT" 340 within the decoder processing block 34, the actual block of spatial domain data is transformed back into a block of coefficient domain data:

Various advantageous embodiments of the overall architecture including the compression codec are described below. A first embodiment uses a single aDSHT. A second embodiment uses multiple aDSHTs in a spectrum band.

A first ("basic") embodiment is shown in Figure 7. HOA time samples with index m of O _3D coefficient channels b(m) are first stored in a buffer 71 to form blocks of M samples and time index μ. B(μ) is transformed to the spatial domain using adaptive iDSHT in the building block pE72 described above. The spatial signal block W _Sd (μ) is input to L _Sd audio compression mono encoders 73 (such as AAC or mp3 encoders) or a single AAC multi-channel encoder (L _Sd channels). The bitstream S73 comprises a multiplexed frame of multiple encoder bitstream frames with integrated side information SI or a single multi-channel bitstream with integrated side information SI (preferably as auxiliary data).

In one embodiment, the corresponding compression decoder building block includes a demultiplexer D1 for demultiplexing the bitstream S73 into L _Sd bitstreams and side information SI and feeding the bitstream to L _Sd mono decoders, decoding them into L _Sd spatial audio channels with M samples to form blocks and feeding the sum SI to pD. In another embodiment where the bitstream is not multiplexed, the compression decoder building block includes a receiver 74 for receiving the bitstream and decoding it into L _Sd multichannel signals, unpacking the SI, and feeding the sum SI to pD.

In the decoder processing block pD 75, φ is transformed to the coefficient domain using adaptive DSHT and SI to form a block of HOA signals B(μ), which are stored in a buffer 76 for de-framing to form the temporal signal of coefficients b(m).

Under certain conditions, the first embodiment described above may have two disadvantages: first, due to the change in spatial signal distribution, there may be blocking artifacts from previous blocks (i.e., from blocks μ to μ+1); second, there may be more than one strong signal at the same time, and the decorrelation effect of aDSHT may be quite small.

In a second embodiment operating in the frequency domain, both drawbacks are addressed. A DSHT is applied to scale factor band data that combines multiple frequency bands. Blocking artifacts are avoided by processing overlapping time-frequency transform (TFT) blocks using Overlay Add (OLA). Improved signal decorrelation can be achieved using the present invention within J spectral bands at the cost of increased overhead in the data rate of transmitting SI _j .

The second embodiment shown in FIG. 9 is described in some further detail below: A time-frequency transform (TFT) 912 is performed on each coefficient channel of the signal b(m). A widely used example of a TFT is the modified cosine transform (MDCT). In the TFT framing unit 911, 50% overlapping data blocks (block index μ) are constructed. The TFT block transform unit 912 performs the block transform. In the spectral banding unit 913, the TFT bands are combined to form J new spectral bands and associated signals, where K _J represents the number of frequency coefficients in band j. These spectral bands are processed in a plurality of processing modules 914. For each of these spectral bands, there is a processing block pE _j that creates the signal and side information SI _j . The spectral bands can match the spectral bands of a lossy audio compression method (such as AAC/MP3 scale factor bands) or have a coarser granularity. In the latter case, channel-independent lossy audio compression without the TFT block 915 requires rearranging the banding. The processing block 914 operates like an _LSd multi-channel audio encoder in the frequency domain, assigning a constant bit rate to each audio channel. The bitstream is formatted in the bitstream wrapper block 916 .

The decoder receives or stores the bitstream (at least parts thereof), unpacks it 921, and feeds the audio data to a multi-channel audio decoder 922 for channel-independent audio decoding without TFT, and feeds the side information SI _j to a plurality of decoding processing blocks pD _j 923. The audio decoder 922 for channel-independent audio decoding without TFT decodes the audio information and formats J spectral band signals as input to the decoding processing blocks pD _j 923, where these signals are transformed into the HOA coefficient domain to form the J spectral bands in the de-banding block 924, which reorganizes the J spectral bands to match the banding of the TFT. They are transformed into the time domain in the iTFT and OLA block 925, which uses an overlap-add (OLA) process with block overlap. Finally, in the TFT de-framing block 926, the output of the iTFT and OLA module 925 is deframed to create the signal

The present invention is based on the discovery that the SNR increase is caused by the cross-correlation between channels. Perceptual encoders only consider the coding noise masking effect that occurs within each individual single-channel signal. However, this effect is typically nonlinear. Therefore, when such a single channel is matrixed into a new signal, noise demasking may occur. This is the reason why coding noise usually increases after the matrixing operation.

The present invention proposes to decorrelate the channels by an adaptive discrete spherical harmonic transform (aDSHT) that minimizes the unwanted noise demasking effect. The aDSHT is integrated into the compression encoder and decoder architecture. Because it includes a rotation operation that adjusts the spatial sampling grid of the DSHT to the spatial characteristics of the HOA input signal, it is adaptive. The aDSHT includes adaptive rotation and actual traditional DSHT. The actual DSHT is a matrix that can be constructed as described in the prior art. Adaptive rotation is applied to this matrix, resulting in minimization of inter-channel correlation and, therefore, minimization of the SNR increase after matrixing. The rotation axis and angle are found by an automatic search operation (rather than analytically). The rotation axis and angle are encoded and transmitted to enable re-correlation after decoding and before matrixing, using an inverse adaptive DSHT (iaDSHT).

In one embodiment, time-frequency transform (TFT) and spectral banding are performed, and aDSHT/iaDSHT is applied to each spectral band independently.

FIG8 a ) shows a flow chart of a method for encoding a multi-channel HOA audio signal for noise reduction in one embodiment of the present invention. FIG8 b ) shows a flow chart of a method for decoding a multi-channel HOA audio signal for noise reduction in one embodiment of the present invention.

In the embodiment shown in FIG8 a ), a method for encoding a multi-channel HOA audio signal for noise reduction comprises the following steps: decorrelating 81 the channels using an inverse adaptive DSHT comprising a rotation operation and an inverse DSHT 812, wherein the rotation operation rotates 811 the spatial sampling grid of the iDSHT; perceptually encoding 82 each decorrelated channel; encoding 83 rotation information (as side information SI) comprising parameters defining the rotation operation; and transmitting or storing 84 the perceptually encoded audio channels and the encoded rotation information.

In one embodiment, inverse adaptive DSHT comprises the steps of: selecting an initial default spherical sample grid; determining the strongest source direction; and, for blocks of M temporal samples, rotating the spherical sample grid so that a single spatial sample position matches the strongest source direction.

In one embodiment, the spherical sample grid is rotated so as to minimize the logarithm of:

where W Sd is the absolute value of the elements of (with matrix row index l and column index j), and W _Sd is the diagonal elements of W Sd, where W Sd is a matrix of the number of audio channels multiplied by the number of blocks of processed samples, and W _Sd is the result of aDSHT.

In the embodiment shown in FIG8 b ), a method for decoding an encoded multi-channel HOA audio signal with reduced noise comprises the following steps: receiving 85 an encoded multi-channel HOA audio signal and channel rotation information (within side information SI); decompressing 86 the received data, wherein perceptual decoding is used; spatially decoding 87 each channel using adaptive DSHT, wherein DSHT 872 and a rotation 871 of the spatial sampling grid of the DSHT according to the rotation information are performed, and wherein the perceptually decoded channels are re-correlated; and matrixing 88 the re-correlated perceptually decoded channels, wherein a reproducible audio signal mapped to a loudspeaker position is obtained.

In one embodiment, the adaptive DSHT comprises the steps of: selecting an initial default spherical sample grid for the adaptive DSHT; and, for blocks of M time samples, rotating the spherical sample grid according to the rotation information.

In one embodiment, the rotation information is a space vector with three components. Note that the rotation axis ψ _rot can be described by a unit vector.

In one embodiment, the rotation information is a vector consisting of three angles: θ _axis , φ _axis , wherein θ _axis , φ _axis define information about a rotation axis with an implicit radius in spherical coordinates and define a rotation angle about the axis.

In one embodiment, the corners are quantized and entropy coded by signaling (ie, indicating) an escape mode (ie, a dedicated bit pattern) that reuses previous values to create side information (SI).

In one embodiment, a device for encoding a multi-channel HOA audio signal for noise reduction includes: a decorrelator for decorrelating channels using an inverse adaptive DSHT, wherein the inverse adaptive DSHT includes a rotation operation and an inverse DSHT (iDSHT), wherein the rotation operation rotates the spatial sampling grid of the iDSHT; a perceptual encoder for perceptually encoding each decorrelated channel; a side information encoder for encoding rotation information, wherein the rotation information includes parameters defining the rotation operation; and an interface for transmitting or storing the perceptually encoded audio channels and the encoded rotation information.

In one embodiment, an apparatus for decoding a multi-channel HOA audio signal with reduced noise includes: an interface device 330 for receiving an encoded multi-channel HOA audio signal and channel rotation information; a decompression module 33 for decompressing the received data using a perceptual decoder for perceptually decoding each channel; a correlator 34 for re-correlating the perceptually decoded channels, wherein a DSHT and a rotation of the spatial sampling grid of the DSHT according to the rotation information are performed; and a mixer for matrixing the correlated perceptually decoded channels, wherein a reproducible audio signal mapped to a speaker position is obtained. In principle, the correlator 34 functions as a spatial decoder.

In one embodiment, an apparatus for decoding a multi-channel HOA audio signal with reduced noise includes: an interface device 330 for receiving an encoded multi-channel HOA audio signal and channel rotation information; a decompression module 33 for decompressing the received data by a perceptual decoder for perceptually decoding each channel; a correlator 34 for correlating the perceptually decoded channels using aDSHT, wherein the DSHT and a rotation of the spatial sampling grid of the DSHT according to the rotation information are performed; and a mixer MX for matrixing the correlated perceptually decoded channels, wherein a reproducible audio signal mapped to a speaker position is obtained.

In one embodiment, the adaptive DSHT in the apparatus for decoding includes means for selecting an initial default sample grid for the adaptive DSHT, a rotation processing means for rotating the default spherical sample grid according to the rotation information for a block of M time samples, and a transform processing means for performing DSHT on the rotated spherical sample grid.

In one embodiment, the correlator 34 in the apparatus for decoding includes a plurality of spatial decoding units 922 for spatially decoding each channel simultaneously using adaptive DSHT, a de-banding unit 924 for performing de-banding, and an iTFT and OLA unit 925 for performing inverse time-frequency transform by overlap-add processing, wherein the de-banding unit provides its output to the iTFT and OLA unit.

In all embodiments, the term reduced noise at least relates to avoiding coding noise demasking.

Perceptual coding of audio signals refers to coding that is tailored to human perception of the audio. It should be noted that when perceptually coding an audio signal, quantization is typically performed not on the wideband audio signal samples, but rather in individual frequency bands relevant to human perception. Consequently, the ratio between signal power and quantization noise can vary between individual frequency bands. Thus, perceptual coding typically involves reducing redundant and/or irrelevant information, while spatial coding typically involves the spatial relationship between channels.

The above-described technique can be viewed as an alternative to decorrelation using the Karhunen-Loève transform (KLT). One advantage of the present invention is that the amount of side information is greatly reduced, consisting of only three corners. The KLT requires the coefficients of the block correlation matrix as side information, and therefore requires significantly more data. In addition, the techniques disclosed herein allow for adjustments (or fine-tuning) of the rotation to reduce transition artifacts when proceeding to the next processing block. This improves the compression quality of subsequent perceptual coding.

Table 1 provides a direct comparison between aDSHT and KLT. Despite some similarities, aDSHT offers significant advantages over KLT.

Table 1 Comparison of aDSHT and KLT

While there have been shown, described, and pointed out the novel features underlying the preferred embodiments of the present invention, it will be understood that various omissions, substitutions, and changes may be made by those skilled in the art in the described apparatus and methods, in the form and details of the disclosed devices, and in their operation, without departing from the spirit of the invention. It is apparent that all combinations of elements intended to perform substantially the same function in substantially the same manner to achieve the same results are within the scope of the present invention. Substitution of elements from one described embodiment to another is fully contemplated and intended.

It will be understood that the present invention has been described by way of example only and modifications of detail may be made without departing from the scope of the invention.

Each feature disclosed in the description and (where appropriate) the claims and drawings may be provided independently or in any appropriate combination.

Features may be implemented as hardware, software or a combination of both where appropriate.Connections may be implemented as wireless connections or wired (not necessarily direct or dedicated) connections where applicable.

Reference signs appearing in the claims are by way of example only and shall have no limiting effect on the scope of the claims.

Cited references

[1] T.D.Abhayapala. Generalized framework for spherical microphone arrays: Spatial and frequency decomposition. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Volume X (Accepted), pp. 196-2008, Las Vegas, USA, April 2008.

[2] James R. Driscoll and Dennis M. Healy Jr.. Computing fourier transforms and convolutions on the 2-sphere. Advances in Applied Mathematics, 15:202-250, 1994.

[3]Fliege. Integration nodes for the sphere, http://www.personal.soton.ac.uk/jf1w07/nodes/nodes.html

[4] Fliege and Ulrike Maier. A two-stage approach for computingcubature formulae for the sphere. Technical report, Fachbereich Mathematik, University of Dortmund, 1999.

[5] R.H.Hardin and N.J.A.Sloane. Webpage: Spherical designs, spherical t-designs. http://www2.research.att.com/-njas/sphdesigns

[6] R.H. Hardin and N.J.A. Sloane. McLaren's improved snub cube and other new spherical designs in three dimensions. Discrete and Computational Geometry, 15: 429-441, 1996.

[7] Erik Hellerud, Ian Burnett, Audun Solvang, and U. Peter Svensson. Encoding higher order ambisonics with AAC. 124th AES Convention, Amsterdam, May 2008.

[8]Peter Jax, Jan-Mark Batke, Johannes Boehm, and Sven Kordon. Perceptual coding of HOA signals in spatial domain. European patent application EP2469741A1 (PD100051).

[9]Boaz Rafaely. Plane-wave decomposition of the sound field on asphere by spherical convolution. J. Acoust. Soc. Am., 4(116): 2149-2157, October 2004.

[10]Earl G.Williams. Fourier Acoustics, Applied Mathematical Sciences Volume 93. Academic Press, 1999.

Claims (5)

1. A method for decoding high-order high-fidelity stereo HOA audio signals, the method comprising: The HOA audio signal is decompressed based on perceptual decoding to at least determine the HOA representation corresponding to the HOA audio signal, which represents the channel after perceptual decoding; The rotation transformation is determined by rotating the default spherical sample grid of the adaptive DSHT based on the received channel rotation information; and The HOA representation of the rotation is determined by multiplying the transformation of the rotation with the HOA representation. 2. An apparatus for decoding high-order high-fidelity stereo HOA audio signals, the apparatus comprising: Decoder, the decoder is configured to: The HOA audio signal is decompressed based on perceptual decoding to at least determine the HOA representation corresponding to the HOA audio signal, which represents the channel after perceptual decoding; The rotation transformation is determined by rotating the default spherical sample grid of the adaptive DSHT based on the received channel rotation information; and The HOA representation of the rotation is determined by multiplying the transformation of the rotation with the HOA representation. 3. A non-transitory computer-readable medium containing instructions that, when executed by a processor, perform the method of claim 1. 4. An apparatus comprising: One or more processors, One or more non-transitory computer-readable storage media having instructions stored thereon that, when executed by the one or more processors, cause the device to perform the method as described in claim 1. 5. An apparatus comprising components for performing the method of claim 1.