HK1238786B - Method and apparatus for compressing and decompressing a higher order ambisonics signal representation - Google Patents

Description

Method and apparatus for compressing and decompressing high-order Ambisonics signal representation

The present application is a divisional application of the invention patent application with application number 201380025029.9, application date May 6, 2013, and invention name “Method and device for compressing and decompressing high-order ambisonics signal representation”.

Technical Field

The present invention relates to a method and apparatus for compressing and decompressing a Higher Order Ambisonics signal representation, wherein directional and ambient components are processed differently.

Background Art

Higher-order Ambisonics (HOA) offers the advantage of capturing the complete sound field around a specific location in three-dimensional space, known as the "sweet spot." Unlike channel-based technologies like stereo or surround sound, this HOA representation is independent of a specific loudspeaker configuration. However, this flexibility comes at the expense of the decoding processing required to play back the HOA representation on a specific loudspeaker configuration.

HOA is based on describing the complex amplitude of the air pressure at a position x near the desired listener position using a truncated spherical harmonics (SH) expansion of a number k of individual angular waves, where, without loss of generality, the desired listener position can be assumed to be the origin of the spherical coordinate system. The spatial resolution of this representation increases with the maximum order N of the expansion. Unfortunately, the number of expansion coefficients O grows quadratically with the order N, i.e., O = (N + 1) ² . For example, a typical HOA representation using order N = 4 requires O = 25 HOA coefficients. Given the desired sampling rate f _S and the number of bits per sample N _b , the total bit rate for transmitting the HOA signal representation is determined as O·f _S ·N _b . Transmission of an HOA signal representation of order N = 4, using N _b = 16 bits per sample and a sampling rate of f _S = 48 kHz, results in a bit rate of 19.2 MBits/s. Therefore, compressing the HOA signal representation is highly desirable.

An overview of existing spatial audio compression methods can be found in patent application EP 10306472.1 or in “Multichannel Audio Coding Based on Analysis by Synthesis” by I. Elfitri, B. Günel, A.M. Kondoz, Proceedings of the IEEE, Vol. 99, No. 4, pp. 657-670, April 2011.

The following technology is more relevant to the present invention.

B-format signals (equivalent to a first-order Ambisonics representation) can be compressed using Directional Audio Coding (DirAC), as described by V. Pulkki in "Spatial Sound Reproduction with Directional Audio Coding" (Journal of Audio Eng. Society, Vol. 55(6), pp. 503-516, 2007). In one version proposed for electronic conferencing applications, the B-format signal is encoded into a single omnidirectional signal together with side information in the form of a single direction and a diffuseness parameter for each frequency band. However, the resulting significant reduction in data rate comes at the expense of a lower signal quality upon reproduction. In addition, DirAC is limited to compression of a first-order Ambisonics representation, which suffers from a very low spatial resolution.

There are relatively few known methods for compressing HOA representations with N > 1. One approach involves directly encoding individual HOA coefficient sequences using the perceptual Advanced Audio Coding (AAC) codec, see E. Hellerud, I. Burnett, A. Solvang, and U. Peter Svensson, “Encoding Higher Order Ambisonics with AAC” (124th AES Convention, Amsterdam, 2008). However, an inherent problem with this approach is the perceptual coding of signals that will never be heard. The reconstructed playback signal is typically obtained by a weighted sum of the HOA coefficient sequences. This explains the high probability of unmasking perceptual coding noise when the decompressed HOA representation is presented on a specific loudspeaker configuration. In more technical terms, the main issue with unmasking perceptual coding noise is the high degree of cross-correlation between the individual HOA coefficient sequences. Because the encoded noise signals in the individual HOA coefficient sequences are generally uncorrelated, overlapping structures of perceptual coding noise can occur, while HOA coefficient sequences unrelated to the noise are eliminated at the overlap. Another issue is that the cross-correlation reduces the efficiency of the perceptual coder.

In order to minimize the extent of these effects, EP 10306472.1 proposes transforming the HOA representation into an equivalent representation in the spatial domain before perceptual coding. The spatial domain signal corresponds to a conventional directional signal and would correspond to a loudspeaker signal if the loudspeakers were placed in exactly the same directions as those assumed for the spatial domain transform.

The transformation to the spatial domain reduces the cross-correlation between the individual spatial domain signals. However, the cross-correlation is not completely eliminated. An example of relatively high cross-correlation is a directional signal whose direction falls between adjacent directions covered by the spatial domain signal.

Another disadvantage of EP 10306472.1 and the aforementioned paper by Hellerud et al. is that the number of perceptually coded signals is (N+1) ² , where N is the order of the HOA representation. Therefore, the data rate of the compressed HOA representation grows quadratically with the Ambisonics order.

The compression process of the present invention decomposes the HOA sound field representation into a directional component and an ambient component. Specifically for computing the directional sound field component, a new process is described below for estimating several dominant sound directions.

Regarding existing methods for direction estimation based on Ambisonics, the aforementioned paper by Pulkki describes a method combined with DirAC coding for estimating direction based on a B-format sound field representation. The direction is obtained from the mean intensity vector, which indicates the direction of the sound field energy flow. An alternative based on the B-format is proposed in D. Levin, S. Gannot, and E.A.P. Habets, "Direction-of-Arrival Estimation using Acoustic Vector Sensors in the Presence of Noise" (IEEE Proc. of the ICASSP, pp. 105-108, 2011). Direction estimation is performed iteratively by searching for the direction that provides the maximum energy to the beamformer output signal directed toward that direction.

However, for direction estimation, both methods are constrained to the B-format, which suffers from relatively low spatial resolution.Another drawback is that the estimation is restricted to only a single main direction.

The HOA representation provides improved spatial resolution, allowing improved estimation of several main directions. Existing methods for estimating several directions based on the HOA sound field representation are quite rare. A method based on compressed sensing was proposed in "The Application of Compressive Sampling to the Analysis and Synthesis of Spatial Sound Fields" by N. Epain, C. Jin, A. van Schaik (127th Convention of the Audio Eng. Soc., New York, 2009) and in "Time Domain Reconstruction of Spatial Sound Fields Using Compressed Sensing" by A. Wabnitz, N. Epain, A. van Schaik, C Jin (IEEE Proc. of the ICASSP, pp. 465-468, 2011). The main idea is to assume that the sound field is spatially sparse, that is, it consists of only a small number of directional signals. After distributing a large number of test directions on the sphere, an optimization algorithm is used to find as few test directions and corresponding directional signals as possible so that they are well described by the given HOA representation. This method provides an improved spatial resolution compared to that actually provided by the given HOA representation, since it avoids the spatial dispersion resulting from the finite order of the given HOA representation. However, the performance of the algorithm is highly dependent on whether the sparsity assumption is met. In particular, the method will fail if the sound field includes any small additional ambient components, or if the HOA representation is affected by noise that would be present when calculated from multi-channel recordings.

Another more intuitive method is to transform the given HOA representation into the spatial domain described in "Plane-wave decomposition of the sound field on a sphere by spherical convolution" (J. Acoust. Soc. Am., Vol. 4, No. 116, pp. 2149-2157, October 2004) by B. Rafaely, and then search for the maximum value in the directional power. The disadvantage of this method is that the presence of an ambient component will result in an ambiguous directional power distribution and will result in a shift in the maximum value of the directional power compared to when no ambient component is present.

Summary of the Invention

The problem to be solved by the present invention is to provide a compression of HOA signals, whereby the high spatial resolution represented by the HOA signals is still maintained. This problem is solved by the methods described in claims 1 and 2. Apparatuses utilizing these methods are disclosed in claims 3 and 4.

The present invention addresses the compression of high-order Ambisonics (HOA) representations of sound fields. In this application, the term "HOA" refers to both the high-order Ambisonics representation and the corresponding encoded or represented audio signal. The dominant sound direction is estimated, and the HOA signal representation is decomposed into several dominant directional signals and associated directional information in the time domain, as well as an ambiance component in the HOA domain. The ambiance component is then compressed by reducing its order. Following this decomposition, the reduced-order ambiance HOA component is transformed into the spatial domain and perceptually encoded along with the directional signal.

At the receiver or decoder side, the coded directional signal and the coded ambience component after order reduction are perceptually decompressed. The perceptually decompressed ambience signal is transformed into a reduced-order HOA domain representation, followed by order expansion. The overall HOA representation is reconstructed from the directional signal and the corresponding directional information, as well as from the original-order ambience HOA component.

Advantageously, the ambient sound field components can be represented with sufficient accuracy by an HOA representation having a lower order than the original, and the extraction of the main directional signal ensures that a high spatial resolution is still obtained after compression and decompression.

In principle, the method of the invention is suitable for compressing a higher-order Ambisonics (HOA) signal representation, said method comprising the following steps:

- estimating a main direction, wherein the main direction estimate depends on the directional power distribution of the main HOA components in energy;

- decomposing or decoding the HOA signal representation into a plurality of main directional signals and associated directional information in the time domain and a residual ambience component in the HOA domain, wherein the residual ambience component represents the difference between the HOA signal representation and a representation of the main directional signal;

- compressing the residual ambience component by reducing the order of the residual ambience component compared to its original order;

- transforming the reduced-order residual ambient HOA component into the spatial domain;

-Perceive coding the main directional signal and the transformed residual ambient HOA component.

In principle, the method of the invention is suitable for decompressing a Higher Order Ambisonics HOA signal representation that has been compressed by:

- estimating a main direction, wherein the main direction estimate depends on the directional power distribution of the main HOA components in energy;

- decomposing or decoding the HOA signal representation into a plurality of main directional signals and associated directional information in the time domain and a residual ambience component in the HOA domain, wherein the residual ambience component represents the difference between the HOA signal representation and a representation of the main directional signal;

- compressing the residual ambience component by reducing the order of the residual ambience component compared to its original order;

- transforming the reduced-order residual ambience component into the spatial domain;

-perceptually encoding the main direction signal and the transformed residual environment HOA component;

The method comprises the following steps:

-perceptually decoding the perceptually coded main directional signal and the perceptually coded transformed residual ambient HOA component;

- inverse transform the perceptually decoded transformed residual ambient HOA components to obtain HOA domain representation;

- performing order expansion on the inverse transformed residual ambient HOA component to create the ambient HOA component of the original order;

- composing the perceptually decoded primary directional signal, the directional information and the original-order expanded ambient HOA components in order to obtain an HOA signal representation.

In principle, the device of the invention is suitable for compressing a higher-order Ambisonics (HOA) signal representation, said device comprising:

- means adapted to estimate a main direction, wherein said main direction estimate depends on the directional power distribution of the main HOA components in energy;

- means adapted to decompose or decode the HOA signal representation into a number of main directional signals and associated directional information in the time domain and a residual ambience component in the HOA domain, wherein the residual ambience component represents the difference between the HOA signal representation and a representation of the main directional signal;

- means adapted to compress said residual ambience component by reducing the order of said residual ambience component compared to its original order;

- means adapted to transform said residual ambience component of reduced order into the spatial domain;

- means adapted to perceptually encode said main directional signal and said transformed residual ambient HOA component.

In principle, the device of the invention is suitable for decompressing a Higher Order Ambisonics HOA signal representation that has been compressed by:

- estimating a main direction, wherein the main direction estimate depends on the directional power distribution of the main HOA components in energy;

- decomposing or decoding the HOA signal representation into a plurality of main directional signals and associated directional information in the time domain and a residual ambience component in the HOA domain, wherein the residual ambience component represents the difference between the HOA signal representation and a representation of the main directional signal;

- compressing the residual ambience component by reducing the order of the residual ambience component compared to its original order;

- transforming the reduced-order residual ambience component into the spatial domain;

-perceptually encoding the main direction signal and the transformed residual environment HOA component;

The device comprises:

- means adapted to perceptually decode the perceptually coded main directional signal and the perceptually coded transformed residual ambient HOA component;

- means adapted to inversely transform the perceptually decoded transformed residual ambient HOA component in order to obtain an HOA domain representation;

- means adapted to perform order expansion on said inverse transformed residual ambience HOA components in order to create ambience HOA components of original order;

- means adapted to compose said perceptually decoded main directional signal, said directional information and said original-order expanded ambient HOA components in order to obtain an HOA signal representation.

Advantageous further embodiments of the invention are disclosed in the respective dependent claims.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the present invention are described with reference to the accompanying drawings, in which:

FIG1 shows the normalized dispersion function v _N (Θ) for different Ambisonics orders N and angles Θ∈[0,π];

FIG2 is a block diagram of a compression process according to the present invention;

FIG3 is a block diagram of a decompression process according to the present invention.

DETAILED DESCRIPTION

The Ambisonics signal uses a spherical harmonics (SH) expansion to describe the sound field within the passive region. This flexibility can be attributed to the physical property that the temporal and spatial behavior of the sound pressure is essentially determined by the wave equation.

Wave equation and spherical harmonic expansion

For a more detailed description of Ambisonics, a spherical coordinate system is assumed below, in which a point in space x = (r, θ, φ) T is represented by a radius r > 0 (i.e., the distance from the coordinate origin), a tilt angle θ∈[0,π] measured from the polar axis z, and an ^azimuth angle φ∈[0,2π] measured from the x-axis in the x=y plane. In this spherical coordinate system, the wave equation for the sound pressure p(t,x) (where t represents time) in a connected passive region is given by Earl G. Williams' textbook "Fourier Acoustics" (Applied Mathematical Sciences, Vol. 93, Academic Press, 1999):

where _cs indicates the speed of sound. Therefore, the Fourier transform of the sound pressure with respect to time is

Here, i represents the imaginary unit, which can be expanded into the SH series according to Williams' textbook:

It should be noted that this expansion is valid for all points x within the connected, passive region (which corresponds to the region of convergence of the sequence).

In equation (4), k represents the number of angular waves defined by:

and denotes the SH expansion coefficients, which depend only on the product kr.

In addition, here is the SH function of order n and degree m:

where represents the associated Legendre function and (·)! represents the factorial.

The associated Legendre functions for non-negative power exponents m are defined by the Legendre polynomials _Pn (x) as follows:

Where m≥0. (7)

For negative power exponents, that is, m < 0, the associated Legendre function is defined as follows:

Where m＜0. (8)

Then the Legendre polynomial P _n (x)(n≥0) can be defined using Rodrigues' formula as:

In the prior art, for example in M. Poletti's "Unified Description of Ambisonics using Real and Complex Spherical Harmonics" (Proceedings of the Ambisonics Symposium 2009, June 25-27, 2009, Graz, Austria), there is also a definition of the SH function, which results from equation (6) by the factor (-1) ^m for the negative power index m.

Alternatively, the Fourier transform of the sound pressure with respect to time can be expressed using the real SH function as

In the literature, there are various definitions of real SH functions (see, for example, the above-mentioned paper by Poletti). One possible definition used in this document is given by:

Where (·) ^* represents the complex conjugate. An alternative representation is obtained by inserting equation (6) into equation (11):

in,

Although real SH functions are real-valued for every definition, in general this is not true for the corresponding expansion coefficients.

Complex SH functions are related to the following real SH functions:

The complex SH functions and the real SH functions with the direction vector Ω:=(θ, φ) ^T form an orthogonal basis of square-integrable complex-valued functions on the unit sphere in three-dimensional space, and thus satisfy the following condition:

Here, δ represents the Kronecker delta function. Using equation (15) and the definition of real spherical harmonics in equation (11), a second result can be obtained.

Internal issues and Ambisonics coefficients

The goal of Ambisonics is to represent the sound field near the origin. Without loss of generality, this region of interest is assumed to be a sphere of radius R centered at the origin, specified by the set {x|0≤r≤R}. A key assumption about this representation is that the sphere contains no sound sources. Finding a representation of the sound field within this sphere is known as the "interior problem"; see the aforementioned Williams textbook.

It can be shown that, with respect to this internal problem, the coefficients of the SH function expansion can be expressed as

where j _n (.) denotes a first-order spherical Bessel function. According to equation (17), it is satisfied that the complete information about the sound field is contained in the coefficients called Ambisonics coefficients.

Similarly, the coefficients of the real SH function expansion can be factored into

where the coefficients are called Ambisonics coefficients with respect to the expansion of the SH function using real values. They are also related to by:

Plane wave decomposition

The sound field within a sound-passive sphere centered at the origin can be represented by the superposition of an infinite number of plane waves of varying angular wavenumbers k impinging upon the sphere from all possible directions, see the aforementioned Rafely's "Plane-wave decomposition..." paper. Assuming that the complex amplitude of a plane wave with angular wavenumber k coming from the direction Ω ₀ is given by D(k, Ω ₀ ), the corresponding Ambisonics coefficients with respect to the real SH function expansion can be shown in a similar manner using equations (11) and (19) as follows:

Therefore, the Ambisonics coefficients for the sound field resulting from the superposition of an infinite number of plane waves with a number k of angular waves are obtained from the integration of equation (20) over all possible directions:

The function D(k,Ω) is called the "amplitude density" and is assumed to be square-integrable on the unit sphere. It can be expanded into a series of real SH functions as follows

where the expansion coefficient is equal to the integral appearing in equation (22), i.e.

By inserting equation (24) into equation (22), it can be seen that the Ambisonics coefficients are scaled versions of the expansion coefficients, i.e.

When applying the inverse Fourier transform with respect to time to the scaled Ambisonics coefficients and the amplitude density function D(k,Ω), the corresponding time-domain quantity is obtained.

Then, in the time domain, equation (24) can be formulated as

The time domain direction signal d(t,Ω) can be expressed by the real SH function expansion according to the following formula

Using the fact that the SH function is real-valued, its complex conjugate can be expressed as

Assuming that the time domain signal d(t,Ω) is real-valued, that is, d(t,Ω)=d ^* (t,Ω), according to the comparison between equation (29) and equation (30), it can be concluded that the coefficient is real-valued in this case, that is,

In the following, the coefficients are referred to as scaled time domain Ambisonics coefficients.

In the following, it is also assumed that the sound field representation is given by these coefficients which will be described in more detail in the following section on processing compression.

Note that the time domain HOA representation by the coefficients used for the processing according to the present invention is equivalent to the corresponding frequency domain HOA representation. Therefore, the compression and decompression can be equivalently implemented in the frequency domain with minor corresponding modifications to the equations.

Finite-order spatial resolution

In practice, only a limited number of Ambisonics coefficients of order n ≤ N are used to describe the sound field near the origin. The magnitude density function computed from the truncated SH function series introduces a spatial dispersion relative to the true magnitude density function D(k,Ω) according to

See the above-mentioned "Plane-wave decomposition..." paper. This can be done by using equation (31)

This is achieved by computing the amplitude density function for a single plane wave coming from the direction Ω ₀ :

in

where Θ represents the angle between two vectors pointing in the direction Ω and Ω ₀ that satisfy the following properties:

cosΘ＝cosθcosθ ₀ +cos(φ-φ ₀ )sinθsinθ ₀ (39)

In equation (34), the plane wave Ambisonics coefficients given in equation (20) are used, while in equations (35) and (36) some mathematical theory is used, see the above-mentioned "Plane-wave decomposition..." paper. The properties in equation (33) can be shown using equation (14).

Compare equation (37) with the true amplitude density function

where δ(·) denotes the Dirac delta function, the spatial dispersion becomes apparent from replacing the scaled Dirac delta function with the dispersion function v _N (Θ) (which is shown in FIG1 for different Ambisonics orders N and angles Θ∈[0,π] after being normalized to its maximum value).

Since the first zero of _vN (Θ) is located approximately at for N≥4 (see the above-mentioned "Plane-wave decomposition ..." paper), the dispersion effect decreases (and thus the spatial resolution increases) with increasing the Ambisonics order N.

As N → ∞, the dispersion function v _N (Θ) converges to the scaled Dirac delta function. This can be seen in the following case: The complete relation for the Legendre polynomials

Used together with Eq. (35) to express the limit of v _N (Θ) with respect to N → ∞ as

In passing

When defining a vector of real SH functions of order n≤N, where O=(N+1) ² and (.) ^T represents the transpose, Equation (37) compared with Equation (33) shows that the deviation function can be expressed by the scalar product of two real SH vectors as

v _N (Θ)＝S ^T (Ω)S(Ω ₀ ) (47)

In the time domain, the dispersion can be equivalently expressed as

sampling

For some applications, it is desirable to determine the scaled time-domain Ambisonics coefficients from samples of the time-domain amplitude density function d(t,Ω) in a finite number J of discrete directions _Ωj . The integral in equation (28) is then approximated by a finite summation according to B. Rafaely, "Analysis and Design of Spherical Microphone Arrays," IEEE Transactions on Speech and Audio Processing, Vol. 13, No. 1, pp. 135-143, January 2005:

where _gj represents some appropriately chosen sampling weights. In contrast to the "Analysis and Design..." paper, approximation (50) refers to the use of a time domain representation of real SH functions rather than a frequency domain representation of complex SH functions. A necessary condition for approximation (50) to be accurate is that the amplitude density is of finite harmonic order N, meaning

For n＞N. (51)

If this condition is not met, the approximation (50) is affected by spatial aliasing errors, see B. Rafaely, “Spatial Aliasing in Spherical Microphone Arrays” (IEEE Transactions on Signal Processing, Vol. 55, No. 3, pp. 1003-1010, March 2007).

The second necessary condition requires that the sampling points Ω _j and the corresponding weights satisfy the corresponding conditions given in the "Analysis and Design..." paper:

For m, m′≤N (52)

Conditions (51) and (52) combined are sufficient for accurate sampling.

The sampling conditions (52) consist of a set of linear equations that can be concisely formulated using a single matrix equation as

ΨGΨ ^H ＝I (53)

Where Ψ represents the mode matrix defined by

And G represents a matrix with weights on its diagonal, that is

G:=diag(g ₁ ,,g _J ) (55)

It can be seen from equation (53) that the necessary condition for satisfying equation (52) is that the number of sampling points J satisfies J ≥ O. The values of the time domain amplitude density at J sampling points are aggregated into the following vector

w(t): =(D(t, Ω ₁ ),..., D(t, Ω _J )) (56)

And the vector of scaled time domain Ambisonics coefficients is defined by

The two vectors are related by the SH function expansion (29). This relationship provides the following system of linear equations:

w(t)＝Ψ ^H c(t) (58)

Using the introduced vector notation, the computation of the scaled time-domain Ambisonics coefficients from the values of the time-domain amplitude density function samples can be written as:

c(t)≈ΨGw(t) (59)

Given a fixed Ambisonics order N, it is often not possible to compute J ≥ O number of sampling points Ω _j and the corresponding weights such that the sampling condition Eq. (52) is satisfied. However, if the sampling points are chosen so as to approximate the sampling condition well, the rank of the pattern matrix Ψ is O and its condition number is low. In this case, there exists a pseudo-inverse of the pattern matrix Ψ

Ψ ⁺ :=(ΨΨ ^H ) ^-1 ΨΨ ⁺ (60)

And a reasonable approximation from the vector of time-domain amplitude density function samples to the scaled time-domain Ambisonics coefficient vector c(t) is given by

c(t)≈Ψ ⁺ w(t) (61)

If J = O and the pattern matrix is of rank O, then its pseudoinverse is identical to its inverse, since

Ψ ⁺ =(ΨΨ ^H ) ^-1 Ψ＝Ψ ^-H Ψ ^-1 Ψ＝Ψ ^-H (62)

If the sampling condition equation (52) is additionally satisfied, then

Ψ ^-H =ΨG (63)

And the two approximations (59) and (61) are equivalent and exact.

The vector w(t) can be interpreted as a vector of spatial-time domain signals. The transformation from the HOA domain to the spatial domain can be performed, for example, using equation (58). This transformation is referred to in this application as the "spherical harmonic transform" (SHT) and is used when the reduced-order ambient HOA components are transformed into the spatial domain. It is implicitly assumed that the spatial sampling points _Ωj of the SHT approximately satisfy the sampling conditions in equation (52) with J=0.

Under these assumptions, the SHT matrix satisfies In cases where the absolute scaling of the SHT is unimportant, the constant can be ignored.

compression

The present invention relates to compression of a given HOA signal representation. As described above, the HOA representation is decomposed into a predefined number of main directional signals in the time domain and an ambient component in the HOA domain, followed by compressing the HOA representation of the ambient component by reducing its order. This operation exploits the assumption, supported by listening tests, that ambient sound field components can be represented with sufficient accuracy by HOA representations having a low order. Extraction of the main directional signals ensures that high spatial resolution is maintained after compression and corresponding decompression.

After decomposition, the reduced-order ambient HOA components are transformed into the spatial domain and perceptually encoded together with the directional signal as described in the Exemplary embodiments section of patent application EP10306472.1.

The compression process comprises two successive steps which are illustrated in Figure 2. The exact definition of the individual signals is described in the Details of Compression section below.

In a first step or stage shown in FIG2 a , the main direction is estimated in a main direction estimator 22 and a decomposition of the Ambisonics signal C(l) into a directional component and a residual or ambience component is performed, where l represents a frame index. The directional components are calculated in a directional signal calculation step or stage 23, whereby the Ambisonics representation is converted into a time domain signal represented by a set of D conventional directional signals x(l) with corresponding directions. The ambience component of the residual is calculated in an ambience HOA component calculation step or stage 24 and represented as HOA domain coefficients _CA (l).

In the second step shown in FIG2 b , perceptual coding is performed on the directional signal X(l) and the ambient HOA component _CA (l) as follows:

The normal time-domain direction signal X(l) may be compressed separately in the perceptual encoder 27 using any known perceptual compression technique.

Compression of the ambient HOA domain components _CA (l) is performed in two sub-steps or stages. A first sub-step or stage 25 performs a reduction of the original Ambisonics order N to _NRED , for example _NRED = 2, resulting in the ambient HOA components CA _,RED (l). Here, the assumption is made that the ambient sound field components can be represented sufficiently accurately by HOA components with low order. A second sub-step or stage 26 is based on the _compression described in patent application EP 10306472.1. By applying a spherical harmonics transformation, ^{the 2} HOA signals CA _{,RED (} l) of the ambient sound field components calculated in sub-step/stage 25 are transformed into _0RED equivalent signals W _A,RED (l) in the spatial domain, resulting in a conventional time domain signal that can be input to a set of parallel perceptual codecs 27. Any known perceptual coding or compression technique can be applied. The coded directional signal and the coded spatial domain signal with reduced order are output and can be transmitted or stored.

Advantageously, the perceptual compression of all time domain signals X(l) and _WA,RED (l) may be performed jointly in the perceptual encoder 27 in order to improve the overall coding efficiency by exploiting possible remaining inter-channel correlations.

Decompression

The decompression process of a received or replayed signal is illustrated in Figure 3. Like the compression process, it comprises two successive steps.

In a first step or stage shown in FIG3 a , perceptual decoding or decompression of the encoded directional signal and the order-reduced encoded spatial domain signal is performed in perceptual decoding 31 , where is a representation component and represents the ambient HOA component. The perceptually decoded or decompressed spatial domain signal is transformed into an HOA domain representation of order N _RED via an inverse spherical harmonics transform in an inverse spherical harmonics transformer 32 . Thereafter, in an order expansion step or stage 33 , an appropriate HOA representation of order N is estimated from by order expansion.

In a second step or stage shown in FIG3 b , the total HOA representation is reconstructed in the HOA signal assembler 34 from the directional signal and the corresponding directional information and from the ambient HOA components of the original order

Achievable data rates decrease

The problem addressed by the present invention is to significantly reduce the data rate compared to existing compression methods for HOA representations. The achievable compression ratios compared to uncompressed HOA representations are discussed below. The compression ratios are derived from a comparison of the data rate required to transmit an uncompressed HOA signal C(l) of order N with the data rate required to transmit a compressed signal representation consisting of D perceptually coded directional signals and the corresponding directions and N _RED perceptually coded spatial domain signals W _A,RED (l) representing ambient HOA components.

To transmit the uncompressed HOA signal C(l), a data rate of O· _fS · _Nb is required. Conversely, transmitting the D perceptually coded directional signals X(l) requires a data rate of D·fb _,COD , where fb _,COD denotes the bit rate of the perceptually coded signal. Similarly, transmitting the _NRED perceptually coded spatial domain signals W _A,RED (l) requires a bit rate of _ORED ·fb _,COD . It is assumed that the directions are calculated based on a rate much lower than the sampling rate _fS , i.e., that they are fixed for the duration of a signal frame consisting of B samples, e.g., B=1200 for a sampling rate of _fS =48kHz, and the corresponding data rate contribution can be ignored for the calculation of the total data rate of the compressed HOA signal.

Therefore, transmitting the compressed representation requires a data rate of approximately (D + _ORED ) · _fb,COD . Therefore, the compression ratio _rCOMPR is

For example, compressing an HOA representation of order N=4 with sampling rate _fs = 48 kHz and _Nb = 16 bits per sample to a representation with D = 3 principal directions using a reduced HOA order _NRED = 2 and a bit rate of _rCOMPR≈ 25 will result in a compression ratio of rCOMPR≈ 25. Transmitting the compressed representation requires a data rate of approximately .

Reduced probability of unmasking coding noise

As described in the background, the perceptual compression of spatial domain signals described in patent application EP 10306472.1 is affected by residual cross-correlation between the signals, which can lead to unmasking of perceptual coding noise. According to the present invention, the main directional signal is first extracted from the HOA sound field representation before being perceptually coded. This means that when forming the HOA representation, after perceptual decoding, the coding noise has exactly the same spatial directionality as the directional signal. Specifically, the effect of the coding noise and the directional signal on any arbitrary direction is deterministically described by a spatial dispersion function interpreted in a spatial resolution portion with a finite order. In other words, at any moment, the HOA coefficient vector representing the coding noise is exactly a multiple of the HOA coefficient vector representing the directional signal. Therefore, any weighted sum of the noise HOA coefficients will not result in any unmasking of the perceptual coding noise.

Additionally, the reduced-order ambient components are processed as proposed in EP 10306472.1, but since the spatial domain signals of the ambient components have a rather low correlation between each other per definition, the probability of perceptual noise being unmasked is low.

Improved direction estimation

The present invention's direction estimation relies on the directional power distribution of the dominant HOA component in terms of energy. The directional power distribution is calculated from a rank-reduced correlation matrix of the HOA representation (obtained by eigenvalue decomposition of the correlation matrix of the HOA representation). This provides the advantage of greater accuracy compared to the direction estimation used in the aforementioned "Plane-wave decomposition..." paper, as focusing on the dominant HOA component in terms of energy rather than using the full HOA representation for the direction estimation reduces spatial ambiguity in the directional power distribution.

Compared to the directional estimates proposed in the aforementioned "The Application of Compressive Sampling to the Analysis and Synthesis of Spatial Sound Fields" and "Time Domain Reconstruction of Spatial Sound Fields Using Compressed Sensing" papers, this offers the advantage of being more robust. The reason is that the decomposition of the HOA representation into directional and ambient components is almost never perfectly achieved, leaving a small amount of ambient component in the directional component. However, compressed sampling methods like those in these two papers cannot provide reasonable directional estimates due to their high sensitivity to the presence of ambient signals.

Advantageously, the direction of the present invention is not expected to be affected by this problem.

HOA represents an alternative application of decomposition

According to the above-mentioned paper "Spatial Sound Reproduction with Diretional Audio Coding" by Pulkki, the decomposition of the HOA representation into several directional signals with relevant directional information and an ambient component in the HOA domain can be used for signal adaptive DirAC-like presentation of the HOA representation.

Each HOA component can be rendered differently because the physical characteristics of the two components are different. For example, a signal panning technique such as vector-based amplitude panning (VBAP) can be used to render a directional signal for a loudspeaker. See V. Pulkki, "Virtual Sound Source Positioning Using Vector Base Amplitude Panning" (Journal of Audio Eng. Society, Vol. 45, No. 6, pp. 456-466, 1997). Standard known HOA rendering techniques can be used to render the ambient HOA component.

Such a rendering is not limited to Ambisonics representations of order "1" and can therefore be seen as an extension of the DirAC-like rendering to HOA representations of order N>1.

The estimation of several directions from the HOA signal representation can be used for any relevant type of sound field analysis.

The following sections describe the signal processing steps in more detail.

compression

Definition of input format

As input, the scaled time-domain HOA coefficients defined in equation (26) are assumed to be sampled at rate . Define vector c(j) to consist of all coefficients belonging to sampling time t= _jTS , according to:

Framing

In a framing step or stage 21, the incoming vector c(j) of scaled HOA coefficients is framed into non-overlapping frames of length B according to:

Assuming a sampling rate of _fs = 48 kHz, corresponding to a frame duration of 25 ms, a suitable frame length is B = 1200 samples.

Estimation of main direction

For the estimation of the main direction, the following correlation matrix is calculated

The summation over the current frame l and the L-1 previous frames indicates that the direction analysis is based on a long overlapping set of frames with L·B samples, that is, for each current frame, the content of the neighboring frames is considered. This helps the stability of the direction analysis for two reasons: longer frames lead to a larger number of observations, and the direction estimate is smoothed due to the overlapping frames.

Assuming _fs = 48 kHz and B = 1200, a reasonable value for L is 4, corresponding to an overall frame duration of 100 ms.

Next, the eigenvalue decomposition of the correlation matrix B(l) is determined according to the following formula

B(l)＝V(l)A(l)V ^T (l) (68)

The matrix V(l) is composed of the eigenvectors _vi (l), 1≤i≤O, as follows

And Λ(l) is a diagonal matrix with corresponding eigenvalues λ _i (l), 1≤i≤O, on its diagonal:

Assume that the eigenvalues are indexed in non-ascending order, that is,

λ ₁ (l)≥λ ₂ (l)≥…≥λ _O (l) (71)

Afterwards, the index set of the main eigenvalues is calculated. One possible way to manage this is to define the desired minimum broadband directional-to-ambient power ratio DAR _MIN and then determine the value such that

and

for

A reasonable choice for DAR _MIN is 15 dB. The number of main eigenvalues is further constrained to be no greater than D, so as to focus on no more than D main directions. This is achieved by replacing the index set with

Next, we can get the rank approximation of B(l) by the following formula:

Of which (74)

This matrix should contain the contributions of the principal direction components to B(l).

Afterwards, calculate the vector

where Ξ represents a pattern matrix for a large number of approximately equally distributed test directions _Ωq :=( _θq , _φq ), 1≤q≤Q, where _θq∈ [0,π] represents the tilt angle θ∈[0,π] measured from the polar axis z, and _φq∈ [-π,π[ represents the azimuth angle measured from the x-axis in the x=y plane.

The pattern matrix Ξ is defined by

Among them, for 1≤q≤Q

The elements in σ ² (l) are approximations of the power of a plane wave corresponding to the main directional signal incident from the direction Ω _q . A theoretical explanation related to this is provided in the following explanation section on the direction search algorithm.

From σ ² ( l ), several main directions for the determination of the directional signal component are calculated, constraining the number of main directions to satisfy in order to ensure a constant data rate. However, if a variable data rate is allowed, the number of main directions can be adapted to the current sound scene.

One possible way to calculate the main directions is to set the first main direction to the one with the maximum power, i.e., where , and assuming that the power maximum is created by the main direction signal, and taking into account the fact that the spatial dispersion of the direction signals is obtained using an HOA representation of finite order N (see, the above-mentioned "Plane-wave decomposition..." paper), it can be concluded that: in the direction field of Ω _CURRDOM,1 (l), power components belonging to the same direction signal should appear. Since the spatial signal dispersion can be represented by a function (see equation (38)), where , represents the angle between Ω _q and Ω _CURRDOM,1 (l), the power belonging to the direction signal decreases according to . Therefore, for the search of the other main directions, it is reasonable to exclude all directions Ω _q in the direction field with Θ _q,1 ≤ Θ _MIN . The distance Θ _MIN can be selected as the first zero of v _N (x) (for N ≥ 4, it is approximately given by ). Then, the second main direction is set to the one with the maximum power in the remaining directions, where the remaining main directions are determined in a similar way.

The number of main directions can be determined by taking into account the power allocated to the individual main directions and searching for a situation where the ratio exceeds the value of the desired direction-to-ambient ratio DAR _MIN . This means that

The overall process for computing all principal directions can be performed as follows:

Next, the direction obtained in the current frame and the direction in the previous frame are smoothed to obtain a smoothed direction. This operation can be divided into two consecutive parts:

(a) Assign the current main direction to the smoothed directions in the previous frame. Determine the assignment function so that the sum of the angles between the assigned directions is

Minimize. Such an assignment problem can be solved using the famous Hungarian algorithm (see HW Kuhn's "The Hungarian method for the assignment problem", Naval research logistics quarterly 2, No. 1-2, pp. 83-97, 1955). The angle between the current direction and the inactive direction in the previous frame (for an explanation of the term "inactive direction", see below) is set to 2Θ _MIN . The effect of this operation is that an attempt is made to assign current directions that are closer than 2Θ _MIN to the previously active direction. If the distance exceeds 2Θ _MIN , it is assumed that the corresponding current direction belongs to a new signal, which means that it is preferably assigned to the previously inactive direction. Note: When a larger waiting time is allowed for the overall compression algorithm, the assignment of successive direction estimates can be performed more robustly. For example, sudden changes of direction can be better identified without mixing them with outliers resulting from estimation errors.

(b) Directions smoothed using the assignments from step (a) Smoothing is based on spherical geometry rather than Euclidean geometry. For each of the current principal directions, smoothing is performed along the minor arc of the great circle spanning two points on the sphere, as specified by the direction θ. Obviously, azimuth and inclination are smoothed independently by computing an exponentially weighted moving average with smoothing factors _αΩ . For inclination, this yields the following smoothing operation:

For azimuth, the smoothing must be modified to get correct smoothing when translating from π-ε (ε>0) to -π and when translating in the opposite direction. This can be taken into account by first computing the differential angle modulo 2π as

It is converted to the interval [-π, π[

The main azimuth angle after smoothing modulo 2π is determined as

And finally converted into a value in the interval [-π, π[

In the case where there is an index set corresponding to the direction in the previous frame that has not yet obtained the assigned current main direction is represented as

Copy the corresponding direction from the previous frame, that is, for

Directions that are not assigned for a predetermined number (L _IA ) of frames are said to be inactive.

After that, the index set of the direction of the activity represented by is calculated. Its cardinality is expressed as

Then, all smoothed directions are concatenated into a single direction matrix as

Calculation of direction signal

The calculation of the directional signal is based on pattern matching. Specifically, a search is performed for directional signals whose HOA representation gives the best approximation to a given HOA signal. Because changes in direction between consecutive frames can lead to discontinuities in the directional signal, an estimate of the directional signal can be calculated for overlapping frames, followed by smoothing the result of consecutive overlapping frames using an appropriate window function. However, this smoothing introduces latency in the individual frames.

The detailed estimation of the direction signal is explained below:

First, the pattern matrix based on the direction of the smoothed activity is calculated according to the following formula

in,

Here, d _ACT,j , 1≤j≤D _ACT (l) represents the index of the direction of the activity.

Next, the matrix X _INST (l) containing the unsmoothed estimates of all directional signals for the (l-1)th and lth frames is calculated:

in,

This is done in two steps. In the first step, the direction signal samples in the rows corresponding to the inactive directions are set to zero, i.e.

if

In the second step, the direction signal samples corresponding to the direction of the activity are obtained by first arranging them in a matrix according to

This matrix is then calculated so that the Euclidean norm of the error

Ξ _ACT (l)X _{INST, ACT} (l)-[C(l-1) C(l)] (97)

Minimize. Its solution is given by

The estimate of the direction signal x _INST,d (l, j) (1≤d≤D) is windowed by an appropriate window function w(j):

x _{INST, WIN, d} (l, j): = x _{INST, d} (l, j)·w (j), 1≤j≤2B (99)

An example of a window function is given by a periodic Hamming window, defined as

Where _Kw represents a scaling factor determined so that the sum of the shifted windows is equal to "1". The smoothed direction signal of the (l-1)th frame is calculated by appropriately overlapping the windowed unsmoothed estimates according to the following formula:

x _d ((l-1)B+j)＝x _{INST, WIN, d} (l-1, B+j)+x _{INST, WIN, d} (l, j) (101)

The samples of all smoothed direction signals for the (l-1)th frame are arranged in the matrix X(l-1) as follows

in,

Calculation of ambient HOA components

The ambient HOA component CA(l-1) is obtained by subtracting the total directional HOA component _CDIR (l-1) from the total HOA representation _C (l-1) according to the following formula:

Wherein, _CDIR (l-1) is determined by the following formula:

Wherein, _ΞDOM (1) represents the mode matrix based on all smooth directions defined by the following formula

Since the calculation of the total directional HOA component is also based on the spatial smoothing of overlapping successive instantaneous total directional HOA components, an ambient HOA component with a latency of a single frame is also obtained.

The order of the ambient HOA component is reduced

It is expressed by the components of _CA (l-1) as

The order reduction is achieved by deleting all HOA coefficients with n>N _RED :

Spherical harmonics transformation of ambient HOA components

The spherical harmonics transformation is performed by multiplying the reduced-order ambient HOA component CA _,RED (l) with the inverse of the mode matrix

in,

Based on the uniform distribution of _ORED , the direction of ΩA _,d

1≤d≤O _RED : W _{A, RED} (l) = (Ξ _A ) ^-1 C _{A, RED} (l) ( ₁₁₁ )

Decompression

Inverse spherical harmonics transform

The perceptually decompressed spatial domain signal is transformed into an HOA domain representation of order N _RED via the inverse spherical harmonics transform by the following formula:

Step expansion

The Ambisonics order represented by the HOA is expanded to N by appending zeros according to

Where O _m×n represents a zero matrix with m rows and n columns.

HOA coefficient composition

The final decompressed HOA coefficient is composed of the sum of the directional and ambient HOA components according to the following formula

At this stage, a latency of a single frame is again introduced to allow the computation of the directional HOA components based on spatial smoothing. Thus, possible undesirable discontinuities in the directional components of the sound field caused by directional changes between consecutive frames are avoided.

To compute the smoothed directional HOA component, two consecutive frames containing estimates of all individual directional signals are concatenated into a single long frame as follows

Each individual signal excerpt contained in the long frame is multiplied by a window function such as equation (100). When the long frame is represented by its components according to the following equation

The window processing operation can be expressed as a formula to calculate the selected information segment after window processing as follows

Finally, the total directional HOA component C _DIR (l-1) is obtained by encoding all windowed directional signal segments into appropriate directions and overlapping them in an overlapping manner:

Explanation of the Direction Search Algorithm

Next, the motivation behind the direction search process described in the main direction estimation section is explained. It is based on some assumptions that are first defined.

Assumptions

The HOA coefficient vector c(j) is usually related to the time-domain amplitude density function d(j,Ω) by

Assume that the HOA coefficient vector c(j) conforms to the following model:

For lB+1≤j≤(l+1)B (120)

The model shows that, on the one hand, the HOA coefficient vector c(j) is created by I main directional source signals x _i (j) (1≤i≤I) from the direction of the lth frame. Specifically, it is assumed that the direction is fixed for the duration of a single frame. It is assumed that the number of main source signals I is significantly smaller than the total number of HOA coefficients O. In addition, it is assumed that the frame length B is significantly larger than O. On the other hand, the vector c(j) is composed of the residual component c _A (j), which can be regarded as representing an ideal isotropic ambient sound field.

It is assumed that the individual HOA coefficient vector components have the following properties:

●Assume that the main source signal has zero mean value, that is

And assume that the main source signals are independent of each other, that is,

where represents the average power of the i-th signal in the l-th frame.

● Assume that the main source signal is independent of the ambient component of the HOA coefficient vector, that is,

● The ambient HOA component vector is assumed to be zero mean and has a covariance matrix

The directional-to-ambient power ratio DAR(l) for each frame l is defined here by

Assume that it is greater than the predefined expected value DAR _MIN , that is

DAR(l)≥DAR _MIN (126)

Explanation of Direction Search

To explain, consider the following case: the correlation matrix B(l) is calculated based only on the samples of the lth frame without considering the samples of the L-1 previous frames (see equation (67)). This operation corresponds to setting L = 1. Therefore, the correlation matrix can be expressed as

By substituting the model assumptions in equation (120) into equation (128), and by using the definitions in equations (122) and (123) and equation (124), the correlation matrix B(l) can be approximated as (129)

According to equation (131), B(l) is approximately composed of two additional components that contribute to the directional and ambient HOA components. Its rank approximation provides an approximation of the directional HOA component, that is,

This is derived from equation (126) regarding the directional to ambient power ratio.

However, it should be emphasized that part of ∑ _A (l) will inevitably leak into ∑ A (l) because ∑ _A (l) generally has full rank, so the columns of the matrix and the subspace spanned by ∑ _A (l) are not orthogonal to each other. By equation (132), the vector σ ² (l) in equation (77) for the main direction search can be expressed as

In equation (135), the following properties of the spherical harmonics shown in equation (47) are used:

S ^T (Ω _q )S(Ω _q′ )＝v _N (∠(Ω _q ,Ω _q′ )) (137)

Equation (136) shows that the components of σ ² (l) are approximations of the power of the signal from the test direction Ω _q (1≤q≤Q).

Claims (13)

1. A method for decompressing a high-order high-fidelity stereo reproduction (HOA) signal representation, the method comprising: Receive encoded direction signals and encoded environmental signals; The encoded direction signal and the encoded environment signal are sensed and decoded to generate the decoded direction signal and the decoded environment signal, respectively. The decoded environmental signal is converted from the spatial domain to its HOA domain representation; and The high-order high-fidelity stereo reproduction (HOA) signal is reconstructed from the HOA domain representation and decoded directional signals of the ambient signal; This transformation includes applying an inverse spatial transformation to the decoded environmental signal. 2. The method according to claim 1, wherein the high-order high-fidelity stereo reproduction (HOA) signal represents a signal having an order greater than 1. 3. The method according to claim 1, wherein the order of the decoded environmental signal is less than the order represented by the high-fidelity stereo reproduction (HOA) signal. 4. The method of claim 1, wherein the encoded direction signal and the encoded ambient signal are received in a bitstream, and the bitstream is sensed and decoded into a plurality of transmission channels, each of the plurality of transmission channels being reassigned to the direction signal or the ambient signal before the conversion and reassembly. 5. An apparatus for decompressing a high-order high-fidelity stereo (HOA) reproduction signal representation, the apparatus comprising: The input interface receives encoded direction signals and encoded ambient signals. An audio decoder that senses and decodes encoded directional signals and encoded ambient signals to generate decoded directional signals and decoded ambient signals, respectively. The inverse transformer converts the decoded environmental signal from the spatial domain to its HOA domain representation; and Synthesizers reconstruct high-order high-fidelity stereo reproduction (HOA) signals from the HOA domain representation and decoded directional signals of ambient signals. The inverse transformer is further configured to perform the conversion by applying an inverse spatial transformation to the decoded ambient signal. 6. The device according to claim 5, wherein the high-order high-fidelity stereo reproduction (HOA) signal represents a signal having an order greater than 1. 7. The device according to claim 5, wherein the order of the decoded ambient signal is less than the order represented by the high-fidelity stereo echo (HOA) signal. 8. The device of claim 5, wherein the encoded direction signal and the encoded ambient signal are received in a bit stream, and the bit stream is sensed and decoded into a plurality of transmission channels, each of the plurality of transmission channels being reassigned to the direction signal or the ambient signal before the conversion and reassembly. 9. A method for decompressing a high-order high-fidelity stereo reproduction (HOA) signal representation, the method comprising: Receive encoded direction signals and encoded environmental signals; The encoded direction signal and the encoded environment signal are sensed and decoded to generate the decoded direction signal and the decoded environment signal, respectively. The decoded environmental signal is converted from the spatial domain to the HOA domain representation of the environmental signal; The high-order, high-fidelity stereo (HOA) reproduction signal is reconstructed from the HOA domain representation and decoding of the ambient signal; and Smooth the reconstructed HOA signal. 10. An apparatus for decompressing a high-order high-fidelity stereo reproduction (HOA) signal representation, the apparatus comprising: The input interface receives encoded direction signals and encoded ambient signals. An audio decoder that senses and decodes encoded directional signals and encoded ambient signals to generate decoded directional signals and decoded ambient signals, respectively. The inverse transformer converts the decoded environmental signal from the spatial domain to the HOA domain representation of the environmental signal; A synthesizer that reconstructs a high-order, high-fidelity stereo reproduction (HOA) signal from the direction signal represented and decoded in the HOA domain of the ambient signal; and A smoother that smooths the reconstructed HOA signal. 11. A non-transitory computer-readable medium comprising instructions that, when executed by a processor, cause to perform the method according to any one of claims 1-4 and 9. 12. An apparatus for decompressing a high-order high-fidelity stereo (HOA) reproduction signal representation, comprising: One or more processors, and One or more storage media storing instructions that, when executed by the one or more processors, cause the method according to any one of claims 1-4 and 9 to be performed. 13. An apparatus comprising components for performing the method according to any one of claims 1-4 and 9.