HK1262529B - Method and apparatus for compressing and decompressing a higher order ambisonics representation for a sound field - Google Patents

Description

Methods and apparatus for compressing and decompressing higher-order stereo reverberation representations of sound fields

This application is a divisional application of the invention patent application with application number 201380064856.9, application date December 4, 2013, entitled "Method and apparatus for compressing and decompressing a high-order stereo reverberation representation of a sound field".

Technical Field

This invention relates to a method and apparatus for compressing and decompressing a high-order stereo reverberation representation of a sound field.

Background Technology

Higher-order stereo reverb (HOA) provides a way to represent three-dimensional stereo sound. Other techniques include wavefield synthesis (WFS) or channel-based methods like 22.2. Compared to channel-based methods, HOA representation offers the advantage of independence from specific speaker configurations. However, this flexibility comes at the cost of decoding; playback of an HOA representation on a specific speaker configuration requires decoding. HOA can also be provided for configurations with fewer speakers compared to WFS, which typically requires a large number of speakers. Another advantage of HOA is that the same representation can be used without any modifications to the binaural presentation of headphones.

HOA is a representation of the spatial density of complex harmonic plane wave amplitudes based on truncated spherical harmonics (SH). Each expansion coefficient is a function of angular frequency, which can be equivalently represented by time-domain functions. Therefore, without loss of generality, it can be assumed that the complete HOA sound field representation consists of O time-domain functions, where O represents the number of expansion coefficients. In the following text, these time-domain functions will be referred to as the HOA coefficient sequence.

The spatial resolution of HOA representation increases with the maximum order N of the expansion. Unfortunately, the number of expansion coefficients O increases quadratically with order N, specifically O = (N+1) ^² . For example, a typical HOA representation using order N = 4 requires O = 25 HOA (expansion) coefficients. Based on the above considerations, given a desired mono sampling rate f_{_s} and the number of bits per sample N_{_b}, the total bit rate for transmission of the HOA representation is determined by O·f _{_s} ·N_{_b}. Transmitting an order N = 4 HOA representation using N_{_b} = 16 bits per sample at a sample rate f _{_s} = 48kHz would result in a bit rate of 19.2 MBits/s, which is very high for many practical applications (e.g., streaming). Therefore, compression of the HOA representation is highly desirable.

Summary of the Invention

Existing methods for compressing HOA representations (with N>1) are scarce. The most straightforward method proposed by E. Hellerud, I. Burnett, A. Solvang, and U.P. Svensson, "Encoding Higher Order Ambisonics with AAC", 124th AES Convention, Amsterdam, 2008, is to perform direct encoding of the individual HOA coefficient sequences using Advanced Audio Coding (AAC), a perceptual coding algorithm. However, an inherent problem with this method is the perceptual encoding of signals that have never been heard. The reconstructed playback signal is often obtained by a weighted sum of the HOA coefficient sequences, and there is a high probability that perceptual coding noise will be exposed when the decompressed HOA representation is presented on a particular speaker configuration. The main problem with the exposure of perceptual coding noise is the high cross-correlation between the individual HOA coefficient sequences. Since the coded noise signals in the individual HOA coefficient sequences are often uncorrelated with each other, a beneficial superposition of perceptual coding noise can occur, while the noise-free HOA coefficient sequences are eliminated at the superposition points. Another problem is that these cross-correlations lead to a decrease in the efficiency of the perceptual encoder.

To minimize the extent of both effects, EP 2469742 A2 proposes transforming the HOA representation into an equivalent representation in the discrete spatial domain before perceptual coding. Formally, this discrete spatial domain is the time-domain equivalent of the spatial density of complex harmonic plane wave amplitudes sampled at discrete directions. Therefore, the discrete spatial domain is represented by O conventional time-domain signals. If the loudspeaker happens to be located in the same direction as assumed for the spatial domain transformation, the conventional time-domain signals can be interpreted as approximately plane waves impacting from the sampling direction, and the conventional time-domain signals will correspond to the loudspeaker signals.

Transforming to the discrete spatial domain reduces the cross-correlation between signals in different spatial domains, but does not completely eliminate it. An example of relatively high cross-correlation is directional signals whose direction is between adjacent directions covered by the spatial domain signal.

The main drawbacks of both methods are that the number of sensing encoded signals is (N+1) ^{^2} , and the data rate for compressed HOA representations increases exponentially with the N-squared order of the stereo reverberation.

To reduce the number of perceptually encoded signals, patent application EP 2665208 A1 proposes decomposing the HOA representation into a given maximum number of dominant orientation signals and residual environment components. The reduction in the number of perceptually encoded signals is achieved by lowering the order of the residual environment components. The principle behind this method is to maintain high spatial resolution with respect to the dominant orientation signals while using sufficient precision to represent the residuals through a lower-order HOA representation.

This method works well as long as the assumptions about the sound field are met: that the sound field consists of a small number of dominant directional signals (represented by a bulk plane wave function encoded with the full order N) and a residual ambient component without any directionality. However, if the residual ambient component still contains some dominant directional components after decomposition, the order reduction leads to errors that are clearly perceptible in the post-decomposition representation. A typical example of a HOA representation that violates the assumptions is a bulk plane wave encoded with an order below N. Such bulk plane waves with orders below N can be produced in artistic creations to make the sound source appear more widespread, and such bulk plane waves with orders below N can also appear when recording HOA sound field representations using a spherical microphone. In both examples, the sound field is represented by a large number of highly correlated spatial domain signals (for further explanation, see Spatial resolution of Higher Order Ambisonics).

The problem this invention aims to solve is to eliminate the drawbacks caused by the process described in patent application EP 2665208 A1, thereby also avoiding the drawbacks of the other cited prior art. This problem is solved by the methods disclosed in the specification. Corresponding apparatuses utilizing these methods are disclosed in the specification.

This invention improves the HOA sound field representation compression process described in patent application EP 2665208 A1. First, as described in EP 2665208 A1, the HOA representation is analyzed for the presence of a dominant sound source, and the direction of the dominant sound source is estimated. Using information about the direction of the dominant sound source, the HOA representation is decomposed into multiple dominant directional signals and residual components representing bulk plane waves. However, instead of immediately reducing the order of the residual HOA component, the order is transformed to a discrete spatial domain to obtain a bulk plane wave function at a uniform sampling direction representing the residual HOA component. These plane wave functions are then predicted based on the dominant directional signals. The reason for this operation is that a portion of the residual HOA component may be highly correlated with the dominant directional signal.

The prediction can be a simple one, generating only a small amount of auxiliary information. In the simplest case, the prediction consists of appropriate scaling and delay. Finally, the prediction error is transformed back into the HOA domain and treated as a residual environment HOA component, against which order reduction is performed.

Advantageously, subtracting the predictable signal from the residual HOA component reduces its total power while maintaining the number of dominant directional signals, thereby reducing decomposition errors due to order reduction.

In principle, the compression method of the present invention is applicable to the high-order stereo reverberation (HOA) representation of compressed sound fields, and the method includes the following steps:

- Estimate the dominant sound source direction based on the current frame of the HOA coefficients;

- Based on the HOA coefficients and the dominant sound source direction, the HOA representation is decomposed into a dominant directional signal and a residual HOA component in the time domain, wherein the residual HOA component is transformed to a discrete spatial domain to obtain a plane wave function at a uniform sampling direction representing the residual HOA component, and wherein the plane wave function is predicted based on the dominant directional signal, thereby providing parameters describing the prediction, and the corresponding prediction error is transformed back to the HOA domain.

- Reduce the current order of the residual HOA component to a lower order to obtain the reduced-order residual HOA component;

-Decorrelate the reduced-order residual HOA component to obtain the corresponding residual HOA component time-domain signal;

- The dominant orientation signal and the residual HOA component time-domain signal are perceptually encoded to provide a compressed dominant orientation signal and a compressed residual component signal.

In principle, the compression device of the present invention is applicable to the high-order stereo reverberation (HOA) representation of compressed sound fields, and the device includes:

- A device suitable for estimating the direction of the dominant sound source based on the current time frame of the HOA coefficient;

- An apparatus suitable for decomposing the HOA representation into a dominant directional signal and a residual HOA component in the time domain based on the HOA coefficients and the dominant sound source direction, wherein the residual HOA component is transformed to a discrete spatial domain to obtain a plane wave function at a uniform sampling direction representing the residual HOA component, and wherein the plane wave function is predicted based on the dominant directional signal, thereby providing parameters describing the prediction, and the corresponding prediction error is transformed back to the HOA domain.

- An apparatus suitable for reducing the current order of the residual HOA component to a lower order to obtain a reduced-order residual HOA component.

- An apparatus suitable for decorrelation of the reduced-order residual HOA component to obtain the corresponding residual HOA component time-domain signal;

- An apparatus suitable for perceptually encoding the dominant orientation signal and the residual HOA component time-domain signal to provide a decompressed dominant orientation signal and a decompressed residual component signal;

In principle, the decompression method of the present invention is applicable to decompressing high-order stereo reverberation representations compressed according to the above-described compression method, and the decompression method includes the following steps:

-Perceptual decoding is performed on the compressed dominant orientation signal and the compressed residual component signal to provide the decompressed dominant orientation signal and the decompressed time-domain signal representing the residual HOA component in the spatial domain.

- Recorrelated the decompressed time-domain signal to obtain the corresponding reduced-order residual HOA component;

- Increase the order of the reduced residual HOA component to the original order to provide the corresponding decompressed residual HOA component;

- The decompressed dominant directional signal, the original-order decompressed residual HOA component, the estimated dominant sound source direction, and the parameters describing the prediction are used to compose a decompressed and reassembled frame with corresponding HOA coefficients.

In principle, the decompression apparatus of the present invention is suitable for decompressing high-order stereo reverberation representations compressed according to the above-described compression method, the decompression apparatus comprising:

- An apparatus suitable for perceptual decoding of compressed dominant orientation signals and compressed residual component signals, thereby providing decompressed dominant orientation signals and decompressed time-domain signals representing residual HOA components in the spatial domain.

- An apparatus suitable for recorrelating the decompressed time-domain signal to obtain the corresponding reduced-order residual HOA component;

- An apparatus suitable for increasing the order of the reduced residual HOA component to the original order, thereby providing a corresponding decompressed residual HOA component;

- An apparatus suitable for assembling and recombining frames of corresponding HOA coefficients by using the decompressed dominant directional signal, the residual HOA component of the original order decompression, the estimated dominant sound source direction, and the parameters describing the prediction.

Advantageous additional embodiments are disclosed in the corresponding dependent claims.

Attached Figure Description

Exemplary embodiments of the present invention will be described with reference to the accompanying drawings, wherein:

Figure 1a Compression step 1: Decompose the HOA signal into multiple dominant directional signals, residual environmental HOA components, and auxiliary information;

Figure 1b Compression Step 2: Order reduction, decorrelation of the environmental HOA component, and perceptual encoding of the two components;

Figure 2a Decompression step 1: Perceptual decoding of the time domain signal, recorrelation of the signal representing the residual environment HOA component, and order increase;

Figure 2b Decompression step 2: Composition represented by the total HOA;

Figure 3 HOA decomposition

Figure 4 HOA Composition

Figure 5 Spherical coordinate system

Figure 6 shows exemplary curves of the normalized function v _{_N} (Θ) for different values of N.

Detailed Implementation

Compression process

The compression process according to the invention comprises two consecutive steps, as shown in Figures 1a and 1b, respectively. The precise definitions of the individual signals are described in the detailed description section on HOA decomposition and reconstruction. Frame-by-frame processing is used for compression of a non-overlapping input frame D(k) of a HOA coefficient sequence of length B, where k denotes the frame index. The frame is defined with respect to the HOA coefficient sequence specified in equation (42) as follows:

D(k):=[d((kB+1)T _s )d((kB+2)T _s )…d((kB+B)T _s )] (1)

Where T _{_s} represents the sampling period.

In Figure 1a, frame D(k) of the HOA coefficient sequence is input to the dominant sound source direction estimation step or stage 11, which analyzes the HOA representation in response to the presence of a dominant directional signal and estimates the direction of the dominant directional signal. The direction estimation can be performed, for example, by the process described in patent application EP 2665208 A1. The estimated direction is represented by where represents the maximum number of direction estimates. Assume the estimated directions are set in the matrix as follows:

It is implicitly assumed that the direction estimates are properly organized by assigning them to direction estimates from previous frames. Therefore, it is assumed that the time series of each direction estimate describes the directional trajectory of the dominant sound source. Specifically, if the d-th dominant sound source should not be operating, it can be indicated by assigning invalid values. Then, in the decomposition step or stage 12, the HOA representation is decomposed using the estimated directions into a maximum dominant directional signal _XDIR (k-1), some parameters ζ(k-1) describing the prediction of the spatial domain signal of the residual HOA component predicted based on the dominant directional signal, and an environmental HOA component _DA (k-2) representing the prediction error. A detailed description of the decompression is provided in the HOA decompression section.

Figure 1b illustrates the perceptual encoding of the directional signal _XDIR (k-1) and the perceptual encoding of the residual environmental HOA component _DA (k-2). The directional signal _XDIR (k-1) is a conventional time-domain signal that can be compressed separately using any existing perceptual compression technique. The compression of the environmental HOA component _DA (k-2) is performed in two consecutive steps or stages. In the step or stage 13 of order reduction, the order _NRED of the stereo reverberation is reduced, where, for example, _NRED = 1, resulting in the environmental HOA component DA _,RED (k-2). This order reduction is achieved by retaining _NRED HOA coefficients in _DA (k-2) and discarding the others. On the decoder side, as explained below, for the omitted values, the corresponding zero values are appended.

It should be noted that, compared to the method in patent application EP 2665208 A1, the reduced order N _RED can generally be chosen to be smaller due to the smaller residual total power and the smaller residual directional amount of the residual environmental HOA component. Therefore, the reduction in order results in a smaller error compared to patent application EP 2665208 A1.

In the subsequent decorrelation step or stage 14, the HOA coefficient sequence of the reduced-order environmental HOA component DA _,RED (k-2) is decorrelated to obtain the time-domain signal WA _{, RED} (k-2), which is input to (a set of) parallel perceptual encoders or a compressor 15 operating according to any known perceptual compression technique. Decorrelation is performed to avoid exposing perceptual coding noise when presenting the HOA representation after decompression (see patent application EP 12305860.4 for an explanation). Approximate decorrelation can be achieved by transforming DA _,RED (k-2) into 0 _RED equivalent signals in the spatial domain, which is implemented by applying the spherical harmonic transform described in patent application EP 2469742 A2.

Alternatively, the adaptive spherical harmonic transform proposed in patent application EP 12305861.2 can be used, in which the grid of the sampling direction is rotated to achieve the best possible decorrelation effect. Another alternative decorrelation technique is the Karhunen-Loève transform (KLT) described in patent application EP12305860.4. It should be noted that for the last two decorrelation methods, some auxiliary information, represented as α(k-2), is required to enable decorrelation recovery during the HOA decompression stage.

In one embodiment, perceptual compression of all time-domain signals X _DIR (k-1) and D _A,RED (k-2) is performed jointly to improve coding efficiency.

The output of perceptual coding is a compressed directional signal and a compressed ambient time-domain signal.

Decompression steps

The decompression process is illustrated in Figures 2a and 2b. Similar to compression, the decompression process consists of two consecutive steps. In Figure 2a, perceptual decompression of the directional signal and the time-domain signal representing the residual environment HOA components is performed in the perceptual decoding or decompression step or stage 21. In the recorrelation step or stage 22, the resulting perceptually decompressed time-domain signal is recorrelated to provide a residual component HOA representation of order N _RED. Optionally, the recorrelation can be performed using a transmitted or stored (depending on the decorrelation method used) parameter α(k-2) in a manner opposite to the two alternative processes described for step/stage 14. Subsequently, in the order-increasing step or stage 23, order-increasing is achieved by appending corresponding 'zero' value rows to the appropriate HOA representation of the estimated order N, thereby assuming zero values for higher-order HOA coefficients.

In Figure 2b, in composition step or stage 24, the overall HOA representation is reconstructed based on the decompressed dominant orientation signal along with the corresponding direction and prediction parameter ζ(k-1), and based on the residual environment HOA components, resulting in a frame of decompressed and reconstructed HOA coefficients.

While performing perceptual compression of all time-domain signals X _DIR (k-1) and _WA,RED (k-2) in conjunction to improve coding efficiency, perceptual decompression of the compressed directional signal and the compressed time-domain signal are also performed in conjunction in a corresponding manner.

A detailed description of reorganization is provided in the HOA Reorganization section.

HOA decomposition

Figure 3 shows a block diagram illustrating the operations performed for HOA decomposition. The operations are summarized as follows: First, the smoothed dominant orientation signal X _DIR (k-1) is calculated, and its output is used for sensing compression. Next, the residual between the HOA representation D _DIR (k-1) of the dominant orientation signal, represented by O orientation signals, and the original HOA representation D(k-1), is calculated, where the O orientation signals can be considered as generally plane waves in a uniformly distributed direction. These orientation signals are predicted based on the dominant orientation signal X _DIR (k-1), and the prediction parameter ζ(k-1) is output. Finally, the residual DA(k-2) between the original HOA representation D(k-2) and the HOA representation D _DIR (k-1) of the dominant orientation _signal , as well as the HOA representation of the predicted orientation signal in a uniformly distributed direction, are calculated and output.

Before going into details, it should be noted that during the composition process, directional changes between consecutive frames can cause signal interruptions in all computations. Therefore, firstly, instantaneous estimates of the corresponding signals for overlapping frames are calculated, with a length of 2B. Secondly, an appropriate window function is used to smooth the results for consecutive overlapping frames. However, each smoothing introduces hysteresis in a single frame.

Calculate the instantaneous dominant orientation signal

In step or stage 30, the calculation of the instantaneous dominant directional signal for the current frame D(k) of the HOA coefficient sequence based on the estimated sound source direction is based on pattern matching described in the following literature: M.A. Poletti, "Three-Dimensional Surround Sound Systems Based on Spherical Harmonics", J. Audio Eng. Soc, 53(11), pages 1004-1025, 2005. Specifically, a search is performed to obtain the best approximation of the directional signal for a given HOA signal from the HOA representation.

Furthermore, without loss of generality, it is assumed that a vector can uniquely specify each direction of the effective dominant sound source. The estimated vector contains the tilt angle θDOM _,d (k)∈[0,π] and the azimuth angle _φDOM,d (k)∈[0,2π] according to the following formula (see Figure 5 for a schematic):

First, according to

The mode matrix based on the direction estimation of effective sound sources is calculated, and its

In equation (4), D _{_ACT} (k) represents the number of valid directions for the k-th frame, and d _{_ACT,j} (k) (1≤j≤D_{_ACT} (k)) indicates their indices. represents a real-valued spherical harmonic function, which is defined in the definition section of the real-valued spherical harmonic function definition.

Second, calculate the matrix containing the instantaneous estimates of all dominant directional signals in the (k-1)th and kth frames, as defined below.

in

This is achieved through two steps. In the first step, the orientation signal samples in the rows corresponding to invalid directions are set to zero, i.e.

This refers to the set indicating the effective directions. In the second step, the orientation signal samples corresponding to the effective directions are obtained by first arranging the orientation signal samples corresponding to the effective directions in a matrix according to the following formula:

Then the matrix is calculated to ensure that the Euclidean norm of the error is...

Minimize. The solution is given by the following equation:

Time smoothing

For step or stage 31, smoothing is explained only for the directional signal, because smoothing of other types of signals can be done in a completely similar manner. The directional signal estimate, whose samples are contained in the matrix according to equation (6), is windowed using the following appropriate window function:

The window function must satisfy the condition that the sum of itself and its offset version in the following overlapping region (assuming the offset of sample B) is '1':

The periodic Hann window, defined by the following equation, provides an example for such a window function:

The smoothed directional signal of the (k-1)th frame is calculated by appropriately superimposing the windowed instantaneous estimates according to the following equation:

The samples of all smoothed directional signals for the (k-1)th frame are set in the following matrix:

in

The smooth dominant orientation signal X _DIR,d (l) should be a continuous signal continuously input to the sensing encoder.

Calculate the HOA representation of the smoothed dominant directional signal.

In step or stage 32, based on the continuous signal X _DIR,d (l), the HOA representation of the smoothed dominant orientation signal is calculated according to X _DIR (k-1) to mimic the operation to be performed on the HOA composition. Since changes in orientation estimation between consecutive frames cause interruptions, the instantaneous HOA representation of overlapping frames of length 2B is calculated again, and the results for consecutive overlapping frames are smoothed using an appropriate window function. Therefore, the HOA representation D _DIR (k-1) is obtained by the following equation:

D _DIR (k-1)＝Ξ _ACT (k)X _DIR,ACT,WIN1 (k-1)+Ξ _ACT (k-1)X _DIR,ACT,WIN2 (k-1) (18),

in,

and

Residual HOA representation is expressed using directional signals on a uniform grid.

In step or stage 33, the residual HOA representation, which is derived from the directional signal on a uniform grid, is calculated based on D _{_DIR} (k-1) and D(k-1) (i.e., D(k)D(k) delayed by frame delay 381). The purpose of this operation is to obtain the directional signal (i.e., the bulk plane wave function) impacted from some fixed, almost uniformly distributed directions (1≤o≤0, also known as grid directions) to represent the residual [D(k-2)D(k-1)]-[D _{_DIR} (k-2)D _{_DIR} (k-1)].

First, regarding the grid orientation, the pattern matrix Ξ _GRID is calculated as follows:

in

Since the grid orientation is fixed throughout the compression process, the pattern matrix Ξ _GRID only needs to be calculated once.

The orientation signal on the corresponding grid is obtained as follows:

Predict the orientation signal on a uniform grid based on the dominant orientation signal.

In step or stage 34, the orientation signal on the uniform grid is predicted according to X _DIR (k-1). The prediction of the orientation signal on the uniform grid composed of the grid directions is based on two consecutive frames for smoothing purposes, i.e., the frame of the expanded grid signal (length 2B) is the frame of the expanded dominant orientation signal according to the smoothing:

Predicted.

First, each grid signal contained in the array is assigned to a dominant directional signal contained in the array. This assignment can be based on the calculation of a normalized cross-correlation function between the grid signal and all dominant directional signals. Specifically, the dominant directional signal is assigned to the grid signal, which provides the highest value of the normalized cross-correlation function. The result of the assignment can be represented by an assignment function that assigns the 0th grid signal to the 1st dominant directional signal.

Second, each grid signal is predicted using the assigned dominant orientation signal. The predicted grid signal is calculated based on the assigned dominant orientation signal through delay and scaling, as follows:

Where _Ko (k-1) represents the scaling factor and _Δo (k-1) indicates the sample delay. These parameters are chosen to minimize the prediction error.

If the power of the prediction error is greater than the power of the grid signal itself, then it is assumed that the prediction has failed. The corresponding prediction parameters can then be set to any invalid value.

It should be noted that other types of prediction are also possible. For example, instead of calculating the full-band scaling factor, it is possible to determine the scaling factor for the sensing-oriented frequency band. However, this improved prediction comes at the cost of an increased amount of auxiliary information.

All prediction parameters can be set in the parameter matrix using the following equation:

Assume all predicted signals are set in a matrix

Calculate the HOA representation of the directional signal on the predicted uniform grid.

In step or stage 35, the HOA representation of the calculated predicted grid signal is determined according to the following formula:

HOA representation for calculating residual ambient sound field components

In step or stage 37, by formula:

The HOA representation of the residual ambient sound field component is calculated based on the time-smoothed version (in step/stage 36) of D(k) with two-frame delay versions (delays 381 and 383) D(k-2) and the frame delay version of D _DIR (k-1) with delay 382 D _DIR (k-2).

HOA indicates

Before describing the process of each step or stage in Figure 4 in detail, a summary is provided. The directional signal with respect to a uniformly distributed direction is predicted based on the decoded dominant directional signal using prediction parameters. Then, the total HOA representation consists of the HOA representation of the dominant directional signal, the HOA representation of the predicted directional signal, and the residual environment HOA component.

Calculate the HOA representation of the dominant directional signal

The inputs are fed into step or stage 41 to determine the HOA representation of the dominant directional signal. After the mode matrices _ΞACT (k) and _ΞACT (k-1) have been calculated based on the direction estimation, the HOA representation of the dominant directional signal is obtained based on the direction estimates of the effective sound field in the k-th and (k-1)-th frames using the following equation:

in,

and

Predict the orientation signal on a uniform grid based on the dominant orientation signal.

The input is fed into step or stage 43 to predict the orientation signal on the uniform grid based on the dominant orientation signal. The unfolded frame of the predicted orientation signal on the uniform grid consists of cells according to the following equation:

The unit is predicted based on the dominant orientation signal using the following equation:

Calculate the HOA representation of the directional signal on the predicted uniform grid.

In step or stage 44, which computes the HOA representation of the predicted orientation signal on a uniform grid, the HOA representation of the predicted grid orientation signal is obtained by an equation, where Ξ _GRID represents the pattern matrix with respect to a predefined grid orientation (see equation (21) for the definition).

Composition of HOA sound field representation

In step or stage 46, the overall HOA generation representation is finally composed of the following equations, based on (i.e., delayed by frame delay 42), (which is the time-smoothed version in step/stage 45), and:

The basic principles of high-order stereo reverberation

Higher-order stereo reverberation is a description of the sound field in a compact region of interest, assuming no sound source in that region. In this case, the spatiotemporal characteristics of the sound pressure p(t,x) at time t and position x in that region of interest are physically determined entirely by the uniform wave equations. The following is based on the spherical coordinate system shown in Figure 5. The X-axis points to the front, the y-axis points to the left, and the z-axis points upward. The position x = (r, θ, φ)T in space is represented by the radius r > 0 (i.e., the distance to the origin), the tilt angle θ ∈ [0, π] measured from the polar axis z, and the azimuth angle φ ∈ [0, π] measured counterclockwise from the x-axis in the xy- ^plane . (·) ^T denotes transpose.

As can be seen (see E.G. Williams, "Fourier Acoustics", volume 93 of Applied Mathematical Sciences, Academic Press, 1999), the Fourier transform of sound pressure with respect to time (represented by), i.e.

(where ω represents the angular frequency and i represents the imaginary unit) can be expanded into a series of spherical functions as follows.

Where c _{_s} represents the speed of sound, and k represents the angular wavenumber, which is related to ω by a formula, j _{_n} (·) represents a spherical Bezier function of the first kind, and represents a real-valued spherical harmonic function of order n and angle m (defined in the real-valued spherical harmonic function section). The expansion coefficients depend only on the angular wavenumber k. It should be noted that it is implicitly assumed here that the sound pressure level is spatially band-limited. Therefore, this series is truncated with respect to the order index n at the upper limit N, which is called the order in HOA representation.

If the sound field is represented by an infinite number of superpositions of harmonic plane waves with different angular frequencies ω, and the sound field can be reached from all possible directions specified by the angle tuple (θ, φ), then it can be seen (see B. Rafaely, "Plane-wave Decomposition of the Sound Field on a Sphere by Spherical Convolution", J. Acoust. Soc. Am., 4(116), pages 2149-2157, 2004) that the corresponding complex amplitude function of the plane wave can be represented by the following spherical harmonic expansion:

The expansion coefficients are related to the expansion coefficients through the following equation:

Assuming that each coefficient is a function of the angular frequency ω, the application of the inverse Fourier transform (represented by ) provides the following time-domain function for each order n and angle m:

The functions can be collected in a single vector as follows:

The position index of the time-domain function in the vector d(t) is given by n(n+1)+1+m.

The final stereo reverberation format provides a version using d(t) sampled at sampling frequency _fS as follows:

Where _TS = 1/ _fS represents the sampling period. The d( _lTS ) unit is called the stereo reverberation coefficient. It should be noted that the time-domain signal and therefore the stereo reverberation coefficient are real values.

Definition of real-valued spherical harmonic functions

The real-valued spherical harmonic function is given by the following equation:

in

Using the Legendre polynomial P _{_n} (x), and unlike the EGWilliams textbook mentioned above, without using the Condon-Shortley term, the associated Legendre function P _{_n,m} (x) is defined as follows:

Spatial resolution of high-order stereo reverberation

The plane wave function x(t) arriving from the direction _Ω0 = ( _θ0 , _φ0 ) ^T is represented in HOA by the following equation:

The spatial density corresponding to the plane wave amplitude is given by the following formula:

As can be seen from equation (48), it is the product of the bulk plane wave function x(t) and the spatial dispersion function _vN (Θ) _, which can be regarded as an angle Θ that depends only on Ω and _Ω0 and has the following characteristics:

Cosθ=Cos6Cosθ ₀ +Cos(φ-φ ₀ )sinθsinθ ₀ (49).

As expected, under the constraint of infinite order, i.e., N→∞, the spatially dispersed function is transformed into the Dirac delta function δ(·), i.e.

However, in the case of a finite order N, the contribution of a substantially plane wave from direction _Ω0 is applied to adjacent directions, and the degree of ambiguity decreases with increasing order. Figure 6 shows the curves of the normalized function _vN (Θ) for different values of N. It should be noted that the temporal characteristic of the spatial density of any plane wave amplitude in the direction Ω is a multiple of its characteristic in any other direction. In particular, for some fixed directions _Ω1 and _Ω2 , the functions d(t, _Ω1 ) and d(t, _Ω2 ) are highly correlated with respect to time t.

Discrete spatial domain

If the spatial density of plane wave amplitudes is discrete along spatial directions Ω _₀ (1≤o≤0) of quantity O and almost uniformly distributed on a unit sphere, then O directional signals d(t, Ω_{_o} ) are obtained. These signals are then aggregated into a vector as shown in the following equation:

d _SPAT (t):=[d(t,Ω ₁ )...d(t,Ω _O )]T (51)

It can be shown by using equation (47) that the vector can be calculated by a single matrix multiplication based on the continuous stereo reverberation representation d(t) defined in equation (41), which is:

d _SPAT (t)＝Ψ ^H d(t), (52)

Where (·) ^H indicates joint permutation and conjugation, and Ψ represents the pattern matrix defined by the following equation:

Ψ:=[S ₁ ... S _O ] (53),

in

Since the direction _Ω₀ is almost uniformly distributed over the unit sphere, the mode matrix is generally invertible. Therefore, through the equation...

d(t)＝Ψ ^-H d _SPAT (t) (55)

A continuous stereo reverberation representation can be calculated from the directional signal d(t, _Ωo ). Two equations constitute the transformation and inverse transformation between the stereo reverberation representation and the spatial domain. In this application, these transformations are referred to as the spherical harmonic transformation and the inverse spherical harmonic transformation.

Since the direction _Ω₀ is almost uniformly distributed on the unit sphere, ^{ΨH ≈Ψ⁻¹} (56) proves that it is feasible to use ^Ψ⁻¹ instead of ^ΨH in equation (52). Advantageously, all of the above relations are also valid for the discrete time domain.

On both the encoding and decoding sides, the process of this invention can be executed by a single processor or circuit, or by several processors or circuits operating in parallel and/or in different parts of the process of this invention.

This invention can be used to process corresponding sound signals that can be presented or played on speaker devices in a home environment or in a movie theater.

Claims (5)

1. A method for decompressing a compressed high-order stereo reverberation (HOA) representation, the method comprising: The compressed dominant orientation signal and the compressed residual component signal are perceptually decoded to provide the decompressed dominant orientation signal and the decompressed time-domain signal representing the residual HOA component in the spatial domain. The decompressed time-domain signal is recorrelated to obtain the corresponding reduced-order residual HOA component; By increasing the corresponding reduced-order residual HOA component to the original order, a decompressed residual HOA component is provided. The predicted directional signal is determined based on at least one parameter; Based on the decompressed dominant directional signal, the predicted directional signal, and the decompressed residual HOA component, the HOA sound field representation is determined, and The parameter indicates the maximum number of effective directional signals used for predicting the dominant sound source. 2. An apparatus for decompressing high-order stereo reverberation (HOA) representations, the apparatus comprising: The decoder performs perceptual decoding on the compressed dominant directional signal and the compressed residual component signal to provide a decompressed dominant directional signal and a decompressed time-domain signal representing the residual HOA component in the spatial domain. A recorrelator is used to recorrelate the decompressed time-domain signal to obtain the corresponding reduced-order residual HOA component. A processor configured to provide a decompressed residual HOA component by amplifying the corresponding reduced-order residual HOA component to the original order, the processor being further configured to determine a predicted directional signal based on at least one parameter. The processor is further configured to determine the HOA sound field representation based on the decompressed dominant directional signal, the predicted directional signal, and the decompressed residual HOA component. The parameter indicates the maximum number of effective directional signals used for predicting the dominant sound source. 3. A non-transitory computer-readable storage medium encoded with a computer program, the computer program being used to cause a computer to perform the method according to claim 1. 4. An apparatus for decompressing a compressed high-order stereo reverberation (HOA) representation, comprising: processor, and A non-transitory computer-readable storage medium encoded with a computer program that causes a processor to perform the method according to claim 1. 5. An apparatus for decompressing a compressed high-order stereo reverberation (HOA) representation, comprising components for performing the method according to claim 1.