HK1263295B - Method and apparatus for compressing and decompressing a higher order ambisonics representation for a sound field - Google Patents

Description

Method and apparatus for compressing and decompressing higher order ambisonic representations of a sound field

The application is a divisional application of an invention patent application with the application number of 201380064856.9, the application date of 2013, 12 and 4 and the invention name of a method and a device for compressing and decompressing the higher-order ambisonic representation of a sound field.

Technical Field

The present invention relates to methods and apparatus for compressing and decompressing higher order ambisonic representations of a sound field.

Background

Higher order ambisonics (denoted HOA) provides one way to represent three dimensional stereo sound. Other techniques are Wave Field Synthesis (WFS) or channel-based methods like 22.2. Compared to channel-based approaches, HOA representation offers the advantage of being independent of a particular speaker configuration. However, this flexibility comes at the expense of a decoding process, which is required for playback of the HOA representation on a particular speaker configuration. HOA may also be provided for configurations comprising only fewer speakers compared to WFS methods, where the number of speakers required is typically large. An additional advantage of HOA is that the same representation can be used without any modification to the binaural rendering of the headphone.

HOA is a representation based on the spatial density of complex harmonic plane wave amplitudes expanded by a truncated Spherical Harmonic (SH). Each expansion coefficient is a function of angular frequency, which can be equivalently represented by a time-domain function. Thus, without loss of generality, it may in fact be assumed that the complete HOA soundfield representation consists of O time-domain functions, where O denotes the number of expansion coefficients. Hereinafter, these time domain functions will be equivalently referred to as HOA coefficient sequences.

The spatial resolution of the HOA representation increases with the maximum order N of the unfolding. Unfortunately, the number of expansion coefficients O grows quadratically with the order N, in particular O ═ N +1)². For example, a typical HOA using order N-4 represents a HOA (expansion) coefficient requiring O-25. Given the above considerations, a desired mono sampling rate f is given_sAnd number of bits per sample N_bThe total bit rate of the transmission for HOA is represented by O · f_s·N_bAnd (4) determining. Using each sample N_b16 bits at a sample rate f_sThe HOA representation with a transmission order N-4 of 48kHz will result in a bit rate of 19.2MBits/s, which is very high for many practical applications, such as streaming. Therefore, compression of the HOA representation is highly desirable.

Disclosure of Invention

Existing methods of handling compression of HOA representations (with N >1) are rare. The most straightforward approach proposed by e.hellerud, i.burnett, asolving and u.p.svensson, "Encoding high Order Ambisonics with AAC",124th aes Convention, Amsterdam,2008 is to perform direct Encoding of the respective sequence of HOA coefficients using Advanced Audio Coding (AAC), which is a perceptual coding algorithm. However, a problem inherent to this approach is perceptual coding of the inaudible signal. The reconstructed playback signal is often obtained by a weighted sum of the HOA coefficient sequences and when the decompressed HOA representation is presented on a specific loudspeaker configuration there is a high probability that perceptual coding noise is exposed. The main problem for perceptual coding noise exposure is the high cross-correlation between the individual HOA coefficient sequences. Since the coding noise signals in the individual HOA coefficient sequences are often uncorrelated with each other, a beneficial superposition of the perceptual coding noise may occur, while the noise-free HOA coefficient sequences cancel at the superposition. Another problem is that these cross-correlations lead to a decrease in the efficiency of the perceptual encoder.

In order to minimize the extent of both effects, it is proposed in EP 2469742 a2 to transform the HOA representation into an equivalent representation in the discrete spatial domain prior to perceptual encoding. Formally, the discrete spatial domain is the time domain equivalent of the spatial density of complex harmonic plane wave amplitudes sampled at some discrete direction. The discrete spatial domain is thus represented by O conventional time domain signals, which can be interpreted as a substantially plane wave impinging from the sampling direction if the loudspeaker is located exactly in the same direction as assumed for the spatial domain transform, and which will correspond to the loudspeaker signal.

The transformation into the discrete spatial domain reduces the cross-correlation between the individual spatial domain signals, but does not completely eliminate it. An example of a relatively high cross-correlation is a directional signal with a direction in the middle of the adjacent directions covered by the spatial domain signal.

The main disadvantages of both methods are: the number of perceptually encoded signals is (N +1)²And the data rate for the compressed HOA representation increases quadratically with the ambisonic order N.

In order to reduce the number of perceptually encoded signals, patent application EP 2665208 a1 proposes to decompose the HOA representation into a given maximum number of dominant directional signals and residual ambient components. The reduction of the number of signals to be perceptually encoded is achieved by reducing the order of the residual ambient component. The principle behind this method is: a high spatial resolution with respect to the dominant directional signal is maintained while using sufficient accuracy to represent the residual by the lower order HOA representation.

This method works well as long as the assumptions about the sound field are met, i.e., the sound field is assumed to be composed of a small number of dominant directional signals (representing a substantially plane wave function encoded using the full order N) and residual ambient components without any directivity. However, if the residual environmental component still contains some dominant directional component after decomposition, the step-down may result in errors that are clearly perceptible at the presentation after decomposition. A typical example of HOA representation that violates the assumption is a generally plane wave encoded at an order of less than N. Such generally plane waves of order below N may result from artistic authoring in order to make sound sources look more extensive, and may also occur as HOA soundfield representations are recorded by spherical microphones. In both examples, the sound field is represented by a large number of highly correlated Spatial domain signals (see also Spatial resolution of high Order Ambisonics for an explanation).

The problem to be solved by the present invention is to eliminate the drawbacks caused by the procedure described in patent application EP 2665208 a1, thereby also avoiding the drawbacks of the other cited prior art mentioned above. This problem is solved by the method disclosed in the specification. Corresponding apparatuses utilizing these methods are disclosed in the specification.

The present invention improves the HOA sound field representation compression process described in patent application EP 2665208 a 1. First, as described in EP 2665208 a1, the HOA representation is analyzed for the presence of a dominant sound source, whose direction is estimated. The HOA representation is decomposed into a number of dominant directional signals representing a substantially plane wave and a residual component using information of the dominant sound source direction. However, instead of immediately reducing the order of the residual HOA component, the order of the residual HOA component is transformed to the discrete spatial domain in order to obtain a substantially plane wave function at a uniform sampling direction representing the residual HOA component. Thereafter, these plane wave functions are predicted from the dominant directional signal. The reason for this is that part of the residual HOA component may be highly correlated with the dominant directional signal.

The prediction may be a simple prediction, resulting in only a small amount of side information. In the simplest case, the prediction consists of appropriate scaling and delay. Finally, the prediction error is transformed back into the HOA domain and, as residual ambient HOA component, an order reduction is performed for the residual ambient HOA component.

Advantageously, the effect of subtracting the predictable signal from the residual HOA component is to reduce its total power and keep the number of dominant directional signals and in this way reduce the decomposition errors due to order reduction.

In principle, the inventive compression method is suitable for compressing a higher order ambisonic (denoted HOA) representation of a sound field, said method comprising the steps of:

-estimating a dominant sound source direction from a current time frame of HOA coefficients;

-decomposing the HOA representation into a dominant directional signal and a residual HOA component in the time domain based on the HOA coefficients and on the dominant sound source direction, wherein the residual HOA component is transformed to the discrete spatial domain in order to obtain a plane wave function at a uniform sampling direction representing the residual HOA component, and wherein the plane wave function is predicted from the dominant directional signal, thereby providing parameters describing the prediction, and a corresponding prediction error is transformed back to the HOA domain;

-reducing a current order of said residual HOA component to a lower order, resulting in a reduced order residual HOA component;

-decorrelating said reduced-order residual HOA components to obtain corresponding residual HOA component time-domain signals;

-perceptually encoding said dominant directional signal and said residual HOA component time domain signal, thereby providing a compressed dominant directional signal and a compressed residual component signal.

In principle, the inventive compression device is suitable for compressing a higher order ambisonic (denoted HOA) representation of a sound field, said device comprising:

-means adapted to estimate a dominant sound source direction from a current time frame of HOA coefficients;

-means adapted to decompose the HOA representation into a dominant directional signal and a residual HOA component in the time domain based on the HOA coefficients and on the dominant sound source direction, wherein the residual HOA component is transformed to the discrete spatial domain in order to obtain a plane wave function at a uniform sampling direction representing the residual HOA component, and wherein the plane wave function is predicted from the dominant directional signal, thereby providing parameters describing the prediction, and the corresponding prediction error is transformed back to the HOA domain;

-means adapted to reduce a current order of said residual HOA component to a lower order, resulting in a reduced order residual HOA component;

-means adapted to decorrelate said reduced-order residual HOA components to obtain corresponding residual HOA component time domain signals;

-means adapted for perceptually encoding said dominant directional signal and said residual HOA component time domain signal, thereby providing a decompressed dominant directional signal and a decompressed residual component signal;

in principle, the decompression method of the present invention is suitable for decompressing a higher order ambisonic representation compressed according to the above-described compression method, said decompression method comprising the steps of:

-perceptually decoding the compressed dominant directional signal and the compressed residual component signal, thereby providing a decompressed dominant directional signal and a decompressed time domain signal representing the residual HOA component in the spatial domain;

-re-correlating said decompressed time domain signal to obtain a corresponding reduced order residual HOA component;

-increasing the order of said reduced-order residual HOA component to the original order, thereby providing a corresponding decompressed residual HOA component;

-composing a decompressed and recomposed frame of corresponding HOA coefficients using the decompressed dominant directional signal, the original order decompressed residual HOA components, the estimated dominant sound source direction and the parameters describing the prediction.

In principle, a decompression apparatus of the present invention is adapted to decompress a higher order ambisonic representation compressed according to the above-described compression method, the decompression apparatus comprising:

-means adapted for perceptually decoding the compressed dominant directional signal and the compressed residual component signal, thereby providing a decompressed dominant directional signal and a decompressed time domain signal representing the residual HOA component in the spatial domain;

-means adapted to re-correlate said decompressed time domain signal to obtain a corresponding reduced order residual HOA component;

-means adapted to increase the order of said reduced-order residual HOA component to the original order, thereby providing a corresponding decompressed residual HOA component;

-means adapted to compose a decompressed and recomposed frame of corresponding HOA coefficients by using said decompressed dominant directional signal, said original order decompressed residual HOA component, said estimated dominant sound source direction and said parameters describing said prediction.

Advantageous additional embodiments are disclosed in the corresponding dependent claims.

Drawings

Exemplary embodiments of the invention are described with reference to the accompanying drawings, in which:

FIG. 1a compression step 1: decomposing the HOA signal into a plurality of dominant directional signals, a residual ambient HOA component, and side information;

FIG. 1b compression step 2: order reduction, decorrelation for the ambient HOA component, and perceptual coding of the two components;

fig. 2a decompression step 1: perceptually decoding the time domain signal, re-correlating the signal representing the residual ambient HOA component, and order boosting;

fig. 2b decompression step 2: composition of total HOA;

FIG. 3 HOA decomposition

FIG. 4 HOA composition

FIG. 5 spherical coordinate system

FIG. 6 normalization function v for different values of N_NExemplary curves of (Θ)

Detailed Description

Compression process

The compression process according to the invention comprises two successive steps shown in fig. 1a and 1b, respectively. The exact definition of the individual signals is described in the detailed description section of HOA decomposition and reassembly. A compressed frame-by-frame processing of non-overlapping input frames d (k) of a HOA coefficient sequence of length B is used, where k denotes the frame index. With respect to the HOA coefficient sequence specified in equation (42), the frame is defined as follows:

D(k):＝[d((kB+1)T_s)d((kB+2)T_s)…d((kB+B)T_s)](1)

wherein T is_sRepresenting the sampling period.

In fig. 1a, a frame d (k) of the HOA coefficient sequence is input to a dominant sound source direction estimation step or stage 11, which analyzes the HOA representation for the presence of dominant directional signals, estimating the direction of the dominant directional signals. The estimation of the direction can be performed, for example, by the procedure described in patent application EP 2665208 Al. The estimated direction is composed ofIs shown in whichRepresenting the maximum number of direction estimates. Assume that the estimated direction is set in the matrix as followsIn A (k):

it is implicitly assumed that the direction estimates are properly sorted by assigning them to direction estimates from previous frames. Thus, it is assumed that the time series of individual direction estimates describes the directional trajectory of the dominant sound source. In particular, if the d-th dominant sound source should not be operated, it may be passed throughA non-valid value is assigned to indicate it. Then, in a decomposition step or stage 12, use is made ofThe direction of medium estimation decomposes the HOA representation intoA maximum dominant directional signal X_DIR(k-1), some parameters describing the prediction of the spatial domain signal of the residual HOA component predicted from the dominant oriented signalAnd an ambient HOA component D representing the prediction error_A(k-2). A detailed description of the decompression is provided in the HOA decompression section.

In FIG. 1b, the directional signal X is shown_DIRPerceptual coding of (k-1) and residual ambient HOA component D_AAnd (k-2) perceptual coding. Directional signal X_DIR(k-1) is a conventional time domain signal that can be separately compressed using any existing perceptual compression technique. Ambient HOA domain component D_AThe compression of (k-2) is performed in two consecutive steps or stages. Order N of the ambisonics is performed in an order-reducing step or stage 13_REDIn which for example N_REDGet the ambient HOA component D as 1_A,RED(k-2). By the reaction of at D_ARetention of N in (k-2)_REDIndividual HOA coefficients and discarding other coefficients to achieve such an order reduction. On the decoder side, corresponding zero values are appended for the omitted values, as explained below.

It should be noted that the reduced order N is due to the smaller residual amount of total power and directionality of the residual ambient HOA component compared to the method in patent application EP 2665208 Al_REDIn general, it may be chosen smaller. The reduction of the order thus leads to smaller errors compared to patent application EP 2665208 Al.

In a subsequent decorrelation step or stage 14, the ambient HOA component D representing the step reduction is evaluated_A,REDDecorrelating the sequence of HOA coefficients of (k-2) to obtain a time-domain signal W_A,RED(k-2), the time domain signal W_A,RED(k-2) input toA parallel perceptual encoder(s) or a compressor 15 operating according to any known perceptual compression technique. Decorrelation is performed in order to avoid exposing perceptual coding noise when rendering the HOA representation after decompression (for its explanation see patent application EP 12305860.4). By mixing D_A,RED(k-2) transformation to O in the spatial domain_REDAn equivalent signal can be approximately decorrelated by applying the spherical harmonic transformation described in patent application EP 2469742 a 2.

Another alternative decorrelation technique is the Karhunen-loeve transform (KLT) described in patent application EP 12305860.4-it should be noted that for the last two decorrelations some kind of side information, denoted α (k-2), is to be provided to enable the recovery of the decorrelation in the HOA decompression stage.

In one embodiment, all time domain signals X are performed jointly_DIR(k-1) and D_A,REDThe perceptual compression of (k-2) to improve coding efficiency.

The output of the perceptual coding is a compressed directional signalAnd compressed ambient time domain signal

Step of decompression

The decompression process is illustrated in fig. 2a and 2 b. Like compression, the decompression process consists of two consecutive steps. In fig. 2a, the directional signal is performed in a perceptual decoding or decompression step or stage 21And a time domain signal representing the residual ambient HOA componentIs sensed byAnd (6) decompressing. Decompressing the resulting perceptually decompressed time domain signal in a re-correlation step or stage 22Performing a re-correlation to provide an order N_REDHOA representation of the residual componentOptionally, the re-correlation may be performed in an inverse manner to the two alternative procedures described for step/stage 14 using transmitted or stored (depending on the decorrelation method used) parameters α (k-2.) thereafter, in an order increase step or stage 23, by order increase, according to the order of the parameters usedEstimating a suitable HOA representation of order NOrder enlargement by appending corresponding 'zero' value rows toIs thus assumed to have zero values with respect to higher order HOA coefficients.

In fig. 2b, the dominant directional signal is decompressed in accordance with a composition step or stage 24Together with the corresponding directionAnd prediction parametersAnd from the residual ambient HOA componentTo reconstruct the overall HOA representation resulting in a frame of decompressed and reconstructed HOA coefficients

Performing all time-domain signals X jointly_DIR(k-1) and W_A,RED(k-2) in order to increase the coding efficiency, the compressed directional signals are also jointly performed in a corresponding mannerAnd compressed time domain signalPerceptual decompression.

A detailed description of the reorganization is provided in the HOA reorganization section.

HOA decomposition

A block diagram illustrating the operations performed for HOA decomposition is given in fig. 3. This operation is summarized as follows: first, a smoothed dominant directional signal X is calculated_DIR(k-1) and its output is used for perceptual compression. Then, from the O directional signalsTo represent the HOA representation D of the dominant directional signal_DIRThe residue between (k-1) and the original HOA representation D (k-1), where the O directional signals can be considered as substantially plane waves in uniformly distributed directions. According to the dominant directional signal X_DIR(k-1) these directional signals are predicted, and prediction parameters are outputtedFinally, the original HOA representation D (k-2) and the HOA representation D of the dominant directional signal are calculated and output_DIRResidue D between (k-1)_A(k-2) and HOA representation of the predicted directional signal in uniformly distributed directions

Before describing the details, it should be noted that during composition, a change in direction between successive frames can cause all the calculated signals to be interrupted. Therefore, an instantaneous estimate of the corresponding signal for the overlapping frame is first calculated, the instantaneous estimate being 2B in length. Second, the results of successive overlapping frames are smoothed using an appropriate windowing function. However, each smoothing introduces a single frame of hysteresis.

Computing instantaneous dominant directional signals

The current frame D (k) for the HOA coefficient sequence in step or stage 30 is based onThe calculation of the instantaneous dominant direction signal is based on pattern matching as described in the following documents: poletti, "Three-Dimensional Surround Systems Based on scientific Harmonics", J.Audio Eng. Soc,53(11), pages 1004-. In particular, a search is made for a directional signal for which the HOA representation yields the best approximation of a given HOA signal.

Furthermore, without loss of generality, it is assumed that a vector can uniquely specify each directional estimate of a valid dominant sound sourceThe vector includes a tilt angle θ according to the following formula_DOM，d(k)∈[0，π]And azimuth angle phi_DOM，d(k)∈[0，2π](for a schematic see FIG. 5):

first, according to

Calculating a mode matrix based on direction estimation of effective sound sources

In equation (4), D_ACT(k) Represents the number of valid directions for the k-th frame, and d_ACT，j(k)(1≤j≤D_ACT(k) Indicating their index.A real-valued spherical harmonic is represented, which is defined in the definition part of the real-valued spherical harmonic.

Second, a matrix is computed that defines the instantaneous estimates of all dominant directional signals for the (k-1) th frame and the k-th frame as follows

Wherein

This is achieved in two steps. In a first step, the directional signal samples in the row corresponding to the invalid direction are set to zero, i.e. zero

WhereinIndicating a set of valid directions. In a second step, directional signal samples corresponding to the effective direction are obtained by first arranging the directional signal samples corresponding to the effective direction in a matrix according to the following formula:

the matrix is then calculated such that the euclidean norm of the error is

And (4) minimizing. The solution is given by the following equation:

time smoothing

For step or stage 31, only for directional signalsSmoothing is explained because smoothing of other types of signals can be done in a completely similar way. The samples are contained in a matrix according to equation (6) by the following appropriate window functionDirectional signal estimation inWindowing is carried out:

the window function must satisfy the condition: its sum with its shifted version (assuming shift of B samples) in the following overlap region is '1':

a periodic Hann window defined by the following equation gives an example for such a window function:

the smoothed directional signal for the (k-1) th frame is computed by appropriate superposition of windowed instantaneous estimates according to the following equation:

the samples of all the smoothed directional signals for the (k-1) th frame are set in the following matrix:

wherein

Smoothed dominant directional signal X_DIR,d(l) Should be a continuous signal that is continuously input to the perceptual encoder.

Computing HOA representation of smoothed dominant directional signal

In step or phase 32, based on the continuous signal X_DIR,d(l) According to X_DIR(k-1) andthe HOA representation of the smoothed dominant directional signal is computed to mimic the same operations that would be performed for the HOA composition. Since a change in direction estimation between successive frames may cause an interruption, the instantaneous HOA representation of the overlapping frame of length 2B is again calculated and the result of successive overlapping frames is smoothed by using an appropriate window function. Thus, the HOA representation D is obtained by the following equation_DIR(k-1)：

D_DIR(k-1)＝Ξ_ACT(k)X_{DIR，ACT，WIN1}(k-1)+Ξ_ACT(k-1)X_{DIR，ACT，WIN2}(k-1) (18)，

Wherein the content of the first and second substances,

and is

Representing residual HOA representation by directional signals on a uniform grid

In step or stage 33, according to D_DIR(k-1) and D (k-1) (i.e., delayed by frame delay 381. D (k))^D(k)) A residual HOA representation represented by the directional signal on the uniform grid is computed. The purpose of this operation is: from a number of fixed, almost uniformly distributed directions(1. ltoreq. o.ltoreq.0, also called the grid direction) to represent the residue [ D (k-2) D (k-1)]-[D_DIR(k-2) D_DIR(k-1)]

First, with respect to the grid direction, the mode matrix xi is calculated as follows_GRID：

Wherein

Since the grid direction is fixed during the whole compression process, the mode matrix xi_GRIDOnly one calculation is needed.

The directional signals on the corresponding grid are obtained as follows:

predicting directional signals on a uniform grid from a dominant directional signal

In step or stage 34, according toAnd X_DIR(k-1), predicting the directional signal on the uniform grid. In the direction of the grid according to the directional signal(1. ltoreq. o.ltoreq.0) is based on two successive frames for smoothing purposes, i.e. the grid signalThe unrolled frame (length 2B) is an unrolled frame from the smoothed dominant directional signal:

and (4) predicting.

First, it is contained inEach of the grid signals(1. ltoreq. o.ltoreq.0) to be included inDominant directional signal inIn (1). The assignment may be based on a calculation of a normalized cross-correlation function between the grid signal and all dominant directional signals. In particular, the dominant directional signal is assigned to the trellis signal, which provides the highest value of the normalized cross-correlation function. The result of the assignment may be determined by assigning the o-th trellis signal to the o-th trellis signalDistribution function of dominant directional signalsTo indicate.

Second, by means of the assigned dominant directional signalTo predict each mesh signalAccording to the distributed dominant directional signalsBy delaying and scaling, the predicted trellis signal is processed as followsAnd (3) calculating:

wherein, K_o(k-1) denotes a scaling factor and Δ_o(k-1) indicates sample delay. These parameters are selected to minimize the prediction error.

If the power of the prediction error is greater than the power of the trellis signal itself, it is assumed that the prediction has failed. The corresponding prediction parameters may then be set to any non-valid values.

It should be noted that other types of prediction are also possible. For example, instead of calculating a full-band scaling factor, it is also possible to determine the scaling factor for the perceptual orientation band. However, this operation improves the prediction at the cost of an increased amount of side information.

All prediction parameters can be set in a parameter matrix as follows:

assuming all predicted signals(1. ltoreq. o.ltoreq.0) is arranged in a matrixIn (1).

Computing HOA representation of directional signals on a predicted uniform grid

In step or stage 35, according to the following formula, according toComputing HOA representation of predicted mesh signal:

computing HOA representation of residual ambient sound field components

In step or stage 37, by the formula:

according toTime smoothed version of (in step/stage 36)Two frame delayed versions (delays 381 and 383) according to D (k) D (k-2), and D_DIRFrame delayed version of (k-1) (delay 382) D_DIR(k-2) computing the HOA representation of the residual ambient sound field component.

HOA representation

Before describing the process in detail at various steps or stages in fig. 4, a summary is provided. Using prediction parametersFrom the decoded dominant directional signalPredicting directional signals with respect to uniformly distributed directionsNext, the overall HOA representationHOA representation by dominant directional signalsHOA representation of predicted directional signalsAnd residual ambient HOA componentAnd (4) forming.

Computing HOA representation of dominant directional signal

Will be provided withAndis input to a step or stage 41 for determining the HOA representation of the dominant directional signal. After having estimated according to the directionAndcalculating the mode matrix xi_ACT(k) Xi and xi_ACT(k-1) thereafter, based on the direction estimates of the effective sound field for the kth and (k-1) th frames, an HOA representation of the dominant directional signal is obtained by the following equation:

wherein the content of the first and second substances,

and is

Predicting directional signals on a uniform grid from a dominant directional signal

Will be provided withAndis input to step or stage 43 for predicting the directional signal on the uniform grid from the dominant directional signal. The expanded frame of the directional signal on the predicted uniform grid is formed by cells according to the following equationConsists of the following components:

the unitIs predicted from the dominant directional signal by the following equation:

computing HOA representation of directional signals on a predicted uniform grid

In the step or stage 44 of computing the HOA representation of the predicted directional signal on the uniform grid, by means of an equationTo obtain a HOA representation of the predicted grid orientation signal, wherein xi_GRIDThe pattern matrix is represented with respect to the predefined grid directions (see equation (21) for definition).

Composing HOA sound field representation

In step or stage 46, according to the following equation(i.e., delayed by frame delay 42)) (is in step/stage 45)Time-smoothed version of (1)Andto finally compose an overall HOA generation representation:

fundamental principle of higher order ambisonics

Higher order ambisonics is based on the description of the sound field in a compact region of interest, assuming no sound sources in the compact region. In this case, in the region of interest, the time-space characteristics of the sound pressure p (t, x) at time t and position x are physically determined entirely by the uniform wave equation. The following is based on the spherical coordinate system shown in fig. 5. The X-axis points to the front position, the y-axis points to the left, and the z-axis points upward. Passing through radius r>0 (i.e., distance to origin of coordinates), fromThe inclination angle theta measured by the polar axis z belongs to [0, pi ]]And an azimuth angle φ e [0, π ] measured counterclockwise from the x-axis in the x-y plane]To indicate the position in space x ═ (r, θ, φ)^T。(·)^TIndicating transposition.

It can be seen (see E.G.Williams, "Fourier Acoustics", volume 93 of applied mathematical Sciences, Academic Press,1999) that the Fourier transform of sound pressure with respect to time (fromExpress), that is

(where ω represents angular frequency and i represents imaginary unit) can be expanded into a series of spherical functions as follows

Wherein c is_sRepresents the velocity of sound, and k represents the angular wavenumber, which is given by the formulaRelated to ω, j_n(. represents a first type of spherical Bessel function, anda real-valued spherical harmonic with an order of n and an angle of m (defined in the real-valued spherical harmonic part) is represented. Coefficient of expansionDepending only on the angular wavenumber k. It is to be noted that it has been implicitly assumed here that the sound pressure is spatially band-limited. Thus, the series is truncated with respect to the order index N at an upper bound N, referred to as the order of the HOA representation.

If the Sound Field is represented by an infinite number of superpositions of harmonic Plane waves of different angular frequencies ω and the Sound Field can arrive from all possible directions specified by the angular tuple (θ, φ), it can be seen (see B. Rafaely, "Plane-wave Decomposition of the Sound Field on a Sphere by Sphere spatial convention", J.Acoust. Soc. am.,4(116), pages 2149-:

wherein the coefficient of expansionBy the following equation and expansion coefficientAnd (3) correlation:

assuming individual coefficientsIs a function of the angular frequency omega, inverse Fourier transform (fromRepresentation) provides each order n and angle m with the following time-domain function:

the functions may be collected in a single vector as follows:

the time-domain function in the vector d (t) is given by n (n +1) +1+ mIs indexed by the location of the location.

Final ambisonic format provides for use of sampling frequency f_SThe sampled versions of (d), (t) are as follows:

wherein T is_S＝1/f_SRepresenting the sampling period. d (lT)_S) The unit is called the ambisonic coefficient. It is noted that the time domain signalAnd thus the ambisonic coefficient is real-valued.

Definition of real-valued spherical harmonics

Real value spherical harmonic functionGiven by the following equation:

wherein

Using Legendre polynomials P_n(x) And unlike the above-mentioned e.g. williams textbook, without using the Condon-Shortley term, the associated Legendre function P is defined as in the following equation_n，m(X)：

Spatial resolution of higher order ambisonics

From direction Ω₀＝(θ₀，φ₀)^TThe arriving plane wave function x (t) is represented in HOA by the following equation:

amplitude of plane waveIs given by the following equation:

it can be seen from equation (48) that it is the generally plane wave function x (t) and the spatial dispersion function v_NProduct of (Θ), the spatial dispersion function v_N(Θ) can be seen as being dependent only on Ω and Ω₀The angle Θ between has the following characteristics:

cosΘ＝cosθcosθ₀+cos(φ-φ₀)sinθsinθ₀(49)。

as expected, under the constraint of infinite order, N → ∞, the spatial dispersion function is converted to a dirac delta function δ (·), i.e., the

However, in the case of finite order N, from the direction Ω₀The contribution of the substantially plane wave of (a) is smeared into adjacent directions and the degree of blurring decreases with increasing step. The normalization function v for different values of N is shown in FIG. 6_N(Θ). It should be noted that the direction Ω of the temporal characteristic of the spatial density of any plane wave amplitude is a multiple of its characteristic in any other direction. In particular, for some fixed directions Ω₁And Ω₂Function d (t, Ω)₁) And d (t, Ω)₂) Are highly correlated with respect to time t.

Discrete spatial domain

If the spatial density of the plane wave amplitudes is in the O number of spatial directions omega which are distributed almost uniformly over the unit sphere₀(1. ltoreq. o.ltoreq.0) are discrete, O directional signals d (t, omega) are obtained_o). These signals are assembled into a vector as in the following equation:

d_SPAT(t)：＝[d(t，Ω₁)...d(t，Ω_O)]^T(51)

it can be demonstrated by using equation (47) that the vector can be calculated from the continuous ambisonic representation d (t) defined in equation (41) by a single matrix multiplication whose equation is:

d_SPAT(t)＝Ψ^Hd(t)， (52)

wherein (·)^HJoint permutation and conjugation is indicated, and Ψ represents a mode matrix defined by the following equation:

Ψ：＝[S₁... S_O](53)，

wherein

Due to the direction omega₀Are almost uniformly distributed on the unit sphere, so the mode matrix is generally invertible. Thus, by the equation

d(t)＝Ψ^-Hd_SPAT(t) (55)

According to the directional signal d (t, omega)_o) A continuous ambisonic representation may be calculated. Two equations form the transform and inverse transform between the ambisonic representation and the spatial domain. In this application, these transforms are referred to as spherical harmonic transforms and inverse spherical harmonic transforms.

Because of the direction omega on a unit sphere₀Is almost uniformly distributed, Ψ^H≈Ψ^-1(56)

This demonstrates the use of Ψ in equation (52)^-1Without using Ψ^HIs feasible. Advantageously, all of the above relationships are valid for the discrete time domain as well.

On the encoding side as well as on the decoding side, the inventive process may be performed by a single processor or circuit, or by several processors or circuits operating in parallel and/or in different parts of the inventive process.

The invention can be used to process corresponding sound signals that can be rendered or played on a loudspeaker device in a home environment or a loudspeaker device in a cinema.

Claims (6)

1. A method for compressing a Higher Order Ambisonic (HOA) representation of a soundfield, the method comprising:

estimating a leading sound source direction according to the current time frame of the HOA coefficient;

decomposing the HOA representation into a dominant directional signal in the time domain and a residual HOA component, wherein the residual HOA component is transformed to the discrete spatial domain to obtain a plane wave function in a uniform sampling direction representing the residual HOA component, and wherein the plane wave function is predicted from the dominant directional signal, thereby providing parameters describing the prediction;

decorrelating the reduced order residual HOA components to obtain corresponding residual HOA component time domain signals;

perceptually encoding the dominant directional signal and the residual HOA component time domain signal to determine a compressed dominant directional signal and a compressed residual component signal.

2. The method of claim 1, wherein the decomposing comprises:

calculating a dominant directional signal according to the estimated sound source direction of the current frame of the HOA coefficient;

temporally smoothing the dominant directional signal to determine a smoothed dominant directional signal;

calculating an HOA representation of the smoothed dominant directional signal from the estimated sound source direction and the smoothed dominant directional signal;

representing a corresponding residual HOA representation by a directional signal on a uniform grid;

predicting the directional signal on a uniform grid from said smoothed dominant directional signal and said residual HOA representation represented by the directional signal, thereby computing a predicted HOA representation of the directional signal on the uniform grid, followed by temporal smoothing;

the HOA representation of the residual ambient sound field component is computed from the directional signal on the smoothed predicted uniform grid, the two frame delayed version of the current frame of HOA coefficients, and the frame delayed version of the smoothed dominant directional signal.

3. An apparatus for compressing a Higher Order Ambisonic (HOA) representation of a soundfield, the apparatus comprising:

an estimator for estimating a dominant sound source direction according to a current time frame of the HOA coefficient;

a decomposer decomposing the HOA representation into a dominant directional signal in the time domain and a residual HOA component, wherein the residual HOA component is transformed to the discrete spatial domain in order to obtain a plane wave function in a uniform sampling direction representing the residual HOA component, and wherein the plane wave function is predicted from the dominant directional signal, thereby providing parameters describing the prediction;

a decorrelator for decorrelating the reduced-order residual HOA component to obtain a corresponding residual HOA component time domain signal;

an encoder that perceptually encodes the dominant directional signal and the residual HOA component time domain signal to provide a compressed dominant directional signal and a compressed residual component signal.

4. The apparatus of claim 3, wherein the decomposer is further configured to:

calculating a dominant directional signal according to the estimated sound source direction of the current frame of the HOA coefficient;

time smoothing the dominant directed signal to obtain a smoothed dominant directed signal;

calculating an HOA representation of the smoothed dominant directional signal from the estimated sound source direction and the smoothed dominant directional signal;

representing a corresponding residual HOA representation by a directional signal on a uniform grid;

predicting the directional signal on a uniform grid from said smoothed dominant directional signal and said residual HOA representation represented by the directional signal, thereby computing a predicted HOA representation of the directional signal on the uniform grid, followed by temporal smoothing;

the HOA representation of the residual ambient sound field component is computed from the directional signal on the smoothed predicted uniform grid, the two-frame delayed version of the current frame of HOA coefficients, and the frame delayed version of the smoothed dominant directional signal.

5. A method for decompressing a compressed Higher Order Ambisonic (HOA) representation, the method comprising:

perceptually decoding the compressed dominant directional signal and the compressed residual component signal, thereby providing a decompressed dominant directional signal and a decompressed time domain signal representing the residual HOA component in the spatial domain;

re-correlating said decompressed time domain signal to obtain a corresponding reduced order residual HOA component;

providing a decompressed residual HOA component by increasing the corresponding reduced order residual HOA component to an original order;

determining a predicted directional signal based on at least one parameter;

determining an HOA sound field representation based on the decompressed dominant directional signal, the predicted directional signal and the decompressed residual HOA component.

6. An apparatus for decompressing a Higher Order Ambisonic (HOA) representation, the apparatus comprising:

a decoder for perceptually decoding the compressed dominant directional signal and the compressed residual component signal, thereby providing a decompressed dominant directional signal and a decompressed time domain signal representing the residual HOA component in the spatial domain;

a re-correlator that re-correlates the decompressed time domain signal to obtain a corresponding reduced order residual HOA component;

a processor configured to provide decompressed residual HOA components by increasing the corresponding reduced-order residual HOA components to original order, the processor further configured to determine a predicted directional signal based on at least one parameter;

wherein the processor is further configured to determine a HOA sound field representation based on the decompressed dominant directional signal, the predicted directional signal and the decompressed residual HOA component.