RU2833441C1 - Method and apparatus for decoding compressed audio representation of audio or sound field using hoa - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: present invention relates to sound recording technologies. When compressing the HOA data frame representation, a gain control is applied for each channel signal before its perceptual coding. Gain values are transmitted in differential mode as side information. However, to start decoding such a streaming compressed representation of a HOA data frame, absolute gain values are required, which must be encoded with a minimum number of bits. To determine such a smallest integer ( β _e ) bits representation of the HOA data frame ( C ( k )) is reproduced in the spatial domain in the form of signals of virtual loudspeakers lying on the unit sphere, with subsequent normalization of HOA data frame representation ( C ( k )). Then the least integer number of bits is taken equal to: ( ΑΑ ).

EFFECT: providing the least integer number of bits required to represent undifferentiated gain values.

3 cl, 6 dwg

Description

Field of technology

The present invention relates to a method and device for determining, for compressing the representation of a HOA data frame, the smallest integer number of bits required to represent non-differentiable gain values associated with channel signals of a specific one of said HOA data frames.

State of the art

Higher Order Ambisonics (HOA) is one of the possibilities for representing stereo sound. Other methods are wave field synthesis (WFS) or channel-based approaches such as 22.2. Unlike channel-based methods, HOA-based representation offers the advantage of not being dependent on a specific loudspeaker setup. However, this flexibility comes at the expense of the decoding process that is required to reproduce the HOA representation on a specific loudspeaker setup. Compared to the WFS approach, where the number of loudspeakers required is usually very large, HOA can also be reproduced in setups consisting of only a few loudspeakers. An additional advantage of HOA is that the same representation can also be used without any modifications for stereo reproduction on headphones.

The HOA technology is based on the representation of the spatial density of complex amplitudes of a harmonic plane wave by a truncated decomposition in spherical harmonics (SH). Each decomposition coefficient is a function of angular frequency, which can be equivalently represented by a time-domain function. Therefore, without loss of generality, the complete HOA sound field representation can indeed be considered as consisting of O time-domain functions, where O denotes the number of decomposition coefficients. These time-domain functions will henceforth be referred to equivalently as HOA coefficient sequences or HOA channels.

The spatial resolution of the HOA representation improves with increasing maximum order N expansion. Unfortunately, the number of expansion coefficients O grows quadratically with order N , in particular, O = ( N + 1)². For example, typical HOA representations using the order N = 4, require HOA coefficients (expansion) O = 25. Total bit rate for transmitting the HOA representation, given the desired single-channel sampling frequency f _s and the number of bits N _b in the sample, is defined as O · f _S · N _b . Transfer of HOA representation with order N = 4 with sampling frequency f _s = 48 kHz, using N _b = 16 bits per sample results in a bit rate of 19.2 Mbps, which is very high for many practical applications, such as streaming data. Thus, compression of the HOA representation is highly desirable.

Previously, compression of HOA sound field representations was proposed in EP 2665208 A1, EP 2743922 A1, EP 2800401 A1, cf. ISO/IEC JTC1/SC29/WG11, N14264, WD1-HOA Text of MPEG-H 3D Audio, January 2014. These approaches have in common that they perform sound field analysis and decompose the given HOA representation into a directional component and a residual ambience component. The final compressed representation is assumed to consist, on the one hand, of a number of quantized signals resulting from perceptual coding of directional and vectorial cues, as well as relevant sequences of HOA ambience component coefficients. On the other hand, it contains additional side information related to the quantized signals, where this side information is required to reconstruct the HOA representation from its compressed version.

Before being passed to the perceptual encoder, these intermediate time domain signals shall have a maximum amplitude within the range of values [-1,1[, which is a requirement arising from the implementation of currently available perceptual coders. To satisfy this requirement when compressing HOA representations, a gain controller processing unit (see EP 2824661 A1 and the above-mentioned ISO/IEC JTC1/SC29/WG11 N14264) is used in front of the perceptual coders, which smoothly attenuate or amplify the input signals. The resulting signal change is assumed to be reversible and is applied frame by frame, wherein in particular the change in signal amplitudes between successive frames is assumed to be a power of "2". To facilitate the inverse transformation of this signal change in the HOA-based decompressor, the corresponding normalization of the side information is included in the total side information. This normalization of the side information may consist of base-2 metrics, where the metrics describe the relative change in amplitude between two successive frames. These metrics are encoded using run-length coding according to the above-mentioned ISO/IEC JTC1/SC29/WG11 N14264 document, since small changes in amplitude between successive frames are more likely than large changes.

Disclosure of the essence of the invention

Using differentially coded amplitude changes to reconstruct the original signal amplitudes in HOA decompression is possible, for example, in the case where a single file is decompressed from start to finish without any time jumps. However, to facilitate random access, independent access units (usually a bit stream) must be present in the coded representation to ensure that decompression starts from the desired position (or at least in its immediate vicinity), regardless of the information from previous frames. Such an independent access unit must contain the total absolute amplitude change (i.e., the non-differentiable gain value) caused by the gain controller processing block from the first frame to the current frame. Assuming that the amplitude changes between two consecutive frames are a power of 2, this is sufficient to describe the total absolute amplitude change using a base 2 exponent. To efficiently encode this exponent, it is important to know the possible maximum gains of the signals before applying the gain controller processing block. However, this information is highly dependent on the specification of the limits on the range of values of the HOA representations to be compressed. Unfortunately, the document concerning the MPEG-H 3D audio standard ISO/IEC JTC1/SC29/WG11 N14264 only provides a description of the format for input of the HOA representation, without specifying any limits on the range of values.

The problem solved by the invention is to provide the smallest integer number of bits necessary to represent non-differentiable gain values. This problem is solved by the method disclosed in claim 1 of the invention formula. The device that uses this method is disclosed in claim 2 of the invention formula.

The advantages of additional embodiments are disclosed in the corresponding dependent claims.

The invention establishes a relationship between the range of values of the input representation of the HOA and the possible maximum gain factors of signals before applying the gain control processing unit in the compressor based on the HOA.

Based on the relationship, the number of bits required (for a given specification in the range of values of the HOA input representation) is determined for the efficient coding of base-2 metrics for describing, within an access block, the total absolute changes in amplitude (i.e., the non-differentiable value of the gain) of the changed signals caused by the gain controller processing block from the first frame to the current frame.

Furthermore, after the rule for calculating the number of required bits for encoding the indicator is satisfied, the invention uses processing to check whether the given HOA representation satisfies the required value range constraints so that it can be compressed properly.

In principle, the method according to the invention is suitable for determining the compression of the representation of a HOA data frame of the smallest integer number β _e of bits required for the representation of non-differentiable gain values for channel signals of a specific one of said HOA data frames, in which each channel signal in each frame comprises a group of sample values, and in which each channel signal of each one of said HOA data frames is assigned a differentiable gain value, and such differentiable gain value causes a change in the amplitudes of the sample values of the channel signal in the current HOA data frame relative to the sample values of this channel signal in the previous HOA data frame, and wherein such channel signals with an adapted gain are encoded in the encoding device,

and wherein said representation of the HOA data frame is reproduced in the spatial domain for O virtual loudspeaker signals w _j ( t ), where the positions of the virtual loudspeakers lie on a unit sphere and are to be distributed uniformly over this unit sphere, wherein said reproduction is represented by the product of matrices w ( t ) = ( Ψ ) ^-1 • c ( t ), where w ( t ) is a vector containing all the virtual loudspeaker signals, Ψ is a mode matrix of the virtual loudspeaker positions, and c ( t ) is a vector of corresponding sequences of HOA coefficients of said representation of the HOA data frame,

and the specified HOA data frame representation was normalized so that ,

The specified method includes the following steps:

- generating said channel signals in one or more sub-steps a), b), c) from said normalized representation of the HOA data frame:

a) for representing dominant audio signals in said channel signals, multiplying said vector of HOA coefficient sequences c ( t ) by a mixing matrix Α , wherein the Euclidean norm of the mixing matrix Α is not greater than "1", wherein the mixing matrix Α represents a linear combination of the coefficient sequences of said normalized representation of the HOA data frame;

b) for representing an ambient component c _AMB ( t ) in said channel signals, subtracting said dominant audio signals from said normalized representation of the HOA data frame, and selecting at least a portion of the coefficient sequences of said ambient component c _AMB ( t ), where || c _AMB ( t )|| ₂ ² ≤ || c ( t )|| ₂ ² , and transforming the resulting minimum ambient component c _AMB,MIN ( t ) by computing w _MIN (t) = Ψ ^-1 _MIN ⋅ c _AMB,MIN ( t ), where

|| Ψ ^-1 _MIN || ₂ < 1 and Ψ _MIN is the mode matrix for the specified minimum component of the environment c _AMB,MIN ( t );

c) selecting a portion of the said sequences of HOA coefficients c ( t ), wherein the selected sequences of coefficients are related to the sequences of coefficients of the HOA environmental component to which the spatial transformation has been applied, and the minimum order N _MIN describing the number of said selected sequences of coefficients is N _MIN ≤ 9 ;

- establishing the specified smallest integer number β _e of bits required to represent the specified non-differentiable gain values for the specified channel signals up to ,

Where , N is the order, N _MAX is the maximum order of interest, directions of the specified virtual loudspeakers, O = ( N + l) ² is the number of sequences of HOA coefficients, and K is the ratio between the square of the Euclidean norm || Ψ || ₂ ² of the specified mode matrix and O .

In essence, the device according to the invention is suitable for determining, when compressing the representation of a HOA data frame, the smallest integer number β _e of bits required to represent non-differentiable gain values for channel signals of a particular one of said HOA data frames, in which each channel signal in each frame comprises a group of sample values, and in which each channel signal of each one of said HOA data frames is assigned a differentiable gain value, and such differentiable gain value causes a change in the amplitudes of the sample values of the channel signal in the current HOA data frame relative to the sample values of this channel signal in the previous HOA data frame, and wherein such channel signals with an adapted gain are encoded in the encoding device,

and wherein said representation of the HOA data frame is reproduced in the spatial domain for O virtual loudspeaker signals w _j ( t ), where the positions of the virtual loudspeakers lie on a unit sphere and are to be distributed uniformly over this unit sphere, wherein said reproduction is represented by the product of matrices w ( t ) = ( Ψ ) ^-1 • c ( t ), where w ( t ) is a vector containing all the virtual loudspeaker signals, Ψ is a mode matrix of the virtual loudspeaker positions, and c ( t ) is a vector of corresponding sequences of HOA coefficients of said representation of the HOA data frame,

and the specified HOA data frame representation was normalized so that ,

The specified device includes:

- means that generate the specified channel signals in one or more operations a), b), c) from the specified normalized representation of the HOA data frame:

a) for representing dominant audio signals in said channel signals, multiplying said vector of HOA coefficient sequences c ( t ) by a mixing matrix Α , wherein the Euclidean norm of the mixing matrix Α is not greater than "1", wherein the mixing matrix Α represents a linear combination of the coefficient sequences of said normalized representation of the HOA data frame;

b) to represent a component of the environment c _AMB ( t ) in said channel signals, subtracting said dominant audio signals from said normalized representation of the HOA data frame, and selecting at least a portion of the coefficient sequences of said environment component c _AMB ( t ), and || c _AMB ( t )||₂ ²≤ || c ( t )||₂ ², and the transformation of the resulting minimum component of the environment c _AMB,MIN ( t ) by calculation w _MIN ( t )= Ψ ^-1 _MIN · c _AMB,MIN ( t ), Where

|| Ψ ^-1 _MIN || ₂ < 1, and Ψ _MIN is the mode matrix for the specified minimum component of the environment c _AMB,MIN ( t ) ;

c) selecting a portion of the said sequences of HOA coefficients c ( t ), wherein the selected sequences of coefficients are related to the sequences of coefficients of the HOA environmental component to which the spatial transformation has been applied, and the minimum order N _MIN describing the number of said selected sequences of coefficients is N _MIN ≤ 9 ;

- means that specify the specified smallest integer number β _e of bits required to represent the specified non-differentiable gains for the specified channel signals ,

Where , N is the order, N _MAX is the maximum order of interest, - the directions of the specified virtual loudspeakers, O = ( N + 1) ² is the number of HOA coefficient sequences, and K is the ratio between the square of the Euclidean norm || Ψ || ₂ ² of the specified mode matrix and O .

Brief description of the drawings

Examples of embodiments of the invention are described with reference to the accompanying drawings, which show:

Fig. 1 shows a compressor based on HOA;

Fig. 2 shows a decompressor based on HOA;

Fig. 3 shows the values of the scale factors K for the virtual directions Ωj ^{( N )} , 1 ≤ j ≤ O , for the HOA orders N = 1, … ,29;

Fig. 4 shows the Euclidean norms of the inverse mode matrices Ψ ^{- 1} for the virtual directions Ω _{MIN, d} , d = 1, … , O _MIN for the orders HOA N _MIN = 1, … ,9;

Fig. 5 shows the definition of the maximum permissible amplitudes γ _{d B} virtual loudspeaker signals in positions Ωj ^{( N )} , 1 ≤ j < O , Where O =( N + 1)²;

Fig. 6 shows a spherical coordinate system.

Description of embodiments

Even if not explicitly described, the following embodiments can be used in any combination or subcombination.

In the following, the principle of HOA compression and decompression is presented in order to provide a more detailed context in which the above problem arises. The basis for this presentation is the processing described in the document concerning MPEG-H 3D audio of the ISO/IEC JTC1/SC29/WG11 standard N14264, see also EP 2665208 A1, EP 2800401 A1 and EP 2743922 A1. In the document N14264, the expression "directional component" is extended to "dominant audio component". For the directional component, it is assumed that the dominant audio component is partially represented by directional cues, which means monophonic cues with the corresponding direction from which they are supposed to affect the listener, together with some calculation parameters for calculating parts of the original HOA representation from the directional cues. In addition, it is assumed that the dominant sound component is represented by "vector signals", which means monophonic signals with a corresponding vector that determines the directional distribution of the vector signals.

HOA compression

The general architecture of a HOA-based compressor described in EP 2800401 A1 is shown in Fig. 1. It has a HOA spatial coding part shown in Fig. 1A and a perceptual coding part and a source coding part shown in Fig. 1B. The HOA spatial coder produces a first compressed HOA representation consisting of I signals together with side information describing how to produce their HOA representation. In the perceptual and side information source coders, the I signals are perceptually coded and the side information is source coded before the two coded representations are multiplexed.

Spatial encoding of HOA

In the first stage, the current k -th frame C ( k ) of the original HOA representation is input to the direction and vector evaluation processing stage or step 11, which involves creating sets of sequences M _DIR ( k ) and M _VEC ( k ). The set of sequences M _DIR ( k ) consists of sequences whose first element denotes the index of the directional signal, and whose second element denotes the corresponding quantized direction. The set of sequences M _VEC ( k ) consists of sequences whose first element denotes the index of the vector signal, and whose second element denotes the vector defining the distribution of the signals by directions, i.e., how the HOA representation of the vector signal is calculated.

Using both sets of sequences M _DIR ( k ) and M _VEC ( k ), initial frame HOA C ( k ) is decomposed at stage or stage 12 of HOA decompression into the frame X _P.S. ( k - 1) of all dominant sound (i.e. based on direction and vector) signals, and the frame C _AMB ( k - 1) HOA environment component. It should be noted that there is a one frame delay, which occurs due to the superposition-addition processing to avoid blocking artifacts. In addition, it is assumed that the HOA decompression stage/step 12 outputs some calculated parameters ζ ( k - 1), describing how to calculate parts of the original HOA representation from the directional signals in order to enrich the dominant sound component of the HOA. Additionally, it is assumed that a target assignment vector should be created v _A,T ( k - 1), containing information on the distribution of dominant sound signals that were determined at stage or step 12 of processing with HOA decomposition, for I available channels. The affected channels can be considered as busy, meaning that they are not available to transmit any sequences of HOA environment component coefficients in the corresponding time frame.

In the step or stage 13 of the environment component change processing, the frame C _AMB ( k - 1) of the environment component HOA is changed in accordance with the information represented by the target assignment vector v _A,T ( k - 1). In particular, it is determined which sequences of coefficients of the environment component HOA are to be transmitted in these I channels, depending (among other aspects) on the information (contained in the target assignment vector v _A,T ( k - 1) on which channels are available and not yet occupied by dominant audio signals. In addition, a smooth increase and a smooth decrease of the coefficient sequences is performed if the indices of the selected coefficient sequences change between successive frames.

Furthermore, it is assumed that the first sequences of coefficients O _MIN of the environmental component HOA C _AMB ( k - 2) are always chosen to be perceptually encoded and transmitted, where O _MIN = ( N _MIN + 1) ² with N _MIN ≤ N , typically of lower order than in the original HOA representation. In order to decorrelate these sequences of HOA coefficients, they can be transformed in step/stage 13 into directional signals (i.e., a generalized planar wave function) incident from some predetermined directions Ω _{MIN, d} , d = 1, … , O _MIN .

Together with the changed environmental component HOA C _M,A ( k - 1 ), the time-estimated changed environmental component HOA C _{P , M,A} ( k - 1) is calculated at step/stage 13 and used at the gain controller processing stages or steps 15, 151 to provide a rational preview, wherein the information about the change in the environmental component HOA is directly related to the distribution of all possible signal types over the available channels at the channel distribution stage or step 14. It is assumed that the final information about this distribution is contained in the final assignment vector v _A ( k - 2). To calculate this vector at step/stage 13, the information contained in the target assignment vector v _A,T ( k - 1) is used.

The channel allocation at stage/step 14 is established, using the information represented by the assignment vector v _A ( k - 2), of the corresponding signals contained in the frame X _PS ( k - 2) and contained in the frame C _M,A ( k - 2) for I available channels, yielding signal frames y _i ( k - 2), i = 1, … , I . In addition, the corresponding signals contained in the frame X _PS ( k - 1) and in the frame C _P,AMB ( k - 1) are also allocated to I available channels, yielding calculated signal frames y _{P, i} ( k - 1), i = 1, … , I .

Each of the signal frames y _i ( k - 2), i = 1, … , I , is finally processed by the gain controller 15, 151, resulting in the exponents e _i ( k - 2) and the exclusion flags β _i ( k - 2), i = 1, … , I , and in the signals z _i ( k - 2), i = 1, … , I , in which the signal gain is smoothly variable, so as to achieve a range of values that is suitable for the stages or steps 16 of the perceptual encoder. The output signal of the stages/steps 16 corresponds to the encoded signal frames , i = 1, … , I .

The calculated frames of the signals y _{P, i} ( k - 1), i = 1, … , I provide a preview view to avoid abrupt gain changes between successive blocks. The side information data M _DIR ( k - 1), M _VEC ( k - 1), e _i ( k - 2), β _i ( k - 2), ζ ( k - 1 ) and v _A ( k - 2) are source coded in the side information source encoder stage or stage 17, resulting in a coded side information frame . In the multiplexer 18 coded signals frames ( k - 2) and encoded side information data for this frame are combined, resulting in the output frame .

It is assumed that in the spatial decoder HOA, the gain changes at stages/steps 15, 151 are fed back by using the gain controller side information consisting of the exponents e _i ( k - 2) and the exclusion flags β _i ( k - 2), i = 1, … , I .

HOA Decompression

The general architecture of the HOA-based decompressor described in EP 2800401 A1 is shown in Fig. 2. It consists of analogs of the HOA-based compressor components, which are arranged in reverse order and include a perceptual decoding part and a source decoding part, shown in Fig. 2A, and a spatial decoding part of the HOA, shown in Fig. 2B.

In the perceptual decoding part and the source decoding part (representing the source decoder of perceptual and side information), the demultiplexing stage or step 21 receives an input framefrom a bit stream and provides a perceptually encoded representation, i = 1, … , I For I signals and coded data of side information, describing how to create their HOA representation. Signalsare perceptually decoded at stage or step 22 of the perceptual decoder, resulting in decoded signals, i = 1, … , I . Coded side information dataare decoded at stage or step 23 of the side information source decoder, resulting in data sets M _DIR ( k + 1), M _VEC ( k + 1), indicators e _i ( k ), exception flags β _i ( k ), calculation parameters ζ ( k + 1) and the destination vector v _AMB,ASSIGN ( k ). Regarding the difference between v _A And v _AMB,ASSIGN , see the above mentioned document N14264 regarding MPEG.

Spatial Decoding of HOA

In the spatial decoding part of HOA, each of the perceptually decoded signals , i = 1, … , I , is input to the gain controller inversion processing stage or step 24, 241 together with the associated gain correction index e _i ( k ) and the gain correction exclude flag β _i ( k ). At the i -th gain controller inversion processing stage/step, a frame of the corrected gain signal is created .

All I frames of adjusted gain signals, i = 1, … , I ,are submitted together with the destination vector v _AMB,ASSIGN ( k ) and sets of sequences M _DIR ( k + 1) and M _VEC ( k + 1) to the stage or step 25 of channel redistribution, see the above-described definition of sequence sets M _DIR ( k + 1) and M _VEC ( k + 1). Destination vector v _AMB,ASSIGN ( k ) consists of I components that indicate for each transmission channel whether it contains a sequence of HOA environment component coefficients and which one it contains. At channel reallocation stage 25, frames of the adjusted gain signalsare redistributed to reconstruct the frameall dominant audio signals (i.e. all direction and vector based signals) and frame C _I,AMB ( k ) intermediate representation of the HOA environment component. In addition, the set J _AMB,ACT ( k ) indicators of the sequences of coefficients of the component of the environment HOA is active in k -m frame, and data sets are presented J _E ( k - 1), J _D ( k - 1) and J _U ( k - 1) the coefficients of the HOA environmental component that must be turned on, off and remain active in ( k - 1)-th frame.

At stage or step 26 of the dominant sound synthesis, the HOA representation of the dominant sound component calculated from the frame of all dominant audio signals using a set of sequences M _DIR ( k + 1 ), a set ζ ( k + 1 ) of estimated parameters, a set of sequences M _VEC ( k + 1 ) and data sets J _E ( k - 1 ), J _D ( k - 1 ) and J _U ( k - 1 ).

At stage or step 27 of the synthesis of the environment of the component of the environment HOA frame is created from frame C _I,AMB ( k ) of the intermediate representation of the HOA environment component using the set J _AMB,ACT ( k ) of indices of the HOA environment component coefficient sequences that are active in the k -th frame. A delay of one frame is introduced due to synchronization with the dominant HOA audio component. Finally, in the HOA environment component mixing stage or step 28, the HOA frame and frame dominant HOA audio component are superimposed in such a way as to create a decoded HOA frame .

The HOA spatial decoder then creates a reconstructed HOA representation from the I signals and side information.

In the case where the HOA environment component has been transformed into directional signals on the encoding side, this transformation is inverted on the decoder side at stage/step 27.

The possible maximum gains of the signals before the gain control processing stages 15, 151 in the HOA-based compressor depend to a large extent on the range of values of the HOA input representation. Therefore, a meaningful range of values for the HOA input representation is first determined, followed by a conclusion on the possible maximum gains of the signals before entering the gain control processing stages.

Normalization of the input representation of HOA

In order to use the processing according to the invention, the (full) input signal of the HOA representation must be normalized first. For HOA compression, the processing is performed frame by frame, where the k -th frame C ( k ) of the original input HOA representation is determined by the vector c ( t ) of time-continuous sequences of HOA coefficients defined in equation (54) in the section Fundamentals of Higher-Order Ambisonics Technology as

(1)

where k denotes the frame index, L is the frame length (in samples), O = ( N + l) ² is the number of HOA coefficient sequences and T _s indicates the sampling period.

As stated in document EP 2824661 A1, a meaningful normalization of the HOA representation, considered from a practical point of view, is not achieved by imposing restrictions on the range of values of the individual sequences of HOA coefficients c _n ^m ( t ), since these time domain functions are not the signals that are actually reproduced by the loudspeakers after reconstruction. Instead, it is more convenient to consider an "equivalent spatial domain representation" that is obtained by reconstructing the HOA representation in O virtual loudspeaker signals w _j ( t ), 1 ≤ j ≤ O . It is assumed that the corresponding positions of the virtual loudspeakers are expressed by means of a spherical coordinate system, where each position is considered to lie on the unit sphere and to have a radius of "1". Therefore, the positions can be equivalently expressed by the order-dependent directions Ω _j ^{( N )} = ( θ _j ^{( N )} , φ _j ^{( N )} ), 1 ≤ j ≤ O , where θ _j ^{( N )} and φ _j ^{( N )} denote the tilt angles and azimuths, respectively (see also Fig. 6 and its description for the definition of a spherical coordinate system). These directions should be distributed over the unit sphere as uniformly as possible, see, for example, the technical report by J. Fliege, U. Maier, "A two-stage approach for computing cubature formulae for the sphere", Technical report, Fachbereich Mathematik, University of Dortmund, 1999. The number of nodes can be found at http://www.mathematik.uni-dortmund.de/lsx/research/projects/fliege/nodes/nodes.html for computing specific directions. These provisions mainly depend on the type of definition of "uniform distribution over a sphere" and are therefore ambiguous.

The advantage of defining value ranges for virtual loudspeaker signals over defining value ranges for HOA coefficient sequences is that the value range for the former can be intuitively set to the interval [-1,1[ , as in the case of conventional loudspeaker signals assuming PCM representation. This leads to a spatially uniformly distributed quantization error, so that quantization is preferably applied in the region relevant for real listening. An important aspect in this context is that the number of bits per sample can be chosen as low as it is typically for conventional loudspeaker signals, i.e. 16, which improves efficiency compared to direct quantization of HOA coefficient sequences, where a higher number of bits (e.g. 24 or even 32) per sample is typically required.

To describe the normalization process in the spatial domain in detail, all virtual loudspeaker signals are summed into a vector as w ( t ) : = [ w ₁ ( t ) … w _O ( t )] ^T , (2)

where ( ) ^T denotes the transformation. Denoting the mode matrix in the virtual directions Ω _j ^{( N )} , 1 ≤ j ≤ O , as Ψ , which is defined as (3)

With , (4)

the reconstruction process can be expressed as matrix multiplication w ( t ) = ( Ψ ) ^-1 · c ( t ). (5)

Using these definitions, a reasonable requirement for virtual loudspeaker signals is:

(6)

which means that the amplitude of each virtual loudspeaker signal must lie in the range [-1,1[ . The time instant t is represented by the sample index l and the sample period T _S for the sample values of the specified HOA data frames.

The total signal power of the loudspeakers therefore satisfies the condition

(7)

The reconstruction and normalization of the HOA data frame representation is performed before the input C ( k ) in Fig. 1A.

Consequences for the signal value range before the gain control

Assuming that the normalization of the input representation HOA is performed in accordance with the description given in the section Normalization of the Input Representation HOA , the range of values of the signals y _i , i = 1, … , I , which are input to the gain controller processing block 15, 151 in the HOA-based compressor, is taken into account in the following. These signals are created by distributing, over the available I channels, one or more sequences of HOA coefficients, or dominant audio signals x _{PS, d} , d = l, … , D , and/or specific sequences of HOA environment component coefficients c _{AMB, n} , n = l, …, O , to a part of which a spatial transformation is applied. Therefore, it is necessary to analyze the possible range of data values of the mentioned different types of signals when assuming normalization in equation (6). Since all types of signals are indirectly calculated from the original sequences of HOA coefficients, their possible ranges of values are considered.

The case in which only one or more HOA coefficient sequences are contained in the I channels is not shown in Fig. 1A and Fig. 2B, i.e. in such a case the HOA decomposition does not require the environment component change and the corresponding synthesis blocks.

Implications for the HOA representation value range

The continuous-time representation of the HOA is obtained from the virtual loudspeaker signals using the equation c ( t ) = Ψw ( t ), (8)

which is the inverse operation of equation (5).

Therefore, the total power of the HOA coefficient sequences is related as follows:

, (9)

using equations (8) and (7).

Assuming N3D normalization of the spherical harmonic functions, the square of the Euclidean norm of the mode matrix can be written as || Ψ ||₂ ²= K · O (10a)

Where (10b)

denotes the ratio between the square of the Euclidean norm of the mode matrix and the number O of HOA coefficient sequences. This ratio depends on the specific order N of the HOA and the specific directions of the virtual loudspeakers Ω _j ^{( N )} , 1 ≤ j ≤ O , which can be expressed by complementing the ratio with the appropriate list of parameters as follows:

K = K ( N , Ω ₁ ^{( N )} _{, …} Ω _O ^{( N )}). (10c)

Fig. 3 shows the values of K for the virtual directions Ω _j ^{( N )} , 1 ≤ j ≤ O , according to the above-mentioned paper by Fliege et al. for HOA orders N = 1, …, 29.

Combining all the previous arguments and considerations gives an upper bound for the magnitude of the HOA coefficient sequences as follows:

(11)

where the first inequality follows directly from the definitions of the norm.

It is important to note that the condition in equation (6) implies the condition in equation (11), but the converse does not hold, i.e. equation (11) does not imply equation (6).

Another important aspect is that, given nearly uniformly distributed virtual speaker positions, the column vectors of the mode matrix Ψ , which represent the mode vectors over the virtual speaker positions, are nearly orthogonal to each other and have Euclidean norm N + 1 each. This property means that the spatial transformation nearly preserves the Euclidean norm, except for a constant factor, i.e.

. (12)

The true norm || c ( lT _S )|| ₂ differs more from the approximation in equation (12) the more the assumption of orthogonality of the mode vectors is violated.

Consequences for the range of values of dominant sound signals

Both types of dominant sound signals (directional and vectorial) have in common that their contribution to the HOA representation is described by a single vector with the Euclidean norm N + 1, i.e. || v ₁ || ₂ = N + 1. (13)

In the case of a directional signal, this vector corresponds to the mode vector in a certain direction of the signal source Ω _{S ,1} , i.e.

v ₁= S ( Ω _S,1) (14)

(15)

This vector describes, using the HOA representation, a directional ray in the direction of the signal source Ω _S,1 . In the case of vector signals, the vector v ₁ is not restricted to being a mode vector in any direction, and can therefore describe a more general directional distribution of a monophonic vector signal.

In what follows, we consider the general case of D dominant sound signals x _d ( t ), d = 1, … , D , which can be contained in the vector x ( t ) in accordance with

x ( t ) = [ x ₁( t ) x ₂( t ) … x _D ( t )] ^T . (16)

These signals must be determined based on the matrix

V : = [ v ₁ v ₂ … v _D ] (17)

which is formed by all vectors v _d , d = l, … , D , representing the distribution in directions of monophonic dominant signals x _d ( t ), d = 1, …, D .

To obtain meaningful extraction of dominant sound signals x ( t ), the following constraints are formulated:

a) Each dominant sound signal is obtained as a linear combination of the sequences of coefficients of the original HOA representation, i.e.

x ( t )= A · c ( t ), (18)

Where denotes the mixing matrix.

b) The mixing matrix A should be chosen in such a way that its Euclidean norm does not exceed the value “1”, i.e.

, (19)

and such that the square of the Euclidean norm (or equivalent power) of the residual between the original HOA representation and the representation of the dominant sound signals is no greater than the square of the Euclidean norm (or equivalent power) of the original HOA representation, i.e.

. (20)

When introducing equation (18) into equation (20), it is seen that equation (20) is equivalent to the constraint

, (21)

where I denotes the identity matrix.

From the constraints in equations (18) and (19) and from the compatibility of the Euclidean matrix and vector norms, the upper bound for the amplitudes of the dominant sound signals is found using the equations

(22)

(23)

, (24)

using equations (18), (19) and (11). Therefore, it is guaranteed that the dominant audio signals remain in the same range as the original HOA coefficient sequences (compare with equation (11)), i.e. (25)

Example of choosing a mixing matrix

An example of how to determine a mixing matrix satisfying constraint (20) is obtained by computing the dominant audio signals in such a way that the Euclidean norm of the residual after extraction is minimized, i.e.

x ( t ) =argmin _{x ( t )} || V • x ( t ) - c ( t ) ||₂. (26)

The solution to minimize the problem in equation (26) is given by x ( t ) = V ⁺ + c ( t ), (27)

where (•) ⁺ indicates the Moore-Penrose pseudo-inverse. Comparing equation (27) with equation (18) shows that, in this case, the mixing matrix is equal to the Moore-Penrose pseudo-inverse of V , i.e., A = V ⁺ . However, V must still be chosen to satisfy constraint (19), i.e., . (28)

In the case of only directional signals, when the matrix V is the mode matrix in some directions of the source signals

Ω _{S, d} , d = 1, … , D , i.e. V = [ S ( Ω _S,1 ) S ( Ω _{S, 2}) … S ( Ω _{S, D} )], (29)

constraint (28) can be satisfied by choosing the directions of the source signals Ω _{S, d} , d = 1, …, D , in such a way that the distance between two adjacent directions is not too small.

Implications for the range of values of the HOA environmental component coefficient sequences

The ambient component of the HOA is the component calculated by subtracting the dominant sound cues HOA representation from the original HOA representation, i.e. c _AMB ( t ) = c ( t ) - V • x ( t ). (30)

If the vector of dominant sound signals x ( t ) is determined in accordance with criterion (20), it can be concluded that

(31)

(32)

(33)

. (34)

Range of values of spatially transformed sequences of HOA environmental component coefficients

An additional aspect of the HOA compression processing proposed in EP 2743922 A1 and in the above-mentioned MPEG N14264 is that the HOA coefficient sequences O _MIN and the ambient component are always chosen to be distributed over the transmission channels where O _MIN = ( N _MIN + 1) ² with N _MIN ≤ N , typically of lower order than in the original HOA representation. In order to decorrelate these HOA coefficient sequences, they can be transformed into virtual loudspeaker signals incident from some predefined directions Ω _{MIN , d} , d = 1, … , O _MIN (similar to the principles described in Normalization of the HOA Input Representation ).

Determination of the vector of all sequences of coefficients of the environment component HOA with the ordinal index n ≤ N _MIN by c _AMB,MIN ( t ) and the mode matrix by virtual directions Ω _{MIN, d} , d = 1, … , O _MIN , on Ψ _{M I N} , the vector of all signals of virtual loudspeakers (determined by) w _MIN ( t ), is obtained from the equation

(35)

Therefore, using the compatibility of the Euclidean matrix and vector norms,

(36)

(37)

(38)

In the above-mentioned document N14264 concerning MPEG, the virtual directions Ω _{MIN, d} , d = 1, … , O _MIN , are chosen in accordance with the above-mentioned paper by Fliege et al. The corresponding Euclidean norms of the inversion of the mode matrices Ψ _MIN are shown in Fig. 4 for the orders N _MIN = 1, … , 9. It is shown that for N _MIN = 1, …, 9. (39).

However, this does not generally hold for N _MIN > 9, where the values is usually greater than "1". However, at least for 1 ≤ N _MIN ≤ 9 the virtual loudspeaker signal amplitudes are related by the equation

(40)

By constraining the input HOA representation to satisfy condition (6), which requires that the signal amplitudes of the virtual loudspeakers from a given HOA representation do not exceed the value "1", it can be guaranteed that these signal amplitudes before the gain control will not exceed the value (see equations (25), (34) and (40)) under the following conditions:

a) The vector of all dominant sound signals x ( t ) is calculated according to equation/constraints (18), (19) and (20);

b) The minimum order N _MIN , which defines the number O _MIN of the first sequences of HOA environment component coefficients to which the spatial transform is applied, shall be less than '9' when used as virtual loudspeaker positions as defined in the above-mentioned Fliege et al. paper.

Further, it can be concluded that the signal amplitudes before the gain controller will not exceed the value for any order N up to the maximum order N _MAX of interest, i.e. 1 ≤ N ≤ N _MAX , where . (41a)

In particular, from Fig. 3, it can be concluded that if the directions of the virtual loudspeakers Ω _j ^{( N )} , 1 ≤ j ≤ O , for the original spatial transform are assumed to be chosen according to the distribution in the paper by Fliege et al., and if additionally the maximum order of interest is assumed to be N _MAX = 29 (as, for example, in the MPEG document N14264), then the signal amplitudes before the gain control will not exceed the value 1.5 O , since < 1.5 in this special case. That is, you can choose = 1.5.

K _MAX depends on the maximum order of interest, N _MAX and directions of virtual loudspeakers Ω _j ^{( N )}, 1 ≤ j ≤ O , which can be expressed as

. (41b)

Therefore, the minimum gain applied by the gain controller to ensure that the signals before perceptual encoding lie within the interval [-1.1] is given by , Where

. (41c)

In the case where the signal amplitudes before the gain controller are too small, the MPEG N14264 document suggests that they can be smoothly amplified by a factor of up to , where e _MAX ≥ 0 is transmitted as side information in the HOA encoded representation.

Thus, each base-2 exponent describing in the access block the total absolute change in the amplitude of the modified signal caused by the gain controller processing block from the first to the current frame can take any integer value within the interval [ e _MIN , e _MAX ]. Therefore, the (least integer) number β _e of bits required for encoding is given by the equation

. (42)

In the case where the signal amplitudes before the gain controller are not too small, equation (42) can be simplified:

. (42a)

This number of bits β _e can be calculated at the input of stages/steps 15, …, 151 of the gain controller.

Using this number β _e bits for the exponent ensures that all possible absolute amplitude changes caused by the HOA-based compressor gain control processing blocks 15, …, 151 can be registered, allowing decompression to start at some predetermined entry points within the compressed representation.

At the start of decompression of the compressed HOA representation in the HOA-based decompressor, non-differentiable gain values representing the total absolute amplitude changes assigned to the side information for some data frames and received from the demultiplexer 21 outside the received data stream , are used in the stages or steps 24,…, 241 of the inverse gain control to apply the corresponding gain control in a mode opposite to the processing that was performed in the stages/steps 15,…, 151 of the gain control.

Additional embodiment

When implementing a specific HOA compression/decompression system as described in sectionsHOA Compression, HOA Spatial Coding, HOA Decompression and HOA Spatial Decoding, number β _e bits for encoding the indicator must be set according to equation (42), depending on the scale factor K _MAX.DES , which itself depends on the desired maximum order N _MAX.DES representation of the HOA to be compressed and the defined directions of the virtual loudspeakers, 1 ≤ N ≤ N _MAX .

For example, assuming N _{MAX , DES} = 29, and choosing the virtual speaker directions according to Fliege et al., a reasonable choice would be . In this situation, correct compression is guaranteed for HOA representations with ordinal number N for 1 ≤ N ≤ N _MAX , which are normalized according to the section Normalization of the HOA input representation using the same virtual speaker directions . However, this guarantee cannot be given in the case of the HOA representation, which is also (for efficiency reasons) equivalently represented by virtual loudspeaker signals in PCM format, but in which the directions , 1 ≤ j ≤ O , the virtual loudspeakers are chosen to differ from the directions of the virtual loudspeakers , assumed at the system design stage.

Due to this different choice of virtual loudspeaker positions, even if the amplitudes of these virtual loudspeaker signals lie within the interval [1,1[ , it can no longer be guaranteed that the amplitudes of the signals before the gain control will not exceed the value . And therefore it is not possible to guarantee that this HOA representation has the appropriate normalization for compression, in accordance with the processing described in MPEG N14264.

In this situation, it is preferable to have a system that provides, based on knowledge of the virtual loudspeaker positions, the maximum permissible amplitude of the virtual loudspeaker signals in order to ensure an appropriate HOA representation suitable for compression according to the processing described in document N14264 concerning MPEG. Fig. 5 illustrates such a system. It takes as input the positions of the virtual loudspeakers , 1 ≤ j ≤ O , where O = ( N + 1) ² at , and produces as output the maximum allowable amplitude γ _{d B} (measured in decibels) of the virtual loudspeaker signals. In step or stage 51, the mode matrix Ψ is calculated from the virtual loudspeaker positions, according to equation (3). In the next step or stage 52, the Euclidean norm || Ψ || ₂ of the mode matrix is calculated. In the third step or stage 53, the amplitude γ is at least "1" and the ratio between the product of the square root of the number of virtual loudspeaker positions and K _MAX,DES , and the Euclidean norm of the mode matrix, i.e.,

. (43)

The value in decibels is obtained using the formula . (44)

To clarify: from the above derivation it is clear that if the magnitude of the HOA coefficient sequences does not exceed the value , i.e., if

, (45)

all signals before the gain controller processing blocks 15, 151 will, accordingly, not exceed this value, which is a requirement for the corresponding HOA compression.

From equation (9) it is found that the magnitude of the HOA coefficient sequences is related by the ratio

. (46)

Therefore, if the value of γ is set according to equation (43) and the virtual loudspeaker signals in PCM format satisfy the relation

, (47)

from equation (7) it follows that (48)

and that requirement (45) is satisfied.

That is, the maximum value of the quantity “1” in equation (6) is replaced by the maximum value of the quantity γ in equation (47).

Fundamentals of Higher Order Ambisonics Technology

Higher Order Ambisonics (HOA) is based on the description of the sound field within a compact region of interest, which is assumed to be free of sound sources. In this case, the spatiotemporal behavior of the sound pressure p ( t,x ) at time t and position x within the region of interest is physically determined by the equation of a homogeneous wave. In what follows, a spherical coordinate system is assumed, as shown in Fig. 6. In the coordinate system used, the x- axis points to the frontal position, the y- axis points to the left, and the z- axis points upward. The spatial position x = ( r, θ, φ ) ^T is represented by the radius r > 0 (i.e., the distance to the origin), the tilt angle , measured from the polar z-axis, and the azimuthal angle formula , measured counterclockwise in the x - y plane from the x- axis. In addition, ( ) ^T denotes the transformation.

Then, it can be shown, according to the textbook "Fourier Acoustics", that the Fourier transform of the sound pressure with respect to time, denoted F _t ( ), i.e.

(49)

with ω denoting the angular frequency and i indicating the imaginary unit, can be expanded into a series of spherical harmonics according to

, (50)

where c _s denotes the speed of sound and k denotes the angular mode index, which is related to the angular frequency ω by the formula . In addition, j _n ( ) denotes the spherical Bessel functions of the first kind, and denotes the real spherical harmonics of order n and degree m , which are defined in the section Definition of real spherical harmonics . Expansion coefficients depend only on the angular mode index k . It should be noted that it is implicitly assumed that the sound pressure is spatially band-limited. Thus, the series is truncated at the ordinal index n at an upper limit N , which is called the order of the HOA representation.

If the sound field is represented as a superposition of an infinite number of harmonic plane waves with different angular frequencies ω arriving from all possible directions specified by a sequence of angles ( θ , φ ), it can be shown (see B. Rafaely, "Plane-wave decomposition of the sound field on a sphere by spherical convolution", J. Acoust. Soc. Am., vol.4(116), pages 2149-2157, October 2004) that the corresponding complex function of the plane wave amplitude c ( ω,θ , φ ) can be expressed by the following expansion in spherical harmonics

, (51)

where are the expansion coefficients are related to the expansion coefficients How . (52)

Assuming that the individual coefficients are functions of the angular frequency ω , applying the inverse Fourier transform (denoted by F ^{- 1} ( )), yields time domain functions

(53)

for each order n and degree m . These time-domain functions are here called time-continuous sequences of HOA coefficients, which can be contained in a single vector c ( t ) as

(54)

HOA Coefficient Sequence Position Index in the vector c ( t ) is given as n ( n + 1) + 1 + m . The total number of elements in the vector c ( t ) is given as O = ( N + 1) ² .

The final Ambisonics format yields a quantized version of c ( t ) using the sampling frequency f _S as

(55)

where T _S = 1/ f _S denotes the sampling period. The elements of c ( lT _S ) are called discrete-time sequences of HOA coefficients, which can be shown to be always real. This property also holds for continuous-time variants .

Definition of real spherical harmonics

Real spherical harmonics (assuming SN3D normalization in accordance with the thesis of J. Daniel, "Representation of acoustic waves, application of transmission and reproduction of complex sound scenes in a multimedia context", PhD thesis, Universite Paris, 6, 2001, chapter 3.1), are given in the form

(56)

With

(57)

The associated Legendre functions P _{n , m} ( x ) are defined as

(58)

with the Legendre polynomial P _n ( x ) and, unlike the textbook by E. G. Williams, "Fourier Acoustics", vol. 93 of Applied Mathematical Sciences, Academic Press, 1999, do not contain the Condon-Shortley phase component (-1) ^m .

The processing according to the invention may be performed by means of a single processor or electronic circuit or by means of several processors or electronic circuits operating in parallel and/or operating on different parts of the processing according to the invention.

Instructions for operating the processor or processors may be stored in one or more memory devices.

Claims (9)

1. A method for decoding a compressed audio representation of a sound recording that creates a "presence effect" of higher order (HOA) sound or a sound field, the method comprising: decoding a bitstream containing a compressed audio representation of an HOA based on the least integer , while the smallest integer are determined on the basis of at the same time Where - order, - the maximum order of interest, - directions of virtual loudspeakers, while where O represents the number of HOA coefficients of the compressed HOA audio representation, and at the same time 2. A device for decoding a compressed audio representation of a sound recording that creates a "presence effect" of higher order (HOA) sound or a sound field, the device comprising: a decoder configured to decode a bitstream containing an audio representation of an HOA based on the least integer , while the smallest integer is determined on the basis of at the same time Where - order,- the maximum order of interest,- directions of virtual loudspeakers, while Where Orepresents the number of HOA coefficients of the compressed HOA audio representation, and at the same time 3. A non-volatile machine-readable data carrier containing instructions that, when executed by a processor, perform the method according to paragraph 1.