HK1260679B - Method and device for applying dynamic range compression to a higher order ambisonics signal - Google Patents

Description

Methods and equipment for applying dynamic range compression to high-order high-fidelity stereo signals

This application is a divisional application of the invention patent application with application number 201580015764.0, application date March 24, 2015, entitled "Method and apparatus for applying dynamic range compression to high-order high-fidelity stereo signals".

Technical Field

The present invention relates to a method and apparatus for performing dynamic range compression (DRC) on high-fidelity stereo signals, particularly high-order high-fidelity stereo (HOA) signals.

Background Technology

Dynamic range compression (DRC) aims to reduce the dynamic range of an audio signal. A time-varying gain factor is applied to the audio signal. Typically, this gain factor depends on the amplitude envelope of the signal used to control the gain. The mapping is usually non-linear. Large amplitudes are mapped to smaller amplitudes, while weak sounds are often amplified. Suitable scenarios include noisy environments, listening late at night, and listening with small speakers or portable headphones.

The general idea behind streaming or broadcasting audio is to generate DRC gains before transmission and apply these gains after reception and decoding. The principle of using DRC (i.e., how DRC is typically applied to an audio signal) is illustrated in Figure 1a). The signal level (typically the signal envelope) is detected, and the associated time-varying gain g _(DRC) is calculated. This gain is used to change the amplitude of the audio signal. Figure 1b) illustrates the principle of using DRC for encoding/decoding, where the gain factor is transmitted along with the encoded audio signal. On the decoding side, the gain is applied to the decoded audio signal to reduce its dynamic range.

For 3D audio, different gains can be applied to speaker channels representing different spatial locations. These locations then need to be known on the emitting side to generate a matching set of gains. This is typically only possible under ideal conditions, while in reality, the number of speakers and their placement vary in many ways. This is more influenced by practical considerations than by prescribed rules. High-order high-fidelity stereo (HOA) is an audio format that allows for flexible rendering. HOA signals contain coefficient channels that do not directly represent sound levels. Therefore, DRC cannot be simply applied to HOA-based signals.

Summary of the Invention

This invention addresses at least the question of how DRC can be applied to HOA signals. The HOA signal is analyzed to obtain one or more gain factors. In one embodiment, at least two gain factors are obtained, and the analysis of the HOA signal includes an iDSHT (individualized spatial domain transformation). One or more gain factors are transmitted along with the original HOA signal. A special indication can be sent to indicate whether all gain factors are equal. This is the case in the so-called simplified mode, but in the non-simplified mode, at least two different gain factors are used. At the decoder, this one or more gains can (but not necessarily) be applied to the HOA signal. The user can choose whether to apply this one or more gains. The advantage of the simplified mode is that it requires far less computation because only one gain factor is used, and because the gain factor can be directly applied to the coefficient channels of the HOA signal in the HOA domain, the transformation to the spatial domain and subsequently the transformation back to the HOA domain can be omitted. In the simplified mode, the gain factor is obtained by analyzing only the zeroth-order coefficient channels of the HOA signal.

According to one embodiment of the present invention, a method for performing DRC on a HOA signal includes transforming the HOA signal to the spatial domain (via inverse DSHT), analyzing the transformed HOA signal, and obtaining a gain factor that can be used for dynamic range compression from the result of the analysis. In a further step, the obtained gain factor is multiplied by the transformed HOA signal (in the spatial domain), wherein a gain-compressed transformed HOA signal is obtained. Finally, the gain-compressed transformed HOA signal is transformed back to the HOA domain (via DSHT), i.e., the coefficient domain, wherein a gain-compressed HOA signal is obtained. Additionally, according to another embodiment of the present invention, a method for performing DRC on a HOA signal in a simplified mode includes analyzing the HOA signal and obtaining a gain factor that can be used for dynamic range compression from the result of the analysis. In a further step, based on an evaluation of an indicator, the obtained gain factor is multiplied by the coefficient channel of the HOA signal (in the HOA domain), wherein a gain-compressed HOA signal is obtained. Also based on an evaluation of an indicator, it can be determined that the transformation of the HOA signal can be omitted. The indicator flag for the simplified mode (i.e., only one gain factor is used) can be implicitly set, for example, if only the simplified mode can be used due to hardware or other limitations, or the indicator flag for the simplified mode can be explicitly set, for example, based on the user's selection of simplified or non-simplified mode.

Additionally, according to one embodiment of the present invention, the method of applying a DRC gain factor to a HOA signal includes receiving a HOA signal, an indicator flag, and a gain factor; determining that the indicator flag indicates a non-simplified mode; transforming the HOA signal to the spatial domain (using inverse DSHT), wherein the transformed HOA signal is obtained; multiplying the gain factor by the transformed HOA signal, wherein a dynamically range-compressed, transformed HOA signal is obtained; and transforming the dynamically range-compressed, transformed HOA signal back to the HOA domain (i.e., the coefficient domain) (using DSHT), wherein a dynamically range-compressed HOA signal is obtained. The gain factor may be received together with the HOA signal or separately.

Additionally, according to one embodiment of the present invention, a method for applying a DRC gain factor to a HOA signal includes receiving a HOA signal, an indicator flag, and a gain factor; determining that the indicator flag indicates a simplified mode; and multiplying the gain factor by the HOA signal based on the determination, wherein a dynamically range compressed HOA signal is obtained. The gain factor may be received together with the HOA signal or separately.

An apparatus for applying a DRC gain factor to a HOA signal is disclosed in claim 11.

In one embodiment, the present invention provides a computer-readable medium having executable instructions that cause a computer to perform a method of applying a DRC gain factor to a HOA signal, the method including the steps described above.

In one embodiment, the present invention provides a computer-readable medium having executable instructions that cause a computer to perform a method for performing DRC on a HOA signal, the method including the steps described above.

Advantageous embodiments of the invention are disclosed in the dependent claims, the following description, and the figures.

Attached Figure Description

Example embodiments of the present invention are described with reference to the accompanying drawings, in which:

Figure 1 shows the general principle of DRC applied to audio.

Figure 2 shows a general method for applying DRC to HOA-based signals according to the present invention.

Figure 3 shows the spherical loudspeaker grid for N=1 to N=6.

Figure 4 shows the creation of DRC gain for HOA.

Figure 5 shows the application of DRC to the HOA signal.

Figure 6 shows the dynamic range compression process on the decoder side.

Figure 7 shows the DRC of the HOA of the QMF domain combined with the rendering steps, and

Figure 8 shows the DRC of the HOA in the QMF domain combined with the rendering step in the simple case of a single DRC gain group.

Detailed Implementation

This invention describes how DRC can be applied to HOA. Traditionally, this is not straightforward because HOA is a sound field description. Figure 2 illustrates the principle of this method. On the encoding or transmission side, as shown in Figure 2a), the HOA signal is analyzed, the DRC gain g is calculated based on the analysis of the HOA signal, and the DRC gain is encoded and transmitted along with an encoded representation of the HOA content. This can be a multiplexed bitstream or two or more separate bitstreams.

On the decoding or receiving side, as shown in Figure 2b), the gain g is extracted from one or more such bitstreams. After decoding the bitstream(s) in the decoder(s), the gain g is applied to the HOA signal as described below. By doing so, the gain is applied to the HOA signal, i.e., a HOA signal with reduced dynamic range is typically obtained. Finally, the dynamically range-adjusted HOA signal is rendered in the HOA renderer.

The assumptions and definitions used are explained below.

It is assumed that the HOA renderer is energy-saving, meaning that N3D normalized spherical harmonics are used, and the energy of the unidirectional signals encoded within the HOA representation is retained after rendering. This energy-saving HOA rendering is described, for example, in WO2015/007889A _(PD130040) .

The terms used are defined as follows.

Let B represent a block of τ HOA samples, where B = [b(1), b(2), ..., b(t), ..., b(τ)], and the vector containing the high-fidelity stereo coefficients in the ACN order (vector index o = ^n² + n + m + 1, where the coefficient order index is n and the coefficient degree index is m). N represents the HOA truncation order. The number of higher-order coefficients in b is (N+1) ^² . The sampling index of a data block is t. τ can range from one sample to 64 samples or more. The zeroth-order signal is the first row of B. Let W = DB, which renders the HOA sampled block into a block of L loudspeaker channels in the spatial domain, where this is the assumed process of the HOA renderer (HOA rendering) in Figure 2b).

Let represent the rendering matrix associated with _LL = (N+1) ^channels , which are placed on a sphere in a very regular manner, such that all adjacent positions share the same distance. _DL is well-conditioned and its inverse exists. Therefore, these two define a pair ^of transformation matrices (DSHT - Discrete _Spherical Harmonic Transform):

W _L = D _L B,

g is a vector of L _L = (N+1) ² gain DRC values. The gain values are assumed to be applied to blocks of τ samples and are assumed to be smooth from block to block. For transmission, gain values sharing the same value can be grouped into gain groups. If only a single gain group is used, then this means that a single DRC gain value (indicated here by g ₁ ) is applied to all τ samples of all speaker channels.

For each HOA cutoff order N, an ideal L _{_L} = (N+1)^² virtual speaker grids and an associated rendering matrix _DL _L are defined. The virtual speaker positions are sampled from the spatial region surrounding the virtual listener. Grids for N = 1 to 6 are shown in Figure 3, where the region associated with the speaker is a shaded cell. A sampling position is always associated with the center speaker position (azimuth = 0, tilt = π/2; note that the azimuth is measured from the frontal direction associated with the listening position). When the DRC gain is created, the sampling positions, DL_L, and DL_{_L} are known on the encoder side. On the decoder side, DL _{_L} and DL_L are needed to apply the gain value.

The DRC gain for HOA is created as follows.

The HOA signal is converted to the spatial domain via W _{_L} = D_{_L}B. Up to L_{_L} = (N+1)^² DRC gains _g _l are created by analyzing these signals. If the content is a combination of HOA and audio object (AO), then AO signals such as dialogue tracks can be used for side chaining. This is shown in Figure 4b). When creating different DRC gain values associated with different spatial regions, care must be taken to ensure that these gains do not affect the spatial image stability on the decoder side. To avoid this, in the simplest case (the so-called simplified mode), a single gain can be assigned to all L channels. This can be achieved by analyzing all spatial signals W, or by analyzing the zeroth-order HOA coefficient sampling block, without requiring a transformation to the spatial domain (Figure 4a). The latter is the same as analyzing the downmixed signal of W. Further details are given below.

Figure 4 illustrates the creation of DRC gains for HOAs. Figure 4a) depicts how a single gain _g1 (for a single gain group) can be derived from a zeroth-order HOA component (optionally, with a side link from the AO). The zeroth-order HOA component is analyzed in DRC analysis function block 41s, and a single gain _g1 is derived. The single gain _g1 is encoded separately in DRC gain encoder 42s. The encoded gain is then encoded together with the HOA signal B in encoder 43, which outputs the encoded bitstream. Optionally, an additional signal 44 may be included in the encoding. Figure 4b) depicts how two or more DRC gains are created by transforming the HOA representation 40 to the spatial domain. The transformed HOA signal _WL is then analyzed in DRC analysis function block 41, and the gain value g is extracted and encoded in DRC gain encoder 42. Here, the encoded gain is encoded together with the HOA signal B in encoder 43, and optionally, an additional signal 44 may be included in the encoding. As an example, sounds from behind (e.g., background sounds) may experience more attenuation than sounds originating from the front or sides. This will result in (N+1) ^² gain values in g, which, in this example, could be sent within ^two gain groups. Optionally, side-linking of the audio object waveforms and their directional information can also be used here. Side-linking means that the DRC gain of one signal is derived from another. This reduces the power of the HOA signal. Dispersed sounds in the HOA mix that share the same spatial source region as the AO foreground sounds can experience stronger attenuation gain than spatially distant sounds.

The gain value is sent to the receiver or decoder side.

A variable number of gain values, from 1 to L_{_L} = (N+1)^², are transmitted, associated with a block of τ samples. Gain values can be assigned to channel groups for transmission. In one embodiment, all equal gains are combined into a single channel group to minimize transmitted data. If a single gain is transmitted, it is associated with all L_{_L} channels. The channel group gain values and their number are transmitted. The use of a channel group is signaled so that the receiver or decoder can correctly apply the gain value.

The gain value is applied as follows.

The receiver/decoder can determine the number of coded gain values transmitted, decode the relevant information (51), and allocate the gain (52-55) to L _L = (N+1) ² channels. If only one gain value (one channel group) is transmitted, it can be directly applied (52) to the HOA signal (B _DRC = g ₁ B), as shown in Figure 5a). This is advantageous because decoding is much simpler and requires far less processing. The reason is that matrix operations are not required; instead, the gain value can be directly applied (52), for example, by multiplying it by the HOA coefficients. Further details are provided below.

If two or more gains are transmitted, then the channel group gain is assigned to each of the L channel gains g = [ _g1 , ..., _gL ].

For a virtual regular loudspeaker grid, the loudspeaker signal with applied DRC gain is calculated by the following formula.

Then, the obtained modified HOA representation is calculated by the following formula.

This can be simplified, as shown in Figure 5b). Instead of transforming the HOA signal to the spatial domain, the gain vector is transformed to the HOA domain by applying the gain and transforming the result back to the HOA domain: 53

In the gain allocation function block 54, the gain matrix is directly applied to the HOA coefficient: B _DRC = GB.

This is more efficient in terms of the computational operations required for (N+1) ^² < τ. That is, this solution has an advantage over the traditional solution because decoding is much simpler and requires far less processing. The reason is that no matrix operations are required; instead, the gain value can be applied directly, for example, by multiplying it with the HOA coefficients in the gain allocation function block 54.

In one embodiment, an even more efficient way to apply the gain matrix is to apply DRC and render the HOA signal in one step by manipulating the renderer matrix in renderer matrix correction function block 57: this is shown in Figure 5c). This is advantageous if L < τ.

In summary, Figure 5 illustrates various embodiments of applying DRC to HOA signals. In Figure 5a), the gain of a single channel group is transmitted and decoded 51 and directly applied to the HOA coefficients 52. The HOA coefficients are then rendered using a normal rendering matrix 56.

In Figure 5b), more than one channel group gain is sent and decoded 51. Decoding results in a gain vector g with (N+1) ² gain values. A gain matrix G is created and applied 54 to the blocks sampled by the HOA. These are then rendered using a normalized rendering matrix 56.

In Figure 5c), the decoded gain matrix/gain value is applied directly to the renderer matrix instead of directly to the HOA signal. This is performed in the renderer matrix correction function block 57 and is computationally advantageous when the DRC block size τ is greater than the number of output channels L. In this case, the HOA sample is rendered using the corrected render matrix 57.

The calculation of the ideal DSHT (Discrete Spherical Harmonic Transform) matrix for DRC is described below. This DSHT matrix is specifically optimized for use in DRC and differs from the DSHT matrix used for other purposes such as data rate compression.

The following yields the requirements for the ideal rendering and encoding matrices DL and _L related to the ideal spherical layout. Finally, these requirements are as follows:

(1) The rendering matrix _DL must be invertible, that is, it needs to exist;

(2) The sum of amplitudes in the spatial domain should be reflected as zeroth-order HOA coefficients after the transformation from the spatial domain to the HOA domain, and should be preserved after the subsequent transformation back to the spatial domain (amplitude requirement); and

(3) When transforming to the HOA domain and transforming back to the spatial domain, the energy of the spatial signal should be preserved (energy preservation requirement).

Even for an ideal rendering layout, requirements 2 and 3 appear to be contradictory. When the DSHT transformation matrix is derived using simple methods, such as those known from the prior art, only one or the other of requirements (2) and (3) can be satisfied without error. Satisfying one of requirements (2) and (3) without error results in an error exceeding 3 dB in the other requirement. This typically leads to audible acoustic artifacts. Methods to overcome this problem are described below.

First, an ideal spherical layout is chosen, where L = (N+1) ^² . The L directions of the (virtual) speaker location are given by _Ω₁ , and the associated mode matrix is represented as a mode vector, each containing a spherical harmonic with direction _Ω₁ . The L quadrature gains associated with the spherical layout location are aggregated in the vector. These quadrature gains are estimated over the spherical region around such a location, and the total gain is 4π related to the surface of a sphere with radius 1.

The first prototype rendering matrix is derived from the following formula.

Note that the division by L can be omitted due to the subsequent normalization step (see below).

Second, perform compact singular value decomposition: and the second prototype matrix is derived from the following equation.

Third, the prototype matrix is normalized:

Here, k represents the matrix norm type. Both matrix norm types show equally good performance. The k=1 norm or the Frobenius norm should be used. This matrix satisfies requirement 3 (energy conservation).

Fourth, in the final step, the amplitude error that satisfies requirement 2 is substituted:

The row vector e is calculated, where [1, 0, 0, ..., 0] is a row vector with (N+1) ² elements, all of which are zero except for the first element which is 1. The sum of the row vectors is represented by . Now, by substituting the amplitude error, the rendering matrix _DL is obtained:

In this matrix, vector e is added to each row. This matrix satisfies requirements 2 and 3. All elements in the first row become 1.

The detailed requirements for DRC are explained below.

First, applying _L identical gains with a value of _g1 in the spatial domain is equivalent to applying gain _g1 to the HOA coefficients:

This leads to the following requirement: it means that L = (N+1)^ ² and that there must be a (trivial) existence.

Second, analyzing the sum signal in the spatial domain is equivalent to analyzing the zeroth-order HOA component. The DRC analyzer uses the signal's energy and its amplitude. Therefore, the sum signal is related to both amplitude and energy.

The signal model of HOA is: B = _{ΨeXs ,} which is a matrix of S directional signals; it is an N3D mode matrix associated with directions _Ω1 ,..., _Ωs . The mode vectors are derived from spherical harmonic combinations. In the N3D representation, the zeroth-order component is independent of the direction.

The zeroth-order component HOA signal needs to be converted into the sum of directional signals to reflect the correct amplitude of the summed signal. _{1 S} is a vector derived from the combination of S elements with a value of 1.

The energy of the directional signal is preserved in this mixture because if the signal X_{_s} is uncorrelated, then this simplifies to...

The sum of amplitudes in the spatial domain is given by, where the HOA translation (panning) matrix _ML = _{DLΨe .}

Therefore, this means that the latter requirement can be compared to the sum of the amplitude requirements sometimes used in translations like VBAP. Experience shows that this can be well approximated for a very symmetrical spherical loudspeaker setup, as we have found that the amplitude requirement can subsequently be achieved with the necessary precision.

This also ensures that the energy requirements for the signal can be met:

In the spatial domain, the energy and are given by . If the desired ideal symmetrical loudspeaker configuration exists, then this equation will become a good approximation.

This leads to the requirement that, in addition, from the signal model we can also conclude that the top row needs to be [1, 1, 1, 1, ...], that is, a vector of length L with "1" elements, so that the recoded zero-order signal retains its amplitude and energy.

Third, energy preservation is a prerequisite: after conversion to the HOA and rendering to the loudspeaker in a direction space independent of the signal (Ωs ₎ , the signal's energy should be preserved. This leads to the fact that this can be achieved by modeling _DL from the rotation matrix and the diagonal gain matrix: _DL = UV ^T diag(a) (for clarity, the dependency on the direction ( _Ωs ) is removed):

This equation will satisfy all gains for spherical harmonics and correlations. If all gains are chosen to be equal, this results in...

The requirement VV ^T = 1 can be realized for L ≥ (N+1) ² , but is only approximated for L < (N+1) ² .

This leads to demand:

As an example, the ideal spherical positions (HOA orders N=1 to N=3) are described below (Tables 1-3). Ideal spherical positions for other HOA orders (N=4 to N=6) are also described below (Tables 4-6). All the positions described below are derived from the corrected positions published in reference [1] below. The methods for deriving these positions and the associated integral/volume gain are published in reference [2] below. In these tables, the azimuth is measured counterclockwise from the frontal direction associated with the listening position, and the tilt is measured from the Z-axis above the listening position with a tilt of 0.

N=1 position

a)

_DL :

b)

Table 1. a) Spherical position of the virtual loudspeaker of HOA order N=1, and b) Rendering matrix generated for spatial transformation (DSHT).

N=2 positions

Inclination angle θ/rad Azimuth φ/rad Integral gain 1.57079633 0.00000000 1.41002219 2.35131567 3.14159265 1.36874571 1.21127801 -1.18149779 1.36874584 1.21127606 1.18149755 1.36874598 1.31812905 -2.45289512 1.41002213 0.00975782 -0.00009218 1.41002214 1.31812792 2.45289621 1.41002230 2.41880319 1.19514740 1.41002223 2.41880555 -1.19514441 1.41002209

a)

_DL :

b)

Table 2.a) The spherical position of the virtual loudspeaker of HOA order N=2, and b) The generated rendering matrix for spatial transformation (DSHT).

N=3 position

Table 3.a) Spherical position of the virtual loudspeaker of HOA order N=3.

_DL :

b)

Table 3.b) shows the rendering matrix generated for the Space Transformation (DSHT).

The term numerical integration is often abbreviated to integration and is synonymous with numerical integration, especially when applied to one-dimensional integration. In this paper, numerical integration with more than one dimension is referred to as volume.

As described above, a typical application scenario for applying DRC gain to a HOA signal is shown in Figure 5. For mixed content applications, such as HOA plus an audio object, DRC gain application can be implemented in at least two flexible rendering methods.

Figure 6 illustrates exemplary dynamic range compression (DRC) processing on the decoder side. In Figure 6a), DRC is applied before rendering and mixing. In Figure 6b), DRC is applied to the loudspeaker signal, i.e., after rendering and mixing.

In Figure 6a), the DRC gain is applied to the audio object and the HOA respectively: the DRC gain is applied to the audio object in the audio object DRC function block 610, and the DRC gain is applied to the HOA in the HOA DRC function block 615. Here, the implementation of function block 615 matches the implementation of the DRC function block in Figure 5. In Figure 6b), a single gain is applied to all channels of the mixed signal of the rendered HOA and rendered audio object signals. Here, neither spatial enhancement nor attenuation is possible. The associated DRC gain cannot be created by analyzing the rendered mixed signal because the speaker layout of the consumer site is not known when it is created at the broadcast or content creation site. The DRC gain can be derived by analysis, where y _{_m} is the mixture of the mono downmixing of S audio objects x _{_s} and the zeroth-order HOA signal b_{_w}:

Below, further details of the publicly disclosed solution are described.

DRC for HOA content

DRC is applied to the HOA before rendering, or it can be combined with rendering. DRC for the HOA can be applied in the temporal domain or in the QMF filter group domain.

For DRC in the time domain, the DRC decoder provides (N+1) ² gain values g _drc = [g ₁ , ..., g(N+1) ² ] ^T based on the number of HOA coefficient channels of the HOA signal c. N is the HOA order.

The DRC gain is applied to the HOA signal according to the following formula:

Where c is a time-sampled vector of HOA coefficients and its inverse is a matrix related to the Discrete Spherical Harmonic Transform (DSHT) optimized for DRC purposes.

In one embodiment, it may be advantageous to reduce the computational load to (N+1) ⁴ operations per sample, including a rendering step and by directly calculating the loudspeaker signal, where D is the rendering matrix and can be pre-computed.

If all gains have the same g _drc value, as in simplified mode, then a single gain group has been used to send the encoder DRC gain. This situation can be flagged by the DRC decoder because, in this case, no calculation is needed in the spatial filter, so the calculation simplifies to:

c _drc = g _drc c

The above describes how to obtain and apply the DRC gain value. Below, the calculation of the DSHT matrix used for DRC is described.

Below, _DL is renamed D _DSHT . The matrix determining the spatial filter D _DSHT and its inverse is calculated as follows:

The set of spherical positions (where _Ωl = [ _θl , _φl ] ^T ) and the associated integral (volume) gain are selected by an index of HOA order N from Tables 1-4. The mode matrix _ΨDSHT associated with these positions is calculated as described above. That is, the mode matrix _ΨDSHT is based on including mode vectors, each of which is a _spherically harmonic mode vector containing a predetermined direction _Ωl = [ _θl , _φl ] ^T . According to Tables 1-6 (for example, for 1≤N≤6), this predetermined direction depends on the HOA order N. The first prototype matrix is calculated (dividing by (N+1) ^² can be omitted due to subsequent normalization). Compact singular value decomposition is performed and a new prototype matrix is calculated. This matrix is normalized. The row vector e is calculated, where [1, 0, 0, ..., 0] is a row vector with (N+1) ^² elements, all of which are zero except for the first element which is 1. The sum of the rows is represented by . Now, the optimized DSHT matrix D _DSHT is obtained by the following equation:

It has been found that if -e is used instead of e, the present invention provides slightly worse, but still usable, results.

The following applies to DRC in the QMF filter bank domain.

The DRC decoder provides a gain value g _ch (n, m) for each time frequency segment (tile) n, m of (N+1) ² spatial channels. The gain for time segment n and frequency band m is arranged in .

Multiband DRC is applied in the QMF filter bank domain. The processing steps are shown in Figure 7. The reconstructed HOA signal is transformed into the spatial domain by (inverse DSHT): W_{_DSHT} = D _{_DSHT} C, where τ is a block of HOA samples and a block of spatial samples matching the input time granularity of the QMF filter bank. Then the QMF analysis filter bank is applied. Let represent the vector of the spatial channel for each time frequency segment (n, m). Subsequently, the DRC gain is applied:

To minimize computational complexity, the DSHT and the rendering to the loudspeaker channel are combined: where D represents the HOA rendering matrix. The QMF signal can then be fed into a mixer for further processing.

Figure 7 shows the DRC of the HOA in the QMF domain in conjunction with the rendering step.

If only a single gain group is used for DRC, should this be flagged by the DRC decoder, or is computational simplification possible? In this case, all gains in the vector g(n,m) share the same _gDRC (n,m) value. A QMF filter bank can be applied directly to the HOA signal, and the gain _gDRC (n,m) can be multiplied in the filter bank domain.

Figure 8 shows the DRC of the HOA in the QMF domain (filter domain of quadrature mirror filters) in conjunction with the rendering step, where the calculation is simplified for the simple case of a single DRC gain group.

As is clear from the foregoing, in one embodiment, the present invention relates to a method for applying a dynamic range compression gain factor to a HOA signal, the method comprising the steps of: receiving a HOA signal and one or more gain factors; transforming the HOA signal 40 to the spatial domain, wherein iDSHT is used in conjunction with a transformation matrix obtained from the spherical position of a virtual loudspeaker and the integral gain q, and wherein the transformed HOA signal is obtained; multiplying the gain factor with the transformed HOA signal, wherein a dynamically range compressed transformed HOA signal is obtained; and transforming the dynamically range compressed transformed HOA signal back to the HOA domain, the HOA domain being the coefficient domain and using Discrete Spherical Harmonic Transform (DSHT), wherein a dynamically range compressed HOA signal is obtained.

Furthermore, the transformation matrix is calculated based on , where is the normalized form of , U, V are obtained from , Ψ _DSHT is the transpose mode matrix of the spherical harmonics related to the spherical position used by the virtual loudspeaker, and e ^T is the transpose form of . .

In another embodiment, the present invention relates to an apparatus for applying a DRC gain factor to a HOA signal, the apparatus comprising a processor or one or more processing elements adapted to receive the HOA signal and one or more gain factors, transform the HOA signal 40 to the spatial domain, wherein iDSHT is used with a transform matrix obtained from the integral gain q and the spherical position of the virtual loudspeaker, and wherein the transformed HOA signal is obtained, the gain factor is multiplied by the transformed HOA signal, wherein a dynamically range-compressed transformed HOA signal is obtained; and the dynamically range-compressed transformed HOA signal is transformed back to the HOA domain, the HOA domain being the coefficient domain and using Discrete Spherical Harmonic Transform (DSHT), wherein a dynamically range-compressed HOA signal is obtained.

Furthermore, according to the calculation of the transformation matrix, where is the normalized form of , U, V are obtained according to , Ψ _DSHT is the transpose mode matrix of the spherical harmonics related to the spherical position of the virtual loudspeaker used, and e ^T is the transpose form of .

In another embodiment, the present invention relates to a computer-readable storage medium having computer-executable instructions that, when executed on a computer, cause the computer to perform a method for applying a dynamic range compression gain factor to a high-order high-fidelity stereo (HOA) signal. The method includes receiving the HOA signal and one or more gain factors; transforming the HOA signal 40 to the spatial domain, wherein iDSHT is used with a transformation matrix obtained from the integral gain q and the spherical position of a virtual loudspeaker, and wherein the transformed HOA signal is obtained; multiplying the gain factors by the transformed HOA signal, wherein a dynamically range compressed transformed HOA signal is obtained; and transforming the dynamically range compressed transformed HOA signal back to the HOA domain, the HOA domain being a coefficient domain and using Discrete Spherical Harmonic Transform (DSHT), wherein a dynamically range compressed HOA signal is obtained.

Wherein, according to the calculation of the transformation matrix, where is the normalized form of , U, V are obtained according to, Ψ _DSHT is the transpose mode matrix of the spherical harmonics related to the spherical position of the virtual loudspeaker used, and e ^T is the transpose form of .

In another embodiment, the present invention relates to a method for performing DRC on a HOA signal, the method comprising the steps of: setting or determining a mode, which is a simplified mode or a non-simplified mode; in the non-simplified mode, transforming the HOA signal to the spatial domain, wherein inverse DSHT is used; in the non-simplified mode, analyzing the transformed HOA signal, while in the simplified mode, analyzing the HOA signal; obtaining one or more gain factors from the result of the analysis, wherein only one gain factor is obtained in the simplified mode, while two or more different gain factors are obtained in the non-simplified mode; in the simplified mode, multiplying the obtained gain factors by the HOA signal, wherein a gain-compressed HOA signal is obtained; in the non-simplified mode, multiplying the obtained gain factors by the transformed HOA signal, wherein a gain-compressed transformed HOA signal is obtained; and transforming the gain-compressed transformed HOA signal back to the HOA domain, wherein a gain-compressed HOA signal is obtained.

In one embodiment, the method further includes the steps of: receiving an indication flag indicating a simplified mode or a non-simplified mode; selecting the non-simplified mode if the indication flag indicates a non-simplified mode, and selecting the simplified mode if the indication flag indicates a simplified mode; wherein the steps of transforming the HOA signal to the spatial domain and transforming the transformed HOA signal with dynamic range compression back to the HOA domain are performed only in the non-simplified mode, and wherein, in the simplified mode, only one gain factor is formed with the HOA signal.

In one embodiment, the method further includes the steps of analyzing the HOA signal in a simplified mode and analyzing the transformed HOA signal in a non-simplified mode; obtaining one or more gain factors from the results of the analysis that can be used for dynamic range compression, wherein in the non-simplified mode, two or more different gain factors are obtained, while in the simplified mode, only one gain factor is obtained; wherein in the simplified mode, a gain-compressed HOA signal is obtained by multiplying the obtained gain factors by the HOA signal, and wherein in the non-simplified mode, the gain-compressed transformed HOA signal is obtained by multiplying the obtained two or more gain factors by the transformed HOA signal, and wherein in the non-simplified mode, the transformation of the HOA signal to the spatial domain uses inverse DSHT.

In one embodiment, the HOA signal is divided into frequency sub-bands, and one or more gain factors are obtained and applied to each frequency sub-band respectively, wherein a separate gain is applied to each sub-band. In one embodiment, the following steps are applied to each frequency sub-band respectively: analyzing the HOA signal (or the transformed HOA signal), obtaining one or more gain factors, multiplying the obtained one or more gain factors by the HOA signal (or the transformed HOA signal), and transforming the gain-compressed transformed HOA signal back to the HOA domain, wherein a separate gain is applied to each sub-band. It should be noted that the order of dividing the HOA signal into frequency sub-bands and transforming the HOA signal to the spatial domain can be interchanged, and/or the order of synthesizing the sub-bands and transforming the gain-compressed transformed HOA signal back to the HOA domain can be interchanged; they are independent of each other.

In one embodiment, the method further includes the step of sending the transformed HOA signal along with the obtained gain factors and the number of such gain factors before the step of multiplying by the gain factors.

In one embodiment, the transformation matrix is calculated based on the mode matrix Ψ _DSHT and the corresponding integral gain, wherein the mode matrix Ψ _DSHT is based on a mode vector, each of which is a spherically harmonic mode vector containing a predefined direction Ω ₁ , Ω _l = [θ _l , φ _l ] ^T , the predefined direction depending on the HOA order N.

In one embodiment, the HOA signal B is transformed to the spatial domain to obtain the transformed HOA signal W _DSHT , and the transformed HOA signal W _DSHT is multiplied sample-by-sample by the gain value diag(g) according to W _DSHT = diag(G)D _L B, and the method includes an additional step of transforming the transformed HOA signal to a different second spatial domain, wherein the transformation is pre-calculated during the initialization phase, and wherein D is a rendering matrix for transforming the HOA signal to a different second spatial domain.

In one embodiment, if at least (N+1) ^² < τ, where N is the HOA order and τ is the DRC block size, then the method further includes the steps of: transforming the gain vector 53 to the HOA domain, where G is the gain matrix and DL is the DSHT matrix defining the DSHT; and applying the gain matrix G to the HOA coefficients of the HOA signal B according to B _DRC = GB, where the DRC-compressed HOA signal B _DRC is obtained.

In one embodiment, if at least L < τ, where L is the number of output channels and τ is the DRC block size, then the method further includes the steps of: applying a gain matrix G to a renderer matrix D, wherein a dynamically range compressed renderer matrix is obtained; and rendering the HOA signal using the dynamically range compressed renderer matrix.

In one embodiment, the present invention relates to a method for applying a DRC gain factor to a HOA signal, the method comprising the steps of: receiving the HOA signal together with an indicator flag and one or more gain factors, the indicator flag indicating a simplified mode or a non-simplified mode, wherein if the indicator flag indicates a simplified mode, only one gain factor is received; selecting a simplified mode or a non-simplified mode according to the indicator flag; in the simplified mode, multiplying the gain factor by the HOA signal, wherein a dynamically range compressed HOA signal is obtained; and in the non-simplified mode, transforming the HOA signal to the spatial domain, wherein a transformed HOA signal is obtained; multiplying the gain factor by the transformed HOA signal, wherein a dynamically range compressed transformed HOA signal is obtained; and transforming the dynamically range compressed transformed HOA signal back to the HOA domain, wherein a dynamically range compressed HOA signal is obtained.

Additionally, one embodiment of the present invention relates to an apparatus for performing DRC on a HOA signal, the apparatus including a processor or one or more processing elements adapted to: set or determine a mode, which is a simplified mode or a non-simplified mode; in the non-simplified mode, transforming the HOA signal to the spatial domain, wherein inverse DSHT is used; in the non-simplified mode, analyzing the transformed HOA signal; and in the simplified mode, analyzing the HOA signal to obtain one or more gain factors from the results of the analysis, wherein only one gain factor is obtained in the simplified mode, and two or more different gain factors are obtained in the non-simplified mode; in the simplified mode, multiplying the obtained gain factors by the HOA signal, wherein a gain-compressed HOA signal is obtained; and in the non-simplified mode, multiplying the obtained gain factors by the transformed HOA signal, wherein a gain-compressed transformed HOA signal is obtained, and transforming the gain-compressed transformed HOA signal back to the HOA domain, wherein a gain-compressed HOA signal is obtained.

In one embodiment, only for the non-simplified mode, the device performing DRC on the HOA signal includes a processor or one or more processing elements adapted to: transform the HOA signal to the spatial domain, analyze the transformed HOA signal, obtain a gain factor from the result of the analysis that can be used for dynamic range compression, multiply the obtained factor by the transformed HOA signal, wherein a gain-compressed transformed HOA signal is obtained, and transform the gain-compressed transformed HOA signal back to the HOA domain, wherein a gain-compressed HOA signal is obtained. In one embodiment, the device further includes a transmission unit that transmits the HOA signal along with the obtained one or more gain factors before multiplying by them.

It should also be noted that the order of dividing the HOA signal into frequency subbands and transforming the HOA signal into the spatial domain can be interchanged, and the order of synthesizing the subbands and transforming the HOA signal with gain compression back into the HOA domain can also be interchanged; they are independent of each other.

In another embodiment, the present invention relates to an apparatus for applying a DRC gain factor to a HOA signal, the apparatus including a processor or one or more processing elements adapted to: receive the HOA signal together with an indicator flag and one or more gain factors, the indicator flag indicating a simplified mode or a non-simplified mode, wherein if the indicator flag indicates a simplified mode, only one gain factor is received; set the apparatus to a simplified mode or a non-simplified mode according to the indicator flag; in the simplified mode, multiply the gain factor by the HOA signal, wherein a dynamically range-compressed HOA signal is obtained; and in the non-simplified mode, transform the HOA signal to the spatial domain, wherein a transformed HOA signal is obtained; multiply the gain factor by the transformed HOA signal, wherein a dynamically range-compressed transformed HOA signal is obtained; and transform the dynamically range-compressed transformed HOA signal back to the HOA domain, wherein a dynamically range-compressed HOA signal is obtained.

In one embodiment, the device further includes a transmission unit that transmits the HOA signal along with the obtained gain factor before multiplying it by the obtained gain factor. In one embodiment, the HOA signal is divided into frequency sub-bands, and the following processes are applied to each frequency sub-band: analyzing the transformed HOA signal, obtaining the gain factor, multiplying the obtained gain factor by the transformed HOA signal, and transforming the gain-compressed transformed HOA signal back to the HOA domain, wherein there is a separate gain for each sub-band.

In one embodiment of the device that applies a DRC gain factor to a HOA signal, the HOA signal is divided into multiple frequency sub-bands, and the following processes are applied to each frequency sub-band: obtaining one or more gain factors, multiplying the obtained gain factors by the HOA signal or a transformed HOA signal, and transforming the gain-compressed transformed HOA signal back to the HOA domain in a non-simplified mode, wherein there is a separate gain for each sub-band.

Additionally, in one embodiment where only the non-simplified mode is used, the present invention relates to an apparatus for applying a DRC gain factor to a HOA signal, the apparatus comprising a processor or one or more processing elements adapted to: receive the HOA signal along with the gain factor; transform the HOA signal to the spatial domain (using iDSHT), wherein the transformed HOA signal is obtained; multiply the gain factor by the transformed HOA signal, wherein a dynamically range-compressed transformed HOA signal is obtained; and transform the dynamically range-compressed transformed HOA signal back to the HOA domain (i.e., the coefficient domain) (using DSHT), wherein a dynamically range-compressed HOA signal is obtained.

Tables 4-6 below list the spherical positions of the virtual loudspeaker for an order N of HOA, where N = 4, 5, or 6.

While the basic novel features of the invention, applicable to its preferred embodiments, have been shown, described, and pointed out, it should be understood that those skilled in the art can make various omissions, substitutions, and changes in the described apparatus and methods, in the form and details of the disclosed devices, and in their operation, without departing from the spirit of the invention. All combinations of these elements that are clearly intended to perform substantially the same function in substantially the same manner to achieve the same result are within the scope of the invention. Element substitutions from one described embodiment to another are also entirely contemplated and conceivable.

It should be understood that the invention has been described by way of example only, and modifications to the details may be made without departing from the scope of the invention. Each feature disclosed in the specification and (suitable) claims and drawings may be provided independently or in any suitable combination. Features may be suitably implemented in hardware, software, or a combination of both.

References:

[1] "Integration nodes for the sphere", Fliege 2010, online accessed2010-10-05http://-www.mathematik.uni-dortmund.de/lsx/research/projects/fliege/nodes/nodes.html

[2] "A two-stage approach for computing cubature formulae for thesphere", Fliege and Ulrike Maier, Technical report, Fachbereich Mathematik, Dortmund, 1999

N=4 positions

Table 4: Spherical position of the virtual loudspeaker of HOA order N=4

N=5 position

 

Table 5: Spherical position of the virtual loudspeaker of HOA order N=5

N=6 positions

 

Table 6: Spherical position of the virtual loudspeaker of HOA order N=6

Claims (8)

1. A method for performing Dynamic Range Compression (DRC), the method comprising: Receives and reconstructs high-order high-fidelity stereo HOA audio signal representation; The reconstructed HOA audio signal representation is transformed to the spatial domain based on W _DSHT = D _DSHT C, where D _DSHT is the inverse discrete spherical harmonic transform DSHT, is a block of τ HOA samples, and is a block of spatial samples that matches the input time granularity of the orthogonal mirror filter QMF group, where N indicates the order of the HOA. The DRC gain value g(n, m) corresponding to the time-frequency segment (n, m) is applied based on the following formula: Where is the vector for the spatial channel of the time-frequency segment (n,m), where n indicates the time slice and m indicates the frequency band, and represents the vector for the spatial channel of the time-frequency segment (n,m) with DRC applied. Wherein, when a simplified mode indication exists, g(n, m) is a single gain factor corresponding to a single gain value. 2. The method of claim 1, wherein the reconstructed HOA audio signal representation is divided into frequency sub-bands, and the DRC gain value is applied to each sub-band respectively. 3. The method according to claim 1, wherein, at least if (N+1) ^² < τ, where N is the order of the HOA, then the method further comprises: The gain vector of the DRC gain value is transformed to the HOA domain, where G is the gain matrix and _DL is the DSHT matrix defining the DSHT; and The gain matrix G is applied to the HOA coefficients B of the reconstructed HOA audio signal based on B _DRC = GB, where the HOA signal B _DRC compressed by DRC is obtained. 4. An apparatus for performing Dynamic Range Compression (DRC), the apparatus comprising: Receiver, receiving the reconstructed high-order high-fidelity stereo HOA audio signal representation; The audio decoder is configured as follows: The reconstructed HOA audio signal representation is transformed to the spatial domain based on W _DSHT = D _DSHT C, where D _DSHT is the inverse discrete spherical harmonic transform DSHT, is a block of τ HOA samples, and is a block of spatial samples that matches the input time granularity of the orthogonal mirror filter QMF group, where N indicates the order of the HOA. The DRC gain value g(n, m) corresponding to the time-frequency segment (n, m) is applied based on the following formula: Where is the vector for the spatial channel of the time-frequency segment (n,m), where n indicates the time slice and m indicates the frequency band, and represents the vector for the spatial channel of the time-frequency segment (n,m) with DRC applied. Wherein, when a simplified mode indication exists, g(n, m) is a single gain factor corresponding to a single gain value. 5. The device of claim 4, wherein the reconstructed HOA audio signal representation is divided into frequency sub-bands, and the gain value is applied to each sub-band respectively. 6. The device of claim 4, wherein, at least if (N+1) ^² < τ, where N is the HOA order, the audio decoder is further configured as follows: The gain vector of the DRC gain value is transformed to the HOA domain, where G is the gain matrix and _DL is the DSHT matrix defining the DSHT; and The gain matrix G is applied to the HOA coefficients B of the reconstructed HOA audio signal based on B _DRC = GB, where the HOA signal B _DRC compressed by DRC is obtained. 7. An apparatus for performing dynamic range compression, comprising: One or more processors, and One or more storage media, storing instructions that, when executed by the one or more processors, cause the method according to any one of claims 1-3 to be performed. 8. A computer-readable storage medium storing instructions that, when executed by one or more processors, cause to perform the method according to any one of claims 1-3.