US20180359582A1

US20180359582A1 - Method and Device for Estimating Acoustic Reverberation

Info

Publication number: US20180359582A1
Application number: US15/778,146
Authority: US
Inventors: Arthur Belhomme; Yves Grenier; Roland Badeau; Eric Humbert
Original assignee: Invoxia SAS
Current assignee: Invoxia SAS
Priority date: 2015-11-26
Filing date: 2016-11-21
Publication date: 2018-12-13
Anticipated expiration: 2036-11-21
Also published as: FR3044509B1; US10313809B2; EP3381204A1; EP3381204B1; WO2017089688A1; FR3044509A1

Abstract

A method for estimating the acoustic reverberations in an environment comprising the following steps: a measurement step in which one acoustic signal emitted in the environment is captured; a step for determination of acoustic energy decay rate distribution during which an acoustic energy decay rate distribution is determined from the acoustic signal captured in step (a); an estimation step during which a reverberation time and a reverberation level of sound in the environment are estimated by regression from the characteristic function of the acoustic energy decay rate distribution determined in step (b).

Description

FIELD OF THE INVENTION

This invention relates to methods and devices for estimating acoustic reverberation.

BACKGROUND OF THE INVENTION

Estimating the acoustic reverberation of an environment is essential for capturing acoustic signals such as speech in a reverberating environment such as for example a room in a building.
When a sound is emitted and then captured by a microphone in a reverberating environment, the microphone captures not only the signal received directly, but also signals reverberating in the environment.
This reverberation is reflected by the impulse response of the environment, from which emerges various known parameters, in particular the reverberation time. The impulse response is directly measurable by emitting an acoustic impulse in the environment, but this method is burdensome and hard to imagine for making repeated measurements while one or more speakers talk in the room.
The reverberation time can be estimated blind, for example while one or more speakers talk. The most commonly used parameter for representing the reverberation time is the reverberation time at 60 dB RT₆₀.
As an example, the document US 2014/169,575 describes a method for blind estimation of reverberation time in a room.
However, the reverberation time is not representative of the distance between the emitter and the microphone, which however has a significant impact on the reverberation level. The captured acoustic signals can therefore not be satisfactorily processed with the known methods of the aforementioned type.

PURPOSE AND SUMMARY OF THE INVENTION

Therefore the purpose of the present invention is to propose a method for estimating the acoustic reverberation with which to avoid this disadvantage.
For this purpose, the invention proposes a method for estimating the acoustic reverberations in an environment comprising the following steps:
(a) a measurement step in which at least one acoustic signal in the environment is captured;
(b) an observation step during which an acoustic energy decay rate distribution is determined from the acoustic signal captured in step (a) and the characteristic function of the acoustic energy decay rate distribution is determined;
(c) an estimation step during which a characteristic reverberation time and a characteristic reverberation level of the sound in the environment are estimated from data representative of the acoustic energy decay rate distribution determined in step (b), where the regression is done with reference to:

- reference characteristic functions representative respectively of several acoustic energy decay rate distributions;
- reference characteristic reverberation times corresponding to said reference characteristic functions; and
- reference characteristic reverberation levels corresponding to said reference characteristic functions.

Because of these arrangements, and in particular because of the fact that the estimation method is applied to the acoustic energy decay rate distribution, both a characteristic reverberation time and a characteristic reverberation level can be reliably determined for the sound in the environment. The captured sound signals can be processed satisfactorily with these two parameters.
In various embodiments of the method according to the invention, one and/or another of the following dispositions can possibly be used:

- during the estimation step (c), a kernel function estimator is used and the characteristic reverberation time and the characteristic reverberation level are determined simultaneously;
- during the estimation step (c), a Nadaraya-Watson estimator is used;
- during the estimation step (c), the characteristic reverberation level of the sound in the environment (7) is chosen among the clarity index C_τ and the definition index D_τ;
- during the observation step (b), the energy decay rates are determined by calculating the energy E_mof the acoustic signal on successive signal frames m, and then calculating a logarithmic ratio between the energy of two successive frames:

$\begin{matrix} ρ (m) = \log (\frac{E_{m}}{E_{m - 1}}); & (5) \end{matrix}$

- the method further comprises a preliminary calibration phase comprising the following steps:
  (a′) at least one initial reference signal determination step in which a plurality of reference acoustic signals corresponding to said reference characteristic reverberation times and said reference characteristic reverberation levels are determined;
  (b′) at least one initial observation step during which, an acoustic energy decay rate distribution and the reference characteristic function are determined for each reference acoustic signal;
- during said reference signal determination step, at least one part of the reference acoustic signals and the reference characteristic reverberation times and characteristic reverberation levels corresponding to said reference acoustic signals are determined by calculation from a predetermined set of impulse responses;
- during said reference signal determination step, at least one part of the reference acoustic signals, the characteristic reverberation times and the reference characteristic reverberation levels corresponding to said reference acoustic signals are determined by measurement.

Further, an object of the invention is also a device for estimating the acoustic reverberation in an environment, comprising:
(a) means of measurement for capturing at least one acoustic signal emitted in the environment;
(b) means of determination of an acoustic energy decay rate distribution from the acoustic signal captured by the means of measurement, and for determining the characteristic function of the acoustic energy decay rate distribution;
(c) means of estimation of a characteristic reverberation time and a characteristic reverberation level of the sound in the environment from data representative of the acoustic energy decay rate distribution, where the regression is done with reference to:

BRIEF DESCRIPTION OF THE DRAWINGS

Other features and advantages of the invention will become apparent during the following description of one of the embodiments thereof, given as a nonlimiting example, with reference to the attached drawings.

In the drawings:

FIG. 1 is a schematic view showing the reverberation of sound in a room when a subject speaks so that their speech is captured by a device according to an embodiment of the invention;

FIG. 2 is a conceptual drawing of the device from FIG. 1.

MORE DETAILED DESCRIPTION

In the various figures, the same references designate identical or similar items.
The purpose of the invention is to estimate the acoustic reverberation of an environment 7, for example a room in a building such as shown schematically in FIG. 1, so as to process the acoustic signals captured by an electronic device 1 provided with a microphone 2. The electronic device 1 can for example be a telephone in the example shown, or a computer or something else.
When the sound is emitted in the environment 7, for example by the person 3, this sound propagates to the microphone 2 along various paths 4, either directly, or after reflection from one or more walls 5, 6 of the environment 7.
As shown in FIG. 2, the electronic device 1 can comprise for example a central electronic unit 8 such as a processor or other, connected to the microphone 2 and various other elements, including for example a speaker 9, keyboard 10 and screen 11. The central electronic unit 8 can communicate with an external network 12, for example a telephone network.
With the invention, the electronic device 1 is able to measure blind two characteristic parameters of the reverberation of the environment 7:

- a characteristic reverberation time, for example the reverberation time at 60 dB RT₆₀; and
- a characteristic reverberation level (for example clarity or definition index, or direct signal over reverberated signal index).

These parameters can be used for eliminating the effects of echoes or more generally for optimizing sound signals captured by the microphone 2. The parameters in question are estimated repetitively, so that the device 1 adapts for example to changes of speakers 3, movements of speakers 3, and movements of the device 1 or other objects in the environment 7.
The reverberation time at 60 dB RT₆₀can be defined by the inverse integration method of Manfred R. Schroeder (New Method of Measuring Reverberation Time, The Journal of the Acoustical Society of America, 37(3):409, 1965) by the Energy Decay Curve (EDC):
EDC(n)=Σ_k=n ^N ^h h(k)² (1)
where:

- h is the impulse response of the environment of length N_h,
- n is a temporal index, for example a number of samples obtained with constant time step sampling; n is included between 1 and N_h.

RT₆₀is the time at temporal index n required for EDC(n) to decrease 60 dB.
Although the reverberation time RT₆₀is the most commonly used, another reverberation time characteristic of the environment 7 could be estimated.
The reverberation level is most commonly represented by the clarity index:
$\begin{matrix} C_{τ} = 10 \log_{10} (\frac{\sum_{n = 0}^{N_{τ}} h^{2} (n)}{\sum_{n = N_{τ} + 1}^{\infty} h^{2} (n)}) dB, & (2) \end{matrix}$
or by the definition index:
$\begin{matrix} D_{τ} = 10 \log_{10} (\frac{\sum_{n = 0}^{N_{τ}} h^{2} (n)}{\sum_{n = 0}^{\infty} h^{2} (n)}) dB, & (3) \end{matrix}$
where:

- N_τ is the number of samples at constant time step corresponding to the time τ, generally included between 0.1 ms and 1 s;
- n is a temporal index included between 1 and N_τ, representative of the number of samples of constant time step;
- h(n) is the impulse response of the environment 7.

These indexes were described in particular by P. A. Naylor and N. D. Gaubitch (Speech Dereverberation, Springer, Eds., edition, 2010).
The two most commonly used values of τ are 50 ms and 80 ms, in particular 50 ms (C₅₀and D₅₀indexes), but other lengths are possible and more generally other indexes reflecting the ratio of direct sound to reverberated sound could be estimated in the method according to the invention, implemented for example by the aforementioned electronic central unit 8.
This method comprises the following steps:
(a) an acoustic signal measurement step;
(b) an observation step during which an acoustic energy decay rate distribution is determined from acoustic signals measured in step (a);
(c) an estimation step during which a characteristic reverberation time and a characteristic reverberation level of sound in the environment 7 are estimated by regression from the acoustic energy decay rate distribution determined in step (b).
(a) Measurement Step:
During this step, the microphone 2 captures “blind” (meaning without prior knowledge of the emitted signals) an acoustic signal broadcast in the environment 7, for example while the speaker 3 talks. The signal is sampled and stored in the processor 8 or an attached memory (not shown).
(b) Observation Step:
During this step, an acoustic energy decay rate distribution is determined from the acoustic signal measured in step (a);
To do that, the reverberated signal energy envelope d_x(n) is determined such as described in particular by Wen et al. (J. Y. C. Wen, E. A. P. Habets, and P. A. Naylor, Blind estimation of reverberation time based on the distribution of signal decay rates, Acoustics, Speech and Signal Processing, 2008, ICASSP 2008, IEEE International Conference pages 329-332, March 2008).
By doing a calculation on the signal sample frames N_ω separated by jumps of R signal samples, a total energy of the frame m can be calculated with the formula:
E _m=Σ_i=0 ^N ^ω ⁻¹ d _x(mR+i) (4)
and next estimate the energy decay rate by calculating the logarithmic ratio of two successive frames:
$\begin{matrix} λ_{x} \approx ρ (m) = \log (\frac{E_{m}}{E_{m - 1}}) . & (5) \end{matrix}$
In fact, the energy envelope d_x(n) can be expressed by the formula:
$\begin{matrix} d_{x} (n) = {\begin{matrix} (e^{λ_{h} n} - e^{λ_{s} n}) / (λ_{h} - λ_{s}) & if λ_{h} \neq λ_{s} \\ {ne}^{λ_{h} n} s & if λ_{h} = λ_{s} \end{matrix} & (6) \end{matrix}$
where λ_sand λ_hare respectively the energy decay rate of the anechoic signal emitted and of the environment 7 (the captured signal is a convolution of the emitted anechoic signal (speech) with the impulse response of the environment between the speaker 3 and the microphone 2, where n is the previously defined temporal index).
Since the sum is dominated by the exponential term corresponding to the largest value of λ, the energy decay rate of the reverberated signal λ_xcan be approximated by:
λ_x=max[λ_h,λ_s] (7),
which justifies the formula (5) above.
The calculation of ρ(m) can typically be done on a number of frames, M, at least 2000, corresponding to at least 1 min. of signal depending on the selected analysis parameters. The frames can have an individual length of 10 to 100 ms, in particular of order 32 ms. The frames can mutually overlap, for example with an overlap rate of order 50% between successive frames.
The result is thus different values of the energy decay rate ρ(m), which have some statistical distribution (number of executions, or probability of execution depending on the energy decay rate ρ(m), as discussed for example in the article by Wen et al. above).
The characteristic function of the energy decay rate distribution is next determined by the following formula (see Audrey Feuerverger and Roman A. Mureika [The empirical characteristic function and its applications, Ann. Statist., 5(1):88-97, 01 1977]):
ϕ_X(ƒ)=∫e ^iƒx dF _X(x)=E[e ^iƒx] (8)
where X here represents the aforementioned energy decay rate ρ(m) estimated for various values of m (formula (5)), F_Xrepresents the cumulative distribution of X and f is a dimensionless variable generally called angular frequency.
The characteristic function can be calculated for angular frequencies f ranging for example from 0 to 0.4, by increments of 0.001.
(c) Estimation step:
Start with the characteristic function Φ_ρ(m)(f), calculated for p/2 frequencies f (where p is an even integer), where the frequency range f and their sampling are intended such that |Φ_ρ(m)(f)| is preferably included between 0.1 and 1.
Typically, p can be included between 256 and 2048.
Because the characteristic function is a complex number, it can be represented by a vector X from
^p, constituting the random input vector x of the estimator used. The random output vector y of the estimator, belonging to
², has the two estimated parameters as its components, for example (RT₆₀, C₅₀) or (RT₆₀, D₅₀).
The estimator used can advantageously be a kernel function estimator, for example a Nadaraya-Watson estimator. Such an estimator has the advantage of simultaneously determining the characteristic reverberation time and the characteristic reverberation level.
The estimator in question can be determined in advance in an initial calibration phase, where at least one initial step of reference signal determination (a′) and at least one initial step of observation (b′) is implemented.
During the initial step of reference signal determination a plurality of reference acoustic signals, and corresponding reference characteristic reverberation times and reference characteristic reverberation levels are determined.
During the initial observation step, the acoustic energy decay rate distribution and the reference characteristic function are determined for each reference acoustic signal in away identical or similar to the aforementioned observation step (b).
The reference acoustic signals are N generally voice signals and correspond to N different scenarios (e.g. different speakers, different positions, different environments 7). N can be several hundred or even several thousand.
The initial reference signal determination step can be done:

- with new real measurements done for example with an electronic device 1 of a fixed model (in this case, the characteristic reverberation time and the characteristic reverberation level can also be measured);
- and/or with synthetic acoustic signals.

In the case of real measurements, these will not generally be done in the specific environment 7 where the electronic device 1 will be used, even though this scenario can be considered.
The aforementioned synthetic acoustic signals can be calculated by convolution of the prerecorded impulse responses with anechoic speech signals, also prerecorded, coming from different speakers. Prerecorded impulse responses can, for example, come from impulse response databases, for example, coming from free access databases such as the databases: Aachen Impulse Response (http://www.openairlib.net/auralizationdb), MARDY (Wen et al., Evaluation of speech dereverberation algorithms using the Mardy database, September IWAENC 2006, Paris), QueenMary (R. Stewart and M. Sandler, Database of omnidirectional and b-format room impulse responses, In Acoustics Speech and Signal Processing (ICASSP). 2010 IEEE International Conference on., pages 165-168, March 2010), for example with reverberation times RT₆₀ranging from 0.3 s to 8 s and clarity indexes C₅₀from −10 dB to 25 dB. The anechoic speech signals recorded from various speakers, for example various ages and genders, with for example recording lengths for example of a few minutes, for example of order five minutes.
The energy decay rate distributions can for example be calculated on 10 to 100 ms frames, in particular of order 32 ms. The frames can mutually overlap, for example with an overlap rate of order 50% between successive frames. The characteristic functions can be calculated for angular frequencies f ranging for example from 0 to 0.4, by increments of 0.001.
In that way N executions of the aforementioned x and y vectors result and the Nadaraya-Watson estimator can then be determined with the formula:
$\begin{matrix} \hat{f} (x) = \frac{\sum_{i = 1}^{N} y_{i} K_{λ} (x, x_{i})}{\sum_{i = 1}^{N} K_{λ} (x, x_{i})} . & (9) \end{matrix}$
where:

- x_i, y_i, i=1 to N, are the N executions of the vectors x, y used for the calibration step;
- K_λ(x, x_i) is a kernel function with window X (where X is a constant also called smoothing parameter);
- x is the unknown input vector (measurement done at the measurement step (a) in order to estimate the vector y with the formula y={circumflex over (ƒ)}(x)).

The kernel function K_λ(x, x_i) is a function of x and x_isuch as defined in particular by Scholkopf et al. (B. Scholkopf and A. J. Smola, Learning with Kernels, MIT Press, Cambridge, Mass., 2001).
The Gaussian kernel can in particular be used, for example with a window of λ=5·10⁻⁴(nonlimiting example):
$K_{λ} (x, x_{i}) = \frac{1}{λ} e^{\frac{- { x - x_{i} }^{2}}{2 λ}} .$
The tests performed show that the method from the invention is more precise than the methods from the prior art for the determination of reverberation time and it further serves to determine the reverberation level at the same time as the reverberation time, which is a significant improvement.

Claims

1. A method for estimating the acoustic reverberations in an environment comprising the following steps:

(a) a measurement step in which at least one acoustic signal emitted in the environment is captured;

(b) an observation step during which an acoustic energy decay rate distribution is determined from the acoustic signal measured in step (a) and the characteristic function of the acoustic energy decay rate distribution is determined;

(c) an estimation step during which a characteristic reverberation time and a characteristic reverberation level of the sound in the environment thereof are estimated by regression from said characteristic function determined in step (b), where the regression is done with reference to:

reference characteristic functions representative respectively of several acoustic energy decay rate distributions;

reference characteristic reverberation times corresponding to said reference characteristic functions; and

reference characteristic reverberation levels corresponding to said reference characteristic functions.

2. The method according to claim 1 wherein during the estimation step (c), a kernel function estimator is used and the characteristic reverberation time and the characteristic reverberation level are determined simultaneously.

3. The method according to claim 2 wherein during the estimation step (c), a Nadaraya-Watson estimator is used.

4. The method according to claim 1 wherein during the estimation step (c), the characteristic reverberation level of the sound in the environment is chosen among the clarity index Cτ and the definition index Dτ.

5. The method according to claim 1 wherein during the observation step (b), the energy decay rates are determined by calculating the energy Em of the acoustic signal on successive signal frames m, and then calculating a logarithmic ratio between the energy of two successive frames:

ρ (m) = \log (\frac{E_{m}}{E_{m - 1}})

6. The method according to claim 1 further comprises a preliminary calibration phase comprising the following steps:

(a′) at least one initial reference signal determination step in which a plurality of reference acoustic signals corresponding to said reference characteristic reverberation times and said reference characteristic reverberation levels are determined;

(b′) at least one initial observation step during which, an acoustic energy decay rate distribution and the reference characteristic function are determined for each reference acoustic signal.

7. The method according to claim 6 wherein during said reference signal determination step, at least one part of the reference acoustic signals and the reference characteristic reverberation times and characteristic reverberation levels corresponding to said reference acoustic signals are determined by calculation from a predetermined set of impulse responses.

8. The method according to claim 6 wherein during said reference signal determination step, at least one part of the reference acoustic signals, the characteristic reverberation times and the reference characteristic reverberation levels corresponding to said reference acoustic signals are determined by measurement.

9. A device for estimating the acoustic reverberations in an environment comprising:

at least a measurement device for capturing at least one acoustic signal emitted in the environment;

at least a processor adapted to:

determination an acoustic energy decay rate distribution from the acoustic signal captured by the means of measurement, and for determining the characteristic function of the acoustic energy decay rate distribution;

estimate a characteristic reverberation time and a characteristic reverberation level of the sound in the environment from data representative of the acoustic energy decay rate distribution, where the regression is done with reference to: