EP1305975B1

EP1305975B1 - Adaptive microphone array system with preserving binaural cues

Info

Publication number: EP1305975B1
Application number: EP01942048A
Authority: EP
Inventors: Fa-Long Luo; Jun Yang; Brent Edwards; Nick Michael
Original assignee: GN Resound AS
Current assignee: GN Hearing AS
Priority date: 2000-06-13
Filing date: 2001-06-05
Publication date: 2011-11-23
Anticipated expiration: 2021-06-05
Also published as: EP1305975A2; DK1305975T3; EP1305975A4; WO2002003749A3; ATE535103T1; WO2002003749A2

Abstract

A microphone system using left and right microphones produces good binaural cues as well as noise reduction by using two adaptive filters (56, 58). A noise enhanced reference signal (46) is sent to two different adaptive filters (56, 58). The output of each adaptive filter is combined with the left or right microphone signal, respectively, to produce left and right output signals. These left and right output signals are used as the error signals to modify the coefficients of the adaptive filters (56, 58). By using two adaptive filters, each of the adaptive filters is able to independently operate helping to preserve the binaural cues.

Description

Background of the Invention

The present invention relates to adaptive microphone array systems.
The combination of multiple-microphone-based spacial processing with binaural listening is receiving increasing attention in many application fields such as hearing aids because this combination should be able to provide both the spacial filtering benefits of a microphone array and the natural benefits of binaural listening for sound-location ability and speech intelligibility. Typically, a microphone is positioned on each side of a user's head. Ideally, the system is designed such that the two microphones can be used to improve the signal-to-noise ratio as well as to maintain the binaural cues.
One proposed system is described in the reference, Microphone-Array Hearing Aids With Binaural Output - Part I: Fixed-Processing Systems, and Part II: A Two-Microphone Adaptive System, IEEE Transactions on Speech and Audio Processing. Vol. 5, No. 6 (Nov. 1997) pp 529-551 and shown in Fig. 1. In this example, X_R(n) is the microphone signal received at the right ear, and X_L(n) is the microphone signal received at the left ear. A low-pass filter and a high-pass filter with cut-off frequency fc are used at each channel. The outputs f_R(n) and f _L (n) of the two high-pass filters are sent to an adaptive processor 22 whose output is Y(n). The output of the two low- pass filters 24 and 26 are delayed in delays 28 and 30, and combined with the output Y(n) of the adaptive processor 22 to provide binaural outputs Z_R(n) and Z_L(n). In this prior-art scheme, the combination of the adaptive array processing with binaural listening is accomplished by dividing the frequency spectrum. The low-pass portion of the frequency spectrum is devoted to binaural processing, and the high-pass part of the spectrum is devoted to adaptive array processing. The binaural cues of signal parts with higher frequencies than the cut-off frequency will be lost in the system. Likewise, the benefits of the adaptive array processing for lower frequencies than the cut-off frequency also will be lost. Furthermore, two low-pass filters and two high-pass filters are required. More importantly, appropriate equalization processing between the adaptive array processing and the outputs of the low-pass filters are required so as to avoid unexpected artifacts that result from a simple cutoff of the spectrum and simple summation. These problems make this prior system complicated and fail as a practical solution in achieving maximum array processing benefits and binaural listening benefits.
It is desired to have an improved adaptive array microphone system that maintains binaural cues and provides spacial filtering benefits.

Summary of the Present Intention

The present invention comprises a microphone system using two adaptive filters, each receiving the same reference signal derived from two ear microphones but having different primary and error (filter adjustment) signals. The primary signals are preferably from the right and left microphone output signals, respectively.
One embodiment of the present invention comprises an apparatus including a noise-enhanced data producing unit receiving left- and right-ear microphone data and producing noise-enhanced data. A right adaptive unit receives the right microphone data and the noise-enhanced data, and produces a reduced-noise right data output. The right adaptive unit includes a first adaptive filter receiving the noise-enhanced data. A left adaptive unit receives the left microphone data and the noise-enhanced data. The left adaptive unit produces reduced-noise left data output. The left adaptive unit includes a second adaptive filter receiving the noise-enhanced data.
Another embodiment of the present invention is a method including the steps of: calculating the noise-enhanced data from the left and right microphone data; adaptively filtering the noise-enhanced data to produce first filtered noise-enhanced data; combining the first filtered noise-enhanced data with right microphone data to produce a reduced-noise right data output; adaptively filtering the noise-enhanced data to produce second filtered noise-enhanced data; and combining the second filtered noise-enhanced data with left microphone data to produce a reduced-noise left data output.

Brief Description of the Drawings

Fig. 1 is a diagram of a prior-art microphone system.
Fig. 2 is a diagram of a microphone system of one embodiment of the present invention.
Fig. 3 is a diagram of one embodiment of the present invention implementing the system of Fig. 2.
Fig. 4 is a flow chart illustrating the operation of one embodiment of the present invention.

Detailed Description of the Preferred Embodiment

Fig. 2 is a diagram illustrating the functional portions of the microphone system of the present invention. X_R(n) 42 comprises data from the right microphone. X_L(n) 44 comprises the data from the left microphone. Data r(n) is derived from these two data signals. In a preferred embodiment, a noise-enhanced signal unit 48 produces an output, r(n), with enhanced noise. In one embodiment, the noise-enhanced signal unit 48 includes a summing unit 50 which subtracts one of the microphone signals from the other. This produces a noise-enhanced signal since sound coming from the front, including presumably the desired speech signal, will reach the left and right microphones at nearly the same time, thus forming a null in the output response at the front.
The noise-enhanced signal is applied to the right adaptive unit 52 and left adaptive unit 54. Preferably, the right adaptive unit includes a first adaptive filter 56 receiving the noise-enhance signal r(n). The right adaptive unit 52 preferably receives X_R(n) as a primary signal to adjust the adaptive filter 56. In one embodiment, the unit 52 also includes a summing unit 59, which subtracts the output of the adaptive filter 56 from the right front signal X_R(n) to produce the output Z_R(n). The left adaptive unit also includes a second adaptive filter 58. The second adaptive filter also receives the reference signal r(n) and produces an output a_L(n). The left adaptive unit 54 receives the output X_L(n) as the primary signal to modify the adaptive filter coefficients.
Note that although the first and second adaptive filters receive the same reference signal, the coefficients of the adaptive filters are different since the primary and error signals used to modify the coefficients are different for the first and second adaptive filters.
As described below, the adaptive filters can be of a variety of different types which are known in the art. In one preferred embodiment, the adaptive filters use the same algorithm but, as described above, have different primary signals.
One embodiment of the system of the present invention is described in the following mathematical description. The received signals at the right ear microphone and the left ear microphone are X_R(n) and X_L(n) which consist of the target speech parts S_R(n), S_L(n) and the noise parts N_R(n), N_L(n), respectively, that is $X_{R} (n) = S_{R} (n) + N_{R} (n)$

and $X_{L} (n) = S_{L} (n) + N_{L} (n)$
The reference signal r(n) = X_R(n) - X_L(n) is sent to two adaptive filters with weights W _R(n) = [W _R1(n), W _R2(n)......., W _RN(n)]^T and W _L(n) = [W _L1(n), W_L2(n) ........ W_LN(n)]^T and these two adaptive filters provide the outputs a_R(n) and a_L(n) as follows, respectively: $a_{R} (n) = \sum_{m = 1}^{N} W_{Rm} (n) r (n - m + 1) = {W^{T}}_{R} (n) R (n)$

and $a_{L} (n) = \sum_{m = 1}^{N} W_{Lm} (n) r (n - m + 1) = {W^{T}}_{L} (n) R (n)$

where R(n)=[r(n),r(n-1),.....,r(n-N+1)] ^T and N is the length of two adaptive filters. Note that the length of two filters is here selected to be the same for simplicity but could be different. The primary signals at two adaptive filters are X_R(n) and X_L(n), and two outputs: Z_R(n) and Z_R(n) for the right ear and the left ear, respectively, are obtained by $Z_{R} (n) = X_{R} (n) - a_{R} (n)$

and $Z_{L} (n) = X_{L}^{ʹ} (n) - a_{L} (n)$
The weights of these two adaptive filters are adjusted so as to minimize the average power of the two outputs, that is, $\min_{W_{R} (n)} E ({|Z_{R} (n) ()|}^{2}) = \min_{W_{R} (n)} E ({|X_{R} (n) - a_{R} (n) ()|}^{2})$

and $\min_{W_{L} (n)} E ({|Z_{L} (n) ()|}^{2}) = \min_{W_{L} (n)} E ({|X_{L} (n) - a_{L} (n) ()|}^{2})$

In the ideal case r(n) contains only the noise part and the two adaptive filters can provide two outputs a_R(n) and a_L(n) by minimizing the two foregoing equations. Since the reference signal r(n) does not include the signal portion, the adaptive filters will adjust a_R(n) and a_L(n) to remove the noise portion from the primary signals, that is, a _L (n)≅N _L (n) and a _L (n)≅N _L (n). As a result, two system outputs Z_R(n) and Z_L(n) will approximate the signal parts S_R(n) and S_L(n), respectively. This means that the above processing not only realizes maximum noise reduction by two adaptive filters but also preserves the binaural cues contained in the target signal parts S_R(n) and S_L(n).
The learning algorithm can be an adaptive algorithm such as the LS, RLS, TLS and LMS algorithms. The LMS algorithm version to update the coefficients of two adaptive filters is $W_{R} (n + 1) = W_{R} (n) + λ R (n) Z_{R} (n)$
$W_{L} (n + 1) = W_{L} (n) + λ R (n) Z_{L} (n)$

where λ is a step parameter which is a positive constant less than $\frac{2}{P}$
where P is the power of the input r(n) of these two adaptive filters. For better performance and faster convergence speed, λ can be also time varying as done in the normalized LMS algorithm uses, that is, $W_{R} (n + 1) = W_{R} (n) + \frac{μ}{{‖ R (n) ‖}^{2}} R (n) Z_{R} (n)$
$W_{L} (n + 1) = W_{L} (n) + \frac{μ}{{‖ R (n) ‖}^{2}} R (n) Z_{L} (n)$

where µ is a positive constant less than 2.
Based on the frame-by-frame processing configuration, a further modified algorithm can be obtained as follows (Denoted by Equation 1 and Equation 2): $W_{Rk} (n + 1) = W_{Rk} (n) + \frac{μ}{{‖ R (n) ‖}^{2}} R (n) Z_{Rk} (n)$

and $W_{Lk} (n + 1) = W_{Lk} (n) + \frac{μ}{{‖ R (n) ‖}^{2}} R (n) Z_{Lk} (n)$

where k represents the k'th repeating in the same frame.
In comparison with the prior-art scheme of Fig. 1, the advantages of the present invention include the following: First, no binaural cue of the target speech will be lost because two system outputs Z_R(n) and Z_L(n) will approximate the signal parts S_R(n) and S_L(n) respectively. In the prior scheme, the binaural cues of signals with higher frequencies than the cut-off frequency have been lost. Second, the array processing benefit for the part with frequencies less than the cut-off frequency will be preserved because the frequency spectrum is not divided. Note that in the prior scheme, this benefit also has been lost. Third, in the present invention no low-pass filters and high-pass filters are required, while the prior scheme requires two low-pass filters and two high-pass filters. Consequently, the related equalization processing is not required in the present invention. The only cost involved in implementing the present invention is to include an additional adaptive filter. The two adaptive filters can use the same structure and adaptive algorithm. This property results in a large convenience in hardware implementation because the related assembly code and machine code of two adaptive filters can be shared.
Fig. 3 is a diagram that illustrates an embodiment of the system of the present invention. The system 70 includes a right ear microphone 72 and a left ear microphone 74. The converters 76 and 78 convert the analog signals into digital signals which are sent to the processor 80. In a preferred embodiment, the processor 80 is a digital signal processor. The processor 80 loads the adaptive microphone array program 82 from memory 84. The adaptive microphone array program can implement the functional blocks as shown in Fig. 2. Other programs 86 such as hearing aid algorithms can also be stored in the memory 84 for loading into the processor 80. The output signals of the processor 80 can be sent to speakers if a hearing-aid system is used.
Fig. 4 is a flow chart that illustrates the operation of one embodiment of the present invention. In step 90, an enhanced noise signal is calculated using left and right microphone samples. This enhanced noise signal can be the reference signal constructed by subtracting one of the microphone samples from the other microphone sample. In step 92, a noise-enhanced signal and the right ear signal are used to produce a noise-reduced right-ear signal. This is preferably done by adaptively filtering the noise-enhanced signal and combining the filtered signal with the right-ear microphone system to produce an output. The output being used as an error signal for the adaptive filter. In step 94, the noise-enhanced signal and the left-ear signal are used to produce a noise-reduced left-ear signal. The output of a second adaptive filter is combined with the left-ear signal, producing the noise-reduced left-ear signal. The noise-reduced left-ear signal is then used as an error signal to affect the coefficients of the adaptive filter. Note that the order of steps 92 and 94 is not important. In step 96 the left and right noise-reduced signals are used to produce coefficients which are used in the adaptive filters used in the producing steps 92 and 94.
Any kind of adaptive algorithms, such as LMS-based, LS-based, TLS-based, RLS-based and related algorithms, can be used in this scheme. The weights can also be obtained by directly solving the estimated Wienner-Hopf equations. Moreover, repeated adaptive algorithms like Equations 1 and 2 and adaptive lattice filters can be used in this scheme as well. In one embodiment, the lengths of two adaptive filters are adjustable and can take different values. Also, the step parameters in related adaptive algorithms for two adaptive filters can take different values. Trade-offs between performance and cost (complexity, etc.) in practical applications determine which algorithm is used.
The two adaptive filters in Fig. 2 can be nonlinear filters and can be implemented by some neural networks such as multi-layer perceptron networks, radial basis function networks, high-order neural networks, etc. The corresponding learning algorithms in neural networks such as the back-propagation algorithm can be used for the adaptive filters.
A matching filter could be added in either the left ear channel or the right ear channel before obtaining the difference signal r(n) so as to compensate for the magnitude mismatch of two-ear microphones. The matching filter can be in either a finite impulse response (FIR) filter or an infinite impulse response (IIR) filter. The matching filter could also be either in a fixed model or in an adaptive model.
In one embodiment, to reduce the target signal cancellation problem existing in most adaptive array processing algorithms, a speech pause detection system is used and the weight update of the two adaptive filters is made during a pause in the speech.
Either a directional microphone or an omnidirectional microphone could be used in this invention. Some pre-processing methods could be used to improve the signal-to-noise ratio of two primary signals X_R(n) and X_L(n). These pre-processing methods include that more than one microphone can be used in each ear and then these microphone signals are combined in some way to produce the signals X_R(n) and X_L(n). This pre-processing could be either in a fixed model or in an adaptive model and also either in the spatial domain or in the temporal domain.

Claims

An apparatus comprising:
a noise-enhanced data producing unit receiving left and right ear microphone data and producing noise-enhanced data;

a right adaptive unit (52) receiving the right microphone data (42) and the noise-enhanced data, the right adaptive unit (52) producing reduced-noise right data output, the right adaptive unit (52) including a first adaptive filter (56) receiving the noise-enhanced data; and

a left adaptive unit (54) receiving the left microphone data (44) and the noise-enhanced data, the left adaptive unit (54) producing reduced-noise left data output, the left adaptive unit (54) including a second adaptive filter (58) receiving the noise-enhanced data.
The apparatus of Claim 1 wherein the first adaptive filter (56) uses the reduced-noise right data output as an error signal to modify the coefficients of the first adaptive filter (56).
The apparatus of Claim 1 wherein the second adaptive filter (58) uses the reduced-noise left data output as an error signal to modify the coefficients of the second adaptive filter (58).
The apparatus of Claim 1 wherein the apparatus is implemented as a software program on a processor-based system.
The apparatus of Claim 1 wherein the noise-enhanced-data-producing unit comprises a summer (50) to subtract one of the ear's microphone data from the other ear's microphone data.
The apparatus of Claim 1 wherein the right adaptive unit (52) includes a summer unit to subtract the output of the first adaptive filter (56) from the right microphone data (42) to produce the reduced-noise right-data output.
The apparatus of Claim 1 wherein the left adaptive unit (54) includes a summer unit to subtract the output of the second adaptive filter (58) from the left microphone data (44) to produce the reduced-noise left-data output.
A method comprising:
calculating noise-enhanced data from left and right microphone data; adaptive filtering the noise-enhanced data to produce first filtered noise-enhanced data;

combining the first filtered noise-enhanced data with right microphone data (42) to produce a reduced noise right data output;

adaptive filtering the noise-enhanced data to produce second filtered noise-enhanced data;

and combining the second filtered noise-enhanced data with left microphone data (44) to produce a reduced noise left data output.
The method of Claim 8 wherein the method is implemented on a processor.
The method of Claim 8 wherein the noise-enhanced data-calculating step comprises subtracting one of the microphone data from the other microphone data.
The method of Claim 8 wherein the first adaptive filtering step uses the reduced-noise right data output as an error signal to modify the coefficients of the first adaptive filter (56).
The method of Claim 8 wherein the second adaptive filtering step uses the reduced-noise left data output as an error signal to modify the coefficients of the second adaptive filter (58).
The method of Claim 8 wherein the combining step comprises subtracting the filtered noise-enhanced data from the microphone data.
A computer-readable medium containing a program which executes the following procedure:
calculating noise-enhanced data from left and right microphone data;

adaptive filtering the noise-enhanced data to produce first filtered noise-enhanced data;

combining the first filtered noise-enhanced data with right microphone data (42) to produce a reduced noise right data output;

adaptive filtering the noise-enhanced data to produce second filtered noise-enhanced data; and

combining the second filtered noise-enhanced data with left microphone data (44) to produce a reduced noise left data output.
The computer-readable medium of Claim 14 wherein the procedure is implemented on a processor.
The computer-readable medium of Claim 14 wherein the noise-enhanced data-calculating step comprises subtracting one of the microphone data from the other microphone data.
The computer-readable medium of Claim 14 wherein the adaptive filtering step uses the reduced-noise data output as an error signal to modify the coefficients of the adaptive filter.
The computer-readable medium of Claim 14 wherein the combining step comprises subtracting the filtered noise-enhanced data from the microphone data.