JP2010245984A - Device for correcting sensitivity of microphone in microphone array, microphone array system including the same, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable sensitivity correction to be properly performed without paying any special attention about whether or not conditions for properly correcting dispersion in sensitivity of each microphone configuring a microphone array are available. <P>SOLUTION: A sensitivity correcting device of the microphone produces a separation matrix for performing sound source separation relating to two observed signals to be otained by collecting a mixed sound of two sounds each emitted from different sound sources S (S1, S2) by each of two microphones M1, M2 so that its first line has a dead angle into a normal line direction of an array surface and its second line has the dead angle into an arrangement direction of the microphone. An incoming direction of the sound to be suppressed by matrix elements of the first line of the separation matrix is estimated, then a signal level of an output signal of any one of the microphones is corrected in response to a ratio of each absolute value of the matrix elements of the first line when the incoming direction is not largely separated from the normal line of the array surface. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a technique for correcting variations in sensitivity of microphones constituting a microphone array.

  A microphone array system is an example of a sound collection system in which a directivity pattern can be set so that only sound coming from a specific direction can be collected. The microphone array system includes a microphone array in which a plurality of microphones are arranged one-dimensionally or two-dimensionally, and filters such as FIR (Finite Impulse Response) filter processing are performed on audio signals output from the microphones constituting the microphone array. Processing is performed, and the filtered audio signal is mixed and output. The directivity pattern is adjusted by adjusting the filter coefficient of the filter processing.

  In this type of sound collection system, the sensitivity of each microphone needs to be uniform. This is because if the sensitivity of each microphone varies, it may hinder the adjustment of the directivity pattern. However, since the microphone is a mechanical component, manufacturing variations cannot be avoided, and variations in sensitivity of ± 4 dB or more may occur in the manufacturing stage. When the sensitivity of each microphone constituting the microphone array has a variation of about ± 4 dB, deterioration of directivity is inevitable. Therefore, various techniques for correcting variations in sensitivity of the microphones constituting the microphone array have been proposed (Patent Document 1, Patent Document 2, etc.). In Patent Document 1, any one of a plurality of microphones constituting a microphone array is used as a reference microphone, and the gain is adjusted so that the signal level of the output signal of the other microphone is equal to the level of the output signal of the reference microphone. Thus, a technique for correcting variation in sensitivity is disclosed. On the other hand, Patent Document 2 discloses a technique for correcting the sensitivity of other microphones by using, as a reference microphone, a microphone to which an acoustic signal having a constant frequency and a constant sound pressure is input for a predetermined time or more among a plurality of microphones constituting a microphone array. Is disclosed.

JP 7-131886 A JP 2007-24618 A

  However, in the technology that corrects the variation in sensitivity by adjusting the level of the output signal of another microphone on the basis of any one of the plurality of microphones constituting the microphone array, the sound source is directly opposed to the microphone array. If the sound source is not located in the direction that passes through the center of the array surface and is perpendicular to the array surface (hereinafter referred to as the normal direction of the array surface), the sensitivity correction cannot be performed properly. is there. This is because the sound wave from the remote sound source propagates through the space as a plane wave, and if the sound source is not facing the microphone array, it is observed at the position of each microphone due to the difference in distance between each microphone and the sound source. This is because the sound pressures of the sound waves to be produced are different from each other. Therefore, when the sensitivity of the microphone is corrected by the technique disclosed in Patent Document 1 or the like, it is sufficient whether or not conditions (such as a sound source facing the microphone array) that can appropriately perform sensitivity correction are prepared. There was a problem that it was necessary to pay attention and was troublesome. In order to solve such a problem, it is conceivable to estimate the arrival direction of sound based on the output signal of each microphone, and to perform sensitivity correction in consideration of the arrival direction. However, conventional techniques for estimating the direction of sound arrival, such as methods using steering vectors (including MVDR and MUSIC), assume that the sensitivity of each microphone constituting the microphone array is uniform. For this reason, this kind of direction-of-arrival estimation technique cannot be used as a premise for correcting variations in sensitivity of the microphones constituting the microphone array.

  The present invention has been made in view of the above problems, and without paying special attention to whether or not the conditions for appropriately correcting the sensitivity variations of the microphones constituting the microphone array are met. An object of the present invention is to provide a technique that enables appropriate correction.

  In order to solve the above problems, the present invention collects a mixed sound of M kinds of sounds (M is a natural number of 2 or more) radiated from different sound sources by each of the M microphones constituting the microphone array. A frequency analysis unit that performs frequency analysis on each of the obtained M observation signals and calculates time-series observation data indicating the signal intensity at each of the plurality of frequencies for each microphone, and at least one of the plurality of frequencies A separation matrix generation unit that generates a separation matrix that is a complex value matrix of M rows and M columns for performing sound source separation for the frequency component by independent component analysis with respect to observation data of the frequency component; For each row of the separation matrix generated by the matrix generation unit, a method for estimating the arrival direction of the sound suppressed by the matrix element of the row from the difference in the declination of the matrix element of each row When there is a row of the separation matrix in which the direction of arrival of the sound estimated by the estimation unit and the direction estimation unit is not greatly deviated from the normal direction of the microphone array, the ratio of absolute values of matrix elements of the row And a sensitivity correction unit that corrects variations in the signal level of the output signal of each microphone according to the above, and a microphone sensitivity correction device that constitutes a microphone array, and a computer that functions as the above-described units. Program.

  According to such a sensitivity correction apparatus and program, first, M types of sound sources are separated by independent component analysis using M observation signals output from each of the M microphones constituting the microphone array. A separation matrix of M rows and M columns to perform is calculated, and for each row of the separation matrix, the arrival direction of the sound suppressed by the row is estimated based on the difference in the declination of the matrix elements. If the separation matrix includes a row in which the direction of arrival of the sound estimated as described above is not greatly deviated from the normal direction of the microphone array, the ratio of the absolute values of the matrix elements of the row Accordingly, the variation in the signal level of the output signal of each microphone is corrected. Although details will be described later, when M = 2, the absolute value of the matrix element of the row that forms a blind spot in the normal direction of the array surface (that is, suppresses sound coming from the normal direction of the array surface). The ratio is equal to the ratio of the signal levels of the output signals of the two microphones (ie, the ratio of the sensitivity of the two microphones). Therefore, according to the present invention, a condition for appropriately correcting the sensitivity variation of each microphone constituting the microphone array (the method of the microphone array is included in M rows of the separation matrix generated by independent component analysis). Whether or not the condition that the sound coming from the line direction is included, in other words, the condition that any sound source is located in the normal direction of the array surface) is satisfied. Even if care is not taken, when the condition is satisfied, variations in sensitivity of the microphones constituting the microphone array are automatically corrected.

  When M = 2, the separation matrix generation unit of the sensitivity correction apparatus uses the initial separation matrix as a starting point for the independent component analysis, and the normal direction of the array surface of the microphone array with respect to the matrix element of one row The values are set so as to suppress the sound coming from the sound source, and the matrix elements in the other row are set so as to suppress the sound coming from the arrangement direction of the microphones on the array surface. The initial separation matrix that is the starting point for independent component analysis when M = 2 is set as described above. If sequential learning is performed using such an initial separation matrix, the normal direction of the array surface and the array This is because it is generally known that it becomes easy to obtain a separation matrix having a blind spot in the arrangement direction of microphones on the surface.

  In order to solve the above-mentioned problem, the present invention includes N-1 microphone arrays each including N (N is a natural number of 2 or more) microphones, and the sensitivity correction device when M = 2. Any one of the N microphones is a reference microphone, each of the other N-1 microphones is a sensitivity correction target microphone, and each of the N-1 sensitivity correction devices is the above-described microphone. Each of the N-1 sensitivity correction target microphones is connected to each of the N-1 sensitivity correction target microphones, and each of the N-1 sensitivity correction device is connected to the reference microphone. To provide a microphone array system that corrects the signal level of the output signal of each of the N-1 correction target microphones. According to such an aspect, the process of correcting the sensitivity of the other N−1 microphones based on the signal level of the output signal of the reference microphone is executed by each of the N−1 sensitivity correction devices. The Thereby, the variation in sensitivity of the N microphones constituting the microphone array is corrected.

It is a figure which shows the structural example of 100 A of microphone array systems which are 1st Embodiment of this invention. It is a figure for demonstrating the process which the frequency analysis part 22 of the sensitivity correction apparatus 20 contained in the system performs. It is a figure which shows the structural example of 40 A of separation matrix production | generation parts of the same sensitivity correction apparatus. It is a figure which shows the structural example of the sensitivity correction control part 28 of the sensitivity correction apparatus 20. FIG. It is a flowchart which shows the flow of the process which the correction amount calculation part 76 of the sensitivity correction control part 28 performs. It is a figure for demonstrating the mixing system and separation system of a sound in the embodiment. It is a figure which shows the structural example of the microphone array system 100B which is 2nd Embodiment of this invention. It is a figure which shows the structural example of the microphone array system 100 which is 3rd Embodiment of this invention. 3 is a block diagram illustrating a configuration example of a signal processing unit 24 of the arithmetic device 12 included in the microphone array system 100. FIG. 3 is a block diagram illustrating a configuration example of a separation matrix generation unit 40 of the arithmetic device 12. FIG.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
<A: First Embodiment>
FIG. 1 is a block diagram showing a configuration example of a microphone array system 100A according to the first embodiment of the present invention. The microphone array system 100A includes a microphone array 10A composed of n (n is a natural number of 2 or more) microphones. In the present embodiment, as shown in FIG. 1, a case is assumed in which the microphone array 10A is configured by two microphones M1 and M2 (n = 2). Each of the microphone M1 and the microphone M2 is arranged along the plane PL so as to be spaced from each other so that the sound collection axes are parallel to each other. For this reason, the array surface of the microphone array 10A is parallel to the plane PL. At different positions around the microphones M1 and M2, there are n sound sources S (S1, S2) in a plane including the sound collection axis of each microphone and the normal line of the array surface of the microphone array 10A. The sound source S1 is located in the direction of the angle θ1 with respect to the normal line Ln of the array surface of the microphone array 10A, and the sound source S2 is located in the direction of the angle θ2 (θ2 ≠ θ1) with respect to the normal line Ln.

  The sound SV1 emitted from the sound source S1 and the sound SV2 emitted from the sound source S2 reach both the microphone M1 and the microphone M2. The microphone M1 generates an observation signal V1 that represents the mixed sound waveform of the sound SV1 from the sound source S1 and the sound SV2 from the sound source S2. Similarly, the microphone M2 generates an observation signal V2 that represents a mixed sound waveform of the sound SV1 from the sound source S1 and the sound SV2 from the sound source S2. As shown in FIG. 1, the observation signal V2 output from the microphone M2 is supplied to the signal processing unit 30 through amplification of the signal level by the amplifier G2, while the observation signal V1 output from the microphone M1 is left as it is (amplified by the amplifier). Without passing through).

  The signal processing unit 30 includes a filter unit that performs filtering processing for directional speech on the observation signal V1 and the observation signal V2, and an adder that mixes and outputs the observation signal V1 and the observation signal V2 that have passed through the filter processing, respectively. (Both not shown). In the microphone array system 100A, the directivity pattern is set by adjusting the filter coefficient used in the filter processing. The signal output from the signal processing unit 30 is reproduced as sound by being supplied to a sound emitting device (for example, a speaker or headphones). Note that an A / D converter that converts the observation signal V1 and the observation signal V2 into a digital signal and a D / A converter that converts the output signal of the signal processing unit 30 into an analog signal are omitted.

  The sensitivity correction device 20 corrects variations in sensitivity of the microphone M1 and the microphone M2 by adjusting the signal level of the observation signal SV2 with reference to the signal level of the observation signal SV1. Although details will be described later, the sensitivity correction device 20 calculates a sensitivity correction amount R from the observation signal V1 and the observation signal V2 by a method that remarkably shows the characteristics of the present embodiment, and sets a gain corresponding to the sensitivity correction amount R. Set to amplifier G2. Thereby, the signal levels of the observation signal V1 and the observation signal V2 are substantially uniform, and variations in sensitivity between the microphone M1 and the microphone M2 are corrected.

  The sensitivity correction device 20 is a computer device such as a personal computer. A CPU (Central Processing Unit: not shown) of the sensitivity correction device 20 executes a program stored in the storage device 14 to execute sensitivity correction processing that significantly shows the features of the present embodiment. The storage device 14 stores the above program (hereinafter referred to as a sensitivity correction support program) and various data. As the storage device 14, a known recording medium such as a semiconductor recording medium or a magnetic recording medium is employed.

  The CPU of the sensitivity correction apparatus 20 executes a sensitivity correction support program, and functions as the frequency analysis unit 22, the separation matrix generation unit 40A, and the sensitivity correction control unit 28 illustrated in FIG. In this embodiment, each of the frequency analysis unit 22, the separation matrix generation unit 40A, and the sensitivity correction control unit 28 is realized by software. However, the frequency analysis unit 22, separation matrix generation is performed by an electronic circuit dedicated to signal processing such as a DSP. The unit 40A and the sensitivity correction control unit 28 may be realized, or a configuration in which these units are distributedly mounted on a plurality of integrated circuits may be employed.

  The frequency analysis unit 22 calculates the frequency spectrum Q (the frequency spectrum Q1 of the observation signal V1 and the frequency spectrum Q2 of the observation signal V2) for each of a plurality of frames obtained by dividing the observation signal V (V1, V2) on the time axis. . For the calculation of the frequency spectrum Q, for example, a short-time Fourier transform is used. As shown in FIG. 2, the frequency spectrum Q1 of one frame identified by the number (time) t has an intensity x1 (t, f1) at each of the K frequencies f1 to fK set on the frequency axis. Calculated as ~ x1 (t, fK). Similarly, the frequency spectrum Q2 is calculated as intensities x2 (t, f1) to x2 (t, fK) at each of the K frequencies f1 to fK.

The frequency analysis unit 22 generates observation vectors X (t, f1) to X (t, fK) for each frame for the K frequencies f1 to fK. The observation vector X (t, fK) of the kth (k = 1 to K) frequency fk is common to the intensity x1 (t, fk) at the frequency fk in the frequency spectrum Q1, as shown in FIG. Vector (X (t, fk) = [x1 (t, fk) ^* x2 (t, fk) ^* ] ^H whose elements are the intensity x2 (t, fk) at the frequency fk in the frequency spectrum Q2 of the frame of Symbol * means complex conjugate, and symbol H means matrix transposition (Hermitian transposition) Observation vectors X (t, f1) to X (t, fK) generated by the frequency analysis unit 22 for each frame. ) Is stored in the storage device 14. The observation vectors X (t, f1) to X (t, fK) stored in the storage device 14 are, as shown in FIG. Observation data D (f1 for each unit interval TU consisting of ~D is divided into (fK). Observed data D (fk) of the frequency fk is a time series of observed vectors X (t, fk) of the calculated frequency fk for each frame in the unit interval TU.

  The separation matrix generation unit 40A generates separation matrices W (f1) to W (fK) from the observation data D (fk) by so-called independent component analysis. Here, the separation matrix is essentially two rows and two columns (n rows and n columns) used for signal processing operations for separating the sound SV1 or the sound SV2 (or both) from the observation signal V1 and the observation signal V2. ) Complex-valued matrix. However, this embodiment is characterized in that variations in sensitivity between the microphone M1 and the microphone M2 are corrected using this separation matrix.

FIG. 3 is a block diagram of the separation matrix generator 40A.
As illustrated in FIG. 3, the separation matrix generation unit 40A includes an initial value generation unit 42, a frequency selection unit 54, and a learning processing unit 44. The initial value generation unit 42 generates initial separation matrices (hereinafter referred to as “initial separation matrix”) W ₀ (f1) to W ₀ (fK) for each of the K frequencies f1 to fK. The initial separation matrix W ₀ (fk) corresponding to the frequency fk is generated for each unit interval TU using the observation data D (fk) stored in the storage device 14. Initial separation matrix _W 0 _(f1) as the technique of generating to W-0 (fK) may be appropriately used a known technique. Here, various forms of the initial separation matrices W ₀ (f1) to W ₀ (fK) are generated. In the present embodiment, a so-called blind spot beamformer is employed. . More specifically, when the observation signals V1 and V2 are multiplied by the initial separation matrix for each of the frequencies f1 to fK, these two observation signals and the matrix element in the first row of the separation matrix (that is, ( 1,1) and (1,2) components), the sound coming from the normal direction of the array surface of the microphone array 10A is suppressed (that is, the normal direction becomes a blind spot). In the signal obtained by multiplying these two observation signals and the matrix elements (that is, the (2,1) component and the (2,2) component) of the second row of the initial separation matrix, each microphone in the microphone array 10A. The initial separation matrix is set so that the sound coming from the arrangement direction is suppressed (that is, the arrangement direction becomes a blind spot). In the present embodiment, since the initial separation matrix is set as described above, the separation matrix of the blind spot beamformer, that is, the sound coming from the blind spot direction is suppressed for each row of the separation matrix (in other words, other than the blind spot) By emphasizing the sound coming from the direction), a separation matrix for sound source separation is generated.

  The frequency selection unit 54 selects one or a plurality of frequencies to be learned from the separation matrix by independent component analysis from among the K types of frequencies f1 to fK according to the arrangement interval of the microphone M1 and the microphone M2 in the microphone array 10A. Select accordingly. More specifically, the frequency selection unit 54 selects one or a plurality of frequencies having high array gain and low aliasing noise from among the K types of frequencies f1 to fK in relation to the arrangement interval of the microphones M1 and M2. select. Hereinafter, the frequency selected by the frequency selection unit 54 is referred to as “selected frequency”. That is, in the present embodiment, learning of the separation matrix using independent component analysis is performed only for the frequency selected by the frequency selection unit 54 among the K types of frequencies f1 to fK. The reason is as follows.

  When learning the separation matrix for the purpose of sound source separation, it is ideal to calculate the separation matrix for all of the K types of frequencies f1 to fK. However, the present embodiment is not intended for sound source separation, but for the purpose of estimating the direction of arrival of sound and correcting the sensitivity of each microphone, so long as the separation matrix can be calculated within a range that can achieve that purpose. It is enough. Therefore, in the first embodiment, one or more frequencies having a high array gain and no aliasing noise are selected from the K frequencies f1 to fK based on the arrangement interval of the microphone M1 and the microphone M2 in the microphone array 10A. However, the amount of calculation required for sensitivity correction is reduced by performing sequential learning of the separation matrix W (fk) using the observation data D (fk) only for those frequencies.

以上が分離行列生成部４０Ａの構成である。 The learning processing unit 44 generates a separation matrix W (fk) by sequential learning using the initial separation matrix W ₀ (fk) as an initial value for each of the selection frequencies fk selected by the frequency selection unit 54. For the learning of the separation matrix W (fk), observation data D (fk) of the frequency fk stored in the storage device 14 is used. For example, the separation signal U1 (time series of the intensity u1 (t, fk) defined by Equation 1) and the separation signal U2 (Equation 2) obtained by multiplying the observation data D (fk) by the separation matrix W (fk). Independent component analysis (for example, higher-order ICA) that repeatedly updates the separation matrix W (fk) so that the defined intensity u2 (t, fk) is statistically independent of each other is performed. It is preferably used for generating W (fk). In the following equations 1 and 2, wij (fk) is an i-row j-column component of the separation matrix W (fk).

The above is the configuration of the separation matrix generation unit 40A.

Next, the configuration of the sensitivity correction control unit 28 will be described.
FIG. 4 is a block diagram showing the configuration of the sensitivity correction control unit 28. As shown in FIG. 4, the sensitivity correction control unit 28 includes a direction estimation unit 72 and a correction amount calculation unit 76.

The direction estimation unit 72 is supplied with data indicating the selected frequency fk and the separation matrix W (fk) after learning by the learning processing unit 44. The direction estimation unit 72 determines the arrival direction of the sound (specifically, the normal of the array plane) from each separation matrix W (fk) after learning with respect to the selected frequency fk, by each row of the separation matrix W (fk). The angle formed by Ln and the direction of arrival of the sound is estimated. More specifically, the direction estimating unit 72 determines the difference between the declinations of the matrix elements in the first row of the separation matrix W (fk) after learning by the learning processing unit 44 (that is, the declination of w11 (fk) and w12). The arrival direction θ1 (fk) of the sound suppressed by the matrix element of the first row is estimated from the difference of the declination angle of (fk), and the difference of the declination of the matrix elements of the second row (that is, w21 ( The direction of arrival θ2 (fk) of the sound suppressed by the matrix element in the second row is estimated from the difference between the deviation angle of fk) and the deviation angle of w22 (fk). For estimation of the arrival direction θ1 (fk) and the arrival direction θ2 (fk) using the matrix elements of the separation matrix W (fk), H. Saruwatari, et. Al., “Blind Source Separation Combining
Independent Component Analysis and Beamforming ", EURASIP Journal on
The methods disclosed in Applied Signal Processing Vol. 2003, No. 11, pp. 1135-1146, 2003 can be used. For example, if the difference between the deviation angle of w11 (fk) and the deviation angle of w12 (fk) is zero, the arrival direction θ1 (fk) of the sound suppressed by the matrix element in the first row of the separation matrix is the microphone array 10A. The normal direction of the array surface is estimated.

  The correction amount calculation unit 76 calculates a sensitivity correction amount R for the microphone M2 from the separation matrix W (fk) after learning by the learning processing unit 44, and sets a gain corresponding to the correction amount R to the amplifier G2. Execute. FIG. 5 is a flowchart showing a flow of processing executed by the correction amount calculation unit 76. As shown in FIG. 5, the correction amount calculation unit 76 suppresses the arrival direction θ1 (fk) of the sound estimated by the direction estimation unit 72 for each selected frequency fk (that is, the first row of the separation matrix W (fk)). It is determined whether or not the arrival direction of the generated sound is significantly deviated from the normal direction of the array surface, and the frequency fk determined to be deviated significantly is excluded from the selected frequency (step SA100). For example, when the absolute value of the angle indicating the arrival direction (that is, θ1 (fk) or θ2 (fk)) exceeds a predetermined threshold, the correction amount calculation unit 76 determines that the arrival direction is the normal line of the array surface. Judged to be significantly out of direction. Here, the frequency in which the arrival direction θ1 (fk) is significantly deviated from the normal direction of the array surface is excluded even if the calculation after step SA120 is performed on the separation matrix corresponding to such a frequency. This is because improvement in sensitivity correction cannot be expected.

Next, the correction amount calculation unit 76 determines whether or not all of the selected frequencies have been excluded in Step SA100 (Step SA110). If the determination result is “No” (that is, excludes the selected frequencies fk). Step SA120 and the subsequent steps are executed only when there is something that has not been done. In this step SA120, the correction amount calculation unit 76 does not exclude the frequency in step SA100 (that is, the frequency at which the arrival direction θ1 (fk) is determined not to deviate significantly from the normal direction of the array surface). Sensitivity correction amount R (fk) for the microphone M2 from the matrix elements in the first row of the separation matrix w (fk) for each of fk (ie, w11 (fk) and w12 (fk)) according to the following Equation 3. Is calculated. In Equation 3, || means an absolute value.

Here, the reason why the sensitivity correction amount R (fk) for the microphone M2 can be calculated according to the above-mentioned equation 3 is as follows. A mixed system of the sound SV1 radiated from the sound source S1 and the sound SV2 radiated from the sound source S2 is represented as shown in FIG. 6, and the sensitivities of the microphone M1 and the microphone M2 are not uniform, and the gain is only on the microphone M1 side. When there is a difference between the signal level of the observation signal V1 and the signal level of the observation signal V2 (see FIG. 6) as if p is present, the observation signal V1 and the observation signal V2 are expressed by the following equation 4 from the sound SV1 and the sound SV2. The mixing matrix A generated according to the above is expressed by the following formula 5. In Equation 5 below, a _ij is a transfer function of a sound propagation path from the sound source Sj to the microphone Mi.

In this case, one of the separation matrix W candidates for separating the sound SV1 and the sound SV2 from the observation signal V1 and the observation signal V2 is an inverse matrix A- ¹ of the mixing matrix A. The separation matrix W in this case is given by the following Equation 6. The separation matrix W suppresses the sound emitted from the sound source S2 by the matrix element in the first row, and suppresses the sound emitted from the sound source S1 by the matrix element in the second row.

When the matrix element of the first row of the separation matrix W forms a blind spot in the normal direction of the array surface (the sound suppressed by the first row of the separation matrix W (fk) is the method of the array surface of the microphone array 10A. When coming from the line direction, that is, when the sound source S2 is directly facing the array surface, the distance from the sound source S2 to the microphone M1 is equal to the distance from the sound source S2 to the microphone M2, and a ₁₂ = a ₂₂ It becomes. Accordingly, when the matrix element in the first row of the separation matrix W forms a blind spot in the normal direction of the array surface, the ratio R of the matrix elements W11 and W12 in the first row of the separation matrix W is given by The magnitude of the value R that is calculated and calculated according to Equation 7 is equal to the sensitivity ratio p between the microphone M1 and the microphone M2.

  Therefore, when the first row of the separation matrix W (fk) forms a blind spot in the normal direction of the array surface, a gain corresponding to R (fk) calculated according to the above equation 3 is given to the amplifier G2. By setting, it is possible to correct variations in sensitivity between the microphone M1 and the microphone M2.

Then, the correction amount calculation unit 76 has a plurality of values R (selected frequencies fk) representing the correction amount R (fk) calculated according to Equation 3 for each of the frequencies fk not excluded in step SA100. In the case where only one selected frequency fk remains, such as an arithmetic mean or median value of R (fk) calculated for each of the selected frequencies fk, the R calculated for the selected frequency fk. (Fk)) is obtained (step SA130). Then, the correction amount calculation unit 76 sets the gain corresponding to R calculated in step SA130 in the amplifier G2 (step SA140), and completes the sensitivity correction.
The above is the flow of processing executed by the correction amount calculation unit 76.

  As described above, in the microphone array system 100A, the conditions for appropriately correcting the sensitivity of the microphones constituting the microphone array 10A are met (the sound source is located in the normal direction of the array surface). ) Is automatically detected, and the sensitivity correction device 20 executes a process of correcting variations in sensitivity of the microphones M1 and M2. As a result, variations in sensitivity of each microphone are automatically corrected without paying special attention to the above conditions.

  Note that the sensitivity of each microphone constituting the microphone array 10A needs to be corrected only once at the time of factory shipment or immediately after the start of operation. Therefore, a flag indicating whether sensitivity correction has been executed (if the value is 0) If the value is 1, the sensitivity is corrected if the value is 1, the initial value “0” is set and written in the storage device 14, and while this flag value is 0, the sensitivity correction is periodically supported. The program may be executed by the CPU of the sensitivity correction apparatus 20, and the CPU may be caused to execute a process of updating the flag to 1 when the process of step SA140 is executed. In this embodiment, the ratio of the absolute values of the matrix elements in the first row of the separation matrix W (fk) (value R (fk) calculated according to Equation 3 or R (fk) for a plurality of selection frequencies fk ), The variation in sensitivity of the microphone M1 and the microphone M2 is corrected by adjusting the signal level of the output signal of the microphone M2, and the signal level of the output signal of the microphone M1 is set to the above R (fk). It is of course possible to correct the variation in sensitivity of both microphones by adjusting the reciprocal of (or the reciprocal of the value representing R (fk)).

<B: Second Embodiment>
Next, a second embodiment of the present invention will be described. In the first embodiment, the microphone array system 100A is configured by using the microphone array 10A including two microphones M (M1, M2). On the other hand, in the second embodiment, the microphone array system 100B is configured using a microphone array 10B composed of three or more microphones M (M1, M2... MN: N is a natural number of 3 or more). FIG. 7 is a block diagram illustrating a configuration example of the microphone array system 100B. As shown in FIG. 7, in the microphone array system 100B, (N−1) microphones Mk (k = 2 to N) other than the microphone M1 are each subjected to signal processing via an amplifier Gk (k = 2 to N). Connected to the unit 30. The microphone M1 and the microphone Mk (k = 2 to N) are connected to a sensitivity correction device 20-k (k = 2 to N), and the gain of the amplifier Gk is adjusted by the sensitivity correction device 20-k. Is done. Each of these sensitivity correction devices 20-k (k = 2 to N) has the same configuration as the sensitivity correction device 20 of FIG.

  That is, in the microphone array system 100B, the microphone M1 is used as a reference microphone, and the sensitivity correction of the other (N−1) microphones Mk (k = 2 to N) is performed by the sensitivity correction device 20-k (k = 2 to N). ) Each. Thus, when the conditions for correcting the sensitivity of each microphone Mk constituting the microphone array 10B are met, the sensitivity correction of each microphone Mk is sequentially executed. As described above, according to the present embodiment, even when the microphone array is composed of three or more microphones, the microphones are automatically and without paying special attention to the user of the microphone array system 100B. Variation in sensitivity of each microphone constituting the array 10B can be corrected.

Here, when the microphone array is composed of N microphones as shown in FIG. 7, it is conceivable to perform an N-channel independent component analysis to correct variations in sensitivity of each microphone. In particular,
A frequency analysis is performed on each of the N observation signals obtained by collecting a mixed sound of N kinds of sounds emitted from different sound sources by each of the N microphones constituting the microphone array, and a plurality of frequencies are obtained. A frequency analysis unit that calculates time-series observation data indicating the signal intensity at each microphone;
Select at least one of the plurality of frequencies and generate an N-by-N complex-value matrix separation matrix for performing sound source separation on the frequency component by independent component analysis on the observation data of the frequency component A separating matrix generating unit,
For each row of the separation matrix generated by the separation matrix generation unit, a direction estimation unit that estimates the arrival direction of the sound suppressed by the matrix element of the row from the difference in declination of the matrix element of each row;
When there is a row of the separation matrix in which the direction of arrival of the sound estimated by the direction estimation unit is not greatly deviated from the normal direction of the microphone array, depending on the ratio of absolute values of the matrix elements of the row A sensitivity correction unit is configured by combining a sensitivity correction unit that corrects variation in the signal level of the output signal of each microphone, and the N microphones and N-1 amplifiers are connected to the sensitivity correction device to connect the microphones. Of course, an array system may be configured.

As to which of the mode of performing N-channel independent component analysis and the mode of combining N-1 sensitivity correction devices that perform 2-channel independent component analysis as in this embodiment, the microphone array system is configured. It may be determined depending on whether the configuration of the microphone array system is preferably simplified or it is preferable that the amount of computation required for the computation of the separation matrix is reduced. In the aspect in which the N-channel independent component analysis is performed, the configuration of the microphone array system is simplified because only one sensitivity correction device is required. On the other hand, in the embodiment in which the microphone array system is configured by combining N-1 sensitivity correction apparatuses that perform 2-channel independent component analysis as in the present embodiment, it is compared with the embodiment in which N-channel independent component analysis is performed. Therefore, the calculation amount is reduced. In the N-channel independent component analysis, the amount of calculation required for the sequential learning of the separation matrix is proportional to N ² , whereas in the aspect in which N−1 sensitivity correction devices that perform 2-channel independent component analysis are combined, the amount of calculation is the same. This is because is proportional to 2 ² × (N−1).

<C: Third Embodiment>
In the first and second embodiments described above, the variation in sensitivity of each microphone constituting the microphone array is corrected using the separation matrix W (fk) generated by the separation matrix generation unit 40A. However, it goes without saying that sound source separation may be performed using the separation matrix W (fk). FIG. 8 is a block diagram illustrating a configuration example of the microphone array system 100 that performs filtering (sound source separation) on the observation signal V1 and the observation signal V2 to generate the separation signals U1 and U2. A microphone array system 100 illustrated in FIG. 8 includes a microphone array including a microphone M1 and a microphone M2, an arithmetic device 12 that performs an operation of generating a separation signal U1 and a separation signal U2 from the observation signal V1 and the observation signal V2, and a storage device. 14 and so on. In FIG. 8, the same components as those in FIG. 1 are denoted by the same reference numerals. Hereinafter, a description will be given focusing on differences from the system shown in FIG.

  As shown in FIG. 8, the arithmetic device 12 includes a frequency analysis unit 22, a signal processing unit 24, a signal synthesis unit 26, and a separation matrix generation unit 40. The arithmetic device 12 is a computer device, similar to the sensitivity correction device 20 in the first embodiment described above, and causes the CPU to execute a program stored in the storage device 14, thereby causing the frequency analysis unit 22 and the signal processing unit 24 to operate. , Function as a signal synthesis unit 26 and a separation matrix generation unit 40.

  The signal processing unit 24 in FIG. 8 performs the filtering process (sound source separation) on the intensity x1 (t, fk) and the intensity x2 (t, fk) calculated by the frequency analysis unit 22 to sequentially increase the intensity for each frame. u1 (t, fk) and intensity u2 (t, fk) are generated. The signal synthesis unit 26 converts the strengths u1 (t, f1) to u1 (t, fK) generated by the signal processing unit 24 into time domain signals and connects them with the preceding and subsequent frames to generate the separated signal U1. Similarly, the signal synthesizer 26 converts the strengths u2 (t, f1) to u2 (t, fK) into signals in the time domain and connects them with the preceding and succeeding frames to generate a separated signal U2.

  FIG. 9 is a block diagram of the signal processing unit 24. As illustrated in FIG. 9, the signal processing unit 24 includes K processing units P1 to PK corresponding to the K frequencies f1 to fK, respectively. The processing unit Pk corresponding to the frequency fk includes a filter 32 that generates the intensity u1 (t, fk) from the intensity x1 (t, fk) and the intensity x2 (t, fk), and the intensity x1 (t, fk) and the intensity x2. And a filter 34 that generates an intensity u2 (t, fk) from (t, fk).

  For the filters 32 and 34, a delay addition type (DS (delay-sum) type) beamformer is used. That is, the filter 32 of the processing unit Pk includes a delay element 321 that adds a delay corresponding to the coefficient w11 (fk) to the intensity x1 (t, fk) and a coefficient w12 (fk), as defined by the above equation 1. A delay element 323 that adds a delay corresponding to the intensity x2 (t, fk), and the output of the delay element 321 and the output of the delay element 323 are added to generate the intensity u1 (t, fk) of the separated signal U1. And an adder 325. Similarly, the filter 34 includes a delay element 341 that adds a delay corresponding to the coefficient w21 (fk) to the intensity x1 (t, fk) and a coefficient w22 (fk) as defined in the above equation 2. A delay element 343 that adds a delay to the intensity x2 (t, fk), and an adder 345 that adds the output of the delay element 341 and the output of the delay element 343 to generate the intensity u2 (t, fk) of the separated signal U2. Including.

  FIG. 10 is a block diagram illustrating a configuration example of the separation matrix generation unit 40. The separation matrix generation unit 40 generates a separation matrix by performing independent component analysis using the observation data D (fk) in the same manner as the separation matrix generation unit 40A in the first embodiment described above. As illustrated in FIG. 10, the separation matrix generation unit 40 includes an initial value generation unit 42, a learning processing unit 44, and a frequency selection unit 54. Then, the separation matrix generation unit 40 applies each matrix element of the separation matrix W (fk) generated by the learning processing by the learning processing unit 44 with respect to the selected frequency fk to the filter 32 and the filter 34 of the processing unit Pk of the signal processing unit 24. Set each one.

  In addition, the separation matrix generation unit 40 includes a direction estimation unit 72 and a matrix supplementation unit 74 as shown in FIG. The direction estimation unit 72 estimates arrival directions θ1 (fk) and θ2 (fk) of sounds separated by each row of the separation matrix W (fk) generated by the learning processing unit 44 for each of the selected frequencies fk, and A value θ1 representative of θ1 (fk) (arithmetic mean or median of θ1 (fk)) and a value θ2 representative of θ2 (fk) are calculated, and data indicating θ1 and θ2 is given to the matrix supplementation unit 74. The matrix supplementation unit 74 in FIG. 10 generates a separation matrix for frequencies that are not selected by the frequency selection unit 54 among the K types of frequencies f1 to fK (hereinafter, non-selected frequencies) in the following manner. The signal processing unit 24 is provided. That is, the matrix supplementation unit 74 follows the same algorithm as the generation of the initial separation matrix in the initial value generation unit 42 described above, and the separation matrix for the non-selected frequency becomes a blind spot in the θ1 direction for the first row. The eyes are generated so that the θ2 direction is a blind spot.

  In the conventional sound source separation using the separation matrix, in order to reduce the amount of calculation required to generate the separation matrix, a specific frequency (selected frequency fk in the present embodiment) among the K types of frequencies f1 to fK is selected. In general, only the separation matrix is learned, and the initial separation matrix generated by the initial value generation unit 42 is used as it is for other frequencies. In the frequency band using the separation matrix obtained by the learning process, the variation in sensitivity of each microphone constituting the microphone array is corrected via the separation matrix, but in the frequency band using the initial separation matrix, the sensitivity of each microphone is corrected. There was a problem that the variation was not corrected, the blind spot was not properly formed due to the variation in sensitivity of each microphone, and the sound source separation accuracy deteriorated. On the other hand, in this embodiment, sound source separation is performed with high accuracy by using a separation matrix generated so as to form a blind spot in a direction estimated from the separation matrix obtained by the learning process for the non-selected frequencies. It becomes possible.

<D: Deformation>
As mentioned above, although each embodiment of this invention was described, it is needless to say that the following modifications may be added to these embodiments.
(1) In each of the above-described embodiments, the frequency for learning the separation matrix is selected according to the arrangement interval of the microphones on the array surface of the microphone array. However, the frequency may be selected based on another scale. One example of such a measure is the significance of learning (the degree of improvement when learning the separation matrix improves the accuracy of sound source separation compared to sound source separation using the initial separation matrix). Can be considered. Here, as an index indicating the significance of learning, for example, the determinant z1 (fk) of the covariance matrix Rxx (fk) of the observation data D (fk) for each of the K frequencies f1 to fK is preferable. It is known that there is. Specifically, the frequency fk at which the determinant z1 (fk) exceeds a predetermined threshold is selected as a learning target. The covariance matrix Rxx (fk) is defined by the following formula 8. Symbol E in the following equations 8 and 9 means an expected value (added value), and symbol Σ_ {t} means addition (average) over a plurality of (for example, 50) frames in the unit interval TU. That is, the covariance matrix Rxx (fk) is obtained by multiplying the observation vector X (t, fk) by the transpose of the observation vector X (t, fk) within a unit interval TU (in the observation data D (fk)). It is an n-by-n matrix added for the observation vector X (t, fk). However, in the following Equation 9, the addition of the observation vectors X (t, fk) over all the frames in the unit interval TU is assumed to be a zero matrix (zero average).

(2) In each of the embodiments described above, a blind spot is formed in the normal direction of the array surface of the microphone array by the matrix element in the first row as the initial separation matrix W ₀ (fk), and the matrix element in the second row Thus, a blind spot beamformer that forms a blind spot in the direction of arrangement of each microphone in the microphone array was used, but the role of the matrix element in the first row and the role of the matrix element in the second row were used interchangeably. Also good. In this way, a blind spot beam that forms a blind spot in the microphone array direction in the microphone array by the matrix element in the first row and forms a blind spot in the normal direction of the array surface of the microphone array by the matrix element in the second row. When the former one is used, whether or not the arrival direction of the sound suppressed by the matrix element in the second row of the separation matrix W (fk) generated by the sequential learning is greatly deviated from the normal direction of the array surface. If not, the correction target microphone (in the first embodiment, the microphone) is determined according to the ratio of the absolute values of the matrix elements in the second row (that is, | w22 | / | w21 |). M2, in the second embodiment, sensitivity correction may be performed by adjusting the gain of the output signal of the microphones M2 to MN).

(3) In each of the above-described embodiments, the sensitivity correction apparatus that significantly shows the features of the present invention is incorporated in the microphone array system in advance. However, the sensitivity correction apparatus is provided as a single unit, and each part of the sensitivity correction apparatus is connected to the microphone array. It may be configured to be connected to each unit to have a configuration similar to that of the microphone array system 100A or the microphone array system 100B.

(4) In the above-described embodiment, the program for causing the CPU to perform the microphone sensitivity correction characteristic of the present invention is stored in the storage device 14 in advance. However, the program may be distributed by being written on a computer-readable recording medium such as a CD-ROM (Compact Disk-Read Only Memory), or the program may be distributed by downloading via a telecommunication line such as the Internet. May be.

  100A, 100B, 100 ... microphone array system, 10A, 10B ... microphone array, M1, M2, MN ... microphone, 20, 20-2, 20-3 ... 20-N ... sensitivity correction device, 12 ... arithmetic device, 22 ... Frequency analysis unit, 14 ... storage device, 40A, 40 ... separation matrix generation unit, 42 ... initial value generation unit, 44 ... learning processing unit, 54 ... frequency selection unit, 28 ... sensitivity correction control unit, 72 ... direction estimation unit, 74: Matrix replenishment unit, 76 ... Correction amount calculation unit.

Claims (4)

Each of M observation signals obtained by collecting a mixed sound of M kinds of sounds (M is a natural number of 2 or more) radiated from different sound sources with each of the M microphones constituting the microphone array has a frequency. A frequency analysis unit that performs analysis and calculates time-series observation data indicating the signal strength at each of a plurality of frequencies for each microphone;
Select at least one of the plurality of frequencies, and generate a separation matrix that is a complex value matrix of M rows and M columns to perform sound source separation for the frequency component by independent component analysis on the observation data of the frequency component A separating matrix generating unit,
For each row of the separation matrix generated by the separation matrix generation unit, a direction estimation unit that estimates the arrival direction of the sound suppressed by the matrix element of the row from the difference in declination of the matrix element of each row;
When there is a row of the separation matrix in which the direction of arrival of the sound estimated by the direction estimation unit is not greatly deviated from the normal direction of the microphone array, depending on the ratio of absolute values of the matrix elements of the row And a sensitivity correction unit that corrects variations in the signal level of the output signal of each microphone. A microphone sensitivity correction apparatus constituting a microphone array. When M = 2, the separation matrix generation unit
The initial separation matrix that is the starting point for the independent component analysis is set to a value that suppresses sound coming from the normal direction of the array surface of the microphone array with respect to the matrix element of one row, and the other row. The sensitivity correction apparatus according to claim 1, wherein values are set so as to suppress sound coming from the arrangement direction of the microphones on the array surface. A microphone array composed of N (N is a natural number of 2 or more) microphones;
The sensitivity correction device according to claim 1, wherein M = 2 is provided, and N−1 sensitivity correction devices are provided.
Any one of the N microphones is a reference microphone, each of the other N-1 microphones is a sensitivity correction target microphone, and each of the N-1 sensitivity correction devices is the above-described microphone. Each of the N-1 sensitivity correction target microphones is connected to each of the N-1 sensitivity correction target microphones, and each of the N-1 sensitivity correction device is connected to the reference microphone. The microphone array system, wherein the signal level of the output signal of each of the N-1 correction target microphones is corrected by: Computer
Each of M observation signals obtained by collecting a mixed sound of M kinds of sounds (M is a natural number of 2 or more) radiated from different sound sources with each of the M microphones constituting the microphone array has a frequency. A frequency analysis unit that performs analysis and calculates time-series observation data indicating the signal strength at each of a plurality of frequencies for each microphone;
Select at least one of the plurality of frequencies, and generate a separation matrix that is a complex value matrix of M rows and M columns to perform sound source separation for the frequency component by independent component analysis on the observation data of the frequency component A separating matrix generating unit,
For each row of the separation matrix generated by the separation matrix generation unit, a direction estimation unit that estimates the arrival direction of the sound suppressed by the matrix element of the row from the difference in declination of the matrix element of each row;
When there is a row of the separation matrix in which the direction of arrival of the sound estimated by the direction estimation unit is not greatly deviated from the normal direction of the microphone array, depending on the ratio of absolute values of the matrix elements of the row A program that functions as a sensitivity correction unit that corrects variations in the signal level of the output signal of each microphone.

Images