KR101023211B1 - Microphone array based speech recognition system and target speech extraction method in the system - Google Patents

Abstract

The present invention relates to a microphone array based speech recognition system using blind signal separation and a target speech extraction method in the system, wherein the speech recognition system separates mixed signals inputted through a plurality of microphones through independent element analysis, Among the separated sound source signals, one target voice spoken for speech recognition is extracted using a Gaussian mixture density model or a hidden Markov model, and the desired voice is automatically recognized through the extracted target voice, thereby reducing noise. Higher recognition rates can be attained even when they exist.

Microphone Array, Speech Recognition, Blind Signal Separation, Independent Element Analysis (ICA), Gaussian Mixed Density Model (GMM), Hidden Markov Model (HMM), Target Speech, Feature Vector, Algebraic Likelihood Ratio (LLR).

Description

MICROPHONE ARRAY BASED SPEECH RECOGNITION SYSTEM AND TARGET SPEECH EXTRACTION METHOD OF THE SYSTEM}

The present invention relates to a speech recognition system and a method for extracting a target voice in the system, and more particularly, to a microphone array based speech recognition system using blind signal separation and a target speech extraction method in the system.

The present invention is derived from a study conducted as part of the IT growth engine technology development project of the Ministry of Knowledge Economy and the Ministry of Information and Telecommunication Research and Development. [Task management number: 2006-S-036-02, Task name: Large-capacity interactive dispersion for new growth engine industries Development of processing voice interface technology].

Conventional speech recognition technology has more than 95% word recognition rate when speech is recognized in a relatively quiet environment. However, in a real application environment where various noises exist, the recognition rate drops rapidly. Therefore, it is essential to secure a high recognition rate for the commercialization of the voice recognition technology.

In recent decades, various noise processing methods have been studied in the pre-processing, recognition and post-processing stages of speech recognition, but there is still no known method for handling all kinds of noises well.

Recently, a microphone array based blind signal separation (BSS) technique for separating a desired voice signal using two or more microphones has been actively studied. Independent Component Analysis (hereinafter referred to as ICA) technology, one of the main methodologies of the BSS, allows interference signals such as surrounding speakers, TV, or radio noise to be received from devices that receive voice, such as voice recognizers and wired / wireless mobile phones. Can be effectively removed or attenuated. That is, when there are N sound sources including the input voice and M microphones, the original N sound source signals can be restored from the M microphone input signals when the number of M and N is similar.

However, the N sound source signals separated by ICA have a problem that their order is reversed arbitrarily.

In addition, the conventional ICA technology multiplies each sound source signal with an arbitrary weight in the time domain and adds them to make mixed signals, which are then separated using the ICA algorithm. Recently, however, due to the development of technology, even in the presence of actual room reverberation (room reverberation) has been developed to the level that can separate the original sound sources. However, even in this advanced ICA technology, there is no known method to automatically find out what the separated sound signals correspond to, and it is necessary to automatically find a target voice to be input to the recognition system for speech recognition. There is this.

In order to solve the above problems, the present invention provides a microphone array based speech recognition system for automatically searching for a desired target speech using a microphone array-based blind signal separation technology for speech recognition, and a method of extracting a target voice from the system. In providing.

The microphone array-based speech recognition system for achieving the objects of the present invention, the signal separator for separating each of the mixed signals input through a plurality of microphones into the sound source signals through independent element analysis; A target voice extractor for extracting one target voice spoken for voice recognition from among the sound source signals separated by the signal separator; And a voice recognizer for recognizing a desired voice through the extracted target voice, and further comprising an additional information transmitter for transmitting additional information used for the target voice extraction to the target voice extractor.

And a target voice extraction method in a microphone array based speech recognition system for achieving the objects of the present invention, comprising: separating the mixed signals respectively input through a plurality of microphones into sound source signals through independent element analysis; Extracting one target voice spoken for voice recognition from the separated sound source signals; Recognizing a desired voice through the extracted target voice, characterized in that it comprises.

According to the present invention, a single target voice, which is aimed for speech recognition, is automatically detected using a Hidden Markov Model (HMM) and a Gaussian Mixed Density Model (GMM). Since it is possible to find out what the corresponding sound source signals correspond to, it is effective to secure a higher recognition rate even in the presence of noise in speech recognition.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In describing the present invention, if it is determined that detailed descriptions of related known functions or configurations may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

Speech recognition technology for target speech extraction according to an embodiment of the present invention belongs to the blind source separation (BSS) technology and the pre-processing technology for speech recognition in a noise environment based on the microphone array-based signal processing . Here, the independent element analysis technology has been recently developed in relation to the blind signal separation technology, which can successfully separate sound sources using two or more microphones, and is capable of acoustic signal separation, multi-channel EEG signal separation, image pattern analysis, and the like. Applicable to various fields.

Next, a speech recognition system for microphone array based target speech extraction using the blind signal separation will be described in detail with reference to the accompanying drawings.

2 is a block diagram illustrating a structure of a microphone array based speech recognition system for extracting target speech according to an embodiment of the present invention.

Referring to FIG. 2, the voice recognition system may include a plurality of microphones 101, a signal separator 110, a target voice extractor 120, a voice recognizer 130, and an additional information device 140.

Each of the plurality of microphones 101 receives a plurality of sound source signals S ₁ (t), S ₂ (t), ..., S _N (t). Receive by mixing

The signal separator 110 receives a mixed signal (X ₁ (t), X ₂ (t), ..., X _N (t) output from each of the microphones 101, and performs independent element analysis (ICA). Each of the mixed signals input through the signal is separated and outputs the separated sound source signals r ₁ (t), r ₂ (t), ..., r _N (t).

The target voice extractor 120 receives each of the separated sound source signals and extracts one target voice from the sound source signals. At this time, the target speech extraction includes feature extraction for N separated sound source signals, Log Likelihood Ratio (LLR) reliability calculation using a Gaussian mixture density model (hereinafter referred to as GMM), reliability comparison and target. Perform voice decisions and more. The target voice extractor 120 may extract the target voice by dividing it into a case of using the additional information received from the additional information 140 and a case in which no additional information is given. In addition, the target voice extractor 120 is the target voice male or female and other The target voice is extracted by considering the sound sources as the target voice and all other signals or other signals, the information other than the target voice as the noise signal, and the speaker personal information that the target voice is the voice of the specific speaker.

When the additional information is given, the target voice extractor 120 applies Gaussian mixture density model to each sound source signal. Through the hypothesis test, the most reliable sound source is determined as the target voice. If the additional information is not provided, the target voice extractor 120 calculates the reliability of each sound source using a hidden Markov model (hereinafter, referred to as HMM) built in a voice recognition system.

The voice recognizer 130 receives the extracted target voice signal y (t) to recognize a desired voice.

A method for extracting a target voice in a speech recognition system having such a structure will be described in detail with reference to the accompanying drawings.

3 is a flowchart illustrating a method for extracting a target voice from a target voice extractor of a microphone array based voice recognition system according to an embodiment of the present invention.

Referring to FIG. 3, in operation 210, the target voice extractor 120 of the speech recognition system receives sound source signals separated from the signal separator 110.

Then, in step 220, when the N separate 8 kHz or 16 kHz sound signals are given, the target speech extractor 120 calculates the feature by calculating the Nth feature vector x _t in a frame of 20 ms intervals every 10 ms of the audio signal. Extract. The feature vector may be calculated using a number of methods such as linear prediction cepstral coefficient (LPCC), perceptual linear prediction (LPP), and mel-frequency cepstral coefficient (MFCC).

In operation 230, the target voice extractor 120 calculates logarithmic likelihood ratio (LLR) reliability using a Gaussian mixture density model (GMM) in each extracted feature. The Gaussian mixture density model (GMM) is a statistical model technique for modeling an N-dimensional vector space in the form of M multivariate Gaussian distributions, and is widely used in speech recognition or speaker recognition. The m-th Gaussian distribution of this arbitrary Gaussian mixture density model (GMM) may be represented by Equation 1 below.

In Equation 1, x, μ _m and ∑ _m mean a feature vector, an m-th Gaussian distribution mean vector, and a covariance matrix, respectively. The GMM output probability is obtained by converting the feature vector sequence in T frames corresponding to the entire audio signal interval X = [x ₁ , x ₂ ,... , x _T ] and when adjacent feature vectors are independent, it may be expressed as Equation 2 below.

은 M개의 가우시안 분포 중 m번째 가우시안 분포의 기여도(혹은 가중치)를 의미하며,

이다.Where μ _m, which are the parameters of the GMM Σ _m values can be obtained from the learning database, by using the well-known maximum likelihood (maximum likelihood) estimation algorithm. Also,

Is the contribution (or weight) of the mth Gaussian distribution among the M Gaussian distributions.

to be.

Subsequently, in step 240, the target voice extractor 120 obtains K (maximum value), and when LLR _K > θ in step 250, the target sound extractor 120 sets the separated sound source signal K as the target voice in step 260. In step 270, the target voice member, that is, the input rejection, is set.

As described above, steps 230 to 270 may be performed using various methods. These methods are target voice extraction methods, such as when using gender information, when using voice-music information, when using voice-noise information, when using personal information of a speaker, and when additional information is not separately provided. Each of these methods will then be described in detail.

First, when a target voice is extracted using gender information, that is, when a sound source corresponding to the target voice is a female and other sound sources are provided with information such as a male speaker's voice or other audio signal, Extract the target voice as follows.

First, the voice database is divided into male and female speakers, and a Gaussian mixed model (GMM) λ _Male and λ _Female are generated for each.

)라고 할 때, λ_Male, λ_Female 모델에 대한 상기 특징벡터열 Xⁱ의 대수우도비(LLR_i)를 하기 <수학식 3>과 같이 계산한다. Then, the feature vector sequence extracted from the N separated sound source signals is X ¹ , X ² ,. , X ^N (

), The algebraic likelihood ratio (LLR _i ) of the feature vector sequence X ⁱ for the λ _Male and λ _Female models is calculated as in Equation 3 below.

In this case, when the i-th sound source X ⁱ is actually a female speaker's voice, LLR _i represents a high value because the molecular term has a high value in Equation 3, otherwise LLR _i represents a relatively low value. Indicates.

The maximum value among all the LLR _i may be calculated as in Equation 4 below.

·

로 계산된다. 여기서

는 프레임의 갯수이고, 표준 임계값

는 각 응용 시스템에 따라 실험적으로 결정된다.When the maximum algebraic likelihood ratio LLR _k is greater than a threshold value as compared with a predetermined threshold θ as in step 250, the target voice extractor 120 determines X ^k as the target voice. This is output to the speech recognizer 130. If the LLR _k is smaller than the threshold value, the target voice extractor 120 determines that the target voice does not exist among the separated sound source signals. The threshold value θ is

·

. here

Is the number of frames and the standard threshold

Is determined experimentally for each application system.

On the other hand, when the sound source corresponding to the target voice is male, and other sound sources are provided with the information of the female speaker's voice or other audio signal, LLRi is represented by Equation 5 below, otherwise the target voice is The same as described above in the case where the sound source corresponding to the present invention is a female.

Secondly, when using voice-music information, that is, a sound source corresponding to the target voice is speech, and other sound sources are provided with the information of the music signal, the target voice is extracted as follows.

First, the database is divided into speech and music data, and then Gaussian mixed model (GMM) λ _Speech and λ _Music are generated for each.

Then, the algebraic likelihood ratio LLR _i of the feature vector sequence X ⁱ is calculated as in Equation 6 below.

Since the target voice extraction method is the same as the first method, a description thereof will be omitted.

Third, when the voice-noise information is used, that is, the sound source corresponding to the target voice is speech, and other sound sources are provided with the information of the noise signal, the target voice is extracted as follows.

First, the database is divided into speech and noise data, and then Gaussian mixture model (GMM) λ _Speech and λ _Noise are generated for each.

Then, the feature vector sequence extracted from the N separated sound source signals is X ¹ , X ² ,. , X ^N , the algebraic likelihood ratio LLRi of the feature vector sequence X ⁱ for the λ _Speech and λ _Noise models is calculated as shown in Equation 7 below.

Since the target voice extraction method is the same as the first method, a description thereof will be omitted.

Fourthly, when the speaker's personal information is used, that is, when the sound source corresponding to the target voice is a predetermined speaker and other sound sources are provided with the information of another speaker's voice or other audio signal, as follows. Extract the target voice.

First, the database is divided into a specific speaker and other audio signals, and then a Gaussian mixed model (GMM) λ _Individual and λ _Others are generated for each.

Then, the feature vector sequence extracted from the N separated sound source signals is X ¹ , X ² ,. , X ^N , the algebraic likelihood ratio LLR _i of the feature vector sequence X ⁱ for the λ _Individual and λ _Others models is calculated as shown in Equation 8 below.

Since the target voice extraction method is the same as the first method, a description thereof will be omitted.

Fifthly, when additional information is not given separately, that is, when no additional information is provided for the target voice, it is assumed that there is a signal uttered for the use of the recognizer among the N separate sound source signals. Reliability based on LLR is calculated by using HMM (Hidden Markov Model). In this case, the reliability calculation method is as follows.

First, the HMM of the speech recognizer 130 After passing through the speech recognizer 130 for each of the separated sound source signals, the HMM acoustic model is aligned with respect to the word sequence that is given as a result of the recognition.

라 하고,

에 해당하는 HMM 상태(state)는

로서, m과 j는 m번째 HMM 부단어 모델(subword model)의 j번째 상태를 뜻한다. 이때, LLR_i는 하기 <수학식 9>와 같이 계산한다. Next, the feature vector string of the i-th sound source

,

The HMM state corresponding to

Where m and j represent the j th state of the m th HMM subword model. In this case, LLR _i is calculated as in Equation 9 below.

는 음성 인식기(130)를 통해 얻은 단어열을 뜻하며,

는

의 컴플리먼트(complement),

는

의 컴플리먼트(complement)를 의미한다. 여기서 상기

는 실질적으로 직접 구하기가 어려우므로 상기 <수학식 9>의 맨 마지막 줄과 같이 근사하여 추정할 수 있다. K는

에 상응하는 모든 HMM들에 포함된 각각의 상태의 번호(number)이다.

는 실험적으로 결정될 수 있는 상수항이다. 만약, 이 값이 매우 크게 설정되면, 상기 <수학식 9>의 마지막 줄 내의 두번째 합산 항(summation term)은 가장 큰 우도 값(likelihood value)에 의해 좌우된다. 반면에,

가 더 작은 값을 갖게 될수록, 다른 우도 값들의 기여는 더 현저해진다. In Equation 9 above

Is a word string obtained through the speech recognizer 130,

Is

Complement of,

Is

It means the complement of. Where above

Since it is difficult to find the direct value, it can be estimated by approximating the last line of Equation (9). K is

Is the number of each state contained in all HMMs corresponding to

Is a constant term that can be determined experimentally. If this value is set very large, the second summation term in the last line of Equation 9 depends on the largest likelihood value. On the other hand,

The smaller the is, the more significant the contribution of the other likelihood values.

Since the target voice extraction method is the same as the first method, a detailed description thereof will be omitted.

Lastly, when additional information is provided that the sound source of the target voice is a sound having a specific property A, and the other sound sources are audio signals having a specific property B, the target voice extraction method may be performed as described below. Is performed.

First, the database is divided into speech data having a specific characteristic A and other audio signal data according to the specific characteristic B, and then Gaussian mixed model (GMM) lambda _{properties _A} and lambda _{property_B} are generated.

Next, the feature vector sequence extracted from the N separated sound source signals is X ¹ , X ² ,... , X ^N , for the feature vector column X ⁱ for λ _{property _A} and λ _{property _B} The LLR _i is calculated as shown in Equation 10 below.

Meanwhile, in the detailed description of the present invention, specific embodiments have been described, but various modifications are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the scope of the following claims, but also by the equivalents of the claims.

1 is a block diagram showing a signal separation system using a general independent element analysis technique,

2 is a block diagram illustrating a structure of a microphone array based speech recognition system for extracting target speech according to an embodiment of the present invention;

3 is a flow chart illustrating a method for extracting a target voice in a target voice extractor of a microphone array based speech recognition system in accordance with an embodiment of the present invention.

Claims (18)

A signal separator for separating the mixed signals input through the plurality of microphones into sound source signals through independent element analysis; A target voice extractor for extracting one target voice spoken for voice recognition from among the sound source signals separated by the signal separator; And Including a voice recognizer for recognizing a desired voice through the extracted target voice, The target voice extractor Extracting a feature vector sequence from the separated sound source signals, calculating a log likelihood ratio for the extracted feature vector sequence, calculating a maximum value using the calculated likelihood ratio, and calculating the maximum value And a predetermined threshold value, and when the maximum value is greater than the threshold value, determining the maximum value as the target voice. The method of claim 1, And an additional information unit for transmitting the additional information used to extract the target voice to the target voice extractor. delete The method of claim 1, And the target voice extractor determines that the target voice does not exist among the separated sound source signals when the maximum value is smaller than the threshold value. The method of claim 2, The target voice extractor determines that the target voice is the most reliable sound source signal by performing a hypothesis test on the separated sound source signals using a Gaussian mixture density model when the additional information is given. Array-based Speech Recognition System. The method of claim 5, The additional information is a microphone array based speech recognition system, characterized in that gender information, voice-music information, voice-noise information, personal information of the speaker. The method of claim 1, The target speech extractor calculates a logarithmic likelihood ratio (LLR) based reliability using an acoustic model hidden Markov model (HMM) of speech recognition when no additional information about the target speech is given. Speech recognition system. Separating the mixed signals input through the plurality of microphones into sound source signals through independent element analysis; Extracting one target voice spoken for voice recognition from the separated sound source signals; And Recognizing a desired voice through the extracted target voice, Extracting the target voice Extracting a feature vector sequence (X ⁱ ) from the separated sound source signals; Calculating an algebraic likelihood ratio (LLR _i ) for the extracted feature vector sequence; Calculating a maximum value k using the calculated log likelihood ratio; Comparing the maximum value with a preset threshold value; And And determining the maximum value as the target voice when the maximum value is greater than the threshold value. delete The method of claim 8,  And determining that the target voice does not exist among the separated sound source signals when the maximum value is smaller than the threshold value. The method of claim 8, The logarithmic likelihood ratio (LLR _i ) indicates that the target voice is female. If additional information is given, passing the extracted feature vector sequence (X ⁱ ) to a pre-generated male and female Gaussian mixture model (λ _Male , λ _Female ), and calculating as shown in Equation 3 below. A target speech extraction method in a microphone array based speech recognition system. <Equation 3>

The method of claim 8, The logarithmic likelihood ratio (LLR _i ) indicates that the target voice is male. If additional information is given, passing the extracted feature vector sequence (X ⁱ ) to a pre-generated male and female Gaussian mixture model (λ _Male , λ _Female ) and calculating the following equation (5): A target speech extraction method in a microphone array based speech recognition system. <Equation 5>

The method of claim 8, The algebraic likelihood ratio LLR _i is a feature vector sequence extracted to each Gaussian mixture model (λ _Speech , λ _Music ) previously generated according to the speech-music information when the additional information of the speech-music information is given. X ⁱ ) passing through and calculating as in Equation 6 below. <Equation 6>

The method of claim 8, The logarithmic likelihood ratio LLR _i is a feature vector string extracted to each Gaussian mixture model (λ _Speech , λ _Noise ) previously generated according to the speech-noise information when the additional information of the speech-noise information is given. X ⁱ ) passing through and calculating as in Equation (7). <Equation 7>

The method of claim 8, The logarithmic likelihood ratio LLR _i passes the extracted feature vector sequence X ⁱ to each previously generated Gaussian mixture model λ _Individual , λ _Others when the target voice is given additional information that the target voice is a specific _speaker . And calculating as shown in Equation (8). <Equation 8>

The method of claim 8, The logarithmic likelihood ratio LLR _i may include the extracted feature vector sequence in each generated Gaussian mixture model λ _{Property_A} and λ _{Property_B} when the target voice is given additional information that the target voice is the first specific property Property_A. X ⁱ ) passing through and calculating as in Equation (10). <Equation 10>

The method of claim 8, wherein the extracting of the target voice comprises: Performing additional speech recognition on the separated sound source signals using an acoustic model hidden markov model (HMM) of speech recognition when no additional information on the target speech is given; Calculating a closest hidden Markov model and a corresponding state sequence with respect to the word sequence obtained according to the speech recognition; Calculating an algebraic likelihood ratio LLR _i using the calculated hidden Markov model and the corresponding state string; Calculating a maximum value k using the calculated log likelihood ratio; Comparing the maximum value with a preset threshold value; And determining the maximum value as the target voice when the maximum value is greater than the threshold value. The method of claim 17, The logarithmic likelihood ratio LLR _i is calculated as in Equation 9 below,

Means the word string obtained by performing the voice recognition,

Above

Means complement of,

Is

Means complement, and K is

Means the number of each state included in all hidden Markov models corresponding to

Is a constant term that can be determined experimentally. The method of extracting a target voice in a microphone array-based speech recognition system. &Quot; (9) &quot;

Images