CN106448684A - Deep-belief-network-characteristic-vector-based channel-robust voiceprint recognition system - Google Patents

Abstract

The invention belongs to the field of speech signal processing and machine learning, and relates to a channel robust voiceprint recognition system based on a deep belief network feature vector, which consists of a speech collection and preprocessing module, an original spectrum feature extraction module, a deep belief network training module, and a speaker. It consists of a voiceprint feature vector extraction module, a speaker acoustic model generation module and a speaker identification module. Supervisedly train a deep belief network with speech data from different channels and the corresponding speaker ID numbers, and propose a discriminant ratio to select the hidden layer output of the deep belief network with the best class discrimination to construct the speech Voiceprint feature vector, which is channel robust. Compared with the traditional speaker confirmation system based on i‑vector, this system has a higher accuracy of voiceprint recognition in the case of channel mismatch.

Description

Channel Robust Voiceprint Recognition System Based on Deep Belief Network Feature Vector

technical field

The invention relates to a channel robust voiceprint recognition system based on deep belief network feature vectors, and belongs to the technical field of human-computer voice interaction.

Background technique

Voiceprint recognition technology is a kind of biological verification technology, which uses voice to verify the identity of the speaker, that is, to confirm whether a certain voice is spoken by a designated person. This technology has good convenience and security, and has great application prospects in banking, social security, public security, smart home, mobile payment and other fields. However, in practical applications, the traditional voiceprint recognition system faces the problem of channel mismatch, that is, different mobile devices are used for speaker registration and testing, resulting in a decline in the performance of the voiceprint recognition system and a decline in recognition accuracy. Therefore, in order to solve the problem of channel mismatch in the mobile device environment, the present invention proposes a channel robust voiceprint recognition system based on deep belief network feature vectors.

The present invention uses a deep belief network (DBN) to extract speaker features. Many existing voiceprint recognition systems still use features in speech recognition such as MFCC features, PLP features, etc. The main information in these underlying acoustic features is pronunciation text features, and speaker information is easily affected by text information, channel and noise information. Interference, these features cannot reflect the characteristics of the speaker well, and at the same time, under the condition of channel mismatch, the recognition performance of the system decreases, which restricts the application of voiceprint recognition technology. Channel mismatch refers to the fact that the channels used to collect speech during training and testing are different. Around this problem, Kenny proposed the Joint Factor Analysis (JFA) technology, which opened up new ideas for the research of voiceprint recognition under the environment of channel mismatch. , the main idea is to divide the space where the Gaussian mean supervector of the speaker is located into three components: Eigenchannel (Eigenchannel) space, Eigenvoice (Eigenvoice) space and Diagonal Residual (Diagonal Residual) space, by removing the speaker The influence of the per capita supervector in the eigenchannel space is used to achieve the purpose of resisting channel mismatch. However, when the training data is unbalanced under various channels, the JFA technique has obvious deficiencies. Later, Dehak proposed based on i-vector technology. The motivation of this modeling method comes from the channel factor after JFA modeling not only includes channel effects but also speaker information. The I-vector method uses a global difference space (Total Variability Space) to replace these two spaces, which includes both the difference between speakers and the difference between channels.

The voiceprint recognition system based on i-vector technology can better reflect the characteristics of the speaker and is one of the mainstream voiceprint recognition technologies, but its performance is average under the condition of channel mismatch. As an emerging machine learning technology in recent years, deep learning has achieved remarkable results in a variety of specific pattern recognition tasks. A common application of deep neural networks is feature extraction. Compared with traditional hand-extracted features, the features extracted by deep neural networks can better represent high-level abstract information. Deep Belief Network (DBN) was proposed by Geoffrey Hinton in 2006. It is a generative model that allows the entire neural network to generate training data with maximum probability by training the weights between neurons. Deep Belief Networks (DBNs) are stacked by multiple layers of Restricted Boltzmann Machines (RBMs). Usually, Deep Belief Network (DBN) is mainly used to model one-dimensional data more effectively, such as speech.

Inspired by the successful application of deep belief network in speech recognition in deep learning, the present invention carries out supervised training to deep belief network (DBN) by using a large number of speech data of different channels and corresponding speaker ID numbers, and through the trained Deep Belief Network (DBN) extracts the speaker's speech features. In order to measure the discriminative degree of output of different hidden layers of neural network, a discriminant ratio is proposed to select the output with the best discriminative degree to form a channel-robust speaker feature vector. At the same time, three Chinese speech databases are used to verify that the channel robust voiceprint recognition system based on the deep belief network feature vector has stronger channel robustness than the traditional i-vector system.

Contents of the invention

Based on the analysis of the above-mentioned prior art, the purpose of the present invention is based on voiceprint recognition for mobile devices based on Chinese, and to construct a voiceprint recognition system for mobile devices that is based on deep learning and is robust to channels in practical applications. The system uses the Deep Belief Network (DBN) to extract the speaker's speech features, and proposes a discriminant ratio R _p to measure the discrimination of the output of different hidden layers of the neural network and select the feature with the best discrimination, thereby improving the voiceprint recognition. The channel robustness of the system. The system includes the following modules:

Voice collection and preprocessing module, for collecting the voice signal of the speaker, and preprocessing the voice signal;

The original spectral feature extraction module is used to extract the original spectral feature MFCC to the preprocessed speech;

Deep belief network training module for supervised training of a channel-robust feature vector extractor;

The speaker's voiceprint feature vector extraction module uses the trained deep belief network to extract the channel robust speaker's voiceprint feature vector;

The speaker acoustic model generating module performs acoustic modeling on the speaker according to the extracted speaker voiceprint feature vector;

The speaker identity identification module compares and scores the acoustic model of the speaker to be tested with the acoustic model of the registered speaker to determine the identity of the speaker to be tested.

Further, the voice collection and preprocessing module is used to perform preprocessing such as amplification, gain control, filtering and sampling on the collected voice signal.

Further, the original spectral feature extraction module includes: performing framing, pre-emphasis, windowing, and fast Fourier transform on the preprocessed speech, and finally extracting Mel cepstral coefficients MFCC.

Further, the deep belief network training module takes the MFCC features extracted from the corpus under a large number of different channels as input, and uses the corresponding speaker ID number (ID) as an output to carry out supervised training to the deep belief network, And save the parameters of each layer of the trained deep belief network.

Further, the speaker voiceprint feature vector extraction module regards the depth belief network as a feature vector extractor, uses MFCC as the input of the depth belief network, and the hidden layer output of the depth belief network can be regarded as the original MFCC High-level representations of features (deep features), these feature vectors are channel-robust.

Furthermore, a method for measuring the discrimination of deep features extracted by different hidden layers of the neural network is proposed, and the discriminant ratio R _p =det(S _bp )/det(S _wp ) is defined as the measure of the discrimination of deep features, where S _bp is the inter-class scatter matrix of training data, and S _wp is the intra-class scatter matrix of training data, which is defined as follows:

where s _mj is the MFCC feature, f _p ( ) is the mapping of the deep belief network from the input of the MFCC to the output of the p-th hidden layer, G _pm is the mean vector of the training data class, G _p is the mean vector of all training data, and the mathematical Expressed as follows:

The large inter-class distance and the smallest intra-class distance are beneficial to the distinguishability of the extracted feature vectors. Therefore, the hidden layer feature vector with the largest discrimination ratio R _p is the most discriminative, that is, the output of the hidden layer satisfying k=argmax _p R _p is determined as the best deep feature. Using the k-th layer depth feature f _k (s _mj ) of the speaker's deep belief network, the feature vector k ^th -DBN-vector can be obtained, which is defined as

Among them, m is the speaker's ID number, c _m is the frame length of MFCC extracted from each sentence, and N _p is the dimension of the pth hidden layer of the deep belief network.

Further, the speaker acoustic modeling module uses the speaker's feature vector k ^th -DBN-vector to perform probabilistic linear discriminant analysis (PLDA) modeling, and saves PLDA model parameters.

Further, the speaker identification module can first extract the k ^th -DBN-vector of the registrant and the tester according to the trained deep belief network. Then based on the trained PLDA model, the log likelihood ratio score s is obtained, and finally the score is compared with the given threshold s ₀ , if s≥s ₀ , the tester is considered to be the registrant, otherwise not.

The beneficial effect of the present invention is that: with the popularization of mobile devices, users will use voiceprint recognition for identity verification between different mobile devices, which brings the problem of channel mismatch between registration voice and test voice, while the traditional The voiceprint recognition system of i-vector technology has average system performance in the case of channel mismatch. As a deep network, deep belief network has strong learning ability and has a wide range of applications in fields such as speech recognition. By using a large number of speech data of different channels to train the deep belief network, the trained deep belief network can extract speaker features that are robust to the channel, thereby reducing the impact of channel mismatch. Therefore, the voiceprint recognition system based on Deep Belief Network (DBN) can have better robustness to channel mismatch, and can be deployed across devices and platforms. It is convenient to use the voiceprint recognition service.

Description of drawings

Fig. 1 is a schematic structural diagram of a channel robust voiceprint recognition system based on deep belief network feature vectors according to an embodiment of the present invention;

Fig. 2 is a schematic diagram of the structure of the deep belief network (DBN) described in the embodiment of the present invention.

detailed description

The preferred embodiments of the present invention will be described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described here are only used to illustrate and explain the present invention, and are not intended to limit the present invention.

The channel robust voiceprint recognition system based on the deep belief network feature vector of the present invention considers the use of a large number of corpus of different channels and the corresponding speaker ID number (ID) to carry out supervised training on the deep belief network, so using The feature vector extracted by the trained deep belief network is robust to the channel, thus improving the accuracy of the voiceprint recognition system in the case of channel mismatch. Concrete steps are as follows, and in conjunction with the structural representation of the system of the present invention of accompanying drawing 1:

S01: Speech collection and preprocessing module;

Firstly, the voice data is obtained, and the voice signal is preprocessed such as amplification, gain control, filtering and sampling.

S02: Original spectrum feature extraction module;

It includes framing, pre-emphasis, windowing, fast Fourier transform of the preprocessed speech, and finally the extraction of Mel cepstral coefficient MFCC.

S03: Deep belief network training module;

Assume that the MFCC feature extracted from each sentence of the speaker is expressed as m (1≤m≤M) is the ID number of the speaker, L is the length of each frame of MFCC, and c _m is the number of frames. With MFCC as input and the corresponding speaker ID as output, the training data {s _mj ,m,j=1,2,…,c _m ,m=1,2,…,M} can be used to conduct deep belief network Supervised training, and save the parameters of each layer of the trained deep belief network. The structural diagram of the deep belief network is shown in Figure 2.

S04: Speaker voiceprint feature vector extraction module;

The deep belief network is regarded as a feature vector extractor, and MFCC is used as the input of the deep belief network. The hidden layer output of the deep belief network can be regarded as a high-level representation (feature vector) of the original MFCC features. define function is the mapping from the input of the deep belief network to the output of the p-th hidden layer, then the extra-depth features can be obtained: {f _p (s _mj ),p=1,2,…,P}. In order to measure the discrimination of the feature vectors extracted by different hidden layers of the neural network, the discriminant ratio R _p =det(S _bp )/det(S _wp ) is defined as the measure of the depth feature discrimination, where S _bp is the training data between classes The scatter matrix, S _wp is the scatter matrix within the training data class, which is defined as follows:

where s _mj is the MFCC feature, f _p ( ) is the mapping of the deep belief network from the input of the MFCC to the output of the p-th hidden layer, G _pm is the mean vector of the training data class, G _p is the mean vector of all training data, and the mathematical Expressed as follows:

The large inter-class distance and the smallest intra-class distance are beneficial to the distinguishability of the extracted feature vectors. Therefore, the hidden layer feature vector with the largest discrimination ratio R _p is the most discriminative, that is, the output of the hidden layer satisfying k=argmax _p R _p is determined as the best deep feature.

S05: speaker acoustic model generation module;

Using the k-th layer depth feature f _k (s _mj ) of the speaker's deep belief network, the feature vector k ^th -DBN-vector can be obtained, which is defined as

Among them, m is the speaker's ID number, c _m is the frame length of MFCC extracted from each sentence, and N _p is the dimension of the pth hidden layer of the deep belief network. Finally, the feature vector k ^th -DBN-vector is used for probabilistic linear discriminant analysis (PLDA) modeling, and the PLDA model parameters are saved.

S06: speaker identification module;

The specific steps are: (1) first collect and preprocess the voice of the registrant, and extract the original spectrum MFCC features, and then use the trained deep belief network to extract the registrant’s feature vector k ^th -DBN-vector; (2 ) Collect and preprocess the voice of the tester, and extract the original spectrum MFCC features, and then use the trained deep belief network to extract the feature vector k ^th -DBN-vector of the registrant; (3) use the registrant and the speaker k ^th -DBN-vector, based on the trained PLDA model, the log likelihood ratio score s can be obtained, and finally the score is compared with the given threshold s ₀ , if s≥s ₀ , the tester is considered to be registered people, otherwise not.

Table 1 Selected database details

Table 2 Database allocation

Table 3 Experimental parameter settings

In the actual experiment process, the experimental database is selected first, and the database is all Chinese corpus, and its detailed information is shown in Table 1. Among them, the MTDSR2015 database was recorded by the Modern Signal and Data Processing Laboratory of Peking University, the THCHS-30 database was recorded by Tsinghua University, and the King-ASR-L-018 database was released by Haitian AAC.

The distribution of the above database in the experiment is shown in Table 2, where the bkg data is used for the global background model (UBM), the global difference matrix T, and the training of the PLDA model, and the bkg data and the Part I data in dev are used for the deep belief network. For training, Part II data in dev is used for registration, and eva data is used for testing.

Then set the experimental parameters, as shown in Table 3, the channel robust voiceprint recognition system based on the deep belief network feature vector of the present invention is DBN-vector, the baseline algorithm is i-vector, and the algorithm performance evaluation index is equal error rate (EER) and the minimum detection cost function (minDCF).

Final experimental results and analysis:

Using bkg data and Part I data in dev, a deep belief network can be trained. By analyzing the output of each hidden layer of the deep belief network, the discriminant ratio of each hidden layer can be obtained, as shown in Table 4. From Table 4, it can be found that the discriminative ratio of the fourth hidden layer of the deep belief network is the largest, indicating that the deep feature f ₄ (s _mj ) of the fourth hidden layer has the best discrimination, so f ₄ (s mj ) is selected _mj ) is the best deep feature.

hidden layer index p=1 p=2 p=3 p=4 Discriminant ratio R _p 0.47 0.72 1.03 1.34

Table 4 Discriminant ratios of different hidden layers of deep belief network

Table 5 Performance comparison between i-vector system and 4 ^th -DBN-vector system under different channel mismatch conditions

Considering the system performance under the condition of channel mismatch, Table 5 shows the performance of the system we invented and the i-vector system under different channel mismatch conditions, where a represents HUAWEI mate7, b represents XM4, c represents Samsung Note3, and d represents iPhone 5C. Take ab as an example, ab indicates that the HUAWEI mate7 channel is used for voice signal collection during the registration phase, and the XM4 signal is used for voice signal collection during the test phase. According to Table IV, we choose the feature vector 4th- ^DBN -vector of the fourth hidden layer of the deep belief network. It can be seen from Table 5 that in each case of channel mismatch, the channel robust voiceprint recognition system based on the deep belief network feature vector (4 ^th -DBN-vector) does not matter from the equal error rate EER or the minimum detection cost function The minDCF is much smaller than the traditional i-vector system, and the equal error rate EER of the 4 ^th -DBN-vector system is less than 0.9%, and the minimum detection cost function is less than 0.8, which shows that the channel robustness based on the feature vector of the deep belief network In the case of channel mismatch, the voiceprint recognition system is more accurate than the i-vector system in identifying the speaker's identity.

The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention in any form. Therefore, any modification, Equivalent changes and modifications all still belong to the scope of the technical solutions of the present invention.

Claims (8)

1. A channel robust voiceprint recognition system based on deep belief network feature vectors to identify the identity of the speaker, characterized in that the system includes: Voice collection and preprocessing module, for collecting the voice signal of the speaker, and preprocessing the voice signal; The original spectral feature extraction module is used to extract the original spectral feature MFCC to the preprocessed speech; Deep belief network training module for supervised training of a channel-robust feature vector extractor; The speaker's voiceprint feature vector extraction module uses the trained deep belief network to extract the channel robust speaker's voiceprint feature vector; The speaker acoustic model generating module performs acoustic modeling on the speaker according to the extracted speaker voiceprint feature vector; The speaker identity identification module compares and scores the acoustic model of the speaker to be tested with the acoustic model of the registered speaker to determine the identity of the speaker to be tested. 2. The channel robust voiceprint recognition system based on the deep belief network feature vector according to claim 1, wherein the voice collection and preprocessing module is used to amplify, gain control, and filter the voice signal collected and sampling preprocessing. 3. The channel robust voiceprint recognition system based on deep belief network feature vectors according to claim 1, wherein the original spectrum feature extraction module comprises: framing, pre-emphasizing, Windowing, fast Fourier transform, and finally the extraction of Mel cepstral coefficient MFCC. 4. the channel robust voiceprint recognition system based on depth belief network feature vector according to claim 1, is characterized in that, described depth belief network training module, with the MFCC feature extracted by the corpus under a large amount of different channels as Input, with the corresponding speaker ID number (ID) as the output, conduct supervised training on the deep belief network, and save the parameters of each layer of the trained deep belief network. 5. The channel robust voiceprint recognition system based on the deep belief network feature vector according to claim 1, wherein the speaker voiceprint feature vector extraction module regards the deep belief network as a feature vector extractor , taking MFCC as the input of the deep belief network, the hidden layer output of the deep belief network can be regarded as a high-level representation of the original MFCC features (deep features), and these deep features have the characteristics of channel robustness. 6. The channel robust voiceprint recognition system based on deep belief network feature vectors according to claim 5, characterized in that, a method for measuring the degree of discrimination of the extracted depth features of different hidden layers of neural networks is proposed, and the definition of discrimination Ratio R _p =det(S _bp )/det(S _wp ), as a measure of depth feature discrimination, where S _bp is the inter-class scatter matrix of training data, and S _wp is the intra-class scatter matrix of training data, which is defined as follows : where s _mj is the MFCC feature, f _p ( ) is the mapping of the deep belief network from the input of the MFCC to the output of the p-th hidden layer, G _pm is the mean vector of the training data class, G _p is the mean vector of all training data, and the mathematical Expressed as follows: The large inter-class distance and the smallest intra-class distance are conducive to the distinguishability of the extracted feature vectors. Therefore, the hidden layer feature vector with the largest discriminant ratio R _p is the most discriminative, that is, the hidden layer feature vector that satisfies k=argmax p R p is determined to satisfy k=argmax _p R _p The output of the layer is used as the best depth feature, and using the depth feature f _k (s _mj ) of the kth layer of the speaker's deep belief network, the feature vector k ^th -DBN-vector can be obtained, which is defined as Among them, m is the speaker's ID number, c _m is the frame length of MFCC extracted from each sentence, and N _p is the dimension of the pth hidden layer of the deep belief network. 7. The channel robust voiceprint recognition system based on deep belief network feature vectors according to claim 1, wherein the speaker acoustic model generation module utilizes the speaker's feature vector k ^th -DBN- vector performs probabilistic linear discriminant analysis (PLDA) modeling and saves the PLDA model parameters. 8. The channel robust voiceprint recognition system based on deep belief network feature vectors according to claim 1, wherein the speaker identification module can first extract the registrant according to the trained deep belief network and tester's k ^th -DBN-vector. Then based on the trained PLDA model, the log likelihood ratio score s is obtained, and finally the score is compared with the given threshold s ₀ , if s≥s ₀ , the tester is considered to be the registrant, otherwise not.