CN111816203A - A synthetic speech detection method based on phoneme-level analysis to suppress the influence of phonemes - Google Patents

Abstract

The invention relates to speech signal processing, in order to study the difference on different phonemes of real speech and fraudulent speech, improve the effect of the systematic fraud attack detection of the automatic speaker, the invention, on the basis of analyzing the synthetic speech detection method that inhibits phoneme influence at the phoneme level, use F-ratio to analyze each frequency band of different phonemes in different real speech and fraudulent speech, find out the frequency range more favorable to distinguishing real speech and fraudulent speech, increase the filter density on the frequency band, obtain the new characteristic, and train the Gaussian mixture model GMM of real speech and fraudulent speech separately with the characteristic, input two models separately after the audio frequency to be recognized extracts the characteristic, score the result of two models with the maximum likelihood ratio finally, receive the final recognition result. The invention is mainly applied to the occasion of voice signal processing.

Description

A synthetic speech detection method based on phoneme-level analysis to suppress the influence of phonemes

technical field

The invention relates to the fields of pattern recognition and speech signal processing, in particular to a method for analyzing phoneme features of real speech and synthesized speech by using F-ratio, which is used for more efficient identification of true and false speech.

Background technique

The use of people's personalized biometrics for personal identification has now been widely used in production and life. Personalized biometrics refer to the physiological characteristics that are persistent and unique within a certain period of time, including fingerprints, iris and voiceprints, and can reflect the differences between individuals. Among them, the Voiceprint Recognition technology is also called the Speaker Recognition technology, which can judge the identity information of the speaker in the audio according to a piece of audio. Compared with fingerprint recognition, face recognition and iris recognition, voiceprint recognition technology has certain advantages. For example, the realization cost is low, the operation is simple, and so on. Voiceprint recognition does not require special equipment like fingerprint recognition, nor does it require specific actions like face recognition. It only needs to say a simple sentence for identity authentication. Therefore, voiceprint recognition technology has a high degree of user acceptance, and the market share has reached 15.8%, and it is constantly showing an upward trend.

But recently, with the maturity of speech synthesis (Speech Synthetic) technology and voice conversion (VoiceConversion) technology, many criminals can use these technologies to easily imitate the acoustic characteristics of the target speaker, and then break the defense of the voiceprint recognition system and steal information and property of others. In order to protect voiceprint recognition systems from synthetic speech or converted speech attacks, the need for Spoofing Attack Detection techniques has become increasingly strong. The research of this technology plays a vital role in the promotion and use of voiceprint recognition systems.

Currently, Interspeech, the top international conference on speech, holds the Automatic Speaker Verification Spoofing and Countermeasures Challenge every two years. Analyzing the strategies of each participating team, it can be found that the research on this topic at home and abroad is mainly divided into two aspects, namely the research based on the front-end speech feature analysis and the research based on the back-end classifier. In terms of features, the commonly used features include Constant Q Cepstral Coefficients (CQCC) obtained by constant Q transform and Linear-Frequency Cepstral Coefficients (Linear-Frequency Cepstral Coefficients) obtained by using linear filters. In terms of classifiers, in addition to Gaussian Mixture Model (GMM), Linear Discriminant Analysis (LDA) and Support Vector Machine (SVM) and other traditional machine learning classics In addition to classifiers, some popular deep neural network models are also used in this task, such as Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN).

In a 2019 study by Gajan Suthokumar et al., it was shown that different phonemes in the process of speech pronunciation have different discrimination abilities when identifying fraudulent attacks, and the discrimination ability of light phonemes is generally higher than that of voiced phonemes.

SUMMARY OF THE INVENTION

In order to overcome the deficiencies of the prior art, the present invention aims to study the difference between real speech and fraudulent speech in different phonemes, so as to improve the effect of fraud attack detection by automatic speaker system. To this end, the technical solution adopted by the present invention is to use F-ratio to analyze each frequency band of different phonemes in different real voices and fraudulent voices, based on a phoneme-level analysis that suppresses the influence of phonemes, and F-ratio is called The variance ratio test is to find the difference distribution in each classification by comparing the variance within and between classes, and find out the frequency range that is more conducive to distinguishing real speech and fraudulent speech through the analysis, and increase the filtering on this frequency band. Obtain new features, and use the features to train the Gaussian mixture model GMM of real speech and fraudulent speech respectively, extract the features of the audio to be recognized and input them into the two models respectively, and finally use the results of the two models with the maximum likelihood ratio. Score to get the final recognition result.

Specific steps are as follows:

Step 1, data preparation:

First, annotate the speech data, that is, obtain each phoneme in the audio and their starting time information, then study each phoneme in the real speech and the fraudulent speech separately, and use a uniform subband filter to process the speech audio In each frame, and then obtain the data of each frequency band on each frame of different phonemes;

Step 2, data analysis:

Use the phoneme-level F-ratio method to analyze the data obtained in the previous step. The F-ratio value on a certain frequency band is used to represent the ability of the frequency band to identify real speech and fraudulent speech. The larger the value of F-ratio, , indicating that this channel carries more information for identification, and the identification ability is stronger. Then, according to the F-ratio value on all channels, the results are normalized, and then the frame number of each phoneme is used as the weight, The normalized data on each frequency band of the phoneme is weighted and averaged, and finally the discrimination ability on each frequency band after the influence of the phoneme is suppressed is obtained. The larger the result, the stronger the discrimination ability;

Step 3, extract features:

According to the experimental results of the second step, in areas with strong discrimination ability, increase the number of filters to increase the density of filters in these areas, and then use these filters to The speech signal after short-time Fourier transform is filtered, and finally a new feature that suppresses the influence of phonemes is obtained through discrete cosine transform (DCT);

Step 4, model training

Using the audio extraction features in the training set as input, train the Gaussian mixture model GMM of real speech and fraudulent speech respectively;

Step 5: Score confirmation

The features extracted from the speech to be tested are input into the models of real speech and fraudulent speech respectively for scoring, and then the maximum likelihood ratio classification method is used to obtain the final result.

Step 2, the specific steps of data analysis are as follows:

Use the phoneme-based F-ratio analysis method PF (Phoneme F-ratio) to analyze each frequency band in different phonemes. The analysis idea of PF is to calculate the variance of a phoneme k on the lth filter between different methods and The ratio of variance within the same method, the higher the value, the greater the difference between different methods in this area. The specific calculation formula of PF is as follows:

表示第t个类型第k个音素第j帧中第l个滤波器上的数据；

表示第t个类型第k个音素的每一帧第l个滤波器上的数据平均值；u_k表示所有类型第k个音素的每一帧第l个滤波器上的数据平均值，

和u_k的公式如下：Among them, T represents the method type, and Ntk represents the frame number of the k-th phoneme in the t-th type;

represents the data on the lth filter in the jth frame of the kth phoneme of the tth type;

represents the average value of data on the lth filter of each frame of the kth phoneme of the tth type; u _k represents the data average value of the lth filter of each frame of the kth phoneme of all types,

The formulas for and u _k are as follows:

After normalizing the obtained PF value, the F-ratio contribution rate PFC (Phoneme F-ratio Contribution) of the phoneme on the lth filter band can be obtained, and its calculation formula is as follows:

Among them, L is the number of uniform subband filters; the calculated PFC can reflect the frequency distribution of the information used to identify fraudulent speech in different phonemes, and then the PFC of each phoneme is weighted and averaged according to its frame number to obtain The overall F-ratio value GF (General F-ratio), its calculation formula is as follows:

Where P is the total number of all phonemes, N is the total number of frames of all phonemes, and the calculation formula of N is as follows:

The calculated GF is then normalized to obtain the PESSDID (Phoneme Effect Suppressed Spoof Detection Information Distribution) that suppresses the influence of the phoneme. The calculation formula is as follows:

Here, the higher the value of PESSDID of filter 1, the more information on the frequency band of the filter that can be used to identify fraudulent attacks.

Step 3, in the feature extraction step:

In addition to the difference in filter distribution, other feature extraction processes include: the steps before using the filter include pre-emphasis, framing and windowing, and then through short-time Fourier transformation to obtain the spectral features of each frame, and then use The filter processes the spectral features, and then undergoes DCT transformation to obtain the final features.

The characteristics and beneficial effects of the present invention are:

The present invention uses the method of F-ratio to analyze the difference in different phonemes between the fraudulent attack voice faced by the voiceprint recognition system and the real voice, and finds the frequency distribution of the information that helps to identify the fraudulent voice. According to the analysis results, a new feature that can suppress the influence of different phonemes in the recognition task is designed by improving the filter. Preliminary experiments were carried out on the test set of ASVspoof2019, and the obtained Equal Error Rate (EER) was 4.16%, which was 48.58% higher than the commonly used LFCC feature (baseline system EER was 8.09%). promote.

Description of drawings:

Figure 1 is a flowchart of feature extraction of fraud attack detection based on F-ratio analysis to suppress the influence of phonemes.

FIG. 2 is a schematic diagram of the distribution of filters for suppressing the influence of phonemes obtained based on F-ratio analysis.

Detailed ways

The object of the present invention is to study the difference between real voice and fraudulent voice on different phonemes, use the method of F-ratio to carry out comparative analysis on different frequency bands of each factor, find out the frequency band that each factor is more conducive to identifying fraudulent voice, and then Increasing the feature extraction is the filter density on these frequency bands, and finally a more robust personalized feature is obtained, which improves the effect of automatic speaker system fraud attack detection.

The technical solution that realizes the purpose of the present invention is:

A synthetic speech detection method with suppressed phoneme influence based on F-ratio analysis. Use F-ratio to analyze each frequency band of different phonemes in different real voices and fraudulent voices, find out the frequency range that is more conducive to distinguishing real voices and fraudulent voices, increase the filter density on this frequency band, and obtain new features, And use this feature to train the Gaussian Mixture Model (GMM) of real speech and fraudulent speech respectively, extract the features of the audio to be recognized and input them into the two models respectively, and finally score the results of the two models with the maximum likelihood ratio to obtain the final recognition. result.

The implementation of the system includes the following steps:

Step 1, data preparation:

First, annotate the speech data, that is, obtain information such as each phoneme in the audio and their starting time. The individual phonemes in real speech and fraudulent speech are then studied separately. Each frame in the speech audio is processed using a uniform subband filter to obtain data for each frequency band on each frame of different phonemes.

Step 2, data analysis:

Use the F-ratio method to analyze the data obtained in the previous step. The F-ratio value on a certain frequency band can be used to characterize the ability of the frequency band to identify real speech and fraudulent speech. Indicates that the more information available for identification carried on this channel, the stronger the identification ability. After that, the results are normalized according to the F-ratio values on all channels, and then the frame number of each phoneme is used as the weight, and the normalized data on each frequency band of the phoneme is weighted and averaged, and finally the suppressed phoneme is obtained. The discriminative ability on each frequency band is affected, and the larger the result, the stronger the discrimination ability.

Step 3, extract features:

According to the experimental results of the second step, in areas with strong discriminating ability, appropriately increase the number of filters to increase the density of filters in these areas. Then use these filters to filter the speech signal after framing, windowing and short-time Fourier transform, and finally obtain new features that suppress the influence of phonemes through discrete cosine transform (DCT).

Step 4, model training

Taking the audio extraction features in the training set as input, GMM models for real speech and fraudulent speech are trained separately.

Step 5: Score confirmation

The features extracted from the speech to be tested are input into the models of real speech and fraudulent speech respectively for scoring, and then the maximum likelihood ratio classification method is used to obtain the final result.

Below in conjunction with accompanying drawing, the synthetic speech detection method that suppresses the influence of phoneme based on F-ratio analysis that the present invention implements is described, mainly comprises the following steps:

Step 1, data preparation:

In order to verify the effect of the present invention, a fraud attack detection experiment is carried out on the database of the ASVSpoof2019 competition. The ASVSpoof2019 database consists of three parts: training set, development set and test set. The training set and development set include 7 kinds of fraud attack algorithms for speech synthesis and speech conversion, while the test set includes 12 different algorithms from the training set and development set. Fraud attack algorithm. All audio in the database is sampled at 16KHz. Since the competition did not provide the text information corresponding to the audio in the database, here we used a speech recognition system to recognize the speech content in the 25,380 audios in the training set. Afterwards, the phoneme information in these audios was extracted through the tool of speech annotation. Each frame in the speech audio is then processed using a uniform subband filter to obtain data for each frequency band on each frame of different phonemes.

Step 2, data analysis:

Here, the method based on the phoneme-level F-ratio (Phoneme F-ratio, PF) is used to analyze each frequency band in different phonemes. The analysis idea of PF is to calculate the difference between different methods of a phoneme k on the lth filter. The ratio of the variance to the variance within the same method. The higher the value, the greater the difference between different methods in this area. The specific calculation formula of PF is as follows:

表示第t个类型第k个音素第j帧中第l个滤波器上的数据；

表示第t个类型第k个音素的每一帧第l个滤波器上的数据平均值；u_k表示所有类型第k个音素的每一帧第l个滤波器上的数据平均值。

和u_k的公式如下：Among them, T represents the type of method, which is only divided into two types: real voice and fraudulent voice, so T is 2; Ntk represents the frame number of the kth phoneme in the tth type;

represents the data on the lth filter in the jth frame of the kth phoneme of the tth type;

Represents the average value of data on the l-th filter of each frame of the k-th phoneme of the t-th type; u _k represents the data average of the l-th filter of each frame of the k-th phoneme of all types.

The formulas for and u _k are as follows:

After that, the obtained PF value is normalized to obtain the F-ratio contribution rate (Phoneme F-ratio Contribution, PFC) of the phoneme on the lth filter band, and its calculation formula is as follows:

Where L is the number of uniform subband filters; the calculated PFC can reflect the frequency distribution of information used to identify fraudulent speech in different phonemes. Next, in order to suppress the influence of the difference of different phonemes in the process of speaker recognition, the PFC of each phoneme is weighted and averaged according to its frame number, and the overall F-ratio value (General F-ratio, GF) is obtained. The calculation formula is as follows:

Where P is the total number of all phonemes, N is the total number of frames of all phonemes, and the calculation formula of N is as follows:

By normalizing the calculated GF, the phoneme effect suppression information distribution (Phoneme Effect Suppressed Spoof Detection Information Distribution, PESSDID) can be obtained. The calculation formula is as follows:

Here, the higher the value of PESSDID of filter 1, the more information on the frequency band of the filter that can be used to identify fraudulent attacks.

Step 3, extract features:

According to the distribution obtained in the previous step, we carry out the filter design of the new feature proposed by the present invention, in which the number of filters is appropriately increased in the frequency band with more information, and the number of filters is appropriately reduced in the frequency band with less information , the adjusted filter can be seen in Figure-2. Except for the difference of filter distribution, the process of other feature extraction is the same as the traditional method. The steps before using the filter include pre-emphasis, framing and windowing, and then after short-time Fourier transformation, the spectral features of each frame are obtained. , and then use the filter to process the spectral features, and then go through DCT transformation after processing to obtain the final features.

Step 4, model training

When training the model, for the speech in the training set, it is no longer necessary to perform speech annotation, and the original audio can be directly processed by the new feature extraction method. According to the true and false labels of the audio in the training set, the obtained features are used to train the GMM model of real speech and the GMM model of fraudulent speech, respectively.

Step 5: Score confirmation

After the audio to be tested is divided into frames and other processing, new features are extracted, and then the features are input into the GMM models of real voice and fraudulent voice respectively for scoring. The specific method is to input each frame of an audio to a GMM model one by one to obtain a similarity score, and then average the results of all frames as the scoring result of the audio in the GMM model. Finally, the maximum likelihood ratio method is used to calculate the scores of the tested audio in the two models, and then the final result is obtained.

The results of the experiments are evaluated using the Equal Error Rate (EER), which represents the error rate when the False Acceptance Rate (FAR) and the False Rejection Rate (FRR) are equal.

The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection of the present invention. within the range.

Claims (4)

1. a synthetic speech detection method that suppresses the influence of phonemes based on phoneme-level analysis is characterized in that, each frequency band of different phonemes in different real voices and fraudulent voices is analyzed using F-ratio, and F-ratio is called variance ratio test. , is to find the difference distribution in each classification by comparing the variance within and between classes, find out the frequency range that is more conducive to distinguishing real voice and fraudulent voice through the analysis, and increase the filter density on this frequency band, Obtain a new feature, and use the feature to train the Gaussian mixture model GMM of real speech and fraudulent speech respectively, extract the features of the audio to be recognized and input them into the two models respectively, and finally score the results of the two models with the maximum likelihood ratio to get the final recognition result. 2. the synthetic speech detection method that suppresses phoneme influence based on phoneme level analysis as claimed in claim 1, is characterized in that, concrete steps are as follows: Step 1, data preparation: First, annotate the speech data, that is, obtain each phoneme in the audio and their starting time information, then study each phoneme in the real speech and the fraudulent speech separately, and use a uniform subband filter to process the speech audio In each frame, and then obtain the data of each frequency band on each frame of different phonemes; Step 2, data analysis: Use the phoneme-level F-ratio method to analyze the data obtained in the previous step. The F-ratio value on a certain frequency band is used to represent the ability of the frequency band to identify real speech and fraudulent speech. The larger the value of F-ratio, , indicating that this channel carries more information for identification, and the identification ability is stronger. Then, according to the F-ratio value on all channels, the results are normalized, and then the frame number of each phoneme is used as the weight, The normalized data on each frequency band of the phoneme is weighted and averaged, and finally the discrimination ability on each frequency band after the influence of the phoneme is suppressed is obtained. The larger the result, the stronger the discrimination ability; Step 3, extract features: According to the experimental results of the second step, in areas with strong discrimination ability, increase the number of filters to increase the density of filters in these areas, and then use these filters to The speech signal after short-time Fourier transform is filtered, and finally a new feature that suppresses the influence of phonemes is obtained through discrete cosine transform (DCT); Step 4, model training Using the audio extraction features in the training set as input, train the Gaussian mixture model GMM of real speech and fraudulent speech respectively; Step 5: Score confirmation The features extracted from the speech to be tested are input into the models of real speech and fraudulent speech respectively for scoring, and then the maximum likelihood ratio classification method is used to obtain the final result. 3. the synthetic speech detection method that suppresses phoneme influence based on phoneme level analysis as claimed in claim 2 is characterized in that, step 2, the concrete steps of data analysis are as follows: Use the phoneme-based F-ratio analysis method PF (Phoneme F-ratio) to analyze each frequency band in different phonemes. The analysis idea of PF is to calculate the variance of a phoneme k on the lth filter between different methods and The ratio of variance within the same method, the higher the value, the greater the difference between different methods in this area. The specific calculation formula of PF is as follows:

Among them, T represents the method type, and Ntk represents the frame number of the k-th phoneme in the t-th type;

represents the data on the lth filter in the jth frame of the kth phoneme of the tth type;

represents the average value of data on the lth filter of each frame of the kth phoneme of the tth type; u _k represents the data average value of the lth filter of each frame of the kth phoneme of all types,

The formulas for and u _k are as follows:

After normalizing the obtained PF value, the F-ratio contribution rate PFC (Phoneme F-ratio Contribution) of the phoneme on the lth filter band can be obtained, and its calculation formula is as follows:

Among them, L is the number of uniform subband filters; the calculated PFC can reflect the frequency distribution of the information used to identify fraudulent speech in different phonemes, and then the PFC of each phoneme is weighted and averaged according to its frame number to obtain The overall F-ratio value GF (General F-ratio), its calculation formula is as follows:

Where P is the total number of all phonemes, N is the total number of frames of all phonemes, and the calculation formula of N is as follows:

The calculated GF is then normalized to obtain the PESSDID (Phoneme Effect Suppressed Spoof Detection Information Distribution) that suppresses the influence of the phoneme. The calculation formula is as follows:

Here, the higher the value of PESSDID of filter 1, the more information on the frequency band of the filter that can be used to identify fraudulent attacks. 4. the synthetic speech detection method that suppresses the influence of phoneme based on phoneme-level analysis as claimed in claim 2, it is characterized in that, step 3, in the extraction feature step: except the difference of filter distribution, the process of other feature extraction comprises: using The steps before the filter include pre-emphasis, framing and windowing, and then through short-time Fourier transformation, the spectral characteristics of each frame are obtained, and then the spectral characteristics are processed by the filter, and then subjected to DCT transformation after processing to obtain final feature.

Images