CN111564163A - An RNN-based voice detection method for multiple forgery operations - Google Patents

Abstract

The invention discloses a method for detecting various fake operations based on RNN, which comprises the following steps: 1) obtaining an original voice sample, performing M kinds of counterfeiting processing on the original voice sample to obtain M voices after counterfeiting operation and 1 original voice without processing, performing feature extraction on the voices to obtain an LFCC matrix of a training voice sample, and sending the LFCC matrix into an RNN classifier network for training to obtain a multi-classification training model; 2) obtaining a section of test voice, extracting the characteristics of the test voice to obtain an LFCC matrix of the test voice data, sending the LFCC matrix into the RNN classifier trained in the step 1) for classification, obtaining an output probability for each test voice, and combining all the output probabilities as a final prediction result: if the prediction result is the original voice, the test voice is recognized as the original voice; if the prediction result is a voice subjected to a certain falsification operation, the test voice is recognized as a falsified voice subjected to a corresponding falsification operation.

Description

An RNN-based voice detection method for multiple forgery operations

technical field

The invention relates to a voice detection method, in particular to a voice detection method for multiple forgery operations based on RNN.

Background technique

With the continuous enhancement of the functions of voice editing software, non-professionals can easily modify the voice content. If some criminals maliciously falsify and modify the voice, or even use the modified voice in news reports, judicial forensics, scientific research and other fields, this will bring huge threats and even have an inestimable impact on social stability. . The digital voice forensics method is the detection of forgery operations, and plays a vital role in identifying the originality and authenticity of audio materials. It is a key research topic in the field of multimedia forensics.

Most of the existing digital voice forensics detection technologies detect a single forgery operation, that is, forensics personnel assume that the voice to be detected will undergo a specific forgery operation. Mengyu Qiao et al. proposed a detection algorithm based on the statistical features of quantized MDCT coefficients and their derivatives to detect up-converted and down-converted MP3 audio files, recompress and calibrate the audio to generate a reference audio signal, and then use the support vector The experimental results show that the method can effectively detect MP3 double compression and can detect the audio processing history of digital forensics. For example, Wang Lihua et al. proposed a tone-shifting speech processing history detection based on convolutional neural network. By applying four different tone-shifting software to three kinds of voice databases, the tone-shifting software was used, and CNN was used to analyze the tone-shifting factor of the voice in the voice library and the library. The detection rate of between and between the modulation methods has reached more than 90%.

Existing digital voice forensics detection technology can detect a single forgery operation, and the detection rate can reach a very high level. However, in practical applications, forensics are often unable to predict the specific operations of forgery, and using a specific operation classifier for detection may result in misjudgment.

At present, most of the existing digital forensics work applicable to a variety of forgery operations focus on the field of digital images, and there are still few researches on digital voice forensics. In the field of digital speech, Luo Weiqi's team designed a convolutional neural network model that can be used to detect audio processing operations with default settings in two different audio editing software, and provided better results. This method can significantly outperform existing Some forensics methods are based on handcrafted features. However, although this experiment has pioneered research on the detection of multiple forgery operations in speech, there are some problems that cannot be ignored, such as the high computational complexity and the ideal application scenarios for forgery operations.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to provide an RNN-based voice detection method for multiple forgery operations, which can improve the detection accuracy.

The technical solution adopted by the present invention to solve the above-mentioned technical problems is: an RNN-based multiple forgery operation voice detection method, which is characterized in that: it comprises the following steps:

1) Train the network: obtain the original voice samples, perform M kinds of forgery processing on the original voice samples, obtain M voices after forgery operations and 1 unprocessed original voice, and compare the above-mentioned M forged voices and 1 Extract the features of the original voices, obtain the LFCC matrix of the training voice samples, and send them to the RNN classifier network for training to obtain a multi-class training model;

2) Speech recognition: obtain a test voice, perform feature extraction on the test voice, obtain the LFCC matrix of the test voice data, and send it to the RNN classifier trained in step 1) for classification, and each test voice obtains an output probability , combine all output probabilities as the final prediction result: if the prediction result is the original speech, the test speech is recognized as the original speech; if the prediction result is the speech that has undergone a certain forgery operation, the test speech is recognized as the corresponding forgery operation. fake voice.

Preferably, in steps 1) and 2), the step of obtaining the LFCC matrix is:

1) FFT: First, preprocess the speech, and calculate the spectral energy E(i,k) of each speech frame after FFT:

Wherein, i is the number of speech frames, k is the frequency component, x _i (m) is the speech signal data of the ith frame, and N is the number of Fourier transforms;

Then calculate the spectral energy E(i,k) of each frame after passing through the triangular filter bank:

Among them, H _i (k) represents the frequency response of the triangular filter, f(l) is the filter function of the l-th triangular filter, S(i,l) is the spectral line energy after the triangular filter bank, and l represents Number of triangular filters, L is the total number of triangular filters;

2) DCT: Calculate the output data lfcc(i,n) of each triangular filter bank using DCT:

Among them, n represents the spectral line after the i-th frame DCT;

3) Obtain the LFCC statistical moment: take lfcc(i,n) as the 12th-order LFCC coefficient, calculate the mean and correlation coefficient, and obtain the LFCC matrix extracted from a piece of speech as:

where x _s,1 ...x _1,n are the calculated n LFCCs of the s-th frame of speech data.

Preferably, the RNN classifier includes an LSTM network, a Dropout layer, a fully connected layer and a Softmax layer connected in sequence, and the Dropout layer is connected to the last LSTM network.

Preferably, the LSTM network has two, and the parameters are set as (64, 128) and (128, 64) respectively.

Preferably, the LSTM network uses a tanh activation function.

Preferably, the Dropout function value of the Dropout layer is 0.5.

Preferably, the original voice is in WAV format.

Compared with the prior art, the present invention has the advantages that: the speech cepstrum feature is used, the probability of outputting the result is classified by the cyclic neural network, the accuracy of speech detection is improved, and it is more suitable for digital speech carriers and can identify different forgery traces; The shared parameters in the RNN network are significantly less computationally complex than existing deep learning-based methods.

Description of drawings

Fig. 1 is the extraction process diagram of the LFCC statistical moment of the speech detection method of the embodiment of the present invention;

2 is a schematic diagram of an overall framework of a speech detection method according to an embodiment of the present invention;

FIG. 3 is a network structure diagram of a speech detection method according to an embodiment of the present invention.

Detailed ways

The following describes in detail the embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions.

In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", " Rear, Left, Right, Vertical, Horizontal, Top, Bottom, Inner, Outer, Clockwise, Counterclockwise, Axial, The orientations or positional relationships indicated by "radial direction", "circumferential direction", etc. are based on the orientations or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying the indicated devices or elements. must have a specific orientation, be constructed and operated in a specific orientation, and since the disclosed embodiments of the present invention may be arranged in different orientations, these directional terms are for illustration only and should not be regarded as limiting, such as "up", "Down" is not necessarily defined as a direction opposite or in line with the direction of gravity. Furthermore, features delimited with "first", "second" may expressly or implicitly include one or more of that feature.

An RNN (Recurrent Neural Network)-based speech detection method for multiple forgery operations is implemented by constructing a cepstral feature-based Recurrent Neural Network framework. Referring to Figure 2, the framework consists of two parts: first, the cepstral features of the speech samples are extracted, and then sent to the designed network framework for classification, so as to achieve the task of identifying various forgery operations.

Specifically, in the present invention, the feature extraction of speech is realized in the following manner. The cepstral feature adopted in the present invention is Linear Frequency Cepstral Coefficients (LFCC). Speech cepstrum feature is one of the most commonly used feature parameters in speech technology. It represents the auditory characteristics of human beings and is widely used in speaker recognition.

LFCC is an even distribution of bandpass filters from low to high frequencies. The extraction process of the LFCC statistical moment of the present invention can refer to Fig. 1:

1) FFT: First, preprocess the speech, and calculate the spectral energy E(i,k) of each speech frame after Fast Fourier Transform (FFT):

Among them, i is the number of speech frames, k is the frequency component, x _i (m) is the speech signal data of the ith frame, and N is the number of Fourier transforms.

Calculate the spectral energy E(i,k) of each frame after passing through the triangular filter bank:

Among them, H _i (k) represents the frequency response of the triangular filter, f(l) is the filter function of the l-th triangular filter, S(i,l) is the spectral line energy after the triangular filter bank, and l represents Number of triangular filters, L is the total number of triangular filters.

2) DCT: Then, the discrete cosine transform (Discrete Cosine Transform, DCT) is used to calculate the output data lfcc(i,n) of each triangular filter bank:

Among them, n represents the spectral line after the DCT of the ith frame.

3) Obtain the LFCC statistical moment: take lfcc(i,n) as the 12th-order LFCC coefficient, and calculate the mean and correlation coefficient. The above steps can be implemented by existing matlab functions. It is assumed that a certain segment of preprocessed speech has a total of s frames , the LFCC matrix extracted from this segment of speech is:

where x _s,1 ...x _1,n are the calculated n LFCCs of the s-th frame of speech data.

See Figure 3, the network framework adopts RNN classifier. The selection of the number of network layers of the RNN classifier is very important for the optimization algorithm. A deeper network can learn more knowledge, but at the same time, the training also takes a long time and may will overfit. Therefore, in the present invention, the network structure of the proposed RNN classifier is shown in FIG. 3 . The network structure contains 2 LSTM networks, the parameters are set to (64, 128) and (128, 64) respectively, and the tanh activation function is used to improve the model performance. It also includes a Dropout layer, a fully connected layer (dense), and a Softmax layer that are connected in sequence, and the Dropout layer is connected to the last LSTM network. Set the value of the Dropout function to 0.5, which helps to reduce overfitting. After dimensionality reduction by the fully connected layer, the Softmax layer (Softmax classifier) is used to output the probability. The overall iterative training of the network framework is set to 50 epochs. Certain adjustments can be made during specific training.

2 again, the speech detection method includes the following steps:

1) First, the network framework needs to be trained. Assuming that there are M kinds of forgery operations, M kinds of forgery processing are performed on the original speech respectively, and M+1 kinds of speech samples are obtained, including M pieces of speech after forgery operations and 1 unprocessed original speech. In the present invention, there are certain constraints on the input of the original speech, and a certain amount of WAV format audio sample library needs to be provided as the training data of the network framework. Perform feature extraction on the above M+1 speech samples to obtain the LFCC matrix of the training speech samples, send them to the designed RNN classifier network for training, and obtain a multi-classified training model; multiple original speech can be stored in the database. Samples are extracted for each original speech sample and sent to the RNN classifier for training;

2) Then, the detection and recognition results are obtained through the trained network framework: when a test voice is obtained, feature extraction is performed on it, the LFCC matrix of the test voice data is obtained, and it is sent to the trained RNN classifier. Classification. Each test speech will get an output probability, and all output probabilities are combined as the final prediction result. If the predicted result is the original voice, the test voice is identified as the original voice; if the predicted result is the voice that has undergone a certain forgery operation, the test voice is identified as the forged voice. The forensic person can judge whether a certain speech has been forged according to the experimental results.

Claims (7)

1. a multiple forgery operation voice detection method based on RNN, is characterized in that: comprise the steps: 1) Train the network: obtain the original voice samples, perform M kinds of forgery processing on the original voice samples, obtain M voices after forgery operations and 1 unprocessed original voice, and compare the above-mentioned M forged voices and 1 Extract the features of the original voices, obtain the LFCC matrix of the training voice samples, and send them to the RNN classifier network for training to obtain a multi-class training model; 2) Speech recognition: obtain a test voice, perform feature extraction on the test voice, obtain the LFCC matrix of the test voice data, and send it to the RNN classifier trained in step 1) for classification, and each test voice obtains an output probability , combine all output probabilities as the final prediction result: if the prediction result is the original speech, the test speech is recognized as the original speech; if the prediction result is the speech that has undergone a certain forgery operation, the test speech is recognized as the corresponding forgery operation. fake voice. 2. the multiple forgery operation voice detection method based on RNN according to claim 1, is characterized in that: in step 1) and 2) in, the step that obtains LFCC matrix is: 1) FFT: First, preprocess the speech, and calculate the spectral energy E(i,k) of each speech frame after FFT:

Wherein, i is the number of speech frames, k is the frequency component, x _i (m) is the speech signal data of the ith frame, and N is the number of Fourier transforms; Then calculate the spectral energy E(i,k) of each frame after passing through the triangular filter bank:

Among them, H _i (k) represents the frequency response of the triangular filter, f(l) is the filter function of the l-th triangular filter, S(i,l) is the spectral line energy after the triangular filter bank, and l represents Number of triangular filters, L is the total number of triangular filters; 2) DCT: Calculate the output data lfcc(i,n) of each triangular filter bank using DCT:

Among them, n represents the spectral line after the i-th frame DCT; 3) Obtain the LFCC statistical moment: take lfcc(i,n) as the 12th-order LFCC coefficient, calculate the mean and correlation coefficient, and obtain the LFCC matrix extracted from a piece of speech as:

where x _s,1 ...x _1,n are the calculated n LFCCs of the s-th frame of speech data. 3. the multiple forgery operation voice detection method based on RNN according to claim 1, it is characterized in that: described RNN classifier comprises LSTM network, the Dropout layer, fully connected layer and Softmax layer that are connected successively, described Dropout layer Connect with the last LSTM network. 4. The RNN-based voice detection method for multiple forgery operations according to claim 3, wherein the LSTM network has two, and the parameters are respectively set as (64, 128) and (128, 64). 5. The RNN-based voice detection method for multiple forgery operations according to claim 3, wherein the LSTM network uses a tanh activation function. 6 . The RNN-based voice detection method for multiple forgery operations according to claim 3 , wherein the Dropout function value of the Dropout layer is 0.5. 7 . 7 . The RNN-based voice detection method for multiple forgery operations according to claim 1 , wherein the original voice is in WAV format. 8 .

Images