CN107068167A - Merge speaker's cold symptoms recognition methods of a variety of end-to-end neural network structures - Google Patents

Abstract

The invention relates to a method for identifying cold symptoms of a speaker who integrates multiple end-to-end neural network structures. Terminal neural network A; S2. Construction and training input is speech spectrum, and the recognition network is an end-to-end neural network B of convolutional neural network and long-term short-term memory network; S3. Construction and training input is speech spectrum, and the recognition network is convolution End-to-end neural network C of neural network and fully connected network; S4. Construction and training input is speech MFCC feature/CQCC feature, and recognition network is end-to-end neural network D of long short-term memory network; S5. Combine the above four kinds of training A good end-to-end neural network for speaker cold symptom recognition.

Description

A speaker's cold symptom recognition method integrating multiple end-to-end neural network structures

technical field

The present invention relates to the field of voiceprint recognition, and more specifically, relates to a speaker's cold symptom recognition method that integrates multiple end-to-end neural network structures.

Background technique

Speaker recognition, also known as voiceprint recognition, is a technology that uses pattern recognition technology to automatically identify speakers. The current speaker recognition technology has achieved good performance in experimental conditions, but in practice, the recognized speech will be affected by environmental noise and the speaker's health conditions, which reduces the robustness of the existing speaker recognition technology. Existing speaker recognition methods are mainly used to determine the speaker's identity, and there is no relevant recognition method applied to the speaker's cold symptoms at present.

In the research of speech technology, researchers always hope to find the features that represent the target type, and describe the characteristics that are clearly different from normal speech from the recognition of target speech. Speech feature extraction is to extract the speaker's speech features and vocal tract features. Currently, , the mainstream feature parameters include MFCC, LPCC, CQCC, etc., all of which are based on a single feature, and the information representing the speaker's cold symptoms is insufficient, which affects the recognition accuracy. At the same time, a lot of knowledge is needed to distinguish the target speech for classification. In the recognition algorithm, the method based on the vocal tract model and the speech model started earlier, but because of the complexity of the model, it has not achieved good practical results. And model matching methods such as dynamic time warping, hidden Markov model, vector quantization and other technologies have begun to play a good role in recognition. Separate research on feature extraction and pattern classification is a common method for recognition research, but there are problems such as mismatch between features and models, difficulty in training, and difficulty in finding features. The classic recognition framework has the above problems.

In recent years, with the development of deep learning, the recognition of images and speech based on deep neural networks has shown great energy, and a series of neural network structures have also been proposed, such as automatic encoding networks, convolutional neural networks, and recurrent neural networks. . Many scholars have found that learning speech through neural networks can better describe the hidden structural features of speech. The end-to-end recognition method is to process feature learning and feature recognition with as little prior knowledge as possible. Good recognition effect.

Contents of the invention

The present invention provides a language that integrates multiple end-to-end neural network structures in order to solve the problems of feature extraction and pattern classification that are caused by the separation of feature extraction and pattern classification in the prior art. A method for recognizing cold symptoms of a person. By unifying feature learning and pattern classification, the method makes the entire speaker's cold symptom recognition process simpler and faster, and has broad application prospects.

For realizing above-mentioned purpose of the invention, the technical scheme that adopts is:

A speaker's cold symptom recognition method that integrates multiple end-to-end neural network structures, including the following steps:

S1. Construct and train an end-to-end neural network A in which the input is speech, and the recognition network is a convolutional neural network and a long-term short-term memory network;

S2. Construction and training input is speech spectrum, and the recognition network is an end-to-end neural network B of convolutional neural network and long-term short-term memory network;

S3. Construction and training input is speech spectrum, and the recognition network is an end-to-end neural network C of convolutional neural network and fully connected network;

S4. Construction and training input is the voice MFCC feature/CQCC feature, and the recognition network is an end-to-end neural network D of a long short-term memory network;

S5. Combining the above four trained end-to-end neural networks to identify the speaker's cold symptoms.

Preferably, the convolutional neural network of the end-to-end neural network A includes 8 modules, each of which includes a one-dimensional convolutional layer, a ReLU activation layer and a one-dimensional maximum pooling layer, wherein the one-dimensional convolutional layer The size of the convolution kernel is 32, the size of the pooling kernel of the one-dimensional maximum pooling layer is 2, and the pooling stride is 2.

Preferably, the convolutional neural network of the end-to-end neural network B includes 6 modules, and each module includes a two-dimensional convolutional layer, a ReLU activation layer and a two-dimensional maximum pooling layer; wherein the first convolutional layer uses 7*7 convolution kernels, the second layer uses 5*5 convolution kernels, and the remaining 4 layers use 3*3 convolution kernels; all maximum pooling layers use 3*3 pooling kernels, pooling The step size is 2.

Preferably, the convolutional neural network of the end-to-end neural network C includes 6 modules, and each module includes a two-dimensional convolutional layer, a ReLU activation layer and a two-dimensional maximum pooling layer; wherein the first convolutional layer uses 7*7 convolution kernels, the second layer uses 5*5 convolution kernels, and the remaining 4 layers use 3*3 convolution kernels; all maximum pooling layers use 3*3 pooling kernels, pooling The step size is 2.

Compared with prior art, the beneficial effect of the present invention is:

Existing recognition technologies study features and pattern classification separately, and there are problems such as mismatch between features and models, difficulty in training, and difficulty in finding features. However, the method provided by the present invention unifies feature learning and pattern classification by fusing four different end-to-end neural networks, so that the entire speaker's cold symptom recognition process is simpler and faster, and has broad application prospects.

Description of drawings

Figure 1 is a schematic diagram of the specific implementation of the method.

Fig. 2 is a flow chart of extracting Mel cepstral coefficients (MFCC) from speech.

Fig. 3 is a flow chart of speech extraction constant Q cepstrum coefficient (CQCC).

FIG. 4 is a schematic diagram of an end-to-end neural network A.

FIG. 5 is a schematic diagram of an end-to-end neural network B.

FIG. 6 is a schematic diagram of an end-to-end neural network C.

FIG. 7 is a schematic diagram of an end-to-end neural network D.

detailed description

The accompanying drawings are for illustrative purposes only and cannot be construed as limiting the patent;

The present invention will be further elaborated below in conjunction with the accompanying drawings and embodiments.

Example 1

Fig. 1 is a specific implementation process diagram of the method provided by the present invention. As shown in Fig. 1, the speaker's cold symptom recognition method that fuses multiple end-to-end neural network structures provided by the present invention includes the following steps:

S1. Construct and train an end-to-end neural network A in which the input is speech, and the recognition network is a convolutional neural network and a long-term short-term memory network;

S2. Construction and training input is speech spectrum, and the recognition network is an end-to-end neural network B of convolutional neural network and long-term short-term memory network;

S3. Construction and training input is speech spectrum, and the recognition network is an end-to-end neural network C of convolutional neural network and fully connected network;

S4. construction and training input is speech MFCC feature/CQCC feature, and recognition network is the end-to-end neural network D of long short-term memory network, specifically as shown in Figure 7;

S5. Combining the above four trained end-to-end neural networks to identify the speaker's cold symptoms.

Among them, as shown in Figures 2 and 3, the MFCC features in step S4 are pre-emphasized on the speech, windowed and framed, fast Fourier transform, calculated energy spectral density, Mel scale triangular filter bank filtering, and paired The CQCC feature is obtained by performing constant Q transformation on the speech, calculating the energy spectral density, taking logarithmic operation, and discrete cosine transform.

In the specific implementation process, as shown in Figure 4, the convolutional neural network of the end-to-end neural network A includes 8 modules, and each module includes a one-dimensional convolution layer, a ReLU activation layer and a one-dimensional maximum pooling layer, where the size of the convolution kernel of the one-dimensional convolutional layer is 32, the size of the pooling kernel of the one-dimensional maximum pooling layer is 2, and the pooling step size is 2.

In the specific implementation process, as shown in Figure 5, the convolutional neural network of the end-to-end neural network B includes 6 modules, each module includes a two-dimensional convolutional layer, a ReLU activation layer and a two-dimensional maximum pooling layer; where the first convolutional layer uses a 7*7 convolutional kernel, the second layer uses a 5*5 convolutional kernel, and the remaining 4 layers use a 3*3 convolutional kernel; all maximum pooling layers are Use a 3*3 pooling kernel with a pooling step size of 2.

In the specific implementation process, as shown in Figure 6, the convolutional neural network of the end-to-end neural network C includes 6 modules, each module includes a two-dimensional convolutional layer, a ReLU activation layer and a two-dimensional maximum pooling layer; where the first convolutional layer uses a 7*7 convolutional kernel, the second layer uses a 5*5 convolutional kernel, and the remaining 4 layers use a 3*3 convolutional kernel; all maximum pooling layers are Use a 3*3 pooling kernel with a pooling step size of 2.

Apparently, the above-mentioned embodiments of the present invention are only examples for clearly illustrating the present invention, rather than limiting the implementation of the present invention. For those of ordinary skill in the art, on the basis of the above description, other changes or changes in different forms can also be made. It is not necessary and impossible to exhaustively list all the implementation manners here. All modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the claims of the present invention.

Claims (4)

1. speaker's cold symptoms recognition methods of a variety of end-to-end neural network structures of fusion, it is characterised in that：Including following Step：

S1. it is voice to build and train input, and identification network is the end-to-end god of convolutional neural networks and shot and long term memory network Through network A；

S2. it is voice spectrum that structure and training, which are inputted, and identification network arrives for the end of convolutional neural networks and shot and long term memory network Terminal nerve network B；

S3. it is voice spectrum to build and train input, and identification network is the end-to-end god of convolutional neural networks and fully-connected network Through network C；

S4. it is voice MFCC features/CQCC features to build and train input, and identification network is end-to-end for shot and long term memory network Neutral net D；

S5. the end-to-end neutral net that more than fusion four kinds train carries out speaker's cold symptoms identification.

2. speaker's cold symptoms recognition methods of a variety of end-to-end neural network structures of fusion according to claim 1, It is characterized in that：The convolutional neural networks of the end-to-end neutral net A include 8 modules, and each module includes one-dimensional volume Lamination, ReLU active coatings and one-dimensional maximum pond layer, wherein the size of the convolution kernel of one-dimensional convolutional layer is 32, one-dimensional maximum pond The Chi Huahe of layer size is 2, and pond step-length is 2.

3. speaker's cold symptoms recognition methods of a variety of end-to-end neural network structures of fusion according to claim 1, It is characterized in that：The convolutional neural networks of the end-to-end neutral net B include 6 modules, and each module includes two-dimensional convolution Layer, ReLU active coatings and Two-dimensional Maximum pond layer；Wherein first convolutional layer uses 7*7 convolution kernel, and the second layer uses 5*5's Convolution kernel, is left 4 layers of convolution kernel using 3*3；All maximum pond layers use 3*3 Chi Huahe, and pond step-length is 2.

4. speaker's cold symptoms recognition methods of a variety of end-to-end neural network structures of fusion according to claim 1, It is characterized in that：The convolutional neural networks of the end-to-end neutral net C include 6 modules, and each module includes two-dimensional convolution Layer, ReLU active coatings and Two-dimensional Maximum pond layer；Wherein first convolutional layer uses 7*7 convolution kernel, and the second layer uses 5*5's Convolution kernel, is left 4 layers of convolution kernel using 3*3；All maximum pond layers use 3*3 Chi Huahe, and pond step-length is 2.