JP2018141925A - Acoustic model learning device and acoustic model learning program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an acoustic model learning device which has the expressive power required for a Japanese acoustic model, has a short learning time, and has an improved WER. SOLUTION: An acoustic model learning means 100C comprises a deep learning means 111A composed of three layers of BLSTM and a linear mapping means 112B for converting the dimension of a vector output by BLSTM30c of the final layer of the deep learning means by a predetermined operation. Be prepared. The BLSTM30d makes the memory cell c that stores information in the time direction smaller than the other layers and compresses the number of dimensions of its output from 640 to 320. The linear mapping means 112B is a first transformation matrix and a second transformation matrix obtained by matrix-decomposing the transformation matrix represented by 640, which is the number of dimensions of the output of BLSTM30c, and 2934, which is the number of dimensions of the character output vector 7, at rank r = 320. Multiply the transformation matrix sequentially. [Selection diagram] FIG. 7

Description

  The present invention relates to an acoustic model learning device and an acoustic model learning program.

  In recent years, several end-to-end speech recognition methods using DNN (Deep Neural Network) have been proposed in the field of speech recognition (Non-patent Documents 1 and 2). For this purpose, the acoustic model learning device directly learns the correspondence between speech and characters by using one acoustic model, and performs end-to-end conversion from speech to characters without going through an intermediate state of phonemes. In the end-to-end speech recognition technique, RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory), or BLSTM (Bi-directional LSTM) may be used to store time direction information. is there.

  A network structure in which the number of units in a specific layer of the DNN is reduced is called a bottleneck structure, and the bottleneck structure may be used as an input for another DNN (see Non-Patent Document 3). . Here, reducing the number of units corresponds to reducing the number of parameters (the number of dimensions) to be determined by learning.

  Further, in Non-Patent Document 4, in the field of a speech recognition method (DNN-HMM) based on HMM (Hidden Markov Model) using DNN, if a matrix decomposition is used as a transformation matrix of Affine transformation (linear transformation), WER It describes that learning time can be shortened without reducing (Word error rate).

Amodei, D., et al., “Deep Speech 2: End-to-End Speech Recognition in English and Mandarin” the Computing Research Repository (CoRR), arXiv: 1512.02595v1 [cs.CL] 8 Dec 2015 Miao, Y., et al., "ESSEN: END-TO-END SPEECH RECOGNITION USING DEEP RNN MODELS AND WFST-BASED DECODING" the Computing Research Repository (CoRR), arXiv: 1507.08240v3 [cs.CL] 18 Oct 2015 Wollmer M., et al., "FEATURE ENHANCEMENT BY BIDIRECTIONAL LSTM NETWORKS FOR CONVERSATIONAL SPEECH RECOGNITION IN HIGHLY NON-STATIONARY NOISE", 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Pages 6822-6826 (2013) Sainath T., et al., "LOW-RANK MATRIX FACTORIZATION FOR DEEP NEURAL NETWORK TRAINING WITH HIGH-DIMENSIONAL OUTPUT TARGETS", 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Pages 6655-6659 (2013)

However, the following problems existed in the prior art.
Many of the conventional speech recognition technologies are targeted for English speech recognition with about 30 conversion candidates. In the case of Japanese, if hiragana, katakana, kanji, etc. are combined, the number of conversion candidates is as large as 2000 or more. Therefore, the number of parameters to be determined by learning (the number of dimensions of the vector when the parameter is regarded as a vector) is remarkably large. Increase.
Further, the conventional technique has a problem that the learning time increases when the number of parameters (number of dimensions) to be determined by learning increases. There is also a problem (generalization ability problem) that if too many parameters, too much detail is expressed too much, and conversely, general and essential features cannot be expressed. On the other hand, if there are too few parameters, the required number of characters cannot be expressed.

  Therefore, in the acoustic model learning apparatus using a neural network capable of storing information in the time direction such as RNN, LSTM, or BLSTM, if the number of parameters to be determined by learning can be appropriately reduced, Japanese speech recognition It is expected to be applicable to

  Further, in the speech recognition system to be researched described in Non-Patent Document 4, DNN-HMM passing through a phoneme string is used as an acoustic model, and the acoustic used in the end-to-end speech recognition method. It does not target the model.

  The present invention has been made in view of the above problems, and has an expressive power necessary for a Japanese acoustic model, an acoustic model learning apparatus and an acoustic model learning that have a short learning time and an improved WER. The challenge is to provide a program.

  In order to solve the above-mentioned problem, the present invention, as an acoustic model learning device, learns the correspondence between the input speech and the characters that are output when the input speech is speech-recognized. An acoustic model learning apparatus that learns an acoustic model that converts to a character using a to-end speech recognition method and outputs the character, and has a neural network having a multilayer structure of three or more layers, and features of speech Is stored in each layer of the multilayer structure, and information on the time direction about the feature amount is stored in each layer of the multilayer structure, and any one of a plurality of target characters from the feature amount of the voice is stored using the information on the time direction. A deep learning means for outputting a feature vector representing the probability of predicting whether or not, and applying a predetermined transformation matrix to the feature vector that is the output of the last layer of the deep learning means And a linear mapping unit that transforms the dimension of the feature vector output from the deep learning unit by a predetermined calculation, and is handled by at least one of the calculations performed by the deep learning unit and the linear mapping unit. The acoustic model is learned by compressing the dimension of the vector.

The present invention has the following excellent effects.
According to the acoustic model learning device according to the present invention, the number of parameters to be determined when learning an acoustic model using an end-to-end speech recognition method by performing dimension compression processing of a vector handled by calculation. Reduced.
Moreover, according to the acoustic model learning device of the present invention, the Japanese acoustic model has the expressive power necessary, the word recognition error rate (WER) is improved, and the learning time and the number of learnings are remarkably shortened.

It is a block diagram which shows the whole structure of the Japanese speech recognition apparatus provided with the Japanese acoustic model learning apparatus which concerns on this embodiment. It is a figure which shows an example of the standard network structure which has a BLSTM structure among End-to-end acoustic models. It is a schematic diagram explaining the network structure of the acoustic model used with the acoustic model learning means which concerns on 1st Embodiment. It is a figure which shows an example of the network structure of the acoustic model used with the acoustic model learning means which concerns on 1st Embodiment. It is a schematic diagram explaining a linear conversion part among the network structures of the acoustic model used with the acoustic model learning means which concerns on 2nd Embodiment. It is a figure which shows an example of the network structure of the acoustic model used with the acoustic model learning means which concerns on 2nd Embodiment. It is a figure which shows an example of the network structure of the acoustic model used with the acoustic model learning means which concerns on 3rd Embodiment.

Hereinafter, a Japanese acoustic model learning device according to an embodiment of the present invention will be described with reference to the drawings.
[Configuration of Japanese speech recognition device]
A Japanese speech recognition device 1 shown in FIG. 1 includes a Japanese acoustic model learning device 10 and a Japanese language model learning device 20.

  The Japanese acoustic model learning device 10 learns an acoustic model that converts input speech into characters by End-to-end and outputs it by learning correspondence between input speech and output characters. It is a device to do. Hereinafter, the learning data 2 for creating a Japanese acoustic model will be described as a pair of a voice 2a and a text 2b. The voice 2a and the text 2b represent a large amount of Japanese voice data and a large amount of text. For example, the program audio of a broadcast program for pre-learning can be used as the audio 2a, and the content of the program audio can be transcribed or equivalent to the text 2b.

  Here, the Japanese acoustic model learning device 10 includes an acoustic model learning unit 100 and an acoustic model storage unit 101.

  The acoustic model learning means 100 performs learning using a pair of a speech 2a and a text 2b and a character label (hereinafter simply referred to as a label) in the learning data 2 for creating an acoustic model in Japanese, and the speech is any of the labels. The model (acoustic model) parameter (weighting coefficient, etc.) for outputting (which character is) is learned, and the acoustic model is stored in the acoustic model storage means 101. Labels corresponding to Japanese include hiragana and katakana phonetic characters, kanji ideographs, and symbols such as punctuation marks. Hereinafter, a label including a symbol may be simply referred to as a character, or a label string may be referred to as a character string. The acoustic model learning unit 100 can be applied to all end-to-end acoustic models that specify a character sequence as described in Non-Patent Document 2.

In this acoustic model, acoustic features (mel frequency cepstrum coefficients, filter bank output, etc.) extracted in advance from a large amount of speech data are classified into a deep neural network (Deep Neural Network) and a connectionist time series classification method (CTC) for each set label. : Connectionist Temporal Classification) etc. It should be noted that the likelihood calculation of the acoustic feature quantity by the acoustic model is a long short term memory (LSTM: Long Short Term) even if the output is a grapheme containing kanji, even if it is a recurrent neural network (RNN). Memory).
The acoustic model storage unit 101 stores an acoustic model generated by learning by the acoustic model learning unit 100, and is a general storage medium such as a hard disk.

The above description corresponds to the description of the process in the pre-learning phase among the two phases (pre-learning phase and evaluation phase) to which the acoustic model is applied.
On the other hand, in the evaluation phase after the learning is completed, the evaluation voice 3 is input instead of the learning data 2 to the acoustic model storage unit 101 (Japanese acoustic model learning device 10). At this time, the acoustic model learning unit 100 recognizes the evaluation speech 3 using the acoustic model generated by the prior learning stored in the acoustic model storage unit 101, and outputs a corresponding character string. To do.

  That is, in the evaluation phase, the acoustic model learning unit 100 converts the input evaluation sound 3 into a feature amount (feature vector), and the acoustic model stored in the acoustic model storage unit 101 is converted to the feature model. And functioning as a character string generating means for generating a character string by sequentially converting to a label (character).

In the evaluation phase, when the feature amount (feature vector) is input instead of the evaluation voice 3, the acoustic model learning unit 100 does not perform the conversion process, but the input feature amount. Using the acoustic model, it may be sequentially converted into a label.
Further, a character string generation unit that performs processing corresponding to the evaluation phase may be provided separately, and the acoustic model learning unit 100 may be configured to perform only processing corresponding to the pre-learning phase.

  The Japanese language model learning device 20 is a device that learns a language model that outputs a word string from a label using a large amount of Japanese text. Here, the Japanese language model learning device 20 includes a language model learning unit 200 and a language model storage unit 201.

  The language model learning unit 200 learns parameters of a model (language model) that outputs a word string from a label using the label and the language model corpus 4, and stores the language model in the language model storage unit 201. The language model corpus 4 is a corpus in which natural language sentences are accumulated on a large scale. The language model corpus 4 includes a larger amount of data than the text 2b of the learning data 2 for creating the acoustic model.

The language model storage unit 201 stores a language model generated by learning by the language model learning unit 200 and is a general storage medium such as a hard disk.
The language model stored in the language model storage unit 201 is obtained by inputting the evaluation speech 3 or its feature amount to the acoustic model storage unit 101 like the model described in Non-Patent Document 2. The present invention is applicable to all models that take a character string including an ideographic character as an input, estimate a word string from the relationship between previous and subsequent words, and output a word string as an estimation result. The language model is obtained by modeling the appearance probability of an output sequence (words and the like) learned in advance from a large amount of text. For example, a general N-gram language model can be used.

In the evaluation phase, when the speech 3 or its feature value is continuously input to the acoustic model having the learned parameters stored in the Japanese acoustic model learning device 10, the corresponding character string is continuously output. And input to the language model storage unit 201 (Japanese language model learning device 20). At this time, the language model learning unit 200 uses the language model having the learned parameters stored in the language model storage unit 201 to recognize the recognition result 5 (as a natural Japanese sentence from the input character string). Output a word string).
That is, in the evaluation phase, the language model learning unit 200 uses the language model stored in the language model storage unit 201 to generate a word string by sequentially converting the input character string into a word. Functions as word string generation means. Note that a word string generation unit that performs processing corresponding to the evaluation phase may be provided separately, and the language model learning unit 200 may be configured to perform only processing corresponding to the pre-learning phase.

[Configuration of Japanese acoustic model learning device 10]
Before describing the network structure of the acoustic model used by the acoustic model learning means 100 of the Japanese acoustic model learning device 10, the network structure of the end-to-end acoustic model will be described with reference to FIG. FIG. 2 shows an example of a standard network structure having a BLSTM structure. However, the present invention can be similarly applied to a general RNN that does not have an LSTM structure or an LSTM structure. It is.

As shown in FIG. 2, an acoustic model learning unit 100R that learns an acoustic model using this standard network structure includes a deep learning unit 111R, a linear mapping unit 112, and a normalizing unit 113.
The deep learning means 111R includes a first layer BLSTM 30a, a second layer BLSTM 30b, and a third layer BLSTM 30c. The deep learning means 111R is a means for learning which voice is the label by using the voice as input. Although the three-layer structure is used here, the deep learning means 111R may be a neural network having a multilayer structure of four or more layers. The deep learning means 111R continuously receives voice feature values, stores time direction information about the sound feature values in each layer of the multilayer structure, and uses the time direction information to store the sound feature values. A feature vector representing the probability of predicting which of the plurality of target characters is output. The deep learning means 111R can define its internal structure by parameters. In the case of the BLSTM structure, the parameters are the number of layers and the memory cell. In the LSTM structure, the memory cell represents a parameter for determining the number of dimensions of a vector for storing information in the time direction, in other words, how far the data on the time axis is taken into the calculation. Since the memory cell in the LSTM structure is described in detail in Non-Patent Document 2, description thereof is omitted here.

In the case of the acoustic model learning unit 100R shown in FIG. 2, the BLSTMs 30a, 30b, and 30c of the layers of the deep learning unit 111R are all the same scale. Specifically, the BLSTM of each layer has a feature vector of 640 dimensions to be output. The memory cells that store the forward time direction information and the memory cells that store the backward time direction information of each BLSTM 30a, 30b, and 30c have the same size (two memory cells are C = 320, respectively). Output a vector of dimensions. The numerical value 320 of the memory cell C corresponds to the memory capacity of one memory cell C. Depending on this value, the number of dimensions of the feature vector output from the memory cell of each layer changes.
The deep learning means 111R receives a feature quantity (feature vector) 6 of 120-dimensional speech and outputs a 640-dimensional feature vector from the BLSTM 30c of the final layer.

  The linear mapping unit 112 receives an acoustic feature amount (feature vector) expressed by the number of dimensions defined by each parameter (in the case of a BLSTM structure, the number of layers and a memory cell) by the deep learning unit 111. The linear mapping unit 112 converts the dimension of the feature vector output from the deep learning unit 111 by a predetermined calculation by applying a predetermined conversion matrix using the feature vector as an input. That is, the linear mapping means 112 converts the dimension of the feature vector output from the BLSTM 30c into the dimension of the character output vector 7. Here, the linear mapping means 112 applies a single Affine transformation matrix to the output vector of the BLSTM 30c. Specifically, the linear mapping unit 112 multiplies the 640-dimensional feature vector input from the BLSTM 30c by a matrix of 640 rows and 2934 columns (hereinafter referred to as a 640 * 320 matrix; the same applies hereinafter) to obtain 2934 dimensions. Returns a vector of. Here, 2934 is the number of Japanese hiragana, katakana, kanji and symbols to be identified. The vector output from the linear mapping unit 112 is input to the normalization unit 113.

  The normalizing means 113 normalizes the dimensional objective function adjusted by the linear mapping means 112. The normalizing means 113 normalizes the dimensional objective function adjusted by the linear mapping means 112 using the Softmax function, and outputs the result as a 2934-dimensional character output vector 7. As a result, the 2934 label can be finally identified. Note that if the number of outputs (number of characters = 2934) to be identified by voice recognition is changed, the number of parameters (number of dimensions) to be determined by learning also changes depending on the number.

(First embodiment)
FIG. 3 is a schematic diagram for explaining the network structure of the acoustic model used in the acoustic model learning means according to the first embodiment. Here, the three layers of deep learning means 111R of the same scale described with reference to FIG. 2 are generalized to N layers and expressed as deep learning means 111. The deep learning means 111 assumes that the number of layers is N (N ≧ 3). Further, the BLSTM of FIG. 2 will be described with reference to a pair of Fw-LSTM and Bw-LSTM. The number of dimensions of the deep learning means 111 depends on the memory cell C if the number of layers N is a constant value.

The acoustic model learning unit 100 (FIG. 1) of the Japanese acoustic model learning device 10 according to the first embodiment includes a memory cell C in the LSTM in the front (Fw) in the first layer of the deep learning unit 111 in FIG. The memory cell C is also set in the rear (Bw) LSTM.
Similarly, the memory cell C is set in the front (Fw) LSTM in the Nth layer of the deep learning means 111, and the memory cell C is also set in the rear (Bw) LSTM.
On the other hand, in the predetermined n-th layer excluding the first layer and the N-th layer of the deep learning means 111, the memory cell c (c <C) is set in the front (Fw) LSTM, and the rear (Bw The memory cell c (c <C) is also set in the LSTM.
Further, in the other layers other than the first layer, the nth layer, and the Nth layer, the memory cell C is set in the front (Fw) LSTM, and the memory cell C is also set in the rear (Bw) LSTM. Is set.

That is, among the N layer BLSTMs (Fw-LSTM and Bw-LSTM pairs) constituting the deep learning means 111, a memory cell c in a predetermined nth layer excluding the first layer and the Nth layer is denoted by reference numeral 301. As shown, it is set smaller than the memory cell C of the other layer.
Therefore, the dimension of the feature vector output from the n-th layer is reduced more than the dimension of the feature vector output from the other layers, and dimensional compression (bottleneck structure) of the network structure of the acoustic model is realized. Thereby, the dimension of the feature vector handled by the calculation by the deep learning means 111 can be compressed. In FIG. 3, the size of the memory cell is represented by the horizontal width of each block representing Fw-LSTM and Bw-LSTM.

FIG. 4 is a diagram illustrating an example of a network structure of an acoustic model used in the acoustic model learning unit according to the first embodiment.
As shown in FIG. 4, the acoustic model learning unit 100A according to the first embodiment includes, as an example, a deep learning unit 111A having a layer number N of 3, a linear mapping unit 112, and a normalizing unit 113. . In addition, the same code | symbol is attached | subjected to the same structure as the acoustic model learning means 100R shown in FIG. 2, and description is abbreviate | omitted.
The deep learning means 111A includes a first layer BLSTM 30a, a second layer BLSTM 30d, and a third layer BLSTM 30c.
The BLSTM 30a of the first layer and the BLSTM 30c of the final layer (third layer) both output feature vectors of 640 dimensions, and two memory cells in each layer have C = 320, respectively.
On the other hand, in the BLSTM 30d of the second layer, the dimension of the feature vector to be output is 320 dimensions, and each of the two memory cells is c = 160.

Further, the network structure of the acoustic model used in the acoustic model learning unit 100A is not limited to the BLSTM structure, but is also realized on the time axis for a realization using LSTM or a more general RNN having no LSTM structure. The present invention can be similarly applied as long as it can set the length of how far away data is taken into the calculation.
According to the Japanese acoustic model learning device 10 according to the first embodiment, the dimensional compression of the network structure of the acoustic model is realized by compressing the dimension of the feature vector handled by the computation of the deep learning means 111A. The number of parameters to be determined is reduced by learning the model.

(Second Embodiment)
FIG. 5 is a schematic diagram for explaining a linear conversion part in the network structure of the acoustic model used by the acoustic model learning means according to the second embodiment. Here, in the acoustic model learning unit 100R in FIG. 2, the feature vector input to the linear mapping unit 112 is assumed to be four-dimensional, and the vector representing the output character is assumed to be 100-dimensional. According to the acoustic model learning means 100R of FIG. 2, the linear mapping means 112 performs a 4 * 100 matrix with respect to the input four-dimensional vector (1 * 4 matrix) as shown in FIG. 5 (a). Are multiplied to output a 100-dimensional vector (1 * 100 matrix). In this case, the number of elements of the matrix multiplied by the input four-dimensional vector is 4 × 100 = 400. The number of elements in this matrix is a measure of the number of parameters (number of dimensions) to be determined by learning the acoustic model.

  The acoustic model learning means 100 (FIG. 1) of the Japanese acoustic model learning apparatus 10 according to the second embodiment compresses the dimension of the feature vector handled by the operation in the linear mapping means 112 of the acoustic model learning means 100R shown in FIG. By doing so, dimensional compression of the network structure of the acoustic model is realized. More specifically, according to the second embodiment, instead of multiplying the 4 * 100 matrix shown in FIG. 5 (a), as shown in FIG. 5 (b), the rank r = Two matrices obtained by matrix decomposition at 2, that is, a 4 * 2 matrix and a 2 * 100 matrix are sequentially multiplied. In this case, the total number of elements of the matrix is 4 × 2 + 2 × 100 = 208, and the number of parameters to be determined by learning of the acoustic model is significantly reduced compared to 400, which is the number of elements in the case of FIG. Is done.

By using the acoustic model learning unit 100R shown in FIG. 2, the dimensionality of the feature vector output from the deep learning unit 111R and the dimensionality of the vector output from the linear mapping unit 112 will be described more generally. Here, if the dimension number of the feature vector output from the BLSTM 30c as the final layer of the deep learning means 111R is D _L and the dimension number of the vector output from the linear mapping means 112 is D _A , the number of parameters in the linear mapping means 112 P _a is represented by the following formula (a). In Expression (a), the first term on the right side represents a linear transformation part (transformation matrix), and the second term on the right side represents a translation component (bias).

P _A = D _L × D _A + D _A Formula (a)

When matrix decomposition of such transformation matrices of the linear mapping means 112 in the low-rank r, the number of parameters P _r at this time is expressed by the following formula (b).

_{P r = D L × r +} r × D A + D A ... formula (b)

Here, when the low rank r satisfies the following expression (1), P _A > P _r , and the number of parameters (number of dimensions) can be reduced by matrix decomposition.

D _L × D _A > D _L × r + r × D _A Formula (1)

FIG. 6 is a diagram illustrating an example of a network structure of an acoustic model used by the acoustic model learning unit according to the second embodiment.
As shown in FIG. 6, the acoustic model learning unit 100B according to the second embodiment includes a deep learning unit 111R, a linear mapping unit 112B, and a normalizing unit 113. In addition, the same code | symbol is attached | subjected to the same structure as the acoustic model learning means 100R shown in FIG. 2, and description is abbreviate | omitted.
The linear mapping unit 112B includes a first linear mapping unit 40 and a second linear mapping unit 42.
The first linear mapping unit 40 multiplies the 640-dimensional feature vector input from the BLSTM 30c that is the final layer (third layer) of the deep learning unit 111R by a 640 * 320 matrix to obtain a 320-dimensional vector. Output.
The second linear mapping means 42 multiplies the 320-dimensional feature vector input from the first linear mapping means 40 by a 320 * 2934 matrix and outputs a 2934-dimensional vector. The vector output from the second linear mapping unit 42 is input to the normalizing unit 113.

This specific example will be described by comparing FIG. 6 with FIG.
In the case of the acoustic model learning unit 100R shown in FIG. 2, that is, when the linear mapping unit 112 does not perform matrix decomposition, when attention is paid to the matrix that the linear mapping unit 112 multiplies the input vector, the number of elements of the matrix is ,
640 × 2934 = 1,877,760.

On the other hand, in the case of the acoustic model learning unit 100B according to the second embodiment, that is, when the linear mapping unit 112B performs matrix decomposition, the total number of elements of each matrix subjected to matrix decomposition decreases. Specifically, the sum of the number of elements of the matrix that the first linear mapping unit 40 multiplies the input vector and the number of elements of the matrix that the second linear mapping unit 42 multiplies the input vector is:
640 × 320 + 320 × 2934 = 1,143,680.

Therefore, according to the Japanese acoustic model learning device 10 according to the second embodiment, the linear mapping unit 112B performs matrix decomposition and compresses the dimension of the feature vector handled by the calculation of the linear mapping unit 112B, thereby generating a network of acoustic models. Dimensional compression of the structure is realized, which greatly reduces the number of parameters to be determined by learning the acoustic model.
Moreover, the dimension of the vector which the 1st linear mapping means 40 with which the linear mapping means 112B is provided is compressed to 320 dimensions, and it is anticipated that generalization capability will increase.

(Third embodiment)
The network structure of the acoustic model used in the acoustic model learning unit according to the third embodiment is a network structure in which the first and second embodiments are combined. That is, by replacing the second layer of the deep learning means 111R shown in FIG. 2 with the BLSTM 30d, the deep learning means 111A having a bottleneck structure is provided, and the linear mapping means 112 shown in FIG. 2 is subjected to matrix decomposition. By replacing the linear mapping means 112B that can be used, dimensional compression of the network structure of the acoustic model is realized. FIG. 7 is a diagram showing an example of a network structure of an acoustic model used in acoustic model learning means according to the third embodiment.

As shown in FIG. 7, the acoustic model learning unit 100C according to the third embodiment includes, as an example, a deep layer learning unit 111A having a layer number N of 3, a linear mapping unit 112B, and a normalizing unit 113. . In FIG. 7, the same components as those described with reference to FIGS. 2, 4, and 6 are denoted by the same reference numerals, and further description thereof is omitted.
According to the Japanese acoustic model learning device 10 according to the third embodiment, the dimensional compression of the network structure of the acoustic model is performed by compressing the dimension of the feature vector handled by the operations of both the deep learning means 111A and the linear mapping means 112B. This reduces the number of parameters to be determined by learning the acoustic model.

  As mentioned above, although each embodiment of this invention was described, this invention is not limited to these, It can implement in the range which does not change the meaning. For example, in each of the embodiments described above as a Japanese acoustic model learning device, it can be regarded as a Japanese acoustic model learning program written in a general-purpose or special computer language so as to enable processing of the configuration of each device. Is also possible.

  In order to confirm the performance of the Japanese acoustic model learning device according to each embodiment, the speech recognition experiment results of each model learned for a plurality of network structures corresponding to each embodiment were compared. As the evaluation sound, the program sound (32k words = 32,000 words) for the June 2013 broadcast of the information program “Hiruma Ehot” on the general television was used. In each method, the learning data used a total of 120 dimensions, that is, 1023 hours of a pair of broadcast audio and subtitles, and an input feature amount of Filter bank 40 dimensions + delta + deltadelta. The language model used was a 200k vocabulary model learned from 600 million words such as NHK (registered trademark) manuscripts and subtitles of past programs. The network used for learning has the standard structure of FIG. 2 and the three structures of FIG. 4, FIG. 6, and FIG. The results are shown in Table 1.

According to Table 1, compared to the standard structure of FIG. 2, the word recognition error rate (WER) was improved and the learning time and the number of times of learning were remarkably shortened in any of the embodiments.
Specifically, in the second embodiment and the third embodiment in which the dimension is compressed in the linear mapping unit 112B, the WER is further improved, and the generalization ability is further increased. ing. Among these, in the model to which only the matrix decomposition of the Affine transformation is applied, that is, in the second embodiment, the WER is improved by 20.2% compared to the method using the standard structure of FIG. This is presumably because the generalization ability of the model was improved by compressing the dimensions once to the number of dimensions equivalent to the reading of kanji (= 320).

  In addition, in the technique for reducing the parameters of the BLSTM part, that is, in the first embodiment and the third embodiment in which the dimension is compressed in the deep learning means 111A, the effect of shortening the learning time is greater. Among these, in the model employing both the bottleneck structure and the matrix decomposition, that is, the third embodiment, the average learning time per learning is improved by 9.3% from the method using the standard structure of FIG. did. Since the BLSTM dimension reduced in each embodiment affects the time direction, it is considered that the learning time can be further shortened as compared with the matrix decomposition of the Affine transform.

DESCRIPTION OF SYMBOLS 1 Japanese speech recognition apparatus 10 Japanese acoustic model learning apparatus 100,100A, 100B, 110C Acoustic model learning means 101 Acoustic model storage means 111,111A, 111R Deep learning means 112,112B Linear mapping means 113 Normalization means 30a, 30b , 30c, 30d BLSTM
40 First linear mapping means 42 Second linear mapping means

Claims (4)

The input voice is converted into a character by using an end-to-end voice recognition method by learning the correspondence with the character output by the voice recognition of the input voice, and the character An acoustic model learning device for learning an acoustic model that outputs
It has a multi-layered neural network of three or more layers, and feature values of speech are continuously input, information on the time direction about the feature values is stored in each layer of the multi-layer structure, and information on the time direction is stored. A deep learning means for outputting a feature vector representing a probability of predicting which of a plurality of target characters from the feature amount of the speech,
Linear mapping means for converting the dimension of the feature vector output from the deep learning means by a predetermined calculation by applying a predetermined transformation matrix to the feature vector that is the output of the last layer of the deep learning means,
An acoustic model learning apparatus, wherein the acoustic model is learned by compressing a dimension of the feature vector handled by at least one of computations by the deep learning unit and the linear mapping unit. The acoustic model learning device according to claim 1,
In order to compress the dimension of the feature vector, the deep learning means
The number of dimensions of a vector for storing information in the time direction in a predetermined layer excluding the first layer and the last layer of the multilayer structure is the number of dimensions of a vector for storing information in the time direction in the first layer and the last layer. An acoustic model learning device that predicts a character from a feature amount of the input speech in a state where the character is set smaller than the above. In the acoustic model learning device according to claim 1 or 2,
The linear mapping means includes
The dimension number of the feature vector output from the last layer of the deep learning means is D _L , and the dimension number of the vector output from the linear mapping means is D _A.
Instead of applying the transformation matrix to the feature vector output from the last layer of the deep learning means, the transformation matrix is expressed by the following formula: D _L × D _A > D _L × r + r × D _A. 1)
An acoustic model learning device characterized in that the dimension of the feature vector is compressed by sequentially applying two matrices obtained by matrix decomposition with a rank r satisfying.   The acoustic model learning program for functioning a computer as the acoustic model learning apparatus as described in any one of Claims 1-3.

Images