CN1201284C - Rapid decoding method for voice identifying system - Google Patents

Abstract

The invention relates to a fast decoding method in a speech recognition system. The method comprises the following steps: (1) initializing the decoding operation unit in the speech recognition system; (2) taking out the feature of the next speech frame successively from the speech feature codeword sequence whose length is T in the input decoding operation unit Code word vector, set it as the speech frame O _t at time t, 1≤t≤T; (3) filter the speech frame O _t at time t; (4) based on the effective speech frame O _t ^V , set the dictionary tree at time t as Each active node in the token resource L _t [I] of each layer I of the card resource L _t is judged; (5) processing is in the token of the dictionary tree node; (6) according to the maximum probability of the local path at time t And the maximum probability of the local path at the moment  corresponding to the previous effective speech frame, and adaptively adjust the threshold related to pruning; (7) Repeat the above steps (2)-(6) to output the generated acoustic model at this moment The text string that has the best match with the language model produces a speech recognition result. Adopting this strategy can speed up the decoding operation compared with traditional methods.

Description

A Fast Decoding Method in Speech Recognition System

technical field

The invention relates to a fast decoding method in a speech recognition system.

Background technique

The decoding operation is the main component of the speech recognition system. Its function is: under the given acoustic model and language model, for the input acoustic observation feature vector sequence, let the computer automatically find out in the search space constructed statically or dynamically. The text string that has the best match with the acoustic model and the language model to convert the user's speech input into the corresponding text.

Shown in Fig. 1 is a kind of structural block diagram of known speech recognition system, analog speech is transformed into the digital signal that computer can process after analog-to-digital conversion unit 11, utilize feature extraction unit 12 to carry out frame processing to this digital signal then, usually frame The length is 20ms, and the frame shift is 10ms. Extract the MFCC parameters of each frame of speech to obtain the MFCC vector sequence. The decoding operation unit 14 adopts a certain search strategy according to the feature vector sequence of the input speech, the acoustic model 13 and the language model 15, such as Depth-first search (Viterbi algorithm) or breadth-first search to obtain the recognition results, in which the language model is used to apply the knowledge of the language layer to the speech recognition system to improve the recognition accuracy of the system when performing continuous speech recognition of large vocabulary.

The speech recognizer based on Fig. 1 has very high requirements to the central processing unit speed and memory capacity of the computer, and some current commercialized dictation machine systems, for example, the dictation machine module in the ViaVoice system of IBM and Microsoft Office XP all requires High-speed central processing unit (IntelPentium II above 400MHz) and large-capacity memory resources (above 100MByte). Generally speaking, the decoding operation occupies more than 90% of the CPU computing resources and almost all memory resources in the entire speech recognizer; the analog-to-digital conversion module and feature extraction unit occupy less than 10% of the CPU computing resources and very few memory resources.

The current commercial embedded speech recognition system mainly adopts small word volume specific person speech recognition based on simple template matching, for example, voice dialing and simple command recognition in mobile phones. Since this technology requires users to register voice data, its ease of use , The applicability is not strong; some non-specific embedded speech recognition systems are mainly for command word recognition with a small vocabulary, and the amount of calculation and memory requirements are still relatively large. For example, IBM's personal voice assistant speech recognition system is for The task domain requires a computing device with a computing capability of 50DMIPS.

The basic principles and concepts of known decoding operations are as follows:

1. Dictionary tree

The dictionary tree is a tree structure used to organize the pronunciation of all words in the recognition system. A phoneme is the basic unit that constitutes the pronunciation of a word. The TRIPHONE phoneme is a commonly used phoneme unit in the current speech recognition system. For example, the TRIPHONE phoneme representation sequence of the word "China" is: "sil-zh+ongzh-ong+g ong-g+uo g -uo+sil" (where "sil" is a special phoneme used to describe the pause in the user's voice), the TRIPHONE phoneme is a context-sensitive phoneme, which can describe phonemes in different contexts compared with the usual pinyin representation The pronunciation variation produced in the word can more accurately describe the acoustic characteristics of word pronunciation. Words in the recognition system may have the same prefix or subwords, for example: the words "middle" and "China", they have the same prefix "zhong", which can be described in a tree structure, assuming that the vocabulary of the recognition system contains the following The five words "abe", "ab", "acg", "acgi", and "ac" of the vocabulary, the dictionary tree of the vocabulary is shown in Figure 4:

The TRIPHONE phoneme corresponding to each node in the dictionary tree is associated with a Hidden Markov Model (HMM) corresponding to the TRIPHONE. Figure 5 shows a HMM topology representing the TRIPHONE phoneme. An HMM consists of several HMM states.

2. Token Definition and Token Expansion Strategy

A token refers to an active search path from the start frame of the user's speech to the current speech frame, which contains path identification information and the matching score of the path with the acoustic model and language model, where the path identification information includes all words in the path And word boundary information, each token corresponds to an active search path, the difference between different tokens is that they have different acoustic contexts and different language contexts.

Each state in the HMM associated with each node in the dictionary tree can hold movable tokens, and each state of the node has a token list, which is used to store all the tokens that are active in the state at any time . Assuming that at time t, the score of an extensible token in the token linked list of state i of a node in the dictionary tree is s _i (t-1), then in the search process, if the token’s score s _j (t-1) plus the transition probability from state i to state j, plus the observation probability of state i for the current speech frame t exceeding the current pruning threshold, a new token is generated with a score of s _j (t), and associated with state j. After completing the processing of all tokens residing in the dictionary tree at time t-1, token resources to be expanded that reside in the dictionary tree at time t will be generated, and all token resources residing in the dictionary tree at time t-1 will be deleted All tokens on the tree.

During token propagation, possible words and word boundary information are recorded in a linked list structure identifying paths. Therefore, at the end time T of speech input, the word sequence with the best match and the corresponding word boundary position can be extracted by going back to the path identification information list in the token with the best score.

3. The token resource definition of the dictionary tree node

Suppose the dictionary tree node contains M HMM states: s ₁ …s _M , then the token resource definition of a dictionary tree node includes the following token resource information:

Node token resources: H _S1 H _S2 … H _SM

Wherein, H _s1 (1≤i≤M) is the head of the token linked list about the HMM state S _i in the node.

The traditional decoding operation method requires too much hardware computing power and memory, and the cost performance is low.

In the Chinese patent application 02131086.6, a method for compressing the feature vector set of the speech recognition system is disclosed. The step of dynamically merging and splitting the sub-sets by the distance measure reduces the sum of the distance measures between the vectors in the clustered set and their corresponding codewords, improves the accuracy of the clustering algorithm, and applies the compressed codebook of the inventive method to In the speech recognition system, the memory capacity of the system can be greatly reduced while ensuring the recognition performance of the speech system. The invention also discloses a speech recognition system, the structural block diagram of which is shown in Figure 2, the system replaces the acoustic model with a feature codebook and a probability table, and does not need to calculate the Gaussian probability in the decoding process, only needs to use the pre-stored probability The required probability value can be found in the table, which greatly reduces the cost of calculating the Gaussian probability in the decoding operation, thus improving the recognition speed of the system to a considerable extent.

Contents of the invention

The purpose of the present invention is to provide a kind of fast decoding operation method in the continuous speech recognition system of improving current non-specific NPC vocabulary, and this method further solves current embedded speech recognition technology, the market acceptability of relative product price, to hardware calculation The problem of high capacity and memory requirements makes the current speech recognition technology also applicable to embedded hardware platforms, such as PDA, Mobil Phone, Smart Phone, etc.

The purpose of the present invention can be achieved through the following measures:

A fast decoding method in a speech recognition system, comprising the steps of:

(1) initialize the decoding operation unit in the speech recognition system;

(2) take out the characteristic codeword vector of next speech frame successively from the speech characteristic codeword sequence that the length in the input decoding operation unit is T, set it as t moment speech frame _Ot , 1≤t≤T;

(3) voice frame O _t is filtered at t moment, if this voice frame is filtered out, then go to step (2) to carry out, otherwise put this voice frame O _t as valid voice frame O _t ^V ;

(4) Based on the effective speech frame O _t ^V , judge each active node in the token resource L _t [I] of each layer I of the dictionary tree token resource L _t at time t, and judge whether it belongs to the scalable expand the token in the token resource table of the node, and link the newly generated token into the token resource table of the target node; where I is an index variable, 1≤I≤H; H is a dictionary tree height; otherwise, step (7);

(5) process the token in the dictionary tree node;

的局部路径最大概率，对与剪枝相关的阈值做自适应调整；(6) According to the maximum probability of the local path at time t and the time corresponding to the previous effective speech frame

The maximum probability of the local path of , and make adaptive adjustments to the thresholds related to pruning;

(7) Repeat the above steps (2)-(6) to obtain the global path with the best score token at the end time T of the input speech, end the token expansion, and output the best value of the acoustic model and the language model that have been generated at this moment. Matched text strings generate speech recognition results.

The present invention does not involve the expansion of word node tokens and related processing algorithms, and users can customize related processing algorithms according to task domains (for example: command word recognition, Chinese monosyllable recognition, large word volume continuous speech recognition, etc.).

The dictionary tree token resource L _t at time t is the sum of token resources of all active nodes in the dictionary tree at this time. The index method of the active node in the dictionary tree at time t is: according to the level index of the active node in the dictionary tree at time t, that is, all active nodes in the same layer are concatenated together to form a linked list, and each layer of the dictionary tree There is such a linked list, which is a two-dimensional linked list as a whole.

The first layer token resource L _t [I] of the dictionary tree token resource at time t is the first layer linked list of the dictionary tree active node token resource L _t indexed in the above manner.

The maximum probability of the local path at time t is: the maximum value of all local path scores in the local path set corresponding to all newly generated tokens at time t.

The previous valid speech frame corresponding to The maximum probability of the local path at a moment is: the moment corresponding to the previous valid speech frame In the local path set corresponding to all newly generated tokens, the maximum value of all local path scores.

Described initialization step (1) also comprises the following steps:

a. Generate a token with a score of zero, and link the token to the token resource header of the root node in the dictionary tree. The active node of the current dictionary tree only contains the root node root, which is at the root of the dictionary tree level one;

b. Initialize the global pruning threshold L _g as the logarithmic minimum;

c. Initialize the local pruning baseline threshold L _b as the logarithmic minimum;

d. Initialize the pruning width threshold L _w as a normal constant L _w ^c , and L _w ^c is preset by the user.

Described filtering step (3) also comprises the following steps:

3a, if voice frame 0 _t is the initial voice frame of user voice input at t moment, then put it as effective voice frame, and filter operation is finished; Otherwise execution step b;

3b, compare the Y feature code word vector f ₁ ^t f ₂ ^t Λ f _Y ^t of the speech frame O _t at time t with the Y feature code vector f 1 _{t-1 f of speech frame O t-1} at time ^t- _{1 2} ^t-1 Λ f _Y ^t-1 degree of similarity, get a similarity measure V;

3c. Compare the similarity measure value V with the decision threshold θ, if V ≤ θ, it is determined that the speech frame O _t at time t is an invalid speech frame for decoding operation; otherwise, it is determined that the speech frame O t at time _t is an effective speech frame for decoding operation .

The decision threshold θ is a constant greater than 0 set by the user.

The node token resource expansion step (4) also includes the following steps:

4a. Based on the effective voice frame O _t ^V , externally expand each token in the token resource linked list corresponding to the last state of the HMM associated with the current node, that is, the command corresponding to the last state of the HMM associated with the current node Each token in the card resource list is extended to the token resource tables of all child nodes of the node in the dictionary tree;

4b. Take one HMM state of the HMM with M states associated with the current node as the current pending HMM state S _n , where 1≤n≤M;

4c. Take a token in the token resource table corresponding to state s _n as the current token to be processed;

的全局剪枝阈值L_g，则根据当前节点关联的HMM模型的拓扑结构，取一个由状态s_n可达的状态，置为当前待处理状态s_m，否则转到步骤k开始执行；4d. If the score of the current token to be processed in the state S _n is greater than the time corresponding to the previous valid speech frame

The global pruning threshold L _g of the current node, then according to the topology structure of the HMM model associated with the current node, take a state that is reachable from the state s _n and set it as the current pending state s _m , otherwise go to step k to start execution;

4e. Calculate the score s _m (t) of the token from S _n to state s _m ; the score s _m (t) is the current score of the token plus the transition probability from state s _n to state s _m , plus Upper state s _m is for the observation probability of current speech frame O _t , and this observation probability can be done table look-up operation to obtain from the probability table of input decoding operation unit;

4f. Calculate the current local pruning threshold L _p , the calculation formula is: L _p =L _b -L _w , in the formula, L _b is the current local pruning baseline threshold; L _w is the current pruning width threshold;

4g. If the score of the token from s _n to the state s _m is greater than the current local pruning threshold L _p , generate a new token and set its score to s _m (t); otherwise, execute step j;

4h. Link the new token generated in step g into the token resource table whose head is H _sm in the node, and check whether the node is already in the active node table of the layer where the node is located in the dictionary tree. If not, then chain into it;

4i. According to the score s _m (t) of the new token, if s _m (t)-L _w > L _b holds true, then update the local pruning baseline threshold L _b to L _b = s _m (t);

4i. Take another state reachable from state s _n , set it as the current pending state s _m , repeat the above step ei until all states reachable from state s _n are processed; go to step k to execute;

4k. Take another token in the token resource table corresponding to the state s _n as the current token to be processed; repeat the above dj steps until the expansion operations on all tokens in the token resource table corresponding to the state s _n are Complete; go to step 1 to execute;

4l. Take another HMM state of the HMM with M states associated with the node as the current HMM state S _n to be processed, where 1≤n≤M, repeat the above step ck until all token resource expansion operations of the current node are Finish.

In the described node token resource expansion step (4) a step, comprise the following steps:

的全局剪枝阈值L_g，则不需要作扩展当前令牌到其所在节点的所有子节点的操作，否则执行步骤ii；4a-i If the current score of the token is less than or equal to the time corresponding to the previous valid speech frame

global pruning threshold L _g , it is not necessary to extend the current token to all child nodes of the node where it is located, otherwise perform step ii;

4a-ii Take the jth child node node _j among the J child nodes of the node where the current token is located in the dictionary tree as the current node to be processed;

4a-iii Accumulate the score s ₁ (t) of the first state s ₁ when the token reaches node _j . The score s ₁ (t) is the current score of the token plus the last value of the node where the current token is located. The transition probability of a state to the first state of node _j , plus the observation probability of the first state _s1 of node _j for the current speech frame O _t , the observation probability can be checked from the probability table of the input decoding operation unit Table operation gets;

4a-iv Calculate the current local pruning threshold L _p , the calculation formula is: L _p =L _b -L _w , where L _b is the current local pruning baseline threshold; L _w is the current pruning width threshold;

4a-v If the token reaches the first state s ₁ of node _j and the score is greater than the current local pruning threshold L _p , then execute step vi; otherwise, execute step ix;

4a-vi generates a new token and sets its score to s ₁ (t);

4a-vii Link the token into the token resource table whose header is H _s1 in the node _j node, and check whether node _j is already in the active node table of the node at the layer where the dictionary tree is located, if not, link into it;

4a-viii Based on the score s ₁ (t), if s ₁ (t)-L _w >L _b , update the local pruning baseline threshold L _b ie L _b =s ₁ (t);

4a-ix Take another child node of the node where the current token is located in the dictionary tree as the current node _j to be processed, repeat the above steps iii-viii, until the extension of the current token to all child nodes of the node where it is located in the dictionary tree The operation is complete.

The described adaptive pruning step (6) based on the maximum probability of the local path comprises the following steps:

其中， 为到t时刻为止的所有有效语音帧的个数，当前的全局剪枝阈值L_g为前一有效语音帧对应的时刻 的局部路径最大概率，当前的局部剪枝基线阈值L_b为t时刻的局部路径最大概率；6a. According to the current time t and the previous effective speech frame corresponding The local maximum probability at the moment, calculate the pruning width threshold adjustment factor L _f as:

in, is the number of all valid speech frames up to time t, and the current global pruning threshold L _g is the time corresponding to the previous valid speech frame The maximum probability of the local path of , the current local pruning baseline threshold L _b is the maximum probability of the local path at time t;

6b. Regularize the calculated pruning width threshold adjustment factor L _f : if $L_f>L_f^{\mathrm{MAX}}$ Then set L _f to L _f ^MAX , if $L_f<L_f^{\mathrm{MIN}},$ Then set L _f as L _f ^MIN , where: L _f ^MAX is the upper bound of the adjustment factor L _f , and L _f ^MIN is the lower bound of the adjustment factor L _f , both of which are normal numbers and can be set by the user;

6c. According to the calculated pruning width threshold adjustment factor L _f , update the pruning width threshold L _w as: $L_w=L_f{L}_w^c,$ L _w ^c is the initial pruning width threshold, which can be obtained in the initialization step (1);

6d. The global pruning threshold at update time t is L _g : L _g =L _b , to prepare for token expansion for the next valid speech frame;

6e. Reset the local pruning baseline threshold L _b to a logarithmic minimum value to prepare for token expansion for the next valid speech frame.

Compared with the prior art, the present invention has the following advantages:

Compared with the traditional decoding operation method, the present invention includes the following improvements: an adaptive pruning strategy based on the maximum probability of the local path; a speech frame filtering strategy based on the feature code word vector.

The decoding operation involved in the present invention adopts a width-first search frame with pruning based on dictionary tree and token expansion, and the computational complexity of this algorithm is O(MT), wherein: T is the voice frame that enters the search calculation M is the average value of the number of active paths at the corresponding time of each speech frame in the search calculation process.

The traditional decoding operation is to search and calculate all the voice frames of the user's voice input. In fact, the user's voice input is a time-varying signal with local stationarity. Therefore, in the stable segment of the user's voice input, some Speech frames that are similar to adjacent speech frames, that is, these speech frames do not enter the search calculation process, and do not affect the accuracy of the decoding operation. For this reason, the present invention provides a kind of speech frame filtering strategy based on speech frame feature codeword vector, this strategy can effectively remove the speech frame that is redundant to search computation, makes the speech frame number that actually enters search computation be less than user speech The actual number of speech frames included in the input can be seen from the above formula. Compared with the traditional method, using this strategy can speed up the decoding operation.

On the other hand, it can be seen from the above formula that the speed of search calculation also depends on the average number of active paths at the corresponding time of each speech frame in the search process, that is: when T is constant, the larger M is, the greater the search cost ; The smaller M is, the smaller the search overhead. The size of M depends on the pruning strategy adopted by the search calculation. For this reason, the present invention provides an adaptive pruning strategy with the maximum probability of local paths. Compared with the traditional method, the value of M can be effectively reduced without significantly affecting the recognition accuracy, so that the speed of decoding operation can be further accelerated .

Description of drawings

Fig. 1 is the structural block diagram of known speech recognition system

Fig. 2 is the structural block diagram of another known speech recognition system

Fig. 3 is the decoding operation flowchart of the present invention

Fig. 4 is a known dictionary tree structure diagram

Fig. 5 is a schematic diagram of the HMM topology structure of the known TRIPHONE phoneme

specific implementation

Fig. 3 shows the flow chart of the decoding operation of the present invention. By Fig. 2, in conjunction with Fig. 3, the operation process of the speech recognition system based on the decoding operation subsystem of the present invention is: the speech analog signal of input is transformed into digital signal; This digital signal is carried out sub-frame processing, extracts each frame of speech The feature parameters of each speech frame correspond to a feature vector to obtain the feature vector sequence of the input speech; use the feature codebook to quantize and encode the feature vector sequence, and each speech frame corresponds to a feature codeword vector to obtain the corresponding feature Codeword vector sequence; the speech feature codeword vector sequence is input into the speech frame filtering unit of the decoding operation subsystem, and the speech frame filtering operation is performed, and the feature codeword vector corresponding to the invalid speech frame is removed from the speech feature codeword vector sequence to obtain an effective Speech feature codeword vector sequence; search and calculate the recognition result to the effective speech feature codeword vector sequence, in the search calculation process, adopt the adaptive pruning strategy based on the local path maximum probability to do local search path pruning, to For each codeword of the effective speech feature codeword vector sequence, its observation probability on the (local) search path can be directly found from the probability table.

The definition of the dictionary tree active node token resource and its indexing method

In the present invention, at any time t, a node with an active token in the dictionary tree is called an active node at time t. The index method of the active node in the dictionary tree at time t is: according to the level index of the active node in the dictionary tree at time t, that is, all active nodes in the same layer are concatenated together to form a linked list, and each layer of the dictionary tree There is such a linked list, which is a two-dimensional linked list as a whole.

At any time t, the sum of the token resources of all active nodes in the dictionary tree is called the token resource of the active node of the dictionary tree at time t, which specifies the token resources to be expanded at time t. For the convenience of the following description, the dictionary tree active node token resource at time t according to the above index is L _t , and its index variable is I (1≤I≤H), where H is the height of the dictionary tree.

Based on the above basic principles of decoding operations and related concepts, a specific implementation manner of the fast decoding method of the present invention is given.

Decoding method of the present invention comprises the following steps:

1. Initialize the decoding operation unit in the speech recognition system;

2, take out the characteristic codeword vector of first speech frame from the speech characteristic codeword vector sequence that the length of input decoding operator is T, put it as t moment speech frame O _t (t=1);

3, the voice frame O _t of time t is filtered;

4. If O _t is an effective speech frame, then to each active node of the dictionary tree token resource L _t [I] of each layer I (1≤I≤H) of _the dictionary tree token resource L t at time t, expand The token in the token resource table of the node, and link the newly generated token into the token resource table of the target node; otherwise, go to step 7;

5, process the token that is in word node; The present invention does not relate to the expansion and relevant processing algorithm about word node token, the user can according to task field (for example: command word recognition, Chinese monosyllable recognition, large word volume continuous speech recognition etc.) to customize relevant processing algorithms;

6. According to the maximum probability of the local path at time t and the time corresponding to the previous valid speech frame The maximum probability of the local path, and adaptively adjust the thresholds related to pruning, including: global pruning threshold L _g , local pruning baseline threshold L _b and pruning width threshold L _w ;

7, take out next speech frame from the speech feature vector sequence that the length of input decoding operator is T, if can get, then put it as t moment speech frame O _t (t≤T) and go to step 3 and carry out, Otherwise go to step 8;

8. End token expansion to generate recognition results: output the text string that best matches the acoustic model and language model through the global path of the token with the best score at time T.

The sub-steps for extending the current node token resource in step 4 of the above decoding method are as follows:

4a. For each token in the token resource linked list corresponding to the last state of the HMM associated with the node, extend the token to the token resource tables of all child nodes of the node in the dictionary tree;

4b. Take the first HMM state of the HMM with M states associated with the node as the current pending HMM state S _n (n=1);

4c. Take a token in the token resource table corresponding to state s _n as the current token to be processed;

的全局剪枝阈值L_g，则根据当前节点关联的HMM模型的拓扑结构，取一个由状态s_n可达的状态，置为当前待处理状态s_m，否则转到步骤k执行；4d. If the current score of the token is greater than the time corresponding to the previous valid speech frame

The global pruning threshold L _g of the current node, according to the topology structure of the HMM model associated with the current node, take a state reachable from the state s _n and set it as the current pending state s _m , otherwise go to step k for execution;

4e. Calculating the score s _m (t) of the token reaching the state s _m is: the current score of the token plus the transition probability from the state s _n to the state s _m , plus the state s _m for the current speech frame O _t Observation probability, the observation probability can be obtained from the probability table of the input decoding operation unit by a table look-up operation;

4f. Calculate the current local pruning threshold L _p , the calculation formula is: L _p =L _b -L _w , in the formula, L _b is the current local pruning baseline threshold; L _w is the current pruning width threshold;

4g. If the score of the token reaching the state s _m is less than or equal to the current local pruning threshold L _p , go to step j for execution; otherwise, do the following: generate a new token and set its score to s _m ( t);

4h. Link the token into the token resource table whose head is H _sm in the node, and check whether the node is already in the active node table of the layer where the node is located in the dictionary tree, if not, link into it;

4i. According to the score s _m (t), update the local pruning baseline threshold L _b , the steps are: if s _m (t)-L _w >L _b , then: L _b =s _m (t); Otherwise, do not update;

4j. Take another state s _m that is reachable by state s _n , if it is obtained, set it as the current pending state s _m and go to step e for execution, otherwise go to step k for execution;

4k. Take another token in the token resource table corresponding to the state s _n , if it is obtained, set it as the current token to be processed and go to step d for execution, otherwise go to step l for execution;

41. Take the next HMM state of the HMM with M states associated with the node. If it is obtained, set it as the current HMM state S _n (n≤M) to be processed and go to step c for execution, otherwise it indicates that the current node The token resource extension operation has completed.

Step a in the first sub-step of step 4 of the above-mentioned decoding method is about extending the current token to all child nodes of the node where it is located:

4a-1. If the score of the current token is less than or equal to the time corresponding to the previous valid speech frame The global pruning threshold L _g , there is no need to extend the current token to all the child nodes of the node where it is located, otherwise go to step 2;

4a-2. Take a child node of the node where the current token is located as the current pending node node _j (j=1);

4a-3. The score s ₁ (t) of the cumulative token reaching the first state s ₁ of node _j is: the current score of the token plus the last state of the node where the token is located to the first state of node _j The transition probability of each state, plus the observation probability of the first state _s1 of node _j for the current speech frame O _t , this observation probability can be done table look-up operation from the probability table of input decoding operation unit and obtains;

4a-4. Calculate the current local pruning threshold L _p , the calculation formula is: L _p =L _b -L _w , where L _b is the current local pruning baseline threshold; L _w is the current pruning width threshold;

4a-5. If the token reaches the first state s ₁ of node _j and the score is less than or equal to the current local pruning threshold L _p , go to step 9, otherwise go to step 6;

4a-6. Generate a new token and set its score to s ₁ (t);

4a-7. Link the token into the token resource table whose head is H _s1 in node _j , and check whether node _j is already in the active node table of the node at the layer where the dictionary tree is located. If not, then chain into it;

4a-8. According to the score s ₁ (t), update the local pruning baseline threshold L _b , the steps are: if s ₁ (t)-L _w >L _b , then: L _b =s ₁ (t) , otherwise, do not update;

4a-9. Take another child node of the node where the current token is located. If it is obtained, set it as the current pending node node _j (j≤N) and go to step 3 for execution, where N is the node where the current token is located. The number of all child nodes in the dictionary tree, otherwise it indicates that the operation of extending the current token to all child nodes of the node where it is located is completed.

Step 1 of the above-mentioned decoding method initializes the decoding operation unit and includes the following steps:

a. Generate a token with a score of zero, and link the token to the token resource header of the root node in the dictionary tree. The active node of the current dictionary tree only contains the root node root, which is at the root of the dictionary tree level one;

d. Initialize the global pruning threshold L _g as the logarithmic minimum;

c. Initialize the local pruning baseline threshold L _b as the logarithmic minimum;

b. Initialize the pruning width threshold L _w as a normal constant L _w ^c , which can be read from the configuration file of the decoding operator set by the user.

The step 3 of above-mentioned decoding method is done the step of filter operation to current speech frame and comprises the following steps:

3a, if current t moment voice frame O _t is the initial voice frame of user's voice input, then put it as effective voice frame, and filter operation is finished; Otherwise execution step 2;

3b. Compare the feature codeword vector f ₁ ^t f ₂ ^t ... f _Y ^t of the speech frame O _t at the current moment t with the feature codeword vector f 1 t-1 f _{2 t-1 of the speech frame Q t-1} ^at the moment ^t - ₁ The degree of similarity of ¹ ... f _Y ^t-1 obtains a similarity measure value V, and Y in the above-mentioned feature codeword vector expression is the number of feature codewords contained in the speech frame feature codeword vector, and the similarity measure value V can be obtained by the following The above formula is calculated to get:

$V=\underset{i=1}{\overset{Y}{\mathrm{\&Sigma;}}}C(f_i^t,f_i^{t-1}),$ in, is defined as the following formula:

3c. Compare the similarity measure value V with the decision threshold θ (a normal number set by the user, which can be read from the configuration file of the decoding operator set by the user), and if V≤θ, determine the speech frame O _t is an invalid speech frame for decoding operation, otherwise it is determined that speech frame O _t is an effective speech frame for decoding operation.

Experimental results show that the above speech frame filtering operation can remove 20% to 30% of invalid speech frames in the speech feature codeword vector sequence, so the speed of decoding operation can be accelerated, and its recognition performance is better than that of traditional methods. Insensitive, because the above speech frame filtering operation can remove more invalid speech frames for users who speak slowly, and less invalid speech frames for users who speak faster Frame, that is, the above-mentioned voice frame filtering operation can regularize the speech rates of different users of users to a certain extent.

Step 6 of the above-mentioned decoding method also includes the following steps:

的局部路径最大概率，当前的局部剪枝基线阈值L_b为t时刻的局部路径最大概率。据此，解码运算器的操作步骤中的步骤6的执行分步骤：From the first five steps of the decoding operator, it can be obtained that the current global pruning threshold L _g is the moment corresponding to the previous effective speech frame

The maximum probability of the local path of , the current local pruning baseline threshold L _b is the maximum probability of the local path at time t. Accordingly, the execution of step 6 in the operation steps of the decoding operator is divided into steps:

其中， 为到t时刻为止，所有有效语音帧的个数；6a. According to time t, The local maximum probability of , calculate the pruning width threshold adjustment factor L _f as:

in, For the time t, the number of all valid speech frames;

6b. Regularize the calculated pruning width threshold adjustment factor L _f : if $L_f>L_f^{\mathrm{MAX}}$ but $L_f=L_f^{\mathrm{MAX}},$ like $L_f<L_f^{\mathrm{MIN}}$ but $L_f=L_f^{\mathrm{MIN}},$ Among them: L _f ^MAX is the upper bound of the adjustment factor L _f (for example: 1.05), L _f ^MIN is the lower bound of the adjustment factor L _f (for example: 0.5), both are normal numbers, which can be obtained from the decoder configuration file set by the user read in;

6c. According to the calculated pruning width threshold adjustment factor L _f , update the pruning width threshold L _w as: $L_w=L_f{L}_w^c;$

6d. The global pruning threshold at the update time t is L _g : L _g =L _b , which is to prepare for the token expansion of the next valid speech frame;

6e. Reset the local pruning baseline threshold L _b to a logarithmic minimum value to prepare for token expansion for the next valid speech frame.

In the traditional search algorithm, the pruning width threshold L _w is constant, but in the present invention, after performing search calculation on the current effective speech frame, the pruning width threshold L _w can be adaptively adjusted according to the maximum probability of the local path , so that the adaptive pruning of the local path can be realized when the next effective speech frame is searched and calculated. The experimental results show that, under the premise of not affecting the recognition accuracy, this method can effectively reduce the average token in the decoding process. The number M (10%-20%) can further speed up the speed of decoding operation.

Claims (10)

1. A fast decoding method in a speech recognition system, comprising the steps of: (1) initialize the decoding operation unit in the speech recognition system; (2) take out the characteristic codeword vector of next speech frame successively from the speech characteristic codeword sequence that the length in the input decoding operation unit is T, set it as t moment speech frame _Ot , 1≤t≤T; (3) filter the voice frame O _t at t moment, if this voice frame is filtered out, then perform step (2), otherwise put this voice frame as the current effective voice frame O _t ^V ; (4) Based on the effective speech frame O _t ^V , judge each active node in the token resource L _t [I] of each layer I of the dictionary tree token resource L _t at time t, and judge whether it belongs to the scalable expand the token in the token resource table of the node, and link the newly generated token into the token resource table of the target node; where I is an index variable, 1≤I≤H; H is a dictionary tree height; otherwise, step (7); (5) process the token in the dictionary tree node; (6) According to the maximum probability of the local path at time t and the time corresponding to the previous effective speech frame The maximum probability of the local path of , and make adaptive adjustments to the thresholds related to pruning; (7) Repeat the above steps (2)-(6) to obtain the global path with the best score token at the end time T of the input speech, end the token expansion, and output the best value of the acoustic model and the language model that have been generated at this moment. Matched text strings generate speech recognition results. 2, the fast decoding method in the speech recognition system as claimed in claim 1 is characterized in that described described t moment dictionary tree token resource L _t is the summation of the token resource of all active nodes in this moment dictionary tree . 3. The fast decoding method in the speech recognition system according to claim 1, wherein the maximum probability of the local path at the time t is that in the set of local paths corresponding to all newly generated tokens at the time t, all local path scores the maximum value. 4. The fast decoding method in the speech recognition system as claimed in claim 1, characterized in that the previous effective speech frame corresponds to

The maximum probability of the local path at a moment is the moment corresponding to the previous valid speech frame

In the local path set corresponding to all newly generated tokens, the maximum value of all local path scores. 5, the fast decoding method in the speech recognition system as claimed in claim 1 is characterized in that described initialization step (1) also comprises the following steps: a. Generate a token with a score of zero, and link the token to the token resource header of the root node in the dictionary tree. The active node of the current dictionary tree only contains the root node root, which is at the root of the dictionary tree level one; b. Initialize the global pruning threshold L _g as the logarithmic minimum; c. Initialize the local pruning baseline threshold L _b as the logarithmic minimum; d. Initialize the pruning width threshold L _w as a normal constant L _w ^c , and L _w ^c is preset by the user. 6, the fast decoding method in the speech recognition system as claimed in claim 1 is characterized in that described filtering step (3) also comprises the following steps: 3a, if voice frame 0 _t is the initial voice frame of user voice input at t moment, then put it as effective voice frame, and filter operation is finished; Otherwise execution step b; 3b. Compare the Y feature codeword vectors f ₁ ^t f ₂ ^t Λ f _Y t of the voice frame _O ^t at time t with the Y feature code vector f ₁ t-1 f of the voice frame O _t -1 at time ^t-1 ₂ ^t-1 Λ f _Y ^t-1 degree of similarity, get a similarity measure V; 3c. Compare the similarity measure value V with the decision threshold θ, if V ≤ θ, it is determined that the speech frame O _t at time t is an invalid speech frame for decoding operation: otherwise, it is determined that the speech frame O t at time _t is an effective speech frame for decoding operation . 7. The fast decoding method in the speech recognition system according to claim 1, characterized in that said decision threshold θ is a constant greater than 0 set by the user. 8, the fast decoding method in the speech recognition system as claimed in claim 1 is characterized in that described node token resource expansion step (4), also comprises the following steps: 4a. Based on the effective voice frame O _t ^V , externally expand each token in the token resource linked list corresponding to the last state of the HMM associated with the current node, that is, the command corresponding to the last state of the HMM associated with the current node Each token in the card resource list is extended to the token resource tables of all child nodes of the node in the dictionary tree; 4b. Take one HMM state of the HMM with M states associated with the current node as the current pending HMM state S _n , where 1≤n≤M; 4c. Take a token in the token resource table corresponding to state s _n as the current token to be processed; 4d. If the score of the current token to be processed in the state S _n is greater than the time corresponding to the previous valid speech frame The global pruning threshold L _g of the current node, then according to the topology structure of the HMM model associated with the current node, take a state that is reachable from the state s _n and set it as the current pending state s _m , otherwise go to step k to start execution; 4e. Calculate the score s _m (t) of the token from S _n to state s _m ; the score s _m (t) is the current score of the token plus the transition probability from state s _n to state s _m , plus The observation probability of the upper state s _m for the current speech frame O _t ; 4f. Calculate the current local pruning threshold L _p , the calculation formula is: L _p =L _b -L _w , in the formula, L _b is the current local pruning baseline threshold; L _w is the current pruning width threshold; 4g. If the token’s score from S _n to state s _m is greater than the current local pruning threshold L _p , generate a new token and set its score to s _m (t); otherwise, execute step j; 4h. Link the new token generated in step g into the token resource table whose head is H _sm in the node, and check whether the node is already in the active node table of the layer where the node is located in the dictionary tree. If not, then chain into it; 4i. According to the score s _m (t) of the new token, according to the formula s _m (t)-L _w >L _b , then update the local pruning baseline threshold L _b ie L _b =s _m (t); 4j. Take another state reachable from state s _n , set it as the current pending state s _m , repeat the above step ei until all states reachable from state s _n are processed; go to step k to execute; 4k. Take another token in the token resource table corresponding to the state s _n as the current token to be processed; repeat the above dj steps until the expansion operations on all tokens in the token resource table corresponding to the state s _n are Complete; go to step 1 to execute; 4l. Take another HMM state of the HMM with M states associated with the node as the current HMM state S _n to be processed, where 1≤n≤M, repeat the above step ck until all token resource expansion operations of the current node have been completed Finish. 9, the fast decoding method in the speech recognition system as claimed in claim 8 is characterized in that in described node token resource expansion step (4) a step, comprises the following steps: 4a-i) If the current score of the token is less than or equal to the time corresponding to the previous valid speech frame global pruning threshold L _g , it is not necessary to extend the current token to all child nodes of the node where it is located, otherwise perform step ii; 4a-ii) take the jth child node among the J child nodes of the node where the current token is located as the current pending node node _j ; 4a-iii) Accumulate the score s ₁ (t) of the first state s ₁ when the token reaches node _j , the score s ₁ (t) is the current score of the token plus the current score of the node where the token is located The transition probability from the last state to the first state of node _j , plus the observation probability of the first state s ₁ of node _j for the current speech frame O _t ; 4a-iv) Calculate the current local pruning threshold L _p , the calculation formula is: L _p =L _b -L _w , where L _b is the current local pruning baseline threshold; L _w is the current pruning width threshold; 4a-v) If the token reaches the first state s ₁ of node _j and the score is greater than the current local pruning threshold L _p , then execute step vi; otherwise, execute step ix; 4a-vi) Generate a new token and set its score to s ₁ (t); 4a-vii) Link the token into the token resource table whose head is H _s1 in the node _j node, and check whether node _j is already in the active node table of the node at the layer where the dictionary tree is located, if not, then chain into it; 4a-viii) According to the score s ₁ (t), if s ₁ (t)-L _w >L _b , update the local pruning baseline threshold L _b ie L _b =s ₁ (t); 4a-ix) Take another child node of the node where the current token is located in the dictionary tree as the current node _j to be processed, repeat the above steps iii-viii, until the current token reaches all the child nodes of the node where it is located in the dictionary tree The extension operation is complete. 10. The fast decoding method in the speech recognition system as claimed in claim 1, characterized in that the adaptive pruning step (6) based on the maximum probability of the local path comprises the following steps: 6a. According to the current time t and the previous effective speech frame corresponding

The local maximum probability at time, the calculation pruning width threshold adjustment factor L _f is: $L_f=\frac{(L_b-L_g)\tilde{t}}{L_b},$ in,

is the number of all valid speech frames up to time t, and the current global pruning threshold L _g is the time corresponding to the previous valid speech frame The maximum probability of the local path of , the current local pruning baseline threshold L _b is the maximum probability of the local path at time t; 6b. Regularize the calculated pruning width threshold adjustment factor L _f : if $L_f>L_f^{\mathrm{MAX}}$ Then set L _f to L _f ^MAX , if $L_f<L_f^{\mathrm{MIN}},$ Then set L _f to L _f ^MIN , where: L _f ^MAX is the upper bound of the adjustment factor L _f , and L _f ^MIN is the lower bound of the adjustment factor L _f , both of which are normal numbers, which can be obtained from the decoder configuration file set by the user read; 6c. According to the calculated pruning width threshold adjustment factor L _f , update the pruning width threshold L _w as: $L_w=L_f{L}_w^c,$ L _w ^c is the initialization pruning width threshold obtained in the initialization step (1); 6d. The global pruning threshold at update time t is L _g : L _g =L _b , which is to prepare for token expansion for the next valid speech frame; 6e. Reset the local pruning baseline threshold L _b to a logarithmic minimum value to prepare for token expansion for the next valid speech frame.

Images