CN1326112C - Voice recognition device and integrated circuit realizing method - Google Patents

Abstract

The invention provides a speech recognition device, comprising an embedded processor, a dynamic time correction algorithm module, the dynamic time correction algorithm module is connected with the embedded processor through a control bus, and the embedded processor is respectively connected with a program memory, a data The memory and the display are connected to the microphone through the analog/digital conversion module at the same time, the embedded processor is provided with keys and RS232 interface, and the dynamic time correction algorithm module includes a calculation control module, a processing unit array, and a template buffer. The present invention generates "movement direction" through the calculation control module, and inputs the reference template feature and the test template feature parameter into the processing unit array from two opposite directions, so that each processing unit can still be correct when the length of the reference template and the test template change. The matching process of the two templates is performed, and at the same time, the complexity of hardware resources and input interface control logic is greatly reduced.

Description

A voice recognition device and its integrated circuit implementation method

technical field

The invention relates to the field of speech recognition and integrated circuit design, in particular to a speech recognition device and an integrated circuit implementation method thereof.

Background technique

At present, there are generally two types of implementation methods for speech recognition algorithms: software implementation and integrated circuit implementation. Software implementation of speech recognition algorithm refers to the implementation of programming on a general-purpose computer or using embedded software. Among them, the available resources for programming on a general-purpose computer are sufficient, and the algorithm can be optimized and upgraded flexibly. The disadvantage is that the computer is inconvenient to carry and cannot be carried with you. Use, poor mobility; using embedded software, its disadvantages are limited resources, and there is also a bottleneck problem in processor calculation speed. Integrated circuit implementation of voice recognition algorithm refers to the implementation of logic circuits, which are characterized by fast and real-time operation, small size and light weight of the recognition device, which is convenient for mobile use, and can also be used as a component module to be loaded into other intelligent devices that require human-computer interaction. , such as a voice dialing cell phone/telephone.

The Dynamic Time Warping (DTW) algorithm is one of the commonly used speech pattern recognition algorithms, which is mainly used in small and medium vocabulary speech recognition.

There are usually two methods for implementing integrated circuits of speech recognition algorithms:

(1) Microcontroller plus a processing unit structure. This structure uses a microcontroller to control the data input, calculation, and output of the processing unit, making full use of the advantages of the hardware processing unit in computing and the advantages of the microcontroller in control. Compared with the pure software implementation using a general-purpose processor The speech recognition method has greatly improved the recognition speed, but due to the limitation of the processing speed of the microcontroller, this implementation structure is still difficult to meet the real-time requirements.

(2) Processing unit array structure. The processing speed of this structure is orders of magnitude faster than non-processing array implementations. According to the classification of processor arrays, the processing unit arrays currently used to implement the DTW algorithm are almost all cardiac arrays. Cardiac arrays are usually only applicable to the problem of determining the size of the calculation. The current implementation structure of cardiac arrays adopts the method of fixing the number of frames of the reference template and the test template, and the number of frames of the reference template and the number of frames of the test template must be equal, but usually Under certain circumstances, the pronunciation duration of each character and word is different, even the same person speaks the same word at different times. The amount of calculation and complexity, more importantly, increases the distortion of speech feature expression, which is not conducive to subsequent recognition. Therefore, the versatility of speech recognition using a processing unit array structure is relatively poor.

Embedding the speech recognition algorithm on the integrated circuit chip not only brings convenience in application, but also greatly expands the application field of speech recognition. Among the existing speech recognition devices in the world, the typical speech recognition chips used include Sensory's RSC speech recognition chip and Philips' Hello IC speech recognition chip. Among them, the RSC speech recognition chip can recognize 16 pre-set words. If an external SRAM (data memory) is used and a specific speaker is used, the vocabulary can be expanded to about 100 after training. The shortcoming of these speech recognition chips is: all realize based on 8-bit MCU (Micro Controller Unit, single-chip microcomputer) plus a speech processing unit, the number of recognizable commands is very limited, it is difficult to meet the requirements of most practical applications.

Domestic integrated circuit design has only developed greatly in recent years, and it is still relatively lagging behind. There are few reports on the design of special speech recognition chips, and there are no reports on special speech recognition chips.

Contents of the invention

The purpose of the present invention is to overcome the shortcomings and problems of the existing speech recognition technology implementation methods, and provide a speech recognition device with strong versatility and practicability, which can be configured with parameters and adopts a dynamic time normalization algorithm.

The object of the present invention is also to provide an integrated circuit implementation method of the above dynamic time correction algorithm.

The object of the present invention is achieved through the following technical solutions: the voice recognition device includes an embedded processor, a dynamic time correction algorithm module, the dynamic time correction algorithm module is connected with the embedded processor through a control bus, and the embedded The embedded processor is respectively connected with a program memory, a data memory, and a display, and is connected with a microphone through an analog/digital conversion module. The embedded processor is provided with buttons and an RS232 interface, and the dynamic time correction algorithm module includes a calculation control module. , processing unit array, template buffer, and set with system clock port, reset signal port, control signal port, reference template valid port, test module valid port, feature data port, system busy flag port, output data valid port, matching result output port, the processing unit array is a one-dimensional linear array, each processing unit is connected in turn, the system clock port, the reset signal port are respectively connected to each processing unit, and the last processing unit is connected to a matching result output port, so The calculation control module and the first and last processing unit and the template buffer, system clock port, reset signal port, control signal port, reference template valid port, test module valid port, system busy flag port, output data valid The ports are respectively connected, and the template buffer is respectively connected to the first and last processing unit, the system clock port, the reset signal port, and the characteristic data port.

The stencil buffer includes a test stencil buffer and a reference stencil buffer.

The control bus is respectively connected with the system clock port, the reset signal port, the control signal port, the effective port of the reference template, the effective port of the test module, the characteristic data port, the system busy flag port, the effective port of the output data, and the output port of the matching result. The control signal port is connected with three control signal lines of the control bus, which can represent 8 different control states, so that the embedded processor can realize the control of the dynamic time normalization algorithm module.

The integrated circuit implementation method of the above-mentioned speech recognition device adopting the dynamic time correction algorithm comprises the following steps:

First, after the calculation control module detects that there is a reset template buffer signal on the control bus, it resets the template buffer and waits for the input of data;

Second, when the calculation control module detects that the effective port of the test template or the effective port of the reference template is a high level, it outputs the corresponding address and control signal, and the test template characteristic parameter and the reference template characteristic on the characteristic data port Parameters are saved in the template buffer, and the input of data is completed;

Third, after the calculation control module detects that there is a start matching command on the control bus, it outputs the corresponding address and control signal to the template buffer, reads the reference template characteristic parameters and the test template characteristic parameters, and sends them to the template buffer respectively. The first processing unit and the last processing unit, while the calculation control module calculates the direction of motion at the next moment, and sends it to the first processing unit, and simultaneously makes the system busy flag port output high level, indicating that the system is working;

Fourth, the calculation control module sequentially reads the required reference templates and test templates from the template buffer according to the direction of motion, and sends them to the processing units for processing;

Fifth, after the data processing is completed, the last processing unit outputs the matching weighted distance between the test template and the reference template through the matching result output port, and at this time, the calculation control module makes the output data valid port output a positive pulse , making the system busy flag port output low level at the same time;

Sixth, after the embedded processor receives the positive pulse output from the output data valid port, it reads the matching result into the data memory and caches it, then repeats the above-mentioned steps 2 to 5 for the next match, and so on until all references Template matching is complete.

The direction of motion is determined by the calculation control module as the search direction of the first processing unit at the next matching moment, and at the same time, the parallelism of the search paths of the multi-processing units is used to make the signal of the direction of motion and the reference template feature together in the cardiac array flow to complete the matching of unequal-length templates.

In the third step, the characteristic parameters of the reference template are passed down sequentially from the first processing unit, and the characteristic parameters of the test template are input from the last processing unit, and made to flow in the opposite direction along the reference template parameters to form a reference template Parameters and test template parameters move towards each other in two opposite directions.

The working principle of the speech recognition device based on the dynamic time correction algorithm is:

(1) Generate a reference template. Through the buttons, select the training function and perform the following steps:

1. The voice of the speaker is converted into an electrical signal by the microphone and then converted into a digital voice signal by the analog/digital conversion module;

2, described embedded processor carries out preprocessing to digital voice signal, then extracts LPC (linear predictive coding) feature parameter as reference module feature parameter;

3. Saving the characteristic parameters of the reference module into the program memory to form a reference template for use in identification.

(2) Speech recognition. Select the recognition function through the buttons, and perform the following steps:

1. The voice of the speaker is converted into an electrical signal by the microphone and then converted into a digital voice signal by the analog/digital conversion module;

2. The embedded processor preprocesses the digital voice signal, extracts the LPC characteristic parameter as the test template characteristic parameter, and saves it in the data memory to form the test template;

3. The embedded processor reads out the test template characteristic parameter and the reference template characteristic parameter respectively from the data memory and the program memory, and sends it to the test template buffer, the template buffer in the dynamic time normalization algorithm module, reference stencil buffer;

4. The embedded processor sends a start matching command, and the dynamic time normalization algorithm module matches the test template with the reference template, and the embedded processor performs calculation of the dynamic time normalization algorithm module after obtaining the calculation result. Reset the reference template buffer, and take out the next reference template parameter from the program memory and put it into the reference template buffer of the dynamic time correction algorithm module, then start the next match, and so on, until all reference templates match Finish;

5. The embedded processor performs recognition judgment according to the matching weighted distances between the test template and all reference templates, obtains the recognition result and displays it on the display, thereby completing the speech recognition process.

In the present invention, the embedded processor and the dynamic time normalization algorithm module are implemented with a FPGA (Field Programmable Gate Array, Field Programmable Gate Array).

The integrated circuit implementation of the dynamic time normalization algorithm is described by using VHDL (Very High Speed Integrated Circuit Hardware Description Language, the hardware description language of ultra-high speed integrated circuits).

Compared with the prior art, the present invention has the following advantages and beneficial effects:

(1) Using the concept of "direction of motion", the DTW algorithm is implemented using a one-dimensional cardiac array. In order to improve the recognition speed to the greatest extent, the present invention adopts a method in which multiple processing units, that is, a processing unit array, operate simultaneously. The processing speed of this implementation method is an order of magnitude higher than that of the non-processing array implementation method, and is suitable for applications with relatively high real-time requirements. The circuit implementation structure of the current DTW algorithm adopts the method of fixing the number of frames of the reference template and the test template, and the number of frames of the reference template and the number of frames of the test template must be equal, so the versatility is relatively poor. To solve this problem, the present invention adds a calculation control module (cal_ctrl) in the DTW implementation structure, and the "movement direction" is generated by the calculation control module to determine the search direction of the first processing unit at the next matching moment, while using The parallelism of the search path of the multi-processing unit makes the motion direction signal and the reference template feature flow together in the cardiac array, and the test template feature is controlled by the motion direction to flow in the reverse direction in the cardiac array, so that each processing unit is within the length of the reference template and the test template. The matching process of the two templates can still be performed correctly under changing conditions.

(2) In the matching process of the DTW algorithm, for a one-dimensional parallel array, the characteristic parameters of the test template are input from the last processing unit, and make it flow in the opposite direction along the flow of the reference template parameters, and the reference template characteristic parameters are obtained from the first Processing units begin to pass down in turn, and the flow is sequential. Therefore, the template characteristic parameters are input from two opposite directions. Compared with the template characteristic parameters adopted by the traditional parallel array, the template characteristic parameters are input from one direction and the template processing method is stored first and then calculated, which greatly reduces the complexity of hardware resources and input interface control logic. Spend.

Description of drawings

Fig. 1 is the structural representation of speech recognition device of the present invention;

Fig. 2 is a schematic diagram of the internal structure of the dynamic time normalization algorithm module shown in Fig. 1;

Fig. 3 is a schematic diagram of the external structure of the dynamic time normalization algorithm module shown in Fig. 1;

FIG. 4 is a schematic diagram of the matching process between the test template and the reference template shown in FIG. 3 .

Detailed ways

The present invention will be described in further detail below in conjunction with the examples, but the embodiments of the present invention are not limited thereto.

Example 1

Fig. 1 has provided the concrete structure of the speech recognition device of the present invention based on dynamic time normalization algorithm, and this speech recognition device comprises embedded processor (Nios II) 1, dynamic time normalization algorithm module (DTW) 2, dynamic time normalization The positive algorithm module (DTW) 2 is connected with the embedded processor (Nios II) 1 through the control bus, and the embedded processor (Nios II) 1 is respectively connected with the program memory (FLASH) 3, the data memory (SRAM) 4, the display 5. At the same time, the analog/digital conversion module 6 is connected to the microphone 7, and the embedded processor (Nios II) 1 is provided with a button 8 and an RS232 interface 9.

Fig. 2, Fig. 3 have provided the structure of dynamic time normalization algorithm module (DTW) 2, comprise calculation control module (cal_ctrl) 10, processing unit array 11, template buffer 12, and be provided with system clock port (clk) 13 , control signal port (ctrl_sig) 14, feature data port (dat_ptn) 15, reference template valid port (ref_dat_en) 16, reset signal port (rst) 17, test module valid port (test_dat_en) 18, system busy flag port (busy) 19. Matching result output port (dout) 20 , output data effective port (dout_en) 21 . The processing unit array 11 is a one-dimensional linear array, each processing unit is connected sequentially, the system clock port (clk) 13, the reset signal port (rst) 17 are respectively connected to each processing unit, and the last processing unit 11-2 is connected with a matching result 0 Output port (dout) 20, calculation control module and the first and last processing unit 11-1, 11-2 and template buffer 12, system clock port (clk) 13, control signal port (ctrl_sig) 14, reference template Valid port (ref_dat-en) 16, reset signal port (rst) 17, test module valid port (test_dat_en) 18, system busy flag port (busy) 19, output data valid port (dout_en) 20 are respectively connected, template buffer 12 Connect 17 to the first and last processing units 11-1, 11-2, system clock port (clk) 13, feature data port (dat_ptn) 15, and reset signal port (rst) respectively.

The stencil buffer 12 includes a test stencil buffer and a reference stencil buffer.

The control bus is respectively connected with the system clock port (clk) 13, the control signal port (ctrl_sig) 14, the feature data port (dat_ptn) 15, the reference template valid port (ref_dat_en) 16, the reset signal port (rst) 17, the test module valid port ( test_dat_en) 18, system busy sign port (busy) 19, matching result output port (dout) 20, output data valid port (dout_en) 21 are connected, control signal port (ctrl_sig) 14 is connected with three control signal lines of control bus, can Indicates 8 different control states, enabling the embedded processor (Nios II) 1 to control the dynamic time correction algorithm module.

Example 2

As shown in Figure 1, Figure 2, and Figure 3, the working process of this speech recognition device based on the dynamic time correction algorithm is:

(1) Generate a reference template. Press button 8 to select the training function and proceed to the following steps:

1. The speaker's voice is converted into an electrical signal by the microphone 7 and then converted into a digital voice signal by the analog/digital conversion module 6;

2. The embedded processor (Nios II) 1 preprocesses the digital voice signal, and then extracts the LPC (Linear Predictive Coding) feature parameter as the reference module feature parameter;

3. Save the characteristic parameters of the reference module into the program memory (Flash) 3 to form a reference template for identification.

(2) Speech recognition. Press key 8 to select the identification function and proceed to the following steps:

1. The speaker's voice is converted into an electrical signal by the microphone 7 and then converted into a digital voice signal by the analog/digital conversion module 6;

2, the embedded processor (Nios II) preprocesses the digital voice signal, extracts the LPC feature parameter as the test template feature parameter, and saves it in the data memory (SRAM) 4 to form the test template;

3. The embedded processor (Nios II) 1 sends the reset template buffer signal to the reset signal port (rst) 17 through the control bus. After the calculation control module (cal_ctrl) 10 detects the reset template buffer signal, the template buffer is 12 reset and wait for data input;

4. Embedded processor (Nios II) 1 inputs test template feature parameters and reference template feature parameters sequentially to feature data port (dat_ptn) 15 through the control bus, and sets test template effective port (test_dat_en) 18 and reference template effective port successively (ref_dat_en) 16 is a high level, and when the calculation control module (cal_ctrl) 10 detects that the test template effective port (test_dat_en) 18 and the reference template effective port (ref_dat_en) 16 are high level, it outputs corresponding addresses and control signals in sequence, and Save the test template feature parameter and the reference template feature parameter to the test template buffer in the template buffer 12 and the reference template buffer respectively to complete the input of data;

5. The embedded processor (Nios II) 1 sends a matching command to the control signal port (ctrl_sig) 14 through the control bus. After the calculation control module (cal_ctrl) 10 detects the matching command, it outputs the corresponding address and control to the template buffer 12 signal, read out the reference template feature parameter and the test template feature parameter, and send them to the first processing unit 11-1 and the last processing unit 11-2 respectively, and the calculation control module 10 calculates the motion direction at the next moment , and send it to the first processing unit, and simultaneously make the system busy flag port (busy) 19 output a high level, indicating that the system is working;

6. Calculation control module (cal_ctrl) 10 sequentially reads the required reference template characteristic parameters and test template characteristic parameters from the template buffer 12 according to the motion direction, and sends them to each processing unit for calculation until the last data is calculated ;

7. After the calculation is completed, the last processing unit 11-2 outputs the matching result through the matching result output port (Dout) 20, that is, the matching weighted distance between the test template and the reference template. At this time, the calculation control module (cal_ctrl) 10 is valid by outputting data Port (dout_en) 21, the control bus outputs a positive pulse to the embedded processor (Nios II) 1, proposes an interrupt application, reminds the embedded processor (Nios II) 1 that the calculation has been completed, and simultaneously makes the system busy flag port (busy) 19 output low level;

8. After the embedded processor (Nios II) 1 receives the positive pulse output by the output data effective port (dout_en) 21, it reads the matching result into the data memory (SRAM) 4 and caches it, then repeats the above steps 3 to 7, Carry out the next match, and so on, until all reference template matches are completed;

G, embedded processor (Nios II) 1 carries out recognition judgment according to the matching weighted distance of test template and all reference templates, obtains recognition result and shows on display 5, thereby finishes the process of speech recognition.

As shown in Figure 3, the processing unit array of this dynamic time correction algorithm module is a one-dimensional parallel array, and the test template characteristic parameter is input from the last processing unit 11-2, and it is made to flow in the opposite direction along the reference template parameter flow , the reference template feature parameters are passed down sequentially from the first processing unit 11-1, and the flow is sequential. As shown in Figure 4, the search direction of the matching process between the test template and the reference template is from the lower left corner to the upper right corner, when the processing unit B is calculating the i-th row, the processing unit A is already calculating the i+1th row; and The characteristics of each column flow in the opposite direction to the row characteristics. When processing unit A calculates column j, processing unit B is already calculating column j+1. Therefore, the characteristic parameters of the reference template and the characteristic parameters of the test template are input from two opposite directions. Compared with the template characteristic parameters adopted by the traditional parallel array, the template processing method is input from one direction and stored first and then calculated, which greatly reduces hardware resources and The input interface controls the complexity of the logic.

In this device, program memory (FLASH) 3 is AM29LV065DU of Atmel Corporation, data memory (SRAM) 4 is IDT71V416 of IDT Corporation, analog/digital conversion module 6 adopts 8-bit analog/digital converter ADC0809 of National Semiconductor Corporation of the United States, and the display 5 is the LCM-S16020 of LUMEX Company, the embedded processor (Nios II) 1 and the dynamic time correction algorithm module 2 are realized by an FPGA (Field Programmable Gate Array, Field Programmable Gate Array) 22 of Altera Company whose model is EP1C20F400C7 , and its implementation process is:

1. Use VHDL (Very High Speed Integrated Circuit Hardware Description Language, the hardware description language for ultra-high speed integrated circuits) to describe the system circuit;

2. Use Synplicity's integrated circuit design synthesis software Synplify to synthesize the hardware circuit described by VHDL, and obtain the netlist file corresponding to Altera's FPGA model EP1C20F400C7;

3. Use Altera's FPGA design implementation software QuartusII to layout and route the obtained netlist and extract delay information;

4. Use Cadence's integrated circuit design simulation software NC-SIM for timing simulation;

5. Download the hardware configuration information to the above-mentioned FPGA (model EP1C20F400C7) with the FPGA design implementation software QuartusII of Altera Company.

As described above, the present invention can be preferably carried out.

Claims (3)

1, adopt the speech recognition method that integrated circuit realizes, it is characterized in that comprising the steps: First, after the calculation control module detects that there is a template buffer reset signal on the control bus, it resets the template buffer and waits for the input of data; Second, when the calculation control module detects that the test template is valid or the reference template is valid, the corresponding address and control signal are output, and the test template characteristic parameters and the reference template characteristic parameters on the characteristic data port are saved in the template buffer, and the completion data entry; The 3rd, after described calculation control module detects that there is a start matching command on the control bus, it outputs corresponding address and control signal to the template buffer, reads the reference template characteristic parameters and test template characteristic parameters, and sends them to the first processing unit and the last processing unit, while the calculation control module calculates the movement direction at the next moment, and sends it to the first processing unit, and simultaneously makes the system busy flag port output high level, indicating the system is working; Fourth, the calculation control module sequentially reads the required reference templates and test templates from the template buffer according to the direction of motion, and sends them to the processing units for processing; Fifth, after the data processing is completed, the last processing unit outputs the matching weighted distance between the test template and the reference template through the matching result output port, and at this time, the calculation control module makes the output data effective port output a positive pulse, At the same time, make the system busy flag port output low level; Sixth, after the embedded processor receives the positive pulse output from the output data valid port, it reads the matching result into the data memory and caches it, then repeats the above-mentioned steps 2 to 5 for the next match, and so on until all references Template matching is complete. 2. The speech recognition method implemented by an integrated circuit according to claim 1, wherein the direction of motion is determined by the calculation control module to determine the search direction of the first processing unit at the next matching moment, and simultaneously utilizes multiprocessing The parallelism of the search path of the unit makes the motion direction signal and the reference template feature flow together in the cardiac array to complete the matching of the unequal-length templates. 3, adopt the speech recognition method that integrated circuit realizes according to claim 1, it is characterized in that, in the described 3rd step, reference template feature parameter is started to pass down successively by first processing unit, and test template feature parameter is from last A processing unit inputs and makes it flow in opposite directions along the flow of the reference template parameters, so that the reference template parameters and the test template parameters move toward each other in two opposite directions.