TWI520131B - Speech Recognition System Based on Joint Time - Frequency Domain and Its Method - Google Patents

Description

Speech recognition system based on joint time-frequency domain feature and method thereof

The present invention relates to a speech and non-speech recognition system and method thereof, and more particularly to a speech and non-speech discrimination system based on combined time domain frequency domain modulation energy and a method thereof.

With the rapid development of mobile communication devices, in order to provide a more convenient operation interface, so that smart electronic products can provide better service functions, voice input mode has become one of the popular solution solutions.

When the mobile device is used outdoors or in a noisy environment, the background noise of the environment is likely to cause misjudgment of the speech recognition device, thereby affecting the service quality. Therefore, in order to solve this technical problem, the conventional techniques for eliminating background noise are (1). The filter designed by the time domain characteristics of the signal (such as the sound box energy of the signal and the zero-crossing rate) is used to eliminate noise, 2). Use the filter designed by the frequency domain characteristics of the signal (for the specific frequency band of the signal) to eliminate noise, and (3) use one of the above or a combination of filters designed to eliminate noise and pass through Machine learning to optimize its anti-noise power.

However, the environmental noise has great uncertainty, so that the filter designed by only part of the time and frequency can not accurately recognize the voice and noise in the environment of complex or low signal noise ratio, thus affecting the voice. The quality of the service; the filter optimized by machine learning requires a lengthy training process and a large amount of resources, making this solution not widely available in practice.

In order to solve the technical problem of the prior art, one of the objects of the present invention is to solve the problem of speech recognition filled with various background noises by analyzing and recognizing the specific structure of the speech.

To achieve the above objective, the present invention provides a voice recognition and non-speech system including a voice conversion module, a feature analysis module, an extraction module, and a decision module. First, the sound conversion module converts the input sound signal into a two-dimensional image in the time-frequency domain, and transmits the two-dimensional image in the time-frequency domain to the feature analysis module. The feature analysis module then analyzes the two-dimensional image of the aforementioned time-frequency domain to obtain a plurality of sound features, and transmits a plurality of sound features to the extraction module. Then, the extraction module extracts the speech feature identification value for the plurality of sound features, and then transmits the speech feature recognition to the decision module. Finally, the decision module compares the speech feature identification value with the speech threshold value to distinguish the voice signal of the audio signal from the non-speech signal.

For the above purposes, the present invention further provides a method for recognizing speech and non-speech, comprising the steps of: converting an input audio signal into a two-dimensional image in a time-frequency domain via a sound conversion module; and then, time-frequency The two-dimensional image of the domain is further analyzed using the feature module to obtain a plurality of sound features; further, a plurality of sound features are extracted using an extraction module to obtain a speech characteristic identification value; finally, a decision mode is used. The group compares and analyzes the speech feature identification value with the built-in speech threshold to distinguish between speech and non-speech signals in the audio signal.

Since the conventional filter can only filter the design for specific noise, the filter performance and filtering range of the filter are limited; in contrast, the speech and non-speech identification system and the method thereof according to the present invention are analyzed by analyzing the specificity of the speech. The structure is analyzed so that under complex background noise, the desired signal can still be extracted to provide a high quality speech recognition service.

1‧‧‧ Identifying voice and non-speech systems

3‧‧‧Sound Conversion Module

5‧‧‧Characteristic Analysis Module

7‧‧‧Extraction module

9‧‧‧Decision module

20‧‧‧Sound signal

22‧‧‧2D image of the time-frequency domain

24‧‧‧Sound characteristics

26‧‧‧Voice characteristic identification value

28‧‧‧Output signal

Figure 1 is an architectural diagram of a speech recognition and non-speech system of the present invention.

2 and 3 are schematic views of a two-dimensional filter used in the present invention.

Fig. 4 and Fig. 5 are schematic diagrams showing the sound characteristics of the sound signal of the present invention after being extracted by a two-dimensional filter.

Figure 6 is a flow chart of the method for recognizing speech and non-speech according to the present invention.

The specific embodiments are described below to illustrate the embodiments of the invention, but are not intended to limit the scope of the invention.

Embodiments for identifying voice and non-speech systems are disclosed below. Please refer to FIG. 1 , which is an architectural diagram of the recognized speech and non-speech system of the present invention. The voice non-speech recognition system 1 mainly includes a voice conversion module 3, a feature analysis module 5, an extraction module 7, and a decision module 9.

The sound conversion module 3 converts the received sound signal 20 into a two-dimensional image 22 in the time-frequency domain, and transmits the two-dimensional image 22 in the time-frequency domain to the feature analysis module connected to the sound conversion module 3. Group 5. Next, the feature analysis module 5 analyzes the two-dimensional image 22 in the time-frequency domain to obtain a plurality of sound features 24 and transmits the plurality of sound features 24 to the extraction module 7 coupled to the feature analysis module 5. The extraction module 7 performs an extraction operation on the plurality of sound features 24 and transmits the generated speech characteristic identification value 26 to the decision module 9 connected to the extraction module 7. The decision module 9 then compares the speech threshold value with the speech characteristic identification value 26 to distinguish between the speech signal and the non-speech signal.

In addition, after receiving the sound signal 20, the sound conversion module 3 can further sound The signal 20 is further divided into a plurality of sound boxes, which are respectively converted into short-time Fourier transforms and then combined to form a two-dimensional image 22 in the time-frequency domain. The aforementioned time-frequency domain two-dimensional image 22 is an auditory spectrogram that simulates the output of the cerebral cortex in the auditory sense.

Please refer to FIG. 2 and FIG. 3 at the same time. FIG. 2 and FIG. 3 are schematic diagrams of the speech modulation directionality based on the time-frequency domain two-dimensional image. Since the voice signal has a specific harmonicity and frequency modulation directionality, when the sound signal is transmitted through the axis and the frequency axis, the direction of the frequency modulation of the signal to be resolved can be clearly seen. Please continue to refer to Figure 2, where the frequency of the input signal is decremented over time in a specific time interval, and the voice modulation direction is the next shift mode, and the packet change rate on the time axis is 4 Hz, and the frequency axis is encapsulated. The rate of change is 2ms; please refer to Figure 3, the frequency of the input signal in a specific time interval is increasing with time, the direction of speech modulation is an upshift mode, and the rate of packet change on the time axis is 8Hz. The packet change rate on the frequency axis is 4 ms. The time-frequency domain 2D image 22 can analyze the direction of the modulation signal 20 and provide a connection module for analysis.

To further obtain the analytical parameters, the feature analysis module 5 uses the two-dimensional time-frequency impulse response bandpass filter bank to generate a plurality of sound features 24 in the combined time domain and frequency domain of the two-dimensional image 22 in the time-frequency domain. The aforementioned two-dimensional time-frequency impulse response bandpass filter bank can use the rate of change of the packet on the time axis and the rate of change of the packet on the frequency axis as parameters of the design to generate a plurality of bandpass filters. The number of the band pass filters can be determined by multiplying the rate of change of the time packet, the rate of change of the packet on the frequency axis, and the amount of directionality of the impulse response in the time domain. The plurality of filters described above can be distinguished by a combination of speech and non-speech characteristics into a plurality of groups of speech characteristic filters and non-speech characteristic filters to process speech and non-speech signals, respectively. The plurality of sound features 24 mainly include time, frequency, rate of change on the time axis, and The parameter of the rate of change of the packet on the frequency axis.

For a clearer explanation, please refer to Figure 4 and Figure 5. They are all schematic diagrams of the sound features 24. When the sound signal 22 including both the voice and the uncertain wind sound is sequentially processed by the sound module 3 and the feature analyzing module 5, the sound features 24 shown in FIGS. 4 and 5 can be obtained. 4 is a modulation energy distribution based on the rate of change of the packet on the time axis and the rate of change of the packet on the frequency axis; FIG. 5 is based on the rate of change of the packet on the time axis and the rate of change of the packet on the frequency axis. The modulation energy distribution of the voice signal. The present invention can clearly classify speech and non-speech signals by observing the difference in sound energy distribution.

In Fig. 4, the speech modulation energy mostly falls on the time axis, the packet change rate = 4 Hz, and the frequency axis packet change rate = 5 ms; in Fig. 5, the non-speech modulation energy mostly falls on The packet change rate on the time axis = 32 Hz, and the packet change rate on the time axis = 2 ms. Therefore, the present invention only needs to extract the desired signal by selecting the above specific interval.

Next, the extraction module 7 obtains the modulation energy of the speech characteristic and the non-speech characteristic on the time and frequency axis by using the above-mentioned plurality of speech characteristics and the non-speech characteristic filter, respectively, and then the aforementioned speech The modulation energy of the characteristic and the non-speech characteristic are respectively multiplied by the respective weight values (for example, the weight value of the modulation energy of the speech characteristic filter can be set to 1, and the modulation energy of the non-speech characteristic filter can be set to -1), respectively. The speech characteristics and the weighted scores with non-speech characteristics are obtained.

Finally, the decision module 9 compares the weighted scores of the voice and non-speech characteristics with the voice threshold to distinguish the voice signal 20 as a voice signal or a non-speech signal, thereby achieving the purpose of voice and non-speech recognition.

Embodiments of the recognized speech and non-speech methods are further disclosed below. Please refer to Referring to Figure 6, Figure 6 is a flow chart of the method for recognizing speech and non-speech according to the present invention. The method includes the following steps: Step 601: converting a sound signal into a two-dimensional image in a time-frequency domain via a sound conversion module; and step 603, analyzing a two-dimensional image in a time-frequency domain using a feature analysis module to obtain a plurality of sound features; in step 605, the plurality of sound features are extracted into a speech characteristic identification value by using an extraction module; and in step 607, the speech characteristic identification value is compared with the speech threshold value using a decision module to distinguish the sound signal. The part of the voice signal and the non-voice signal.

In this embodiment, the sound signal obtained in step 601 can be first divided into a plurality of sound frames through a sound conversion module, and a plurality of sound frames are used to generate a two-dimensional image in a time-frequency domain using a short-time Fourier transform, and The two-dimensional image of the aforementioned time-frequency domain is transmitted to the feature analysis module for processing. The two-dimensional image in the time-frequency domain converted by the sound conversion module is an auditory spectrum image of the cerebral cortex output in the simulated hearing.

The feature analysis module of step 603 is a two-dimensional time-frequency domain impulse response bandpass filter bank, and the two-dimensional time-frequency domain impulse response bandpass filter bank is a joint time-frequency of a two-dimensional image in a time-frequency domain. And the frequency change rate of the frequency domain is used for speech structure analysis to generate a plurality of sound features, and the sound features are transmitted to the extraction module for further processing. The above two-dimensional time-frequency domain impulse response band-pass filter bank can be implemented by using a plurality of band-pass filters, and the number of band-pass filters is based on the rate of change of the packet on the time axis, on the frequency axis. The product of the rate of change of the packet and the amount of directionality of the impulse response in the time-frequency domain is determined by the product of the multiplication. The filter bank can be divided into a plurality of speech characteristic filters and a plurality of non-speech characteristic filters to process speech and non-speech signals respectively through speech characteristics and non-speech characteristics. The plurality of sound characteristics described above include parameters of time, frequency, rate of change of the packet on the time axis, and rate of change of the packet on the frequency axis.

Next, in step 605, the plurality of sound features are passed through a speech characteristic filter in the plurality of band pass filters and multiplied by the weight value of the speech characteristic filter to obtain a weighted score having a speech characteristic, and the speech characteristic is The weighted score may be part of the speech characteristic identification value. In addition, step 605 may also pass the plurality of sound features to pass through the non-speech characteristic filter of the plurality of band pass filters and multiply the weight value of the non-speech characteristic filter. To obtain a weighted score with non-speech characteristics, and the weighted score with non-speech characteristics can be another part of the speech characteristic identification value.

Finally, in step 607, the weighted scores of the speech characteristics and the weighted scores of the non-speech characteristics are discriminated from the speech threshold value, and then the sound signal is used as the basis for the voice non-speech signal.

The detailed description above is a detailed description of one of the possible embodiments of the present invention, and is not intended to limit the scope of the present invention. The patent scope of this case.

1‧‧‧ Identifying voice and non-speech systems

3‧‧‧Sound Conversion Module

5‧‧‧Characteristic Analysis Module

7‧‧‧Extraction module

9‧‧‧Decision module

20‧‧‧Sound signal

22‧‧‧2D image of the time-frequency domain

24‧‧‧Sound characteristics

26‧‧‧Voice characteristic identification value

28‧‧‧Output signal

Claims (21)

A system for recognizing voice and non-speech includes: a sound conversion module for converting a sound signal into a two-dimensional image in a time-frequency domain; a feature analysis module connected to the sound conversion module and The two-dimensional image in the time-frequency domain is analyzed to obtain a plurality of sound features, and the sound features include a voice signal and a non-speech signal, wherein the modulation energy of the voice signal falls on the time axis, and the packet change rate=4 Hz, the frequency axis The change rate of the upper packet = 5ms, the modulation energy of the non-speech signal falls on the time axis, the rate of change of the packet = 32 Hz, and the rate of change of the packet on the time axis = 2 ms; an extraction module, and the feature analysis module Connecting and extracting the plurality of sound features into a speech characteristic identification value; a decision module is connected to the extraction module and comparing the speech characteristic identification value with a speech threshold value to distinguish a speech in the audio signal The signal and a part of a non-speech signal. The system for recognizing speech and non-speech according to claim 1, wherein the sound conversion module divides the sound signal into a plurality of sound boxes, and generates the plurality of sound boxes by short-time Fourier transform respectively. A two-dimensional image of the time-frequency domain. The system for recognizing speech and non-speech according to claim 1, wherein the two-dimensional image of the time-frequency domain in the sound conversion module simulates an auditory spectrogram output of the cerebral cortex in the auditory sense. The system for recognizing speech and non-speech according to claim 1, wherein the feature analysis module generates a two-dimensional image of the time-frequency domain by using a two-dimensional time-frequency domain impulse response bandpass filter bank. The complex sound features of the combined time domain and frequency domain. A system for recognizing speech and non-speech as described in claim 4, wherein the two-dimensional time The frequency domain impulse response bandpass filter bank consists of a plurality of bandpass filters. The system for recognizing speech and non-speech according to claim 5, wherein the number of the plurality of band pass filters is a rate of change of the packet on the time axis and a rate of change of the packet on the frequency axis, and a direction of the frequency response of the frequency domain. The product of the number of sex is multiplied by the product. The system for recognizing speech and non-speech according to claim 4, wherein the plurality of sound characteristics are time, frequency, rate of change of the packet on the time axis, and rate of change of the packet on the frequency axis. The system for recognizing speech and non-speech according to claim 6, wherein the plurality of band pass filters are divided into a plurality of speech characteristic filters and a plurality of non-speech characteristic filters according to speech characteristics and non-speech characteristics. . The system for recognizing speech and non-speech according to claim 8, wherein the extraction module obtains the modulation energy of the speech characteristic through the plurality of speech characteristic filters and the plurality of non-speech characteristic filters respectively a modulated energy value of the value and the non-speech characteristic, and multiplying the modulated energy value having the speech characteristic by the weight value of the plurality of speech characteristic filters and the modulation energy value having the non-speech characteristic multiplied by the plurality of non-speech After the weight value of the speech characteristic filter, the weighted scores of the weighted scores of the speech characteristics and the non-speech characteristics are respectively obtained. The system for recognizing a voice and a non-speech according to claim 9 , wherein the decision module determines the weighted score of the voice characteristic and the weighted score of the non-speech characteristic to determine the voice signal by using the voice threshold Voice signal or the non-speech signal. A method for recognizing speech and non-speech includes the following steps: Step 1: converting the audio signal into a two-dimensional image in the time-frequency domain via the sound conversion module; Step 2: performing a two-dimensional image of the time-frequency domain Use the feature analysis module for analysis to get the a plurality of sound features, wherein the sound characteristics include a voice signal and a non-speech signal, wherein the modulation energy of the voice signal falls on a time axis: a packet change rate=4 Hz, and a frequency axis packet change rate=5 ms, the non-speech signal The modulation energy falls on the time axis, the packet change rate=32 Hz, and the packet change rate on the time axis=2 ms; step 3: extracting the plurality of sound features into the speech characteristic identification value by using the extraction module; 4: The voice characteristic identification value and the voice threshold value are compared and analyzed by using the decision module to distinguish the voice signal and the non-voice signal part of the voice signal. The method for recognizing speech and non-speech according to claim 11, wherein the audio signal is divided into a plurality of sound boxes by the sound conversion module, and the plurality of sound frames are respectively generated by short time Fourier transform to generate the time frequency. A two-dimensional image of the domain, and the two-dimensional image of the time-frequency domain is transmitted to the feature analysis module for processing. The method for recognizing speech and non-speech according to claim 11, wherein the two-dimensional image of the time-frequency domain in the sound conversion module simulates an auditory spectrogram outputted by the cerebral cortex in the auditory sense. The method for recognizing speech and non-speech according to claim 11, wherein the feature analysis module is a two-dimensional time-frequency domain impulse response bandpass filter bank, and the two-dimensional time-frequency domain impulse response is used. Performing speech structure analysis on the joint time domain and frequency domain energy change rate of the two-dimensional image of the band-pass filter bank in the time-frequency domain to generate the plurality of sound features, and transmitting the plurality of sound features to the extraction module Process it. The method for recognizing speech and non-speech according to claim 14, wherein the two-dimensional time-frequency domain impulse response band pass filter group is composed of the plurality of band pass filters. The method for recognizing speech and non-speech according to claim 14, wherein the plurality of sound characteristics are time, frequency, rate of change of the packet on the time axis, and rate of change of the packet on the frequency axis. The method for recognizing speech and non-speech according to claim 15, wherein the number of the plurality of band pass filters is a rate of change of the packet on the time axis and a rate of change of the packet on the frequency axis in time and frequency domain. The product obtained by multiplying the number of directionalities is derived. The method for recognizing speech and non-speech according to claim 15 is characterized in that the plurality of band pass filters are divided into a plurality of speech characteristic filters and a plurality of non-speech characteristic filters according to a speech characteristic and a non-speech characteristic. The method for recognizing speech and non-speech according to claim 18, wherein the speech characteristic identification value is passed by the plurality of sound characteristics through the speech characteristic filter of the plurality of band pass filters and multiplied by the The weighted value of the speech characteristic filter can be followed by a weighted score with speech characteristics. The method for recognizing speech and non-speech according to claim 19, wherein the non-speech characteristic identification value is passed by the plurality of sound characteristics through the non-speech characteristic filter of the plurality of band pass filters and multiplied by the non-speech The weight value of the speech characteristic filter can be followed by a weighted score with non-speech characteristics. The method for recognizing speech and non-speech according to claim 20, wherein the weighted score of the speech characteristic and the weighted score of the non-speech characteristic are discriminated from the speech threshold, and the voice signal is used as the voice signal or The basis for determining the non-speech signal.