CN119479708B - Speech recognition method for parkinsonism - Google Patents
Speech recognition method for parkinsonism Download PDFInfo
- Publication number
- CN119479708B CN119479708B CN202510064918.0A CN202510064918A CN119479708B CN 119479708 B CN119479708 B CN 119479708B CN 202510064918 A CN202510064918 A CN 202510064918A CN 119479708 B CN119479708 B CN 119479708B
- Authority
- CN
- China
- Prior art keywords
- tremor
- frequency
- voice
- pronunciation
- frequency spectrum
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 25
- 208000027089 Parkinsonian disease Diseases 0.000 title claims description 21
- 206010034010 Parkinsonism Diseases 0.000 title claims description 21
- 206010044565 Tremor Diseases 0.000 claims abstract description 128
- 238000001228 spectrum Methods 0.000 claims abstract description 116
- 238000012545 processing Methods 0.000 claims abstract description 67
- 239000013598 vector Substances 0.000 claims abstract description 44
- 230000008859 change Effects 0.000 claims description 27
- 230000010354 integration Effects 0.000 claims description 22
- 208000018737 Parkinson disease Diseases 0.000 claims description 10
- 238000006243 chemical reaction Methods 0.000 claims description 10
- 230000004913 activation Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000012216 screening Methods 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 6
- 238000005070 sampling Methods 0.000 description 6
- 238000000605 extraction Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 210000000056 organ Anatomy 0.000 description 3
- 239000000284 extract Substances 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 210000003205 muscle Anatomy 0.000 description 2
- 210000000653 nervous system Anatomy 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 238000010183 spectrum analysis Methods 0.000 description 2
- 210000001260 vocal cord Anatomy 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010924 continuous production Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000003412 degenerative effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 208000035475 disorder Diseases 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 230000001771 impaired effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 210000004717 laryngeal muscle Anatomy 0.000 description 1
- 230000004630 mental health Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
- G10L25/66—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for extracting parameters related to health condition
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
- G10L25/18—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/90—Pitch determination of speech signals
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Signal Processing (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Epidemiology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Electrically Operated Instructional Devices (AREA)
Abstract
本发明公开了一种帕金森病人的语音识别方法，属于语音识别技术领域。本发明首先对帕金森病人的语音信号进行震颤频率剔除，获得去震颤语音频谱集合；其次，基于相邻时刻去震颤语音频谱的差异，将频谱集合划分为多个发音子集；然后，在每个发音子集中提取峰值频率、泛音质心值和泛音偏差值，构建特征性发音向量；最后，利用多层发音向量处理模型，最终解析出帕金森病人的语义内容。本发明显著提升了帕金森病患者语音识别的准确性。
The present invention discloses a speech recognition method for Parkinson's patients, and belongs to the field of speech recognition technology. The present invention first removes the tremor frequency of the speech signal of the Parkinson's patient to obtain a detremorized speech spectrum set; secondly, based on the difference of the detremorized speech spectrum at adjacent moments, the spectrum set is divided into multiple pronunciation subsets; then, the peak frequency, overtone centroid value and overtone deviation value are extracted in each pronunciation subset to construct a characteristic pronunciation vector; finally, a multi-layer pronunciation vector processing model is used to finally parse the semantic content of the Parkinson's patient. The present invention significantly improves the accuracy of speech recognition of Parkinson's patients.
Description
Technical Field
The invention relates to the technical field of voice recognition, in particular to a voice recognition method for parkinsonism.
Background
Parkinson's disease is a common degenerative disorder of the nervous system that primarily affects motor ability and speech expression in patients. With the progress of the illness, the voice characteristics of the patient can change significantly, including the tone, volume and clarity of voice. These changes not only affect the patient's ability to communicate, but also negatively impact their mental health and quality of life. Therefore, development of effective speech recognition technology to assist parkinson's disease patients in communication has become an important research direction.
The prior art generally employs conventional spectral analysis and acoustic feature extraction methods, such as mel-frequency cepstral coefficients (MFCCs), linear Predictive Coding (LPC), and the like. However, these methods have significant limitations in processing voices of parkinson's disease patients, and cannot separate the voice signal segments belonging to the same voice, and feature extraction must be performed on the whole voice signal, so that the accuracy and effectiveness of signal processing are greatly reduced, and there is a problem that the voice recognition accuracy is low.
Disclosure of Invention
Aiming at the defects in the prior art, the voice recognition method for the parkinsonism solves the problem of low voice recognition precision in the prior art.
In order to achieve the aim of the invention, the technical scheme adopted by the invention is that the voice recognition method for parkinsonism comprises the following steps:
S1, eliminating tremor frequency of voice signals of parkinsonism to obtain tremor-removed voice frequency spectrum set;
s2, in the tremor-removing voice frequency spectrum set, dividing the tremor-removing voice frequency spectrum set according to the difference of tremor-removing voice frequency spectrums at adjacent moments to obtain a plurality of sounding tremor-removing voice frequency spectrum subsets;
s3, extracting peak frequency, overtone centroid value and overtone offset value from frequencies in the tremor-removing voice frequency spectrum subset of each pronunciation to construct a pronunciation vector;
and S4, processing the multiple pronunciation vectors by adopting a multi-layer pronunciation vector processing model to obtain the semantics of the parkinsonism.
Further, S1 comprises the following sub-steps:
S11, sliding on a voice signal by adopting a sliding window, and advancing for 1 moment each time;
s12, carrying out Fourier transform on a signal section under a sliding window when sliding each time to obtain a voice frequency spectrum at each moment;
s13, calculating the average value of all the amplitude values in the voice frequency spectrum at each moment, and screening out the frequency value corresponding to the amplitude value lower than the average value to form a tremor candidate frequency set;
s14, intersection sets are taken from the candidate tremor frequency sets at all times to obtain tremor frequencies;
S15, eliminating tremor frequency from the voice frequency spectrum at each moment to obtain tremor-removing voice frequency spectrum at each moment, and arranging tremor-removing voice frequency spectrums at each moment in sequence according to time occurrence to form tremor-removing voice frequency spectrum sets.
Further, S2 comprises the following sub-steps:
s21, in the tremor-removing voice frequency spectrum set, the frequency in the tremor-removing voice frequency spectrum at the time t+1 and the frequency in the tremor-removing voice frequency spectrum at the time t are combined;
s22, eliminating frequencies in the tremor-removed voice frequency spectrum at the moment t from the union to obtain a new added frequency at the moment t+1;
s23, calculating a frequency change value according to the new frequency at the time t+1;
S24, when the frequency change value is larger than the frequency change threshold value, marking the moment t+1 as a pronunciation conversion point;
s25, dividing the tremor-removing voice frequency spectrum set according to the positions of the pronunciation conversion points to obtain tremor-removing voice frequency spectrum subsets of the pronunciations.
Further, the formula for calculating the frequency variation value in S23 is:, Wherein mu t+1 is the frequency change value at time t+1, f t+1,z,i is the ith newly added frequency at time t+1, f t,s,j is the jth frequency in the tremor-removing voice frequency spectrum at time t, I is the absolute value, M is the number of frequencies in the tremor-removing voice frequency spectrum at time t, and min is M I and j are positive integers, N is the number of newly added frequencies at time t+1, and d t+1,z,i is the minimum difference between the ith newly added frequency at time t+1 and the frequencies in the tremor-removed voice frequency spectrum at time t.
Further, S3 comprises the following sub-steps:
S31, in each tremor-removing voice frequency spectrum of a tremor-removing voice frequency spectrum subset of a pronunciation, finding out the frequency corresponding to the maximum amplitude as a main frequency, and taking the average value of all the main frequencies to obtain a peak frequency;
S32, in a sounding tremor-removing voice frequency spectrum subset, taking other frequencies except a main frequency in each tremor-removing voice frequency spectrum as overtones;
s33, calculating an overtone centroid value of overtone frequency;
S34, calculating overtone deviation values according to the differences between the overtone frequencies and the peak frequencies;
S35, constructing a pronunciation vector by taking the peak frequency, the overtone centroid value and the overtone deviation value as elements.
Further, the calculation formula of the harmonic overtone centroid value in S33 is: Wherein f o,c is the harmonic centroid value, f o,i is the ith harmonic frequency in the sounding tremor-removal audio spectrum subset, A o,i is the amplitude of the ith harmonic frequency f o,i, L is the number of harmonic frequencies in the sounding tremor-removal audio spectrum subset, and i is a positive integer.
Further, the formula for calculating the harmonic offset value in S34 is:
wherein f o,d is the harmonic offset value, f o,i is the ith harmonic frequency in the sounding tremor-removal audio spectrum subset, i is a positive integer, f peak is the peak frequency, and L is the number of harmonic frequencies in the sounding tremor-removal audio spectrum subset.
Further, the multi-layer pronunciation vector processing model in S4 comprises a plurality of first pronunciation processing layers, a plurality of second pronunciation processing layers, a first BiLSTM layers, a second BiLSTM layers, a feature integration layer and a CRF unit;
The input end of each first pronunciation processing layer is used for inputting a pronunciation vector, the input end of each second pronunciation processing layer is used for inputting a pronunciation vector, the input end of each first BiLSTM layer is respectively connected with the output ends of the plurality of first pronunciation processing layers, the input end of each second BiLSTM layer is respectively connected with the output ends of the plurality of second pronunciation processing layers, the input end of the characteristic integration layer is respectively connected with the output ends of the first BiLSTM layer and the output end of the second BiLSTM layer, the output ends of the characteristic integration layer are connected with the input ends of the CRF unit, and the output end of the CRF unit serves as the output end of the multi-layer pronunciation vector processing model.
Further, the expression of the first sounding processing layer is: Wherein y 1 is the output of the first sounding processing layer, tanh is a hyperbolic tangent activation function, x n is the nth element in the sounding vector, ω 1,n is the nth weight in the first sounding processing layer, b 1,n is the nth bias in the first sounding processing layer, and the value range of n is a positive integer ranging from 1 to 3;
the expression of the second pronunciation processing layer is: Where y 2 is the output of the second pronunciation processing layer, sigmoid is the S-type activation function, ω 2,n is the nth weight in the second pronunciation processing layer, and b 2,n is the nth bias in the second pronunciation processing layer.
Further, the expression of the feature integration layer is: Wherein H m is the mth integrated feature output by the feature integration layer, H 1,m is the mth feature output by the first BiLSTM layer, H 2,m is the mth feature output by the second BiLSTM layer, ω 1,m is the mth first weight in the feature integration layer, and ω 2,m is the mth second weight in the feature integration layer.
The beneficial effects of the invention are as follows:
1. The invention eliminates tremble frequency for voice signal, avoids tremble frequency affecting voice recognition, and improves precision of voice recognition.
2. According to the invention, in the tremor-removing voice frequency spectrum set, according to the difference of tremor-removing voice frequency spectrums at adjacent moments, the segmentation point of each pronunciation is found, and the tremor-removing voice frequency spectrum set is divided, so that different pronunciation paragraphs can be more accurately distinguished, and the problem of inaccurate feature extraction caused by whole-segment voice signal processing is avoided.
3. The invention extracts peak frequency, overtone centroid value and overtone deviation value from the tremor-removing voice frequency spectrum subset of each pronunciation, and reflects the voice characteristics of the pronunciation of the section.
4. The invention adopts the multi-layer pronunciation vector processing model to process a plurality of pronunciation vectors, synthesizes pronunciation vectors corresponding to the voice signals, recognizes the semantics of parkinsonism, and improves the voice recognition precision.
Drawings
Fig. 1 is a flowchart of a voice recognition method of parkinson's disease person.
FIG. 2 is a schematic diagram of a multi-layered pronunciation vector processing model.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and all the inventions which make use of the inventive concept are protected by the spirit and scope of the present invention as defined and defined in the appended claims to those skilled in the art.
As shown in fig. 1, a voice recognition method for parkinson's disease comprises the following steps:
S1, eliminating tremor frequency of voice signals of parkinsonism to obtain tremor-removed voice frequency spectrum set;
s2, in the tremor-removing voice frequency spectrum set, dividing the tremor-removing voice frequency spectrum set according to the difference of tremor-removing voice frequency spectrums at adjacent moments to obtain a plurality of sounding tremor-removing voice frequency spectrum subsets;
s3, extracting peak frequency, overtone centroid value and overtone offset value from frequencies in the tremor-removing voice frequency spectrum subset of each pronunciation to construct a pronunciation vector;
and S4, processing the multiple pronunciation vectors by adopting a multi-layer pronunciation vector processing model to obtain the semantics of the parkinsonism.
In this embodiment, S1 includes the following sub-steps:
S11, sliding on a voice signal by adopting a sliding window, and advancing for 1 moment each time;
s12, carrying out Fourier transform on a signal section under a sliding window when sliding each time to obtain a voice frequency spectrum at each moment;
s13, calculating the average value of all the amplitude values in the voice frequency spectrum at each moment, and screening out the frequency value corresponding to the amplitude value lower than the average value to form a tremor candidate frequency set;
s14, intersection sets are taken from the candidate tremor frequency sets at all times to obtain tremor frequencies;
S15, eliminating tremor frequency from the voice frequency spectrum at each moment to obtain tremor-removing voice frequency spectrum at each moment, and arranging tremor-removing voice frequency spectrums at each moment in sequence according to time occurrence to form tremor-removing voice frequency spectrum sets.
The invention adopts the sliding window to slide on the voice signal, advances for 1 moment each time, carries out Fourier transform on the signal section under the sliding window after each sliding to obtain the voice frequency spectrum at each moment, realizes the dynamic and real-time frequency spectrum analysis of the voice signal, and can capture the fine change of the voice signal. Because of impaired nervous system motor control, reduced muscle control, and reduced coordination of vocal cords and vocal muscles in parkinsonism, parkinsonism suffers from speech tremors. Tremors are manifested as low amplitude, persistent abnormal frequencies that tend to be masked in normal speech signals. Therefore, the invention screens out the frequency corresponding to the low amplitude value in the voice frequency spectrum at each moment, and the frequency which is always present in the voice signal is extracted by taking the intersection of the candidate tremor frequency sets at each moment, and the frequency is distributed in the whole stage of the voice signal, so that the frequency which is continuously present in the whole stage of the voice signal is effectively extracted, and the tremor frequency which is stably present in the voice of the parkinsonism can be accurately positioned, and the frequency is not sporadic noise.
In this embodiment, the sliding window is a one-dimensional window, the length of the sliding window is K, K is a positive integer, and the sliding window can cover K times each time, i.e. a signal section with a length of K can be obtained.
In this embodiment, the fundamental frequency range of human voice is typically 50-250Hz, the frequency of parkinsonian voice tremor is typically 4-7Hz, the sliding window can be set to 20-40 ms, the length can cover the pitch period (typically 10-20 ms), the typical frequency range of parkinsonian voice tremor is covered, the balance of time resolution and frequency resolution is ensured, for example, when the sampling rate is 16kHz, K can take 32 ms, and the corresponding sampling point number is about 16kHz multiplied by 0.032 s=512 sampling points. The length of the forward one time is in the range of 5-20 milliseconds, for example, when the sampling rate is 16kHz, 15 milliseconds are 1 time length, and the corresponding sampling points are 16kHz multiplied by 0.015=240 sampling points.
In this embodiment, the sliding window and the length of 1 moment may be adjusted, and when the sliding window is too small, the frequency resolution is low, tremble cannot be accurately captured, and when the sliding window is too large, the time resolution is reduced, and the detail features are lost. When the length of 1 moment is too small, the calculation amount is large, the system burden is increased, and important voice features can be lost when the length is too large.
In this embodiment, S2 includes the following substeps:
s21, in the tremor-removing voice frequency spectrum set, the frequency in the tremor-removing voice frequency spectrum at the time t+1 and the frequency in the tremor-removing voice frequency spectrum at the time t are combined;
s22, eliminating frequencies in the tremor-removed voice frequency spectrum at the moment t from the union to obtain a new added frequency at the moment t+1;
s23, calculating a frequency change value according to the new frequency at the time t+1;
S24, when the frequency change value is larger than the frequency change threshold value, marking the moment t+1 as a pronunciation conversion point;
s25, dividing the tremor-removing voice frequency spectrum set according to the positions of the pronunciation conversion points to obtain tremor-removing voice frequency spectrum subsets of the pronunciations.
The invention marks the pronunciation conversion points in the tremor-removed voice frequency spectrum set, and divides the tremor voice frequency spectrum of each section according to the pronunciation conversion points to form a subset.
The speech is a dynamically changing signal, the frequency characteristics of different phonemes and syllables are converted, and the pronunciation process is continuous but discrete language expression. Therefore, the invention compares the frequencies in the tremor-removing voice frequency spectrums at the adjacent time, finds out the newly added frequency at the next time, calculates the frequency change value according to the newly added frequency at the time t+1, can embody the change condition of the pronunciation, and indicates that the pronunciation has change between the time t and the time t+1 if the frequency change value is larger than the frequency change threshold. Pronunciation involves coordination of organs such as vocal cords, mouth, tongue, etc., each phoneme emission requires a specific organ position and muscle state, and this conversion may lead to a significant change in frequency. The invention marks the pronunciation conversion points so that each pronunciation has a tremor-removing voice frequency spectrum subset.
In this embodiment, if there is a small number of tremor-removed speech spectra in a subset of tremor-removed speech spectra, only one tremor-removed speech spectrum may be discarded, for example, where the frequency differences between the tremor-removed speech spectra and the time of the left and right are large, so that the subset does not have continuous pronunciation and cannot provide valuable information, and by discarding these unstable spectra, the overall quality of the data set may be improved.
Speech signals are a continuous process, with usually fluent utterances, and natural transitions between phonemes. The features of a pronunciation are similar at multiple times. During normal pronunciation, certain frequency characteristics remain relatively stable for a short period of time. The tremor-free speech spectrum at multiple moments may reflect this stability, forming a subset of moments.
When the frequency change value is larger than the frequency change threshold value, the invention belongs to the last tremor voice frequency spectrum subset at the time t and belongs to the next tremor voice frequency spectrum subset at the time t+1.
In the present embodiment, the formula for calculating the frequency variation value in S23 is:, Wherein mu t+1 is the frequency change value at time t+1, f t+1,z,i is the ith newly added frequency at time t+1, f t,s,j is the jth frequency in the tremor-removing voice frequency spectrum at time t, I is the absolute value, M is the number of frequencies in the tremor-removing voice frequency spectrum at time t, and min is M I and j are positive integers, N is the number of newly added frequencies at time t+1, and d t+1,z,i is the minimum difference between the ith newly added frequency at time t+1 and the frequencies in the tremor-removed voice frequency spectrum at time t.
The invention calculates the difference between each newly added frequency and each frequency in the tremor-removing voice frequency spectrum at the moment t, finds out the minimum difference, determines the frequency offset condition, and combines the number of the newly added frequencies to obtain the frequency change value, when the frequency change value is larger, the pronunciation at the moment t+1 changes obviously, so the moment t+1 is probably the pronunciation of the next word.
In the present invention, the frequency change threshold is a threshold set for the frequency change value, and in the present embodiment, the frequency change threshold is set to 0.5.
In this embodiment, S3 includes the following substeps:
S31, in each tremor-removing voice frequency spectrum of a tremor-removing voice frequency spectrum subset of a pronunciation, finding out the frequency corresponding to the maximum amplitude as a main frequency, and taking the average value of all the main frequencies to obtain a peak frequency;
S32, in a sounding tremor-removing voice frequency spectrum subset, taking other frequencies except a main frequency in each tremor-removing voice frequency spectrum as overtones;
s33, calculating an overtone centroid value of overtone frequency;
S34, calculating overtone deviation values according to the differences between the overtone frequencies and the peak frequencies;
S35, constructing a pronunciation vector by taking the peak frequency, the overtone centroid value and the overtone deviation value as elements.
In the sound signal, the frequency point with the highest amplitude corresponds to the fundamental frequency, which is a main feature of sound. The fundamental frequency determines the pitch of the sound and is a key factor in identifying the pitch and timbre. The main frequency corresponding to the maximum amplitude exists in each tremor-removing voice frequency spectrum, so that the average value of all main frequencies of the section of pronunciation is obtained, the peak frequency is obtained, and the main characteristics of the section of pronunciation are reflected.
In this embodiment, the calculation formula of the harmonic overtone centroid value in S33 is: Wherein f o,c is the harmonic centroid value, f o,i is the ith harmonic frequency in the sounding tremor-removal audio spectrum subset, A o,i is the amplitude of the ith harmonic frequency f o,i, L is the number of harmonic frequencies in the sounding tremor-removal audio spectrum subset, and i is a positive integer.
In the present embodiment, the formula for calculating the overtone deviation value in S34 is: wherein f o,d is the harmonic offset value, f o,i is the ith harmonic frequency in the sounding tremor-removal audio spectrum subset, i is a positive integer, f peak is the peak frequency, and L is the number of harmonic frequencies in the sounding tremor-removal audio spectrum subset.
The invention forms a subset of tremor-removing voice frequency spectrums belonging to the same pronunciation, finds out the frequency corresponding to the maximum amplitude value in each tremor-removing voice frequency spectrum in the subset as the main frequency, averages each main frequency to obtain the peak frequency, and the main frequency reflects the basic pronunciation characteristics, so that the peak frequency of the pronunciation can embody the pronunciation main characteristics, the overtone reflects the subtle state of a pronunciation organ, the overtone centroid value of the overtone frequency is extracted, the main frequency range of the overtone is embodied, the overtone deviation value is calculated again, the difference between each overtone frequency and the peak frequency is embodied, and the distribution situation of the overtone frequency and the peak frequency during the pronunciation is further excavated.
As shown in fig. 2, the multi-layered pronunciation vector processing model in S4 includes a plurality of first pronunciation processing layers, a plurality of second pronunciation processing layers, a first BiLSTM layers, a second BiLSTM layers, a feature integration layer, and a CRF unit;
The input end of each first pronunciation processing layer is used for inputting a pronunciation vector, the input end of each second pronunciation processing layer is used for inputting a pronunciation vector, the input end of each first BiLSTM layer is respectively connected with the output ends of the plurality of first pronunciation processing layers, the input end of each second BiLSTM layer is respectively connected with the output ends of the plurality of second pronunciation processing layers, the input end of the characteristic integration layer is respectively connected with the output ends of the first BiLSTM layer and the output end of the second BiLSTM layer, the output ends of the characteristic integration layer are connected with the input ends of the CRF unit, and the output end of the CRF unit serves as the output end of the multi-layer pronunciation vector processing model.
In the present invention, the number of the first pronunciation processing layers and the second pronunciation processing layers is equal to the number of pronunciation vectors.
In this embodiment, the expression of the first sounding processing layer is: wherein y 1 is the output of the first sounding processing layer, tanh is the hyperbolic tangent activation function, x n is the nth element in the sounding vector, ω 1,n is the nth weight in the first sounding processing layer, b 1,n is the nth bias in the first sounding processing layer, and the value range of n is a positive integer ranging from 1 to 3.
In this embodiment, the expression of the second pronunciation processing layer is: Where y 2 is the output of the second pronunciation processing layer, sigmoid is the S-type activation function, ω 2,n is the nth weight in the second pronunciation processing layer, and b 2,n is the nth bias in the second pronunciation processing layer.
In this embodiment, the expression of the feature integration layer is: Wherein H m is the mth integrated feature output by the feature integration layer, H 1,m is the mth feature output by the first BiLSTM layer, H 2,m is the mth feature output by the second BiLSTM layer, ω 1,m is the mth first weight in the feature integration layer, and ω 2,m is the mth second weight in the feature integration layer.
According to the invention, two pronunciation processing layers are arranged to process the same pronunciation vector, characteristics of voice signals are extracted from different layers, the understanding capability of a model on complex voice signals is enhanced, two pronunciation characteristics of the same pronunciation vector are obtained, a first BiLSTM layer is adopted to process one pronunciation characteristic of each pronunciation vector, a second BiLSTM layer is adopted to process the other pronunciation characteristic of each pronunciation vector, so that a BiLSTM layer can capture a context relation better, and a characteristic integration layer is adopted to integrate the characteristics of the first BiLSTM layer and the second BiLSTM layer, so that the characteristics of a double channel are fused, and the accuracy of semantic recognition is improved.
In this embodiment, the CRF unit may also be replaced with a Softmax layer or CTC layer.
In the invention, all pronunciation vectors corresponding to the voice signal are converted into readable text through a multi-layer pronunciation vector processing model.
In this embodiment, weights and biases in the multi-layer pronunciation vector processing model are trained using the existing gradient descent method.
The invention eliminates tremble frequency for voice signal, avoids tremble frequency affecting voice recognition, and improves precision of voice recognition.
According to the invention, in the tremor-removing voice frequency spectrum set, according to the difference of tremor-removing voice frequency spectrums at adjacent moments, the segmentation point of each pronunciation is found, and the tremor-removing voice frequency spectrum set is divided, so that different pronunciation paragraphs can be more accurately distinguished, and the problem of inaccurate feature extraction caused by whole-segment voice signal processing is avoided.
The invention extracts peak frequency, overtone centroid value and overtone deviation value from the tremor-removing voice frequency spectrum subset of each pronunciation, and reflects the voice characteristics of the pronunciation of the section.
The invention adopts the multi-layer pronunciation vector processing model to process a plurality of pronunciation vectors, synthesizes pronunciation vectors corresponding to the voice signals, recognizes the semantics of parkinsonism, and improves the voice recognition precision.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method for voice recognition of parkinson's disease, comprising the steps of:
S1, eliminating tremor frequency of voice signals of parkinsonism to obtain tremor-removed voice frequency spectrum set;
s2, in the tremor-removing voice frequency spectrum set, dividing the tremor-removing voice frequency spectrum set according to the difference of tremor-removing voice frequency spectrums at adjacent moments to obtain a plurality of sounding tremor-removing voice frequency spectrum subsets;
s3, extracting peak frequency, overtone centroid value and overtone offset value from frequencies in the tremor-removing voice frequency spectrum subset of each pronunciation to construct a pronunciation vector;
and S4, processing the multiple pronunciation vectors by adopting a multi-layer pronunciation vector processing model to obtain the semantics of the parkinsonism.
2. The method for voice recognition of parkinson' S disease according to claim 1, wherein said S1 comprises the following sub-steps:
S11, sliding on a voice signal by adopting a sliding window, and advancing for 1 moment each time;
s12, carrying out Fourier transform on a signal section under a sliding window when sliding each time to obtain a voice frequency spectrum at each moment;
s13, calculating the average value of all the amplitude values in the voice frequency spectrum at each moment, and screening out the frequency value corresponding to the amplitude value lower than the average value to form a tremor candidate frequency set;
s14, intersection sets are taken from the candidate tremor frequency sets at all times to obtain tremor frequencies;
S15, eliminating tremor frequency from the voice frequency spectrum at each moment to obtain tremor-removing voice frequency spectrum at each moment, and arranging tremor-removing voice frequency spectrums at each moment in sequence according to time occurrence to form tremor-removing voice frequency spectrum sets.
3. The method for voice recognition of parkinson' S disease according to claim 1, wherein said S2 comprises the following sub-steps:
s21, in the tremor-removing voice frequency spectrum set, the frequency in the tremor-removing voice frequency spectrum at the time t+1 and the frequency in the tremor-removing voice frequency spectrum at the time t are combined;
s22, eliminating frequencies in the tremor-removed voice frequency spectrum at the moment t from the union to obtain a new added frequency at the moment t+1;
s23, calculating a frequency change value according to the new frequency at the time t+1;
S24, when the frequency change value is larger than the frequency change threshold value, marking the moment t+1 as a pronunciation conversion point;
s25, dividing the tremor-removing voice frequency spectrum set according to the positions of the pronunciation conversion points to obtain tremor-removing voice frequency spectrum subsets of the pronunciations.
4. A method for voice recognition of parkinson' S disease according to claim 3, wherein said formula for calculating the frequency variation value in S23 is:, Wherein mu t+1 is the frequency change value at time t+1, f t+1,z,i is the ith newly added frequency at time t+1, f t,s,j is the jth frequency in the tremor-removing voice frequency spectrum at time t, I is the absolute value, M is the number of frequencies in the tremor-removing voice frequency spectrum at time t, and min is M I and j are positive integers, N is the number of newly added frequencies at time t+1, and d t+1,z,i is the minimum difference between the ith newly added frequency at time t+1 and the frequencies in the tremor-removed voice frequency spectrum at time t.
5. The method for voice recognition of parkinson' S disease according to claim 1, wherein said S3 comprises the following sub-steps:
S31, finding out the frequency corresponding to the maximum amplitude value from each tremor-removing voice frequency spectrum of a voice tremor-removing voice frequency spectrum subset as a main frequency, and taking the average value of all the main frequencies to obtain a peak frequency;
S32, in a sounding tremor-removing voice frequency spectrum subset, taking other frequencies except a main frequency in each tremor-removing voice frequency spectrum as overtones;
s33, calculating an overtone centroid value of overtone frequency;
S34, calculating overtone deviation values according to the differences between the overtone frequencies and the peak frequencies;
S35, constructing a pronunciation vector by taking the peak frequency, the overtone centroid value and the overtone deviation value as elements.
6. The method for recognizing the voice of parkinsonism according to claim 5, wherein the calculation formula of the harmonic-overtone centroid value in S33 is: Wherein f o,c is the harmonic centroid value, f o,i is the ith harmonic frequency in the sounding tremor-removal audio spectrum subset, A o,i is the amplitude of the ith harmonic frequency f o,i, L is the number of harmonic frequencies in the sounding tremor-removal audio spectrum subset, and i is a positive integer.
7. The method for recognizing the voice of parkinsonism according to claim 5, wherein the formula for calculating the overtone deviation value in S34 is: wherein f o,d is the harmonic offset value, f o,i is the ith harmonic frequency in the sounding tremor-removal audio spectrum subset, i is a positive integer, f peak is the peak frequency, and L is the number of harmonic frequencies in the sounding tremor-removal audio spectrum subset.
8. The method for recognizing the voice of the parkinsonism according to claim 1, wherein the multi-layered pronunciation vector processing model in S4 comprises a plurality of first pronunciation processing layers, a plurality of second pronunciation processing layers, a first BiLSTM layers, a second BiLSTM layers, a feature integration layer and a CRF unit;
The input end of each first sounding processing layer is used for inputting a sounding vector, the input end of each second sounding processing layer is used for inputting a sounding vector, the input ends of the first BiLSTM layers are respectively connected with the output ends of the first sounding processing layers, the input ends of the second BiLSTM layers are respectively connected with the output ends of the second sounding processing layers, the input ends of the feature integration layers are respectively connected with the output ends of the first BiLSTM layers and the output ends of the second BiLSTM layers, the output ends of the feature integration layers are connected with the input ends of the CRF units, and the output ends of the CRF units serve as the output ends of the multi-layer sounding vector processing model.
9. The method for recognizing the voice of parkinsonism according to claim 8, wherein the expression of the first sound processing layer is: Wherein y 1 is the output of the first sounding processing layer, tanh is a hyperbolic tangent activation function, x n is the nth element in the sounding vector, ω 1,n is the nth weight in the first sounding processing layer, b 1,n is the nth bias in the first sounding processing layer, and the value range of n is a positive integer ranging from 1 to 3;
the expression of the second pronunciation processing layer is: Where y 2 is the output of the second pronunciation processing layer, sigmoid is the S-type activation function, ω 2,n is the nth weight in the second pronunciation processing layer, and b 2,n is the nth bias in the second pronunciation processing layer.
10. The method for recognizing the voice of parkinsonism according to claim 8, wherein the expression of the feature integration layer is: Wherein H m is the mth integrated feature output by the feature integration layer, H 1,m is the mth feature output by the first BiLSTM layer, H 2,m is the mth feature output by the second BiLSTM layer, ω 1,m is the mth first weight in the feature integration layer, and ω 2,m is the mth second weight in the feature integration layer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202510064918.0A CN119479708B (en) | 2025-01-15 | 2025-01-15 | Speech recognition method for parkinsonism |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202510064918.0A CN119479708B (en) | 2025-01-15 | 2025-01-15 | Speech recognition method for parkinsonism |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN119479708A CN119479708A (en) | 2025-02-18 |
| CN119479708B true CN119479708B (en) | 2025-03-14 |
Family
ID=94587798
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202510064918.0A Active CN119479708B (en) | 2025-01-15 | 2025-01-15 | Speech recognition method for parkinsonism |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119479708B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110070888A (en) * | 2019-05-07 | 2019-07-30 | 颐保医疗科技(上海)有限公司 | A kind of Parkinson's audio recognition method based on convolutional neural networks |
| CN111354338A (en) * | 2020-02-26 | 2020-06-30 | 重庆大学 | Parkinson's speech recognition system optimized by sparse transfer learning based on PSO convolution kernel |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011180308A (en) * | 2010-02-26 | 2011-09-15 | Masatomo Okumura | Voice recognition device and recording medium |
| US9183849B2 (en) * | 2012-12-21 | 2015-11-10 | The Nielsen Company (Us), Llc | Audio matching with semantic audio recognition and report generation |
| CN118116412A (en) * | 2024-03-29 | 2024-05-31 | 重庆科技大学 | Parkinson's disease voice recognition system based on multisource depth domain generalization |
-
2025
- 2025-01-15 CN CN202510064918.0A patent/CN119479708B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110070888A (en) * | 2019-05-07 | 2019-07-30 | 颐保医疗科技(上海)有限公司 | A kind of Parkinson's audio recognition method based on convolutional neural networks |
| CN111354338A (en) * | 2020-02-26 | 2020-06-30 | 重庆大学 | Parkinson's speech recognition system optimized by sparse transfer learning based on PSO convolution kernel |
Also Published As
| Publication number | Publication date |
|---|---|
| CN119479708A (en) | 2025-02-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20010010039A1 (en) | Method and apparatus for mandarin chinese speech recognition by using initial/final phoneme similarity vector | |
| Wand et al. | Deep Neural Network Frontend for Continuous EMG-Based Speech Recognition. | |
| Yerigeri et al. | Speech stress recognition using semi-eager learning | |
| Sharma et al. | Automatic evaluation of song intelligibility using singing adapted STOI and vocal-specific features | |
| Zolnay et al. | Using multiple acoustic feature sets for speech recognition | |
| Salim et al. | Automatic speaker verification system for dysarthric speakers using prosodic features and out-of-domain data augmentation | |
| Zheng et al. | Comparing the performance of classic voice-driven assistive systems for dysarthric speech | |
| CN119479708B (en) | Speech recognition method for parkinsonism | |
| CN115050387A (en) | Multi-dimensional singing playing analysis evaluation method and system in art evaluation | |
| CN113129923A (en) | Multi-dimensional singing playing analysis evaluation method and system in art evaluation | |
| Lekshmi et al. | An acoustic model and linguistic analysis for Malayalam disyllabic words: a low resource language | |
| Nakagawa et al. | Speech, hearing and neural network models | |
| Hossain et al. | Finding emotion from multi-lingual voice data | |
| Nunes | Whispered speech segmentation based on Deep Learning | |
| Ramos-Aguilar et al. | A general overview of language pronunciation analysis based on machine learning | |
| Mun et al. | A speech corpus for chronic kidney disease | |
| Lv et al. | Objective evaluation method of broadcasting vocal timbre based on feature selection | |
| Phuong et al. | Development of high-performance and large-scale vietnamese automatic speech recognition systems | |
| Sahoo et al. | A physical exertion inspired multi-task learning framework for detecting out-of-breath speech | |
| Malik et al. | Efficacy of Current Dysarthric Speech Recognition Techniques | |
| Sehgal | Dysarthric speech analysis and automatic recognition using phase based representations | |
| Singh et al. | A review paper on emotion recognition | |
| El Belghiti et al. | Automatic detection of speech breath events using deep neural networks | |
| Achanta | Multilingual Text-to-Speech Synthesis using Sequence-to-Sequence Neural Networks | |
| BHABAD | SPEECH RECOGNITION & RECTIFICATION FOR ARTICULATORY HANDICAPPED PEOPLE |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |