CN107300971A - The intelligent input method and system propagated based on osteoacusis vibration signal - Google Patents

Abstract

The present invention provides an intelligent input method and system based on bone conduction vibration signal propagation. The intelligent input method includes: S1. Receiving step, the intelligent device receives the vibration signal with a vibration sensor, and performs noise reduction processing on the vibration signal; S2. In the extraction step, the double-threshold endpoint detection method is used to detect and extract the vibration signal segment generated by knocking on the designated part of the human body; S3. The processing step is to extract the signal features, and classify the signal position based on the RNM algorithm. The beneficial effects of the present invention are: the present invention not only solves the problem of the smart watch text input method, but also achieves a high recognition rate, and can input quickly without losing the battery life of the smart watch.

Description

Intelligent input method and system based on bone conduction vibration signal propagation

technical field

The invention relates to the technical field of smart wearable devices, in particular to an intelligent input method and system based on bone conduction vibration signal propagation.

Background technique

In recent years, we have witnessed the rapid development of smart wearable devices. Wearable devices such as smart bracelets, smart earphones, smart glasses, smart helmets and smart watches have become increasingly popular and accepted in people's daily life. With the innovation space of smart phones shrinking and the market growth approaching saturation, smart wearable devices, as the next hot spot in the mobile terminal industry, have been widely recognized by the market and predicted to be an invention that will soon replace mobile phones.

Smart watches are popular as a portable device. However, its compactness and lightness also face unavoidable technical problems. Due to the small screen, only a few keys can be displayed at a time, and the display of other content will be blocked, so the efficiency is very low. At present, there are three main ways for smart watches to realize text input: traditional keyboard, text prediction and speech recognition. But the above methods are either not convenient and flexible enough, or not safe enough. In the presence of environmental noise, the voice recognition rate is difficult to achieve the desired effect, and for the protection of user passwords and other privacy, it is obviously not suitable to use voice input in public places. Although multi-national scientific research teams have developed finger tracking and recognition technology, the user experience of keyboard use has never been compared with the effective, fast and comfortable text input of large-screen smartphones. For example, in 2016, the research team of the University of Washington in the United States achieved millimeter-level precise finger tracking. technology, allowing users to implement handwriting input based on acoustic positioning on mobile devices, but handwriting input is still too slow to meet people's needs. If you want to expand the market demand for smart watches, you must dig deep into its applications and solve the text input problem.

Contents of the invention

The present invention provides an intelligent input method based on bone conduction vibration signal propagation, comprising the following steps:

S1. Receiving step, the smart device receives the vibration signal with the vibration sensor, and performs noise reduction processing on the vibration signal;

S2. The extraction step is to detect and extract the vibration signal segment generated by knocking the designated part of the human body by using the double-threshold endpoint detection method;

S3. A processing step, extracting signal features, and classifying signal positions based on the RNM algorithm.

As a further improvement of the present invention, in the S2. extracting step, include:

Step S21, setting two thresholds, high and low, for the processed signal;

Step S22, when the energy or zero-crossing rate of the signal exceeds the low threshold, the starting point of the knocking signal is initially determined, and when the energy or zero-crossing rate of the signal breaks through the high threshold, the real starting point of the knocking signal is determined; when the signal energy and zero-crossing When the rate is lower than the low threshold at the same time, determine the end of the signal;

Step S23, keeping the data from the start point to the end point, and obtaining the vibration signal segment generated by tapping the back of the hand.

As a further improvement of the present invention, the S3. processing step includes:

Step S31, normalize the segmented signal, extract the Mel-frequency cepstral coefficient, and obtain the signal feature;

Step S32, using the RNM algorithm based on the random subspace and the nearest center point algorithm to classify the signal features, and then determine the tapping position;

Described step S32 comprises:

Step S321, according to the different tapping positions, collect the signal features obtained in step S31 as training samples, and classify them;

Step S322, using the nearest center point algorithm to calculate each type of center point of the training sample;

Step S323, based on the random subspace, the center points of the test sample and the training sample are compared in the subspace multiple times to obtain multiple classification results;

Step S324, using the principle of simple majority voting for the classification results, and setting a certain proportion of votes, and obtaining the final classification results when a majority of votes is obtained and a certain number of votes is reached at the same time;

Step S325, classify the successfully classified samples into new training samples, and recalculate the new class center points of the new training samples.

As a further improvement of the present invention, in the step S321, knock on different positions on the back of the hand, collect the signal features of the step S31 as a training sample, mark the position and divide it into n categories according to the position, and knock m times for each category, where n and m are greater than or equal to 1;

In the step S323, T attributes are randomly sampled from each class center generated in the step S322, and the above operation is repeated Q times to obtain Q subspaces, and each class center of the test sample and the training sample is compared one by one in the subspace The Euclidean distance of the point, find the nearest center point, that is, get the result of Q subspace classification, where T and Q are greater than 1.

As a further improvement of the present invention, in the S2. extraction step, the designated part of the human body is the back of the hand;

The vibration sensor is a piezoelectric ceramic vibration sensor;

In the S1. receiving step, performing noise reduction processing on the vibration signal includes:

Step S11, using a 20Hz Butterworth high-pass filter to filter out the DC component and low-frequency noise;

Step S12, use 800Hz low-pass filter to filter out high-frequency noise.

The present invention also provides an intelligent input system based on bone conduction vibration signal propagation, including:

The receiving module, the smart device uses the vibration sensor to receive the vibration signal, and performs noise reduction processing on the vibration signal;

The extraction module uses the double-threshold endpoint detection method to detect and extract the vibration signal fragments generated by knocking on the designated parts of the human body;

The processing module extracts signal features and classifies signal positions based on the RNM algorithm.

As a further improvement of the present invention, the extraction module includes:

The first extraction module is used to set two thresholds, high and low, for the processed signal;

The second extraction module is used to preliminarily determine the starting point of the knocking signal when the energy or zero-crossing rate of the signal exceeds the low threshold, and determine the real starting point of the knocking signal when the energy or zero-crossing rate of the signal breaks through the high threshold; When the energy and zero-crossing rate are lower than the low threshold at the same time, the end of the signal is determined;

The third extraction module is used to retain the data from the start point to the end point, and obtain the vibration signal segment generated by tapping the back of the hand.

As a further improvement of the present invention, the processing module includes:

The first processing module is used to normalize the segmented signal, extract mel frequency cepstral coefficients, and obtain signal features;

The second processing module is used to classify the signal features using the RNM algorithm based on the random subspace and the nearest center point algorithm, and then determine the knocking position;

The second processing module includes:

The first processing unit is used to collect the signal features obtained in step S31 as training samples according to the different tapping positions, and classify them;

The second processing unit is used to calculate each type of center point of the training sample using the nearest center point algorithm;

The third processing unit is used to compare the center points of the test sample and the training sample in the subspace multiple times based on the random subspace to obtain multiple classification results;

The fourth processing unit is used to use the principle of simple majority voting on the classification results, and set a certain proportion of votes, and obtain the final classification results when a majority of votes is obtained and a certain number of votes is obtained;

The fifth processing unit is used to classify the successfully classified samples into new training samples, and recalculate the new center points of the new training samples.

As a further improvement of the present invention, in the first processing unit, different positions on the back of the hand are tapped, and the signal features obtained in step S31 are collected as training samples, and the positions are marked and divided into n categories according to different positions, and each type of tap m times, where n and m are greater than or equal to 1;

In the third processing unit, T attributes are randomly sampled from each class center generated in the second processing unit, and the above operation is repeated Q times to obtain Q subspaces, and the test samples and training samples are compared one by one in the subspaces The Euclidean distance of the center points of each class, find the nearest center point, that is, get the results of Q subspace classification, where T and Q are greater than 1.

As a further improvement of the present invention,

In the extraction module, the designated part of the human body is the back of the hand;

The vibration sensor is a piezoelectric ceramic vibration sensor;

In the receiving module, performing noise reduction processing on the vibration signal includes:

The first receiving module is used to use a 20Hz Butterworth high-pass filter to filter out DC components and low-frequency noise;

The second receiving module is used to filter out high-frequency noise by using an 800Hz low-pass filter.

The beneficial effects of the present invention are: the present invention not only solves the problem of the smart watch text input method, but also achieves a high recognition rate, and can input quickly without losing the battery life of the smart watch.

Description of drawings

Figure 1 is a schematic diagram of the user typing on the virtual Jiugongge on the back of the hand;

Fig. 2 is a schematic diagram of a piezoelectric ceramic vibration sensor;

Fig. 3 is the structural diagram of piezoelectric ceramic vibration sensor;

Fig. 4 is an original signal waveform diagram;

Figure 5 is an adaptive filtering diagram;

Fig. 6 is a low-pass filtering figure;

Fig. 7 is a flowchart of the RNM algorithm.

detailed description

As shown in Figure 1, the present invention discloses an intelligent input method based on bone conduction vibration signal propagation, including the following steps:

S1. Receiving step, the smart device receives the vibration signal with the vibration sensor, and performs noise reduction processing on the vibration signal;

S2. The extraction step is to detect and extract the vibration signal segment generated by knocking the designated part of the human body by using the double-threshold endpoint detection method;

S3. A processing step, extracting signal features, and classifying signal positions based on the RNM algorithm.

In the S2. extraction step, the designated part of the human body is the back of the hand;

The vibration sensor is a piezoelectric ceramic vibration sensor. Smart devices include smart watches, and the piezoelectric ceramic vibration sensor is built into the smart watch. Figures 2 and 3 show the schematic diagram and structure diagram of the piezoelectric ceramic vibration sensor. Due to the piezoelectric effect, the internal polarity changes, and the voltage change is displayed externally, allowing the operator to tap the back of the hand to collect the vibration signal generated by the tap.

Figure 4 is the waveform diagram of the original signal. It can be seen that the collected original signal has strong anti-interference ability to the outside world and less noise. Figure 5 shows waveforms using adaptive filtering and Figure 6 using low-pass filtering. Low-pass filtering retains more features of the signal, and has a better effect on the subsequent classification of vibration signals with small differences.

In the S1. receiving step, performing noise reduction processing on the vibration signal includes:

Step S11, using a 20Hz Butterworth high-pass filter to filter out the DC component and low-frequency noise;

Step S12, use 800Hz low-pass filter to filter out high-frequency noise.

Include in described S2. extraction step:

Step S21, setting two thresholds, high and low, for the processed signal;

Step S22, when the energy or zero-crossing rate of the signal exceeds the low threshold, the starting point of the knocking signal is initially determined, and when the energy or zero-crossing rate of the signal breaks through the high threshold, the real starting point of the knocking signal is determined; when the signal energy and zero-crossing When the rate is lower than the low threshold at the same time, determine the end of the signal;

Step S23, only keep the data from the start point to the end point, and obtain the vibration signal segment generated by tapping the back of the hand.

In the S3. processing step, extracting signal features includes: initializing the longest start point and end point in the training sample as the unified length of the segmented signal, and normalizing the signals with the same length after segmenting, using the formula: where x is the vibration signal and n is the dimension of the signal. Feature extraction is performed on the normalized signal to reduce the amount of calculation and retain most of the information of the original signal. The feature extraction is the Mel frequency cepstral coefficient of the signal, while retaining the features of the time domain and frequency domain.

Include in described S3. processing step:

Step S31, normalize the segmented signal, extract the Mel-frequency cepstral coefficient, and obtain the signal feature;

Step S32, using the RNM algorithm based on the random subspace and the nearest center point algorithm to classify the signal features, and then determine the tapping position;

As shown in FIG. 7, in the step S32, classifying signal positions based on the RNM algorithm includes: including:

Step S321, according to the different tapping positions, the signal features obtained in the acquisition step S31 are used as training samples, and classified (divided into 9 categories according to the position of the Jiugong grid);

Step S322, using the nearest center point algorithm to calculate each type of center point of the training sample;

Step S323, based on the random subspace, the center points of the test sample and the training sample are compared in the subspace multiple times to obtain multiple classification results;

Step S324, using the principle of simple majority voting for the classification results, and setting a certain proportion of votes, and obtaining the final classification results when a majority of votes is obtained and a certain number of votes is reached at the same time;

Step S325, classify the successfully classified samples into new training samples, and recalculate the new class center points of the new training samples.

In said step S321, knock on different positions on the back of the hand, collect the signal features of step S31 as a training sample, mark the position and divide it into n categories according to the different positions, and knock m times for each category, wherein n and m are greater than or equal to 1;

In the step S323, T attributes are randomly sampled from each class center generated in the step S322, and the above operation is repeated Q times to obtain Q subspaces, and each class center of the test sample and the training sample is compared one by one in the subspace The Euclidean distance of the point, find the nearest center point, that is, get the result of Q subspace classification, where T and Q are greater than 1.

The invention also discloses an intelligent input system based on bone conduction vibration signal propagation, including:

The receiving module, the smart device uses the vibration sensor to receive the vibration signal, and performs noise reduction processing on the vibration signal;

The extraction module uses the double-threshold endpoint detection method to detect and extract the vibration signal fragments generated by knocking on the designated parts of the human body;

The processing module extracts signal features and classifies signal positions based on the RNM algorithm.

In the extraction module include:

The first extraction module is used to set two thresholds, high and low, for the processed signal;

The second extraction module is used to preliminarily determine the starting point of the knocking signal when the energy or zero-crossing rate of the signal exceeds the low threshold, and determine the real starting point of the knocking signal when the energy or zero-crossing rate of the signal breaks through the high threshold; When the energy and zero-crossing rate are lower than the low threshold at the same time, the end of the signal is determined;

The third extraction module is used to keep only the data from the start point to the end point, and cut the rest into segments to obtain the vibration signal segments generated by tapping the back of the hand.

Included in the processing module:

The first processing module is used to normalize the segmented signal, extract mel frequency cepstral coefficients, and obtain signal features;

The second processing module is used to classify the signal features using the RNM algorithm based on the random subspace and the nearest center point algorithm, and then determine the knocking position;

The second processing module includes:

The first processing unit is used to collect the signal features obtained in step S31 as training samples according to the different tapping positions, and classify them;

The second processing unit is used to calculate each type of center point of the training sample using the nearest center point algorithm;

The third processing unit is used to compare the center points of the test sample and the training sample in the subspace multiple times based on the random subspace to obtain multiple classification results;

The fourth processing unit is used to use the principle of simple majority voting on the classification results, and set a certain proportion of votes, and obtain the final classification results when a majority of votes is obtained and a certain number of votes is obtained;

The fifth processing unit is used to classify the successfully classified samples into new training samples, and recalculate the new center points of the new training samples.

In the first processing unit, knock on different positions on the back of the hand, collect the signal features of step S31 as a training sample, mark the position and divide it into n categories according to the position, and knock m times for each category, where n and m are greater than or equal to 1;

In the third processing unit, T attributes are randomly sampled from each class center generated in the second processing unit, and the above operation is repeated Q times to obtain Q subspaces, and the test samples and training samples are compared one by one in the subspaces The Euclidean distance of the center points of each class, find the nearest center point, that is, get the results of Q subspace classification, where T and Q are greater than 1.

In the extraction module, the designated part of the human body is the back of the hand; the vibration sensor is a piezoelectric ceramic vibration sensor.

In the receiving module, performing noise reduction processing on the vibration signal includes:

The first receiving module is used to use a 20Hz Butterworth high-pass filter to filter out DC components and low-frequency noise;

The second receiving module is used to filter out high-frequency noise by using an 800Hz low-pass filter.

The invention embeds a tiny and thin piezoelectric sensor on the smart watch, and the piezoelectric sensor can realize the conversion from mechanical energy to electrical energy. A nine-grid keyboard is virtualized on the back of the hand. When the finger taps the grid at different positions, the mechanical wave will spread out in all directions and reflect back when it hits an object. Therefore, based on the broadcast nature of mechanical waves, the piezoelectric sensor will receive mechanical wave signals with different multipath propagation at one time. This mechanical wave signal spreads into the air through the voice signal on the one hand, and on the other hand spreads inside the hand, and the so-called bone conduction. This part of the mechanical wave signal is not affected by environmental noise, and can be better received by the piezoelectric sensor, converted into an electrical signal and processed by the controller of the smart watch. Because the mechanical wave multipath effect produced by different grids is different, the signals received by the smart watch are different. Using this difference, combined with the classification algorithm of machine learning, each button of the Jiugong grid can be classified. Thus, a text input method and system for smart watches based on back bone conduction technology can be realized.

The present invention has a built-in piezoelectric ceramic vibration sensor in the smart watch. For the first time, the collected vibration of knocking on the back of the hand is used as the text input mode of the smart watch, and the back of the hand is used as a virtual large screen of the small screen of the smart watch, which is convenient for text input; the collected It is the vibration signal after the multi-path propagation on the human hand behind the percussion hand. It has strong anti-interference ability, and after the signal is denoised, segmented, normalized, and the Mel frequency cepstral coefficient is extracted, the invented RNM algorithm for classification, the recognition rate reached 92%. The complexity of the algorithm used here is only linear order, so the fast input of text can be realized. In addition, the piezoelectric ceramic vibration sensor consumes extremely low power and will not significantly reduce the battery life of the smart watch.

The invention not only solves the problem of the text input method of the smart watch, but also achieves a high recognition rate, and can input quickly without losing the battery life of the smart watch.

The invention has the advantages of low hardware cost, simple system and convenient use, and can simply and quickly realize the input of the smart watch based on the bone conduction vibration signal transmission of the back of the hand.

The above content is a further detailed description of the present invention in conjunction with specific preferred embodiments, and it cannot be assumed that the specific implementation of the present invention is limited to these descriptions. For those of ordinary skill in the technical field of the present invention, without departing from the concept of the present invention, some simple deduction or replacement can be made, which should be regarded as belonging to the protection scope of the present invention.

Claims (10)

1. An intelligent input method based on bone conduction vibration signal propagation, is characterized in that, comprises the steps: S1. Receiving step, the smart device receives the vibration signal with the vibration sensor, and performs noise reduction processing on the vibration signal; S2. The extraction step is to detect and extract the vibration signal segment generated by knocking the designated part of the human body by using the double-threshold endpoint detection method; S3. A processing step, extracting signal features, and classifying signal positions based on the RNM algorithm. 2. the intelligent input method according to claim 1, is characterized in that, comprises in described S2. extracting step: Step S21, setting two thresholds, high and low, for the processed signal; Step S22, when the energy or zero-crossing rate of the signal exceeds the low threshold, the starting point of the knocking signal is initially determined, and when the energy or zero-crossing rate of the signal breaks through the high threshold, the real starting point of the knocking signal is determined; when the signal energy and zero-crossing When the rate is lower than the low threshold at the same time, determine the end of the signal; Step S23, keeping the data from the start point to the end point, and obtaining the vibration signal segment generated by tapping the back of the hand. 3. the intelligent input method according to claim 1, is characterized in that, comprises in the described S3. processing step: Step S31, normalize the segmented signal, extract the Mel-frequency cepstral coefficient, and obtain the signal feature; Step S32, using the RNM algorithm based on the random subspace and the nearest center point algorithm to classify the signal features, and then determine the tapping position; Described step S32 comprises: Step S321, according to the different tapping positions, collect the signal features obtained in step S31 as training samples, and classify them; Step S322, using the nearest center point algorithm to calculate each type of center point of the training sample; Step S323, based on the random subspace, the center points of the test sample and the training sample are compared in the subspace multiple times to obtain multiple classification results; Step S324, using the principle of simple majority voting for the classification results, and setting a certain proportion of votes, and obtaining the final classification results when a majority of votes is obtained and a certain number of votes is reached at the same time; Step S325, classify the successfully classified samples into new training samples, and recalculate the new class center points of the new training samples. 4. The intelligent input method according to claim 3, characterized in that: In said step S321, knock on different positions on the back of the hand, collect the signal features of step S31 as a training sample, mark the position and divide it into n categories according to the different positions, and knock m times for each category, wherein n and m are greater than or equal to 1; In the step S323, T attributes are randomly sampled from each class center generated in the step S322, and the above operation is repeated Q times to obtain Q subspaces, and each class center of the test sample and the training sample is compared one by one in the subspace The Euclidean distance of the point, find the nearest center point, that is, get the result of Q subspace classification, where T and Q are greater than 1. 5. The intelligent input method according to claim 1, characterized in that: In the S2. extraction step, the designated part of the human body is the back of the hand; The vibration sensor is a piezoelectric ceramic vibration sensor; In the S1. receiving step, performing noise reduction processing on the vibration signal includes: Step S11, using a 20Hz Butterworth high-pass filter to filter out DC components and low-frequency noise; Step S12, use 800Hz low-pass filter to filter out high-frequency noise. 6. An intelligent input system based on bone conduction vibration signal propagation, characterized in that it comprises: The receiving module, the smart device uses the vibration sensor to receive the vibration signal, and performs noise reduction processing on the vibration signal; The extraction module uses the double-threshold endpoint detection method to detect and extract the vibration signal fragments generated by knocking on the designated parts of the human body; The processing module extracts signal features and classifies signal positions based on the RNM algorithm. 7. The intelligent input system according to claim 6, characterized in that, comprising in the extraction module: The first extraction module is used to set two thresholds, high and low, for the processed signal; The second extraction module is used to preliminarily determine the starting point of the knocking signal when the energy or zero-crossing rate of the signal exceeds the low threshold, and determine the real starting point of the knocking signal when the energy or zero-crossing rate of the signal breaks through the high threshold; When the energy and zero-crossing rate are lower than the low threshold at the same time, the end of the signal is determined; The third extraction module is used to retain the data from the start point to the end point, and obtain the vibration signal segment generated by tapping the back of the hand. 8. The intelligent input system according to claim 6, wherein the processing module comprises: The first processing module is used to normalize the segmented signal, extract mel frequency cepstral coefficients, and obtain signal features; The second processing module is used to classify the signal features using the RNM algorithm based on the random subspace and the nearest center point algorithm, and then determine the knocking position; The second processing module includes: The first processing unit is used to collect the signal features obtained in step S31 as training samples according to the different tapping positions, and classify them; The second processing unit is used to calculate each type of center point of the training sample using the nearest center point algorithm; The third processing unit is used to compare the center points of the test sample and the training sample in the subspace multiple times based on the random subspace to obtain multiple classification results; The fourth processing unit is used to use the principle of simple majority voting on the classification results, and set a certain proportion of votes, and obtain the final classification results when a majority of votes is obtained and a certain number of votes is obtained; The fifth processing unit is used to classify the successfully classified samples into new training samples, and recalculate the new center points of the new training samples. 9. The intelligent input system according to claim 8, characterized in that: In the first processing unit, knock on different positions on the back of the hand, collect the signal features of step S31 as a training sample, mark the position and divide it into n categories according to the position, and knock m times for each category, where n and m are greater than or equal to 1; In the third processing unit, T attributes are randomly sampled from each class center generated in the second processing unit, and the above operation is repeated Q times to obtain Q subspaces, and the test samples and training samples are compared one by one in the subspaces The Euclidean distance of the center points of each class, find the nearest center point, that is, get the results of Q subspace classification, where T and Q are greater than 1. 10. The intelligent input system according to claim 6, characterized in that: In the extraction module, the designated part of the human body is the back of the hand; The vibration sensor is a piezoelectric ceramic vibration sensor; In the receiving module, performing noise reduction processing on the vibration signal includes: The first receiving module is used to use a 20Hz Butterworth high-pass filter to filter out DC components and low-frequency noise; The second receiving module is used to filter out high-frequency noise by using an 800Hz low-pass filter.