CN107411935A - A kind of multi-mode brain-computer interface control method for software manipulators in rehabilitation - Google Patents

Abstract

The invention discloses a multi-mode brain-computer interface control method for a software rehabilitation manipulator. When a user performs active rehabilitation training in daily life, a multi-source signal control mode is constructed by synchronously collecting EEG, EMG and EoG signals , use the double blink of EOG signal to switch brain/muscle/oculoelectric mode in turn, and preprocess the signal of the corresponding mode, feature extraction, pattern recognition, and the classification results are converted into control signals through real-time processing of the lower computer and sent to the software The rehabilitation manipulator drives the user's fingers to complete relevant training tasks, and its control results are fed back to the user in real time, so as to achieve the purpose of active rehabilitation training for the user. The multi-mode brain-computer interface control method of the present invention has the advantages of multiple active control modes, multiple control commands, and strong real-time performance. The software rehabilitation manipulator has high safety, good comfort, low noise, light weight, portability, and low cost, and has great advantages. prospects for clinical application.

Description

A multi-mode brain-computer interface control method for soft rehabilitation manipulators

technical field

The invention belongs to the interdisciplinary technical field of rehabilitation medicine and neurological function information, and relates to a multi-mode brain-computer interface control method for a software rehabilitation manipulator.

Background technique

In modern society, with the arrival of an aging society, a large number of stroke users cause motor dysfunction. In addition, the number of users with nerve or limb injuries due to work-related injuries, traffic accidents, wars and diseases has also increased significantly, which brings great inconvenience to users' life and work. At present, brain injury (stroke, traumatic brain injury, brain tumor, spinal cord injury, etc.) is one of the main reasons for the loss of hand motor function. Studies have shown that the 3 months after the initial onset is the golden period for functional recovery of brain-injured users. To a certain extent, intensive training can repair damaged nerves, which is beneficial to the recovery of user's hand motor function. However, the traditional treatment method is to complete the correct action under the one-on-one guidance of the rehabilitation professional physical therapist to the rehabilitation user. This method is labor-intensive, inefficient and labor-intensive. However, there is a physical therapy in the training process. Insufficient teachers, users do not actively cooperate, cannot guarantee timely and quantitative training, and the lack of human-computer interaction in the training process, the lack of attractive training content and other factors seriously restrict the training effect. A large number of clinical medicine shows that the effect of active rehabilitation of users during training is more significant than that of passive rehabilitation. Therefore, users actively participate in training in daily life, increase the interest of human-computer interaction and training content, and can reshape the central nervous system. And better restore the damaged nerves, thereby greatly improving the rehabilitation effect of the user's hand motor function.

The motor function of the hand occupies a large proportion in daily life. The physiological structure of the hand is also the most complicated part, and the joints are the most concentrated, which makes the prosthesis for hand motor function rehabilitation relatively complicated. The traditional mechanical rehabilitation hand has many defects such as poor safety, inability to fully fit the fingers, easy pressure causing pain to the user, excessive weight, and loud noise, while the soft rehabilitation robot has a multi-segment joint bending similar to a human finger , the hollow structure composed of pneumatically driven soft materials drives the fingers to move passively. At the same time, it has the advantages of high safety, high comfort, low noise and light weight and portability. secondary damage". Due to the above advantages of the software rehabilitation manipulator, the user is willing and active to participate in the action tasks, and is also easy to accept and actively control the deformation of the software rehabilitation manipulator, so as to assist or assist the user's fingers to complete the basic movements of daily life, and at the same time meet the user's daily needs. The basic needs of life (such as drinking water, eating, etc.), so as to achieve the purpose of training in family life.

Brain-computer interface technology is a special communication channel between the human brain and external devices. By extracting the user's motion intention, the brain can actively control the software rehabilitation manipulator. The main source of the active control signal is the electromyography signal (EMG signal). , EEG signal (EEG signal) and oculoelectric signal (EOG signal) and other brain/muscle/oculoelectric signals. The bionic prosthesis based on myoelectricity controls the movement of the prosthesis by detecting the EMG signals on the user's limbs. It has fast response and natural movements. However, the performance of nerve endings in stroke users is weakened, resulting in low action recognition rate and affecting myoelectricity. Prosthetic applications. EEG-based bionic prosthetics help users with neuromuscular impairments to actively control prosthetic movements through EEG signals, but EEG prosthetics respond slowly and have low discrimination against fine hand movements. The bionic prosthesis using electro-oculogram has many advantages such as fast response, strong applicability, simple and convenient operation, and low cost. However, long-term control of electro-oculogram prosthesis by trained users will cause eye fatigue, dry eyes and many other discomforts. The above-mentioned methods for actively controlling bionic prostheses using a single mode of EEG, EMG or EEG already exist. In terms of dual-mode control, the method of actively controlling bionic prostheses has also emerged in the combination of two modes in brain/muscle/ocular electrical signals. However, regardless of the single-mode or dual-mode active control method, the generated control commands are not many, the real-time control is not strong, and it cannot meet the needs of active control of basic hand movements in daily life. At present, no method has been found to actively control the software rehabilitation manipulator using the brain-computer interface of the three modes of brain/muscle/eye electricity.

Contents of the invention

The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a multi-mode brain-computer interface control method for software rehabilitation manipulators. To obtain the user's active movement intention and convert it into the control command of the software rehabilitation manipulator, so as to achieve the purpose of assisting the user to perform hand training in daily life.

In order to achieve the above object, the present invention adopts the following technical solutions to achieve:

A multi-mode brain-computer interface control method for a software rehabilitation manipulator, comprising the following steps:

1) Synchronously collect brain/muscle/oculoelectric signals to build a multi-source signal control mode;

2) Extract the motion intention according to the brain/muscle/oculogram signal during the time period when the user generates the motion idea, and use the prompt sound or the muscle/oculograph signal feature to divide the corresponding task event;

3) Select different control modes according to the user's wishes, and use double blinking to switch the control mode;

4) Perform preprocessing, feature extraction, and pattern recognition on the brain/muscle/oculograph signals of a task event under different control modes to decode the user's movement intention in real time;

5) Divide different control commands in real time according to the decoded motion intention, and convert them into control commands through the lower computer in real time and send them to the air pump control unit, and then control the inflation and deflation of the software rehabilitation manipulator.

The further improvement of the present invention is:

In step 1), the brain/muscle/oculoelectric signals include EMG signals, EEG signals and EOG signals.

In step 2), the brain/muscle/oculogram signal within the time period generated by each movement intention is called a task event; for the EEG pattern, EMG pattern or Oculogram pattern, extract the corresponding EEG task event, EMG task event or oculograph task event;

a. In the EEG mode, when the EEG signal is collected, the computer gives an initial prompt tone, and then the EEG signal within 2-4 seconds of continuous motor imagery is intercepted as an EEG task event;

b. In the myoelectric mode, according to the addition of the average absolute value of the EMG signal, the start and end points of the action are judged by setting the threshold. The average absolute value of the EMG signal starts from being greater than the set threshold to the time period when it is less than the set threshold. The EMG signal in is a myoelectric task event;

c. In the electro-oculogram mode, when the EOG signal is collected online, the low-frequency component of the EOG signal is obtained by wavelet transform. If the component is greater than the set threshold, it is determined that the position is a conscious blink. If the two positions are less than the length threshold, the blink is double. Blink, otherwise it is a single blink; if the component exceeds the threshold range, it is regarded as a pulse generated by saccade, and the position of left saccade and right saccade is found; EOG within 0.1-0.4s before and after the blink position or 0.2-0.3s before and after the saccade position The signal is an oculoelectric task event.

In step 3), the user switches between the brain/muscle/EoG mode in turn by double blinking; first enter the default EoG mode, judge whether this mode can realize the action task, and then enter this mode and complete the corresponding action task; otherwise, switch the brain/muscle/eye electricity mode sequentially by double blinking until it switches to the mode that can complete the action task.

In step 4), the processing of EMG signal, EEG signal and EOG signal is specifically as follows:

EOG signal processing:

Based on wavelet transform and EOG signal in the vertical direction to locate the blink position, and then process and identify conscious single blink and double blink in an oculoelectric task event according to the threshold value; Carry out identification and positioning, and then correct the identified left glance/right glance signal according to the threshold method. By integrating the pulse amplitude in the horizontal and vertical directions, the identification and classification are carried out according to the two values of large amplitude and small amplitude. When the component is greater than the set When the threshold is fixed, it is determined that the position is a left saccade, otherwise, the position is a right saccade, thereby completing the real-time detection of a left saccade/right saccade in an oculoelectric task event;

EEG signal processing:

The power feature selection method based on wavelet analysis extracts the synchronization and desynchronization features of EEG signals, and realizes the classification of left-hand motor imagery events or right-hand motor imagery events in an EEG task event through the support vector machine classification algorithm;

EMG signal processing:

By collecting the EMG signal of the user's upper limb forearm, the feature values of the average absolute value MAV, the number of zero-crossing points ZC, the number of slope changes SSC, the waveform length WL, and the average absolute value slope MAVS are respectively extracted in the time domain, and are classified by a support vector machine. The algorithm performs feature recognition on the fusion of five time-domain statistical eigenvalues to realize online classification of hand training actions.

In step 5), the decoded motion intention is divided into different control commands in real time by the upper computer, and then the corresponding control instructions are sent to the lower computer, and the air pump control unit is used to adjust the flow of the corresponding passage of the solenoid valve of the two-position three-way valve. control the inflation and deflation of the soft rehabilitation manipulator.

Compared with the prior art, the present invention has the following beneficial effects:

The present invention controls the software rehabilitation manipulator based on collecting brain/muscle/oculoelectric signals and identifying movement intentions, thereby assisting users to perform finger movement tasks anytime and anywhere in daily life, and can also independently customize corresponding movement tasks and complete them. The basic movements in daily life, while meeting the basic needs of users and mobilizing their enthusiasm for active training, so as to achieve better rehabilitation training effect. The present invention can be used for hand training for users who have lost or lacked hand motor function due to brain damage or accidents, etc., can help users complete active training tasks, realize brain damage or assist users in active control in daily life The software rehabilitation manipulator drives the user's hand to achieve the purpose of rehabilitation training. At the same time, the invention has the advantages of multiple active control modes, multiple control commands, and strong real-time performance. The software rehabilitation manipulator has the characteristics of high safety, good comfort, low noise, light weight, portability, and low cost, and has great clinical application prospects.

Description of drawings

Fig. 1 is a schematic diagram of the system of the user's active control software rehabilitation manipulator in the present invention;

Fig. 2 is a technical roadmap for the user to actively control the software rehabilitation manipulator in the present invention;

Fig. 3 is the electrode connection diagram of the brain/muscle/oculoelectric signal of the present invention; wherein, (a) is the layout diagram of EEG/eye electrodes, and (b) is the layout diagram of EMG electrodes;

Fig. 4 is the task event division schematic diagram of brain/muscle/oculoelectric signal of the present invention; Wherein, (a) is the task event division schematic diagram of EEG signal, (b) is the task event division schematic diagram of EMG signal, (c) is EOG signal Schematic diagram of the division of task events;

Fig. 5 is a flow chart of the signal processing algorithm for realizing software rehabilitation manipulator control in the present invention;

Fig. 6 is a schematic diagram of the control system of the software rehabilitation manipulator of the present invention.

detailed description

The present invention is described in further detail below in conjunction with accompanying drawing:

Referring to Figures 1-6, the multi-mode brain-computer interface control method for the software rehabilitation manipulator of the present invention includes the following steps:

1) During training, the user wears EEG electrodes on the head, EEG electrodes on the eyes and EMG electrodes on the forearm of the upper limbs. The acquisition of EEG signals and EOG signals shares an EEG amplifier, while the acquisition of EMG signals uses EMG amplifier, through the development platform of EEG amplifier and EMG amplifier, and using the Windows platform to design and develop a set of human-computer interaction control interface based on VC++, to realize the synchronous online acquisition of EEG signal, EOG signal and EMG signal ;

2) When the user wants to actively control the software rehabilitation manipulator according to his own wishes, it is necessary to extract the user's movement intention according to the corresponding time period of the brain/muscle/oculoelectric signal to generate the idea, and within the time period of each movement intention The corresponding brain/muscle/oculogram signal is called a task event. In the EEG mode, the computer gave an initial prompt tone while collecting EEG signals, and then intercepted the EEG signals within the duration of motor imagery (2-4 seconds) as a task event. In the EMG mode, according to the addition of the average absolute value of the EMG signal, the start and end points of the action are judged by setting the threshold. The EMG signal is a mission event. In the electro-oculogram mode, when the EOG signal is collected online, the low-frequency component of the EOG signal is obtained by wavelet transform. If the component is greater than the set threshold, it is determined that the position is a conscious blink. If the two positions are less than the length threshold, the blink is a double blink. Otherwise it is a single blink. If the component exceeds the threshold range, it is regarded as the pulse generated by the saccade, and the location of the left and right saccades is found. The EOG signal before and after the blink position (0.1-0.4 seconds) or before and after the glance position (0.2-0.3 seconds) is a task event;

3) When the user actively controls the software rehabilitation manipulator during the training process, the brain/muscle/eye electricity multi-mode can be switched sequentially by double blinking. First enter the default EEG mode, judge whether this mode can realize the action task, and can complete the relevant task, then enter this mode and complete the corresponding action task, otherwise, switch the brain/muscle/Eogle mode in turn by double blinking until it is able to switch Until the mode of the action task is completed;

4) In terms of EOG signal processing, the position of the blink is located based on the EOG signal in the vertical direction based on wavelet transform, and then the conscious single blink and double blink are identified according to the threshold value processing and within a task event. Similarly, based on wavelet transform and differential operation, the left saccade/right saccade signal is identified and positioned, and then the identified left saccade/right saccade signal is corrected according to the threshold method. When the component is greater than the set threshold, it is determined that the position is a left glance, otherwise, the position is a right glance, so as to complete the real-time detection of left glance/right glance in a task event ;

5) In terms of EEG signal processing, the power feature selection method based on wavelet analysis extracts the synchronization and desynchronization features of EEG signals, and realizes the left-hand motor imagery event or right-hand motor imagery event within a task event through the support vector machine classification algorithm (SVM). Classification of motor imagery events, so that the user can use the motion intention to control the software rehabilitation manipulator online and drive the user's fingers to complete the corresponding rehabilitation movement;

6) In terms of EMG signal processing, by collecting the EMG signal of the forearm of the user's upper limbs, the average absolute value (MAV), the number of zero crossing points (ZC), the number of slope changes (SSC), and the waveform length (WL) are respectively extracted in the time domain. and average absolute value slope (MAVS) and other EMG signal eigenvalues, and use the support vector machine classification algorithm to perform feature recognition on the fusion of five time-domain statistical eigenvalues, and realize the online classification of hand training actions commonly used in daily life .

Principle of the present invention is as follows:

The soft rehabilitation manipulator is made of soft materials poured into the hand mold to form a hollow cavity structure. Fibers and strain-limiting layers are evenly arranged on the periphery of it. The bending deformation and elongation deformation of the hollow cavity structure are combined to design a humanoid finger. However, it is more convenient for the pneumatic actuator to achieve linear displacement, and the compressibility of the gas ensures the flexibility of the actuator, so the pneumatic actuator is safe, convenient, and pollution-free. A trachea is embedded in the hollow cavity of the humanoid finger, and the soft rehabilitation manipulator inflates and deflates the trachea of the humanoid finger to drive the user's finger to perform corresponding action tasks;

The classification results of the brain/muscle/oculoelectric signals by the upper computer are converted into control instructions, and then the corresponding control instructions are sent to the lower computer, and the software rehabilitation manipulator is controlled by adjusting the on-off of the corresponding passage of the solenoid valve of the two-position three-way valve The user can also adjust the proportional valve in the control unit to adjust the air pressure to control the force level required by the soft manipulator according to the degree of hand rehabilitation in daily life, so that the user can actively control the air pressure drive and control the software rehabilitation manipulator. Inflate, deflate, and air volume of different fingers to flexibly control the motion deformation of the pneumatic software rehabilitation manipulator and drive the user's fingers to perform corresponding rehabilitation exercises;

Combining the commonly used basic forms of hand motor function with the basic hand movements in daily life and the frequency of use, select the most commonly used and most frequently used movement tasks, and use common hands to train basic movements (grip, pull, push, loosen). opening and hitting) or by adjusting the control unit of the soft rehabilitation manipulator to customize the training actions to meet the basic needs of the user in daily life.

Example:

Referring to Fig. 1, the present invention proposes a multi-mode brain-computer interface control method for software rehabilitation manipulators. Users do not need to be accompanied by rehabilitation professional physical therapists in hospitals, but carry out active training at home in daily life. A multi-source signal control mode is constructed by synchronously collecting three kinds of brain/muscle/oculoelectric signals (EEG signal, EMG signal and EOG signal). Select different control modes according to the user's wishes, and use the double blinking of the EOG signal to switch the brain/muscle/oculoelectric mode in turn, and extract the user's movement intention from the brain/muscle/oculoelectric signals under the control mode. Through brain/muscle/oculoelectric signal preprocessing, feature extraction, and pattern recognition, different control commands are classified in real time, and processed in real time by the lower computer into control commands and sent to the air pump control unit. According to the solenoid valve of the two-position three-way valve The inflation and deflation of the software rehabilitation manipulator is controlled by the connection and disconnection of the corresponding pathways, so as to drive the user's fingers to complete relevant action tasks, and the control results are fed back to the training user in real time, so as to realize the purpose of active rehabilitation training for the user.

As shown in Figure 2, the user wears 4-lead EEG electrodes on the head (including 1-lead ground electrode and 1-lead reference electrode on the mastoid behind the right ear), 4-lead eye electrodes on the eyes, and 8-lead electrodes on the forearm of the upper limbs. The EMG electrodes collect EEG/EMG/EEG signals at the same time. If the user is using the software rehabilitation manipulator system for the first time, click the training button on the human-computer interaction control interface to enter the EEG mode, EMG mode and EEG mode in turn. The corresponding training in the EEG mode, the corresponding brain/muscle/oculoelectric signal enters the training model, and the training parameters of the respective training models are obtained, and the training parameters are passed to the training system to complete the EEG mode, EMG mode and Oculoelectric mode respectively. Model training in . After the model training is over, you can click the test button on the human-computer interaction control interface to enter the testing stage of the model. In the test phase, firstly enter the default electrooculogram mode, automatically divide a task event by the threshold value of the EOG signal, and then identify the left saccade and right saccade in the EOG signal through the threshold value processing method and the threshold value after model training, and put the corresponding The control command is converted into the control command S1/S2, and the corresponding control command is converted into the control signal of the software rehabilitation manipulator through the lower computer. According to the actual situation, the user can switch to the EMG mode by double blinking. If it is not the desired control mode, blink again to switch to the EEG mode. At most two double blinks can complete any of the three modes of brain/muscle/eye electricity mode. switch. After the user enters the EMG mode by double blinking, a task event is divided according to the threshold value of the average absolute value of the EMG signal, and the characteristic values of five EMG signals such as MAV, ZC, SSC, WL and MAVS are extracted according to the collected EMG signals , and use the support vector machine to identify the user's motion intentions online through the five time-domain statistical eigenvalues, and convert the motion intentions classified online into six basic hand movement control commands S5/S6/S7/S8 /S9/S10, the lower computer automatically converts into the motion signal of the active control software rehabilitation manipulator. The user enters the EEG mode by double blinking. When the computer makes a "beep", the user starts to imagine the movement of the left and right hands and the time for motor imagination is longer than 2 seconds. The EEG signal corresponding to this period after 2s. According to the phenomenon of synchronization and desynchronization, the power of the EEG signal in the motor area changes, and the change of the power spectrum can be used to distinguish whether the subjects imagined the left hand or the right hand. Extract the functional spectrum in the EEG signal through wavelet changes, and use the support vector machine to identify the user's motion intention online, convert the classified motion intention into control instructions S3/S4, and then control the software rehabilitation manipulator to drive the user's hand to do corresponding exercises sports.

Figure 3 shows the connection diagram of the user's brain/muscle/oculoelectric signal electrodes. According to Figure 3(a), the user’s head is connected with 6-lead EEG electrodes. The electrodes are distributed in the brain areas: C3 and C4 in the brain motor area, the ground electrode GND placed at the upper center of the forehead, and the right ear. The mastoid reference electrode (A2, on the mastoid behind the ear), the EEG electrode position adopts the 10-20 system electrode method. The user's eyes are connected to the 4-lead eye electrode by bipolar lead connection method, and the two electrodes in the horizontal direction are respectively placed on the horizontal line of the double outer canthus of the eyeball, about 10mm away from the outer canthus of the eye (HR and HL) Two electrodes in the vertical direction were respectively placed on the central line of the eye, about 30mm away from the pupil (VU and VL). The wet electrode method is used uniformly for EEG acquisition and oculoelectric acquisition. The electrode is a bowl-shaped silver/silver chloride (Ag/AgCl) alloy electrode, and conductive paste is injected into it. The impedance of all electrodes is less than 5kΩ to improve the conductivity. Both use a unified reference electrode (A2) and ground electrode (GND). The EEG signal and EOG signal use the same NuAmps EEG amplifier (NuAmp, Neuroscan, Inc.), the sampling frequency is 500Hz, and the supporting host computer system is scan 4.5 . According to Figure 3(b), the contraction and relaxation of the forearm muscles of the user's upper limbs will generate weak biovoltage signals, and the finger movement can be controlled by identifying the muscle fibers of the finger movement. The acquisition of EMG signals is to use the MYO armband that supports more channels of sensors and has better integration as a gesture control armband (ThalmicLabs, a Canadian startup company). The sampling frequency is 200Hz. The sensor contains 8 surface electromyography sensors, 1 An accelerometer, a gyroscope, and 8 EMG signal sensors are evenly spaced around the forearm of the user's upper limb.

As shown in FIG. 4 , a schematic diagram of task event division of the brain/muscle/oculoelectric signal of the present invention is described. When the brain-computer interface system extracts the user's motion intention online, it must clearly judge the start and end points of the motion intention, and extract and use the brain/muscle/oculogram signals in the corresponding time period when the corresponding brain/muscle/oculogram signals generate ideas. the player's movement intention. As shown in the picture (a) in the EEG mode, when the EEG signal is collected, the computer will give a start beep (“beep”). After the beep ends 2s, a task event will be completed, and the beep will be saved from the beginning to the beep. EEG signal over a period of 2 s. As shown in (b) in the EMG mode, add the time-domain characteristic MAV of the 8-channel EMG signal of the 10 experiments of the MYO armband, and compare it with the MAV of the non-moving EMG signal, and set the threshold to judge the six kinds of hand movements. The start and end points of the internal movement, the time period of the 8-lead EMG signal from the beginning to the end of the movement is a task event. As shown in (c) in the oculoelectric mode, the 10-layer 'db4' discrete wavelet decomposition is performed on the original EOG signal and the low-frequency approximation signal is reconstructed to obtain the low-frequency component of the signal, and the low-frequency component is subtracted from the original vertical EOG signal , That is to obtain the approximate multi-peak pulse signal of the return to zero line, and perform the first threshold processing on it, set 1 when it is greater than the threshold T1 (value 80uV), and set 0 if it is less than T1, then the signal becomes a rectangular wave signal and each high voltage The flat interval is the approximate blink interval including the exact location of the blink. In order to identify this interval, the following difference operation is performed on the rectangular wave, and the second threshold value processing is performed on it. If it is greater than the threshold T2 (value 540uV), it is set to 3, and if it is less than T2, it is set to 1, so as to identify single blink and double blink. The 1 on the right side of the figure represents unconscious blinking, 2 represents conscious single blinking, and 3 represents double blinking. The EOG signal before and after the blink position (0.1-0.4 seconds) is a task event. Set the threshold to 30. If it exceeds the threshold range, it will be regarded as the pulse generated by the saccade, and the positions of the left and right saccades will be found. The EOG signal before and after the saccade position (0.2-0.3 seconds) is a task event.

Figure 5 is a flow chart of the signal processing algorithm for realizing software rehabilitation manipulator control in the present invention. For the EEG data of motor imagery, it is amplified by the EEG amplifier, and the collected EEG signal is preprocessed accordingly, and then the Mu frequency band is obtained through band-pass filtering (8-14Hz), and then after 7 layers of wavelet transformation, the obtained The wavelet coefficient energy, the sum of the energy of the C3 channel and the C4 channel is classified and tested in the test mode of the support vector machine, and the left hand motor imagery (control command: S3) and the right hand motor imagery (control command: S4) are respectively obtained. Similarly, the oculoelectric data of blinking and saccade are amplified by the EEG amplifier, and the DC component is removed from the collected EOG signal, and then the EOG signal is decomposed by 10-layer 'db4' discrete wavelet and the low-frequency approximation signal is reconstructed. The low-frequency component of the signal is obtained, and after differential motion calculation, single blink and double blink can be classified according to the threshold method. Similarly, the corresponding position and classification of left saccade and right saccade can be identified (control command: S1/S2). Finally, the data of the rehabilitation action is amplified by the myoelectric amplifier, and the collected EMG signal is preprocessed accordingly, and then the corresponding myoelectric frequency band is obtained through band-pass filtering (45-195Hz), and five time-domain statistical features are used. (MAV, ZC, SSC, WL, MAVS) as the classification standard, the five features are classified and tested in the test mode of the support vector machine, and six basic hand movements (including fist, palm adduction, palm adduction, etc.) are respectively obtained. Abduction, five fingers spread, thumb and middle finger double tap, no action, etc.) control commands S5/S6/S7/S8/S9/S10. The classification results of the brain/muscle/oculoelectric patterns are converted into corresponding control commands by the lower computer into control signals of the software rehabilitation manipulator, so as to drive the user's hands to perform corresponding rehabilitation training.

As shown in FIG. 6 , a schematic diagram of the control system of the software rehabilitation manipulator of the present invention. After the user's movement intention is recognized and converted into the control command of the software rehabilitation manipulator through the lower computer, the air flows through the air pump through the proportional valve, solenoid valve, and throttle valve, and is sent to the air pipe in the corresponding finger through the compressed air to the software rehabilitation manipulator. Inflate and deflate, so as to realize the humanoid fingers to drive the user's fingers to do corresponding action tasks. The humanoid fingers are 5 artificial fingers designed and manufactured according to the size of the user's fingers. The fingers are made of soft material (silicone rubber) and poured into a 3D printed hand mold to form a hollow cavity structure. Fibers and strain-limiting layers are evenly arranged, and humanoid fingers are designed through the combination of bending deformation and elongation deformation of the hollow cavity structure. Each finger is composed of multiple joints to ensure that the user's hand and the artificial finger can fully fit , and have enough force to drive the finger movement. There are bandages on the outside of the anthropomorphic fingers to meet the tight constraints between the user's hand and the artificial fingers. A trachea is embedded in the cavity of each humanoid finger, and the trachea is connected with a portable control unit. The user can adjust the strength level required by the software rehabilitation manipulator according to the degree of hand rehabilitation in daily life. The air pressure can be adjusted through the proportional valve in the control unit. Air, the throttle valve can control the flow rate, realize the pneumatic drive to inflate and deflate different fingers of the soft rehabilitation manipulator, so as to flexibly control the motion deformation of the pneumatic software rehabilitation manipulator (grip, hook, push, release and hit) ), to meet the user's purpose of active rehabilitation training in daily life.

The above content is only to illustrate the technical idea of the present invention, and cannot limit the protection scope of the present invention. Any changes made on the basis of the technical solution according to the technical idea proposed in the present invention, all fall into the scope of the claims of the present invention. within the scope of protection.

Claims (6)

1. A multi-mode brain-computer interface control method for software rehabilitation manipulator, is characterized in that, comprises the following steps: 1) Synchronously collect brain/muscle/oculoelectric signals to build a multi-source signal control mode; 2) Extract the motion intention according to the brain/muscle/oculogram signal during the time period when the user generates the motion idea, and use the prompt sound or the muscle/oculograph signal feature to divide the corresponding task event; 3) Select different control modes according to the user's wishes, and use double blinking to switch the control mode; 4) Perform preprocessing, feature extraction, and pattern recognition on the brain/muscle/oculograph signals of a task event under different control modes to decode the user's movement intention in real time; 5) Divide different control commands in real time according to the decoded motion intention, and convert them into control commands through the lower computer in real time and send them to the air pump control unit, and then control the inflation and deflation of the software rehabilitation manipulator. 2. The multi-mode brain-computer interface control method for software rehabilitation manipulator according to claim 1, characterized in that, in step 1), the brain/muscle/oculoelectric signals include EMG signals, EEG signals and EOG signals. 3. The multi-mode brain-computer interface control method for software rehabilitation manipulator according to claim 2, characterized in that, in step 2), the brain/muscle/oculoelectric signal in the time period produced by each movement intention is called A task event; for the EEG pattern, EMG pattern or EEG pattern, extract the corresponding EEG task event, EMG task event or Oculoelectric task event; a. In the EEG mode, when the EEG signal is collected, the computer gives an initial prompt tone, and then the EEG signal within 2-4 seconds of continuous motor imagery is intercepted as an EEG task event; b. In the myoelectric mode, according to the addition of the average absolute value of the EMG signal, the start and end points of the action are judged by setting the threshold. The average absolute value of the EMG signal starts from being greater than the set threshold to the time period when it is less than the set threshold. The EMG signal in is a myoelectric task event; c. In the electro-oculogram mode, when the EOG signal is collected online, the low-frequency component of the EOG signal is obtained by wavelet transform. If the component is greater than the set threshold, it is determined that the position is a conscious blink. If the two positions are less than the length threshold, the blink is double. Blink, otherwise it is a single blink; if the component exceeds the threshold range, it is regarded as a pulse generated by saccade, and the position of left saccade and right saccade is found; EOG within 0.1-0.4s before and after the blink position or 0.2-0.3s before and after the saccade position The signal is an oculoelectric task event. 4. The multi-mode brain-computer interface control method for software rehabilitation manipulator according to claim 3, characterized in that, in step 3), the user realizes sequential switching between multiple modes of brain/muscle/eye electricity by double blinking ;First enter the default electro-oculogram mode, judge whether the mode can realize the action task, then enter the mode and complete the corresponding action task; otherwise, switch the brain/muscle/oculo-electric mode by double blinking until the action task can be completed mode so far. 5. the multi-mode brain-computer interface control method for software rehabilitation manipulator according to claim 4, is characterized in that, in step 4), the processing to EMG signal, EEG signal and EOG signal is specifically as follows: EOG signal processing: Based on wavelet transform and EOG signal in the vertical direction to locate the blink position, and then process and identify conscious single blink and double blink in an oculoelectric task event according to the threshold value; Carry out identification and positioning, and then correct the identified left glance/right glance signal according to the threshold method. By integrating the pulse amplitude in the horizontal and vertical directions, the identification and classification are carried out according to the two values of large amplitude and small amplitude. When the component is greater than the set When the threshold is fixed, it is determined that the position is a left saccade, otherwise, the position is a right saccade, thereby completing the real-time detection of a left saccade/right saccade in an oculoelectric task event; EEG signal processing: The power feature selection method based on wavelet analysis extracts the synchronization and desynchronization features of EEG signals, and realizes the classification of left-hand motor imagery events or right-hand motor imagery events in an EEG task event through the support vector machine classification algorithm; EMG signal processing: By collecting the EMG signal of the user's upper limb forearm, the feature values of the average absolute value MAV, the number of zero-crossing points ZC, the number of slope changes SSC, the waveform length WL, and the average absolute value slope MAVS are respectively extracted in the time domain, and are classified by a support vector machine. The algorithm performs feature recognition on the fusion of five time-domain statistical eigenvalues to realize online classification of hand training actions. 6. The multi-mode brain-computer interface control method for software rehabilitation manipulator according to claim 5, characterized in that, in step 5), the decoded motion intention is divided into different control commands in real time by the host computer, and then Send the corresponding control instructions to the lower computer, and adjust the on-off of the corresponding passage of the solenoid valve of the two-position three-way valve through the air pump control unit to control the inflation and deflation of the software rehabilitation manipulator.