CN111816309A - Rehabilitation training prescription adaptive recommendation method and system based on deep reinforcement learning - Google Patents

Abstract

The invention provides a rehabilitation training prescription self-adaptive recommendation method and system based on deep reinforcement learning. The method comprises the following steps: 1) collecting basic information and medical record information of a patient; 2) acquiring cerebral cortex blood oxygen data of different brain areas of a patient and movement and myoelectric data of an affected limb of the patient; 3) calculating a brain function evaluation index in the exercise rehabilitation training process of the patient by utilizing the cerebral blood oxygen data, and calculating a motor function evaluation index and a muscle function evaluation index in the exercise rehabilitation training process of the patient by utilizing the exercise data and the myoelectric data so as to dynamically evaluate the brain function, the motor function and the muscle function of the patient; 4) inputting the evaluation indexes of the brain function, the motor function and the muscle function obtained in the step 3) into a pre-established deep reinforcement learning model so as to train the deep reinforcement learning model and automatically generate a rehabilitation training prescription; 5) feeding back the rehabilitation training prescription generated in the step 4) to the doctor and the patient for rehabilitation training. By utilizing the method and the system, the self-adaptive adjustment of the training prescription can be realized.

Description

Rehabilitation training prescription adaptive recommendation method and system based on deep reinforcement learning

technical field

The invention relates to the field of limb movement rehabilitation training, in particular to a method and system for adaptive recommendation of rehabilitation training prescriptions based on deep reinforcement learning.

Background technique

There are more than 2 million new stroke patients in my country every year, and the trend is increasing year by year. Among them, 55-75% of stroke patients show motor dysfunction. At the same time, brain function damage caused by cerebral palsy, traumatic brain injury, etc. can also lead to limb motor dysfunction, which brings a heavy burden to patients, their families, and society. Rehabilitation training is the most important means to restore the patient's motor function. However, whether it is traditional manual rehabilitation training or rehabilitation training based on rehabilitation training robots, formulating personalized rehabilitation training prescriptions for different conditions of patients is an important condition to ensure the training effect. However, at present, rehabilitation training prescriptions can only be issued by doctors based on the evaluation results of patients, which largely depends on the experience of doctors. In addition, the functional assessment of patients is generally performed several times at different stages of rehabilitation. Therefore, the update of training prescriptions also depends on the period of assessment. As a result, the update of training prescriptions may not keep up with the recovery process of patients, and it is difficult to improve the efficiency of rehabilitation. . The development of artificial intelligence enables rehabilitation training robots to use a variety of sensory information to perform real-time functional evaluation during patient training, which solves the problem of long manual evaluation cycles. Prescriptions will undoubtedly greatly increase the workload of doctors. On the other hand, the active participation and fatigue level of patients in rehabilitation training will also have an important impact on the training effect. The training efficiency is often inefficient under low active participation and fatigue state, and it is difficult to manually adjust the training prescription. It can be adjusted in time according to the patient's state in a single training process, which will result in a waste of training and treatment resources to a certain extent.

SUMMARY OF THE INVENTION

Based on the above problems, the purpose of the present invention is to provide an adaptive recommendation method and system for rehabilitation training prescription based on deep reinforcement learning, which can perform real-time evaluation of brain function and motor function according to the patient's near-infrared cerebral oxygen, exercise, electromyography and other information. , input various medical record information and functional evaluation indicators into the pre-established deep reinforcement learning model for learning, and adaptively recommend rehabilitation training prescriptions according to the functional status of patients.

One aspect of the present invention provides an adaptive recommendation method for rehabilitation training prescriptions based on deep reinforcement learning, wherein the method includes the following steps:

1) Collect basic patient information and medical record information;

2) Using near-infrared cerebral blood oxygen monitoring equipment to obtain cerebral cortex blood oxygen data in different brain regions of the patient, and obtain the motion data and EMG data of the patient's affected limb;

3) Using cerebral blood oxygen data to calculate the evaluation index of brain function in the process of exercise rehabilitation training of patients, using exercise data and EMG data to calculate the evaluation index of motor function and muscle function in the process of exercise rehabilitation training of patients, to dynamically evaluate Brain function, motor function and muscle function of the patient;

4) Input the evaluation indexes of brain function, motor function and muscle function obtained in step 3) into the pre-established deep reinforcement learning model to train the deep reinforcement learning model and automatically generate a rehabilitation training prescription;

5) Feeding back the rehabilitation training prescription generated in step 4) to the doctor and the patient for rehabilitation training.

According to one embodiment, the training of the deep reinforcement learning model in the above step 4) includes taking the evaluation indexes of brain function, motor function and muscle function as the state according to the basic information and medical record information of the patient, taking the rehabilitation training prescription as the action, and using The functional improvement of the current rehabilitation training prescription after training is used as a reward to train the deep reinforcement learning model, and the prior knowledge in the rehabilitation training prescription knowledge base is introduced in the training process to accelerate the training of the learning model.

According to an embodiment, the deep reinforcement learning model in the above step 4) may include a large amount of patient medical record information, functional evaluation indicators, and a knowledge base of rehabilitation training prescriptions based on pre-inputted training prescriptions issued by doctors. Knowledge assists in the training of the model.

According to another embodiment, the deep reinforcement learning model in the above step 4) can use the patient's brain function, motor function and muscle function evaluation indicators as the state, use the rehabilitation training prescription as the action, and use the current prescription after training. Functional improvement is rewarded with reinforcement learning.

In one embodiment, the evaluation indexes of brain function, motor function and muscle function generated in the above step 3) and the rehabilitation training prescription generated in step 4) can be added to the knowledge base of the deep reinforcement learning model in real time, and the knowledge base is continuously expanded.

In one embodiment, the brain function evaluation index, motor function evaluation index and muscle function evaluation index obtained in step 3) are input into the trained deep reinforcement learning model, and the model calculates and outputs the different categories or types included in each rehabilitation training prescription. The Q value of the grade, the prescription items with the highest Q value are combined to automatically generate a rehabilitation training prescription, where the Q value is a numerical representation of the pros and cons of the corresponding action.

In another embodiment, the cerebral blood oxygen parameter data and EMG data obtained in the above step 2) can be further analyzed to obtain the active participation and fatigue of the brain and muscles of the patient during the training process, and the depth in step 4). Reinforcement learning models are able to adjust training prescriptions based on patient active engagement and fatigue.

According to a preferred embodiment, the active participation of the brain and muscles can be reflected by the activation degrees of different brain regions and the amplitude information of different muscle EMG signals, and the frequency domain information such as the average power frequency and median frequency of the EMG signals can be used It reflects the degree of fatigue of the patient's muscles.

According to another preferred embodiment, the training of the deep reinforcement learning model in the above step 4) may include first initializing the experience pool and network weights, inputting the medical records and state parameters of each functional evaluation index, and if the current state cannot match the prior knowledge If the current state can match the feature state in the prior knowledge base, the training prescription is output based on the comprehensive judgment of the prior action Q value and the estimated action Q value. The information is stored in the experience pool, and the above steps are repeated to train the network. After each training, the network weights are automatically updated to correct the network. The selected learning strategy is, for example, the ε-greedy strategy.

According to another embodiment, the evaluation index generated in the above step 3) and the rehabilitation training prescription generated in the above step 4) can be added to the knowledge base of the deep reinforcement learning model in real time to expand the knowledge base.

According to another aspect of the present invention, an adaptive recommendation system for rehabilitation training prescriptions based on deep reinforcement learning is provided, including:

The human-computer interaction module is used to receive the patient's basic information and case information and manage the pre-stored rehabilitation training knowledge base;

The near-infrared cerebral blood oxygen information acquisition module is used to collect the near-infrared cerebral blood oxygen signal of the corresponding brain region of the patient, and transmit the collected near-infrared cerebral blood oxygen signal to the evaluation and analysis module;

The movement and physiological data acquisition module is used to collect movement signals and surface EMG signals during the movement of the patient's limbs, and transmit these signals to the evaluation and analysis module;

The evaluation and analysis module is used to calculate the brain function of different brain regions of the patient according to the near-infrared cerebral blood oxygen signal transmitted from the near-infrared cerebral blood oxygen acquisition module and the motion signal and surface EMG signal transmitted from the exercise and physiological data acquisition module evaluation metrics and motor function evaluation metrics; and

The intelligent learning and prescription recommendation module is used for intelligent learning based on the patient's medical record information in the human-computer interaction module and the evaluation indicators of the patient's brain function and motor function obtained by the evaluation and analysis module, so as to output the rehabilitation training prescription and feedback through the human-computer interaction module. for doctors and patients.

According to one embodiment, the intelligent learning and prescription recommendation module may include a pre-established rehabilitation training prescription knowledge base and a deep reinforcement learning model, wherein the rehabilitation training prescription knowledge base allows modification, add content.

According to another embodiment, the evaluation and analysis module can reflect the active participation of the brain and muscles with the activation degree of different brain regions and the amplitude information of different muscle EMG signals, by calculating the average power frequency and median frequency of the EMG signals, etc. The frequency domain information reflects the fatigue level of the patient's muscles.

The present invention also provides an adaptive recommendation method for rehabilitation training prescriptions based on deep reinforcement learning, the method comprising the following steps:

1) Enter basic information such as gender, age, and medical history of the patient, as well as medical record information such as etiology, brain injury, onset time, initial functional level, and course of disease, including original medical examination data such as images and laboratory tests, and initial assessment data.

2) Use the near-infrared cerebral blood oxygen monitoring equipment to obtain the blood oxygen data of the cerebral cortex in different brain regions such as the motor area and the prefrontal lobe during the exercise training of the patient, including the local oxyhemoglobin concentration, deoxyhemoglobin concentration, blood oxygen saturation, etc. Use inertial sensors, surface electromyography sensors, etc. to obtain motion data such as acceleration and angular velocity of the patient's affected limb and surface electromyography data of exercise-related muscles.

3) During the patient's exercise rehabilitation training process, the brain function evaluation indicators such as the activation degree, activation mode, functional connection between brain regions, and laterality of different brain regions are calculated by using the cerebral blood oxygen data, and the patient's brain function is dynamically evaluated. ;Use acceleration, angular velocity and other motion data to calculate and obtain joint motion, motion smoothness, motion trajectory deviation and other motor function evaluation indicators, and dynamically evaluate the patient's motor function; use surface EMG data to obtain muscle function indicators such as muscle strength and muscle tension The degree of activation of different brain regions and the amplitude information of different muscle EMG signals reflect the active participation of the brain and muscles, and the frequency domain information such as the average power frequency and median frequency of EMG signals reflects the degree of muscle fatigue of patients.

4) Pre-establish a deep reinforcement learning model including a knowledge base of rehabilitation training prescriptions based on a large number of patient medical record information, functional evaluation indicators and training prescriptions issued by doctors. Input the evaluation indexes of brain function, motor function, and muscle function obtained in step 3) into the pre-established deep reinforcement learning model, and automatically generate a rehabilitation training prescription, including training tasks, training programs, sports training modes, training frequency, training intensity, etc.

Specifically, the deep reinforcement learning model uses the patient's brain function and motor function evaluation indicators as the state, the rehabilitation training prescription as the action, and the functional improvement after training with the current prescription as the reward to perform reinforcement learning. The training programs in the rehabilitation training prescription include unilateral exercise training, limb linkage exercise training, exercise training + functional electrical stimulation, exercise training + transcranial magnetic stimulation, exercise training + virtual reality feedback, etc. The exercise training modes include active, passive, assistive The training frequency includes the number of training sessions per week, the number of training sessions for a single task in each training process, etc. The training intensity includes the duration of each training session, the difficulty of the training task, the location, intensity, and frequency of magnetoelectric stimulation.

Further, the deep reinforcement learning model will adjust the training prescription according to the patient's active participation and fatigue level obtained in step 3).

5) Feed back the rehabilitation training prescription generated in step 4) to the doctor and the patient for rehabilitation training, and repeat step 2).

Further, the evaluation index generated in step 3) and the rehabilitation training prescription generated in step 4) are added to the knowledge base of the deep reinforcement learning model in real time, and the knowledge base is continuously expanded.

The present invention also provides an adaptive recommendation system for rehabilitation training prescriptions based on deep reinforcement learning, the system comprising:

Human-computer interaction module, including medical record information input module, rehabilitation training knowledge base management module and recommended prescription display module.

Further, the medical record information input module is used to input basic information such as gender, age, and medical history of the patient, as well as medical record information such as etiology, brain injury, onset time, initial functional level, and disease course, including original medical examination data such as images and laboratory tests. Initial evaluation data.

Near-infrared cerebral blood oxygen information acquisition module, including near-infrared light source and probe, fixed device, optical fiber, data acquisition system, etc., is used to collect the local oxyhemoglobin concentration, deoxyhemoglobin concentration, blood oxygen saturation, etc. blood oxygen signal, and transmit the collected cerebral blood oxygen signal to the evaluation and analysis module.

Movement and physiological data acquisition module, including inertial sensors, electromyography sensors and data acquisition circuits distributed in different parts of the limbs, used to collect acceleration, angular velocity and surface electromyography signals of exercise-related muscles during the movement of the patient's limbs, and use these The signal is transmitted to the evaluation analysis module.

An evaluation and analysis module, used for cerebral blood oxygen signals such as local oxyhemoglobin concentration, deoxyhemoglobin concentration, blood oxygen saturation and other cerebral blood oxygen signals transmitted by the near-infrared cerebral blood oxygen acquisition module, and the acceleration transmitted by the exercise and physiological data acquisition module , angular velocity and surface EMG signal, calculate the activation degree, activation mode, functional connection between brain areas, lateral deviation and other brain function evaluation indicators of different brain regions of the patient, joint mobility, motion smoothness, trajectory deviation, etc. Motor function evaluation indicators and muscle function indicators such as muscle strength and muscle tension. The degree of activation of different brain regions and the amplitude information of different muscle EMG signals reflect the active participation of the brain and muscles, and the average power frequency and median frequency of EMG signals are calculated to reflect the degree of muscle fatigue of patients.

The intelligent learning and prescription recommendation module includes a pre-established rehabilitation training prescription knowledge base and a deep reinforcement learning model using a large amount of patient medical record information, functional evaluation indicators, training prescriptions issued by doctors and other information. It is used to perform intelligent learning according to the patient's medical record information input by the medical record information input module and the patient's brain function and motor function evaluation indicators obtained by the evaluation and analysis module, output the rehabilitation training prescription based on the patient's condition and current functional state, and pass all The recommended prescription display module of the human-computer interaction module is fed back to doctors and patients.

Specifically, the rehabilitation training prescription knowledge base in the intelligent learning and prescription recommendation module allows users with authority to modify and add content through the knowledge base management module of the human-computer interaction module.

The beneficial effects of the present invention are: by using the method and system, the training prescription can be adjusted in real time and adaptively according to the patient's condition, motor function and training state in the process of rehabilitation training, which not only reduces the workload of doctors to manually evaluate and adjust the prescription , and it is more dynamic and precise than the fixed-period prescription adjustment, which is conducive to improving the efficiency of rehabilitation training.

The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present invention will become apparent by reference to the accompanying drawings and the following detailed description.

Description of drawings

In the drawings, unless stated otherwise, the same reference numbers refer to the same or like parts or elements throughout the several figures. The drawings are not necessarily to scale. It should be understood that these drawings depict only some embodiments according to the disclosure and should not be considered as limiting the scope of the invention.

FIG. 1 is an overall composition diagram of a rehabilitation training prescription adaptive recommendation system based on deep reinforcement learning according to an embodiment of the present invention;

2 is a schematic structural diagram of an adaptive recommendation system for rehabilitation training prescriptions based on deep reinforcement learning according to an embodiment of the present invention;

3 is an application flowchart of a deep reinforcement learning-based adaptive recommendation method for rehabilitation training prescriptions according to an embodiment of the present invention;

FIG. 4 is a flow chart of calculating a deep reinforcement learning model according to an embodiment of the present invention.

Detailed ways

In the following, only certain exemplary embodiments are briefly described. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. Accordingly, the drawings and description are to be regarded as illustrative in nature and not restrictive.

As shown in FIG. 1 and FIG. 2 , the adaptive recommendation system for rehabilitation training prescriptions based on deep reinforcement learning of the present invention generally includes a human-computer interaction module 1 , a near-infrared cerebral blood oxygen information collection module 2 , and an exercise and physiological data collection module 3 , evaluation analysis module 4 and intelligent learning and prescription recommendation module 5.

The human-computer interaction module 1 is used to receive the basic information and case information of the patient and manage the pre-stored rehabilitation training knowledge base, and includes a medical record information input module 11 , a rehabilitation training knowledge base management module 12 and a recommended prescription display module 13 .

The medical record information input module 11 is used to input basic information such as gender, age, and medical history of the patient, as well as medical record information such as etiology, brain injury, onset time, initial functional level, and disease course, including original medical examination data such as images and laboratory tests, and initial assessment. data.

The near-infrared cerebral blood oxygen information acquisition module 2 is used to collect the near-infrared cerebral blood oxygen signal of the corresponding brain region of the patient, and transmits the collected near-infrared cerebral blood oxygen signal to the evaluation and analysis module 4, and mainly includes a near-infrared light source 21 and a Probe 22 , fixing device (head cap) 23 , optical fiber 24 , data acquisition system 25 .

The near-infrared light source 21 and the probe 22 are arranged at a fixed distance, and are fixed at corresponding positions in different brain regions of the patient's head by a fixing device (a headgear in the figure) 23 . The near-infrared light signal collected by the probe 22 is transmitted to the data acquisition system 25 through the optical fiber 24, and the local oxyhemoglobin concentration, deoxyhemoglobin concentration, and blood oxygen saturation in the corresponding brain region of the patient are calculated according to the intensity of the light signal at different probe positions. Wait for the cerebral blood oxygen signal, and transmit the collected cerebral blood oxygen information to the evaluation and analysis module 4 .

The motion and physiological data acquisition module 3 is used to collect motion signals and surface EMG signals during the movement of the patient's limbs, and transmit these signals to the evaluation and analysis module 4, and includes inertial sensors 31 and EMG sensors distributed in different parts of the limbs. 32 and a data acquisition circuit 33.

The inertial sensor 31 is used to obtain the acceleration, angular velocity and other information during the movement of the patient's limbs, and the EMG sensor 32 is used to obtain the surface EMG signals of the muscles involved in the movement of the patient's limbs, and these information are synchronously collected and transmitted to the data acquisition circuit 33. Assessment Analysis Module 4.

The evaluation and analysis module 4 is used to transmit cerebral blood oxygen signals such as local oxyhemoglobin concentration, deoxyhemoglobin concentration, blood oxygen saturation, etc. transmitted by the near-infrared cerebral blood oxygen acquisition module 2, and the exercise and physiological data acquisition module 3 transmits The acceleration, angular velocity and surface EMG signals of the patient were calculated to obtain the activation degree, activation mode, functional connection between brain regions, lateral deviation and other brain function evaluation indicators of different brain regions of the patient. Movement function evaluation indicators such as displacement, and muscle function indicators such as muscle strength and muscle tension. The degree of activation of different brain regions and the amplitude information of different muscle EMG signals reflect the active participation of the brain and muscles, and the average power frequency and median frequency of EMG signals are calculated to reflect the degree of muscle fatigue of patients.

The intelligent learning and prescription recommendation module 5 includes a pre-established rehabilitation training prescription knowledge base and a deep reinforcement learning model using a large amount of patient medical record information, functional evaluation indicators, training prescriptions issued by doctors and other information. It is used to perform intelligent learning according to the patient's medical record information input by the medical record information input module 11 and the patient's brain function, motor function and muscle function evaluation indicators obtained by the evaluation and analysis module 4, and output the rehabilitation training based on the patient's condition and current functional state. Prescriptions, and fed back to doctors and patients through the recommended prescription display module 13 of the human-computer interaction module, wherein the rehabilitation training prescription knowledge base in the intelligent learning and prescription recommendation module 5 allows users with authority to pass the human-computer interaction module. The knowledge base management module 12 is modified to add content.

As shown in Figure 3, the deep reinforcement learning-based rehabilitation training prescription adaptive recommendation method of the present invention includes the following steps:

S1: Enter the patient's basic information and medical record information.

Basic information includes gender, age, medical history, etc. The medical record information includes etiology, brain injury, onset time, initial functional level, disease course, as well as original imaging, laboratory and other medical examination data and initial assessment data.

S2: Collect near-infrared cerebral blood oxygen and exercise and physiological information.

The near-infrared cerebral blood oxygen acquisition module is used to obtain the blood oxygen parameters of the cerebral cortex in different brain regions such as the motor area and the prefrontal lobe during the patient's exercise training, including the local oxyhemoglobin concentration, deoxyhemoglobin concentration, blood oxygen saturation, etc. Use inertial sensors, surface electromyography sensors, etc. to obtain motion data such as acceleration and angular velocity of the patient's affected limb and surface electromyography data of exercise-related muscles.

S3: Assess the patient's brain function and motor function.

During the patient's exercise rehabilitation training process, the brain function evaluation indicators such as the activation degree, activation mode, functional connection between brain regions, and laterality of different brain regions were calculated by using the cerebral blood oxygen data, and the patient's brain function was dynamically evaluated.

Specifically: perform continuous complex wavelet transform on the near-infrared cerebral blood oxygen signal collected by each acquisition channel, and use the wavelet amplitude to represent the degree of brain activation; obtain the frequency-domain wavelet phase matrix by calculation, and then perform every pairwise wavelet phase matrix. The wavelet phase coherence calculation of the near-infrared cerebral blood oxygen signal of the channel is used to obtain the brain functional connection indicators, including the brain functional connection strength and effect connection strength; the difference between the brain function indicators of a certain cerebral hemisphere and the contralateral cerebral hemisphere is divided by a certain cerebral hemisphere. The laterality coefficient was calculated as the sum of the brain function indexes of the contralateral cerebral hemisphere.

Using acceleration, angular velocity and other motion data and surface electromyography data to calculate the joint range of motion, motion coordination, muscle strength, muscle tension and other motor function and muscle function evaluation indicators, and dynamically evaluate the patient's motor function and muscle function.

Specifically: According to the approximate linear relationship between EMG and muscle strength, the muscle strength and muscle tension of the corresponding muscle are calculated by using the amplitude of the EMG signal. By establishing a human body dynamics model, the joint angles and motion trajectories are calculated using the acceleration and angular velocity data of different segments of the limb. The maximum joint angle in motion is used as the degree of motion of the joints, and the motion trajectories are used to reflect the motion coordination, including: motion trajectories and The deviation of the target trajectory, the smoothness of the motion trajectory, etc.

The degree of activation of different brain regions and the amplitude information of different muscle EMG signals reflect the active participation of the brain and muscles, and the frequency domain information such as the average power frequency and median frequency of EMG signals reflect the degree of muscle fatigue of patients.

S4: Based on medical record information and functional evaluation indicators, a deep reinforcement learning model is used for prescription recommendation.

A rehabilitation training prescription knowledge base is established in advance, which contains a large amount of patient medical record information, functional evaluation indicators and the mapping relationship between training prescriptions issued by doctors, and the mapping relationship between medical records, functional indicators and training prescriptions is used as prior knowledge. The DQN (Deep Q-Learning) algorithm is used to build a deep reinforcement learning model. According to the patient's basic information and medical record information, as well as the evaluation indicators of brain function, motor function and muscle function as the state, the rehabilitation training prescription is used as the action, and the current prescription is used. The functional improvement after training is used as a reward to train the learning model, and the prior knowledge in the knowledge base of rehabilitation training prescriptions is introduced in the training process to accelerate the training of the model. The evaluation indexes of brain function include the degree of activation of different brain regions, activation patterns, functional connections between brain regions, laterality and so on. Motor function evaluation indicators include joint range of motion, movement coordination, muscle strength, muscle tension, etc.

Brain function evaluation indicators such as the activation degree, activation mode, functional connection between brain regions, laterality, etc. of various different brain regions obtained in step S3, and motor function evaluation such as joint mobility, motion smoothness, and trajectory offset degree. The indicators and muscle function evaluation indicators such as muscle strength and muscle tension are input into the trained deep reinforcement learning model, and the model calculates and outputs the training tasks, training programs, sports training modes, training frequency, training intensity and other rehabilitation training prescription items contained in it. For the Q values of different categories or grades, the prescription items with the highest Q value are combined to automatically generate rehabilitation training prescriptions. Among them: training programs include different categories such as unilateral exercise training, limb linkage exercise training, exercise training + functional electrical stimulation, exercise training + transcranial magnetic stimulation, exercise training + virtual reality feedback; exercise training modes include active, passive, assisted movement , resistance and other categories; training frequency includes weekly training times, single task training times in each training process, etc. 5 times and other different levels; the training intensity includes the duration of each training, the difficulty of the training task, and the location, intensity and frequency of magnetoelectric stimulation, etc. The training duration includes 10 minutes, 20 minutes, 30 minutes, 40 minutes, 50 minutes, 60 minutes There are different levels such as minutes, the difficulty of the task includes different levels such as easy, medium, difficult, and difficult, the magnetoelectric stimulation site includes different categories, and the intensity and frequency of the magnetoelectric stimulation include different levels.

In the process of use, the evaluation index generated in step S3 and the rehabilitation training prescription generated in step S4 will be added to the knowledge base of the deep reinforcement learning model in real time, and the knowledge base will be continuously expanded. Automatically correct the model.

The specific calculation method of the deep reinforcement training model is shown in Figure 4: First, initialize the experience pool and network weights, and input state parameters such as medical records and functional evaluation indicators. - The greedy strategy selects the action (training prescription); if the current state can match the feature state in the prior knowledge base, the output action (training prescription) is comprehensively judged according to the prior action Q value and the estimated action Q value. According to the functional improvement of the patient after the action (training prescription), the reward value and the next state are obtained, and the information is stored in the experience pool, and the above steps are repeated to train the network. After each training, the network weights are automatically updated to correct the network.

On the other hand, the prior knowledge base of the rehabilitation training prescription contains state characteristics such as the patient's active participation and fatigue level. The deep reinforcement learning model will adjust the training prescription according to the patient's active participation and fatigue level obtained in step S3. Change the training mode or reduce the training intensity if the patient's active participation decreases or fatigue occurs after training with the current training prescription for a period of time.

S5: Rehabilitation training prescription feedback.

Specifically: the rehabilitation training prescription generated in step S4 is fed back to the doctor and the patient for rehabilitation training, and step S2 is repeated.

Finally, it should be noted that the above-mentioned embodiments are only specific implementations of the present invention, and are used to illustrate the technical solutions of the present invention, but not to limit them. The protection scope of the present invention is not limited thereto, although referring to the foregoing Embodiments have been described in detail the present invention, those of ordinary skill in the art should understand: any person skilled in the art is within the technical scope disclosed by the present invention, and it can still modify the technical solutions recorded in the foregoing embodiments Or can easily think of changes, or equivalently replace some of the technical features, and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be covered in the present invention. within the scope of protection. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (9)

1. A rehabilitation training prescription self-adaptive recommendation method based on deep reinforcement learning comprises the following steps:

1) collecting basic information and medical record information of a patient;

2) acquiring cerebral cortex blood oxygen data of different brain areas of a patient, and acquiring motion data and myoelectric data of an affected limb of the patient;

3) calculating to obtain a brain function evaluation index in the exercise rehabilitation training process of the patient by utilizing the cerebral blood oxygen data, and calculating to obtain a motor function evaluation index and a muscle function evaluation index in the exercise rehabilitation training process of the patient by utilizing the motor data and the myoelectric data so as to dynamically evaluate the brain function, the motor function and the muscle function of the patient;

4) inputting the evaluation indexes of the brain function, the motor function and the muscle function obtained in the step 3) into a pre-established deep reinforcement learning model so as to train the deep reinforcement learning model and automatically generate a rehabilitation training prescription;

5) feeding back the rehabilitation training prescription generated in the step 4) to the doctor and the patient for rehabilitation training.

2. The rehabilitation training prescription self-adaptive recommendation method based on the depth-enhanced learning of claim 1, wherein the training of the depth-enhanced learning model in the step 4) comprises training the depth-enhanced learning model by taking the evaluation indexes of the brain function, the motor function and the muscle function as states, taking the rehabilitation training prescription as an action, taking the function improvement condition after training by adopting the current rehabilitation training prescription as a reward according to the basic information and the medical history information of the patient, and introducing the prior knowledge in the rehabilitation training prescription knowledge base in the training process to accelerate the training of the learning model.

3. The rehabilitation training prescription self-adaptive recommendation method based on deep reinforcement learning of claim 1, wherein the brain function evaluation index, the motor function evaluation index and the muscle function evaluation index obtained in step 3) are input into a trained deep reinforcement learning model, Q values of different categories or levels contained in each rehabilitation training prescription are output through model calculation, the prescription items with the highest Q values are combined, and the rehabilitation training prescription is automatically generated, wherein the Q values are numerical representations of the superiority and inferiority of corresponding actions.

4. The adaptive deep reinforcement learning-based rehabilitation training prescription recommendation method according to claim 1, wherein the brain blood oxygen parameter data and the myoelectric data obtained in step 2) are further analyzed to obtain the active participation degree and the fatigue degree of the brain and the muscle in the training process of the patient, and the deep reinforcement learning model in step 4) can adjust the training prescription according to the active participation degree and the fatigue degree of the patient.

5. The adaptive recommendation method for rehabilitation training prescription based on deep reinforcement learning of claim 4, wherein the degree of activation of different brain areas and the amplitude information of different muscle electromyographic signals are used for reflecting the active participation degree of the brain and the muscle, and the frequency domain information of the electromyographic signals is used for reflecting the fatigue degree of the muscle of the patient.

6. The rehabilitation training prescription self-adaptive recommendation method based on deep reinforcement learning of claim 1, wherein the training of the deep reinforcement learning model in the step 4) comprises the steps of firstly initializing an experience pool and network weights, inputting medical records and state parameters of each function evaluation index, and selecting a training prescription according to a selected learning strategy if the current state cannot be matched with the feature state in the prior knowledge base; if the current-stage state can be matched with the characteristic state in the prior knowledge base, the training prescription is comprehensively judged and output according to the prior action Q value and the estimated action Q value, the information is stored in an experience pool, the steps are repeated to train the network, and the network weight is automatically updated after each training to correct the network.

7. A rehabilitation training prescription self-adaptive recommendation system based on deep reinforcement learning comprises:

the human-computer interaction module is used for receiving the basic information and the case information of the patient and managing a pre-stored rehabilitation training knowledge base;

the near-infrared brain blood oxygen information acquisition module is used for acquiring near-infrared brain blood oxygen signals of a corresponding brain area of a patient and transmitting the acquired near-infrared brain blood oxygen signals to the evaluation analysis module;

the motion and physiological data acquisition module is used for acquiring motion signals and surface electromyographic signals of the limb of the patient in the motion process and transmitting the signals to the evaluation analysis module;

the evaluation analysis module is used for calculating and obtaining evaluation indexes of brain functions, motor functions and muscle functions of different brain areas of the patient according to the near-infrared cerebral blood oxygen signals transmitted from the near-infrared cerebral blood oxygen acquisition module and the motor signals and surface electromyographic signals transmitted from the motor and physiological data acquisition module; and

and the intelligent learning and prescription recommending module is used for intelligently learning according to the patient medical record information in the human-computer interaction module and the evaluation indexes of the brain function, the motor function and the muscle function of the patient, which are obtained by the evaluation and analysis module, so as to output a rehabilitation training prescription, and feeding the rehabilitation training prescription back to the doctor and the patient through the human-computer interaction module.

8. The adaptive deep reinforcement learning-based rehabilitation training prescription recommendation system according to claim 7, wherein the intelligent learning and prescription recommendation module comprises a pre-established rehabilitation training prescription knowledge base and a deep reinforcement learning model, wherein the rehabilitation training prescription knowledge base allows modification and content addition through a knowledge base management module included in the human-computer interaction module.

9. The adaptive recommendation system for rehabilitation training prescriptions based on deep reinforcement learning of claim 7, wherein the evaluation and analysis module reflects active participation of the brain and muscles according to the activation degrees of different brain areas and the amplitude information of different muscle electromyographic signals, and reflects the fatigue degree of the muscle of the patient by calculating the frequency domain information of the electromyographic signals.

Images