CN114814832A - Real-time monitoring system and monitoring method of human falling behavior based on millimeter wave radar - Google Patents

Abstract

The invention discloses a millimeter wave radar-based real-time human body falling behavior monitoring system which comprises a millimeter wave radar module, a computing board card, an acousto-optic module and a power supply module, wherein the millimeter wave radar module is arranged in a protective shell; millimeter wave radar module and power module are connected with the calculation integrated circuit board through USB respectively, and the reputation module is connected with the calculation integrated circuit board, and the power module provides electric power for monitoring system, and the calculation integrated circuit board passes through the WIFI module and links to each other with remote terminal. The invention also discloses a real-time monitoring method for the falling behavior of the human body based on the millimeter wave radar, which firstly reduces the noise of the millimeter wave radar, selects the Tsfresh algorithm for characteristic extraction, and extracts abundant characteristic parameters related to falling. And secondly, a LightGBM algorithm is adopted to replace a height threshold value method, so that the robustness of the millimeter wave radar for detecting the falling behavior of the human body is improved.

Description

Real-time monitoring system and monitoring method of human falling behavior based on millimeter wave radar

technical field

The invention belongs to the field of industrial sensor technology and artificial intelligence technology, in particular to a real-time monitoring system for human falling behavior based on millimeter-wave radar; and a real-time monitoring method for human falling behavior based on millimeter-wave radar.

Background technique

According to the World Health Organization, about 28-35% of people aged 65 and over fall at least once a year, and falls have become one of the leading causes of death and unintentional injury among the elderly. As a public health problem, falls have become a common obstacle for the elderly to live independently. Therefore, in the context of home care, timely and effective medical and health monitoring has become crucial. If the elderly fall in time, they can take proactive measures. Rescue; if the fall is not detected in time and treatment is delayed, the consequences will be disastrous. Therefore, a reliable and accurate fall detection system for the elderly is very important, and it is of great significance for the safety monitoring of the elderly at home.

At present, the technical solutions that can be applied to fall detection mainly include acceleration sensor, camera-based vision technology and millimeter-wave radar. The acceleration sensor is usually integrated in smart watches or wristbands. The advantages are low cost and low recognition rate for general falls. High, the disadvantage is that the recognition rate of slow falls is low, the comfort is poor, charging is troublesome, and the user needs to wear it to achieve detection; the camera needs to have enough light for fall detection, and there must be no objects between the camera and the detected person. The potential risk of privacy leakage is more worrying. As an emerging perception method, millimeter wave technology is widely used in traffic flow recognition, vehicle speed detection, obstacle perception and other scenarios. In the home environment, point cloud information can be generated for the human body through millimeter wave technology to describe the relevant behavioral characteristics of the human body. However, the spatial position information of the human body generated by the millimeter-wave radar has large noise with the change of the environment, and has a large false detection rate for the fall detection behavior of the human body in the home scene. Secondly, the fall detection method based on height threshold method is not suitable for use in complex environment due to the limitation of specific environment, thus reducing the recognition accuracy of falling behavior.

Therefore, in the fall scene of the elderly at home, the big data analysis algorithm in the field of artificial intelligence can be used to reduce the problem of excessive noise caused by the millimeter-wave radar due to its own attributes. Secondly, the machine learning algorithm is used instead of the height threshold method to improve the accuracy of the millimeter-wave radar to the human body. Robustness of fall behavior detection.

SUMMARY OF THE INVENTION

The purpose of the present invention is to solve the problem that relatives cannot quickly know when the elderly at home fall, so as to provide a real-time monitoring system for human falling behavior based on millimeter wave radar, which can accurately detect falling behavior under the premise of protecting privacy.

The invention also provides a real-time monitoring method for human body falling behavior based on millimeter wave radar, which realizes multi-angle collection of human body echo information, and uses machine learning algorithm to replace the height threshold method to realize accurate identification of human body falling behavior.

The first technical solution adopted by the present invention is a real-time monitoring system for human falling behavior based on millimeter-wave radar, including a millimeter-wave radar module, a computing board, an acousto-optic module, and a power module arranged inside the protective casing; The millimeter-wave radar module and the power module are respectively connected to the computing board through USB, the acousto-optic module is connected to the computing board through the GPIO interface, the power module provides power for the monitoring system, and the computing board is connected to the remote terminal through the WIFI module .

The millimeter wave radar module includes a millimeter wave sensor and a digital signal processing chip. The millimeter wave sensor is used to obtain the spatial position information of the human body based on the millimeter wave signal, and the digital signal processing chip is used to complete the modulation, demodulation and coordinate conversion calculation of the millimeter wave signal. , to obtain the human body spatial position data based on the point cloud information.

The millimeter-wave radar is a multi-input multi-channel millimeter-wave radar, and the millimeter-wave radar is placed within a range of 1.8-2.5 meters above the ground in the detection area.

The computing board implements algorithms such as point cloud information preprocessing, feature extraction, and behavior classification to identify human falling behavior.

The sound and light module is used to send an alarm to the outside world through the form of sound and light on the detected falling behavior of the human body.

The protective casing protects the millimeter-wave radar module, computing board, acousto-optic module and power module inside the casing.

The power module provides power for the entire hardware system.

The second technical solution adopted by the present invention is a real-time monitoring method for human falling behavior based on millimeter-wave radar, comprising the following steps:

Step 1: The millimeter-wave radar transmits electromagnetic waves to the monitoring space at the installation height, and the electromagnetic waves are received by the radar receiving antenna after being reflected by the human body. The digital signal processing chip of the millimeter-wave radar preprocesses the echo signal. Fast Fourier transform and other processing are used to obtain body echo point information, including distance, azimuth, elevation, radial velocity and signal-to-noise ratio.

Step 2: Preprocess the obtained target point cloud containing distance, azimuth, elevation, radial velocity and signal-to-noise ratio information, and use the K-Means algorithm to cluster the human point cloud;

The preprocessing is as follows: the data obtained by the radar is further filtered by the tracking filter based on the Gaussian algorithm to filter the original position data to eliminate the noise brought by the hardware environment; the Kalman filter is used to filter out the ghost generated by the multipath reflection.

Step 3: Transmit the human body point cloud data preprocessed in Step 2, including distance, azimuth, elevation, radial velocity and signal-to-noise ratio information, to the computing board, and the computing board further uses the sliding window to ID the data, After the identification data is obtained.

After the data set is segmented based on the sliding window, the Tsfresh algorithm is used for feature extraction, and the five parameters of the input data, including distance, azimuth, elevation, radial velocity and signal-to-noise ratio information, are extracted to obtain a variety of different digital features.

The parameters of the sliding window include the sliding window length LW and the sliding step size LI. The preprocessed data is continuously identified by ID. The data in the same sliding window is identified as the same row ID, and the sliding window is regarded as another sliding window after moving a certain step. , is identified as a new ID, the present invention selects the sliding window length LW=20, and the sliding step length LI=5.

Further use the extract_relevant_features() method provided by tsfresh, which is specially used to filter the digital features with zero value, retain the non-zero part, filter the extracted features, and retain the features with high correlation with the classification label.

Step 4: Use the Tsfresh algorithm to perform feature extraction on the data identified in step 3, thereby obtaining a variety of different digital features; use the extract_relevant_features() method that comes with tsfresh to filter the digital features with zero value, retain the non-zero part, and extract the features. Filter and retain features that are highly correlated with classification labels.

Step 5: Input the digital features obtained in Step 4 into the fall detection model for detection and classification. When an abnormal fall behavior is detected, the sound and light module will give an alarm.

The present invention is also characterized in that,

Step 1 is as follows:

After the electromagnetic signal acts on the human body, the receiving antenna receives the echo signal with the human body motion information, and the reflected echo is subjected to one-dimensional fast Fourier transform to obtain the target distance dimension information, and further uses the Capon beamforming algorithm to obtain the echo point azimuth, Elevation angle information, further extracting the Doppler spectrum from the radar waveform data, and performing a maximum peak search in the Doppler spectrum to measure the radial velocity of the detection point.

The implementation method of the fall detection model is as follows:

Step 5.1: Use millimeter-wave radar to detect the physical sign data when the person under test falls and send it to the computer; collect data when falling in different postures, including front fall, back fall, left fall, right fall, and vertical fall. The data set collected by the millimeter wave radar is recorded as A. Record the real fall state, record a fall as 1, and record a fall as 0, and treat the fall detection problem as a two-category problem;

Step 5.2: Repeat step 5.1 to collect experimental data as training, validation and test data sets;

Step 5.3: Divide the data in step 5.1 into training set, validation set and test set, and execute steps 2-4;

Step 5.4: Use the LightGBM algorithm model as the classification model. First, use the histogram algorithm to discretize the training data set from continuous floating-point eigenvalues into k integers. According to the discrete values of the histogram, traverse to find the optimal segmentation point, and further use The leaf-wise algorithm parallelizes the operation of the leaf nodes of the same layer, finds the leaf node with the largest gain, and splits it. Further, the LightGBM algorithm model outputs a probability list, and the output result B has a value in the range of [0, 1]. After training Model;

Step 5.5: Transplant the model trained in step 5.4 to the computing board as a fall detection model.

The beneficial effects of the present invention are:

The real-time monitoring system for human falling behavior based on millimeter wave radar of the present invention has the following advantages:

1. The present invention is a real-time monitoring system for human falling behavior based on millimeter-wave radar. The device body is composed of a millimeter-wave radar module, a computing board, an acousto-optic module, a protective casing, and a power module. The device as a whole can be miniaturized and integrated, which is convenient for installation and deployment in actual home scenarios.

2. The present invention is a real-time monitoring system for human body falling behavior based on millimeter-wave radar. First, the human body space position data based on point cloud information is collected by the millimeter-wave radar module, and secondly, the computing board adopts the tracking filter to perform the human body position data. The filtering process reduces the influence of the quality of the observation data generated by the millimeter-wave radar on the accuracy of the spatial position information of the human body.

3. Further, compared with the traditional height threshold method, the present invention adopts the Tsfresh algorithm to perform multi-dimensional feature extraction, and the behavior classifier built based on the machine learning algorithm can effectively improve the detection accuracy of falling behavior, and solve the problem of human body detection in the traditional method of using millimeter wave radar. A problem with a high probability of misidentification of fall behavior.

Description of drawings

Fig. 1 is the overall structure schematic diagram of the real-time monitoring system of human body falling behavior based on millimeter wave radar of the present invention;

FIG. 2 is a flowchart of a real-time monitoring method for human falling behavior based on millimeter wave radar according to the present invention.

In the figure, 1. millimeter wave radar module, 2. computing board, 3. acousto-optic module, 4. power module, 5. protective casing.

Detailed ways

The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments of the real-time monitoring system for human falling behavior based on millimeter wave radar.

The real-time monitoring system for human body falling behavior based on millimeter-wave radar of the present invention, as shown in FIG. 1 , includes a millimeter-wave radar module 1 arranged inside a protective casing 5 , a computing board 2 , an acousto-optic module 3 , and a power module 4; The millimeter-wave radar module 1 and the power module 4 are respectively connected to the computing board 2 through USB, the acousto-optic module 3 is connected to the computing board 2 through the GPIO interface, the power module 4 provides power for the monitoring system, and the computing board The card 2 is connected with the remote terminal through the WIFI module.

The millimeter-wave radar module 1 includes a millimeter-wave sensor and a digital signal processing chip. The millimeter-wave sensor is used to obtain the spatial position information of the human body based on the millimeter-wave signal, and the digital signal processing chip is used to complete the modulation, demodulation, and coordinate conversion of the millimeter-wave signal. Calculate to obtain human body spatial position data based on point cloud information.

The real-time monitoring method of human falling behavior based on millimeter wave radar of the present invention is shown in Figure 2, and the specific operation steps are as follows:

Step 1: The millimeter-wave radar transmits electromagnetic wave signals to the detection space, and the reflected echo is subjected to one-dimensional windowing and one-dimensional fast Fourier transform to obtain the echo point information of the human body in the detection space, including distance, azimuth, elevation, radial Speed and SNR information 5 channels of data;

After the electromagnetic signal acts on the human body, the receiving antenna receives the echo signal with the human body motion information, and the reflected echo is subjected to one-dimensional fast Fourier transform to obtain the target distance dimension information, and further uses the Capon beamforming algorithm to obtain the echo point azimuth, Elevation angle information, further extract the Doppler spectrum from the radar waveform data, and perform the maximum peak search in the Doppler spectrum to measure the radial velocity of the detection point;

Step 2: Preprocess the obtained target point cloud containing distance, azimuth, elevation, radial velocity and signal-to-noise ratio information, and use the K-Means algorithm to cluster the human point cloud;

The preprocessing is as follows: the data obtained by the radar is further filtered by the tracking filter based on the Gaussian algorithm to filter the original position data to eliminate the noise caused by the hardware environment; the Kalman filter is used to filter out the ghost generated by the multipath reflection;

Step 3: Transmit the human body point cloud data preprocessed in Step 2, including distance, azimuth, elevation, radial velocity and signal-to-noise ratio information, to the computing board, and the computing board further uses the sliding window to ID the data, Get the data after identification;

The parameters of the sliding window include the sliding window length LW and the sliding step size LI. The preprocessed data is continuously identified by ID. The data in the same sliding window is identified as the same row ID, and the sliding window is regarded as another sliding window after moving a certain step. , identified as a new ID, the present invention selects the sliding window length LW=20, and the sliding step length LI=5;

Step 4: Use the Tsfresh algorithm to perform feature extraction on the data identified in step 3, thereby obtaining a variety of different digital features, as follows:

Use the extract_relevant_features() method that comes with tsfresh to filter the digital features with zero value, retain the non-zero part, filter the extracted features, and retain the features with high correlation with the classification label

Step 5: Input the digital features obtained in Step 4 into the fall detection model for detection and classification, and the sound and light module will give an alarm when an abnormal fall behavior is detected at that time;

The implementation method of the fall detection model is as follows:

Step 5.1: Use millimeter-wave radar to detect the physical sign data when the person under test falls and send it to the computer; collect data when falling in different postures, including front fall, back fall, left fall, right fall, and vertical fall. The data set collected by the millimeter wave radar is recorded as A. Record the real fall state, record a fall as 1, and record a fall as 0, and treat the fall detection problem as a two-category problem;

Step 5.2: Repeat step 5.1 to collect experimental data as training, validation and test data sets;

Step 5.3: Divide the data in step 5.1 into training set, validation set and test set, and execute steps 2-4;

Step 5.4: Use the LightGBM algorithm model as the classification model. First, use the histogram algorithm to discretize the training data set from continuous floating-point eigenvalues into k integers. According to the discrete values of the histogram, traverse to find the optimal segmentation point, and further use The leaf-wise algorithm parallelizes the operation of the leaf nodes of the same layer, finds the leaf node with the largest gain, and splits it. Further, the LightGBM algorithm model outputs a probability list, and the output result B has a value in the range of [0, 1]. After training Model;

Step 5.5: Transplant the model trained in step 5.4 to the computing board as a fall detection model.

The invention designs a real-time monitoring system for human falling behavior based on millimeter-wave radar. The device body is composed of a millimeter-wave radar module, a JetsonNANO computing board, an acousto-optic module, a protective casing and a power module. The device as a whole can be miniaturized and integrated, which is convenient for installation and deployment in actual home scenarios.

Different from the combination of the millimeter-wave radar and other image sensors, the present invention only uses the millimeter-wave radar as the monitoring sensor, which realizes non-sensing monitoring while protecting personal privacy.

Different from the fall detection scheme using multiple radars, the present invention uses only one millimeter-wave radar, supports a viewing angle of 120 degrees in the horizontal and vertical directions, and can obtain rich point cloud information of human body echoes in space.

The algorithm part uses the big data analysis algorithm in the field of artificial intelligence to reduce the problem of excessive noise caused by the millimeter-wave radar due to its own attributes. The Tsfresh algorithm is used for feature extraction, and the feature parameters related to falls are extracted from the limited data parameters. Compared with traditional feature extraction methods, the algorithm has high efficiency and wide range, and can automatically calculate a large number of time series features. Secondly, the LightGBM classification algorithm is used to replace the height threshold method to improve the robustness of the millimeter-wave radar for human falling behavior detection.

Example:

The height threshold only uses the elevation data of the target in the millimeter wave data set as the feature quantity, and identifies the falling behavior by judging the relationship between the height and the set threshold, while the feature quantity of LightGBM includes the speed and acceleration information of the target in addition to the elevation data. The method of pattern recognition can comprehensively judge the fall behavior, so it can improve the accuracy and robustness of fall recognition.

Table 1 below shows the accuracy of fall recognition between the traditional threshold method and the machine learning algorithm LightGBM adopted in the present application. It can be seen that the accuracy of the fall detection methods of the present application is all above 90%, and the accuracy is higher.

Table 1

Threshold method (accuracy rate) LightGBM (accuracy rate) fall forward 82.35% 95.05% fall after 88.97% 91.28% Left fall 87.15% 93.51% right down 85.55% 92.66% vertical drop 92.01% 99.21%

.

Claims (8)

1. the real-time monitoring system of human body falling behavior based on millimeter-wave radar, is characterized in that, comprises the millimeter-wave radar module (1) that is arranged in protective casing (5) inside, computing board (2), acousto-optic module (3) ), a power supply module (4); the millimeter wave radar module (1) and the power supply module (4) are respectively connected to the computing board (2) through USB, and the acousto-optic module (3) is connected through a GPIO interface is connected with a computing board (2), the power module (4) provides power for the monitoring system, and the computing board (2) is connected with a remote terminal through a WIFI module. 2. The real-time monitoring system for human falling behavior based on millimeter-wave radar according to claim 1, wherein the millimeter-wave radar module (1) comprises a millimeter-wave sensor and a digital signal processing chip, and the millimeter-wave sensor The digital signal processing chip is used to obtain the human body space position information based on the millimeter wave signal, and the digital signal processing chip is used to complete the modulation, demodulation and coordinate conversion calculation of the millimeter wave signal, and obtain the human body space position data based on the point cloud information. 3. A real-time monitoring method for human falling behavior based on millimeter-wave radar, characterized in that the specific operation steps are as follows: Step 1: The millimeter-wave radar transmits electromagnetic wave signals to the detection space, and the reflected echo is subjected to one-dimensional windowing and one-dimensional fast Fourier transform to obtain the echo point information of the human body in the detection space, including distance, azimuth, elevation, radial Speed and SNR information 5 channels of data; Step 2: Preprocess the obtained target point cloud containing distance, azimuth, elevation, radial velocity and signal-to-noise ratio information, and use the K-Means algorithm to cluster the human point cloud; Step 3: Transmit the human body point cloud data preprocessed in Step 2, including distance, azimuth, elevation, radial velocity and signal-to-noise ratio information, to the computing board, and the computing board further uses the sliding window to ID the data, After getting the identification data; Step 4: use the Tsfresh algorithm to perform feature extraction on the data identified in step 3, thereby obtaining a variety of different digital features; Step 5: Input the digital features obtained in Step 4 into the fall detection model for detection and classification. When an abnormal fall behavior is detected, the sound and light module will give an alarm. 4. the real-time monitoring method of human body falling behavior based on millimeter wave radar according to claim 3, is characterized in that, the described fall detection model realization method of step 5 is specifically as follows: Step 5.1: Use millimeter-wave radar to detect the physical sign data when the person under test falls and send it to the computer; collect data when falling in different postures, including front fall, back fall, left fall, right fall, and vertical fall. The data set collected by the millimeter wave radar is recorded as A. Record the real fall state, record a fall as 1, and record a fall as 0, and treat the fall detection problem as a two-category problem; Step 5.2: Repeat step 5.1 to collect experimental data as training, validation and test data sets; Step 5.3: Divide the data in step 5.1 into training set, validation set and test set, and execute steps 2-4; Step 5.4: Use the LightGBM algorithm model as the classification model. First, the histogram algorithm discretizes the training data set from continuous floating-point eigenvalues into k integers. According to the discrete values of the histogram, traverse to find the optimal segmentation point, and further use leaf The -wise algorithm parallelizes the operation of the leaf nodes of the same layer, finds the leaf node with the largest gain, and splits it. Further, the LightGBM algorithm model outputs a probability list, and the value of the output result B is in the range of [0, 1], and the trained model is obtained. ; Step 5.5: Transplant the model trained in step 5.4 to the computing board as a fall detection model. 5. The real-time monitoring method for human body falling behavior based on millimeter wave radar according to claim 3, wherein the preprocessing described in step 2 is: the data obtained by the radar is further passed through a tracking filter based on a Gaussian algorithm to the original position data. Perform filtering to eliminate the noise brought by the hardware environment; use the Kalman filter to filter out the ghosts generated by multipath reflections. 6. The real-time monitoring method of human body falling behavior based on millimeter wave radar according to claim 3, is characterized in that, the feature extraction described in step 4 is as follows: Use the extract_relevant_features() method that comes with tsfresh to filter the digital features with zero value, retain the non-zero part, filter the extracted features, and retain the features with high correlation with the classification label. 7. the real-time monitoring method of human body fall behavior based on millimeter wave radar according to claim 3, is characterized in that, the parameter of the described sliding window of step 3 comprises sliding window length LW and sliding step length LI, and the data after preprocessing is continuous. Carry out ID identification, the data in the same sliding window is identified as the same row ID, after the sliding window moves a certain step length, it is regarded as another sliding window, and the identification is a new ID. The present invention selects the sliding window length LW=20, and the sliding step length LI =5. 8. the real-time monitoring method of human body falling behavior based on millimeter wave radar according to claim 3, is characterized in that, step 1 is as follows: After the electromagnetic signal acts on the human body, the receiving antenna receives the echo signal with the human body motion information, and the reflected echo undergoes one-dimensional fast Fourier transform to obtain the target distance dimension information, and further uses the Capon beamforming algorithm to obtain the echo point azimuth, The elevation angle information is further extracted from the radar waveform data, and the Doppler spectrum is further extracted, and the maximum peak search is performed in the Doppler spectrum to measure the radial velocity of the detection point.

Images