CN111887803A - Multi-dimensional monitoring and evaluation system for man-machine work efficiency of aircraft cockpit - Google Patents

Abstract

A multi-dimensional monitoring and evaluation system for ergonomics in an aircraft cockpit, comprising: a human-machine state monitoring unit, a feature extraction unit, a multi-channel feature synchronous normalization processing unit, an ergonomic element description unit, an ergonomic diagnosis unit, and an ergonomic comprehensive evaluation unit, The man-machine state monitoring unit is connected with the feature extraction unit by outputting the original man-machine state measurement data, the feature extraction unit is connected with the ergonomics element description unit by outputting the feature extraction result, and the multi-channel feature synchronous normalization processing unit performs the output results of each module of the feature extraction unit. Time alignment processing and feature index normalization processing, and the features are grouped according to the formation mechanism of the features. The ergonomics element description unit is connected with the human factors diagnosis unit and the ergonomic comprehensive evaluation unit by outputting the results of the description of the ergonomics elements in multiple dimensions. The ergonomic comprehensive evaluation unit calculates and generates continuous and quantitative ergonomic overall evaluation results. The invention solves the problem that traditional ergonomics research pays attention to decision and ignores diagnosis, and can provide an effective human factor objective analysis tool for design optimization and risk tracing.

Description

Multi-dimensional monitoring and evaluation system for aircraft cockpit ergonomics

technical field

The invention relates to a technology in the field of aviation human factors safety, in particular to a multi-dimensional monitoring and evaluation system for ergonomics in an aircraft cockpit.

Background technique

In civil aircraft flight accidents, accidents caused by crew errors account for about 60% to 80% of the total accidents. The evaluation of cockpit ergonomics is of great significance for optimizing the design of the cockpit human-machine interface, reviewing the human factors related clauses in the airworthiness compliance of the aircraft, detecting the human factors risks in the operation process, and ensuring flight safety.

After investigation, there are a series of limitations in the current cockpit ergonomic evaluation: First, the current cockpit ergonomic evaluation method based on anthropometric measurements is partial and static, and can only meet the evaluation of accessibility. requirements, it is difficult to evaluate the pilot's cognitive state and mental workload during the mission; second, the evaluation results of the existing commonly used subjective evaluation methods such as NASA-TLX are poorly consistent and unconvincing, and cannot be used in the mission process. Continuous measurements are performed; third, existing evaluation methods cannot provide diagnostic information to guide design optimization.

SUMMARY OF THE INVENTION

Aiming at the above-mentioned deficiencies in the prior art, the present invention proposes a multi-dimensional monitoring and evaluation system for ergonomics in the cockpit of an aircraft, which uses objective measurement technology to continuously monitor the human-machine state in the process of human-machine interaction in multiple directions, and uses mathematical statistics methods. , integrates multi-source ergonomics features, realizes the objective and comprehensive evaluation of ergonomics in the task process, and solves the problem that the existing ergonomics evaluation relies too much on static measurement and subjective feeling; and builds ergonomics diagnostic indicators to analyze and locate the source of ergonomics problems. , which solves the problem that traditional ergonomic research focuses on decision-making and ignores diagnosis, and provides an effective human-factor objective analysis tool for design optimization and risk traceability.

The present invention is achieved through the following technical solutions:

The invention relates to a multi-dimensional monitoring and evaluation system for ergonomics in an aircraft cockpit, comprising: a man-machine state monitoring unit, a feature extraction unit, a multi-channel feature synchronous normalization processing unit, an ergonomics element description unit, an ergonomic diagnosis unit and an ergonomic synthesis unit The evaluation unit, wherein: the man-machine state monitoring unit is connected with the feature extraction unit by outputting the original man-machine state measurement data, the feature extraction unit is connected with the ergonomics element description unit by outputting the feature extraction result, and the multi-channel feature synchronous normalization processing unit is connected to the feature extraction unit. The output results of each module are subjected to time alignment processing and feature index normalization processing, and the features are grouped according to the formation mechanism of the features. The ergonomics element description unit outputs the results of the ergonomics element description in multiple dimensions and integrates the human factors diagnosis unit and ergonomics. The evaluation units are connected, the human factors diagnosis unit provides the results of multiple human factors diagnosis indicators to the user, and the ergonomic comprehensive evaluation unit calculates and generates continuous and quantitative ergonomic overall evaluation results.

The man-machine state monitoring unit includes: currently mainstream head-mounted eye trackers, chest strap cardiopulmonary activity measurement systems, cockpit manipulation recording equipment, and flight parameter recording equipment, wherein the eye tracker collects the coordinates of the pilot's gaze point. , Pupil diameter measurement, the cardiopulmonary activity measurement system collects the pilot's heart rate, breathing frequency, and breathing amplitude measurement values, the cockpit control recording device collects the position measurement values of the flight joystick, accelerator stick, and pedals, and the flight parameter recording device collects the aircraft. latitude, longitude, altitude, airspeed, and acceleration measurements.

The feature extraction unit includes: an eye movement feature extraction module, a control feature extraction module, a physiological feature extraction module and a flight task feature extraction module.

The ergonomics element description unit includes: a visual activity description module, a control activity description module, a workload description module and a task performance description module.

The human factors diagnosis unit includes: a visual perception difficulty diagnosis module, a manipulation difficulty diagnosis module, an information saliency diagnosis module and a manipulation efficiency diagnosis module.

technical effect

The overall problem solved by the present invention is to conduct a comprehensive, quantitative and continuous objective evaluation of the ergonomics of the cockpit during the flight mission, and to locate the source of the ergonomics problem from the design point of view.

Compared with the prior art, the present invention utilizes the cognitive psychology mechanism to clarify the elements that need to be investigated for the ergonomic evaluation problem; utilizes the continuous objective measurement technology, the quantitative feature extraction method and the multi-source feature fusion method to realize the task process. The quantitative description and objective comprehensive evaluation of cockpit ergonomic elements in the ergonomics field effectively solve the problem of static measurement and subjective evaluation in the field of ergonomics, and the lack of continuous objective evaluation methods; using the correlation between evaluation dimensions to construct ergonomic diagnostic indicators, It can provide targeted auxiliary information for design optimization, and solve the problem of traditional ergonomics analysis focusing on decision and ignoring diagnosis.

Description of drawings

Figure 1 is a schematic diagram of a multi-dimensional monitoring and evaluation system for ergonomics in the cockpit of an aircraft;

Fig. 2 is a schematic diagram of the characteristic index system for objective evaluation of ergonomics in the aircraft cockpit;

FIG. 3 is a multi-dimensional graphical presentation mode of the comprehensive evaluation result of the ergonomics of the aircraft cockpit constructed by the present invention.

Detailed ways

As shown in FIG. 1 , this embodiment relates to a multi-dimensional monitoring and evaluation system for ergonomics in an aircraft cockpit, including: a human-machine state monitoring unit, a feature extraction unit, a multi-channel feature synchronous normalization processing unit, an ergonomic element description unit, a human-machine state monitoring unit, a feature extraction unit, a Due to the diagnosis unit and the ergonomic comprehensive evaluation unit.

This embodiment relates to an ergonomic monitoring and evaluation method using the above system, including: ergonomic state monitoring in flight missions, basic raw measurement data processing and ergonomic feature extraction, ergonomic element calculation, calculation of diagnostic indicators, and comprehensive ergonomic evaluation Calculation of results and presentation of evaluation results.

The head-mounted eye tracker in the human-machine state monitoring unit uses the eye camera and the target detection method to locate the pupil position and detect the pupil size from the image, calculate the line of sight direction, and finally output the pixel coordinates of the gaze point in the foreground image and the pupil Diameter value.

The chest belt type cardiopulmonary activity measurement system in the man-machine state monitoring unit measures the ECG data of the subjects through the ECG sensor, and reflects the changes in the thoracic contour caused by breathing through the voltage value changes on the pressure-sensitive sensors in the chest belt.

The cockpit manipulation recording device in the man-machine state monitoring unit collects the deflection angle values of the flight joystick, the accelerator stick, and the pedal pedal relative to the neutral position.

The flight parameter recording device in the man-machine state monitoring unit collects and outputs the longitude, latitude, altitude, airspeed and acceleration values of the aircraft.

The eye movement feature extraction module in the feature extraction unit uses the coordinates of the fixation point and the sampling time to calculate the movement speed of the fixation point. When the movement speed of the fixation point is less than 30 pixels/second, it is recorded as the occurrence of gaze activity, and if it is greater than 30 pixels/second, it is recorded as the occurrence of saccades. Activity, count the length of each gaze activity sequence to obtain the value of the gaze time feature, and count the occurrences of the saccade activity per unit time to obtain the value of the scan rate feature. By detecting the 0 value in the pupil diameter measurement sequence, when a 0 value occurs, it is recorded as blinking, and the number of blink sequence segments in unit time is counted to obtain the value of the blink frequency feature.

The control feature extraction module in the feature extraction unit performs differential processing on the time series of the flight stick, throttle stick, and pedal angle, and obtains the characteristic values of the aileron control rate, the elevator control rate, the rudder control rate, and the throttle control rate.

The physiological feature extraction module in the feature extraction unit detects the R wave from the ECG data, and calculates the value of the heart rate feature through the reciprocal of the R-R interval. Using the amplitude of the voltage value on the pressure-sensitive sensor as the value of the breathing depth feature, by detecting the position where the first-order derivative of the voltage waveform on the pressure-sensitive sensor is 0 and the second-order derivative is less than 0, the maximum value in the thoracic circumference waveform is obtained. And the reciprocal of the interval time between the two maximum values is obtained to obtain the respiratory frequency feature.

The flight task feature extraction module in the feature extraction unit obtains the value of the track deviation feature by calculating the deviation between the actual track latitude and longitude and the planned flight route, and obtains the altitude deviation feature by calculating the difference between the actual flight altitude and the planned flight altitude. The value of the speed deviation feature is obtained by calculating the difference between the actual airspeed and the planned airspeed, and the value of the acceleration feature is obtained by calculating the acceleration vector modulus value.

The multi-channel feature synchronization normalization processing unit realizes multi-channel feature data synchronization by searching for timestamps that match various feature data, and normalizes each feature by calculating z-score to eliminate dimensional differences between features. The features are grouped according to their mechanism, and a feature index system for objective evaluation of ergonomics is constructed, as shown in Figure 3.

The visual activity description module in the ergonomic element description unit uses the principal component analysis method to fuse the features of fixation time, scanning frequency and blink frequency. First, the eigenvalues and eigenvectors of the matrix formed by the above eigendata are calculated, and the original data are projected in the direction of the eigenvectors to obtain the principal component data. Then, the eigenvalues are normalized to represent the variance explanation rate of the principal components, and each principal component is weighted and summed by the variance explained rate to obtain the result of the visual activity description index.

The control activity description module in the ergonomic element description unit uses the principal component analysis method to fuse the characteristics of aileron control rate, elevator control rate, rudder control rate and throttle control rate. First, the eigenvalues and eigenvectors of the matrix formed by the above eigendata are calculated, and the original data are projected in the direction of the eigenvectors to obtain the principal component data. Then the eigenvalues are normalized to characterize the variance explanation rate of the principal components, and each principal component is weighted and summed by the variance explained rate to obtain the result of the control activity description index.

The workload description module in the ergonomics element description unit uses the principal component analysis method to fuse the features of heart rate, respiration frequency, respiration amplitude, and pupil diameter. First, the eigenvalues and eigenvectors of the matrix formed by the above eigendata are calculated, and the original data are projected in the direction of the eigenvectors to obtain the principal component data. Then, the eigenvalues are normalized to characterize the variance explanation rate of the principal components, and each principal component is weighted and summed by the variance explained rate, and the result of the workload description index is obtained.

The task performance description module in the ergonomics element description unit uses the principal component analysis method to fuse the features of aircraft deviation, altitude deviation, speed deviation and acceleration. First, the eigenvalues and eigenvectors of the matrix formed by the above eigendata are calculated, and the original data are projected in the direction of the eigenvectors to obtain the principal component data. Then, the eigenvalues are normalized to characterize the variance explanation rate of the principal components, and each principal component is weighted and summed by the variance explained rate, and the result of the task performance description index is obtained.

The visual perception difficulty diagnosis module in the human factors diagnosis unit obtains the visual perception difficulty diagnosis result by calculating the Pearson correlation coefficient between the visual activity description index and the workload description index in a sliding window of 30 seconds.

The manipulation difficulty diagnosis module in the human factors diagnosis unit obtains the manipulation difficulty diagnosis result by calculating the Pearson correlation coefficient between the control activity description index and the workload description index in a sliding window of 30 seconds.

The information saliency diagnosis module in the human factors diagnosis unit obtains the information saliency diagnosis result by calculating the Pearson correlation coefficient between the visual activity description index and the task performance description index in a sliding window of 30 seconds.

The control efficiency diagnosis module in the human factors diagnosis unit obtains the control efficiency diagnosis result by calculating the Pearson correlation coefficient between the control activity description index and the workload description index in a sliding window of 30 seconds.

The comprehensive ergonomic evaluation unit integrates the features of visual activity description results, control activity description results, workload description results and task performance description results. First, the eigenvalues and eigenvectors of the matrix formed by the above data are calculated, and the original data are projected in the direction of the eigenvectors to obtain the principal component data. Then, the eigenvalues are normalized to characterize the variance explanation rate of the principal components, and each principal component is weighted and summed by the variance explained rate, and the result of the comprehensive evaluation index of ergonomics is obtained.

In order to provide designers, evaluators, and analysts with intuitive and detailed ergonomic evaluation and diagnosis results, this embodiment constructs a graphical method for presenting ergonomic evaluation results. The graph contains 8 directions, of which the horizontal and vertical directions include <visual activity>, <control activity>, <workload> and <task performance> 4 ergonomic elements, and the diagonal direction includes <visual perception difficulty> , <manipulation difficulty>, <information significance> and <manipulation efficiency> four diagnostic fingertips. At each moment during the flight mission, the ergonomics can be evaluated in detail and intuitively through this graphical presentation.

This embodiment proposes for the first time to define ergonomics by using four ergonomic elements of <visual perception activity>, <control activity>, <workload> and <task performance>, and to measure the above elements through objective measurement technology, through data Through processing and feature extraction, a feature index system for objective evaluation of ergonomics was constructed, and finally, the objective and comprehensive evaluation of ergonomics in the aircraft cockpit was realized by grouping and synthesizing the feature indexes by multivariate mathematical statistics method. At the same time, the use of correlations between evaluation dimensions to construct diagnostic indicators is also the main innovation of this embodiment. The introduction of diagnostic indicators can help designers and testers locate the source of ergonomics problems, enabling them to locate and improve problems in a targeted manner. , which makes up for the deficiency of the weak diagnostic ability of the existing ergonomics evaluation methods.

The system and method constructed in this embodiment are applied in a real pilot mission process. The experimental data show that this embodiment distinguishes the ergonomics differences caused by different task difficulties, different flight experiences and different cockpit designs.

Compared with the prior art, the performance index improvement of this embodiment is that: compared with the most commonly used ergonomic subjective evaluation method, this embodiment provides an objective quantitative evaluation method, which is not subject to the subjective consciousness of the evaluator. Influence; this embodiment combines cognitive science theory to monitor and evaluate the ergonomics of the aircraft cockpit from multiple aspects, and the results are more comprehensive and effective than the existing methods; this embodiment can continuously and dynamically evaluate the task process, evaluate The frequency of the results reaches 20 Hz, which has a higher temporal resolution than existing evaluation methods, and is able to provide more detailed information on ergonomic changes during the task.

The above-mentioned specific implementation can be partially adjusted by those skilled in the art in different ways without departing from the principle and purpose of the present invention. The protection scope of the present invention is subject to the claims and is not limited by the above-mentioned specific implementation. Each implementation within the scope is bound by the present invention.

Claims (6)

1. an aircraft cockpit ergonomics multi-dimensional monitoring and evaluation system, is characterized in that, comprises: man-machine state monitoring unit, feature extraction unit, multi-channel feature synchronization standardization processing unit, ergonomics element description unit, human factor diagnosis unit and human The ergonomics comprehensive evaluation unit, wherein: the man-machine state monitoring unit is connected with the feature extraction unit by outputting the original man-machine state measurement data, the feature extraction unit is connected with the ergonomics element description unit by outputting the feature extraction result, and the multi-channel feature synchronous normalization processing unit pair The output results of each module of the feature extraction unit are subjected to time alignment processing and feature index normalization processing, and the features are grouped according to the formation mechanism of the features. The ergonomic comprehensive evaluation unit is connected, the human factors diagnosis unit provides the results of multiple human factors diagnostic indicators to the user, and the ergonomic comprehensive evaluation unit calculates and generates continuous and quantitative ergonomic overall evaluation results. 2. The multi-dimensional monitoring and evaluation system for ergonomics in an aircraft cockpit according to claim 1, wherein the man-machine state monitoring unit comprises: current mainstream head-mounted eye trackers, chest strap cardiopulmonary activity measurement system, cockpit manipulation recording equipment, and flight parameter recording equipment, among which: the eye tracker collects the pilot's gaze point coordinates and pupil diameter measurements, the cardiopulmonary activity measurement system collects the pilot's heart rate, breathing frequency, and breathing amplitude measurements, and the cockpit controls The recording device collects the position measurements of the flight joystick, the accelerator stick, and the pedal, and the flight parameter recording device collects the longitude, latitude, altitude, airspeed, and acceleration measurements of the aircraft. 3. The multi-dimensional monitoring and evaluation system for ergonomics in aircraft cockpit according to claim 1, wherein the feature extraction unit comprises: an eye movement feature extraction module, a control feature extraction module, a physiological feature extraction module and a flight task Feature extraction module. 4. The multi-dimensional monitoring and evaluation system for ergonomics in an aircraft cockpit according to claim 1, wherein the ergonomics element description unit comprises: a visual activity description module, a control activity description module, a workload description module and a task performance Describe the module. 5. The multi-dimensional monitoring and evaluation system for ergonomics in an aircraft cockpit according to claim 1, wherein the human factors diagnosis unit comprises: a visual perception difficulty diagnosis module, a manipulation difficulty diagnosis module, an information significance diagnosis module and a Handling efficiency diagnostic module. 6. An ergonomic monitoring and evaluation method based on the system according to any of the preceding claims, characterized in that it comprises: ergonomic state monitoring in flight tasks, basic raw measurement data processing and ergonomic feature extraction, ergonomics element calculation, Calculation of diagnostic indicators, calculation of comprehensive ergonomic evaluation results, and presentation of evaluation results; The head-mounted eye tracker in the human-machine state monitoring unit uses the eye camera and the target detection method to locate the pupil position and detect the pupil size from the image, calculate the line of sight direction, and finally output the pixel coordinates of the gaze point in the foreground image and the pupil diameter value; The chest belt type cardiopulmonary activity measurement system in the man-machine state monitoring unit measures the ECG data of the subjects through the ECG sensor, and reflects the changes in the thoracic contour caused by breathing through the voltage value change on the pressure sensitive sensor in the chest belt; The cockpit manipulation recording device in the man-machine state monitoring unit collects the deflection angle values of the flight joystick, the accelerator stick, and the pedal pedal relative to the neutral position; The flight parameter recording device in the man-machine state monitoring unit collects and outputs the latitude and longitude, altitude, airspeed and acceleration values of the aircraft; The eye movement feature extraction module in the feature extraction unit uses the coordinates of the fixation point and the sampling time to calculate the movement speed of the fixation point. When the movement speed of the fixation point is less than 30 pixels/second, it is recorded as the occurrence of gaze activity, and if it is greater than 30 pixels/second, it is recorded as the occurrence of saccades. Activity, count the length of each gaze activity sequence to get the value of the gaze time feature, and count the number of saccade activities in unit time to get the value of the scan rate feature; by detecting the 0 value in the pupil diameter measurement sequence, the occurrence of 0 value is recorded as blinking , count the number of blink sequence segments in unit time to get the value of blink frequency feature; The control feature extraction module in the feature extraction unit performs differential processing on the time series of the angles of the flight stick, the throttle stick, and the pedal pedal, and obtains the characteristic values of the aileron control rate, the elevator control rate, the rudder control rate, and the throttle control rate; The physiological feature extraction module in the feature extraction unit detects the R wave from the electrocardiogram data, and calculates the value of the heart rate feature through the reciprocal of the R-R interval; the amplitude of the voltage value on the pressure-sensitive sensor is used as the value of the respiratory depth feature, and by detecting the pressure When the first derivative of the voltage waveform on the sensitive sensor is 0 and the second derivative is less than 0, the maximum value in the chest cavity circumference waveform is obtained, and the reciprocal time between the two maximum values is obtained to obtain the respiratory frequency characteristic; The flight task feature extraction module in the feature extraction unit obtains the value of the track deviation feature by calculating the deviation between the actual track latitude and longitude and the planned flight route, and obtains the altitude deviation feature by calculating the difference between the actual flight altitude and the planned flight altitude. value, obtain the value of the speed deviation feature by calculating the difference between the actual airspeed and the planned airspeed, and obtain the value of the acceleration feature by calculating the acceleration vector modulus value; The multi-channel feature synchronization normalization processing unit realizes multi-channel feature data synchronization by searching for timestamps that match various feature data, and normalizes each feature by calculating z-score to eliminate dimensional differences between features; According to the mechanism, the features are grouped, and the feature index system for objective evaluation of ergonomics is constructed; The visual activity description module in the ergonomics element description unit uses the principal component analysis method to fuse the gaze time, scan rate and blink frequency features; first calculate the eigenvalues and eigenvectors of the matrix formed by the above feature data, and convert the original data to the eigenvectors. Direction projection to obtain the data of each principal component; then the eigenvalues are normalized to represent the variance explanation rate of the principal component, and each principal component is weighted and summed by the variance explained rate to obtain the result of the visual activity description index ; The control activity description module in the ergonomics element description unit uses the principal component analysis method to fuse the characteristics of the aileron control rate, elevator control rate, rudder control rate and throttle control rate; first, the eigenvalues and eigenvectors of the matrix formed by the above feature data are calculated. , and project the original data to the eigenvector direction to obtain the principal component data; then normalize the eigenvalues to characterize the variance explanation rate of the principal component, and use the variance explanation rate to weight the principal components and sum them up , to obtain the results of the control activity description indicators; The workload description module in the ergonomics element description unit fuses the features of heart rate, respiration rate, respiration amplitude, and pupil diameter by using the principal component analysis method; firstly, the eigenvalues and eigenvectors of the matrix formed by the above feature data are calculated, and the original data are converted into The eigenvectors are projected in the direction to obtain the data of each principal component; then the eigenvalues are normalized to represent the variance explanation rate of the principal components, and each principal component is weighted and summed by the variance explained rate to obtain the workload description index the result of; The task performance description module in the ergonomics element description unit uses the principal component analysis method to fuse the aircraft deviation, altitude deviation, speed deviation and acceleration features; Project to the eigenvector direction to obtain the data of each principal component; then normalize the eigenvalues to represent the variance explanation rate of the principal components, and use the variance explanation rate to weight and sum each principal component to obtain the task performance description the result of the indicator; The visual perception difficulty diagnosis module in the human factors diagnosis unit obtains the visual perception difficulty diagnosis result by calculating the Pearson correlation coefficient between the visual activity description index and the workload description index in a 30-second sliding window; The control difficulty diagnosis module in the human factors diagnosis unit obtains the control difficulty diagnosis result by calculating the Pearson correlation coefficient between the control activity description index and the workload description index in a sliding window of 30 seconds; The information saliency diagnosis module in the human factors diagnosis unit obtains the information saliency diagnosis result by calculating the Pearson correlation coefficient between the visual activity description index and the task performance description index in a 30-second sliding window; The control efficiency diagnosis module in the human factors diagnosis unit obtains the control efficiency diagnosis result by calculating the Pearson correlation coefficient between the control activity description index and the workload description index in a sliding window of 30 seconds; The ergonomic comprehensive evaluation unit integrates the features of visual activity description results, control activity description results, workload description results and task performance description results; firstly, the eigenvalues and eigenvectors of the matrix formed by the above data are calculated, and the original data are converted to the features. The vector direction projection is used to obtain the data of each principal component; then the eigenvalues are normalized to characterize the variance explanation rate of the principal component, and each principal component is weighted and summed by the variance explained rate to obtain a comprehensive evaluation of ergonomics the result of the indicator.

Images