CN115169829A - Team cooperation state evaluation system and method based on support vector machine - Google Patents

Abstract

The invention provides a team collaborative state evaluation system and method based on a support vector machine. The data acquisition module acquires four parts of information of a team: team information acquisition, eye movement equipment acquisition, electrocardio equipment acquisition and video recording acquisition. And the data interface module receives the information transmitted by the data acquisition module and converts and extracts the index characteristics. And the online evaluation module performs online analysis and evaluation on the team experimental data. And the output report module comprehensively outputs the read team information and the online evaluation result. The system can be used for rapidly and effectively judging team cooperation states of teams in various task environments, is beneficial to improving team cooperation level of ships, and provides basis for reducing human errors.

Description

System and method of team collaboration state assessment based on support vector machine

technical field

The present invention relates to a team collaboration state assessment system and method based on a support vector machine (SVM).

Background technique

In modern military and industrial work environments, operational tasks often have high cognitive demands and require close and efficient teamwork to successfully complete. Team synergy refers to the collection of interconnected thoughts, actions, and feelings that each member needs to form a team that promotes coordinated, adaptive performance and task goals that yield value-added outcomes. The goal of team collaboration is to create good team performance, and excellent team performance requires accurate information sharing through team collaboration. Therefore, team collaboration status is an important indicator to measure whether a team can complete various tasks accurately and efficiently.

As a typical complex man-machine system, modern ships are characterized by harsh operating environment under long-term navigation conditions; complex tasks and high requirements for personnel coordination; man-machine interface is rich in information, and the crew's mental load is high. Common accidents on ships include collisions, fires, flooding, explosions, etc. According to statistics, more than 70% of ship accidents are caused by human error, and poor team coordination is one of the reasons why ship accidents cannot be ignored.

The assessment methods of team collaboration status are mainly divided into four categories, namely Self-report measures, Observational measures, Event-based measurement and Automated measurement.

①Self-report assessment mainly includes questionnaires and rating scales to assess the state of team collaboration at the individual and team levels, especially those that are inherently emotional (such as trust) and difficult to observe. For example, The Team Climate Inventory (TCI) assesses a variety of team synergies. Specifically, TCI has 5 top-level scales, corresponding to 15 sub-scales; the 5 top-level scales respectively assess participation security, innovation support, team vision, task orientation, and social desirability. Other tools assess individual factors associated with team coordination status, such as the Psychological Safety Scale proposed in 1999, which is one of the most widely used assessment criteria for team coordination status. A major challenge in using self-reports to assess team synergy status is how to aggregate or translate individual responses into team-level characteristics. Even with a detailed theoretical understanding of the respective structures, the various aggregation methods are likely to yield very different results and fail to capture the unique features of individual responses. In addition, self-report assessments tend to produce inflated scores because subjects rate themselves higher than observers; self-report assessments are often performed before or after team tasks and fail to capture the dynamic nature of team performance as tasks are performed .

② Observational assessments can be collected in real time or with videos of team collaboration tasks, thus having the potential to capture the dynamic nature of team collaboration processes. Behaviorally Anchored Rating Scales (BARS) is a typical observational assessment technique: for excellent, average, and poor team performance, a short behavioral description is used as an anchor for observational assessment. These behaviors can be rated at the individual level and aggregated into a team score or team rating. BARS is widely used in the assessment of team function in medical institutions, non-technical skill systems for assessing pilot behavior in the crew environment, etc. While providing specific descriptions of behaviors contributes to more accurate ratings, it may lead observers to focus only on behaviors on the list.

③ Event-Based Measurement (EBAT) is a structured observation technique that is usually used in team simulation scenarios. By systematically introducing events into training exercises, EBAT provides the opportunity to observe specific behaviors of interest, which are then flagged as present or absent. For example, in an aviation scenario, situational awareness can be assessed by introducing an event in which the trainer deliberately steers the plane off course. The target behavior response is "Check Navigation" and the result is a detection success or a detection failure. Using this approach, learning objectives can be clearly quantified, and specific concerns for teamwork ability training can be set. However, due to event-specific scenario limitations, EBAT can only be used in training scenarios, but not in real performance scenarios. Furthermore, the development of the EBAT metric is time-consuming.

④ Automated assessment Collecting team-related data through a computer system that interacts with the team has the advantages of reducing interruptions, minimizing measurement errors, and reducing experimental staff resources. This type of assessment has been most frequently and successfully applied to the assessment of team performance and team synergy status.

Related studies on team collaboration status assessment have shown inconsistent reliability and validity test reports, as well as a lack of related research and applications in the field of ship command and control systems.

Based on the above considerations, it is necessary to develop an experiment based on a simulated ship team, collect and process physiological data, communication data and other data information automatically through computers, and use the support vector machine in the machine learning algorithm to analyze the data information, so that the data information can be analyzed directly and conveniently. A system for automatically assessing team collaboration status online. It is especially hoped that the evaluation system can be applied to the ergonomic evaluation and analysis of the collaborative state of the ship team, so as to provide a basis for improving the collaborative operation state of the ship team and enhancing the collaborative operation ability of the ship team.

SUMMARY OF THE INVENTION

According to one aspect of the present invention, a support vector machine-based team collaboration status assessment system is provided, which includes four sub-modules: a data acquisition module, a data interface module, an online assessment module, and an output report module. Among them, the data collection module is used to collect four parts of information of the team: team information collection, eye tracking equipment collection, ECG equipment collection, and video recording collection; the data interface module is used to receive the information transmitted by the data collection module and convert and extract index features. ; Online evaluation module for online analysis and evaluation of team experimental data; output report module for comprehensive output of the read team information and online evaluation results.

According to another aspect of the present invention, there is provided a team collaboration state assessment method, which is characterized by comprising: an initial interface before the assessment operation for collecting team information; a data acquisition and processing interface during the assessment operation for performing data analysis The online evaluation interface in the evaluation operation is used to analyze and evaluate the team experimental data online; the output result interface after the evaluation operation is used to comprehensively output the read team information and online evaluation results.

The beneficial effects of the present invention are:

The present invention overcomes the deficiency of the current online assessment technology of team collaboration state level, aiming at the team collaboration state in the ship experiment process, the interface module is connected to the eye tracker device and the electrocardiogram device to read the team's physiological data in real time, and the video recording and analysis device is connected to real-time data. Collect team communication data, realize online assessment of team coordination status, and quickly identify team coordination status in various task environments, so as to ensure effective team coordination and provide a basis for reducing human errors.

Advantages of the present invention include:

(1) An automated team collaboration status assessment system is provided, which does not need to rely on cumbersome questionnaires and post-event analysis and evaluation. It only needs to collect eye movement data, ECG data and video recordings during the team collaborative experiment. Online automatic assessment of the level of collaboration status, the system is simple and feasible to use, and provides an optimized user interface and use process, which is especially suitable for evaluating and analyzing the collaborative status of ship crews, helping to improve the collaborative work status of ship crews.

(2) Provide an objective assessment method of team collaboration state. Based on support vector machine, the team collaboration state level can be assessed directly, conveniently and automatically only by collecting the physiological data and communication data of the team collaboration experiment.

Description of drawings

1 is an architectural diagram of a team collaboration state assessment system based on a support vector machine according to an embodiment of the present invention;

2 is a flowchart of a user interface operation method of a support vector machine-based team collaboration state assessment system according to an embodiment of the present invention;

3(a) to 3(d) are schematic diagrams of user interfaces of a support vector machine-based team collaboration state assessment system in various operating states according to an embodiment of the present invention.

Detailed ways

1 General situation

Aiming at real-time collection of different scenarios and/or applications of different equipment and different usage requirements, the present invention proposes an online assessment system and method for team collaboration status based on the eye movement, electrocardiogram and communication characteristics of team members during the simulated ship team experiment process. By classifying the physiological and communication data collected and combed during the experiment based on support vector machine, the level of team collaboration was assessed.

2 Architecture design of team collaboration status assessment system based on support vector machine

As shown in FIG. 1 , a support vector machine-based team collaboration state assessment system according to an embodiment of the present invention includes four modules: a data acquisition module, a data interface module, an online assessment module, and an output report module. The specific description is as follows.

(1) Data acquisition module

This module collects four parts of team data information: team information collection, eye tracking equipment collection, ECG equipment collection, and video recording collection. Among them, ① the team information collection includes the team test number, which is input through the user interface; ② the eye tracking device collection reads the original eye movement data in real time by wearing the eye tracking device, such as the original spatial position coordinates of the eye movement, blink marks, gaze marks, saccades Marking, etc., ③ ECG equipment collection By wearing ECG equipment, real-time reading of raw ECG data, such as heart rate, ECG original waveform data, etc. ④Video recording collection: Record the video of the activity, including the picture and audio, by recording the whole process of the activity (team collaboration).

(2) Data interface module

This module is used to receive the information transmitted by the data acquisition module and convert and extract the index features. It includes three main modules: a data transceiver interface module, a data storage module, and a data conversion module. in:

①Data transceiver interface module: First set the UDP port number and Host address in the eye tracking device and the ECG device, and then establish the UDP data communication between the data interface module and the eye tracking device and the ECG device through the data transceiver interface based on the C++ platform. Read the raw data collected by the eye tracking device at a frequency of 60Hz, and read the raw ECG data at a frequency of 1Hz; extract the communication text from the communication audio in the team experiment through audio analysis software;

②Data storage module: store the real-time collection of eye movement and ECG raw data of different frequencies and the extracted communication text in the memory space;

3. Data conversion module: in one embodiment, based on the entire activity, the system converts the raw data of eye movement and electrocardiogram into the calculation conversion and average value of the index, and converts it into gaze rate, saccade rate, pupil diameter, average gaze time. Eye movement characteristic index data, as well as ECG characteristic index data such as average heart rate, average RR interval, normalized low-frequency power, and normalized high-frequency power.

For communication texts, text analysis is performed through text analysis software, and the frequency of occurrence of communication codes and word frequency data of communication keywords are counted to obtain characteristic data of communication texts.

(3) Online assessment module

This module is mainly used for online analysis and evaluation of team experimental data, including four modules: eye movement feature preprocessing module, ECG feature preprocessing module, communication feature preprocessing module, and online assessment classifier module. in:

①The eye movement feature preprocessing module reads the eye movement feature indicators (gazing rate, saccade rate, pupil diameter, average fixation time, etc.) sent in the data interface module, and performs normalization processing such as null or outlier elimination, and standardization Preprocessing operations such as processing;

②The ECG feature preprocessing module reads the ECG feature indicators (average heart rate, average RR interval, normalized low-frequency power, normalized high-frequency power, etc.) sent in the data interface module, and performs normalization such as null or abnormal value elimination processing, and preprocessing operations such as normalization.

③ Communication feature preprocessing module, which reads the communication text feature indicators (communication coding frequency, communication keyword word frequency, etc.) sent in the data interface module, and performs normalization processing such as null value or outlier elimination, and preprocessing operations such as normalization processing. ;

④Online evaluation classifier module, which reads in the standardized and normalized eye movement, ECG and communication feature data, and transmits them to three "One-Against-One" (OAO) support vector machines for online calculation. : Among them, SVM1 takes the vector corresponding to the high team coordination state as the positive set +1, and the vector corresponding to the medium team coordination state as the negative set -1; SVM2 takes the vector corresponding to the high team coordination state as the positive set +1, and the low team coordination state as the positive set +1. The vector corresponding to the collaborative state is taken as the negative set -1; SVM3 takes the vector corresponding to the collaborative state of the middle team as the positive set +1, and the vector corresponding to the collaborative state of the low team as the negative set -1. The three SVMs are calculated separately, and three classification results are obtained: classification 1, classification 2, and classification 3. Use the voting method to determine the final classification of the team collaboration state level, that is, the result with the most appearance among the three classification results is the final classification. For example, if SVM1 is judged to be a high team collaboration state, SVM2 is judged to be a high team collaboration state, and SVM3 is judged to be a medium team collaboration state, then the final classification result is a high team collaboration state, thus realizing the online assessment of the level of different team collaboration states.

(4) Output report module

The team information read in the above modules and the online assessment results are comprehensively output, and the assessment results are the final classification of the team collaboration status level, which can support selecting a specified path to save and export the data list.

3 User interface operation method of team collaboration state assessment system based on support vector machine

As shown in FIG. 2 , according to an embodiment of the present invention, the flow chart of the user interface operation method of the team collaboration state assessment system based on support vector machine includes four interfaces in turn: before the assessment operation (initial interface), during the assessment operation (data Acquisition and processing interface), during evaluation operation (online evaluation interface), and after evaluation operation (output result interface), respectively corresponding to Figures 3(a) to 3(d). The specific description is as follows.

As shown in FIGS. 3( a ) to 3 ( d ), it is a schematic diagram of a user interface of a team collaboration state assessment system based on a support vector machine according to an embodiment of the present invention.

After entering the team collaboration state assessment system based on support vector machine, the pre-assessment operation (initial interface) is shown in Figure 3(a). The system interface includes four parts: title, input, status, and evaluation result. The title part is "Online Evaluation System of Team Collaboration Status"; the input part includes the test number input field and the "Start Evaluation" button; the status part can display the operating status of the system; the evaluation result part includes the team cooperation state level and the evaluation result output column, and " Save data", "Export results" and "Close system" buttons.

At the beginning of the experiment, the user enters the test number in the test number column, and clicks the "Start Evaluation" button to start data collection and sorting. During the experiment, the status part displays "Data collection and processing...", as shown in Figure 3 As shown in (b), the evaluation is in operation (data acquisition and processing interface).

After the experiment is over, the status part will display "Data acquisition and processing completed, online assessment in progress...", which means that online analysis of the collected data is started, and the team collaboration status level is assessed based on support vector machine classification, as shown in Figure 3(c). For evaluation in operation (online evaluation interface).

After the online assessment is completed, the status section displays "Online assessment completed", and the assessment results section outputs the team collaboration status level and assessment results. Click the "Save Data" button to save the data and evaluation results, click the "Export Results" button to export the test number and the corresponding test results, and click "Close System" to close the team collaboration status online assessment system, as shown in Figure 3 (d) shows after the evaluation operation (output result interface).

Claims (9)

1. A team cooperation state evaluation system based on a support vector machine is characterized by comprising:

data acquisition module for gather team's data information, include through user input collection team information, through the eye movement data of eye movement equipment collection team, through electrocardio equipment collection team's electrocardio data, through the video recording data to the activity video recording collection team, wherein:

the team information includes a team test number,

the team's eye movement data includes the raw eye movement data read in real time,

the electrocardiographic data of the team comprises real-time read original electrocardiographic data,

team video recording data includes full-range video and/or recorded video, including picture and audio,

the data interface module is used for receiving the data transmitted from the data acquisition module and converting and extracting the index characteristics, and comprises:

the data receiving and transmitting interface module is used for firstly setting UDP port numbers and Host addresses in the eye movement equipment and the electrocardio equipment, then establishing UDP data communication between the data interface module and the eye movement equipment and the electrocardio equipment through the data receiving and transmitting interface based on a C + + platform, reading original eye movement data collected by the eye movement equipment, reading original electrocardio data and extracting a communication text of a team cooperative communication audio;

the data storage module is used for storing the original eye movement data, the original electrocardiogram data and the extracted communication text which are collected in real time in a memory space;

the data conversion module is used for converting the indexes of the original eye movement data and the original electrocardio data and averaging the indexes, wherein the conversion of the indexes comprises the steps of converting the original eye movement data into eye movement characteristic index data comprising fixation rate, saccade rate, pupil diameter and average fixation time and converting the original electrocardio data into electrocardio characteristic index data comprising average heart rate, average RR interval, standardized low-frequency power and standardized high-frequency power,

the data interface module is also used for carrying out text analysis through text analysis software, counting the occurrence frequency of communication codes and word frequency data of communication keywords for communication texts to obtain communication text characteristic data,

the online evaluation module is used for performing online analysis and evaluation on the eye movement data of the team, the electrocardio data of the team and the communication text characteristic data of the team, and comprises:

the eye movement characteristic preprocessing module is used for reading eye movement characteristic index data sent by the data interface module, and performing standardized processing including null value and/or abnormal value elimination and preprocessing operation including standardized processing;

the electrocardio characteristic preprocessing module is used for reading electrocardio characteristic index data sent by the data interface module, and performing standardized processing including null value and/or abnormal value elimination and preprocessing operation including standardized processing;

the communication characteristic preprocessing module is used for reading communication text characteristic index data sent by the data interface module, and performing standardized processing including null value and/or abnormal value elimination and preprocessing operation including standardized processing;

the online evaluation classifier module is used for reading in standardized and normalized eye movement characteristic index data, electrocardiogram characteristic index data and communication text characteristic index data, and respectively transmitting the eye movement characteristic index data, the electrocardiogram characteristic index data and the communication text characteristic index data to three OAO support vector machines for online calculation: wherein:

the first OAO support vector machine SVM1 takes a vector corresponding to the high team cooperation state as a positive set +1, and takes a vector corresponding to the medium team cooperation state as a negative set-1;

the second OAO support vector machine SVM2 takes the vector corresponding to the high team cooperation state as a positive set +1, and takes the vector corresponding to the low team cooperation state as a negative set-1;

the third OAO support vector machine SVM3 takes the vector corresponding to the middle team cooperation state as a positive set +1, the vector corresponding to the low team cooperation state as a negative set-1,

SVM1, SVM2 and SVM3 are respectively calculated to obtain 3 classification results, namely classification 1, classification 2 and classification 3,

and determining the final classification result of the team collaborative state level by using a voting method, namely taking the result which appears most in the 3 classification results as the final classification result,

and the output report module is used for comprehensively outputting the team information and the final classification result.

2. The support vector machine-based team collaborative status assessment system according to claim 1, wherein:

team information is input through the user interface terminal,

the eye movement device comprises a wearable eye movement device,

the raw eye movement data includes raw spatial position coordinates of the eye movement, blink markers, gaze markers, glance markers,

the electrocardio-device comprises a wearable electrocardio-device,

the original electrocardio data comprises heart rate and electrocardio original waveform data.

3. The support vector machine-based team collaborative status assessment system of claim 1, wherein:

in the online evaluation classifier module, if the SVM1 judges that the team cooperation state is high, the SVM2 judges that the team cooperation state is high, and the SVM3 judges that the team cooperation state is medium, the final classification result is the high team cooperation state.

4. The support vector machine-based team collaborative status assessment system according to claim 1, wherein:

the data receiving and transmitting interface module reads original eye movement data at the frequency of 60Hz, reads original electrocardio data at the frequency of 1Hz, and extracts communication texts from communication audios in team experiments through audio analysis software.

5. The team cooperation state evaluation system based on the support vector machine is characterized by comprising the following steps of:

a) Gather the data information of team, include through user input gather team information, through the eye movement data of eye movement equipment collection team, through the electrocardio data of electrocardio equipment collection team, through the video recording data to the activity video recording collection team, wherein:

the team information includes a team test number,

the team's eye movement data includes the raw eye movement data read in real time,

the electrocardiographic data of the team comprises original electrocardiographic data read in real time,

team video recording data includes full-range video and/or recorded video, including picture and audio,

b) Receiving data transmitted from a data acquisition module and converting and extracting index characteristics, wherein the index characteristics comprise:

b1 Firstly, UDP port numbers and Host addresses in the eye movement equipment and the electrocardio equipment are set, then UDP data communication between a data interface module and the eye movement equipment and between the data interface module and the electrocardio equipment is established through a data receiving and sending interface based on a C + + platform, original eye movement data collected by the eye movement equipment are read, original electrocardio data are read, and communication texts are extracted from communication audios coordinated by a team;

b2 Storing the original eye movement data, the original electrocardio data and the extracted communication text which are collected in real time in a memory space;

b3 Original eye movement data and original electrocardio data are subjected to index conversion and averaged, the index conversion comprises the steps of converting the original eye movement data into eye movement characteristic index data comprising fixation rate, saccade rate, pupil diameter and average fixation time and converting the original electrocardio data into electrocardio characteristic index data comprising average heart rate, average RR interval, standardized low-frequency power and standardized high-frequency power,

c) Text analysis is carried out through text analysis software, the occurrence frequency of communication codes and word frequency data of communication keywords are counted for communication texts, communication text characteristic data are obtained,

d) Carrying out online analysis and evaluation on the eye movement data of the team, the electrocardio data of the team and the communication text characteristic data of the team, wherein the online analysis and evaluation comprises the following steps:

d1 Read in the eye movement characteristic index data sent in the data interface module, and carry on the normalization processing including null value and/or abnormal value rejection and include the preconditioning operation of the normalization processing;

d2 Read in the electrocardio characteristic index data sent in the data interface module, and carry out standardized processing including null value and/or abnormal value elimination and preprocessing operation including standardized processing;

d3 Read in the communication text characteristic index data sent in the data interface module, and carry out standardized processing including null value and/or abnormal value elimination, and preprocessing operation including standardized processing;

d4 The standardized and standardized eye movement characteristic index data, the standardized and standardized electrocardio characteristic index data and the standardized and communicated text characteristic index data are read in, and the eye movement characteristic index data, the electrocardio characteristic index data and the communicated text characteristic index data are respectively transmitted to three OAO support vector machines for online calculation: wherein:

the first OAO support vector machine SVM1 takes a vector corresponding to the high team cooperation state as a positive set +1, and takes a vector corresponding to the medium team cooperation state as a negative set-1;

the second OAO support vector machine SVM2 takes the vector corresponding to the high team cooperation state as a positive set +1, and takes the vector corresponding to the low team cooperation state as a negative set-1;

the third OAO support vector machine SVM3 takes the vector corresponding to the middle team cooperation state as a positive set +1, the vector corresponding to the low team cooperation state as a negative set-1,

d5 Respectively calculated by SVM1, SVM2 and SVM3 to obtain 3 classification results, namely classification 1, classification 2 and classification 3,

d6 Using voting to determine a final classification result of team collaborative status level, taking the most appeared result of the 3 classification results as the final classification result,

e) And comprehensively outputting the team information and the final classification result.

6. The support vector machine-based team collaborative status assessment method according to claim 5, wherein:

team information is input through the user interface terminal,

the eye movement device comprises a wearable eye movement device,

the raw eye movement data includes raw spatial position coordinates of the eye movement, blink marks, gaze marks, glance marks,

the electrocardio-device comprises a wearable electrocardio-device,

the original electrocardio data comprises heart rate and electrocardio original waveform data.

7. The support vector machine-based team collaborative status assessment method according to claim 5, wherein:

in the step D4), if the SVM1 is judged to be in the high team cooperation state, the SVM2 is judged to be in the high team cooperation state, the SVM3 is judged to be in the medium team cooperation state, and the final classification result is in the high team cooperation state.

8. The support vector machine-based team collaborative status assessment method according to claim 5, wherein:

in the step B1), reading original eye movement data at the frequency of 60Hz, reading original electrocardio data at the frequency of 1Hz, and extracting a communication text of a communication audio in a team experiment through audio analysis software.

9. A computer-readable storage medium storing a computer program enabling a processor to execute the support vector machine-based team collaborative status assessment method according to one of claims 5-9.

Images