[go: up one dir, main page]

CN111407269A - A Reinforcement Learning-Based EEG Signal Emotion Recognition Method - Google Patents

A Reinforcement Learning-Based EEG Signal Emotion Recognition Method Download PDF

Info

Publication number
CN111407269A
CN111407269A CN202010239133.XA CN202010239133A CN111407269A CN 111407269 A CN111407269 A CN 111407269A CN 202010239133 A CN202010239133 A CN 202010239133A CN 111407269 A CN111407269 A CN 111407269A
Authority
CN
China
Prior art keywords
layer
emotion
eeg
output
eeg data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010239133.XA
Other languages
Chinese (zh)
Inventor
杜广龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South China University of Technology SCUT
Original Assignee
South China University of Technology SCUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China University of Technology SCUT filed Critical South China University of Technology SCUT
Priority to CN202010239133.XA priority Critical patent/CN111407269A/en
Publication of CN111407269A publication Critical patent/CN111407269A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/18—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state for vehicle drivers or machine operators
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2503/00—Evaluating a particular growth phase or type of persons or animals
    • A61B2503/20—Workers
    • A61B2503/22—Motor vehicles operators, e.g. drivers, pilots, captains
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/02—Preprocessing
    • G06F2218/04—Denoising
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/08—Feature extraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
    • G06F2218/12—Classification; Matching

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • Psychiatry (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Animal Behavior & Ethology (AREA)
  • Surgery (AREA)
  • Medical Informatics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Pathology (AREA)
  • Theoretical Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Psychology (AREA)
  • Signal Processing (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Physiology (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Child & Adolescent Psychology (AREA)
  • Developmental Disabilities (AREA)
  • Educational Technology (AREA)
  • Hospice & Palliative Care (AREA)
  • Social Psychology (AREA)
  • Fuzzy Systems (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)

Abstract

本发明公开了一种基于增强学习的EEG信号情感识别方法。包括以下步骤：S1、采用半监督学习机制来过滤训练集中标签错误的脑电图信号；S2、采用向量化卷积来提取EEG通道与频段的关系；S3、将模糊推理与RNN相结合，提取脑电图的有效时态信息，以更好地理解情感；S4、输出该EEG信号对应受试者的情感。本发明采用向量化卷积来提取EEG频段和信道之间的联系，将模糊推理与递归神经网络结合来应对情绪EEG信号的模糊性并提取其中的时间信息。同时，使用无监督学习和标记数据，使用于训练的数据更加准确。

Figure 202010239133

The invention discloses an EEG signal emotion recognition method based on reinforcement learning. It includes the following steps: S1, using a semi-supervised learning mechanism to filter the EEG signals with wrong labels in the training set; S2, using vectorized convolution to extract the relationship between EEG channels and frequency bands; S3, combining fuzzy reasoning with RNN to extract Effective temporal information of EEG to better understand emotion; S4, output the EEG signal corresponding to the emotion of the subject. The invention adopts vectorized convolution to extract the relationship between EEG frequency bands and channels, and combines fuzzy reasoning with recurrent neural network to deal with the ambiguity of emotional EEG signals and extract the time information therein. At the same time, using unsupervised learning and labeled data makes the data used for training more accurate.

Figure 202010239133

Description

EEG signal emotion recognition method based on reinforcement learning
Technical Field
The invention belongs to the technical field of intelligent transportation, and particularly relates to an EEG signal emotion recognition method based on reinforcement learning.
Background
Drivers are in a state of fatigue driving and their ability to recognize road conditions and driving skills is significantly reduced. The results of the study show that 25% -30% of traffic accidents are caused by fatigue driving. To overcome this problem, a system must be developed that can effectively detect fatigue driving of drivers and warn them in time.
The fatigue of the driver can be detected using a wearable device to measure the heart rate of the driver or using an RGB camera to extract facial features. However, the wearable device may cause inconvenience and discomfort to the driver, and the detection accuracy of the RGB camera may be affected by light, glasses, and head orientation. In addition, most existing methods ignore the time information of fatigue features and the relationship between the features thereof, and reduce the recognition accuracy. Furthermore, some existing fatigue detection methods focus on processing fatigue features with temporal slices, ignoring temporal variations in the features.
Vehicle behavior-based methods mainly measure vehicle data such as steering angle, speed, acceleration, and turning angle, without considering the physiological signals that detect driver fatigue and make early warnings. Physiological signal based methods mainly apply Electrograms (EOGs) and electrocardiograms and other physiological signals. Drivers must wear associated equipment that is intrusive, prevents driving, and results in a poor user experience.
In terms of the behavior-based approach, fatigue (as a percentage of eyelid closure (PERC L OS)) is determined by detecting eyelid closure frequency.
Driver fatigue is a continuous time process, so temporal changes in fatigue characteristics are very important for fatigue driving identification. However, existing algorithms focus on dealing with fatigue features that have a short time, ignoring the temporal variation of the fatigue features. Some methods may detect fatigue by measuring heart rate using a wearable device. However, wearing the device is inconvenient for the driver and may make them uncomfortable. Existing fatigue driving detection methods use RGB images to extract eye openness, which can affect light, glasses, and head orientation. There is little existing model to capture temporal information of features and temporal relationship information between features for driver fatigue detection.
Disclosure of Invention
In order to solve the defects in the prior art, the invention provides an EEG signal emotion recognition method based on reinforcement learning, which comprises the following steps:
s1, filtering electroencephalogram signals with label errors in a training set by adopting a semi-supervised learning mechanism;
s2, extracting the relation between an EEG channel and a frequency band by adopting vectorization convolution; a
S3, combining fuzzy reasoning with RNN to extract effective temporal information of electroencephalogram so as to better understand emotion;
and S4, outputting the EEG signal corresponding to the emotion of the subject, wherein the emotion refers to outputting which of positive emotion, negative emotion and neutral emotion the subject belongs to.
Further, labeling the subject's three emotions, positive emotion, negative emotion, and neutral emotion in the EEG data, the label error comprising neutral emotion EEG data wrongly labeled as positive emotion in the positive emotion EEG data, or neutral emotion EEG data wrongly labeled as negative emotion in the negative emotion EEG data,
the electroencephalogram signal with the wrong label in the filtering training set is that the electroencephalogram data with the neutral emotion wrongly labeled as the positive emotion or the negative emotion is screened out by processing the EEG data through a fuzzy clustering algorithm (FCM) based on a target function.
In an EEG dataset, when the overall emotional state of the video used to stimulate the mood of the subject is positive, negative or neutral, the respective EEG data emotional state from the subject will be labeled as positive, negative or neutral. However, during EEG acquisition, positive or negative mood EEG data may contain partially neutral mood data that is incorrectly labeled as positive or negative mood. Such mislabeled electroencephalographic data can greatly interfere with the accuracy of the emotion recognition method.
Further, neutral emotion EEG data is screened from negative emotion EEG data containing neutral emotions by the following specific screening method:
assume that the input EEG data for FCM is X ═ X1,x2,…,xn]Negative emotion EEG data containing negative emotions and neutral emotions,
the EEG data X is divided into two groups by FCM in the time dimension, where one group is neutral electroencephalographic data X that is falsely labeled as negative emotions1＝[x1,x2,…,xi]The other group is negative emotional electroencephalogram data X2＝[xi+1,xi+2,...,xn]，
Computing X by Euclidean1And Xneural、X2And XneuralThe distance between the electroencephalograms is determined to determine the similarity, and the real neutral emotion electroencephalogram data are screened out through the similarity
Figure BDA0002431967890000021
ρ is the length of the selected element sequence.
Further, the vectorization convolution is adopted to extract the relation between the EEG channel and the frequency band; by fusing the information of the two dimensions, the EEG signal can be more comprehensively utilized for emotion classification. The method specifically comprises the following steps: by performing a convolution operation on the vectorized convolution layer,
assuming X is the EEG feature over a period of time, X for each elementiPerforming vectorized convolution, wherein xiThe rows and columns of channels are frequency bands, and x is obtained by a series of convolution operationsiBecomes a vector and the length of the final vector is the number of lanes.
Further, the combination of fuzzy inference and RNN in the time information for extracting heart rate characteristics from EEG signals specifically comprises 6 layers,
the first layer is an input layer and the second layer is an output layer,
the second layer is a fuzzy layer, and the membership value of the data from the first layer is calculated by using a Gaussian membership function which is as follows:
Figure BDA0002431967890000031
wherein m isi、
Figure BDA0002431967890000032
Respectively representing the mean and variance of the gaussian membership functions,
Figure BDA0002431967890000033
the output of the second layer is represented by,
Figure BDA0002431967890000034
indicates the output of layer 1, i indicates the number of neurons, and λ indicates the number of neurons in the second layer.
Electroencephalograms are physiological signals that contain noise. The noise can be attenuated to some extent by the blur layer.
The third layer is a spatial activation layer for calculating the membership degree of each node of the fourth layer, the nodes of the third layer provide spatial activation degrees by using the membership degree operation received from the second layer, and the spatial activation strength is calculated by using continuous cumulative multiplication as a fuzzy operator as follows:
Figure BDA0002431967890000035
wherein,
Figure BDA0002431967890000036
the output of the third layer is shown,
Figure BDA0002431967890000037
λ means the dimensions of the j-th output of the i-th output of layer 2 and the i-th output of layer 2 neurons, respectively.
The fourth layer is a cycle layer for acquiring time-varying features of EEG, and the layer outputs the spatial activation intensity transmitted by the third layer and the last time activation intensity
Figure BDA0002431967890000038
In combination, t represents the current time step number, t-1 represents the last time step number, and each neuron in this layer is calculated as follows:
Figure BDA0002431967890000039
wherein
Figure BDA00024319678900000310
Representing the temporal activation intensity, the output of the third layer and the output of the fourth layer, respectively.
The mood of the subject is related to the change in the electroencephalographic signal over time. RNNs can effectively extract the characteristics of time series data and are therefore applied in this layer. Obtaining time-varying features of the EEG can help accurately identify emotions that vary over time.
The fifth layer is the result layer, and since the defuzzification operation of the sixth layer requires the use of the input of the fuzzification layer, temporal information needs to be merged with the output of the second layer in this layer, specifically, this layer uses the first layer
Figure BDA00024319678900000311
And a fourth layer
Figure BDA00024319678900000312
The outputs of which are weighted linear summation calculations, each node in layer 5 having an output as input for the next layer,
Figure BDA0002431967890000041
where i, j ∈ [1, ρ]，νijωiIs a weight parameter, vij、ωiRepresenting the weight passed by layer 2 to the current layer value and the weight passed by layer 4 to the current layer, respectively.
Figure BDA0002431967890000042
Respectively representing the jth dimension in the ith output of layer 2, the output of the fourth layer, and the output of the fifth layer.
The sixth layer is an output layer, and the defuzzification is executed by adopting a weighted average defuzzification method, which is as follows:
Figure BDA0002431967890000043
y represents the output of the sixth layer and is the corresponding positive or negative or neutral mood of the subject.
In the real model, network parameters and structures are adjusted by utilizing back propagation, and the loss function of the model is as follows:
Figure BDA0002431967890000044
wherein,
Figure BDA0002431967890000045
is the emotion label for the electroencephalogram data and y represents the output of the sixth layer. y isiThe ith element denoted as y.
Compared with the prior art, the invention has the following beneficial effects:
(1) the unsupervised learning approach filters the raw EEG data, making the data used to train the model more accurate. The method enables the model after circulation to be capable of predicting the model more effectively, and improves the identification precision of the model.
(2) Considering that the emotion of a subject is related to the change of electroencephalogram signals along with time, fuzzy reasoning is combined with RNN, wherein the fuzzy reasoning can well process noise, the RNN can effectively extract the characteristics of time series data, and the time-varying characteristics of EEG can be obtained to help accurately identify the emotion along with time. The robustness of the model is effectively improved.
(3) The proposed neural network is enabled to fuse information in both EEG signal channel and frequency dimensions by vectorized convolution. By fusing the information in these two dimensions, the EEG signal can be more fully utilized for emotion classification.
Drawings
Fig. 1 is a flowchart of an EEG signal emotion recognition method based on reinforcement learning according to the present invention.
Fig. 2 is a network structure diagram of the fuzzy neural network in the present invention.
Detailed Description
The invention will be further described with reference to examples and figures, but the embodiments of the invention are not limited thereto.
As shown in fig. 1, the present embodiment provides an EEG signal emotion recognition method based on reinforcement learning, which includes the following steps:
s1, filtering electroencephalogram signals with label errors in a training set by adopting a semi-supervised learning mechanism;
s2, extracting the relation between an EEG channel and a frequency band by adopting vectorization convolution, and performing emotion classification by fusing the information of the two dimensions by utilizing the EEG signal more comprehensively.
S3, combining fuzzy reasoning and RNN, extracting effective temporal information of electroencephalogram to better understand emotion, and outputting the EEG signal corresponding to the emotion of the subject, namely outputting the emotion of the subject to be one of positive emotion, negative emotion and neutral emotion.
In particular, in an EEG dataset, when the overall emotional state of the video used to stimulate the mood of the subject is positive, negative or neutral, the respective EEG data emotional state from the subject will be labeled as positive, negative or neutral mood. However, during EEG acquisition, positive or negative mood EEG data may contain partially neutral mood data that is incorrectly labeled as positive or negative mood. The overall emotional state of the video selected from the data set is negative, and therefore, the electroencephalographic signal collected from the subject is labeled as negative electroencephalographic data. When the subject begins to watch the video clip, it stimulates the subject to use EEG data that is neuro-mood neutral, rather than negative mood. But the data set would mark the emotion tags of these EEG data as negative. Such badly labeled EEG data can greatly interfere with the accuracy of the emotion recognition method.
The electroencephalogram signal with the wrong label in the filtering training set is that the electroencephalogram data with the neutral emotion wrongly labeled as the positive emotion or the negative emotion is screened out by processing the EEG data through a fuzzy clustering algorithm (FCM) based on a target function.
FCM is a fuzzy clustering algorithm based on an objective function, and improves emotion recognition accuracy of a model by calculating Euclidean distance of the distance from each sample to a clustering center to screen out wrongly labeled neutral electroencephalogram data from positive and negative emotion electroencephalogram data. For ease of understanding, a specific step of screening out neutral mood EEG data from negative mood EEG data (containing neutral mood incorrectly labeled as negative mood) by FCM is given below.
Assume that the input EEG data for FCM is X ═ X1,x2,…,xn]Negative emotion EEG data including negative emotions and neutral emotions, and the EEG data X is divided into two groups in a time dimension by FCM, wherein one group is neutral electroencephalogram data X which is wrongly labeled as a negative emotion1＝[x1,x2,…,xi]The other group is negative emotional electroencephalogram data X2＝[xi+1,xi+2,...,xn]Computing X by Euclidean1And Xneutral、X2And XneutralThe distance between the electroencephalograms is determined to determine the similarity, and the real neutral emotion electroencephalogram data are screened out through the similarity
Figure BDA0002431967890000051
Wherein, XneutralThe expression means that the emotion sample data marked as neutral is represented, and ρ is the length of the element sequence to be screened.
In step S2, specifically, the extracting of the relationship between the EEG channel and the frequency band by using vectorized convolution(ii) a By fusing the information of the two dimensions, the EEG signal can be more comprehensively utilized for emotion classification. The method specifically comprises the following steps: the convolution operation is performed by vectorizing the convolutional layer. This embodiment pairs each element X in XiPerforming vectorized convolution, wherein xiThe rows and columns of channels are frequency bands, and x is obtained by a series of convolution operationsiBecomes a vector and the length of the final vector is the number of lanes.
In step S3, the fuzzy inference is combined with RNN, specifically including 6 layers,
the first layer is an input layer, and the input information of the input layer is X.
The second layer is a fuzzy layer, and the membership value of the data from the first layer is calculated by using a Gaussian membership function which is as follows:
Figure BDA0002431967890000061
wherein m isi、
Figure BDA0002431967890000062
Respectively representing the mean and variance of the gaussian membership functions,
Figure BDA0002431967890000063
the output of the second layer is represented by,
Figure BDA0002431967890000064
indicates the output of layer 1, and i indicates the number of the neuron. The meaning of λ is the number of layer 2 neurons.
Electroencephalograms are physiological signals that contain noise. The noise can be attenuated to some extent by the blur layer.
The third layer is a spatial activation layer for calculating the membership degree of each node of the fourth layer, the nodes of the third layer provide spatial activation degrees by using the membership degree operation received from the second layer, and the spatial activation strength is calculated by using continuous cumulative multiplication as a fuzzy operator as follows:
Figure BDA0002431967890000065
wherein,
Figure BDA0002431967890000066
the output of the third layer is shown,
Figure BDA0002431967890000067
λ means the output of layer 2 and the number of layer 2 neurons, respectively.
The fourth layer is a cycle layer for acquiring time-varying features of EEG, and the layer outputs the spatial activation intensity transmitted by the third layer and the temporal activation intensity of the last time
Figure BDA0002431967890000068
And (4) combining. t represents the current time step number, and t-1 represents the last time step number. Each neuron of this layer is calculated as follows:
Figure BDA0002431967890000069
wherein
Figure BDA00024319678900000610
Representing the temporal activation intensity, the output of layer 3 and the output of layer 4, respectively.
The mood of the subject is related to the change in the electroencephalographic signal over time. RNNs can effectively extract the characteristics of time series data and are therefore applied in this layer. Obtaining time-varying features of the EEG can help accurately identify emotions that vary over time.
The fifth layer is the result layer, and since the defuzzification operation of the sixth layer requires the use of the input of the fuzzification layer, temporal information needs to be merged with the output of the second layer in this layer, specifically, this layer uses the first layer
Figure BDA00024319678900000611
And a fourth layer
Figure BDA00024319678900000612
The outputs of which are weighted linear summation calculations, each node in layer 5 having an output as input for the next layer,
Figure BDA00024319678900000613
where i, j ∈ [1, ρ]，νij、ωiAre weight parameters representing the weight passed by layer 2 to the current layer value and the weight passed by layer 4 to the current layer, respectively.
Figure BDA0002431967890000071
Respectively representing the jth dimension, the 4 th layer output and the 5 th layer output in the ith output of the 2 layers.
The sixth layer is an output layer, and the defuzzification is executed by adopting a weighted average defuzzification method, which is as follows:
Figure BDA0002431967890000072
y represents the output of the sixth layer, and the emotion of the output subject is either a positive emotion or a negative emotion or a neutral emotion.
In the model, network parameters and structures are adjusted by utilizing back propagation, and the loss function of the model is as follows:
Figure BDA0002431967890000073
wherein,
Figure BDA0002431967890000074
is an emotion label for electroencephalogram data, y represents the output of layer 6, yiThe ith element denoted as y.
The present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which are made without departing from the spirit and principle of the invention are equivalent substitutions and are within the scope of the invention.

Claims (5)

1.一种基于增强学习的EEG信号情感识别方法，其特征在于，包括以下步骤：1. an EEG signal emotion recognition method based on reinforcement learning, is characterized in that, comprises the following steps: 采用半监督学习机制来过滤训练集中标签错误的脑电图信号；Adopt a semi-supervised learning mechanism to filter the EEG signals that are mislabeled in the training set; 采用向量化卷积来提取EEG通道与频段的关系；Use vectorized convolution to extract the relationship between EEG channels and frequency bands; 将模糊推理与RNN相结合，提取脑电图的时态信息；Combine fuzzy reasoning with RNN to extract the temporal information of EEG; 输出该EEG信号对应受试者的情感。Outputting the EEG signal corresponds to the emotion of the subject. 2.根据权利要求1所述的一种基于增强学习的EEG信号情感识别方法，其特征在于：EEG数据中标注受试者的积极情绪、消极情绪和中性情绪三种情感，所述标签错误是指在积极情绪EEG数据中包含有错误标记为积极情绪的中性情绪EEG数据，或在消极情绪EEG数据中包含有错误标记为消极情绪的中性情绪EEG数据，2. a kind of EEG signal emotion recognition method based on reinforcement learning according to claim 1, is characterized in that: label experimenter's positive emotion, negative emotion and neutral emotion three kinds of emotion in EEG data, described label is wrong Refers to positive emotion EEG data containing neutral emotion EEG data incorrectly labeled as positive emotion, or negative emotion EEG data containing neutral emotion EEG data incorrectly labeled as negative emotion, 所述过滤训练集中标签错误的脑电图信号是指，采用基于目标函数的模糊聚类算法FCM对EEG数据进行处理，筛选出被错误标注为积极情绪或消极情绪的中性情绪脑电图数据。The filtering of the wrongly labeled EEG signals in the training set refers to using the objective function-based fuzzy clustering algorithm FCM to process the EEG data, and to screen out the neutral emotional EEG data that are wrongly labeled as positive emotions or negative emotions. . 3.根据权利要求2所述的一种基于增强学习的EEG信号情感识别方法，其特征在于，从包含有中性情绪的消极情绪EEG数据中筛选出中性情绪EEG数据，具体筛选方法如下：3. a kind of EEG signal emotion recognition method based on reinforcement learning according to claim 2, is characterized in that, from the negative emotion EEG data that comprises neutral emotion, screen out neutral emotion EEG data, concrete screening method is as follows: 假设FCM的输入EEG数据是X＝[x1,x2,…,xn]，是包含有消极情绪和中性情绪的消极情绪EEG数据，Assuming that the input EEG data of FCM is X=[x 1 ,x 2 ,...,x n ], it is negative emotion EEG data containing negative emotion and neutral emotion, 通过FCM将EEG数据X按时间维度分成两组，其中一组是被错误标记为消极情绪的中性脑电图数据X1＝[x1,x2,…,xi]，另一组是消极情绪脑电图数据X2＝[xi+1,xi+2,...,xn]，The EEG data X is divided into two groups by time dimension by FCM, one of which is the neutral EEG data X 1 =[x 1 ,x 2 ,..., xi ] which is wrongly labeled as negative emotion, and the other group is Negative emotion EEG data X 2 =[x i+1 ,x i+2 ,...,x n ], 采用欧几里得计算X1和Xneural、X2和Xneural之间的距离，以确定相似性，通过相似性筛选出真实的中性情绪脑电图数据
Figure FDA0002431967880000011
ρ是筛选要素序列的长度。Euclidean is used to calculate the distance between X 1 and X neural , X 2 and X neural to determine the similarity, and filter out the real neutral emotional EEG data through the similarity
Figure FDA0002431967880000011
ρ is the length of the screening feature sequence. 4.根据权利要求1所述的一种基于增强学习的EEG信号情感识别方法，其特征在于，所述采用向量化卷积来提取EEG通道与频段的关系，具体为：通过向量化卷积层进行卷积操作，对X中的每个元素xi执行向量化卷积，其中xi的行是通道列是频段，通过一系列卷积操作将xi变成一个向量，最终向量的长度是通道的数量。4. a kind of EEG signal emotion recognition method based on reinforcement learning according to claim 1, is characterized in that, described adopting vectorized convolution to extract the relationship between EEG channel and frequency band, specifically: by vectorized convolution layer Perform a convolution operation, perform a vectorized convolution on each element x i in X, where the rows of x i are the channels and the columns are the frequency bands, turn x i into a vector through a series of convolution operations, and the length of the final vector is number of channels. 5.根据权利要求1所述的一种基于增强学习的EEG信号情感识别方法，其特征在于，所述将模糊推理与RNN相结合，具体包括6层，5. a kind of EEG signal emotion recognition method based on reinforcement learning according to claim 1, is characterized in that, described combining fuzzy reasoning and RNN, specifically comprises 6 layers, 第一层为输入层，输入层的输入信息是X，The first layer is the input layer, the input information of the input layer is X, 第二层为模糊层，使用高斯隶属函数计算来自第一层的数据的隶属资格值，高斯隶属函数如下：The second layer is the fuzzy layer, which uses the Gaussian membership function to calculate the membership value of the data from the first layer. The Gaussian membership function is as follows:
Figure FDA0002431967880000021
Figure FDA0002431967880000021
 其中，mi、
Figure FDA0002431967880000022
分别表示高斯隶属函数的均值和方差，
Figure FDA0002431967880000023
表示第二层的输出，
Figure FDA0002431967880000024
表示第1层的输出，i表示是神经元的序号，λ代表的含义是第2层神经元的个数Among them, m i,
Figure FDA0002431967880000022
are the mean and variance of the Gaussian membership function, respectively,
Figure FDA0002431967880000023
represents the output of the second layer,
Figure FDA0002431967880000024
Represents the output of the first layer, i represents the serial number of the neuron, and λ represents the meaning of the number of neurons in the second layer 第三层为空间激活层，用于计算第四层的每个节点的隶属化程度，第三层的节点使用从第二层接收的隶属度操作提供空间激活度，使用连续累积乘法作为模糊运算符，空间激活强度计算如下：The third layer is the spatial activation layer, which is used to calculate the membership degree of each node of the fourth layer, the nodes of the third layer provide the spatial activation degree using the membership degree operation received from the second layer, and use the continuous cumulative multiplication as the fuzzy operation , the spatial activation strength is calculated as follows:
Figure FDA0002431967880000025
Figure FDA0002431967880000025
 其中，
Figure FDA0002431967880000026
表示第三层的输出，
Figure FDA0002431967880000027
λ分别是第2层的第i个输出中的第j维和第2层神经元的第i个输出的维度，in,
Figure FDA0002431967880000026
represents the output of the third layer,
Figure FDA0002431967880000027
λ is the jth dimension in the ith output of layer 2 and the dimension of the ith output of the neuron in layer 2, respectively, 第四层为循环层，用于获取EEG的时变特征，此层输出将第三层传输的空间激活强度与上次的时间激活强度
Figure FDA0002431967880000028
相结合，t表示当前时间步数，t-1表示上一时间步数，此层的每个神经元按如下方式计算：The fourth layer is the recurrent layer, which is used to obtain the time-varying features of EEG. This layer outputs the spatial activation intensity transmitted by the third layer and the last time activation intensity.
Figure FDA0002431967880000028
Combined, t represents the current time step number, t-1 represents the previous time step number, and each neuron in this layer is calculated as follows:
Figure FDA0002431967880000029
Figure FDA0002431967880000029
 其中
Figure FDA00024319678800000210
分别代表时间激活强度、第3层的输出和第4层的输出in
Figure FDA00024319678800000210
represent the temporal activation strength, the output of layer 3 and the output of layer 4, respectively 第五层为结果层，此层使用第一层
Figure FDA00024319678800000211
和第四层
Figure FDA00024319678800000212
的输出进行加权线性求和计算，第五层中的每个节点都有一个输出作为下一个图层的输入，The fifth layer is the result layer, which uses the first layer
Figure FDA00024319678800000211
and the fourth floor
Figure FDA00024319678800000212
A weighted linear summation is performed on the outputs of
Figure FDA00024319678800000213
Figure FDA00024319678800000213
 其中，i,j∈[1,ρ]，νij、ωi是权重参数，分别代表第2层传递到当前层值的权重和第4层传递到当前层的权重，
Figure FDA00024319678800000214
分别表示第二层、第四层和第五层的输出，Among them, i,j∈[1,ρ], ν ij , ω i are the weight parameters, which represent the weight passed to the current layer value from the second layer and the weight passed to the current layer from the fourth layer, respectively,
Figure FDA00024319678800000214
represent the outputs of the second, fourth and fifth layers, respectively, 第六层为输出层，执行去模糊化，采用加权平均脱模糊法，如下所示：The sixth layer is the output layer, which performs defuzzification, using the weighted average deblurring method, as follows:
Figure FDA00024319678800000215
Figure FDA00024319678800000215
 y表示第六层的输出，输出受试者的情感。y represents the output of the sixth layer, which outputs the emotion of the subject. 利用反向传播来调整网络参数和结构，模型的损耗函数如下：Using backpropagation to adjust the network parameters and structure, the loss function of the model is as follows:
Figure FDA0002431967880000031
Figure FDA0002431967880000031
 其中，
Figure FDA0002431967880000032
是脑电图数据的情感标签，y代表第六层的输出，yi表示为y的第i个元素。in,
Figure FDA0002431967880000032
is the sentiment label of the EEG data, y represents the output of the sixth layer, and yi is the ith element of y.
CN202010239133.XA 2020-03-30 2020-03-30 A Reinforcement Learning-Based EEG Signal Emotion Recognition Method Pending CN111407269A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010239133.XA CN111407269A (en) 2020-03-30 2020-03-30 A Reinforcement Learning-Based EEG Signal Emotion Recognition Method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010239133.XA CN111407269A (en) 2020-03-30 2020-03-30 A Reinforcement Learning-Based EEG Signal Emotion Recognition Method

Publications (1)

Publication Number Publication Date
CN111407269A true CN111407269A (en) 2020-07-14

Family

ID=71486294

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010239133.XA Pending CN111407269A (en) 2020-03-30 2020-03-30 A Reinforcement Learning-Based EEG Signal Emotion Recognition Method

Country Status (1)

Country Link
CN (1) CN111407269A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112949684A (en) * 2021-01-28 2021-06-11 天津大学 Multimodal dialogue emotion information detection method based on reinforcement learning framework
CN115905837A (en) * 2022-11-17 2023-04-04 杭州电子科技大学 A Semi-supervised Adaptive Marker Regression EEG Emotion Recognition Approach for Automatic Anomaly Detection
CN116010858A (en) * 2023-03-28 2023-04-25 同心智医科技(北京)有限公司 Channel Attention MLP-Mixer Network Model Based on Self-Supervised Learning and Its Application
CN117195153A (en) * 2023-09-20 2023-12-08 安徽大学 EEG-EOG multi-modal emotion recognition method and system based on contrastive learning

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170135621A1 (en) * 2015-11-16 2017-05-18 Samsung Electronics Co., Ltd. Apparatus and method to train autonomous driving model, and autonomous driving apparatus
CN108898166A (en) * 2018-06-13 2018-11-27 北京信息科技大学 A kind of image labeling method
CN110069958A (en) * 2018-01-22 2019-07-30 北京航空航天大学 A kind of EEG signals method for quickly identifying of dense depth convolutional neural networks
CN110236533A (en) * 2019-05-10 2019-09-17 杭州电子科技大学 Epileptic seizure prediction method based on the study of more deep neural network migration features
CN110399857A (en) * 2019-08-01 2019-11-01 西安邮电大学 A EEG Emotion Recognition Method Based on Graph Convolutional Neural Network
CN110428003A (en) * 2019-07-31 2019-11-08 清华大学 Method, device and electronic device for correcting sample category label
CN110866537A (en) * 2019-09-27 2020-03-06 华南理工大学 Brain wave-based emotion recognition method for game evaluation

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170135621A1 (en) * 2015-11-16 2017-05-18 Samsung Electronics Co., Ltd. Apparatus and method to train autonomous driving model, and autonomous driving apparatus
CN110069958A (en) * 2018-01-22 2019-07-30 北京航空航天大学 A kind of EEG signals method for quickly identifying of dense depth convolutional neural networks
CN108898166A (en) * 2018-06-13 2018-11-27 北京信息科技大学 A kind of image labeling method
CN110236533A (en) * 2019-05-10 2019-09-17 杭州电子科技大学 Epileptic seizure prediction method based on the study of more deep neural network migration features
CN110428003A (en) * 2019-07-31 2019-11-08 清华大学 Method, device and electronic device for correcting sample category label
CN110399857A (en) * 2019-08-01 2019-11-01 西安邮电大学 A EEG Emotion Recognition Method Based on Graph Convolutional Neural Network
CN110866537A (en) * 2019-09-27 2020-03-06 华南理工大学 Brain wave-based emotion recognition method for game evaluation

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112949684A (en) * 2021-01-28 2021-06-11 天津大学 Multimodal dialogue emotion information detection method based on reinforcement learning framework
CN115905837A (en) * 2022-11-17 2023-04-04 杭州电子科技大学 A Semi-supervised Adaptive Marker Regression EEG Emotion Recognition Approach for Automatic Anomaly Detection
CN116010858A (en) * 2023-03-28 2023-04-25 同心智医科技(北京)有限公司 Channel Attention MLP-Mixer Network Model Based on Self-Supervised Learning and Its Application
CN117195153A (en) * 2023-09-20 2023-12-08 安徽大学 EEG-EOG multi-modal emotion recognition method and system based on contrastive learning
CN117195153B (en) * 2023-09-20 2025-08-05 安徽大学 EEG-EOG multimodal emotion recognition method and system based on contrastive learning

Similar Documents

Publication Publication Date Title
Rastgoo et al. Automatic driver stress level classification using multimodal deep learning
Hu et al. EEG-based driver fatigue detection using spatio-temporal fusion network with brain region partitioning strategy
Dwivedi et al. Drowsy driver detection using representation learning
Phyo et al. TransSleep: Transitioning-aware attention-based deep neural network for sleep staging
CN111407269A (en) A Reinforcement Learning-Based EEG Signal Emotion Recognition Method
Liu et al. Real time detection of driver fatigue based on CNN‐LSTM
Batool et al. An ensemble deep learning model for human activity analysis using wearable sensory data
Dipu et al. Real-time driver drowsiness detection using deep learning
Bhatia et al. Retinal vessel extraction via assisted multi-channel feature map and U-net
Kayadibi et al. An eye state recognition system using transfer learning: AlexNet-based deep convolutional neural network
CN115743137B (en) Driving situation understanding method based on man-machine enhanced perception
CN111516700A (en) Driver distraction fine-granularity monitoring method and system
CN116129405A (en) Method for identifying anger emotion of driver based on multi-mode hybrid fusion
CN116524380A (en) Target detection method based on brain-computer signal fusion
CN111652036B (en) A Vision-Based Fatigue Driving Recognition Method Fused with Heart Rate and Facial Features
CN117076980A (en) Driving intention recognition method based on functional connection and graph neural network
Xiao et al. FDAN: Fuzzy deep attention networks for driver behavior recognition
Hashemi et al. Towards safer roads: a deep learning-based multimodal fatigue monitoring system
Chen et al. Deep learning approach for detection of unfavorable driving state based on multiple phase synchronization between multi-channel EEG signals
Duan et al. Enhancing cross-dataset performance of distracted driving detection with Score Softmax classifier and dynamic Gaussian smoothing supervision
Amadori et al. Decision anticipation for driving assistance systems
Gawande et al. Deep-learning approach for efficient eye-blink detection with hybrid optimization concept
Papapanagiotou et al. Self-supervised feature learning of 1D convolutional neural networks with contrastive loss for eating detection using an in-ear microphone
CN120199473B (en) Multi-mode autism diagnosis method, system, medium and product based on resting state fMRI and phenotype text information
Yang et al. An improved CapsNet based on data augmentation for driver vigilance estimation with forehead single-channel EEG

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200714