On some generalized Raina-type fractional-order integral operators and related Chebyshev inequalities

1.   Introduction

Emotion is a reflection of a person's psychological or physiological state. It guides people's behavior to a certain extent. In daily life, emotions always play a very important role, especially in human perception and decision making ^[1]. Emotion analysis is an interdisciplinary research, involving biology, computer science, artificial intelligence and other fields. It is widely used in medical monitoring, safe driving, product design and other fields.

At present, there are two ways of emotion recognition. One kind is based on the non-physiological signal, through the external behavior of the human body, such as tone of voice, facial expression, body posture, text, etc. ^[2]. The non-physiological signal information is easy to collect. Natural language processing and speech recognition has been widely used in the field of machine learning research. Technology is relatively mature. Satisfactory results have been obtained in emotion recognition of non-physiological signals. However, external signals may be disguised. Because people can control their own behavior. Even text data will be affected by regional and cultural differences, so the accuracy of emotion recognition of non-physiological signals cannot be guaranteed. The other is emotion recognition based on physiological signals. The common physiological signals such as electrocardiogram, EEG, electromyogram ^[3]. Compared with the emotion generated by external signals, physiological signals are controlled by the central nervous system and cannot be manipulated subjectively. So the reflection of emotions is more objective, authentic and reliable. Among these physiological signals, EEG signals are mostly used.

In the aspect of EEG emotion recognition, the early studies mainly used the combination of feature extraction and shallow machine learning algorithms. Hosseini and Naghibi-Sistani ^[4] used pictures from the International Affective Picture System (IAPS) to stimulate subjects, and extracted approximate entropy and wavelet entropy from the EEG signals of five channels. The classification was carried out by support vector machine (SVM). The accuracy was 73.25% in two emotional states, calm-neutral and negatively excited. Daimi and Saha ^[5] used Dual-Tree Complex Wavelet Packet Transform to extract time-frequency features. The SVM was used in emotion recognition. DEAP dataset is used to validate the method. The identification accuracy was 65.3, 66.9, 71.2 and 69.1% in the four dimensions of valance, arousal, liking and dominance, respectively. Mehmood and Lee ^[6] extracted the time domain and frequency domain features of Hjorth parameters. Particle swarm optimization (PSO) and genetic search (GS) were used for feature selection, and SVM was used as classifier. Bastos-Filho et al. ^[7] extracted three kinds of EEG signal features, used K-nearest neighbor (KNN) for emotion recognition, and classified calm and stress emotions in DEAP data. The highest accuracy was 70.1% based on power spectral density (PSD) feature. Yosi et al. ^[8] used wavelet function to extract features and combined multi-layer perceptron (MLP) to classify four emotions of EEG.

In addition, empirical mode decomposition (EMD) has been widely used in EEG signal processing. Zhuang et al. ^[9] decomposed EEG signals into IMFs by EMD. The first difference of IMFs sequence, the first difference of phase and the energy were extracted as features. The experimental results on DEAP data showed that the accuracy of arousal was 71.99%, and the accuracy of valence was 69.1%. Ozel and Akan ^[10] used multiple EMD to get the IMFs. Gram-Schmidt ortho-method was used to extract effective IMF, empirical wavelet transform was used to extract the features of effective IMF. Decision tree and ensemble classifiers are used to classify arousal, valence and dominance, and the classification performance is 72.7, 62.0 and 64.7%, respectively.

In recent years, with the development of artificial intelligence technology, emotion recognition of EEG signals is no longer limited to traditional machine learning methods. Neural networks have developed from the original MLP to deep learning, which is popular at present. Deep learning has been widely used to recognize emotion of EEG signals. Li et al. ^[11] proposed a hybrid deep learning model. The continuous wavelet transform and scale-map transform were used to preprocess the multi-channel EEG signals. Convolutional Neural Network (CNN) was used to extract features automatically. Long short-term memory (LSTM) was used for emotion recognition. Pandey and Seeja ^[12] proposed a VMD and deep neural network (DNN) EEG Signal Emotion Recognition method. Hwang et al. ^[13] considered local information within multiple channels or multiple frequency bands of EEG signals, and used CNN to evaluate the positive, neutral, and negative emotional states on the SEED dataset. Li et al. ^[14] introduced local and global attention mechanisms to highlight the transferable brain region samples, and proposed a transferable attention neural network. Three datasets of SEED, SEED-IV and MPED were used for verification. Wang et al. ^[15] used Transformer encoder to learn spatial information from different brain regions hierarchically and combined attention mechanisms to emphasize different contributions of brain regions. Cheng et al. ^[16] converted the data into 2D frame sequences and used deep forest for emotion recognition based on subject-dependent EEG.

Through literature analysis, we found that the classification research on EEG emotion mainly uses different classification methods to classify different emotional states. Most of them use DEAP datasets for emotion analysis, because it is typical EEG emotion dataset that is publicly available. Most of the existing studies use a single variable selection method, and the univariate filtering method only considers the correlation with the dimension of emotion, so the selected features may be too redundant. The forest with poor fitting quality in ensemble learning will have a negative impact on the overall performance of the model, thus reducing the accuracy of EEG recognition. To solve this issue, we propose a new EEG emotion recognition method. The main contributions of this study are as follows.

1) A compositional framework for EEG emotion recognition is proposed, which integrates VMD, Pearson correlation coefficient, AEN and CF.

2) A new feature selection method is proposed. The quadratic penalty term of the AEN is weighted. The minimum common redundancy maximum relevance criterion is used to calculate the weight of the penalty term, which takes into account not only the redundancy among variables, but also the relevance and redundancy between variables and targets.

3) An adaptive weighted CF algorithm is constructed. Classifiers with good performance are given higher weights to reduce the negative impact of classifiers with poor performance.

2.   Materials and methods

2.1.   Materials

The dataset used in this paper are from a public multimodal database established by Sander Koelstra's team at the University of London ^[17]. DEAP is a classical dataset widely used in EEG emotion analysis. The EEG signals was collected from 32 healthy subjects. The experimental process was to make 32 subjects wear "10–20" international lead standard electrode cap. There were 40 electrode channels, including 32 channels of EEG signals. The remaining 8 channels generated other physiological signals such as electrooculography, electromyography. The electrode distribution of the EEG signal channel is shown in Figure 1, the sampling frequency was 512 HZ. Each subject watched 40 pieces of music videos. Each video was 1 minute long. 63s of EEG were collected each time, including 60s of watching the music video and 3s of baseline EEG without emotional stimulation before watching each video. The labels were generated by the subjects' subjective evaluation of arousal, valence, dominance and liking degree on a scale of 1–9 according to self-assessment manikin (SAM). In this study, 14 channels of EEG signals are used to experiment on the two dimensions of arousal and valence. The 14 channels used are AF3, F3, F7, FC5, T7, P7, O1, AF4, F4, F8, FC6, T8, P8 and O2, which are highlighted in red in Figure 1.

2.2.   Methods

2.2.1.   Proposed EEG emotion recognition model

The proposed EEG emotion recognition framework mainly includes five stages: decomposition, extraction of effective components, feature extraction, variable selection and classification. The framework of emotion recognition of EEG is shown in Figure 2.

The process is as follows:

1) Data preprocessing. The data are downsampled. The EEG signals needed for the experiment were selected.

2) EEG signal decomposition. There seem to be few studies using VMD in sentiment analysis. Each preprocessed EEG signal sequence is decomposed by VMD to obtain K IMFs.

3) Effective component extraction. Since some components after decomposition have noise, the Pearson correlation coefficient between each IMF and the original signal is calculated. The components whose correlation value is greater than a certain threshold are selected as the effective components. The components whose correlation coefficient is less than the threshold are deleted.

4) Data segmentation. The IMF signals obtained in 3) are segmented with a sliding window of 1s in length, without overlap. The sliding step is 1s.

5) Feature extraction. PE, PSD, DE and Hjorth parameter characteristics are calculated for each segment data.

6) Variable selection. In order to reduce the complexity of the model, the dimension of extracted features is reduced. The proposed CRRAEN algorithm is used to select the feature variables and obtain the most valuable features.

7) EEG emotion classification. In the classification stage, the features after dimensionality reduction are used to train the WCF classifier. Model performance is evaluated using the evaluation metrics.

2.2.2.   VMD

VMD ^[18] is a quasi-orthogonal and adaptive signal decomposition method. The main idea of VMD is to decompose the original signal into IMFs with different center frequencies and bandwidths by iteratively solving the optimal solution of the variational equation. VMD is suitable for processing nonlinear and non-stationary signals, which has obvious advantages over EMD ^[19], and effectively solves the mode aliasing problem of EMD.

The mathematical expression of the variational problem with constraints is as follows:

$f$ is original input signal, $j$ is an imaginary unit, ${\partial _t}$ is the partial derivative of $t$, $*$ represents the convolution, $\delta \left(t \right)$ is the impact signal, ${v_k}: = \left\{ {{v_1}, \cdots, {v_K}} \right\}$ represents IMFs after VMD decomposition, K represents the number of IMFs, $\left\{ {{\omega _k}} \right\}: = \left\{ {{\omega _1}, \cdots, {\omega _K}} \right\}$ represents the center frequency of each component. $\sum\nolimits_k : = \sum\nolimits_{k = 1}^K {}$ represents the sum of all modes.

The quadratic penalty factor $\alpha$ and Lagrange multiplication operator $\lambda \left(t \right)$ are introduced into Eq (1) to convert the constrained problem into an unconstrained problem. $\alpha$ is the balance parameter of data fidelity constraint, then the function expression is:

In order to obtain the optimal solution of Eq (2), the alternating direction multiplier method is adopted to continuously update $v_k^{n + 1}, {\text{ }}\omega _k^{n + 1}, {\text{ }}\lambda _k^{n + 1}$ iteratively to obtain the "saddle point" of Eq (2), where the expression of $v_k^{n + 1}$ is:

where ${\omega _k} = \omega _k^{n + 1}$, $\sum\nolimits_i {{v_i}\left(t \right)} = \sum\nolimits_{i \ne k} {} v_i^{}{\left(t \right)^{n + 1}}$.

Fourier transform is applied to Eq (3) to obtain the frequency domain expressions of IMF and center frequency, respectively.

$\hat v_k^{n + 1}$ is Wiener filtering of $\hat f\left(\omega \right) - \sum\nolimits_{i \ne k} {\hat v_i^{}\left(\omega \right)}$, $\omega _k^{n + 1}$ is center frequency; n is the number of iterations, ${\hat v_k}\left(\omega \right)$ denotes the inverse Fourier transform. The real part of the result is $v_k^{}\left(t \right)$.

EEG signals have obvious nonlinear non-stationary characteristics. VMD is beneficial to reduce the non-stationarity of EEG signals.

2.2.3.   Feature extraction

EEG signal is a kind of physiological signal with strong randomness and high time-varying sensitivity. When the EEG signal is directly input into the classifier, the classifier cannot get the ideal result. So feature extraction is necessary. This paper mainly extracts the PSD, permutation entropy (PE), differential entropy (DE) and Hjorth parameter features of EEG signals.

1) PSD

PSD represents the signal power in the unit frequency band of the input signal, which is utilized to measure the mean square value of random variables. The simplest method to solve the PSD is the periodogram method. Because the periodogram method is biased estimation, the obtained power spectrum is not smooth. The Welch method is used in this paper to calculate the PSD ^[20].

The sequence data $x\left(t \right)$ is divided into L segments. Each segment has M data. The window function $w\left(t \right)$ operates on each segment of data. The power spectral of the $i\begin{array}{*{20}{c}} {{\text{th}}} \end{array}$segment is calculated.

$U = \frac{1}{M}\sum\limits_{t = 0}^{M - 1} {{w^2}\left(t \right)}$, the PSD is as follows:

2) PE

PE is a simple and fast nonlinear method to quantitatively analyze the complexity of sequences. It can detect the real-time dynamic characteristics of sequences and has strong anti-noise ability ^[21].

Suppose there is a set of time series $X = \left\{ {{x_1}, {x_2}, \cdots, {x_n}} \right\}$. According to the reconstruction theory, the phase space of element ${x_i}$ in X is reconstructed. The obtained reconstruction matrix is ${X_i} = \left\{ {{x_i}, {x_{i + 1}}, \cdots, {x_{i + \left({m - 1} \right)\tau }}} \right\}$. $m$ is the embedding dimension and $\tau$ represents the delay time.

Each sequence in the reconstructed vector is rearranged in ascending order, resulting in a new vector consisting of the original position coordinates of the elements in the vector.

${j_1}, {j_2}, \cdots, {j_m}$ is the position of each element in ${X_i}$.

For any type of positional order, there are $m!$ possible permutations. The frequency ${p_1}, {p_2}, \cdots, {p_k}$ of occurrence of the number of each permutation corresponding to the k reconstructed components in the whole permutation is calculated as the probability, where $k \leqslant m!$. Calculate the information entropy of all types of probabilities.

The PE of time series is defined as:

PE measures the randomness of time series. The larger the PE, the more irregular the time series.

3) DE

DE is used to measure the complexity of continuous variables. Previous studies have shown that DE can provide relatively stable EEG features ^[22,23]. DE is calculated as follows.

The distribution of EEG signal sequence approximately follows the Gaussian distribution of $N\left({\mu, {\sigma ^2}} \right)$. According to the formula (11), the DE is equal to the logarithm of the energy spectrum in a certain frequency band.

4) Hjorth parameter

Hjorth parameter ^[24] is widely used in EEG signal analysis ^[25,26] due to its low computational cost. It includes three parameters: Activity, Mobility and Complexity.

Activity represents the amplitude characteristics of the EEG, which is defined as the variance of the signal.

Mobility represents the time scale of the EEG signal. It is defined as the ratio of the standard deviation of the first difference to the standard deviation of the EEG signal.

Complexity is used to characterize the complexity of EEG signals. It is defined as the ratio of the mobility of the first derivative of the EEG signal to the mobility of the vibration signal.

$\sigma ''$ is the standard deviation of the second-order difference of the original vibration signals.

2.2.4.   AEN based on minimum common redundancy maximum relevance

The coefficient of AEN ^[27] is estimated as

${\hat w_i}$ is the adaptive penalty weight, ${\hat w_i} = {\left| {{{\hat \beta }_{i, en}}} \right|^{ - \gamma }}$, ${\hat \beta _{i, en}}$ is the coefficient estimate of the elastic net, $\gamma > 0$ is constant.

The minimum redundancy maximum relevance (mRMR) criterion proposed by Ding and Peng ^[28] uses mutual information (MI) as a metric criterion, and aims to find out M features that have the largest relevance with the target variable and the minimum mutual redundancy from the feature set space.

where, $S$ and $\left| S \right|$ represent the set of features and the number of features, respectively. c is the target variable. $I\left({{x_j}, c} \right)$ represents the MI between variable ${x_j}$ and target variable c. $I\left({{x_j}, {x_i}} \right)$ represents the MI between variable ${x_j}$ and variable ${x_i}$.

The redundant part of mRMR only considers the MI between features, but ignores the common mutual information (CMI) between features and target variables. The estimation of CMI is constructed by Bennasar's "maximum of the minimum" method ^[29]. Based on AEN and mRMR, a new variable selection method called AEN based on minimum common redundancy maximum relevance, abbreviated as CRRAEN, is proposed. Weight is given to the quadratic penalty term of the AEN. At the same time, the minimum common redundancy maximum relevance (mCRMR) criterion is used to calculate the weight matrix.

The maximum common redundant MI can be expressed as follows. Assume that X is the candidate variable and Y is the selected variable. The CMI is determined by the redundancy information rate of the candidate variable and the MI between the selected variable, the candidate variable and the target variable. CMI is expressed as

The Eq (17) represents the redundancy of candidate variable ${X}_{k}$ and selected variable subset $S$ with respect to c. According to the minimum common redundancy maximum relevance criterion, the MI can be expressed as

For $n \times p$-dimensional data, the importance of defining the $k\begin{array}{*{20}{c}} {{\text{th}}} \end{array}$variable is expressed as:

The weight coefficient of the $k\begin{array}{*{20}{c}} {{\text{th}}} \end{array}$variable is

where, $0 < \eta \leqslant 1$ indicates the given threshold. When ${I_k} > \eta$, the $k\begin{array}{*{20}{c}} {{\text{th}}} \end{array}$variable has obvious significance. When ${I_k} \leqslant \eta$, the $k\begin{array}{*{20}{c}} {{\text{th}}} \end{array}$variable is not significant in predicting the explanatory variable. The weight matrix is expressed as:

The penalty term of the AEN is reconstructed. The weight is calculated according to (21).

The CRRAEN adaptive variable selection model is expressed as

$0 \leqslant \alpha \leqslant 1$, $\lambda$ is the penalty parameter, and the estimated value can be expressed as

MCRMR is used to remove the common redundant information in the affective features. The purer emotional feature information is incorporated into the calculation of the weight of the penalty term, which is more conducive to get the real coefficient estimate. The model considers the global normalization of sentiment dimension, avoids the complex calculation of MI and redundant estimation, and realizes the control of role of redundant variables.

2.2.5.   Weighted CF

CF is an ensemble algorithm proposed by Zhou and Feng ^[30] in 2017. Inspired by DNN's layer-by-layer feature representation learning, a CF is established. Each cascade is composed of multiple layers. Generally, each layer is composed of two random forests (RF) and two completely random forests (CRF), respectively. The data characteristics are processed layer by layer through the CF stage, which strengthens the representation learning ability of the algorithm and is conducive to improving the accuracy of prediction. In the stage of the CF, in addition to the first layer with original features as input data. Each layer from the previous layer to obtain the processed characteristic information. The processing result output of this layer is passed to the next layer. Each subsequent layer concatenates the output result of the previous layer with the original input feature vector as the input of this layer. The CF structure is shown in Figure 3.

The difference of RF generalization ability is not considered in the CF. Forests with poor generalization ability will have a negative impact on the overall performance of the model. In order to alleviate this problem, we propose a Weighted Cascade Forest (WCF). Each learner at each level is given a weight. The weight β is assigned to each classifier according to its classification performance, as shown in Figure 4. The area AUC (Aera Under Curve) under the Receiver Operating Characteristic (ROC) curve is used to calculate the weight. The weight calculation process is as follows.

1) For the first level, the True Positive Rate (TPR) and False Positive Rate (FPR) are calculated based on the classification results of each classifier.

TPR represents the proportion of positive samples with correct prediction to the actual positive samples. FPR represents the proportion of positive samples with incorrect prediction to the actual negative samples. Where, tp and tn are the number of true positive and true negative, respectively. fp and fn are the number of false positive and false negative, respectively.

2) With TPR as the vertical axis and FPR as the horizontal axis, the ROC curve is drawn along the lines of the tracing points. The area under the ROC curve is calculated to obtain the AUC. Then the weight of each classifier is

where I represents the number of learners in each layer.

3) The class probability vector output by each classifier is multiplied with the corresponding $\beta_i$, and then concatenated with the original input as the input of the next level. The weights of other classifiers are calculated using the same method.

4) Finally, the class with the greatest probability is selected as the classification result.

The number of cascaded forest layers is adaptive by the algorithm. If adding a cascade layer does not improve the performance of the model, the training process is terminated.

3.   Experiment settings

3.1.   Configuration details

This experiment studies the subject-independent sentiment analysis. For partitioning the data set, 10-fold cross-validation is used to evaluate the performance. The metrics are as follows.

Parameters settings: PE takes two parameters: $m$ and $\tau$. If m is selected too large, the computational complexity will increase; If $m$ is too small, the permutation space will be small. In many applications, m = 3 to 7 are recommended. We set $m$ to be 4 and $\tau$ to be 1. VMD has two parameters: $K$ and $\alpha$. In order to avoid the arbitrariness of manual setting, Ensemble Empirical Mode Decomposition (EEMD) ^[31] is used for reference. EEMD is an adaptive signal decomposition method using a noise-assisted program to alleviate the mode mixing problem in EMD by repeatedly adding white Gaussian noise. K is determined by the model itself and does not need to be manually set. After the EEG signal is decomposed by EEMD, 12 IMFs can be obtained adaptively. The value of K for VMD is set to 12, the same as that for EEMD. The default value for $\alpha$ is 2000. For the weighted cascade classifier, the number of trees and forests on the arousal dimension is set to 250 and 5, respectively. There are three RF and two CRF. On the valence dimension, the number of trees and forests is 150 and 5, respectively. There are two RF and three CRF, respectively. The Gini coefficient is used to select the partition attributes. The minimum sample number of leaf nodes is set to 1.

3.2.   Data preprocessing

In this paper, two experiments are set to classify high/low arousal and high/low valence respectively. The original label is a score from 1 to 9. The score less than 5 is set as the low dimension, and the score greater than or equal to 5 is set as the high dimension. The class distribution is shown in Figure 5. First, the sample is downsampled to 128 HZ, and a bandpass filter from 4 HZ to 45 HZ is utilized to remove the ocular artifacts. Since the subjects did not watch the video for the first 3s, they could not show emotion. At the beginning and end of watching the video, subjects will have a process of entering the state and coming out of the state. Their emotions are mainly concentrated in the middle segment of the video. Therefore, the EEG signals in the first 13s and the last 10s are removed, and only the data in the middle 40s are retained for analysis. After data preprocessing, the dimension of EEG data of each subject is 40 × 14 × 5120 (video × channel × sample). Among them, 5120 is the number of sampling points within 40s at the sampling frequency of 128 HZ.

4.   Results and discussion

For the purpose of results presentation, the EEG of a single subject is illustrated. Figure 6 shows the EEG signal of AF3 channel of music video watched by a single subject. After VMD decomposition, a set of IMFs with vibration around different center frequencies are obtained, as shown in Figure 7. As can be seen from the figure, IMFs is arranged from low frequency to high frequency.

The high frequency components after VMD decomposition usually contain noise. In order to eliminate the interference of noise, pearson correlation coefficient method is used to calculate the correlation between each IMF and the original signal. The larger the absolute value of the correlation coefficient, the stronger the correlation. In this paper, the IMFs whose absolute value of correlation coefficient are greater than 0.35 is retained. Figure 8 shows the correlation coefficient between each IMF and the original signal. By comparison, IMF1–IMF5 is finally retained.

The features of all IMF1–IMF5 are extracted by sliding window. 16,800 dimensional features are obtained. In order to reduce the time cost, the CRRAEN is used for variable selection, and the 168-dimensional features obtained after variable selection are input into the WCF for classification. Figure 9 shows the confusion matrix obtained by the proposed method of valence and arousal emotion recognition. The element on the positive diagonal represents the accuracy of the correct classification of the sample. The confusion matrix shows that high arousal has higher recognition accuracy than low arousal. The recognition accuracy of high valence and low valence have similar recognition accuracy.

In terms of accuracy, precision and recall, the proposed method is compared with CF, WCF, RFVS-CF, mRMR-CF, EN-CF, AEN-CF and CRRAEN-CF methods. All methods are based on VMD decomposition and feature extraction. CF and WCF indicates that all features are used for classification. Other methods use different variable selection methods combined with CF. RFVS represents RF variable selection method.

Table 1 shows the results of the proposed method on the arousal dimension. For CF and WCF methods, the accuracy, precision and recall of WCF are 3.05, 2.55 and 1.83% higher than CF, respectively. It shows that the WCF is helpful to improve the accuracy. The accuracy of the proposed CRRAEN-WCF is 4.3, 8.44, 9.69, 10.79 and 12.43 higher than CRRAEN-CF, RFVS-CF, mRMR-CF, AEN-CF, EN-CF, respectively, and higher than that of CF and WCF. The precision of the proposed CRRAEN-WCF is 76.14% higher than other methods. The recall of the proposed method is 78.88% better than CRRAEN-CF, AEN-CF, EN-CF, mRMR-CF, RFVS-CF. The above results demonstrate the effectiveness of the proposed method.

Table 2 shows the results of the proposed method on the valence dimension. The accuracy of the proposed CRRAEN-WCF is 3.67, 4.22, 7.19, 6.25, 4.46, 4.06 and 6.17% higher than CRRAEN-CF, AEN-CF, EN-CF, mRMR-CF, RFVS-CF, WCF, CF, respectively. The precision of the proposed method is 83.82% higher than CRRAEN-CF, AEN-CF, EN-CF, mRMR-CF, RFVS-CF. The recall of CRRAEN-WCF is 82.50% better than other comparison methods.

The experimental results of arousal and valence show that the proposed CRRAEN feature selection method has more obvious advantages than RFVS, mRMR, EN and AEN. CRRAEN takes full account of the redundant information of variables, and combines the advantages of AEN, which excludes irrelevant variables. CRRAEN can significantly improve the accuracy of emotion recognition. Regardless of variable selection, WCF has a higher accuracy of emotion recognition than CF, which also confirms the effectiveness of weighting.

To evaluate the classification performance of the proposed approach from a statistical perspective, we perform Friedman test with 90% confidence on the performance of the different algorithms. It is commonly used in medical statistical analysis. Figure 10 is the Friedman test chart drawn according to the ranking results of the accuracy values of each algorithm. The vertical axis represents the algorithm, and the horizontal axis represents the average order value. If the horizontal lines of the two algorithms overlap, it means that there is no significant difference between the two algorithms; otherwise, there is a significant difference. It is obvious from Figure 10(b) that the horizontal line segments of the proposed CRRAEN-CF, AEN-CF, EN-CF, MRMR-CF, RFVS-CF, WCF and CF algorithms do not overlap, respectively. For valence, CRRAEN-WCF is significantly different from the comparison algorithm. In Figure 10(a), except for CRRAEN-CF, the horizontal segments of CRRAEN-WCF and AEN-CF, EN-CF, mRMR-CF, RFVS-CF, WCF and CF do not overlap, respectively. It demonstrates that CRRAEN-WCF is significantly different from other algorithms. However, there is overlap with the horizontal line segment of CRRAEN-CF, which is caused by the low accuracy of the proposed method in low arousal.

Table 3 lists the performance of emotion recognition methods using the same DEAP dataset in the existing literature. The items listed in the table are related to whether the methods in the existing literature use decomposition methods, which features are extracted, which feature selection method is used, which classifier is used and the number of classes. The accuracy of our proposed method is higher than that of some classical classification methods, such as SVM ^[3,32,36,38], RF ^[35] and KNN ^[37]. Among them, Atkinson and Campos ^[36] achieved relatively high performance by using MRMR reduction and GA-SVM classification, thanks to the optimization of SVM parameters by genetic algorithm. Compared with CWT-CRNN ^[11], VMD-DNN ^[12], DBN ^[33], PCA-SAE ^[34] and Transformer ^[15] in the deep model, the proposed method achieves higher accuracy. Among them, reference ^[11] combined CNN and LSTM. The accuracy of valence is 72.06%, and the accuracy of arousal is 74.12%, which is higher than other deep models. Literature ^[39] used MRMR variable selection combined with CF to classify five emotions, and the accuracy is 51.3%. The variable selection method proposed in this paper considers the common redundant information to construct mCRMR, and combines the embedded variable selection method AEN, which reduces the feature redundancy compared with MRMR. In addition, weight is given to the classifiers with good performance in the CF, which improves the role of strong classifiers compared with CF. The above results show that the proposed VMD-CRRAEN-WCF method has certain competitiveness.

Confirmation of hyper-parameters of WCF: We investigate the change of classification performance with the number of RF and decision tree. For the arousal and valence classification tasks, we use line graphs to visualize the trend of accuracy with the number of trees. When we study the changing trend of accuracy as the number of trees, the other parameters remain unchanged, the number of RF and CRF set to 2. The change interval of tree is 50. According to Figure 11, at the beginning, the accuracy improves gradually with the increase of the number of trees; when the tree reaches a certain amount, accuracy reaches maximum; when the number of trees continues to increase, the accuracy of the model fluctuates in a small range. The accuracy of WCF is highest when the number of trees reached 250 on arousal dimension, after which there is a slight fluctuation. When the number of trees in WCF is 150 on valence dimension, the accuracy is higher. From the perspective of model complexity, the number of trees is set as 150.

When we observed the effect of the number of forests on the accuracy, the other parameters are fixed. We use the confusion matrix to visualize, where the horizontal axis represents the number of CRF, the vertical axis represents the number of RF, and the label value represents the accuracy. It can be seen that with the increase of forest, the accuracy rate does not show a certain regularity. When 2 CRF and 3 RF in arousal, the accuracy value of arousal was the highest. When there were 3 CRF and 2 RF, the best accuracy rate was achieved on the valence dimension. In the arousal dimension, the optimal accuracy was obtained when two CRF and three RF were selected.

Running time:The configuration used in the experiment is Intel i7-10510U CPU (4 cores), and the memory is DDR4 2666 MHz 16G. Python3.9 version is used for training. The computing cost is shown in Table 4. The calculational cost of the algorithm without dimension reduction is relatively large, and the time cost of the algorithm after variable selection is significantly reduced. The proposed method in this paper combines two variable selection methods, and the classification time is higher than other single variable selection methods, but the accuracy is significantly improved.

5.   Conclusions

This paper mainly studies and analyzes the emotion recognition algorithm based on the EEG signal. In this paper, a new hybrid framework based on VMD-CRRAEN-WCF is proposed to improve the accuracy of EEG emotion recognition. Firstly, VMD is used to decompose the EEG signal and select the most relevant IMF with the original signal. CRRAEN is used to select features that are significantly correlated with the target variables. Finally, a weighted CF classifier is used for emotion recognition. Compared with EMD, VMD is a non-recursive signal analysis method, which can effectively resist noise interference and reduce the mode aliasing problem. CRRAEN is a new feature selection method, which combines the AEN and the minimum common redundancy maximum relevance criterion. CRRAEN fully considers the common redundant MI, and can effectively select related variables and reduce redundant. CF is an ensemble learning algorithm, which does not rely on large samples and precise tuning of hyper-parameters. The weighted CF mainly considers the different contributions of the forest learner, and has achieved satisfactory results on the public DEAP dataset.

Compared with some existing research methods, the proposed method has higher accuracy, which indicates the effectiveness of the proposed algorithm framework. The framework can also be applied to emotion recognition of other physiological signals.

Acknowledgments

This work was supported by the Science and Technology Department Project of Jilin Province (Grant No. 20210101149JC), the Science and Technology Department Project of Jilin Province (Grant No. 20200403182SF), and the Education Department Project of Jilin Province (Grant No. JJKH20210625KJ). The authors are very grateful for the supports.

Conflict of interest

The authors declare there is no conflict of interest.