Comparing Features from ECG

Pattern and HRV Analysis for

Emotion Recognition System

Hany Ferdinando, Tapio Seppnen, Esko

Alasaarela
Outline
Background
Materials and Methods
Results and Discussions
Conclusions
Acknowledgement
Background
The Mahnob-HCI database provides
database for research in affective
computing.
The baseline accuracy for valence and
arousal from ECG signal only using HRV
based features are 42.6% and 47.7%
respectively.
Are there any other features which can
increase these accuracies?
Materials and Methods
The Mahnob-HCI database
Recorded from 27 healthy subjects.
32 channels EEG signal
Peripheral physiological signals (ECG,
temperature, respiration, skin conductance).
Face and body videos
Synchronized measurement for multimodal
emotional response study.
Materials and Methods
Complex ECG Signals
Feature Extractions
Each ECG signal is divided into 5 second segment.
Complex ECG signal is made based on this
segment.
Bivariate empirical mode decomposition (BEMD)
is applied to the complex ECG signal, resulted 5-6
intrinsic mode functions (IMFs).
Only the first three of IMFs are used for further
processes.
Spectrogram analysis is used to get the dominant
frequencies from the IMFs.
Feature Extractions
Dominant frequency (DF) for each
segment is combined with the ones from
other segments to represent dominant
frequency over the entire ECG signal.
Features are statistical distribution of DFs
and their first difference.
Apply sequential forward floating search
algorithm to find the best features
combination.
Feature Extractions
Features are calculated based on the
combinations of window size and overlap
parameters in spectrogram analysis.
It is important to find the optimum
combination to get the highest accuracy.
As a comparison, features are also
calculated from the instantaneous
frequencies (IFs) of IMFs after applying
empirical mode decomposition (EMD) and
BEMD.
Validation
10-fold cross validation.
80% of the data is for training and testing
using 10-fold cross validation.
20% of the data is for validation.
Leave-one-subject-out (LOSO) validation.
It is similar to leave-one-out (LOO) but it is
applied to all data from one particular
subject.
The final accuracy is the average accuracy
from individuals.
 Results for 10-fold cross
validation
Valence Arousal

55.8 7.3 59.7 7.0

Window size = 150 samples Window size = 250 samples
Overlap
0 = 30%
1 2 0 1 2 Overlap = 30%
76. 13. 0 73.2 17.9 8.9
0 9.5
6 9 1 44.5 46.1 9.4
38. 44. 16. 56.
1 2 29.9 13.8
8 7 5 3
0:low, 37.
1:medium, 2: high
18. 43. 0:low, 1:medium, 2: high
2
9 5 6
Compare with the others
(10-fold cross validation)
DF of
BEMD
Feature
(propose IF of IF of
calculatio HRV
d EMD BEMD
n method
features
)
Classifier SVM KNN KNN KNN
Valence 42.6% 55.8% 45.3% 45.6%
Arousal 47.7% 59.7% 47.1% 46.6%
 Results for LOSO validation
Valence Arousal

59.2 11.4 58.7 9.1

Window size = 150 samples Window size = 250 samples
Overlap
0 = 30%
1 2 0 1 2 Overlap = 30%
81. 11. 0 75.9 18.3 5.8
0 7.4
6 0 1 48.6 43.6 7.8
35. 46. 18. 47.
1 2 40.4 11.9
6 2 2 7
0:low, 37.
1:medium, 2: high
19. 43. 0:low, 1:medium, 2: high
2
4 0 6
Compare with the others
(LOSO validation)
DF of
Feature
BEMD IF of IF of
calculation
(proposed EMD BEMD
method
features)
Classifier KNN KNN KNN
Valence 59.2% 45.4% 41.6%
Arousal 58.7% 47.0% 46.4%
Conclusions
The proposed features improve the
accuracy to classify emotion in valence
and arousal for 3-class problem: from
42.6% to 55.8% for valence and from
47.7% to 59.7% for arousal in 10-fold cross
validation (increased by 12%).
LOSO validation tests if the proposed
features are subject independent
Valence: 59.2 11.4
Arousal: 58.7 9.1
Conclusions
The proposed features outperformed the
other features not only for 10-fold cross
validation but also for LOSO validation.
Large overlap is not suitable for ECG
signal as the frequency contents in P, Q,
R, S, and T in the ECG signals have
different characteristic.
The proposed features may possess
potential for building emotion tracking
systems because it requires short-time
ECG signals.
Acknowledgement
This research is supported by the
Directorate General of Higher Education,
Ministry of Higher Education and
Research, Republic of Indonesia, No.
2142/E4.4/K/2013 and the
Optoelectronics and Measurement
Techniques unit, University of Oulu,
Finland.
The presentation is supported by the IEEE
Computational Intelligence Society (IEEE
CIS) through Travel Grant award.
Thank you