Sie sind auf Seite 1von 9

See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/224090298

Twin SVM for Gesture Classification Using the Surface Electromyogram

Article  in  IEEE Transactions on Information Technology in Biomedicine · April 2010


DOI: 10.1109/TITB.2009.2037752 · Source: IEEE Xplore

CITATIONS READS
89 137

3 authors:

Ganesh Naik Dinesh Kumar


Western Sydney University RMIT University
140 PUBLICATIONS   1,394 CITATIONS    352 PUBLICATIONS   2,769 CITATIONS   

SEE PROFILE SEE PROFILE

Jayadeva Dr
Indian Institute of Technology Delhi
102 PUBLICATIONS   1,607 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Our recent outcomes for PD diagnosis got significant media coverage View project

wound management View project

All content following this page was uploaded by Ganesh Naik on 27 May 2014.

The user has requested enhancement of the downloaded file.


IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE, VOL. 14, NO. 2, MARCH 2010 301

Twin SVM for Gesture Classification Using the


Surface Electromyogram
Ganesh R. Naik, Member, IEEE, Dinesh Kant Kumar, Senior Member, IEEE, and Jayadeva, Senior Member, IEEE

Abstract—Surface electromyogram (sEMG) is a measure of the the muscle activity from the surface. It is closely related to the
muscle activity from the skin surface, and is an excellent indica- strength of muscle contraction, and an obvious choice for control
tor of the strength of muscle contraction. It is an obvious choice of prostheses and other similar applications [3]–[6] because
for control of prostheses and identification of body gestures. Using
sEMG to identify posture and actions that are a result of overlap- mechanical sensors are unsuitable for people who have suffered
ping multiple active muscles is rendered difficult by interference amputations. Earlier attempts to use sEMG signals to identify
between different muscle activities. In the literature, attempts have hand and other body gestures were able to provide reliable
been made to apply independent component analysis to separate command signals only when one primary muscle was active
sEMG into components corresponding to the activities of differ- and the contraction level was high. When the muscle activity is
ent muscles, but this has not been very successful, because some
muscles are larger and more active than the others. We address small and multiple muscles are simultaneously active, such as
the problem of how to learn to separate each gesture or activity during maintained hand gestures, there is a relatively very high
from all others. Multicategory classification problems are usually level of noise, and this makes identification of gestures based
solved by solving many one-versus-rest binary classification tasks. on sEMG very unreliable.
These subtasks naturally involve unbalanced datasets. Therefore, In the recent past, there has been work done to classify sEMG
we require a learning methodology that can take into account un-
balanced datasets, as well as large variations in the distributions associated with relatively smaller and complex actions to iden-
of patterns corresponding to different classes. This paper reports tify hand or facial actions. Momen et al. [7] used strength of
the use of twin support vector machine for gesture classification the signal measured using rms to classify subject determined
based on sEMG, and shows that this technique is eminently suited wrist actions. This has number of applications, but is unsuitable
to such applications. for controlling prosthetic hand because the actions are user de-
Index Terms—Independent component analysis (ICA), learning, fined and not universal. Gabor transform and absolute mean [8]
multiclass, support vector machines (SVMs), surface electromyo- were used to identify ten wrist and finger motions in real-time
gram (sEMG), unbalanced data. learning. Canonical component analysis was used to obtain suit-
able features to identify four wrist motions and five finger mo-
I. INTRODUCTION
tions [9]. Yu et al. have used entropy of sEMG with statistical
ACTILE input devices, including keypads, keyboards, and
T mice, support only very low levels of information commu-
nication. We must consciously translate our intent into an exact
classifiers to identify five finger flexions [10]. Tenore et al. [11]
have used a large array of electrodes to identify hand actions.
However, both these techniques are limited because large array
sequence of binary key presses before a device can use this of electrodes makes the system unsuitable for people who had
information. There is a need for devices to understand more suffered transradial amputation.
natural forms of human communication such as hand gestures. In the recent past, support vector machines (SVMs) have
A hand gesture mode of communication and controlling offers been used to improve the accuracy of classification. Bitzer and
privacy and naturalness [1], [2]. Such systems are required to Smagt [12] have used a large set of electrodes to identify finger
help people with special needs such as with hand amputation actions using SVM. The system described in this paper requires
and defence personnel. On a broader level, all people who use an array of electrodes placed very close to the wrist to identify
devices such as computer mouse would benefit from such de- the actions, making it unsuitable for most transradial amputees.
vices, as it will allow them to have a natural control. This will Lucasa et al. [13] have used a combination of wavelets and svm,
increase productivity and reduce computer-related injuries such but it is not clear how the experiments were conducted. Jorgesen
as repetitive strain injury (RSI). and van der Binsted [14] have used SVM to classify subvocal
Hand actions and maintained gestures are a result of complex sEMG to identify unspoken speech, which has similar problems
combinations of contractions of multiple muscles in the forearm. as wrist sEMG. However, their confusion matrix suggests a
Surface electromyogram (sEMG) is the electrical recording of very low accuracy system, which may be suitable for subvocal
applications, but would not be suitable for prosthetic control.
Manuscript received November 21, 2008; revised July 18, 2009 and The authors are unaware of any systems that use upper forearm
November 1, 2009. First published December 15, 2009; current version pub-
lished March 17, 2010.
recordings of EMG to identify finger and wrist actions.
G. R. Naik and D. K. Kumar are with the Department of Electrical and Com- To reliably identify small movements and gestures of the
puter Engineering, Royal Melbourne Institute of Technology, Melbourne 3001, hand, there is a need to decompose the sEMG into components
Australia (e-mail: ganesh.naik@rmit.edu.au; dinesh@rmit.edu.au).
Jayadeva is with the Department of Electrical Engineering, Indian Institute
corresponding to activity originating from different muscles.
of Technology, New Delhi 110016, India (e-mail: jayadeva@ee.iitd.ac.in). Spectral and temporal filtering is not suitable for this because of
Digital Object Identifier 10.1109/TITB.2009.2037752 overlapping spectra and simultaneously active muscles. Blind

1089-7771/$26.00 © 2009 IEEE


302 IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE, VOL. 14, NO. 2, MARCH 2010

source separation (BSS) techniques have been developed re- constraints is M K 2 , where K is the number of classes and M
cently, and they provide a solution for such a situation. Inde- is the number of patterns, making the system very complex for
pendent component analysis (ICA) is a statistical technique that larger problems.
is suitable for estimating independent components (ICs) from Classification of the sEMG for gesture recognition thus
a mixture. It has been used very successfully for audio and presents all of these challenges. This paper reports a novel tech-
biosignal applications [15], [16]. One of the shortcomings of nique that uses Twin SVM (Twin SVM) [18] to classify the
ICA is that the estimated outputs are scaled and the order is not ICA-separated sEMG to accurately identify hand gestures from
known. While this is generally not an issue for audio signals, it is sEMG recorded from upper regions of the forearm, making it
very important for estimating hand gestures, where the relative suitable for prosthetic applications. The Twin SVM solves two
strength of contractions and location of the different muscles related SVMs, one for each class, and generates two separate
is important [15], [17]. One of the difficulties with classifying hyperplanes. In each binary classification task, the hyperplane
decomposed sEMG using traditional approaches is that some for class 1 is close to patterns of class 1, with the constraints aris-
weak muscle activities are associated with a number of actions. ing from patterns only of class (–1). The converse is true for the
Since each gesture involves a different pattern of activation of other SVM. Each SVM thus has half the constraints of a regular
a number of muscles, the sEMG signals corresponding to each SVM. Solving two independent problems allows us to over-
gesture can be understood to arise from a different distribution. come the skew in class cardinalities, and also provides speedup,
Therefore, we require a learning methodology that can take into since the individual SVMs are smaller. The results demonstrate
account large variations in the distributions of patterns corre- a striking improvement in classification results. Twin SVM also
sponding to different classes or categories. does not assume that patterns in each class arise from similar
Multiclass classification problems are usually solved by solv- distributions. It allows the use of a different kernel for each
ing many one-versus-rest binary classification tasks. These class, which can be separately optimized based on the data. The
subtasks naturally involve unbalanced datasets. In the pres- facility for data dependent kernel optimization for each class
ence of unbalanced datasets, conventional error-minimization- is particularly valuable in our application, and the results show
based learning, as commonly employed in neural networks and that significant improvements can be obtained by exploiting this
SVsMs, tends to favor the larger set, since we minimize the sum feature.
of errors across patterns. Although smaller class tests yield poor The remainder of this paper is organized as follows.
performance, the accuracy is good enough for practical use. Section II describes the Twin SVM and section III describes the
Consider a multiclass problem with M samples and K classes, basis of ICA related to this study. Section IV is the methodology
and (M /K) patterns in each class. We usually build one binary section. Section V contains experimental results and observa-
classifier for each class, which is learnt by using the one-versus- tions. Section VI contains concluding remarks.
rest approach. Consider how the classifier for, say, class 1 is
built. We construct a binary or two-class classifier that learns to II. TWIN SVMS
distinguish patterns of class 1 from other class patterns. Sam-
ples for class 1 comprise (M /K) patterns and samples for class A. Summary of the Twin SVM Formulation
(–1) are obtained by merging patterns from the remaining (K– Let the patterns to be classified be denoted by a set of m
1) classes, i.e., M – (M /K) patterns. It is clear that the resultant row vectors Ai ∈ Rn , where Ai = (Ai1 , Ai1 , . . . ,Ain )T . Let
binary classification problem is highly unbalanced, with a small y ∈ {−1, 1} denote the class to which the ith pattern belongs.
fraction of patterns in class 1 and a significant number of pat- The classical L1 norm soft margin SVM classifier [18] finds a
terns in the other. Therefore, multiclass problems naturally lead hyperplane wT x + b = 0 that separates patterns from the two
to highly unbalanced binary classification tasks. Most learning classes. The parameters of the hyperplane are determined by
algorithms minimize a sum of misclassification errors across all solving the optimization problem
patterns. Consequently, the error on M – (M /K) patterns will 1 m
dominate on the relatively fewer (M /K) patterns of class 1. Minw ,b w2 + C qi (1)
2 i=1
For instance, with M = 1000 and K = 10, we can conceiv-
ably obtain a classifier that is 95% accurate (it makes no error subject to the constraints
on the 900 patterns in one binary class), but misclassifies 50% yi (Ai w + b) + qi ≥ 1, i = 1, 2, . . . ,m (2)
of the patterns belonging to category 1. In practice, the classi-
fier is useless because it is as bad as a coin toss in recognizing where qi is an error variable associated with the ith pattern and
category 1 patterns. This problem has been highlighted in the lit- C is a hyperparameter that determines the relative importance
erature, where researchers have observed that classes might not of the misclassification error in comparison to the regularization
be recognized at all, particularly if they have a small number term in the objective function.
of patterns [18]–[20]. Recognizing gestures from sEMG data If the patterns are not linearly separable, then the data are
thus demands an appropriate classification methodology that is implicitly transformed to a higher dimensional space through a
suitable for handling unbalanced datasets. map φ(x), and the corresponding quadratic optimization prob-
There are a few other notable approaches in the literature that lem is given by
address multiclass SVMs. Notable among these is the M-SVM 1 m
[21] that solves a single optimization problem. The number of Minw ,b w2 + C qi (3)
2 i=1
NAIK et al.: TWIN SVM FOR GESTURE CLASSIFICATION USING THE SURFACE ELECTROMYOGRAM 303

subject to the constraints B points from the hyperplane (w(2) )T x + b2 = 0.. The second
  term is the misclassification error. The scalar C2 indicates the
yi wT φ (Ai ) + b + qi ≥ 1, i = 1, 2, . . . , m. (4)
relative importance of the misclassification error in the objective
The SVM classifier is usually obtained by solving the dual of function. The decision about the membership of a point x is
(3) and (4), which is given by made by determining which of the two hyperplanes it is closest
1 m m m to. Note that the number of constraints in each of the two SVMs
Min λi λj yi yj Kij − λi (5) is smaller than in the original SVM. Therefore, the Twin SVM
2 i=1 j =1 i=1
formulation enhances the speed of operation [18].
subject to the constraints
The extension of the Twin SVM to nonlinear kernels is
0 ≤ λi ≤ C (6) straightforward. The kernel-generated surfaces (in place of the
linear hyperplanes) are given by K(A, C T )u(1) + b1 = 0 and
where λi ’s are Lagrange parameters and Kij is the (i, j)th entry
K(A, C T )u(2) + b2 = 0. The parameters of the two hypersur-
of the Kernel matrix; the entries of the Kernel matrix are values
faces are obtained by solving the following pair of optimization
of the Kernel function evaluated at pairs of training samples,
problems:
which are given by
1   2 m 2
Kij = K (Ai , Aj ) = φ (Ai )T φ (Aj ) . (7) Minu ( 1 ) ,b 1 K A, C T u(1) + e1 b1  + C1 q1i
2 i=1
The primal objective function in (1) or (3) sums up classification (12)
errors over all patterns. It does not distinguish between the num- subject to the constraints
bers of training patterns in either class. As a consequence, if the
   
two classes are highly unbalanced, the error term is dominated − K B, C T u(1) + b1 + q1i ≥ 1, i = 1, 2, . . . , m2 .
by the larger class; the solution attempts to minimize any clas- (13)
sification errors over the larger class. As illustrated in Section I, The second problem tries to similarly find a hyperplane that
this may lead to classifiers that cannot discriminate a specified is close to patterns of class (–1) and at least a distance of 1
category in a multiclass scenario, even though overall accuracy away from the points of class 1. This leads to the following
figures may be high. optimization task:
To explain the Twin SVM better, we assume that the patterns
of classes (+1) and (–1) are represented by matrices A and 1   2 m 1
Minu ( 1 ) ,b 1 K B, C T u(2) + e2 b2  + C2 q2i
B, respectively. The sizes of A and B are thus (m1 × n) and 2 i=1
(m2 × n), respectively. The Twin SVM with a linear Kernel (14)
finds two hyperplanes (w(1) )T x + b1 = 0 and (w(2) )T x + b2 = subject to the constraints
0 by solving a pair of optimization problems. The first problem    
tries to find a hyperplane that is close to patterns of class 1 and K A, C T u(2) + b2 + q2i ≥ 1, i = 1, 2, . . . , m2
at least a distance of 1 away from the points of class (–1) (15)
m 2 where C T = [AB]T and K is the kernel matrix. Note that C1
1 2
Minw ( 1 ) ,b 1 Aw(1) + e1 b1  + C1 q1i (8) and C2 are two constants analogous to the constant C in the
2 i=1
case of the SVM.
subject to the constraints In multiclass problems, patterns in each of the categories can-
  not be assumed to come from similar distributions. The note-
− Bi w(l) + b1 + q1i ≥ 1, i = 1, 2, . . . , m2 . (9)
worthy aspect of Twin SVMs is that we can optimize the kernel
where e1 is a vector of m1 1s. The first term in the objective for each of the Twin problems separately [22]. In the sequel,
function is the sum of squared distances of points belonging to we show that significant improvements in prediction accuracies
class A from the hyperplane (w(1) )T x + b1 = 0. The second can be obtained by applying kernel optimization to Twin SVMs
term is the misclassification error. The scalar C1 indicates the on the sEMG data.
relative importance of the misclassification error in the objective
function.
The second problem tries to similarly find a hyperplane that III. INDEPENDENT COMPONENT ANALYSIS
is close to patterns of class (–1) and at least a distance of 1 A. ICA Formulation
away from the points of class 1. This leads to the following
optimization task: ICA is a BSS technique for estimating the s independent
sources from x recordings, where the x recordings are the linear
1 2 m 1
Minw ( 2 ) ,b 2 Bw(2) + e2 b2  + C2 q2i (10) mixtures of s independent sources, and when neither the orig-
2 i=1 inal sources nor the actual mixing matrix A is known. This is
subject to the constraints achieved by exploiting higher order signal statistics and opti-
  mization techniques [15], [16].
Ai w(2) + b2 + q2i ≥ 1, i = 1, 2, . . . , m1 . (11)
The aim of BSS techniques is to recover unobserved signals
where e2 is a vector of m2 1s. The first term in the objective or sources from a mixture when the observer is blind to the
function (equation 10) is the sum of squared distances of class signal properties and the mixing matrix. Given a set of
304 IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE, VOL. 14, NO. 2, MARCH 2010

observations recorded by the sensors x (t) = The successful separation of the original signals is dependent
[x1 (t) , . . . , xn (t)], the mathematical formulation of BSS on the fulfillment of the following conditions.
is typically given in the form of a statistical estimation problem. 1) The source signals s(t) are independent
This model is generative, and it estimates how the observed data 2) The source signals must have non-Gaussian distributions.
are generated by a process of mixing the source components. It 3) The number of sensors (recordings) must be greater than
is assumed that s (t) = [s1 (t) , . . . , sn (t)]T are the unknown or equal to the number of sources N ≥ M.
source signals, which have undergone mixing according to a 4) The recorded signals must be (approximately) linear com-
vector function f . binations of the independent sources.
For linear mixing models, ICA is a valuable tool for BSS, and 5) There should be no (little) noise common to the sources.
the mathematical formulation of the classical ICA is a simplified While ICA has demonstrated success in separating signals,
form of the BSS problem the output of ICA suffers from the following ambiguities.
1) The order of the ICs cannot be fixed and this may change
x (t) = As (t) (16)
for each estimate.
where A is an N × M scalar matrix representing the unknown 2) The amplitude and sign of the ICs cannot be determined.
mixing coefficients and it is called transfer or mixing matrix. In most applications such as the cocktail party problem, these
For most ICA applications, the noise is either assumed to be are not serious problems. The supervisor is able to identify the
white Gaussian with variance σ 2 or negligible. Apart from the different sources and determine the quality of the separation
source signals, the noise can also be assumed to be part of the based on the audio information by listening to the sounds [15],
sources. In this case, the noise is assumed to be statistically [24]. When dealing with signals such as sEMG, the order of
independent of other source components. The goal of ICA is to signals and the relative value of the signal magnitude are very
find a linear transformation W of the dependent sensor signals important for estimating the action correctly.
x(t) that makes the outputs as independent as possible:
ŝ (t) = W x (t) = W As (t) (17) IV. METHODOLOGY

where ŝ (t) is an estimate of the sources. The sources are exactly Experiments were conducted to evaluate the performance of
recovered when W is the inverse of A up to a permutation and the proposed system to identify finger and wrist flexion from
scale change. Since both the sources and the mixing coefficients sEMG recorded on the forearm. The Royal Melbourne Institute
are unknown, it is impossible either to determine the variances of Technology (RMIT University) Human Experiments Ethics
or the order of the ICs. Numerous ICA algorithms are used for Committee approved the experimental protocol.
signal processing research. For isometric hand gesture identifi-
cation, we use FastICA algorithm [15], [23]. FastICA technique A. Subjects
and the algorithm are briefly explained in the next section. Seven healthy subjects (six males and one female) volun-
teered to participate in this study. Mean age was 25.2 (σ = 2.05)
B. FastICA Algorithm
years; mean weight 70.2 (σ = 6.56) kg; and mean height was
FastICA is a fixed point algorithm that employs higher order 170.2 (σ = 7.42) cm. The participants inclusion criteria were: 1)
statistics for the recovery of independent sources. Separation is no history of myo or neuropathology and 2) no evident abnormal
performed by minimization of negentropy of the mixture such motion restriction. Only right-hand dominant participants were
that uncorrelated and independent sources whose amplitude dis- included in this study. Each participant was given an oral and
tributions as non-Gaussian as possible are obtained. The non- written summary of the experimental protocol and the purpose
Gaussianity is measured with the differential entropy J, called of the study, and then was required to sign a consent form prior
negentropy, which is defined as the difference between the en- to participation.
tropy of a Gaussian random variable yGauss (having the same
mean and variance of the observed random variable y) and the B. Data Acquisition
entropy y
The sEMG was recorded using DELSYS (Boston, MA) pro-
J (y) = H (yGauss ) − H (y) (18) prietary sEMG acquisition system. Each channel consists of a
where the entropy H is given by pair of differential electrodes with a fixed interelectrode dis-
 tance of 10 mm and a gain of 1000 (see Fig. 1). The system has
H (y) = − f (y) log (f (y)) dy. (19) a fixed bandpass filter range of 20–450 Hz and a sampling rate
of 1024 samples/s. The frequency range ensures that artefacts
Since Gaussian random variables have the largest entropy H due to muscle and electrode movement are removed.
among all random variables having equal variance, maximizing Prior to placing the electrode, the skin of the participant was
the negantropy, J (y) leads to the separation of independent prepared by shaving (if required) and exfoliation to remove dead
source signals [23]. skin. The skin was cleaned with a 70% v/v alcohol swab to re-
FastICA can estimate ICs one by one (deflation approach) or move any oil or dust from the skin surface. The skin impedance
simultaneously (symmetric approach). FastICA uses simple es- between the electrodes was measured, and in all cases, it was
timates of negentropy based on the maximum entropy principle. less than 60 kΩ. Standard isometric manual muscle testing was
NAIK et al.: TWIN SVM FOR GESTURE CLASSIFICATION USING THE SURFACE ELECTROMYOGRAM 305

comfortable to the user. This is also important for amputees who


do not get any feedback because the muscles do not move any
finger.

D. Data Analysis
The data analysis was performed in MATLAB on a Windows
platform. As a first step, the sEMG was segmented to remove
the start and end of each recording corresponding to the mark-
ers. This was done to eliminate transient conditions. FastICA
was then used to separate the four channels of sEMG using
4 × 4 matrix structures for the first day experiments. The es-
timated unmixing matrix W was saved and given an identifier
corresponding to the participant. The rms was computed for
a 2500-samples window size of the four estimated separated
signals to obtain one number corresponding to each source of
Fig. 1. Hand gesture experimental setup with four electrodes.
sEMG for each action. Calculating the rms is computationally
TABLE I
efficient, a good estimate of the strength of contraction of the
MUSCLE ANATOMY FOR THE PLACEMENT OF ELECTRODES OVER THE FOREARM muscle, and suitable for real-time applications. It also allows a
direct comparison with most other techniques reported in litera-
ture for similar applications. The aforementioned was repeated
for each of the seven participants. There were 84 datasets, cor-
responding to seven classes, for each of our subjects.
This paper proposes the use of Twin SVM for identifying
the finger actions. Most approaches in the literature deal with
a neural network. The intention was to present a comparison
with the baseline. We have proposed that standard SVMs have
clear drawbacks with skewed or unbalanced datasets. On the
performed to verify electrode placement. Four muscle groups in other hand, Twin SVMs solve two smaller problems instead of a
forearm were chosen for this study. These muscles are responsi- large one. They can be applied to a multiclass problem by using
ble for wrist and finger flexion movements. The locations of the the one-versus-rest approach, in which a separate classifier is
electrodes and associated actions have been tabulated in Table I. built for each category. Neural networks can be directly applied
to multicategory discrimination by using multiple neurons in
the output layer. Since the hidden neurons are common, multi-
layer neural networks could possibly exploit correlations among
C. Experimental Protocol
classes to yield superior accuracies. However, our results show
Subjects were asked to keep the forearm resting on the ta- that Twin SVMs can significantly outperform them.
ble with the elbow at an angle of 90◦ (approximately) and in a The inputs to the classifiers were the four rms values. In a
comfortable position. Seven subtle (low-level contraction) ac- later experiment, the kernels of the Twin SVM were optimized
tions involving finger and wrist flexion followed by relaxation for this dataset. The design of the neural network was obtained
were performed and repeated 12 times. Markers were manually heuristically such that it gave the minimum classification error.
inserted using a pushbutton connected to the marker channel. The network that was used had four input nodes, a total of 20
These identified the start and completion of each action. The ac- hidden nodes spread over two hidden layers and four output
tions were chosen such that these corresponded to simultaneous nodes, and had a sigmoid threshold function. The network was
contraction by multiple muscles. This was done to test the ability trained using the error backpropagation algorithm. The neural
of the system to decompose the measured signal into compo- network and Twin SVM were trained for each subject separately.
nents corresponding to simultaneously active multiple muscles. Each of these was trained using 35 randomly selected datasets,
The seven actions are as follows: and was tested using the remaining 49 sets of data.
1) wrist flexion; The performance of the neural network and Twin SVM classi-
2) finger flexion—ring finger and the middle finger together; fiers were evaluated using the standard tenfold cross-validation
3) wrist flexion toward little finger; methodology. The total number of data samples for all the seven
4) wrist flexion toward thumb; subjects, each performing seven gestures, repeated 12 times, and
5) fingers and wrist flexion together; for two days is 1176 (7 × 7 × 12 × 2). This set was partitioned
6) fingers and wrist flexion toward little finger; into ten folds, each with approximately 117 data points. One
7) fingers and wrist flexion toward thumb. fold was left out and the balance nine folds out of ten were used
The subjects were allowed to conduct the actions at their own to train the system. The fold that was left out was used to test
preferred speed. This made the actions more reproducible and the trained system. This was repeated ten times such that all ten
306 IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE, VOL. 14, NO. 2, MARCH 2010

TABLE II TABLE IV
OVERALL AVERAGE RESULTS FOR HAND GESTURES USING ICA WITH NEURAL GESTURE CLASSIFICATION WITH THE TWIN SVM (IN PERCENT)
NETWORK (IN PERCENT)

TABLE III
OVERALL AVERAGE RESULTS FOR SEVEN HAND GESTURES USING ICA WITH
NEURAL NEURAL NETWORK (IN PERCENT)

TABLE V
GESTURE CLASSIFICATION WITH THE TWIN SVM USING AN OPTIMIZED
KERNEL CONSISTING OF A SUM OF RBF KERNELS

folds were tested against the other nine. The results from the ten
tests were averaged.
To test the reliability of the techniques, sensitivity and speci-
ficity of Twin SVM and neural networks were calculated using
the following formula:
Number of recognized true gestures
Sensitivity = (20)
Number of true gestures
Number of rejected false gestures
Specificity = . (21)
Number of false gestures
same kernel is not the best choice for different categories; the
V. EXPERIMENTAL RESULTS
linear kernel is better for some, while the polynomial kernel is
Table II shows the results obtained by running the neural better for others.
network on the test data. The results indicate that the perfor- The earlier observation motivated us to examine kernel op-
mance rapidly degrades with an increase in the number of ges- timization in the Twin SVM. The kernels for each of the Twin
tures, i.e., categories; for three classes, the overall efficiency problems were optimized and the classifier built by using an
was 99.34%, whereas for seven classes, it decreased to 59.34%. alternating optimization scheme, as outlined in [22]. Tables V
The detailed results for the seven gestures’ case are presented and VI summarize the results. In Table V, the optimal kernel
in Table III. Note that on one category, the classification ac- consists of a weighted sum of three Gaussians, with σ = 0.1,
curacy is worse than a coin toss. This reinforces our earlier 0.01, and 0.001, respectively. The kernel weights indicate the
viewpoint—conventional learning algorithms tend to work well weights of the three Gaussian kernels in the optimal mix. Note
for binary classification tasks, but are not as effective in a multi- that the weights µ(1) that correspond to the class 1 kernel are
class scenario, since minimizing the sum of classification errors different from µ(2) , the weights for the optimal kernel for the
favors fewer errors on the larger class. This is not effective in a other class.
multiclass scenario. In Table VI, the optimal kernel consists of a weighted sum of
The tenfold cross-validation prediction results on test data for a linear kernel, a degree two polynomial kernel, and a Gaussian
the Twin SVM are presented in Table IV. The table shows results kernel with σ = 0.5. The kernel weights indicate the weights of
for the Twin SVM with linear as well as polynomial kernels. the individual primary kernels in the mixture corresponding to
The results show significant performance improvements over the the optimal kernel.
neural network. On an average, the gesture recognition accuracy The confusion matrix has been provided in Table VII. This
varies from 79.07% to 85.07%. The results indicate that the table is a measure of the accuracy and specificity of the system.
choice of kernel can have a significant bearing on the prediction The overall performance of this classifier was 86.0%. The recog-
accuracy. Moreover, the best kernel is category dependent—the nition accuracy ranged from 84.8% to 87.3%, and the order of
NAIK et al.: TWIN SVM FOR GESTURE CLASSIFICATION USING THE SURFACE ELECTROMYOGRAM 307

TABLE VI problems need to take into account differences in the pattern


GESTURE CLASSIFICATION WITH THE TWIN SVM USING AN OPTIMIZED
KERNEL CONSISTING OF A SUM OF THREE DIFFERENT KERNELS
distributions across categories.
These results indicate that appropriately optimized Twin
SVM can accurately identify hand gestures and actions from
sEMG that has been separated using ICA. Such a system over-
comes the weakness of previous sEMG-based hand gesture iden-
tification systems, where there are issues related to the reliability.
Such a system could find applications for giving the user with
a natural control, and may be used for controlling devices such
as prosthetic hand or the computer cursor.

VI. CONCLUSION
Learning from multiclass data, where patterns in different
classes arise from different distributions, requires taking these
differences into account. Multiclass data inherently lead to un-
balanced binary classification tasks. Biosignal analysis and in-
TABLE VII
terpretation abounds with applications that exhibit both of the
CONFUSION MATRIX FOR SEVEN GESTURES aforementioned characteristics. In this paper, we addressed the
task of gesture recognition using sEMG data and observed sim-
ilar challenges. The Twin SVM was shown to be an effective
approach in meeting these challenges, since it is a more natural
choice for applying to unbalanced datasets.
In the literature, classifiers used for similar applications in-
clude neural networks [25], [26] and SVMs [7]. More recently,
the multiclass problem of identifying complex actions has been
addressed using statistical classifiers [10] and one-versus-rest
TABLE VIII
style SVM [12]. However, all these systems are based on the
SENSITIVITY AND SPECIFICITY FOR G1 USING TWIN SVM AND NEURAL implicit assumption that the dataset is balanced, which is not an
NETWORK accurate assumption. Thus, even though high accuracy results
may be obtained, the outcomes of such an approach are not
consistent.
A notable multiclass approach is the M-SVM [21] that solves
a single optimization problem. However, the M-SVM requires
solving a problem with M K 2 constraints, where K is the num-
ber of classes and M is the number of patterns. This is difficult
the recognition was: G7 (87.3%), G6 (86.9%), G2 (86.3%), G3 for larger problems.
(85.9%), G4 (85.6%), G5 (85.4%), and G1 (84.8%). G1 (wrist This paper overcomes the hurdles posed by unbalanced
flexion versus the rest) was misclassified highly with G4 (4.2%) datasets by employing the Twin SVM. The use of ICA leads
and partly with G5 (1.2%), while G2 was misclassified highly to a smaller dataset where the user does not have to conduct
with G4 and G6 (2.9%), and partly with G5 (1.2%). Similarly, numerous repetition of the experiment. The system is able to
G3 was misclassified highly with G4 (3.1%) and partly with G2 accurately identify actions by using sEMGs recorded from near
(1.2%). From this list, it can be observed that there is little dif- the upper region of the elbow, making it suitable for applications
ference between the misclassification of the different gestures. such as transradial amputees.
The sensitivity and specificity of the classifiers (Twin SVM The facility for separately optimizing the kernel for each class
and neural network) results for G1 are presented in Table VIII. in a binary classification problem turns out to be very valuable
From these results, it is observed that for G1, the Twin SVM has when learning multiclass problems with disparate categories.
sensitivity and specificity of 84.8% and 88.1%, respectively, and The proposed approach lends itself to the development of a va-
neural network sensitivity and specificity for the same gesture riety of people friendly interfaces that use biosignals such as
are 58.3% and 42.5%, respectively. This indicates that Twin prosthetic hand or even to control the computer cursor. Experi-
SVM is more reliable compared with neural networks. mental results clearly demonstrate the efficacy of the proposed
The results indicate that choosing an appropriate data- approach. This technique can be extended for other biosignals
dependent kernel can significantly improve prediction accu- and other applications using similar methodology.
racies. The optimal weights show that for class (–1), which
comprises many categories merged together, multiple Gaussians REFERENCES
need to be combined since the data are a mix of several distribu- [1] B. W. Miners, O. A. Basir, and M. S. Kamel, “Understanding hand gestures
tions. The kernel optimization is different across different cat- using approximate graph matching,” IEEE Trans. Syst., Man, Cybern. A,
egories; this also reinforces our observation that multicategory Syst. Humans, vol. 35, no. 2, pp. 239–248, Mar. 2005.
308 IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE, VOL. 14, NO. 2, MARCH 2010

[2] S. Mitra and T. Acharya, “Gesture recognition: A survey,” IEEE Trans. Ganesh R. Naik (S’05–M’09) received the B.E. de-
Syst., Man, Cybern. C, Appl. Rev., vol. 37, no. 3, pp. 311–324, May 2007. gree in electronics and communication engineering
[3] P. C Doerschuk, D. E Gustafson, and A. S Willsky, “Upper extremity from the University of Mysore, Mysore, India, in
limb function discrimination using EMG signal analysis,” IEEE Trans. 1997, the M.E. degree in communication and infor-
Biomed. Eng., vol. BME-30, no. 1, pp. 18–28, Jan. 1983. mation engineering from Griffith University, Bris-
[4] M. Z. Kermani, B. C. Wheeler, K. Badie, and R. M. Hashemi, “EMG bane, Australia, in 2002, and the Ph.D. degree in the
feature evaluation for movement control of upper extremity prostheses,” area of digital signal processing from Royal Mel-
IEEE Trans. Rehabil. Eng., vol. 2, no. 4, pp. 1267–1271, Dec. 1995. bourne Institute of Technology (RMIT University),
[5] A. D. C. Chan, K. B. Englehart, and C. C. Jorgensen, “Continuous my- Melbourne, Australia, in 2009.
oelectric control for powered prostheses using hidden Markov models,” He is currently a Postdoctoral Fellow in the De-
IEEE Trans. Biomed. Eng., vol. 52, no. 1, pp. 121–124, Jan. 2005. partment of Electrical and Computer Engineering,
[6] A. B. Ajjiboye and R. F. Weir, “A heuristic fuzzy logic approach to EMG RMIT University. His research interests include pattern recognition, blind source
pattern recognition for multifunctional prosthesis control,” IEEE Trans. separation techniques, audio signal processing and biosignal processing, and
Neural Syst. Rehabil. Eng., vol. 13, no. 3, pp. 280–291, Sep. 2005. human–computer interface.
[7] K. Momen, S. Krishnan, and T. Chau, “Real-time classification of fore- Dr. Naik was the Chair for the IEEE Computer Society CIT08 Conference,
arm electromyographic signals corresponding to user-selected intentional Sydney. He was a recipient of the Baden–Württemberg Scholarship from the
movements for multifunction prosthesis control,” IEEE Trans. Neural University of Berufsakademie, Stuttgart, Germany (2006–2007).
Syst. Rehabil. Eng., vol. 15, no. 4, pp. 535–542, Dec. 2007.
[8] D. Nishikawa, W. Yu, H. Yokoi, and Y. Kakazu, “EMG prosthetic hand
controller discriminating ten motions using real-time learning method,”
in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Kyongju, Korea, Oct.
1999, pp. 1592–1597.
[9] K. Nagata, K. Ando, K. Magatani, and M. Yamada, “Development of the
hand motion recognition system based on surface EMG using suitable
measurement channels for pattern recognition,” in Proc. 29th Annu. Int. Dinesh Kant Kumar (M’95) received the B.E. de-
IEEE EMBS Conf., Lyon, France, Aug. 2007, pp. 5214–5217. gree in electrical engineering from the Indian Institute
[10] K. J. Yu, K. M. Cha, and H. C. Shin, “Maximum likelihood method for fin- of Technology (IIT), Madras, India, in 1982, and the
ger motion recognition from sEMG signals,” in Proc. ICBME, Singapore, Ph.D. degree from IIT, Delhi, India, in 1990.
2008, pp. 452–455. He was with the engineering industry for more
[11] F. Tenore, A. Ramos, A. Fahmy, S. Acharya, R. E. Cummings, and N. than ten years in various capacities. Since 1996, he
V. Thakor, “Towards the control of individual fingers of a prosthetic hand has been an Academic in the Department of Electrical
using surface EMG signals,” in Proc. 29th Annu. Int. Conf. IEEE EMBS, and Computer Engineering, Royal Melbourne Insti-
Aug. 2007, pp. 6145–6148. tute of Technology (RMIT University), Melbourne,
[12] S Bitzer and P. van der Smagt, “Learning EMG control of a robotic hand: Australia. His research interests include iterative sig-
Towards active prostheses,” in Proc. IEEE Int. Conf. Robot. Autom., May nal processing, computer vision, and intelligent sys-
2006, pp. 2819–2823. tems for application such as biometrics, human–computer interface, and helping
[13] M. F. Lucasa, A. G. S. Pascuala, C. Doncarlia, and D. Farinab, “Multi- the disabled.
channel surface EMG classification using support vector machines and
signal-based wavelet optimization,” Biomed. Signal Process Control,
vol. 3, no. 2, pp. 169–174, Apr. 2008.
[14] C. Jorgensen and K. Binsted, “Web browser control using EMG based sub
vocal speech recognition,” in Proc. 38th Annu. Hawaii Int. Conf. Syst. Sci.
(HICSS2005), vol. 9, p. 294 c.
[15] A. Hyvarinen, J. Karhunen, and E. Oja, Independent Component Analysis.
New York: Wiley, 2001. Jayadeva (S’87–M’88–SM’05) received the B.Tech.
[16] A. Bell and T. Sejnowski, “An information maximisation approach to blind and Ph.D. degrees from the Department of Electri-
separation and blind deconvolution,” Neural Comput., vol. 7, pp. 1129– cal Engineering, Indian Institute of Technology (IIT),
1159, 1995. Delhi, India, in 1988 and 1993, respectively.
[17] A. Cichocki and S. Amari, Adaptive Blind Signal and Image Processing: He is currently a Professor in the Department of
Learning Algorithms and Applications. New York: Wiley, 2003. Electrical Engineering, IIT, Delhi. He is the holder of
[18] Jayadeva, R. Khemchandani, and S. Chandra, “Twin support vector ma- one U.S. patent on analog-to-digital conversion, He
chines for pattern classification,” IEEE Trans. Pattern Anal. Mach. Intell., is the author or coauthor of the book Numerical Opti-
vol. 29, no. 5, pp. 905–910, May 2007. mization and Applications. His research interests in-
[19] C. Burges, “A tutorial on support vector machines for pattern recognition,” clude machine learning, optimization, and very large
Data Min. Knowl. Discovery, vol. 2, pp. 1–43, 1998. scale integration.
[20] C. Cortes and V. N. Vapnik, “Support vector networks,” Mach. Learn., Prof. Jayadeva is a Speaker on the IEEE Computer Society Distinguished
vol. 20, pp. 273–297, 1995. Visitor Programme. He was a recipient of the Young Engineer Award from
[21] J. Arenas-Garcia and F. Perez-Cruz, “Multi-class support vector machines: the Indian National Academy of Engineering, the Young Scientist Award from
A new approach,” in Proc. IEEE Int. Conf. Acoust., Speech, Signal Pro- the Indian National Science Academy, and the Better Opportunities for Young
cess. (ICASSP), Apr. 2003, pp. 781–784. Scientists in Chosen Areas of Science and Technology Fellowship from the
[22] R. Khemchandani, Jayadeva, and S. Chandra, “Optimal kernel selection Department of Science and Technology, Government of India. He was also a
in twin support vector machines,” Optim. Lett., vol. 3, no. 1, pp. 77–88, recipient of Best Paper Awards from the Institution of Electronics and Telecom-
Jan. 2009. munication Engineers Journal of Research, and three other conference papers.
[23] A. Hyvarinen, “A fast and robust fixed-point algorithm for independent
component analysis,” IEEE Trans. Neural Netw., vol. 10, no. 3, pp. 626–
634, May 1999.
[24] P. Comon, Independent Component Analysis, a New Concept? New
York: Wiley, 2001.
[25] A. Subasia, M. Yilmazb, and H. R. Ozcalika, “Classification of EMG
signals using wavelet neural network,” J. Neurosci. Methods, vol. 156,
no. 1–2, pp. 360–367, Sep. 2006.
[26] Z. Liu and Z. Luo, “Hand motion pattern classifier based on EMG using
wavelet packet transform and LVQ neural networks,” in Proc. IEEE Int.
Symp. IT Med. Educ. (ITME), Dec. 2008, pp. 28–32.

View publication stats