HSI 2011

Yokohama, Japan, May 19-21, 2011

Development of a Potential System for Upper

Limb Rehabilitation Training Based on Virtual
Reality
Zhibin Song1, Shuxiang Guo2, and Mohd Yazid2
1
Graduate School of Engineering, Kagawa University, Kagawa, Japan,
2
Dept. of Intelligent Mechanical Systems Engg, Kagawa University, Kagawa, Japan,
s09d505@stmail.eng.kagawa-u.ac.jp, guo@eng.kagawa-u.ac.jp, s07t461@stmail.eng.kagawa-u.ac.jp


have some level of hemiparesis. These patients needed

locally based multi-disciplinary assessments and
appropriate rehabilitative treatments after they were
dismissed from hospital. Rehabilitation for stroke also can
be divided into three types of therapy: physical therapy,
speech therapy, occupational therapy. In physical therapy,
the goal is to strengthen muscles and improve the patients
movement. This paper focuses on the upper extremity
rehabilitation in physical therapy. The conventional
physical rehabilitation for stroke patients which relies on
the use of physiotherapy and the therapists training and
experience is called passive rehabilitation. Meanwhile there
is another method called active rehabilitation that patients
can restore their motor function through using certain
system by themselves [2].
With the development of robotics technology,
robot-assisted rehabilitation for physical rehabilitation and
neurological rehabilitation of patients who have suffered
physical or neurological injuries has been developed and
become an active research area. Until now, there are a
series of robots developed in rehabilitation. Some results
have indicated the robot-assisted movement has an effect
on patients recovery [3]. In 1997, with the cooperation of
Stanford University and Rehabilitation Research and
Development Centre, another rehabilitation system named
MIME has been developed [4]. It uses a PUMA 560
manipulator to provide assistance to move the subjects arm
with a pre-programmed position trajectory. Another work
done at MIT-MANUS[5] on the development of a new
robot that allowed the patient toexercise against therapist
nominated stiffness and damping parameters uses a
different approach from thesystems. The device assisted
planar pointing and drawing movements with an impedance
controller. Another upper limb rehabilitation system (ARM
Guide) can move along horizontal and vertical orientation
with resistance and support to the patient [6]. Gentle/s was a
three year project funded by the European Commission to
develop
machine
mediated
therapies
for
neurorehabilitation of people with stroke. Gentle/s had the
aim to improve quality of treatment and reduce costs [7].
These systems take advantages of sufficient assistance and
range of motion (ROM). However, they are expensive and
heavy so they are not suitable for home-rehabilitation. In

Abstract. This paper proposed a novel rehabilitation system

for rehabilitation training of the upper limbs for patients
whose brain injured such as stroke. We also proposed some
strategies of rehabilitation using this system based on Virtual
Reality. In this paper, Virtual Reality (VR) was adapted in
this system in which the water simulation provided
fundamental environment for rehabilitation training and we
used a haptic device (Phantom Omni) and an inertial sensor
(MTx) to implement tasks proposed. In this paper, three types
of tasks were designed so that comprehensive analyses of
performance would be obtained. As preliminary phase, five
healthy subjects were invited to participate in experiments.
The experimental results showed that the virtual force model
was effective for the upper limbs rehabilitation and the
subjects showed improvement during the experiment. Though
there is some limitation due to the haptic device, this system is
promising, because strategies proposed are potential to be
used in real rehabilitation.
Keywords: Virtual Reality, Home-rehabilitation, Upper
limb, Inertial sensor, Haptic device.

I. INTRODUCTION

troke is a condition where a blood vessel interrupts

blood flow to an area of the brain. A lack of oxygen and
glucose flowing to the brain leads to the death of brain cells
and brain damage, often resulting in impairment in speech,
movement, and memory. Stroke is the leading cause of
disability in the U.S. Approximately 700,000 people
experience a stroke each year. There are an estimated 5
million stroke survivors living in the United States. Stroke
places a financial burden on our society with the projected
cost of stroke care in 2007 approaching 62 billion dollars
[1]. Japan Ministry of Health, Labour and Welfare release
the statistic of the stroke patients and in 2005, the number
of patients already over 1,400,000 persons. In addition, at
least 130,000 of patients are confirmed died because of
stroke every year. In the stroke survivor population, 50%

This research is supported by Kagawa University Characteristic Prior

Research fund 2011.
Zhibin Song, Hayashi-cho, Takamatsu, 761-0396, Japan. Graduate
School of Engineering, Kagawa University, Japan (e-mail:
s09d505@stmail.eng.kagawa-u.ac.jp).

978-1-4244-9640-2/11/$26.00 2011 IEEE

352

recent years, the virtual reality technology has increasingly

been used in the rehabilitation of upper limbs and lower
limbs [8]-[11].
In this research, we want to develop a compact
home-rehabilitation system for the upper limbs. The
approach involves the designing the system that can train
patients upper limbs; around elbow joint and wrist joint in
specific. This system will train 4 degrees of freedom (elbow
flexion/extension;
hand
extension/flexion;
hand
pronation/supination; hand radial deviation/ulnar deviation)
of the injured upper limbs. The rehabilitation training is
based on tracking task where the patients need to track
down a virtual arms movement and control another virtual
arm through manipulating a haptic device and an inertial
sensor. The system will provide the vision through the
monitor and force feedback through the haptic device. This
system also needs to be safe, user friendly, fun and compact
for home-rehabilitation. This paper proposed the three
kinds of rehabilitation strategies and implemented the
active rehabilitation that involved virtual water impedance.
This paper is organized as follows. It first presents the
proposed rehabilitation system in Section 2. Experiment
results are presented in Section 3. The last section presents
paper conclusion.

manipulators hand.

Fig. 2 Phantom omni

II.PROPOSED REHABILITATION SYSTEM

Fig. 3 MTx coordinates M relative to the reference
coordinates R

A. Structure of system proposed

The proposed system provides visual and haptic
feedback to patients and patients are required to perform
several tracking tasks designed. Fig. 1 shows the schematic
diagram of the proposed system.

C. Force model
Reasonable impedance is more effective than no load
paid to patients in active rehabilitation [12]. In this paper,
we implemented water impedance simulation using haptic
device. In addition, water impedance can be adjusted by
setting the viscoelastic coefficient (1) and (2).
(1)
(2)
where F is the force output; L is the length of the arm; k is
viscoelastic coefficient; U is the density of water and  is
the angular velocity of the manipulated arm. Based on this
equation, force will be exerted in opposite direction to
velocity. In this model, the force exerted on arm is
considered and force on hand is not included. The stiffness
of the system can be change easily by setting the
viscoelastic coefficient of the system.

Fig. 1. schematic diagram of the proposed system

B. Apparatus used in this system
The components of system comprise one haptic device
(PHANTOM Omni), an advanced inertial sensor (MTx)
and a computer. To make sure that haptic device can work
smoothly, the performance of computer should be good.
The CPU of the computer is Pentium 4 (3.40 GHz), random
memory is 1GB. The operating system of computer is
Windows XP, the program of virtual simulator is developed
with Visual C++ and OPENGL.PHANTOM Omni is a
kind of haptic device, the effector can move in 3 degree of
freedoms, the maximum exertable force at nominal position
is 3.3N. PHANTOM Omni is connected to computer
through IEEE-1394 FireWire port. (Fig.6) shows the
apparatus of self-assisted rehabilitation system. We use one
tri-axial inertial sensor (MTx) to get the accurate posture of

Fig. 4 schematic diagram of force model

353

m and overlapped each other. And m is programmed to

become transparent so that the manipulated virtual arm m
can be seen easily even when they are in same place. For
this type, the lighting of the background is disabled in order
to put different color for both virtual arms. By doing this, it
is easier for the subjects to differentiate between
manipulated virtual arm m and tracked virtual arm m.

Doing training inside real water will have the effect like
this. Rotating the arm around elbow joint will experience
resistance or force along all of arm. However, this system
did not use real water as resistance and Phantom Omni, this
kind of haptic device can exert force at the stylus only. In
order to simulate the water resistance as much as possible,
we calculated the force through torque since both of torque
inside water is same with torque inside the virtual
environment. Equation (3)-(5) can be got according to
Fig.4.

2). Design of 2nd Type of Training Task

In this 2nd type of training task, the tracked virtual arm
m is placed beside (different x coordinate) manipulated
virtual arm m at same y and z coordinate. By doing this, it is
easier for the subject to see the movement for both virtual
arms.

(3)
(4)
(5)

3). Design of 3rd Type of Training Task

Compare to the 2nd type of training task, this training is
called mirror therapy. The tracked virtual arm m is placed
as the left hand and manipulated virtual arm m is placed as
right hand. The idea of this mirror therapy is to trick the
brain there is no injured part of the body. This type of
therapy was introduced by Vilayanur S. Ramachandran in
1995 to help alleviate phantom limb pain, in which patients
feel they still have a limb after having it amputated. In this
paper, mirror training was implemented in virtual reality
environment. Patients can manipulate virtual devices to
perform mirror training according to another virtual limb.

Assume length of arm as L; velocity of the Phantom Omni

as V; torque applied to the stylus as T; the angular velocity
Z around the arm can be determined by following
expression.
D.Design concept
The environment of the system is projected into the
monitor. This system consists of 2 virtual arms (m and m).
One manipulated virtual arm m is to be operated by the
subject through Phantom Omni haptic device and MTx
inertial sensor. Manipulating the Phantom Omni haptic
device will control the rotation around the elbow joint
(elbow extension/flexion) while manipulating the MTx
inertial sensor will control the movement of hand around
wrist
joint
(hand
extension/flexion;
hand
pronation/supination;
hand
radial
deviation/ulnar
deviation). Another virtual arm which is tracked virtual arm
m will rotate randomly in 4 degree of freedom within the
range. This system will only train the subjects 4 degree of
freedoms of the upper limbs. The training task for the
subject is to track the randomly move virtual arm m and
manipulate another virtual arm m until it is same position
and gesture as randomly move virtual arm m while
overcome the force exerted by the stylus of the haptic
device.

Fig. 5 the 1st Type of Training Task

E. Subjects
As preliminary experiment, five students were invited as
subjects. Subjects are between 22-30 years old and all of
them in healthy condition. Before the experiment, all of the
subjects were explained how the system worked and they
were given time to practice. At first, they were seated on a
height adjusted chair and then, they worn a glove that was
attached with inertial sensor in the back. Then, the subjects
grasped the stylus of the haptic device and the experiment
started. The experiment was conducted for straight one
week in order to compare the performance.
F. Task design
1) Design of 1st Type of Training Task

Fig. 6 the 2nd Type of Training Task

In this 1st type of training task, the manipulated virtual

arm m is placed at same coordinate with tracked virtual arm
354

Fig. 7 the 3rd Type of Training Task

III. EXPERIMENTAL RESULTS
In this paper, we proposed three kinds of training
approaches to upper limb rehabilitation. As preliminary
phase, five young and healthy subjects were required to
perform three kinds of experiments for one week. For
evaluation of system proposed, we analyzed the
performance of all the subjects including tracking results of
four DOFs. In experiments, the elbow flexion/extension
was detected according to haptic device and other three
DOFs on wrist joint were detected using the inertia sensor
which was attached on the back of subjects hand.
Fig.8 shows the typical experimental results of subject A.
(a), (b) and (c) stand for the tracking performance of the
subject A around elbow joint for 1st, 2nd and 3rd type task.
(d), (e) and (f) stand for the tracking performance of the
subject A around wrist joint for 1st, 2nd and 3rd type task. Fig.
8 (a) stands for the tracking performance of the subject A
around elbow joint for 1st type task. In this figure, the blue
single line represents the movement of tracked virtual arm
m while the red dash line represents movement of the

(a)

(d)

manipulated virtual arm m for elbow flexion/extension. In

figure (d), red, green and blue single lines show the
movement of tracked hand m for wrist flexion/extension,
wrist pronation/supination and wrist radial/ulnar deviation.
Meanwhile, the red, green and blue dash line show the
movement of manipulated hand m for wrist
flexion/extension, wrist pronation/supination and wrist
radial/ulnar deviation. These lines also represent same
meaning for the other types of training task. In the 1st type
of tracking performance, subject performed quite well for
all 4 DOFs. In tracking elbow flexion/extension, errors
made by subject were very small compare to the other types
of training. The position of the virtual arms make this
training is easy to perform. The subject knows exactly the
position of tracked virtual arm and doesnt need to estima6
the position. On the other hand, 2nd type of training task
requires the subject to estimate position of the tracked
virtual arm m as it is placed beside the manipulated virtual
arm m (Fig 6). For the tracking performance of 4DOF,
subjects managed to follow the movement of tracked
virtual arm m. However it is not accurate compare to the
1st type of training task. The errors performed for all 4DOF
are bigger. On the other hand, based on the questionnaire
from the subjects, this task has advantage of the vision. It is
easier to distingue manipulated and tracked arms with
comparing to the first type. In this 3rd type (c) and (f), the
movement of manipulated virtual arm around elbow
flexion/extension was not obviously different from 2nd type,
as this motion performed in the same way with it from 2nd
type. However tracking motion in wrist joint was rougher
compared the 2nd type and errors performed were higher
compared to the 1st type. The requirement to transform the
tracked virtual arm m as left arm can cause the rough
movement. The accuracy also decreased as tracked virtual
arm m was place left from manipulated virtual arm m, so
that the subject only can estimate the position of the tracked
virtual arm m and posture of the tracked hand.

(b)

(c)

(e)

(f)
st

Fig 8: (a), (b) and (c) stand for the tracking performance of the subject A around elbow joint for 1 , 2nd and 3rd type task.
(d), (e) and (f) stand for the tracking performance of the subject A around wrist joint for 1st, 2nd and 3rd type task.

355

HSI 2011

Yokohama, Japan, May 19-21, 2011

TABLE 1: COMPARISON OF AVERAGE ERROR OF THE 1ST TYPE TASK.

subjec
t

Elbow
flexion
/extension

Wrist
flexion
/extension

Wrist
pronation
/supination

for the patients under minimum supervisor at home and

reduce the work load of doctors and therapists. As active
training strategies, all three types of tasks proposed adapted
water impedance model exerted on the haptic device. As
preliminary experiment, five healthy subjects participated
in training. For tracking accuracy, the 1st type task is best.
However, subjects had to distinguish both of virtual arm
though the tracked one was designed to be transparent.
Sometimes subjects can not pay much attention to it. The
2nd type task solved this problem, but subjects should
estimate the position and posture of the tracked virtual hand,
therefore, accuracy is lower than the 1st type. We also
implemented mirror training in virtual environment, which
is used in neuro-rehabilitation. This type task gained best
outcome. Above all, these strategies proposed are potential
to be used in upper limb rehabilitation, especially in
home-rehabilitation. In the future, these strategies of
training will be implemented using another powerful haptic
device and mild stroke patients will be invited to
participate.

Wrist
radial/ulnar
deviation

F(%)

L(%)

F(%)

L(%)

F(%)

L(%)

F(%)

L(%)

7.8

5.6

15.9

7.5

11.1

7.4

19.8

14.3

5.4

5.0

8.7

5.8

10.1

7.6

24.1

7.6

7.6

6.3

11.6

9.2

10.2

7.2

13.9

12.9

6.7

4.3

9.8

8.4

9.7

7.2

21.8

13.4

5.2

3.9

7.1

6.3

12.4

8.1

24.3

14.7

TABLE 2: COMPARISON OF AVERAGE ERROR OF THE 2ND TYPE TASK.

subjec
t

Elbow
flexion
/extension

Wrist
flexion
/extension

Wrist
pronation
/supination

Wrist
radial/ulnar
deviation

F(%)

L(%)

F(%)

L(%)

F(%)

L(%)

F(%)

L(%)

13.4

7.8

18.1

13.1

16.7

13.0

26.6

21.6

13.6

6.8

16.1

13.4

17.5

12.5

19.0

14.8

9.9

4.3

13.1

14.8

12.9

11.1

23.7

17.0

10.1

4.7

19.6

16.3

13.2

10.3

24.7

17.1

11.9

5.8

16.0

10.7

14.3

10.2

23.9

14.4

ACKNOWLEDGMENT
This research is supported by Kagawa University
Characteristic Prior Research fund 2011.
REFERENCES

TABLE 3: COMPARISON OF AVERAGE ERROR OF THE 3RD TYPE TASK.

subjec
t

Elbow
flexion
/extension

Wrist
flexion
/extension

Wrist
pronation
/supination

Rosamond W., Flegal K., Friday G., Furie K., et al. Heart Disease
and Stroke Statistics-2007 update. American Heart Association.
2007.
[2] David Czell, Reinhard Schreier, Rudiger Rupp, Influence of passive
leg movements on blood circulation on the tilt table in healthy
adults, Journal of Neuroengineering and Rehabilitation, 1:4, 2004.
[3] Mary K. Seaton,
Gail N. Groth,
Leonard Matheson
and
Christine Feely, Reliability and Validity of the Milliken Activities
of Daily Living Scale, Journal of Occupational Rehabilitation, vol.
15, no. 3, pp.343-351, 2005.
[4] L. E. Kahn, W. Z. Rymer and D. J. Reinkensmeyer, Adaptive
Assistance for Guided Force Training in Chronic Stroke,
Proceedings of the 26th Annual International Conference of the
IEEE EMBS, pp.2722-2725, 2004.
[5] R.F. Boian, M. Bouzit, G.C. Burdea and J.E. Deutsch, Dual Stewart
Platform Mobility Simulator, Proceedings of the 26th Annual
International Conference of the IEEE EMBS, pp.4848-4851, 2004.
[6] L. E. Kahn, W. Z. Rymer and D. J. Reinkensmeyer, Adaptive
Assistance for Guided Force Training in Chronic Stroke,
Proceedings of the 26th Annual International Conference of the
IEEE EMBS, pp.2722-2725, 2004.
[7] D. Khalili and M. Zomlefer, An intelligent robotic system for
rehabilitation of joints and estimation of body segment parameters,
IEEE Trans. Biomed. Eng., vol.35, pp. 138-146, 1988.
[8] Kahn, L.E.; Zygman, M.L; Rymer, W.Z. &Reinkensmeyer,D.J.
Robot-assisted reachingexercise promotes arm movement recovery
in chronic hemiparetic stroke: a randomized controlledpilot study.
Journal of NeuroEngineering andRehabilitation, vol3, 12, pp. 1-13
2006.
[9] H. Ring, Is neurological rehabilitation ready for immersion in the
world of virtual reality, Disability and Rehabilitation, vol.20, no. 3,
pp. 98-101, 1998.
[10] L. E. Jones, Does virtual reality have a place in the rehabilitation
world, Disability and Rehabilitation, vol. 20, no. 3, pp.102-103,
1998.
[11] Gang Song, Shuxiang Guo, Development of an Active Self-assisted
Rehabilitation Simulator for Upper Limbs, The 6th world congress
on Intelligent Control and Automation, pp.9444-9448, 2006.
[12] Shuxiang Guo and Zhibin Song, A Novel Motor Function Training
Assisted System for Upper Limbs Rehabilitation. Proceedings of
the 2009 IEEE International Conference on Intelligent Robots and
Systems, 2009. pp. 1025-1030.
[1]

Wrist
radial/ulnar
deviation

F(%)

L(%)

F(%)

L(%)

F(%)

L(%)

F(%)

L(%)

9.6

7.2

27.6

10.1

15.2

10.9

28.9

11.8

13.9

6.1

24.8

7.2

19.3

10.3

30.3

16.7

9.2

3.3

16.1

6.8

11.8

8.8

27.5

12.3

15.2

2.5

11.1

6.4

11.7

7.2

21.9

14.9

8.8

3.5

12.6

7.9

12.7

9.8

28.7

13.7

Table 1,2 and 3 show comparison of average error of the

1st, 2nd and 3rd type tasks between the first day and the last
day by all subjects. To evaluate the performance, equation
(6) is adapted.

Gp

1 N yi  yi,
u100%

N i 1 yi,

(6)

where all data of evaluation G p were shown in Table 1 to 3.

N stands for the sampling. yi shows the manipulating data.

yi, shows the tracked data. From tables, different average

errors were shown to evaluate each subjects manipulation
performance. During one week training, subjects improved
their performance in all three types of tasks, and
improvement degree for each subject is different, especially
in 3rd type task. So these strategies of training are potential
in upper limb rehabilitation.
IV. CONCLUSION
What this project tries to achieve, is to provide therapy
356