Eye Movement Based Electric

Wheel Chair

Under the guidance of:P.S. Shilpashree

Assistant Professor
Dept. of E&C
SIT, Tumkur

Presented by:Abhishek (1SI12EC004)

Abhishek Khandelwal (1SI12EC005)
Gautam Kumar Singh (1SI12EC038)
Kumar Rahul (1SI12EC045)

DEPARTMENT OF ELECTRONICS AND COMMUNICATION

2015-16

Overview
Motivation
Introduction
Literature survey
Block diagrams
Methodology
System Software
System Hardware
Process Flow
Testing and Results
Conclusion
Future Scope
References

Motivation
Over 100 million people uses WHEEL CHAIR worldwide.
Diseases or accidents injuring the nervous system make
people lose their ability to move their voluntary muscle.
Paralysis may cause a person not move their locomotor
organs such as arms and legs.
People with restricted limb movement cannot use traditional
wheel chair.
Scientist Stephen W. Hawking is perhaps the most wellknown victim of major paralysis Hawking was diagnosed
with incurable Amyotrophic Lateral Sclerosis (ALS).

Scientist Stephen Hawking

Photo Courtesy: www.telegraph.co.uk

Introduction
For different disabilities, various kinds of interfaces
for powered wheelchair control.
The model is a possible alternative for diseases such
as ALS.
The key essentials of the system is detecting and
tracking the eye movements.
Patient on the Wheel Chair assembly looks directly at
the camera and is able to move in a direction.
Camera signals are monitored by OpenCV
application.
Guides the motors wired to the Microcontroller over
the Serial Interface.

Literature Survey
Existing computer input devices with finger,
voice, and gesture can be divided into five
categories:
Bio-potential based method
Voice Based method
Motion based method
Image Analysis method
Search coil method

Advantages and Drawback

Method
Bio-potential based method.

Advantages

Drawback

Potential Difference can be calculated Poor gaze direction accuracy compared to

easily

in

both

light

and

dark video tracker, relatively costly.

Productivity,

Can

help Less accurate on Background Noise,

Environments.
Voice Based Methods

Increases

people who have trouble using their Channel variability, Speaker variability,
hands, Can help people who have Speaking style, Sex of the speaker,
cognitive disabilities

Anatomy of vocal tract, Speed of the

speech.

Motion Based Methods

Allows for more physical support, Uses human effort to navigate like
including adjustable seating such as tilt joystick, etc. for a handicapped with any
and recline. Motion based wheelchair of the organs failed is not useful.
users can also adjust the height of the
chair to see their environment more
clearly.

Search Coil

Coil signals provide a better signal Burden to user, here measuring time is
stability. Therefore, coil signals are limited to approximately 30 to 60. They
better suited for the analysis of fine have limited Lifetime.
details of eye movements.

Block Diagrams
System Block Diagram

Wheel Chair Assembly Block Diagram

Methodology
There are two major components from the
system design standpoint:
Eye-Detection and motion tracking.
Webcam is mounted on a cap, continuously staring at the users eyes.
Video is taken and thereafter processed using OpenCV.
Image processing toolbox which uses Viola Jones Algorithm is used.
Eye centre localisation by means of gradients.

Microcontroller controlled Wheel Chair Prototype.

ASCII characters are received serially depending on the pupil
position.
The microcontroller delivers the desired motion of motor.

Continued..

Viola-Jones Algorithm
Face Detection Algorithm has 4 stages:
Haar Features
Integral Image
AdaBoost
Cascading

Picture Courtesy: www.wikipedia.com

Haar Feature that looks similar to the eye region which

is darker than the upper cheeks is applied onto a face.

Haar Feature that looks similar to the bridge

of the nose is applied onto the face.

...
Viola Jones Algorithm uses 24x24 window as the base window size
to start evaluation of features in any image, i.e., 162,336 possible
features.
An integral image at location (x,y), is the sum of the pixels above
and to the left of (x,y), inclusive.

Picture Courtesy:
www.scielo.sld.cu

Adaboost is a machine learning algorithm which helps in finding

only the best features among all these 160,000+ features.
A Cascade classifier is used which composes of stages each
containing a strong classifier.

Picture Courtesy:
www.wikipedia.com

Eye Centre Localisation By

Means of Gradients [4]
A multi-stage scheme:

A face detector is applied first; based on the face location rough eye regions are extracted (left),
which are then used for a precise estimation of each eye centre (middle and right).

The centre of a circular pattern is defined as the location where most of

the image gradients intersect.
Prior knowledge about the eye appearance is incorporated to increase
the robustness.
simple post processing techniques is applied to reduce problems that
arise in the presence of glasses, reflections inside glasses, or prominent
eyebrows.
Continued..

...

Artificial example with a dark circle on

a light background, similar to the iris
and the sclera. On the left the
displacement vector di and the gradient
vector gi do not have the same
orientation, whereas on the right both
orientations are equal.

Evaluation of (1) for an exemplary pupil with

the detected centre marked in white (left). The
objective function achieves a strong maximum at
the centre of the pupil; 2-dimensional plot
(centre) and 3-dimensional plot (right).

Where wc = I*(cx, cy)

System Software
OpenCV 3.0.0
Open Source Computer Vision Library.
Includes several hundreds of computer vision
algorithms.
Modular Structure.
Core functionality
Video
Calib3d
Features2d
Objdetect, etc.

...
Eclipse 3.8
It is an IDE.
A base workspace and an extensible plug-in
system for customizing the environment.
Develop applications in Java, C++, OpenCV, etc.
using plugins.

...
Microvision Keil
Provides the software development tools for the 8051
family of microcontrollers.
3 main tools, assembler, compiler and linker.
An assembler is used to assemble 8051 assembly program.
A compiler is used to compile C source code into an object
file.
A linker is used to create an absolute object module suitable
for your in-circuit emulator.

Steps to develop 8051 project using Keil:

Create source files in C or assembly.
Compile or assemble source files.
Correct errors in source files.
Link object files from compiler and assembler.
Test linked application.

System Hardware
A Web camera mounted on a cap/helmet.

Camera Specification:
Video Resolution: 2.1MP, USB Interface:2.0
Frame Rate: 30 Frames/sec, 1.5 m Cable

...
AT 89S52 Development Board
RS232 Tx-Rx Interface with MAX232 IC
Crystal of 11.0592 MHz Frequency
Reset Button

...
AT 89S52 Microcontroller IC
8-bit Microcontroller with 8K Bytes In-System
Programmable Flash Memory.
256 byte RAM
32 Programmable I/O Lines
Full Duplex UART Serial
Channel

...
RS 232 Standard
USB to RS232 Converter
MAX232 IC
RS232 Voltages are not compatible
with TTL Voltage level.
MAX232 converts RS232 to
TTL level.
Capable of handling Two Receivers
and Two Transmitters, and
full duplex communication.

...
Motor Driver: L293D IC
Allows DC Motor to drive in either direction.
16 pin IC, controls two DC motors simultaneously.
Works on concept of H-Bridge.

...
Wheel Chair Prototype
A chassis.
Two DC Motors connected to Wheels.
One Dummy wheel.

Process Flow

Testing and Results

Testing
After detection of the eye, rectangular box is
created around the eye.
We divide this rectangle vertically into four parts
and horizontally into three parts.
If the pupil is found in the 1st horizontal division
(shown in Purple), the wheelchair moves forward.
Similarly if it is found in the centre of horizontal
as well as vertical division or in the 3rd horizontal
division (shown in Brown), the wheelchair halts.

...
If the pupil is found in the 4th vertical division
(shown in Blue), the wheelchair turns left.
If the pupil is found in the 1st vertical division
(shown in Pink), the wheelchair turns right.

...
Results

Person looking in the centre,

wheel chair doesnt move (or stops).

Person looking right,

wheel chair turns right.

...

Person looking left,

wheel chair turns left.
Name
Abhishek
Abhishek Khandelwal
Gautam Kumar Singh
Kumar Rahul

Person looking up,

wheel chair moves forward.

Attempts

Successful Attempts

Accuracy

200

185

92.5%

100

91

91%

100

92

92%

100

91

91%

Conclusion
The project aims on setting an idea for a system which could
better the lives of millions of paralysed person in world.
A prototype of an electric wheelchair which works on eye
movement is designed.
The design processes the video, taken by the web camera, uses
Viola-Jones algorithm and pupil detection for eye movement
which is communicated serially to the wheel chair via a
microcontroller.
The system functions with an accuracy rate of 90% which is
above expectations.
The electric wheelchair is designed so that it works in almost all
conditions along with certain safety considerations.

Future Scope
Though the prototype performs satisfactorily, but a lot
of work needs to be done before making the product
commercially viable. Some sort of sequence of events
should trigger start of detection, because one would not
want the wheel chair to move when the person is just
casually glaring in different directions.
Also since the criticality of the application is so high,
lot of safety precautions need to be incorporated. It
needs to be made sure that the system is not fatal to the
health of the person. A lot of testing needs to be done
before making such a product a reality.

References
[1] Kohei Arai and Ronny Mardiyanto. Eyes based electric wheel chair
control system. International Journal of Advanced Computer Science and
Applications, 2(12):98105, November 2011.
[2] Gunda Gautam, Gunda Sumanth, Karthikeyan K. C., Shyam Sundar,
and D.Venkataraman. Eye movement based electronic wheel chair for
physically challenged persons. International Journal of Scientic &
Technology Research, 3(2):206212, February 2014.
[3] Yi-Qing Wang. An analysis of the viola-jones face detection
algorithm. Image Processing On Line, 4:128148, June 2014.
[4] Fabian Timm and Erhardt Barth. Accurate Eye Centre Localisation By
Means Of Gradients. Proceedings of the Int. Conference on Computer
Theory and Applications (VISAPP), Volume 1:125-130, 2011.