Mid- Term Report For Major Project

Gesture Controlled Robot Using Image Processing

Project Supervisor:
Rajesh Kr. Poudel

Ashish Bhandari (BEX/069/7)

Elisha Thapa - (BEX/069/21)
Priyanka Upadhyaya - (BEX/069/24)
Sarala Thapa- (BEX/069/36)

Introduction
The project is Gesture Controlled Robot
Using Image Processing.
It is a hardware and software based integrated
system .
It is developed for hand gesture based
surveillance robot.
The hardware part is developed based on
Raspberry Pi platform with the help of Python.

Objectives
To provide the real time hand gestures to the
camera and process the obtained image.
To recognize the given gestures, interpret the
given gestures and come to the conclusion.
To be able to monitor the hardware with the help
of hand gesture.
To be able to identify the surrounding objects by
the robot so as to provide the live camera feed to
the controller.

Scope

Defense
Gesture-based wheel chairs
Healthcare environments
Industries
Entertainment field

Requirement Analysis
Hardware components:
o
o
o
o
o

Pi Cam
Motor Driver (L293D circuit)
Raspberry Pi 2 B
DC Motor
Wheels

Software components:
o
o
o
o

Python
Open CV
PCB wizard
Proteus

Block Diagram

Image processing Steps

Image processing Steps

contd

1. Image acquisition and preprocessing

Data acquisition and sensing
Pre-processing

2. Pattern Recognition
Feature Extraction
Classification

3. Post Processing
Evaluation of confidence in decisions.
Exploitation of context to improve
performances.
Combination of experts.

Work Accomplished
Grayscaling:
A grayscale of an digital image is an image in
which the value of each pixel is a single
sample that carries only intensity information.

 Thresholding and binarization:

Replacing each pixel in an image with a black

pixel if the image intensity is less than some fixed
constant
With white pixel if the image intensity is greater
than that constant.

 Edge Detection:

Edge detection is used for extracting real

boundaries of objects in a binary image. Here we
use ordered lists of its edge points
Canny edge detection is used.

 Convex Hull and convexity defects

- The convex hull of a set of points in the euclidean
space is the smallest convex set that contains all the
set of given points.
- Convex curves are the curves which are always
bulged out, or at-least flat.
- Convex hull is drawn around the contour of the
hand, such that all contour points are within the
convex hull.
- This makes an envelope around the hand contour.

- It uses minimum points to form the hull to include all

contour points inside or on the hull and maintain the
property of convexity.
- This causes the formation of defects in the convex
hull with respect to the contour drawn on hand.

 PCB design
PCB for the robotic part was designed and
simulated.

Results

Work remaining
Complete interfacing of the hardware parts and
software.
Complete testing of the hardware and software
parts.

Conclusion
The image acquisition and hardware (PCB)
design required for this project has been
completed.
The project has been going smoothly
according to the planning.

Thank You