A Project presentation on

Human Mouse
Interface
By:
ESHPREET BAJWA(Roll no. H-15)
SMITA CHANDURKAR(Roll no. H-27)
ANAGHA GOLHAR(Roll no. J-19 )

Under guidance of:

Prof. S.A. Khoje
 INDEX
 Problem Definition
 Project Objectives
 Face detection methods
 SSR filter
 Skin color model
 SVM
 Face detection algorithm
 Face tracking
 H/W & S/W requirements
 Feasibility
 Future prospects
Problem definition
 This project aims to present an application
that is able of replacing the traditional
mouse with the human face as a new way
to interact with the computer.

 Facial features (nose tip and eyes) are

detected and tracked in real-time to use
their actions as mouse events.
Project Objectives

 Real Time Response.

 To enable physically handicapped people
to use the mouse.
 Nose tip = mouse pointer.
 Eye blink = mouse click.
Face detection methods
 Two main categories: Feature-based
methods, and image-based methods.

 Feature-based methods

 Image-based methods
Face Detection Algorithm
Overview

Face detection general steps.

Find Face Candidates

 We will be using feature based face detection

methods.

 This is to reduce the area in which we are looking for

the face, so we can decrease the execution time.

 To find face candidates the SSR filter will be used

SSR Filter
 SSR Filter stands for: Six Segmented
Rectangular filters.
The sum of pixels squaren in each sector is
denoted as S along with the sector
number.

SSR Filter.
How do we use SSR?
 Integral Images

 Integral image at
location x, y contains
the sum of pixels
which are above and
to the left of the
pixel x, y
….contd.

 We will need only 3 arithmetic

operations to calculate the
sum of pixels which belong to
it
 Sector = D – B – C + A.
 So each SSR filter requires
6*3 operations to calculate it.
Ideal SSR filter location for a face
candidate
Skin Color Model
 Use the pure r and g values which are the R and G values of the RGB color
model in the absence of brightness, and they are calculated with the following
equations:
 r = R / (R + G + B) (1)
 g = G / (R+ G + B) (2)
 We move the r and g values to the (a, b) color space with the following
equations:
 a = r + g / 2 (3)
 b = √3 / 2 g (4)
 The range of ‘a’ is from 0 to 1, while the range of ‘b’ is from 0 to √3 / 2.
 0.49<a<0.59
 0.24<b<0.29
 SVM
SVM stands for: Support Vector Machines, which are new types of
maximum margin classifiers.
The center of each cluster that is big
enough is set with the following equations:
x = [∑ x(i)] / n
y = [∑ y(i)] / n
i is the pixel from the cluster, n is the
cluster’s area.
 Find Pupils’ Candidates

 Find the pixels that

belong to a dark
area , the sector is
binarized with a
certain threshold.

 The clusters of the

binarized sector
are found.
Extract BTE Templates

 After finding pupils candidates for each of the

clusters (face candidates) we can extract BTE
templates in order to pass them to the support
vector machine.

 We train our support vector machine on

templates of size 35 * 21 pixels.
35 * 21 pixels that
represent the original
template.
An extracted template that was rotated back to a horizontal pupils’ alignment.
The larger version is only for observation. It will not be used in our face detection process.
Face Tracking
 Detecting The Eyebrows
 Eyes Tracking
 Tracking The BTE
 Tracking the nose tip
 Blink Detection
 Hardware Requirements
o Pentium IV processor
o Webcam

Software Requirements
o Windows XP
o JAVA/JAVAX
Feasibility
 Technical Feasibility

 Economical Feasibility

 Operational Feasibility

 Schedule Feasibility
.
Future Works
o Improving the tracking robustness against
lighting conditions.
o Adding the double left click and the drag
mode
o Adding voice commands
o Enable/disable controlling the mouse with
the face
Thank you