SIGN LANGUAGE TO

TEXT/ VOICE
CONVERTER
Project Guide :
MS. REKHA RAJ C T

Group Members :
ALJO JOSE
ANOOP K

KENNY T CHAKOLA
KIRAN JAMES
NAVEEN K U

ABSTRACT

Communications between deaf mute and normal person

have always been a challenging task.

Goal to reduce barrier of communication by developing an assistive device for deafmute persons.

The work is verified under MATLAB environment.

:- DESIGN OF A COMMUNICATION AID FOR PHYSICALLY CHALLENGED
Suganya R , Dr.T.Meeradevi

SIGN LANGUAGE TO TEXT / VOICE CONVERTER

ENHANCEMENT

Implementing the work as an android application

Use deep learning convolution network for implement the work.

SIGN LANGUAGE TO TEXT / VOICE CONVERTER

INTRODUCTION

There are about 70 million deaf people who use sign language as their first language or
mother tongue.

Goal is to implement an android application that capture the sign language (video) in
real time and convert it into text and voice.

The proposed system use the deep learning and artificial neural network features to

perform the conversion.

SIGN LANGUAGE TO TEXT / VOICE CONVERTER

LITERATURE SURVEY
PAPER: DESIGN OF A COMMUNICATION AID FOR PHYSICALLY CHALLANGED

Implementing a s/m without hand held gloves and sensors-capturing the gestures
continuously and converting to voice in MATLAB environment.

In 2 phases: training phase-Feed Forward Neural Network and testing phase-footage

taken segmented and then compared with database created.

PAPER: SIGN LANGUAGE RECOGNITION SYSTEM BY PATTERN MATCHING

Efficient pattern matching technique for automatic translation-uses concept of boundary

tracing and pattern matching.

SIGN LANGUAGE TO TEXT / VOICE CONVERTER

LITERATURE SURVEY
PAPER: INDIAN SIGN LANGUAGE INTERPETTER

Implementing Viola-Jones algorithm in real time-creates classifier using OpenCV library.

Author successfully created M gesture recognizer-1:2 ratio of positive to negative images

Created an android application.

PAPER: INTELLIGENT SIGN LANGUAGE RECOGNITION USING IMAGE PROCESSING

Fast algorithm for identification of no: of fingers opened in gesture representing an alphabet
of binary language.

Use combination of image processing, machine learning and artificial intelligence.

SIGN LANGUAGE TO TEXT / VOICE CONVERTER

LITERATURE SURVEY
PAPER: HAND TRACKING AND GESTURE RECOGNITION SYSTEM USING LOW COST
HARDWARE

Author presents design and development of a robust marker-less hand/finger tracking and gesture
recognition system using low cost hardware.

System is capable of extracting human hand motion and tracking despite complex backgrounds
and motion blur.

Drawback: technique used-human skin colour extraction based on a threshold value- different
persons has different skin tones-calibration has to be done while setting the threshold.

SIGN LANGUAGE TO TEXT / VOICE CONVERTER

PROBLEM DEFINITION

The earlier implementation has less accuracy and efficiency.

It was not completely implemented and tested in MATLAB or similar environments.

SOLUTION

By implementing as android application get high portability and usability.

Use convolution ANN for implementing the work.

SIGN LANGUAGE TO TEXT / VOICE CONVERTER

DESIGN

MODULES

1.

Video acquisition and pre-processing : - Capture video of communication and take frames
from it.

2.

Home Page: - provides sign language gallery and basic information regarding how to use this
application.

3.

Feature extraction: - Responsible for detecting the hand movements from the extracted frames.

4.

Translator:- take extracted features as input and use convolution ANN to find the text of the
corresponding signs.

5.

Speech synthesis:- generates meaningful sentences.

SIGN LANGUAGE TO TEXT / VOICE CONVERTER

DIAGRAMS

SIGN LANGUAGE TO TEXT / VOICE CONVERTER

BLOCK DIAGRAM

10

CONCLUSION

In earlier approaches accuracy and efficiency was very less.

So we proposed a new method using deep learning convolution network.

The implementation as android application enable more portable and easy to use
environment without having any other expenditure.

SIGN LANGUAGE TO TEXT / VOICE CONVERTER

11

REFERENCE

A. Sujith Kumar, Shahabaz Begum Sign Mobiles: An Android App for Specially Able People Volume 4, Issue 9,
September 2014 ISSN: 2277 128X .

Sujeet D.Gawande, Prof. Nitin .R. Chopde Neural Network based Hand Gesture Recognition International Journal of
Emerging Research in Management &Technology ISSN: 2278-9359 (Volume-2, Issue-3) 2013 .

George Corser Why Not Mobile Sign Language Translation? (2012).

Shanmukha Swamy, Chethan M P , Mahantesh Gatwadi Indian Sign Language Interpreter with Android Implementation
International Journal of Computer Applications (0975 8887) Volume 97 No.13, July 2014 .

Ashok kumar sahoo, Kiran kumar ravulakollu Vision based indian sign language character recognition(2014).

Ruslan Kurdyumov, Phillip Ho, Justin Ng.Sign Language Classi

cation Using Webcam Images

Beginning Android 4 by Grant Allen

http://www.tutorialspoint.com/android/

http://deeplearning.net/software/theano/

SIGN
LANGUAGE TO TEXT / VOICE CONVERTER

http://note.sonots.com/SciSoftware/haartraining.html

12

SIGN LANGUAGE TO TEXT / VOICE CONVERTER

13