(SLASL)

Self Learning Aid for Arabic Sign Language

1. Introduction
1.1 Overview

The report proposes a new technique that helps the deaf to learn sign language.
The basic idea is to combine several tools and techniques together. It gives the user
ability to learn the sign language without human help. He has to use data gloves that
are connected to computer, which has a database for signs clips. When user asks for a
sign, the computer will display its clip. Then the user will try to do it correctly using
data gloves, so the computer captures the sign and make a decision. If the sign is not
matched, the computer will ask for another trial.

1.2 Methodology

The figure shown (Figure 1.1) describes the tools and techniques of this project.
We use data gloves for sign acquisition. Then the Artificial Neural Networks (ANNs)
are used for sign recognition. Then, the graphical user interface (GUI) is used to
display the text and clips.

1.3 Organization of the Report

The report is organized as follows: Chapter 1 states report objectives and the
proposed system is overviewed. Chapter 2 discusses data gloves and its interface to
the PC. In Chapter 3, artificial neural networks (ANNs) are introduced and how our
design programmed, also there is some specifics like back propagation (BP) method.
In chapter 4, graphical user interface (GUI) is introduced and different tools are
explained. GUI use as an interface for our project is also discussed in this chapter. In
chapter 5, we try to discuss future steps needed to improve the work accomplished in
this project.

EE 498 KSU Electrical Engineering Department 1

2. Data Glove
2.1 Introduction

The data glove is a magical tool that captures

hand movement and transfers it into data. It takes
63 samples per second according to the stress of
the five fingers, and reflects it into degrees within
the range (0~4095). That range can be normalized
under specific calibration (0~1).

Under this project, we collect a simple

dictionary (Table 2.1) including seven letters and
numbers that are mostly used (If the proposed
system works for a limited dictionary then, in
principle it can be made to work for a larger one).

2.2 How it Works

The data glove that used is called 5DT Data

Glove 5 Ultra, it has two gloves for right and left
hand. The one glove has five sensors. These are
the positions of the sensors on the right hand.
There is one sensor for each finger to capture the
sign.

There is interface program coming with the

data gloves called Glove Manager. It shows the
state of each sensor and the values of the sensors.
Here is the sensor name and the following real-
time values (the amount of information displayed Figure 2.1 Positions of the sensors.
is depending on the resolution available). The
sensor has the range of values between (0-4095)
in figure 2.2.

2.3 Preprocessing

When the user records data, there is auto

calibration by default; so the data will be different
each time using glove according to maximum
stress. Therefore, the data imported in this project
is recorded under specific calibration which is
chosen manually to solve this problem. After that,
Figure 2.2 the Program window (Glove Manager).
the signs are imported under that calibration as
data files (csv files) that have samples of the
movement with respect to the time.

EE 498 KSU Electrical Engineering Department 2

3. Artificial Neural Networks (ANNs)
3.1 Introduction

An artificial neural network is a system based on the operation of biological

neural networks; it is an emulation of biological neural system. It is a set of parallel
elements called neurons, where they work together to achieve a specific goal which is
set by the designer.

3.2 Preprocessing

There are some processes that needed to start training neural network (ANNs) to
achieve good matching. When the user starts recording data, there is a huge range will
be recorded; so we need to make a standard to synchronize the effective data. There
are two steps to solve this problem.

First, the user has to start with completely

opened hand before do the sign as a reference
sign (Figure 3.1) Also open it again after the
sign is done. However, the data will start with
zero value (no signal) and the end too. That
makes us able to detect the effective data that
will compare.
Figure 3.1 Opened hand (Initial movement).
Second, when the user does the sign, there is
unwanted data that recorded before he reaches
the sign shape (Figure 3.2), and when he back to
the reference sign; because he needs time to do
it. Also, there is lost time after do it. Therefore,
we neglect records equivalent to half second
from start and the end of the data.

3.3 Design Steps Figure 3.2 Sample sign (No sign in ASL).

In this project we use Backpropagation (BP)

method to train ANNs with two hidden layers
using {tansig, logsig} transfer functions because
they are differentiable, although the o/p layer we
select 3 purlin transfer functions. Then we start
by initial case and design four combinations for
the layers transfer function. We found the best
case that has [tansig logsig] in the first and
second layer, respectively. Also by
superposition technique, we select the enhanced
number of neuron for each layer with respect to
error and the training time (Figure 3.3). Figure 3.3 [tansig (25) logsig (5)] ANNs.

EE 498 KSU Electrical Engineering Department 3

4. Graphical User Interface (GUI)
4.1 Introduction

A graphical user interface (GUI) is a graphical

display that contains devices, or components, that enable
a user to perform interactive tasks. The GUI components
can be menus, toolbars, push buttons, radio buttons, list
boxes, and sliders. In MATLAB, a GUI can also display
data in tabular form or as plots, and can group related Figure 4.1 Welcome window.

components.

4.2 Designed GUI

The interface program in the project uses five

windows. The main window WELCOME window has
four buttons [Learn, Test, help, about us] (Figure 4.1);
each button refers to next window.
Figure 4.2 Learn window.
When the user selects Learn button, the Learn
window will be shown (Figure 4.2). There are the signs
on the right. When user chooses the sign, it shows on the
left to teach him how to do it. After that, he can press
Do it button to import his sign; so the output will
show if he do it correctly or not.

When the Test button is selected, the Test

window will show (Figure 4.3). It has four buttons
[New, Do it, Show, Back]. New button generates a Figure 4.3 Test window.
random sign without its clip. So, the user tries to do it by
clicking on Do it to import his movement. After that,
the degree shows immediately. He can try again if it is
not matched; also he can get the right sign by choosing
Show button. So, he can try another test by New
button, or back to the main window by Back button.

The other windows are help and about us

windows as shown (Figure 4.4).
Figure 4.4 Help and about us windows.

SLASL: Self Learning Aid for Arabic Sign Language

Prepared by: Mohammed Al Marshad (+966505193578) eng.mhd17@gmail.com

Yousef Saleh Al Hassoun (+966566218181) you17sef@gmail.com

Supervisor: Dr. Salah Foda (+966507861981) sfoda@hotmail.com

EE 498 KSU Electrical Engineering Department 4