i

FACE RECOGNITION USING NEURAL NETWORK

ALFATIHAH BINTI ABDULLAH

This thesis is submitted as partial fulfillment of the requirements for the award of the
Bachelor of Electrical Engineering (Hons.) (Electronics)

Faculty of Electrical & Electronics Engineering

Universiti Malaysia Pahang

NOVEMBER, 2010

"I herebyacknowledgethat the scopeand quality ofthis thesisis qualified for the
awardofthe Bachelor Degreeof Elecfiical Engineering@lectronics)"

signature,

tf'

Name

: MOHD.HISYAM BLNMOHD ARIFF

Date

: 29NOVE\@ER2010

111

*All the uadema* and copyrightsusehereinaxpropertyof their respectiveowner.

Referencesof information from other sourcesarequotedaccordingly;otherwisethe
informationpresentedin this rcport is solely work of the author."

Signature
Author

: ALFATITIAHBINTI ABDULLAH

Date

: 29NOVEMBER2010

ACKNOWLEDGEMENTS

I am grateful and would like to express my sincere gratitude to my supervisor

Mr. Mohd Hisyam bin Mohd Ariff for his germinal ideas, invaluable guidance,
continuous encouragement and constant support in making this research possible. His
guidance and support have been a constant source of encouragement throughout the
work of this thesis. It has been a great honor to have worked under his supervision.
His valuable suggestions and feedback at every critical phase throughout the work
were of utmost importance for timely completion of the thesis. His tremendous
knowledge about the subject has gone a long way in ensuring the successful
completion of this thesis. I also sincerely thanks for the time spent proofreading and
correcting my many mistakes.

I would like to express my gratitude towards to Mrs Nurul Wahidah Bt

Arshad, Mrs. Faradila Naim, and Mrs Rohana Abdul Karim for the short course
training about image processing with MATLAB. Their support this work by agreeing
to be part of the committee and providing me with critical inputs throughout the work
of this thesis.

A special thanks to all the staff of the Faculty of Electrical Engineering without
whose help this work would not have been possible.

Finally, I acknowledge my sincere indebtedness and gratitude to my parents

for their love, dream and sacrifice throughout my life. I would like to thank my friends
for their continued support which has helped me stay strong and focused on the thesis
work and those who love and care for me.

vi

ABSTRACT
Facial recognition systems are computer-based security systems that are able to
automatically detect and identify human faces. Facial recognition has gained
increasing interest in the recent decade. Over the years there have been several
techniques being developed to achieve high success rate of accuracy in the
identification and verification of individuals for authentication in security systems.
This project experiments the concept of neural network for facial recognition that can
differentiate and recognize face of image. This face recognition system begins with
image pre-processing and then the output image is trained using Backpropagation
algorithm. Backpropagation network learns by training the inputs, calculating the error
between the real output and target output, and propagates back the error to the network
to modify the weights until the desired output is obtained. After training the network,
the recognition system is tested to ensure that the system can recognize the pattern of
each face image. The purpose of this project is to recognize face of image for the
recognition analysis using Neural Network. This project is mainly concern with offline
facial recognition systems using purely image processing technique. The system will
find database image has a maximum percentage on similarity of the pattern of the
image. This project is also to design a pattern recognition system by applying Neural
Network Toolbox in MATLAB software.

vii

ABSTRAK

Sistem pengenalan muka (facial recognition) ialah sistem keselamatan yang

berasaskan komputer dengan mengesan dan mengenal muka manusia secara
automatik. Pengenalan muka telah banyak menarik minat, kebelakangan dekad ini.
Beberapa teknik atau cara telah dikaji dan dibangunkan untuk mencapai tahap
ketepatan dengan kadar kejayaan yang tinggi dalam usaha mengenalpasti seseorang
individu untuk diberi kebenaran laluan dalam sistem-sistem keselamatan. Konsep
projek ini adalah pengenalan muka menggunakan rangkaian saraf (neural network)
dimana dapat membandingkan dan mengecam imej muka. Sistem pengenalan muka
bermula dengan proses awal terhadap imej dan kemudian imej keluaran dilatih
menggunakan algoritma Backpropagation. Rangkaian backpropagation belajar dengan
berlatih masukan imej, pengiraan ralat diantara keluaran sebenar dengan keluaran
yang dikehendaki, dan akan mengulang balik kembali ralat untuk mengubah suai
pemberat sehingga keluaran yang diingini diperolehi. Selepas rangkaian dilatih, sistem
pengenalan diuji untuk memastikan yang sistem itu boleh mengenalpasti setiap jenis
bentuk muka. Objektif utama projek ini adalah untuk mengenali imej muka dan
seterusnya untuk menganalisis imej menggunakan rangkaian saraf. Projek ini hanya
ditumpukan kepada sistem pengenalpastian wajah luar talian menggunakan teknik
pemprosesan imej. Sistem ini akan memperolehi imej pangkalan data yang mana
mempunyai peratusan yang paling tinggi pada persamaan corak imej. Projek ini juga
adalah untuk mereka sistem pengenalan menggunakan aplikasi rangkain saraf
alatkotak dalam perisian MATLAB.

viii
TABLE OF CONTENTS

Page
TITLE PAGE

DECLARATION

ii

DEDICATION

iv

ACKNOWLEDGEMENTS

ABSTRACT

vi

ABSTRAK

vii

TABLE OF CONTENTS

viii

LIST OF TABLES

xi

LIST OF FIGURES

xii

LIST OF SYMBOLS

xiii

LIST OF ABBREVIATIONS

xiv

CHAPTER 1

INTRODUCTION

1.1

Introduction

1.3

Problem Statement

1.3

Objectives of the Research

1.4

Project Scope

1.5

Verification vs. Identification

1.6

Thesis Organization

CHAPTER 2

LITERATURE REVIEW

2.1

Preprocessing

2.2

Basis of Facial Recognition Process the PCA

2.3

Neural Networks

10

2.3.1
2.3.2

11
14
14
15

Model of Neuron
Neural Network Architectures
2.3.2.1 Components of A Neural Network
2.3.2.1 Various Subcategories of The Neural Network

ix

2.3.3
2.3.4
2.3.5
2.3.6
2.4

Learning Methods
Backpropagation Network
Backpropagation Learning
Backpropagation Algorithm

16
17
17
18

Facial Recognition Analysis

19

2.4.1
2.4.2
2.4.3

19
20
20

Facial image acquisition

Facial Feature Data Extraction
Facial Classification

2.5

Pattern recognition

2.6

MATLAB, Image Processing Toolbox and Neural

toolbox

2.7

Previous Research

23

2.8

Summary

24

CHAPTER 3

21
Network

22

METHODOLOGY

3.1

Introduction

26

3.2

Preprocessing

27

3.2.1
3.2.2

29
30

3.3

3.4

Histogram equalization
Average Filtering

Feature extraction

31

3.3.1

31

Principal Component Analysis (PCA)

Classification

33

3.4.1 The Neural Network

3.4.2 The binarized image is applied in the neural network
training as the input.
3.4.3 Training of Neural Networks for Recognition

33
35

3.5

Database

39

3.6

Summary

41

38

CHAPTER 4

RESULTS AND DISCUSSION

4.1

Introduction

42

4.2

Preprocessing

42

4.3

Neural Network Training

46

4.3.1
4.3.2
4.3.3

47
47
49

Training Experiment
Test Result for the Face Database
Training and Recognition

4.4

Actual Recognition Test and Result

50

4.5

Recognition Analysis

52

CHAPTER 5

CONCLUSION AND RECOMMENDATIONS

5.1

Introduction

53

5.2

Conclusions

53

5.3

Recommendations

55

5.3.1
5.3.2
5.3.3
5.3.4

55
55
56
56

Practical software implementation

Improving Image Quality
Combination Of Algorithms
User-friendly Console Design

REFERENCES

57

APPENDICES
A

Main Coding

60

xi
LIST OF TABLES

TABLE NO.
4.1

TITLE

Results for Backpropagation Neural Network

PAGE
51

xii

LIST OF FIGURES

FIGURE NO.

TITLE

PAGE

1.1

Identification vs. Verification

2.1

A typical model of an artificial neuron

11

2.2

Typical neural network architecture

14

3.1

Block diagram of the project of facial recognition

26

3.2

Coding for read image

27

3.3

Coding for image pre-processing

27

3.4

Histogram equalization

29

3.5

5*5 Mean Filter

3.6

Architecture of neural network

36

3.7

The set of 10 images for two subjects. Considerable variation

can be seen

39

4.1(a)

The image after binarization

42

4.1(b)

The Binary Image Input Vector

43

4.2(a)

Preprocessing for example dataset

44

4.2(b)

Preprocessing for example dataset

44

4.3

Neural Network Training Window

45

4.4

Mean face for ORL Face Database

48

4.5

Training window

48

4.6

Test image name window

49

4.7(a)

Match the test image with the database image

50

4.7(b)

Match the test image with the database image

50

5.1

Example of a console design, GUI.

55

31

xiii

LIST OF SYMBOLS

Xi

Input grayscale Pixel

Output binarization Pixel

netj

Activation function at hidden layer

Wij

Weight between input layer and hidden layer

Bias weight at hidden layer

Input layer

Hidden layer
Slope parameter which adjust the abruptness of the function
j

Oj
k

Error signal through hidden layer

Output at hidden layer
Error signal at the output layer

Wkj

Weight between output layer and hidden

Ok

Output at output layer

Wij

Weight between input layer and hidden layer

Wij

Adaptation weight between input (i) layer and hidden layer (j)
Learning rate

Iteration number

xiv
LIST OF ABBREVIATIONS

FR

Facial recognition

ANN

Artificial neural network

PCA

Principle component analysis

IC

Identification card

FPGA

Field-programmable gate array

ASCII

American Standard Code for Information Interchange

OCR

Optical character recognition

RGB

Red Green Blue

JPEG

Joint Photographic Experts Group

BMP

Bitmapped image format (Microsoft Corp.)

GIF

Graphics Interchange File; Graphics Interchange Format

ORL

The Olivetti and Oracle Research Laboratory

3D

Three-dimension

CHAPTER 1

INTRODUCTION

1.1

PROJECT BACKGROUND

Face recognition is very important for our daily life. It can be used for remote
identification services for security in areas such as banking, transportation, law
enforcement, and electric industries, etc. For this security access project is aimed at
demonstrating facial recognition techniques that could antiquate, substitute, or
otherwise, supplement, conventional key, and can be used as an alternative to existing
fingerprint biometrics method. A computerized system equipped with a digital camera
can identify the face of a person and determine if the person is authorized to start the
vehicle. This integrated system would be able to authorize a user before switching on
the vehicle with a key. Whilst facial recognition systems are by now readily available in
the market, the vast majority of them are installed at large open spaces, such as in
airport halls. The focus of this project is, thus, to compare the extracted feature with
face image database for the recognition analysis using Neural Network.

Biometric identification is the technique of automatically identifying or

verifying an individual by a physical characteristic or personal trait. The term
automatically means the biometric identification system must identify or verify a

human characteristic or trait quickly with little or no intervention from the user.
Biometric technology was developed for use in high-level security systems and law
enforcement markets. The key element of biometric technology is its ability to identify
a human being and enforce security. Since the beginning of time, humans have relied on
facial recognition (FR) as a way to establish and verify another persons identity. FR
technology isnt any different. Using software, a computer is able to locate human faces
in images and then match overall facial patterns to records stored in a database.

Face recognition, although a trivial task for the human brain has proved to be
extremely difficult to imitate artificially, because although commonalities exist between
faces, it can vary considerably in terms of age, skin color, face angle, facial expression
and presence of facial furniture such as glasses or facial hair.

Artificial neural network (ANN) is a complicated, able-to-learn, nonlinear and

dynamic system. It is very useful for pattern recognition such as face recognition to
increase identification accuracy and robust of the recognition system. Principle
component analysis (PCA) is a statistic method. Though it can overcome some bright
problem, its identifying accuracy is not very high. To overcome the problem of
brightness and to increase the identifying accuracy, this project apply ANN to face
recognition system. [1]

Facial recognition is one of classical applications of the Artificial Neural

Network. This recognition system use neural network approach to recognize the image
according to the neural network, this project use backpropagation network. The
backpropagation learning is a technique discovered by Rumelhart, Hinto, and Williams
in 1986 and it is a supervised learning that learns by propagating the signals through the
network, computing the input and output using a feedforward network, then calculates
the error values and propagates the error back through the network to adapt the weight
during training. To perform this project, a simulator program, Matlab R2008a is applied
in such a way that the image face can be able to assign an input pattern or to train the
network. According to the Matlab software, there is a Neural Network Toolbox that
helps this project to train the network.

1.1

PROBLEM STATEMENT

Face recognition is a pattern recognition task performed specifically on faces. It

can be described as classifying a face either "known" or "unknown", after comparing it
with stored known individuals. It is also desirable to have a system that has the ability
of learning to recognize unknown faces. Computational models of face recognition
must address several difficult problems. This difficulty arises from the fact that faces
must be represented in a way that best utilizes the available face information to
distinguish a particular face from all other faces. Faces pose a particularly difficult
problem in this respect because all faces are similar to one another in that they contain
the same set of features such as eyes, nose, and mouth arranged in roughly the same
manner.

In view of the high rate of crime, fraud and terrorism in the world today, it is
becoming increasingly more important to have remote monitoring systems that work
well with other security devices. As security threats and frauds become increasingly
rampant, it is necessary to have systems that allows recognition and monitoring of
unauthorized people being in an area, using an equipment.

Identity fraud starts when an individual uses multiple identification documents

such as driver's licenses, passports, visas, national Identification Card (IC), etc., under
assumed identities. This is possible because in most countries, documents such as birth
certificates are very easy to fake. Databases may contain facial photographs and thus in
principle, have the information required to prevent duplication. However, in practice, it
is impossible for a human to search over millions of photos to find those duplicates.
Fortunately, computers are able to do this function. Using Face Recognition, millions of
images can be checked for possible matches quickly, automatically and with
phenomenal accuracy. The software returns to the investigator any matches exceeding a
confidence threshold and rank these matches in terms of diminishing resemblance or
likelihood of a match.

With the emergence of biometric recognition systems, it is possible to achieve

the above by combining a password, for example, with a face recognition system or a

fingerprint recognition system. Although face recognition systems are popular and
highly accurate, many do not perform well in real time environment. Factors such as
scale, rotation, pose and lighting prove to be limiting factors in a real time face
recognition system performance. Preprocessing is normally necessary in order to obtain
results that are satisfactory. To overcome the problem and to increase the identifying
accuracy, this project apply ANN and PCA to face recognition system. By using face
recognition biometrics compare to fingerprint recognition, face recognition will reduce
opportunity of theft, and make the system more secured.

1.2

PROJECT OBJECTIVES

The objectives of this project are:-

i.

To compare the extracted feature with face image database for the recognition
analysis using Neural Network.

ii.

To design a pattern recognition system using Neural Network in MATLAB

software.

iii.

1.3

To declare a matches of the same facial features for the recognition.

PROJECT SCOPES

The scopes of this project are limited to:

i.

Interface the system by form of an offline process.

ii.

Design and train the network with backpropagation algorithm learning and
weight adaptation.

iii.

This system cannot distinguish between twins.

iv.

Facial database taken from AT & T Laboratories, Cambridge.

As the project developed, it was decided that hardware implementation of the

experiment such as using FPGA would encounter many technical constraints, such as
the demanding time to learn new programmers and application to real-time conditions,

as well as physical limitations of using FPGA, such as processing speed and need for
very large memory size for data, for image processing. It was also decided that many
algorithms have been developed by researchers in the area of image processing and
recognition, and developing new methods requires extensive knowledge in various
aspects of programming and theory behind the development.

Because of these constraints, it was decided that in the best interest of achieving
the requirements of this course that the coverage is simplified but to a workable product,
and it is certainly better to utilize the tools available in the market, i.e. by using
MATLAB programmer, and exploring the combination of several techniques. This
project uses MATLAB Release R2008a (which contains additional and advanced
blocksets not found in the previous MATLAB suite, especially the Video & Image
Processing Blocksets).

1.4

VERIFICATION VS. IDENTIFICATION

In general, face recognition consists of two different ways of usage, which are face
verification and face identification. The former is used for verification of the owner and
the latter is used for matching input identity with registered identity in the database. For
mobile device applications, since these mobile devices are mainly used for personal use,
the face verification is mainly used rather than the identification. So from now on, we
will describe face verification mainly. There are two different ways to recognize a
person: verification and identification. Verification (answers the question Am I who I
claim I am?) involves confirming or denying a person's claimed identity. In
identification, the system has to recognize a person (addressing the question Who am
I?) from a list of N users in the template database. Identification is a more challenging
problem because it involves 1:N matching compared to 1:1 matching for verification.

Figure 1.1: Identification vs. Verification

1.5

THESIS ORGANIZATION

This thesis composed of 5 chapters each will detail out of details upon every
aspect of this project. The beginning of this thesis will explain on what foundation the
system to be built on. Chapter 1 has introduction, to give definition of the project
background to the facial recognition system and how we could use it in security
propose. The problem statement was also mentioned in this chapter as well. Project
objectives and scopes of the methodology of this project had been included, at the end
of chapter 1 the organization of this thesis had been come.

The preceding chapter 2 will touch on why this project was proposed. The
literature review of the project which contains the background of the problem statement
and previous works on face verification and identification.

Chapter 3 has the discussion of the methodology of this project, image

processing and neural network was explained in this chapter. Also explains the research
methodology used in the project. As the methodology involved only the software, the
chapter provides the overall methodology involved.

The results of this project had enough explanation in the chapter 4. These results
are also analyzed and discussed here to give an in-depth view of the accuracies of this
system in modeling by using MATLAB.

Chapter 5 was included the summarizing of this project, the benefits of this
project and suggestion of future work been mentioned in this chapter. In the end of this
thesis the appendixes were organized.

CHAPTER 2

LITERATURE REVIEW

2.1

PREPROCESSING

Image preprocessing is a significant step before applying any other technique on

images. Image preprocessing can be of different kinds, for noise removal, smoothing,
thresh holding, background removal etc., it can be rightly said that the kind of
preprocessing required is greatly dependent on the application under consideration.
In many applications involving face recognition, the inputs are first binarized to
form a two level image based on the threshold value. It is usual to preprocess the
scanned images to do the image enhancement in the presence of noise and other types of
distortions that occur during the scanning process. In this preprocessing stage, the image
is enhanced, resized and binarized to make the image clear and accurate. It is necessary
to employ the non linear technique for processing the face images prior to binarization
[2, 3]

There are several methods available for thresholding image to produce binary
image. An experimental performance evaluation of several such techniques may found
in [2]. These methods include fixed global threshold, Otsu threshold and other
techniques. The input grayscale pixels are denoted by Xi {0, 1}. The corresponding

output binarization pixels are denoted by bi {0, 1} where 0 refers to black' and 1
refers to 'white'[2].

Image enhancement is necessary to remove noise by filtering the image, adjust

the contrast of the image and enhance the edge of the face image before binarization
process and training the facial recognition. Edge enhancement can be achieved by noise
presents in the input images but tend to smooth the edge details [3].

In this project, the global image threshold using Otsu's method will be chosen as
the technique to binarized the face images. This method finds the global threshold t that
minimizes the infraclass variance of the resulting black and white pixels of the image.
This method is also standard binarization technique and was implemented in Matlab
function "graythresh". The threshold of this method can be shown as: [2, 4]

Where;
Xi = Input grayscale Pixel
= Output binarization Pixel

2.2

BASIS OF FACIAL RECOGNITION PROCESS THE PCA

The recognition process involves comparing the Eigenface weights for two faces
using a proprietary algorithm that generates a match score. Different faces will produce
a poor match score; images of the same face will produce a good match score.

The process begins by reducing the variability of the human face to a set of
numbers. Using a mathematical technique called Principal Components Analysis
(PCA), one can examine a large group of faces and extract the most efficient building
blocks required to describe them. It turns out that any human face can be represented as
the weighted sum of 128 of these building blocks, known as Eigenfaces, based on the
pioneering works of M. Turk and A. Pentland [5]. With this technique, the essence of a

10

human face can be reduced to just 256 bytes of information. The recognition process
involves comparing the Eigenface weights for two faces using a proprietary algorithm
that generates a match score. Different faces will produce a poor match score; images of
the same face will produce a good match score.

The system requires a one-to-one comparison, the Eigenface weights of

authorized personnel are recorded in a central database. When someone appears before
the camera, his or her face is quickly compared to all of the faces in the database to see
if it generates a match.

The scheme is based on an information theory approach that decomposes face

images into a small set of characteristic feature images called eigenfaces, which may be
thought of as the principal components of the initial training set of face images. When
the eigenfaces of a database is constructed, any face in this database can be exactly
represented with the combination of these eigenfaces. In combination of these
eigenfaces, the multipliers of them are called the feature vectors of this face, and this
face can be represented this new descriptors. Each person in database has his/her own
neural network. Firstly, these neural networks are trained with these new descriptors of
the training images. When an image needs to be recognized, this face is projected onto
the eigenface space first and gets a new descriptor. The new descriptor is used as
network input and applied to each persons network. The neural net with maximum
output is selected and reported as the host if it passes predefined recognition threshold.
The eigenface approach used in this scheme has advantages over other face recognition
methods in its speed, simplicity, learning capability and robustness to small changes in
the face image.

2.3

NEURAL NETWORKS

An artificial neural network is a non linear and adaptive mathematical module

inspired by the working of a human brain. It consists of simple neuron elements
operating in parallel and communicating with each other through weighted
interconnections.

11

2.3.1 Model of neuron

A neuron is an information-processing unit that is fundamental to the operation of a

neural network. In this case of artificial neural networks, the strength of the connection

between an input and a neuron is defined as the value of the weight. Negative weight
values correspond to inhibitory connections, while positive values correspond to
excitatory connections. The adder sums up all the inputs modified by their respective
weights. Finally, a transfer function controls the amplitude of the output of the neuron.
An acceptable range of output is usually between 0 and 1, or -1 and 1 depending on the
transfer function selected. Figure 2.1 shows a typical model of an artificial neuron.

Figure 2.1: A typical model of an artificial neuron