1

Department of Electronics and Communication Engineering

School of LSTS
Lovely Professional University, Phagwara
August to December, 2011


CAPSTONE PROJECT REPORT

(Project Term August-December, 2011)



(FACE RECOGNITION)

Submitted by

Manpreet Singh(10808169)
Programme :-B-tech(ECE)-M-tech
Section :-E28M1


Under the Guidance of Mr. Sonit Singh



2


DECLARATION


I hereby declare that the project work entitled Face Recognition is an authentic
record of my own work carried out as requirements of Capstone Project (Part-I) for the
award of degree of B.Tech Electronics and Communication Engineering ( B.tech - M-
tech-ECE ) from Lovely Professional University, Phagwara, under the guidance of Mr.
Sonit singh , during August to December, 2011.



Name of Students:-

Manpreet Singh (10808169)
Date: 26-nov-2011





















3


CERTIFICATE


This is to certify that the above statement made by the student is correct to the
best of my knowledge and belief. The Capstone Project Proposal based on the technology
tool learnt is fit for the submission and partial fulfillment of the conditions for the award
of B.Tech(ECE) from Lovely Professional University, Phagwara.





Name : ..

U.ID :

Designation : .












4





ACKNOWLEDGEMENT

First and foremost, we thank the Almighty for enabling us to complete this project
successfully. Infrastructure and an academically stimulating atmosphere go a long way
in modeling professionals and nurturing their caliber. In this regard, we express to our
mentor Mr. Sonit Singh. We extend our sincere thanks to all the teaching and non-
teaching staff of our Department and our family members and friends for their selfless
help and support.









5



INDEX

1. INTRODUCTION OF PROJECT.6
2. Face Recognition Applications..7
3. WORKING OF PROJECT7
3.1 Method.7
3.2 Flow diagram..12
3.3 Construction.13

4.CODING16

4.1 Code Description
4.2 MATLAB code
5. APPROACHES FOR FACE RECOGNITION.22
6. Technology used26
6.1 ABOUT EIGEN VECTORS
6.2 PCA
7. Differences Between Algorithms.32
8. Recent Developments..39

9. Future Work..39

10. References.40




6


1. Introduction
In this project we will make the program to identify the face or any image using
MATLAB. In this there will be photographs in our database. Software will check the
photo graph we want to check is in database or not. If not then it will display image is not
in database. If yes then it will display , image is available in database. Previous
mathematicians have theorized that certain cognitive processes, such as face recognition,
can be emulated through the use of principal component analysis. we have attempted to
use techniques of principal component analysis, more specifically, eigen-vector analysis,
to develop a computer program capable of face recognition. Specifically, the goal of our
project was to investigate a mathematical basis and model for face recognition using
principal component analysis with eigenvectors, and then implement a working model in
the form of a computer program.
The fundamental idea behind principal component analysis with eigenvectors has its
basis in linear algebra. Put simply, if there are a series of multi-dimensional vectors
representing objects which have similarities, it is possible to use a transformation matrix
and eigenvectors to orient a space which requires fewer dimensions to accurately describe
these multidimensional vectors. For instance, if in three dimensional space, there was a
cloud of particles that lied in a two dimensional plane skewed from the axes (Fig 1), it
would be possible to orient a new space with a new origin and new unit vectors such that
the cloud which previously required a three dimensional representation could now easily
be represented in only two dimensions.

7

Figure 1: Series of objects in 3D space which can be represented by 2D vectors
The new unit vectors of the lower dimensional space, which now sufficiently describes
where the points are located, can be derived by normalizing the eigenvectors of the
transformation matrix used to create the new origin. Lower eigenvalues, that is
eigenvalues approaching zero, correspond to unit vectors in dimensions which dont have
to be represented. Consequently, the vectors corresponding to these eigenvlaues can be
disregarded thus reducing the dimensions of the space even further.
2. Face Recognition Applications
In the past, much research has been performed in the field of face recognition. This problem
has received attention from people in many different disciplines such as medicine,
engineering, computer vision, artificial intelligence and databases. One reason why
researchers have spent time in this field is that there has been a high demand from security
services, such as police, army and customs, for a system that is able to recognize people.
Applications for such recognition include databases of criminals and wanted or missing
persons. Since these databases are typically very large, it is impossible for any user to
memorize every face. Therefore, it is necessary to have machines aid users in the recognition
process.Another motivation for research in the face recognition field is the authentication
problem. These days, most people in the western world have several access codes and credit
cards with pin numbers, as well as login names for computer systems. In a security survey
done by Price Waterhouse Coopers in the United Kingdom, they discovered that the average
employee in British companies must remember three different usernames and passwords to
be able to do his or her job [DTI06]. Personal access codes are not included in these figures.
Although some of these codes are necessary for security reasons, people tend to forget them,
which leads to extra work or cost in getting a new one. In order not to forget these codes
people may compromise security, e.g. by writing their pin numbers on their credit cards or
using the same code for everything. As a result, the security that was supposed to stem from
all these numbers is lost. To counter the lack of security associated with pin numbers, face
recognition has been proposed as a means to authenticate persons. The advantages are that no
numbers of authentication units are needed, only the face.
8



3. WORKING OF PROJECT
3.1 Method
The task of having computers capable of recognizing faces, without using principal
component analysis, is a formidable one. To a computer, a face, like any other image, is
a matrix of several hundred pixels by several hundred pixels. Dealing with many faces ,
in the form of pictures, can be very time consuming and difficult. If, however, one
applies principal component analysis, the task becomes much more manageable. If the
pictures pixel matrix is turned into a vector, then that picture will have a location
vector in a hundred thousand dimensional space. If the same is done with other
pictures, their location vectors will wind up in the same area of this huge multi-
dimensional space. We call this small subsection of the multi-dimensional space the
"face space." Then using eigenvectors to create a new basis for the representing the
images, one can now represent images as coordinates in a much smaller dimensional
space. Images of the same person, will, of course, be located nearer to one another in
this space, as they are similar to one another. At this level, the task of recognition
becomes merely a matter of determining the location of the vector form of a new
picture in a lower dimensional space relative to pictures of recognized faces.

Initialization with Training Set

In order to apply the ideas of ideas of principal component analysis to face recognition,
we had to take care of preliminaries. Firstly, the faces being discussed are nine 512*352
pixel 8-bit TIFF images, consisting of three separate images for three separate people,
which we converted into vectors of 180,224 dimensions. Our first task was to find an
average face,this was easily done by taking the arithmetic mean of the face vector.
9


Figure 2: The "average face" of our nine sample faces
We then constructed a matrix A, which consisted of the differences between each face
and the average face. Then, by creating a covariance matrix C = A * A, we can easily
find the eigenvectors of C, which by proof [Turk and Pentland, p.6], will span the "face
space." The eigenvectors whose corresponding eigenvalues are less than one can be
discarded. Then, if the eigen vectors are normalized (ie. divided by their length), they will
now, also by proof [ibid], compose the basis of this lesser dimensional space. Also, they
will be orthogonal and orthonormal. Now, instead of being a 180,224 byte matrix, the
images we used became eight numerical co-ordinates in our new "face space"!





10

Recognizing Faces
Once faces can be represented as coordinate vectors in a low dimensional space,
the task of face recognition itself is rendered trivial. First we look at the placement of
different vectors representing different pictures of the same person. We average these
vectors to find the "center" image of this person and then calculate a "radius" from this
center to the different images. New pictures are broken down into lower dimensional
vectors using simular techniques as used to establish the coordinates of the test faces.
Essentially, we make a vector and multiply by the eigenvectors to determine coordinates.
Now, face recognition simply becomes the mathematical calculation of distance, to see
whether or not a new face falls into a "sphere of recognition" of a previously recognized
face.
Results
Using the Algorithm for Face Recognition
We did phase1. This took a few hours to run on a Silicon Graphics, which was
running "some" other processes at this time. Here are three of the nine images; theses are
of the same person:

figure 3: Three of the nine images in the training set

11

To recognize an image, we wrote a new function...
Here is another image of that person. After running for some minutes, our program
identified him as being the same as the three above.


Figure 4: A new image that our program identified well
Using the Algorithm for Image Compression
The idea that faces can be represented with only a few "coordinates" and the principal
components, which are only stored once, inspired us to use this method for image
compression. In fact, [Turk, Pentland, p.5] refer to [Sirovich, Kirby] and state that this
algorithm was originally designed for compression!
Here are the results of three experiments that we performed with different images:
1. We took a completely new face of a person that was not in the training set, calculated
its (in our example) eight weights describing the position of the face in the "face space"
and rebuilt the image using the linear combination of the eight principle components.
What comes out is the following:
As one can see, the result is very poor. It has to be said that this could probably be
improved by using more than eight principle components (eigenfaces). [Turk, Pentland]
speaks of using about 40 eigenfaces.
12

2. In another decomposing/rebuilding experiment, we did the same with an unknown face
of a person of whom we have had three different images in the training set. The following
figure shows that the result is much better in this case:
3. As a third experiment, we decomposed/rebuilt the image of a face that was itself part
of the initial training set. As one can imagine, the quality of this image was perfect; no
difference from the original picture could be seen.
Suitability of MATLAB
We were using MATLAB for the realization of our project. Though we had no previous
knowledge and experience with this computer algebra system, we found it very easy to
learn and useful for experimenting. Especially calculations invoking matrices and vectors
are very simple. Still, it has to be stated that we cannot think of using MATLAB in a real
application, say for a security system, because it is far too slow. We would use a C++
program for this purpose; the matrix calculations could be simplified and optimized by
using an industry standard math library such as Matrix. h++ from RogueWave Software
Inc.
Efficiency of the Algorithm & Suitability for Industry Use
Recognizing faces could theoretically (see previous chapter concerning MATLAB
or C++) be very fast, because it consists only of a matrix subtraction, one matrix
multiplication and a bit of comparison. On the other hand, training new faces is a
comparably complicated calculation, especially if the "face space" has to be recalculated
as soon as new persons are added. We would therefore say that this method could be
useful in a serious industry application only to recognize a group of people that does not
often change after initial training.







13

3.2 Flow diagram












































Insert data base
image
Give path
Training set
Normalized the
training set
Taking mean
image
Getting eigen
faces
Giving input
image
14

3.3 Construction
This section gives step-by-step instructions along with photos and formulas on how to
recognize faces and implemented into Matlab. All the necessary files to complete this
tutorial would be provided.
Steps
1. The first step is to obtain a set S with M face images. In our example M = 25 as
shown at the beginning of the tutorial. Each image is transformed into a vector of
size N and placed into the set.
{ }
M
S I I I I = , ......... , , ,
3 2 1

2. After you have obtained your set, you will obtain the mean image



3. Then you will find the difference between the input image and the mean image
+ I = u
i i

4. Next we seek a set of M orthonormal vectors, u
n
, which best describes the
distribution of the data. The k
th
vector, u
k
, is chosen such that

=
I = +
M
n
n
M
1
1
15

( )

=
u =
M
n
n
T
k k
u
M
1
2 1

is a maximum, subject to
otherwise
if

0
1 k l
u u
lk k
T
l
=

= =o

Note: u
k
and
k
are the eigenvectors and eigenvalues of the covariance matrix C

5. We obtain the covariance matrix C in the following manner

T
M
n
T
n n
AA
M
C

1
1
=
u u =

=

} { , ....... , , ,
3 2 1 n
A u u u u =

6. A
T

n
T
m mn
L u u =

7. Once we have found the eigenvectors, v
l
, u
l

M l v u
M
k
k lk l
, ...... , 1
1
= u =

=

16


These are the eigenfaces of our set of original images

Recognition Procedure
1. A new face is transformed into its eigenface components. First we compare our
input image with our mean image and multiply their difference with each
eigenvector of the L matrix. Each value would represent a weight and would be
saved on a vector .

( ) + I =
T
k k
u e

| |
M
T
e e e , ....... , ,
2 1
= O

2. We now determine which face class provides the best description for the input
image. This is done by minimizing the Euclidean distance

2
k k
O O = c


17


3. The input face is considered to belong to a class if
k
is bellow an established
threshold

. Then the face image is considered to be a known face. If the

difference is above the given threshold, but bellow a second threshold, the image
can be determined as a unknown face. If the input image is above these two
thresholds, the image is determined NOT to be a face.
4. If the image is found to be an unknown face, you could decide whether or not you
want to add the image to your training set for future recognitions. You would
have to repeat steps 1 trough 7 to incorporate this new face image.
4. CODING
4.1 Code Description
This algorithm works in the following manner: First, it obtains several images from the
training set. The code was been tested using bmp and jpg images. After loading these
images, we find the mean face and perform several calculation (refer to code and
tutorial). Next, we ask for the name of an image we want to recognize (note: the image
must have the same size as the training images). The default path of your input image is
your desktop. You could change it by modifying the code on line 159. We then project
the input image into the eigen space, and based on the difference from the eigen faces we
decide.








18

4.2 MATLAB CODE

% Source code of Face recognition system
clear all % To clear all the variable
close all % To close all the windows
clc
% number of images on your training set.
M=13;
%Chosen std and mean.
%It can be any number that it is close to the std and mean of most of the images.
um=100;
ustd=80;
%read and show images(bmp);
S=[]; %img matrix
figure(1);
for i=1:M
str=strcat(int2str(i),'.jpg'); %concatenates two strings that form the name of the image
eval('img=imread(str);');
subplot(ceil(sqrt(M)),ceil(sqrt(M)),i)
imshow(img)
if i==3
title('Training set','fontsize',18)
end
drawnow;
[irow icol]=size(img); % get the number of rows (N1) and columns (N2)
temp=reshape(img',irow*icol,1); %creates a (N1*N2)x1 matrix
S=[S temp]; %X is a N1*N2xM matrix after finishing the sequence
%this is our S
end


19

%Here we change the mean and std of all images. We normalize all images.
%This is done to reduce the error due to lighting conditions.
for i=1:size(S,2)
temp=double(S(:,i));
m=mean(temp);
st=std(temp);
S(:,i)=(temp-m)*ustd/st+um;
end
%show normalized images
figure(2);
for i=1:M
str=strcat(int2str(i),'.jpg');
img=reshape(S(:,i),icol,irow);
img=img';
eval('imwrite(img,str)');
subplot(ceil(sqrt(M)),ceil(sqrt(M)),i)
imshow(img)
drawnow;
if i==3
title('Normalized Training Set','fontsize',18)
end
end
%mean image;
m=mean(S,2); %obtains the mean of each row instead of each column
tmimg=uint8(m); %converts to unsigned 8-bit integer. Values range from 0 to
255
img=reshape(tmimg,icol,irow); %takes the N1*N2x1 vector and creates a N2xN1
matrix
img=img'; %creates a N1xN2 matrix by transposing the image.
figure(3);
imshow(img);
20

title('Mean Image','fontsize',18)
% Change image for manipulation
dbx=[]; % A matrix
for i=1:M
temp=double(S(:,i));
dbx=[dbx temp];
end
%Covariance matrix C=A'A, L=AA'
A=dbx';
L=A*A';
% vv are the eigenvector for L
% dd are the eigenvalue for both L=dbx'*dbx and C=dbx*dbx';
[vv dd]=eig(L);
% Sort and eliminate those whose eigenvalue is zero
v=[];
d=[];
for i=1:size(vv,2)
if(dd(i,i)>1e-4)
v=[v vv(:,i)];
d=[d dd(i,i)];
end
end
%sort, will return an ascending sequence
[B index]=sort(d);
ind=zeros(size(index));
dtemp=zeros(size(index));
vtemp=zeros(size(v));
len=length(index);
for i=1:len
dtemp(i)=B(len+1-i);
ind(i)=len+1-index(i);
21

vtemp(:,ind(i))=v(:,i);
end
d=dtemp;
v=vtemp;
%Normalization of eigenvectors
for i=1:size(v,2) %access each column
kk=v(:,i);
temp=sqrt(sum(kk.^2));
v(:,i)=v(:,i)./temp;
end
%Eigenvectors of C matrix
u=[];
for i=1:size(v,2)
temp=sqrt(d(i));
u=[u (dbx*v(:,i))./temp];
end
%Normalization of eigenvectors
for i=1:size(u,2)
kk=u(:,i);
temp=sqrt(sum(kk.^2));
u(:,i)=u(:,i)./temp;
end
% show eigenfaces;
figure(4);
for i=1:size(u,2)
img=reshape(u(:,i),icol,irow);
img=img';
img=histeq(img,255);
subplot(ceil(sqrt(M)),ceil(sqrt(M)),i)
imshow(img)
drawnow;
22

if i==3
title('Eigenfaces','fontsize',18)
end
end
% Find the weight of each face in the training set.
omega = [];
for h=1:size(dbx,2)
WW=[];
for i=1:size(u,2)
t = u(:,i)';
WeightOfImage = dot(t,dbx(:,h)');
WW = [WW; WeightOfImage];
end
omega = [omega WW];
end
% Acquire new image
% Note: the input image must have a bmp or jpg extension.
% It should have the same size as the ones in your training set.
% It should be placed on your desktop
InputImage = input('Please enter the name of the image and its extension \n','s');
InputImage = imread(strcat('E:\MATLAB6p5\work\',InputImage));
figure(5)
subplot(1,2,1)
imshow(InputImage); colormap('gray');title('Input image','fontsize',18)
InImage=reshape(double(InputImage)',irow*icol,1);
temp=InImage;
me=mean(temp);
st=std(temp);
temp=(temp-me)*ustd/st+um;
NormImage = temp;
Difference = temp-m;
23

p = [];
aa=size(u,2);
for i = 1:aa
pare = dot(NormImage,u(:,i));
p = [p; pare];
end
ReshapedImage = m + u(:,1:aa)*p; %m is the mean image, u is the eigenvector
ReshapedImage = reshape(ReshapedImage,icol,irow);
ReshapedImage = ReshapedImage';
%show the reconstructed image.
subplot(1,2,2)
imagesc(ReshapedImage); colormap('gray');
title('Reconstructed image','fontsize',18)
InImWeight = [];
for i=1:size(u,2)
t = u(:,i)';
WeightOfInputImage = dot(t,Difference');
InImWeight = [InImWeight; WeightOfInputImage];
end
ll = 1:M;
figure(6)
subplot(1,2,1)
stem(ll,InImWeight)
title('Weight of Input Face','fontsize',14)
% Find Euclidean distance
e=[];
for i=1:size(omega,2)
q = omega(:,i);
DiffWeight = InImWeight-q;
mag = norm(DiffWeight);
e = [e mag];
24

end
kk = 1:size(e,2);
subplot(1,2,2)
stem(kk,e)
title('Eucledian distance of input image','fontsize',14)
MaximumValue=max(e)
MinimumValue=min(e)
if (MinimumValue <= 1.8180e+004)
disp('Input image exist in the present Database')
else
disp('This Photo does not belong to this Database ')




















25

5. APPROACHES FOR FACE RECOGNITION

Input image normalization

Image normalization is the first stage for all face recognition systems. Firstly face area is
detected in the image. We used template matching to localize a face. Then the eye (iris)
centers should be detected because the distance between them is used as a normalization
factor.

We located the eyes in facial images of different size using the luminance component.We
required the eyes must be open in input images. The gray-scale image was processed by
an edge detector (we used Sobel). Then it was binarized by Otsu method. We calculated
the vertical integral projection of the binary image and smoothed the projection profile by
averaging of neighboring values. The profile was searched for a large valley with an
intervening high peak. The peak indicates to the eye area in the image (see the right part
of Fig.1). To locate eyes we applied Hough transform to a small strip of the binary image
(the shaded area in Fig.1) using a half circle mask as often the upper part of the iris is
covered by eyelid. To speed up the processing, we store several masks corresponding to
different radii and then use for Hough Transform. Among several peaks in the Hough
space we find two highest scoring candidates. The crosses in Fig.1 show the centers of
eyes and some details of their location. The output of this step is the coordinates of the
eyes. In the next step we transformed the initial images by rotation, scaling and cropping
of the central face part. We tried to remove background and hair, and keep the most
invariant (in time) part of a face. The most important fiducial features are situated around
eyes, eyebrows, and nose. We have observed that the nose shape varies a lot as a
consequence of head rotation and lighting conditions. Finally mouth is the most variant
part of the face.


26


Fig. 1. Original face image and located eye centers.

We can rely on the fact that face expression in document photos is usually minimal, at
least discreet, while for a camera control image the test person can be requested to
restrain from excessive expressions. Additional features from the original image can be
gleaned in the form of a gray-scale edge map using Deriche algorithm. Examples of the
input images are given in the top of Fig.2.




Fig.2. Original (upper row) and normalized images (lower row), respectively.

Then we apply the ranking transform to the edge map and change the value of every map
element by the rank of the edge value corresponding to the element. Histogram of the
27

edge map gives ranking of all edge values. Outcome of the normalization step is the rank
map of the cropped original rotated and scaled image (see Fig.2).

Geometric approach to face recognition

The first historical way to recognize people was based on face geometry. There are a lot
of geometric features based on the points. We experimentally selected 37 points (Fig.3).
Geometric features may be generated by segments, perimeters and areas of some figures
formed by the points. It includes 15 segments between the points and the mean values of
15 symmetrical segment pairs. We tested different subsets of the features to looking for
the most important features.

The Olivetti Research Laboratory face database was used in our experiments. Each photo
is an image of size of 92112 pixels, and quantized to 256 gray levels. We tested 70
images of 12 persons. Images of two persons were added from our image database. They
were done with a huge time difference (from 1 to 30 years). We have selected 28
features. In spite of small rotation, orientation and illumination variances, the algorithm
works in a fairly robust manner. Distances in the feature space from a template image to
every image in the database were calculated. Following to the FERET protocol, 5 nearest
face images were derived and if there were photos of the query person then the result was
considered positive. Each image was tested as a query and compared with others. Just in
one case of 70 tests there were no any image of the person in the query through the 5
nearest ones, i.e. the recognition rate was 98.5%. The approach is robust, but it main
problem is automatic point location. Some problems arise if image is of bad quality or
several points are covered by hair.



28



Fig. 3. Some facial points and distances between them are used in face recognition.

The previous approach evaluates the point-based features only. Applying some
elastic transform we can change geometry and image texture and then compare two
images. Given the variability of a face, even under controlled conditions it is futile to try
to compare directly original images or even their feature maps. In fact, we have got very
low similarity scores with the popular method of mosaic images, and using linear
correlation. With the rank transformation differences of luminance and contrast become
compensated for. Furthermore to compensate for the local changes due facial expression
and pose, we apply a nonlinear warping of one rank map vis--vis another map. Among
the many algorithms to warp images we used a technique similar to that described in . It
turns out that comparison of one warped map with another nonwarped one yields much
better results than a direct comparison of the maps. However, the computational cost of
warping is high, as we must test a large set of randomly generated local deformations. A
practical way to maneuver in the warping parameter space consists in checking after
every warping the score against the previously achieved level of similarity. We continue
to warp the first map if the present score is higher than the previous one; otherwise we
start warping with other parameters. Note that the dissimilarity measure we used to
compare two maps was the rank correlation coefficient. The smaller this score, the more
similar are the two rank maps and, hence, the persons presented in the images. In fact, it
equals 0 if the compared maps are identical, equals to 1 if they are very different, and
29

assumes the value 2 if they are reverse images. When the score falls below predefined
threshold T, then one can conclude that the two images delineate the same person.

We experimented with a set of 100 pairs of passport and camera images of 20 persons.
The camera images were captured under various illumination conditions. The better
results in our experiments were obtained when we scale original images doing the
interocular distance about 40 to 50 pixels. This also sets a lower bound on the resolution
level. The recognition rate in the mentioned experiment was 92.5%. The pair of the most
difficult for identification images in our database is presented in Fig 2. Notice the very
different quality and contrast of the camera images and the passport photos (see also
Fig.4). Run time of our non-optimized experimental procedure was 10-12 seconds on a
Pentium PC (up to 800-1000 warping iterations).The system remains to be compared
with its competitors, but we believe that due to the inherent nonlinearity in the rank
correlation it should be more robust vis--vis scheme based on linear correlation



Fig. 4. Image (a) is transformed to (b) to be similar to (c).

Neural Networks for Access Control

Face Recognition is a widespread technology used for Access Control. The task
is stated as follows. There is a group of authorized people, which a recognition system
must accept. All the other people are unauthorized or aliens and should be rejected.We
can train a system to recognize the small group of people that is why application of a
Multilayer Perceptron (MLP) Neural Network (NN) was studied for this task.
Configuration of our MLP was chosen by experiments. It contains three layers. Input for
NN is a grayscale image. Number of input units is equal to the number of pixels in the
image. Number of hidden units was 30. Number of output units is equal to the number of
persons to be recognized. Every output unit is associated with one person. NN is trained
to respond +1 on output unit, corresponding to recognized person and -1 on other
30

outputs. We called this perfect output. After training highest output of NN indicates
recognized person for test image. Most of these experiments were passed on ORL face
database. Any input image was previously normalized by angle, size, position and
lightning conditions. We also studied another image representations: a set of discrete
cosine transform coefficients and a gradient map. Using DCT first coefficients we reduce
the sample size and significantly speedup the training process. DCT representations
allows us to process JPEG and MPEG compressed images almost without
decompression. A gradient map allows to achieve partial invariance to lightning
conditions. In our experiments with NNs we studied several subjects. We explored
thresholding rules allowing us to accept or reject decisions of NN. We introduced a
thresholding rule, which allow improving recognition performance by considering all
outputs of NN. We called this sqr rule. It calculates the Euclidean distance between
perfect and real output for recognized person. When this distance is greater then the
threshold we reject this person. Otherwise we accept this person.

6. Technology used
6.1 ABOUT EIGEN VECTORS
Eigenvalue, eigenvector and eigenspace
In mathematics, a vector may be thought of as an arrow. It has a length, called its
magnitude, and it points in some particular direction. A linear transformation may be
considered to operate on a vector to change it, usually changing both its magnitude and
its direction. An eigenvector (help info) of a given linear transformation is a vector which
is multiplied by a constant called the eigenvalue (info) as a result of that transformation.
The direction of the eigenvector is either unchanged by that transformation (for positive
eigenvalues) or reversed (for negative eigenvalues). For example, an eigenvalue of +2
means that the eigenvector is doubled in length and points in the same direction. An eigen
value of +1 means that the eigen vector is unchanged, while an eigen value of 1 means
that the eigenvector is reversed in direction. An eigenspace of a given transformation is
the span of the eigenvectors of that transformation with the same eigenvalue, together
with the zero vector (which has no direction). An eigen space is an example of a subspace
of a vector space.
31

In linear algebra, every linear transformation between finite-dimensional vector spaces
can be given by a matrix, which is a rectangular array of numbers arranged in rows and
columns. Standard methods for finding eigenvalues, eigenvectors, and eigenspaces of a
given matrix are discussed below. These concepts play a major role in several branches
of both pure and applied mathematics appearing prominently in linear algebra,
functional analysis, and to a lesser extent in nonlinear mathematics. Many kinds of
mathematical objects can be treated as vectors: functions, harmonic modes, quantum
states, and frequencies, for example. In these cases, the concept of direction loses its
ordinary meaning, and is given an abstract definition. Even so, if this abstract direction is
unchanged by a given linear transformation, the prefix "eigen" is used, as in
eigenfunction, eigenmode, eigenstate, and eigenfrequency.
Definitions: the eigen value equation
Linear transformations of a vector space, such as rotation, reflection, stretching,
compression, shear or any combination of these, may be visualized by the effect they
produce on vectors. In other words, they are vector functions. More formally, in a vector
space L a vector function A is defined if for each vector x of L there corresponds a
unique vector y = A(x) of L. For the sake of brevity, the parentheses around the vector on
which the transformation is acting are often omitted. A vector function A is linear if it has
the following two properties:
additive
Homogeneity
where x and y are any two vectors of the vector space L and is any real number.
[10]

Such a function is variously called a linear transformation, linear operator, or linear
endomorphism on the space L.
The key equation in this definition is the eigenvalue equation, Ax = x. Most vectors x
will not satisfy such an equation. A typical vector x changes direction when acted on by
A, so that Ax is not a multiple of x. This means that only certain special vectors x are
eigenvectors, and only certain special numbers are eigenvalues. Of course, if A is a
32

multiple of the identity matrix, then no vector changes direction, and all non-zero vectors
are eigenvectors. But in the usual case, eigenvectors are few and far between. They are
the "normal modes" of the system, and they act independently.
[12]

The requirement that the eigenvector be non-zero is imposed because the equation A0 =
0 holds for every A and every . Since the equation is always trivially true, it is not an
interesting case. In contrast, an eigenvalue can be zero in a nontrivial way. An eigenvalue
can be, and usually is, also a complex number. In the definition given above, eigenvectors
and eigenvalues do not occur independently. Instead, each eigenvector is associated with
a specific eigenvalue. For this reason, an eigenvector x and a corresponding eigenvalue
are often referred to as an eigenpair. One eigenvalue can be associated with several or
even with infinite number of eigenvectors. But conversely, if an eigenvector is given, the
associated eigenvalue for this eigenvector is unique. Indeed, from the equality Ax = x =
'x and from x 0 it follows that = '. Geometrically, the eigenvalue equation means
that under the transformation A eigenvectors experience only changes in magnitude and
sign the direction of Ax is the same as that of x. This type of linear transformation is
defined as homothety (dilatation, similarity transformation). The eigenvalue is simply
the amount of "stretch" or "shrink" to which a vector is subjected when transformed by
A. If = 1, the vector remains unchanged (unaffected by the transformation). A
transformation I under which a vector x remains unchanged, Ix = x, is defined as identity
transformation. If = 1, the vector flips to the opposite direction (rotates to 180); this is
defined as reflection.

If x is an eigenvector of the linear transformation A with eigenvalue , then any vector
y = x is also an eigenvector of A with the same eigenvalue. From the homogeneity of
the transformation A it follows that Ay = (Ax) = (x) = (x) = y. Similarly, using the
additivity property of the linear transformation, it can be shown that any linear
combination of eigenvectors with eigenvalue has the same eigenvalue . Therefore, any
non-zero vector in the line through x and the zero vector is an eigenvector with the same
eigenvalue as x. Together with the zero vector, those eigenvectors form a subspace of the
33

vector space called an eigenspace. The eigenvectors corresponding to different
eigenvalues are linearly independent meaning, in particular, that in an n-dimensional
space the linear transformation A cannot have more than n eigenvectors with different
eigenvalues. The vectors of the eigenspace generate a linear subspace of L
n
which is
invariant (unchanged) under this transformation.
If a basis is defined in vector space L
n
, all vectors can be expressed in terms of
components. Polar vectors can be represented as one-column matrices with n rows where
n is the space dimensionality. Linear transformations can be represented with square
matrices; to each linear transformation A of L
n
corresponds a square matrix of rank n.
Conversely, to each square matrix of rank n corresponds a linear transformation of L
n
at a
given basis. Because of the additivity and homogeneity of the linear trasformation and the
eigenvalue equation (which is also a linear transformation homothety), those vector
functions can be expressed in matrix form. Thus, in a the two-dimensional vector space
L
2
fitted with standard basis, the eigenvector equation for a linear transformation A can
be written in the following matrix representation where the just a position of matrices
means matrix multiplication. This is equivalent to a set of n linear equations, where n is
the number of basis vectors in the basis set. In these equations both the eigenvalue and
the components of x are unknown variables.
The eigenvectors of A as defined above are also called right eigenvectors because they
are column vectors that stand on the right side of the matrix A in the eigenvalue equation.
If there exists a transposed matrix A
T
that satifies the eigenvalue equation, that is, if A
T
x
= x, then x
T
= (x)
T
= (A
T
x)
T
= x
T
A, or x
T
A = x
T
. The last equation is similar to the
eigenvalue equation but instead of the column vector x it contains its transposed vector,
the row vector x
T
, which stands on the left side of the matrix A. The eigenvectors that
satisfy the eigenvalue equation x
T
A = x
T
are called left eigenvectors. They are row
vectors. In many common applications, only right eigenvectors need to be considered.
Hence the unqualified term "eigenvector" can be understood to refer to a right
eigenvector. Eigenvalue equations, written in terms of right or left eigenvectors (Ax = x
and x
T
A = x
T
) have the same eigenvalue .
34

An eigenvector is defined to be a principal or dominant eigenvector if it corresponds to
the eigenvalue of largest magnitude (for real numbers, largest absolute value). Repeated
application of a linear transformation to an arbitrary vector results in a vector
proportional (collinear) to the principal eigenvector. The applicability of the eigenvalue
equation to general matrix theory extends the use of eigenvectors and eigenvalues to all
matrices, and thus greatly extends the scope of use of these mathematical constructs not
only to transformations in linear vector spaces but to all fields of science that use
matrices: linear equations systems, optimization, vector and tensor calculus, all fields of
physics that use matrix quantities, particularly quantum physics, relativity, and
electrodynamics, as well as many engineering applications


6.2 TECHNIQUE PCA- PRINCIPAL COMPONENT ANALYSIS
Principal component analysis (PCA) is a technique used to reduce multidimensional data
sets to lower dimensions for analysis. Depending on the field of application, it is also
named the discrete Karhunen-Love transform, the Hotelling transform or proper
orthogonal decomposition (POD).
PCA is mostly used as a tool in exploratory data analysis and for making predictive
models. PCA involves the calculation of the eigenvalue decomposition of a data
covariance matrix or singular value decomposition of a data matrix, usually after mean
centering the data for each attribute. The results of a PCA are usually discussed in terms
of component scores and loadings.
Details
PCA is mathematically defined as an orthogonal linear transformation that transforms the
data to a new coordinate system such that the greatest variance by any projection of the
data comes to lie on the first coordinate (called the first principal component), the second
greatest variance on the second coordinate, and so on. PCA is theoretically the optimum
transform for a given data in least square terms.
35

PCA can be used for dimensionality reduction in a data set by retaining those
characteristics of the data set that contribute most to its variance, by keeping lower-order
principal components and ignoring higher-order ones. Such low-order components often
contain the "most important" aspects of the data. However, depending on the application
this may not always be the case.PCA has the distinction of being the optimal linear
transformation for keeping the subspace that has largest variance. This advantage,
however, comes at the price of greater computational requirement if compared, for
example, to the discrete cosine transform.
Properties and Limitations of PCA
PCA is theoretically the optimal linear scheme, in terms of least mean square error, for
compressing a set of high dimensional vectors into a set of lower dimensional vectors and
then reconstructing the original set. It is a non-parametric analysis and the answer is
unique and independent of any hypothesis about data probability distribution. However,
the latter two properties are regarded as weakness as well as strength, in that being non-
parametric, no prior knowledge can be incorporated and that PCA compressions often
incur loss of information.
The applicability of PCA is limited by the assumptions made in its derivation. These
assumptions are:
Assumption on Linearity
We assumed the observed data set to be linear combinations of certain basis. Non-linear
methods such as kernel PCA have been developed without assuming linearity.
Assumption on the statistical importance of mean and covariance
PCA uses the eigenvectors of the covariance matrix and it only finds the independent
axes of the data under the Gaussian assumption. For non-Gaussian or multi-modal
Gaussian data, PCA simply de-correlates the axes. When PCA is used for clustering, its
main limitation is that it does not account for class separability since it makes no use of
36

the class label of the feature vector. There is no guarantee that the directions of maximum
variance will contain good features for discrimination.
Assumption that large variances have important dynamics
PCA simply performs a coordinate rotation that aligns the transformed axes with the
directions of maximum variance. It is only when we believe that the observed data has a
high signal-to-noise ratio that the principal components with larger variance correspond
to interesting dynamics and lower ones correspond to noise.
Essentially, PCA involves only rotation and scaling. The above assumptions are made in
order to simplify the algebraic computation on the data set. Some other methods have
been developed without one or more of these assumptions; these are briefly described
below.
Computing PCA using the Covariance Method
Following is a detailed description of PCA using the covariance method. The goal is to
transform a given data set X of dimension M to an alternative data set Y of smaller
dimension L. Equivalently, we are seeking to find the matrix Y, where Y is the
Karhunen-Loeve transform (KLT) of matrix X:
Organize the data set
Suppose you have data comprising a set of observations of M variables, and you want to
reduce the data so that each observation can be described with only L variables, L < M.
Suppose further, that the data are arranged as a set of N data vectors with each
representing a single grouped observation of the M variables. Write as column vectors,
each of which has M rows. Place the column vectors into a single matrix X of dimensions
M N.


37

Calculate the empirical mean
Find the empirical mean along each dimension m = 1...M.
Place the calculated mean values into an empirical mean vector u of dimensions M 1
Calculate the deviations from the mean
Subtract the empirical mean vector u from each column of the data matrix X.
Store mean-subtracted data in the M N matrix B.
where h is a 1 x N row vector of all 1's:
Find the covariance matrix
Find the M M empirical covariance matrix C from the outer product of matrix B with
itself where is the expected value operator, is the outer product operator, and is the
conjugate transpose operator. Note that if B consists entirely of real numbers, which is
the case in many applications, the "conjugate transpose" is the same as the regular
transpose.
Find the eigenvectors and eigenvalues of the covariance matrix
Compute the matrix V of eigenvectors which diagonalizes the covariance matrix
C: where D is the diagonal matrix of eigenvalues of C. This step will typically involve
the use of a computer-based algorithm for computing eigenvectors and eigenvalues.
These algorithms are readily available as sub-components of most matrix algebra
systems, such as MATLAB, Mathematica, SciPy, or IDL(Interactive Data Language).
See, for example, the eig function. Matrix D will take the form of an M M diagonal
matrix, where is the mth eigenvalue of the covariance matrix C, and Matrix V, also of
dimension M M, contains M column vectors, each of length M, which represent the M
eigenvectors of the covariance matrix C. The eigenvalues and eigenvectors are ordered
and paired. The mth eigenvalue corresponds to the mth eigenvector

38

Differences Between Algorithms

Here we will compare two different eigenface algorithms. One was developed by Adam. I. Wilmer
and the other one is the code I have developed so far. The next set of images show the inputs and
outputs of Adam's code. This was done using a set of 350 images. The images and the code can be
obtained at the following address:

First we can see the mean face obtained from the training set of images.

Mean Face
The first image to be recognized was taken from the training set. We would compare this result with
the ones obtained using unknown images.

39


This image is a part of a bigger image which was cut by the program to perform the recognition. We
can see that the results are not very good since the Euclidean distance (shown at the right of the
image) is within the same limits as the one obtained from a known face.

The same image that was used above was resized to be the same size of the ones used in the training
set. We then run the code and obtained the following results. We can see that the Euclidean distance
increased in this case

Bellow are the results obtained using my code. These results were obtained using 50 images.
40

Our mean image is


Next we can see the training set followed by the normalized training set

41


The following graph shows the eigenfaces representing our set. For some reason they look too white.
They might need some normalization.

42

Results of Face Recognition
Given the following training set




Finally we obtained the input and output image. Currently we can not show the Euclidean distance,
but we will try to show it on future versions of the code.

43

The mean face is



The eigenfaces are shown bellow. They look darker than the ones in the tutorial since
they have not been normalized, or passed through filters


44


We then wanted to recognize the following image

It should be recognized as file 15.

Final Words
Although our algorithm works, it is necessary to normalize the images and use some
filtering techniques to obtain better results.

8. Recent Developments

One of the most recent methods that have shown up in the face recognition literature
is a method based on blood vessels. Researchers have discovered that the skin is slightly
warmer around blood vessels, so using pictures of people taken with infrared cameras or
heat cameras, the blood vessels graph of the face appears. Methods for fingerprint
detection can then be used to find who the face belongs to. The Japanese company Fujitsu
has made ATM machines based on blood vessel graphs of hands and a few of these
machines have already been installed in Japanese banks Fujitsu has demonstrated that
this technique is very accurate and reliable, which indicates that it can be suitable for face
recognition, since blood vessel graphs of faces should be at least as unique as graphs of
hands. Because this technique is new and we did not have access to the required
resources and equipment, we could not investigate it, but this method is very interesting
45



9. Future Work

There are many interesting problems that remain in the area of face recognition. One
problem is image preprocessing prior to the application of the eigen face method. It may
be possible to gain better accuracy in classification if one segments the spectrum of
people into different spaces. For example, if one was able to determine if an image was of
a man or a woman, one could use this categorization to send an image to one of two
classifiers, each specifically trained with that type of individual in mind. This would
mean that there would be a set of eigenfaces specifically formales and one specifically for
females (face spaces with gender, so to speak). Work in this area has been done by
Lizama, Waldoestl and Nickolay, however it would be interesting to extend it to use
eigenfaces to act as the gender classifier as well. A general face-space would be created
in addition to the male and female face-spaces, with the sole purpose of being used to
classify an image as male or female.
Another area of future work is improving our neural network classifier. As
mentioned in the previously, it is possible to construct the network to take its input
directly from the image data rather from the vector that results from an images projection
into face-space. Perhaps learning the face projection function could increase the accuracy
of the neural network classifier. Additionally, more experiments are needed to see if there
are other ways to tweak the network














46

10. REFERENCES
[1] Matthew Turk, Alex Pentland: Eigenfaces for Recognition; Vision and Modeling
Group, The Media Laboratory, Massachusetts Institute of Technology; September 1990.
[2] L. Sirovich and M. Kirby: Low-dimensional procedure for the characterization of
human faces; J. Opt. Soc. Am. A, Vol. 4, No. 3, March 1987, 519-524.
[3] M. Kirby and L. Sirovich: Application of the Karhunen-Loeve procedure for the
characterization of human faces; IEEE Trans. Pattern Analysis and Machine Intelligence,
Vo 12, No. 1, Jan. 1990.
[4] www.face-rec.org/interesting-papers/General/ImAna4FacRcg_lu.pdf

[5] www.face-rec.org/interesting-papers/Other/FaceRecognition.pdf

[6] nform.nu/Articles/Vol3/v3n1p01-07.pdf