Digital Image Processing

ADIP
Advance Digital Image Processing.
Advance Digital Image Processing – UCP Fall 2015 Dr. Asim Baig
Lec
25
15.12.12
Applications of Image Processing
Slides taken from
• Dr. Arun Ross– West Virginia University
• Dr. Anil K. Jain – Michigan State University
BIOMETRICS – A DEEPER LOOK
WHAT TO EXPECT TODAY….
• We will take a brief look at different biometric
modalities and how they are used in a
recognition system.
• We will look at
• Fingerprint
• Iris
• Face
• We will also look at Multimodal Biometrics if

we have time…
Lets Start at the Beginning....
FINGERPRINT RECOGNITION
Fingerprints
• What are fingerprints….
• A fingerprint is an INDIVIDUAL CHARACTERISTIC no two are alike

and no two have yet been found to possess identical ridge
characteristics.
• Fingerprints are the product of your genes as well as the environment of the womb
during development in the first trimester (nutrition, blood pressure, contact with the
womb)
• Once developed, fingerprints do not change over the course of your life, although
they may be altered or scarred
• They are comprised of a series of lines corresponding to hills (ridges) and valleys
(grooves).
Identical twins have the same DNA configuration but they do not
have identical friction ridge configuration
History
Prehistoric
Picture writing of a hand with ridge patterns was discovered in
Nova Scotia. In ancient Babylon, fingerprints were used on clay tablets
for business transactions. In ancient China, thumb prints were found
on clay seals.
1880
Dr. Faulds published an article in the Scientific Journal, "Nautre"
(nature). He discussed fingerprints as a means of personal
identification, and the use of printers ink as a method for obtaining
such fingerprints. He is also credited with the first fingerprint
identification of a greasy fingerprint left on an alcohol bottle.
History
1888
Sir Francis Galton - (Father of contemporary fingerprinting)
• Sir Francis Galton, a British anthropologist and a cousin of Charles
Darwin, began his observations of fingerprints as a means of
identification in the 1880's.
• In 1892, he published his book, "Fingerprints", establishing the
individuality and permanence of fingerprints. The book included the
first classification system for fingerprints.
• Galton's primary interest in fingerprints was as an aid in determining
heredity and racial background. While he soon discovered that
fingerprints offered no firm clues to an individual's intelligence or
genetic history, he was able to scientifically prove that fingerprints do
not change over the course of an individual's lifetime, and that no two
fingerprints are exactly the same.
• According to his calculations, the odds of two individual fingerprints
being the same were 1 in 64 billion.
• Galton identified the characteristics by which fingerprints can be
identified. These same characteristics (minutia) are basically still in use
today, and are often referred to as Galton's Details
Types of Fingerprints
Fingerprint type can be identified based on the following 9 different
features
Types of Prints
FINGERPRINTS HAVE GENERAL RIDGE PATTERNS FOR
CLASSIFICATION:
Divided into three classes:

LOOP
60-65% OF THE POPULATION HAS LOOPS
WHORL
30-35% WHORLS
ARCH
AND 5% ARCHES
Minutiae
They are the major features of the fingerprint utilized in comparison
of one print with other
• The type, number and relative location of minutiae impart

individuality.
• For comparison, the minutiae should be in the same relative

location to one another.
http://cnx.rice.edu/content/m12574/latest/match.jpg
Type of Minutiae
Minutiae Detection
Generally, following steps are involved in Minutiae Detection:
• Enhancement / Pre-processing
• The input image is pre-processed to improve the details
and remove the noise
• Binarization
• The enhanced image is converted from greyscale image
to binary image
• Thinning
• The ridges in the binarized image are reduced to single
pixel thickness.
• Minutiae Detection
• The minutiae (bifurcations and terminations) are
detected in the thinned image.
Enhancement
High boost filtering is utilized for enhancement.
• Enhances the high frequency components without eliminating low

frequency details
Noise removal approach using Gaussian filter is also provided for

comparison
Enhancement
Binarization
• Local adaptive binarization technique presented by NiBlack was
utilized where separate threshold is set of each block.
• Global Threshold based binarization technique (Otsu) was also

implemented for comparison
Thinning
A simple thinning algorithm is applied to reduce the binarized ridges to
single pixel width
Minutiae Detection
Minutiae are detected in the thinned image by following the observing
the 8 connected pixel for each point on the thinned ridge. Spurious
Minutiae are removed
Minutiae Detection
Detected Minutiae
Fingerprint Recognition
(Overview)
Template Feature Results
Matcher
Image Input Extractor
Feature
Database
Possible Results:
X Y θ T
X Y θ T
X Y θ T
• Genuine Match 128 246 30 B
245 416 60 T
128 246 30 B
128
Enhance 246 30 B • Genuine Non-Match .
Extract . . .
Binarize . . . . 245 416 60 T

Image
245 416 60 T • False Accept Features
. . . .
. . . .
• False Reject
. . . .
. . . .
The Stable one…
IRIS RECOGNITION
Overview
• The Iris as a Biometrics: The iris is an overt body that is
available for remote assessment with the aid of a machine
vision system to do automated iris recognition.
• Iris recognition technology combines computer vision,
pattern recognition, statistical inference, and optics.
• The spatial patterns that are apparent in the human iris are
highly distinctive to an individual.
– Clinical observations
– Developmental biology
Overview
The structure
Theofstructure
the iris seen
of the
in ahuman
transverse
eye section
The structure of the iris seen in a frontal section
Overview
• Its suitability as an exceptionally accurate biometric derives

from its
– extremely data-rich physical structure
– genetic independence — no two eyes are the same
– patterns apparently stable throughout life
– physical protection by a transparent window (the cornea),
highly protected by internal organ of the eye
– externally visible, so noninvasive — patterns imaged from a
distance
Overview
• The disadvantages to use iris as a biometric measurement

are
– Small target (1 cm) to acquire from a distance (about 1 m)
– Moving target
– Located behind a curved, wet, reflecting surface
– Obscured by eyelashes, lenses, reflections
– Partially occluded by eyelids, often drooping
– Deforms non-elastically as pupil changes size
– Illumination should not be visible or bright
Outline
• Technical Issues **
– Image Acquisition
– Iris Localization
– Pattern Matching
• Systems and Performance
** (Throughout this discussion two different standard systems will be

discussed, namely the iris-recognition systems of Daugman and Wildes et al.)
Technical Issues
Schematic diagram of iris recognition
I. Image Acquisition
• Why important?
One of the major challenges of automated iris recognition is
to capture a high-quality image of the iris while remaining
noninvasive to the human operator.
• Concerns on the image acquisition rigs
– Obtained images with sufficient resolution and sharpness
– Good contrast in the interior iris pattern with proper
illumination
– Well centered without unduly constraining the operator
– Artifacts eliminated as much as possible
I. Image Acquisition - Rigs
• The Daugman image-acquisition rig
I. Image Acquisition - Rigs
• The Wildes et al. image-acquisition rig
I. Image Acquisition - Results
Result Image from Wildes et al. rig -- capture the iris

as part of a larger image that also contains data derived
from the immediately surrounding eye region
Discussion
• In common:
– Easy for a human operator to master
– Use video rate capture
• Difference:
– Illumination
• The Daugman’s system makes use of an LED-based point light source in
conjunction with a standard video camera.
• The Wildes et al. system makes use of a diffuse source and polarization
in conjunction with a low-light level camera.
– Operator self-position
• The Daugman’s system provides the operator with live video feedback
• The Wildes et al. system provides a reticle to aid the operator in
positioning
II. Iris Localization
• Purpose: to localize that portion of the acquired image that

corresponds to an iris
• In particular, it is necessary to localize that portion of the
image derived from inside the limbus (the border between
the sclera and the iris) and outside the pupil.
• Desired characteristics of iris localization:
– Sensitive to a wide range of edge contrast
– Robust to irregular borders
– Capable of dealing with variable occlusions
• The Daugman system fits the circular contours via gradient

ascent on the parameters so as to maximize
Where is a radial Gaussian,

and circular contours (for the limbic and pupillary boundaries)
be parameterized by center location (xc,yc), and radius r
(active contour fitting method)
• The Wildes et al. system performs its contour fitting in two
steps. (histogram-based approach)
– First, the image intensity information is converted into a binary edge-
map
where
and
– Second, the edge points vote to instantiate particular contour
parameter values.
• The voting procedure of the Wildes et al. system is realized via
Hough transforms on parametric definitions of the iris
boundary contours.
Illustrative Results of Iris Localization
only that portion of the image below the upper eyelid and
above the lower eyelid should be included
Obtained by using the Wildes et al. system

Discussion
 Both approaches are likely to encounter difficulties if
required to deal with images that contain broader regions
of the surrounding face than the immediate eye region
 Difference:
 the active contour approach avoids the inevitable
thresholding involved in generating a binary edge-map
 the histogram-based approach to model fitting should avoid
problems with local minima that the active contour model’s
gradient descent procedure might experience
III. Pattern Matching
• Four steps:
1) bringing the newly acquired iris pattern into spatial alignment
with a candidate data base entry;
2) choosing a representation of the aligned iris patterns that
makes their distinctive patterns apparent;
3) evaluating the goodness of match between the newly
acquired and data base representations;
4) deciding if the newly acquired data and the data base entry
were derived from the same iris based on the goodness of
match.
III. Pattern Matching -Alignment
• Purpose: to establish a precise correspondence between

characteristic structures across the two images.
• Both of the systems under discussion compensate for image
shift, scaling, and rotation.
• For both systems, iris localization is charged with isolating
an iris in a larger acquired image and thereby accomplishes
alignment for image shift.
 The Daugman’s system uses radial scaling to compensate
for overall size as well as a simple model of pupil variation
based on linear stretching.
Map Cartesian image coordinates (x, y)
to dimensionless polar (r, ө)
image coordinates according to
• The Wildes et al. system uses an image-registration

technique to compensate for both scaling and rotation.
The mapping function (u,v) is to minimize
while being constrained to capture a similarity

transformation of image coordinates (x, y) to
(x’, y’)
 The two methods for establishing correspondences
between acquired and data base iris images seem to be
adequate for controlled assessment scenarios
 Improvements:
 more sophisticated methods may prove to be
necessary in more relaxed scenarios
 more complicated global geometric compensations will
be necessary if full perspective distortions (e.g.,
foreshortening) become significant
III. Pattern Matching - Representation
• The Daugman’s system uses demodulation with complex-

valued 2D Gabor wavelets to encode the phase sequence of
the iris pattern to an “IrisCode”.
• In implementation, the Gabor filtering is performed via a

relaxation algorithm, with quantization of the recovered
phase information yielding the final representation.
Pictorial Examples of one IrisCode

• The Wildes et al. system makes us of an isotropic bandpass

decomposition derived from application of Laplacian of
Gaussian filters to the image data.
with σ the standard deviation of the Gaussian and ρ the radial distance of a
point from the filter’s center
• In practice, the filtered image is realized as a Laplacian

pyramid. This representation is defined procedurally in terms
of a cascade of small Gaussian-like filters.
• Result: Multiscale representation for iris pattern matching.
Distinctive features of the iris are manifest across a range of
spatial scales.
Obtained by using the Wildes et al. system
IV. Pattern Matching – Goodness of Match
• The Daugman system computes the normalized Hamming

distance as
• The result of this computation is then used as the goodness of

match, with smaller values indicating better matches.
IV. Pattern Matching - Decision
• The Wildes et al. system employs normalized correlation
between the acquired and data base representations.
• For the Daugman system, this amounts to choosing a
separation point in the space of (normalized) Hamming
distances between iris representations.
• In order to calculate the cross-over point, sample populations
of imposters and authentics were each fit with parametrically
defined distributions.
• For the Wildes et al. system, the decision-making process
must combine the four goodness-of-match measurements
that are calculated by the previous stage of processing (i.e.,
one for each pass band in the Laplacian pyramid
representation) into a single accept/reject judgment.
Systems and Performance - The
Daugman iris-recognition system
• Both the enrollment and verification modes take under 1s to complete.
 Empirical test 1: 592 irises from 323 persons  the system exhibited no
false accepts and no false rejects
 Empirical test 2:
– Phase1: 199 irises from 122 persons, 878 attempts in identification
mode over 8 days  no false accepts and 89 false rejects (47 retry
with still 16 rejected)
– Phase2: 96 irises (among 199) with 403 entries for identification  no
false accepts and no false rejects
Systems and Performance - The Wildes
et al. iris-recognition system
• Both the enrollment and verification modes require
approximately 10s to complete.
 Only one empirical test: 60 different irises with 10 images
each (5 at the beginning and 5 about one month later) from
40 persons  no false accepts and no false rejects.
Iris
Template Feature Matcher Results

Image Input Extractor
Feature
Database
Segmentation Noise Removal Polar Image and Noise Mask Iris Code and Noise Mask
The one to watch…
FACIAL RECOGNITION
Face Recognition
• Probably the most common biometric characteristic used by humans
• Face recognition appears to be a dedicated process of the brain
• Holistic and feature information are used in the recognition process
• Face memory is highly viewpoint-dependent
• Analysis of facial expressions is accomplished in parallel to face recognition
• Humans recognize people from their own race better than people from
another race
• Non-intrusive technique which people generally accept as a biometric
characteristic
• Dependent on imaging/devices
• Overt (user aware) and covert (user unaware) applications
• Subject of intensive research for over 25 years
Market Share
Scenarios of Face Recognition
Input Images
• Fully Constrained
– Input and Database images are acquired in a fully
controlled environment with similar lighting condition
and backgrounds using the same sensor.
– This scenario mostly occurs in lab based research
• Semi Constrained
Fully Constrained
– The database consists of fully constrained images but
the input image are acquired under somewhat relaxed
conditions such that either the lighting is changed or
image has some pose variation.
– This scenario occurs in access control applications
where the user has to pass through a pre-defined
location.
Database
• Unconstrained Semi Constrained
– The input images are acquired in any lighting
condition with any kind of pose and with different
acquisition sensor.
– This scenario occurs is systems deployed at public
places where it is nearly impossible to control the user
pose or lighting conditions
Unconstrained
Fully Constrained
Data Acquisition Constraints
ISO/IEC JTC 1/SC 37 N185
Width Distance from Eye to Eye

(Pixels) (Inclusive)
240 (min) 60
480 120
720 180
Fully Constrained
Image Acquisition Process
ISO/IEC JTC 1/SC 37 N185
Current Trends in Facial Recognition
• CCTV system at London Borough • Pennsylvania Justice Network

of Newham, in UK searches crime scene CCTV footage
in mugshot database for previous
• Fully automated border control at arrests.
Frankfurt Airport by using a facial
recognition system setup by the • US Department of State uses over
German Federal Police 75 million photographs for visa
processing
• Australian Customs Service uses a
similar automated border processing • At Super Bowl XXXV, police used
system called SmartGate to search for potential criminals.
• US Department of Motor Vehicle • Utilized at Presidential elections in

uses face recognition to help prevent Mexico in 2000 to prevent voter
identity thefts fraud.
Challenges
• Highly non-convex and nonlinear distribution of faces vs. non faces and
between faces of two different individuals.
(a) Face versus non-face manifolds. (b) Face manifolds of different individuals.
Challenges
• High Interclass similarity
www.marykateandashley.com news.bbc.co.uk/hi/english/in_depth/americas/2000/us_election
s
Twins Father and son
Challenges
• High Intra-class variability
• It occurs due to
• Lighting variation
• Orientation variation (face angle) (In plane rotation)
• Size variation
• Acquisition Geometry
• Acquisition Sensor Quality
• Head Pose (Out of plane rotation)
• Expressions (Smile, Frown etc)
• Facial Accessories (Glasses, Makeup, Facial Hair, etc)
• Aging Effects
Challenges
• High Intra-class variability
Real Time Face Recognition
Useful for……
• Security
– Fight terrorism
– Find fugitives
• Personal information access
– ATM
– Sporting events
– Home access (no keys or passwords)
– Any other application that would want personal identification
• Improved human-machine interaction
• Personalized advertising
Must have features…..

• The system must be inexpensive enough to be used at many locations
• It must Match within seconds

– Before the person walks away from the advertisement
– Before the fugitive has a chance to run away
• It must have the ability to handle a large database (millions of people)
• It must have the ability to do recognition in varying environments (day, night etc.)
Additional challenges…..
• Storage Issues
– They work on very large databases therefore require a large amount of storage space that can
allow quick access and retrieval
• Processing Issues
– These systems are supposed to compare millions/billions of face against an input in order to
evaluate a single user.
• Timing Issues
– These billions of comparison should be performed in seconds to provide real time results.
• Network and Communication Issues
– The user database will be stored at a central location, which means that input images and the
results will be transmitted over the network and as such would require proper network speed,
reliability and security.
Possible Usage Scenarios
• Verification
– The user input his employee
number
– The system acquires the facial
image
– The image is matched against
the users image in database
(one – on – one matching)
– The system outputs decision
• User is Genuine
• User is Imposter
• Identification
– An image of the user is
acquired when he enters gate
at the premises
– The image is compared with
every image in the database
– The system outputs
• Top n best matches
• Confidence in each match
• Identification
– The system detects movement
in no go areas
– The video is processed to
acquire facial image
– Image is compared against a
standard database such as
NADRA database to identify
the person
• Identification
– The system detects arrival of a
vehicle at an entry gate
– The image is processed to
acquire facial image and
vehicle’s number plate.
– Facial Image is compared
against a employee database
to identify the person
– Number Plate image is
compared against a database
to identify the vehicle
• Forensic
– Video of an event is evaluated
after the fact.
– The culprit is marked in the
video
– Facial recognition against
standard database is
performed
Possible Scenarios
Systems development can be categorized into two major areas
Face Recognition Process
• Face recognition is a “Visual Pattern Recognition” problem.
• A face is a three-dimensional object subject to varying illumination, pose,

expression is to be identified based on its two-dimensional image ( or three-
dimensional images obtained by laser scan).
• A face recognition system generally consists of 4 modules divided into two

section
– Pre Processing Consists of

Detection and Alignment,
– Facial Recognition consists of
Feature Extraction and Matching.
Real time Face Recognition System
Overview
Complete system consists of…..
• Pre-Processing Section
– Face Detection Module
Must be able to detect frontal and nonfrontal faces with high accuracy
– Face Alignment Module
Faces must be aligned and centered for proper matching
– Face Image Normalization Module
Face images are normalized for lighting and shadows
• Face Recognition Section

– Feature Extraction
• Global Feature Detection
Global feature detection evaluate the face shape, texture and color e.g. using graph based methods
• Local Feature Detection
Local features of the face are detected for matching
– Matching
• A robust approach to matching should be developed to provide best possible result
Pre processing Steps

• Face Detection
• Face Tracking (optional – For Video).
• Face Alignment
• Normalized based on
– Geometrical Properties, such as size and pose
– Photometrical Properties such as illumination and gray scale.
Face Detection
• Face Detection is the first step in automated face recognition.
• Face Detection can be performed based on several cues:

– Skin Color
– Motion
– Facial/Head Shape
– Facial Appearance “or”
– A combination of these parameters
Face Detection: Challenges
• Out-of-Plane Rotation: frontal, 45 degree, profile

• Presence of beard, mustache, glasses etc
• Facial Expressions
• Occlusions by long hair, hand
• In-Plane Rotation
• Image conditions:
– Size
– Lighting condition
– Distortion
– Noise
– Compression
Face Detection – Different Approaches

• Knowledge-based methods:
– Encode what constitutes a typical face, e.g., the relationship between facial features
• Feature invariant approaches:
– Aim to find structure features of a face that exist even when pose, viewpoint or lighting
conditions vary
• Template matching:
– Several standard patterns stored to describe the face as a whole or the facial features separately
• Appearance-based methods:
– The models are learned from a set of training images that capture the representative variability
of faces.
Appearance based approaches are the most successful in face detection
Face Detection
• Appearance based and Learning based approaches.
– Face detection is treated as a two class classification problem (face vs. nonface)
– Appearance based methods avoid difficulties in modeling 3D structure of face by considering
possible face appearances under various conditions.
• Disadvantages
– Large variations brought about by change in facial appearance, lighting and expression can make
the face distribution and face/nonface boundaries highly complex.
– These approaches are highly dependant on the face size in training data and can only detect faces
of the same size as available in training data.
• Some of the learning based approaches are

– Neural Network based Methods
– Kernel SVM Classifiers
– AdaBoost based Methods
Face Detection
• Conclusions
– AdaBoost based approaches have so far provided the best face detection solution in terms of the
statistical rates and the speed.
– Extended Haar Like Features allow for non-frontal facial recognition and can provide scale
invariance.
– Extensive evaluation is required of different variations of AdaBoost algorithms such as
FloatBoost, GentleBoost, LogitBoost and DiscreteBoost etc. to evaluate which would provide
the best possible results for face detection.
• It is important to note that once properly explored these approaches can also be
utilized in other projects which require object detection.
Face Recognition Methods
Face Recognition
• Main Idea
– Reduce a facial image containing thousands of pixels before making comparisons
One of the most commonly used techniques is the “Eigenfaces” based technique.
Which constitutes of steps presented in next slide. These approaches are
considered useful because the matching process is very fast.
Face Recognition Methods
• Eigenfaces Initialization
– Acquire an initial set of face images (the training set)
– Calculate the eigenfaces from the training set, keeping only the M images that correspond to the
highest eigenvalues. These M images define the face space. As new faces are experienced, the
eigenfaces can be updated or recalculated
– Calculate the corresponding distribution in M-dimensional weight space for each known
individual, by projecting their face images onto the “face space.”
• Eigenfaces Recognition
– Calculate a set of weights based on the input image and the M eigenfaces by projecting the input
image onto each of the eigenfaces.
– Determine if the image is a face at all by checking to see if the image is sufficiently close to
“face space.”
– If it is a face, classify the weight pattern as either a known person or as unknown.
– (Optional) Update the eigenfaces and/or weight patterns.
Eigenfaces for Real time Face Recognition
Not suitable for Real Time Face Recognition …..

• Accuracy drop due to Scaling
– The head size in the input image should be similar to the head size in the database otherwise
the accuracy drops significantly
• Accuracy drop due to variation in orientation
– Even a smallest variation in orientation (in plane or out of plane rotation) of the input image
can cause this approach to fail.
• Accuracy drop due to lighting conditions
– The lightning conditions for the acquisition of input image should be similar to the lightning
conditions setup during database acquisition
• Possible Suppression of Important Details
– The images are projected into a low dimension space by removing the smallest variance
direction, it is possible that these details are important to distinguish interclass differences.
Categorization of Feature Extraction Techniques
Approach Representative Work

Appearance Based Methods
Eigenfaces Direct Application of PCA
Probabilistic Eigenfaces Two-class Problem with Prob. Measure
Fisherfaces FLD on Eigenfaces
SVM Two-class Problem based on SVM
Evolution Pursuit Enhanced GA Learning
Feature Lines Based on Point to Line distance
ICA ICA based feature analysis
Kernel Faces Kernel methods
Feature Based Methods
Pure Geometry Methods Mixture distance based approach
Dynamic Link Architecture Graph Matching Methods
Hidden Markov Model HMM methods
Hybrid Methods
Modular Eigenfaces Eigenfaces and Eigen Modules
Hybrid LFA Local Feature based Methods
Shape Normalization Flexible appearance models
Component based Face regions and components
Gabor Filter Based Approach
The Gabor filter (Gabor Wavelet)

• It represents a band-pass linear filter whose impulse response is defined by
a harmonic function multiplied by a Gaussian function.
• A bi-dimensional Gabor filter constitutes a complex sinusoidal plane of

particular frequency and orientation modulated by a Gaussian envelope.
• It achieves an optimal resolution in both spatial and frequency domains.
• Besides face recognition, Gabor filters are successfully used in many other
image processing and analysis domains, such as
– image smoothing, image coding, texture analysis, shape analysis, edge detection,
fingerprint and iris recognition.
• The processing of facial images by a Gabor filter has been widely used for
its biological relevance and technical properties.
• The Gabor filter kernels have similar shapes as the receptive fields of
simple cells in the primary visual cortex.
• They are multi-scale and multi-orientation kernels.
• The Gabor transformed face images yield features that display scale,
locality and differentiation properties. These properties are quite robust to
variability of face image formation, such as the variations of illumination,
head rotation and facial expressions.
• The two dimensional Gabor Wavelet function g(x,y) and its Fourier
Transform G(u,v) can be defined as follows:
• where and
• Gabor functions can form a complete but non-orthogonal basis set.

Expanding a signal using this basis provides localized frequency
description
• Let g(x, y) be the mother Gabor wavelet, then this self-similar filter
dictionary can be obtained by appropriate dilations and rotations of g(x, y)
through the generating function
• where
– a>1
– m, n = integers
– θ = nπ/k orientations with k = number of orientations.
– a-m is scale factor
• The non-orthogonality of the Gabor wavelets implies that there is

redundant information in the filtered images.
• To reduce this redundancy, let Ul and Uh denote the lower and upper centre
frequencies of interest. Let K be the number of orientations and S be the
number of scales in the multi-resolution decomposition. Then the following
formulas is used for computing the filter parameters σμ and συ (and thus σx
and σy )
where
W = Uh
m = 0,1, … S-1
Gabor filter Based Approach
• Given image I(x,y) its Gabor wavelet transformation is
• Calculating Gabor wavelet transformation of the facial image for five (5)
scales and Eight (8) orientations will yield following forty (40) filtered
images
Gabor filter Based Approach
• The augmented Gabor Face vector can then be defined as
• This vector is quite large in size and for a big database containing hundreds
or thousands of enrolled users it can be huge.
• To avoid this issue the vector size can be reduced by using just mean and
standard deviation as discriminating features of each Gabor wavelet
transform. Then the vector becomes.
X = ((m0,0, σ0,0)t, …….., (mS,K, σS,K)t)t
Conclusions
• Face Recognition is a mature technology
– Offers several advantages over more “traditional” biometric technologies
• Face Recognition is one of the most appealing biometric modality

– And the most challenging, sometimes beyond expectations
• Several Successful working system exists

– They still have several drawbacks and limitations
References
• M. Turk and A. Pentland. Eigenfaces for Recognition. Journal of Cognitive Neuroscience, 3(1), 1991
• C. Nastar and M. Mitschke. Real-Time Face Recognition Using Feature Combination. In Proceedings of the Third
IEEE International Conference on Automatic Face and Gesture Recognition, Nara, Japan, April 1998
• J. Gilbert and W. Yang. A Real-Time Face Recognition System using Custom VLSI Hardware. Harvard
Undergraduate Honors Thesis in Computer Science, 1993.
• Juwei Lu et. al. Face Recognition using LDA-Based Algorithms. IEEE Transactions of Neural Networks. Vol. 14, no.
1, January 2003.
• Peter Belhumeur et. Al. Eigenfaces vs Fisherfaces: Recognition Using Class Specific Linear Projection. Vol 19. no 7.
July 1997.
• Evaluation of Face Recognition Algorithms. Colorado State University.
http://www.cs.colostate.edu/evalfacerec/data/normalization.html
• S. Z. Li and A. K. Jain. Handbook of Face recognition, 2005
• T. Kanade. Picture processing by computer complex and recognition of human faces. Ph.D. thesis, 1973
• K. Fukunaga. Introduction to statistical pattern recognition. Academic Press, 1990
• M. Bichsel and A. P. Pentland. Human face recognition sand the face image set’s topology. CVGIP, 1994
• M. Turk. A random walk through eigenspace. IEICE Tans. Inf. & Syst., 2001
• Peter N. Belhumeur, Joao P. Hespanha, and David J. Kriegman. Eigenfaces vs. Fisherfaces: Recognition. Using
Class Specific Linear Projection, IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997
• E. Hjelmas and B. K. Low Face detection: A survey. Computer Vision and Image Understanding., 2001
• Henrik Wann Jensen, “Digital Face Cloning”, SIGGRAPH 2003 Technical Sketch, San Diego, July 2003
Last but not the Least… Lets talk about
MULTIMODAL BIOMETRICS
Multimodal Biometrics
• Multimodal Biometrics can be defined as:
The approach of using more than

one physical or physiological
biometric traits to recognize a user.
Conventionally this is achieved by merging ‘or’

fusing together the result of multiple unimodal
biometric identification systems
Designing a Multimodal Biometric System
The design process of a multimodal biometric system requires

making decision regarding the following issues:
• Types of Biometrics
Decision is based on number of Biometrics and algorithms used
• Level of Fusion
Decision provides the level at which fusion takes place
• Type of Fusion
Decision is based on the available systems and type of application
Types of Biometrics
Type Description
Single Biometric Trait Based Systems
Multi Samples Single Algorithm Useful in eliminating error due to environmental

Samples obtained by same or different or sensor noise
sources
Single Sample Multi Algorithms Useful in identification system with large

A weak and a Strong classification algorithms are databases
used
Multi Samples Multi Algorithms Useful in eliminating sensor errors in

Overcome the inherent problem of other identification system
approaches
Multiple Biometric Traits Based System
Multi Biometrics Single Algorithm Useful in identification and verification systems

Merges the biometric traits to provide single and it is the most flexible approach in terms of
search space fusion
Multi Biometrics Multi Algorithms Most commonly used multimodal approach to

Different Biometric is used with different both identification and verification
algorithms and the results are merged
Level of Fusion
Feature Level Fusion
Decision Level Fusion
Level of Fusion
Score/Rank Level Fusion
Types of Fusion
• Serial Fusion
•User provides One Biometric at a time
•Decision level Fusion takes place
•Useful when using COTS Biometric Systems.
• Parallel Fusion
•User provide both Biometrics at the same time
•Fusion can take place at any level.
•Strong classifiers are used
•Useful in verification Systems
• Hierarchical Fusion
•User provides both Biometrics at the same time
•Fusion can take place at any level
•A weak and strong classifier is used
•Useful in identification system using large databases
Challenges in Multimodal Biometrics
• Designing Challenges
•Types of Biometrics to Fuse
•Selection of Biometric Algorithms
•Best Fusion Method
•Implementation Constraints
»Processing Times
»Memory Constraints
»System Stability
• Testing & Analysis Challenges

•Availability of Standardized Datasets
•Baseline Performance Measures
Advance Digital Image Processing – UCP Fall 2015 * Rodney Brooks from his book FleshDr.
andAsim
Machines
Baig
Framework 1
Single Input – Single Matcher based Fusion Framework
Framework 1
Results
Identification / Detection Mode Results

FCV2006
Match False Accept False Reject Non Match
DB2_A
(1676 images)
Feature Extractor 1
(FE1)
1430 23 0 223
Feature Extractor 2
(FE2)
1612 17 0 47
Fused Score
(FE1 & FE2)
1675 0 0 1
Verification Mode Results

Proposed Algorithm Test Setup Results Provided in [3]
FE 1 FE 2 Fused % decrease Algo1 Algo2 Fused % decrease
ERR 18 24 13 27.77% 1.97 2.72 1.45 26.39%
Framework 2
Multimodality – Single Matcher based Fusion Framework
Framework 2
Results
Verification Mode Results
Raw Scores
Correct Incorrect
False Accept False Reject
Match Match
System 1 with Chain code based Minutiae Extractor

Fingerprint 60 15 15 10
Iris 65 13 13 9
Fused 72 9 9 10
System 2 with Binarization based Minutiae Extractor
Fingerprint 64 13 13 10
Iris 65 13 13 9
Fused 75 8 8 9
Comparison of ERR Improvement

ERR Fingerprint Iris Fused % improvement
Experimental
15 13 9 40.00%
System 1
Experimental
17 13 10 41.17%
System 2
Traditional
8 13 4 50.00%
System
Results in [10]
5.61 3.2 2.86 49.01%
For P075
Unimodal Fusion System
Conventional
Unimodal
Identification
Match
User
Score System
66
44
User 1
22 f(x) = Max (Match Score) Result = User 2
88
27
14
User 2
18
96
• Conventional Unimodal Systems do not utilize all the match score.
• Standard Averaging functions are biased by the extreme values

(maximum and minimum values).
• Ordered Weighted Average works because it can be used to

eliminate the biasing values.
• This approach is designed to increase the stability for simpler and

reasonably accurate systems.
• Not quite useful for best of breed solutions as in their case almost
always the max value is true solution.
• Can be used conditionally for best of breed solution i.e. only on

incorrect matches to improve performance.
• Ordered Weighted Average

»Symmetric aggregation functions that allocate weights
according to the values.
»Different from weighted means in that the weights are
associated not with a particular input but with their
magnitudes.
If x is the matching score vector for a certain user then let xd denote
the vector obtained from x by arranging its components in
descending order in terms of their values i.e. xd(1) ≥ xd(2) ≥ … ≥
xd(n) also let w denote a weight vector such that ∑wi = 1. Then the
Ordered Weighted Average can be given as:
M 
OWA
(
x)w
x
i iw
,x
d

i1
Ordered Weighted Averages
• Weights Calculations
»Calculated by using fuzzy linguistic qualifiers called Regular
Increasing Monotones (RIM) qualifiers.
Let Q denote the RIM qualifier then the ith weight for any n number of values can
be calculated from Q as follows:
i i
1
i
w  
Q Q 
n n
The value of Q can be calculated as follows:
Value of variable pair

 0, if ta
 (a,b) is set to (0.25,0.75)
t

a
Q
a t  if atb to generate Weights based

,b
b
 a on fuzzy set “most”.
 1, if tb
Proposed Unimodal
Score Fusion based
Identification System
Match
User
Score
66
44
User 1
22 f(x) = OWA (Match Score) Result = User 1
88
27
14
User 2
18
96 Weights = (0,0.5, 0.5, 0)
Results
Iris Dataset (WVU)

(Daugman’s Algorithm used for Detection)
Algorithm Threshold Match False Accept False Reject Non Match
Standard 69 25 19 19 37
Proposed 68.50 27 21 21 31
Fingerprint (FVC 2004 DB1_A)
(NIST algorithm for Feature Extraction and CUBS Matcher for Detection)
Standard 90 2 29 29 40
Proposed 13.50 68 6 7 19
Fingerprint (WVU)
(NIST algorithm for Feature Extraction and CUBS Matcher for Detection)
Standard 100 20 16 19 45
Proposed 66 70 8 8 14
Results – Error Rates
Error Rates for West Virginia University Fingerprint Dataset

Results – Error Rates
Error Rates for FVC 2004 (DB1_A) Fingerprint Dataset

Results – ROC Curve
ROC Curve for West Virginia University Fingerprint Dataset

Results – ROC Curve
True Positive Rate
False Positive Rate
ROC Curve for FVC 2004 (DB1_A) Fingerprint Dataset

Next
Lecture
Applications of Image Processing(contd...)

Digital Image Processing

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Digital Image Processing

Hochgeladen von

Copyright:

Verfügbare Formate

ADIP

Advance Digital Image Processing.

Applications of Image Processing

BIOMETRICS – A DEEPER LOOK

• We will also look at Multimodal Biometrics if

• A fingerprint is an INDIVIDUAL CHARACTERISTIC no two are alike

Divided into three classes:

• The type, number and relative location of minutiae impart

• For comparison, the minutiae should be in the same relative

• Enhances the high frequency components without eliminating low

Noise removal approach using Gaussian filter is also provided for

• Global Threshold based binarization technique (Otsu) was also

Binarize . . . . 245 416 60 T

• Its suitability as an exceptionally accurate biometric derives

• The disadvantages to use iris as a biometric measurement

** (Throughout this discussion two different standard systems will be

Schematic diagram of iris recognition

Result Image from Wildes et al. rig -- capture the iris

• Purpose: to localize that portion of the acquired image that

• The Daugman system fits the circular contours via gradient

Where is a radial Gaussian,

Obtained by using the Wildes et al. system

• Purpose: to establish a precise correspondence between

• The Wildes et al. system uses an image-registration

while being constrained to capture a similarity

• The Daugman’s system uses demodulation with complex-

• In implementation, the Gabor filtering is performed via a

Pictorial Examples of one IrisCode

• The Wildes et al. system makes us of an isotropic bandpass

• In practice, the filtered image is realized as a Laplacian

Obtained by using the Wildes et al. system

• The Daugman system computes the normalized Hamming

• The result of this computation is then used as the goodness of

Template Feature Matcher Results

Width Distance from Eye to Eye

• CCTV system at London Borough • Pennsylvania Justice Network

• US Department of Motor Vehicle • Utilized at Presidential elections in

Twins Father and son

Must have features…..

• It must Match within seconds

• It must have the ability to handle a large database (millions of people)

• Face recognition is a “Visual Pattern Recognition” problem.

• A face is a three-dimensional object subject to varying illumination, pose,

• A face recognition system generally consists of 4 modules divided into two

– Pre Processing Consists of

• Face Recognition Section

Pre processing Steps

• Face Tracking (optional – For Video).

• Face Detection can be performed based on several cues:

Face Detection: Challenges

• Out-of-Plane Rotation: frontal, 45 degree, profile

Face Detection – Different Approaches

Appearance based approaches are the most successful in face detection

• Some of the learning based approaches are

Not suitable for Real Time Face Recognition …..

Approach Representative Work

The Gabor filter (Gabor Wavelet)

• A bi-dimensional Gabor filter constitutes a complex sinusoidal plane of

• It achieves an optimal resolution in both spatial and frequency domains.

• They are multi-scale and multi-orientation kernels.

• Gabor functions can form a complete but non-orthogonal basis set.

• The non-orthogonality of the Gabor wavelets implies that there is

X = ((m0,0, σ0,0)t, …….., (mS,K, σS,K)t)t

• Face Recognition is one of the most appealing biometric modality