50YearsofBiometricResearch:

Almost
The Solved,TheUnsolved,andTheUnexplored
Anil Jain AnilJain
MichiganStateUniversity
June5,2013
1
KeynoteTalkDeliveredattheInternationalConf.onBiometrics,Madrid,Spain,June5,2013
50YearsofBiometricsResearch
Fingerprint
M.Trauring,OntheAutomaticComparisonofFingerRidgePatterns,
Nature,vol.197,pp.938940, 1963
V i Voice
S.Pruzansky,PatternMatchingProcedureforAutomaticTalker
Recognition,J.AcousticSocietyofAmerica,vol.35,pp.354358,1963
Face Face
W.W.Bledsoe,ManMachineFacialRecognition,Tech.Report
PRI22,PanoramicRes.Inc.,1966
T. Kanade, Picture Processing System by Computer Complex and T.Kanade, PictureProcessingSystembyComputerComplexand
RecognitionofHumanFaces,DoctoralDissertation,KyotoUniversity,1973
Handgeometry
R.H.Ernst,HandIDSystem,USPatentNo.3576537,1971 y
Iris
L.Flom andA.Safir,IrisRecognitionSystem,USPatent4641349A,1987
J.G.Daugman,HighConfidenceVisualRecognitionofPersonsbyaTestof
StatisticalIndependence,IEEETrans.PAMI,vol.15,pp.11481160,1993
1960s:BeginningsofresearchinAI,patternrecognition&imageprocessing
2
Over 1 billion people have been covered by biometric
IdentificationforDevelopment:TheBiometricsRevolution
Over1billionpeoplehavebeencoveredbybiometric
identificationprogramsintheLowMiddleIncomeCountries
*IdentificationforDevelopment:TheBiometricsRevolution,A.GelbandJ.Clark,CenterforGlobalDevelopment,NW,Washington
DC,WorkingPaper315,Jan.2013,http://www.cgdev.org/sites/default/files/1426862_file_Biometric_ID_for_Development.pdf
3
Objectives&Outline
Whatisbiometrics?
H did bi t i t t t d? Howdidbiometricsgetstarted?
Wherearewenow?
Wheredowegofromhere?
4
TheBiometricsConundrum
Demographics
Credentials
Sensordata
Transaction
Authentication
Privilege
User
Biometric Recognition Application
Matchscore
Confidence
BiometricRecognition Application
Recognizeapersonbybodytraits&linkthebodytoanexternallyassignedidentity
5
BiometricChallenge
Findarepresentation &similarity measure suchthat
Intrasubjectsimilarityisveryhigh
Intersubject similarity is very low Inter subjectsimilarityisverylow
Probe Gallery
MATCH MATCH
6
Similarityscore>Timpliesamatch
WhyBiometrics?
Security:Doesthepersonhaveapriorcriminalrecord?
Convenience:Noneedtocarrycredentials(pw,ID)
Audit trail: Who accessed the bank vault? Audittrail:Whoaccessedthebankvault?
Fraud:Isthecreditcardholdertherightfulowner?
Deduplication:Oneperson,onedocument!
PalmveinscannersusedforpatientregistrationinHoustonhospitalsystem;2,488
patientsarenamedMariaGarciaand231ofthemhavethesamebirthdate
7
http://www.usatoday.com/story/news/nation/2012/11/25/palmscannerstechnologyschools/1726175/
IndiasAadhaar Project
Basic demographic data and biometrics
Name
Parents
Gender
1568 3647 4958
Basic demographic data and biometrics
stored centrally
UID = 1568 3647 4958
10 fi i t 2 i i & f i
DoB
PoB
Address
10 fingerprints, 2 irises & face image
Central UID database
Togivethepooranidentity
~350 million unique ID numbers have already been issued
g p y
https://portal.uidai.gov.in/uidwebportal/dashboard.do
8
Personalization
Applications
V ifi i
Transactions
Healthcare
Verification
Convenience
Travel
Safety
9
Historical Perspective HistoricalPerspective
10
HabitualCriminalAct(1869)
What is wanted is a means of
classifying the records of habitual
criminal, such that as soon as the
particulars of the personality of any particulars of the personality of any
p r i s o n e r ( w h e t h e r
description, measurements, marks, o p , , ,
r photographs) are received, it may
be possible to ascertain readily, and
i h i h h hi i i with certainty, whether his case is in
the register, and if so, who he is. HabitualCriminalsReturnsBook
G.Pavlich,TheEmergenceofHabitualCriminalsin19thCenturyBritain:Implicationsfor
Criminology,JournalofTheoreticalandPhilosophicalCriminology,2(1),pp.159,2010
11
BertillonSystem(1882)
H.T.F.Rhodes,AlphonseBertillon:FatherofScientificDetection,Harrap,1956
12
Firstuseofsoftbiometricsandmultibiometrics
FrictionRidgePattern FrictionRidgePattern
Perhaps the most beautiful and
characteristic of all superficial marks
(on human body) are the small furrows (on human body) are the small furrows
with the intervening ridges and their
pores that are disposed in a singularly
complex yet even order on the under p y
surfaces of the hands and feet.
Francis Galton Nature June 28 1888 Francis Galton, Nature, June 28, 1888
ThreeMostPopularBiometricTraits
Legacydatabase Legacydatabase 1:Nsearch
1:Nsearch
NISTevaluation
Covertcapture
NISTevaluation
Highaccuracy
NISTevaluation
FingerprintRecognitionMilestones
1880
HenryFaulds
Articleonfingerprints
publishedinNature
1963
MitchellTrauring
Firstpaperonautomatic
fingerprintmatching
2004
DHS
USVISIT
2009
UIDAI
Indiastartedissuing
12digitUIDno.to
1901
ScotlandYard
AdoptedGalton/Henrysystemof
classification
AD600
China
Fingerprint to seal
1893
Argentina
First use of fingerprint
1924
FBI
Set up fingerprint
p g p g g
itsresidents
2008
FBI
NGI
1970s
FBI
Initiation of AFIS Fingerprinttoseal
contractsandlegal
documents
Firstuseoffingerprint
asforensicevidence
Setupfingerprint
identificationdivision
~10002000B.C. 1892
1990s
2005 1997
NGI InitiationofAFIS
AncientBabylon
Fingermarkonclayseal
JuanVucetich
Rolledinkonpaper
Livescantechnique
Opticalsensor
Solidstatesensor
Ultrasoundsensor
TBS
Touchless 3Dsensor
ThomsonCSF
Swipesensor
~1995
2003
USVISIT
http://www.alaskafingerprinting.com/Fingerprinting.html
http://commons.wikimedia.org/wiki/File:USVISIT_(CBP).jpg
http://www.biometricdirect.net/upekeikontouchfingerprintreadereikontouch700.html
http://www.thehindu.com/news/national/uidaitostartinformingaadharnumbersthroughemailssmses/article2624078.ece
http://www.idex.no/technology/swipesensor/
http://www.techneology.org/2009/10/touchless3dfingerprinting.html
15
SiemensIDmouse
FaceRecognitionMilestones
1991
Turk & Pentland
1964
Woodrow Bledsoe
1973
Takeo Kanade
2001
Viola & Jones
2006
Ahonen et
2009
Wright et al.
1997
Belhumeur et
1999
Blanz & Vetter Turk&Pentland
Eigenface
WoodrowBledsoe
Facerecognition
TakeoKanade
FirstFRthesis
Viola & Jones
Facedetector
Ahonen et
al.
LBP
Wrightetal.
Sparserep.
Belhumeur et
al.
Fisherface
Blanz &Vetter
Morphable face
1915
35mmstillcamera
1991
Kodak
Digitalcamera
1024p
2000
Sharp
Firstcameraphone
320p
20102013
Wearablecamera
480P@30fps
GoogleGlass
April2013
SamsungGalaxyS4
1080p@30fps
1990s
Surveillancecamera
480p@30fps
Bledsoe,W.W.1964.TheModelMethodinFacialRecognition,TRPRI15,PanoramicResearch,Inc.,California. http://photodoto.com/camerahistorytimeline/
M.TurkandA.Pentland, Eigenfaces forrecognition.JournalofCognitiveNeuroscience3(1):7186,1991.
TakeoKanade,PictureProcessingSystembyComputerComplexandRecognitionofHumanFaces,KyotoUniv.,1973. Viola,Jones:RobustRealtimeObjectDetection,IJCV2001.
Ahonen,et al.FaceDescriptionwithLocalBinaryPatterns:ApplicationtoFaceRecognition,PAMI, 2006.
J.Wrightetal. RobustFaceRecognitionviaSparseRepresentation,PAMI,312,2009.
http://static7.businessinsider.com/image/4d013ea7cadcbb7033010000/looxcievideocamera.jpg
http://www.techinasia.com/samsunggalaxys4infographic/
720p@30fps
Belhumeur,P.N.etal.,Eigenfaces vs.Fisherfaces:recognitionusingclassspecificlinearprojection,PAMI,197,1997.
V.Blanz andT.Vetter,Amorphable modelforthesynthesisof3Dfaces,SIGGRAPH1999.
http://commons.wikimedia.org/wiki/File:Three_Surveillance_cameras.jpg
IrisRecognitionMilestones
2002
USA
Useofirisrecognition
infieldoperations
2009
UIDAI
NationalID
2010
Mexico
NationalID
2011
Indonesia
NationalID
1936
FrankBurch
Conceptofusingiris
patternsforhuman
identification
1985
Flom andSafir
1991
JohnDaugman
2001
UAE
2005
Flom andSafir
2011
JohnDaugman
identification
Firstirisrecognition
patent
Irisrecognition
patent
1995
Deployedirisrecognition
systemforbordercontrol
Patentexpired Patentexpired
1989 2006 2004 2013 2013
IrisScanner System
Oneoftheearliest
commercialiriscameras
JohnDaugman
Firstiriscamera
Sarnoff
IrisontheMove
SecuriMetrics
Portableirisrecognition
device
AOptix
App&device
forsmartphone
tocaptureiris
DeltaEye
https://www.fbo.gov/index?s=opportunity&mode=form&tab=core&id=1d31acd78d7a20b1fe598bf4b4661d6b
http://www.sri.com/engage/productssolutions/iompassportportalsystem
http://www.cnn.com/2013/04/16/tech/mobile/mashablebiometriciphonescanner
17
OtherBiometricTraits
18
TraitswithLegacyDatabase
Traceevidence
Innocenceproject
RealtimeDNAmatching
~30%oflatents areofpalmprints
FBIsNGIwillhavelatentpalmprint
matchingcapability
Capabilitytoconduct1:Nsearch
19
g p y
TraitsinCommercialSystems
Primarilyfor1:1match
20
TraitsinLaboratoryStage
21
WhichBiometricTrait?
22
RequirementsofaBiometricTrait
Uniqueness (Isitdistinctiveacrossusers?)
Permanence (Does it change over time?) Permanence (Doesitchangeovertime?)
Universality (Doeseveryuserhaveit?)
Collectability (Canitbemeasuredquantitatively?)
Performance (Does it meet error rate, throughput..?) Performance (Doesitmeeterrorrate,throughput..?)
Userexperience(Isitacceptabletotheusers?)
Vulnerability(Canitbeeasilyspoofed?)
Integration (Canitbeembeddedintheapplication?) g ( pp )
23
Nobiometrictraitisoptimal,butmanyareadmissible
RejectedBiometricTraits
24
Where is Biometrics Now? WhereisBiometricsNow?
25
FingerprintMatching
Plaintoplainmatching
NIST FpVTE (2003): 99 4% TAR at 0 01% FAR NISTFpVTE (2003):99.4%TARat0.01%FAR
Latenttorolledmatching
NIST ELFT EFS II (2012): 63 4% rank 1 accuracy NISTELFTEFSII(2012):63.4%rank1accuracy
OnNISTSD27,stateoftheartrank1accuracy:~72%
NIST FpVTE NIST ELFTEFS NISTFpVTE NISTELFT EFS
C.Wilsonetal.,FingerprintVendorTechnologyEvaluation2003:SummaryofResultsandAnalysisReport,NISTIR7123,2004
M.Indovina etal.,EvaluationofLatentFingerprintTechnologies:ExtendedFeatureSets[Evaluation#2],NISTIR7859,2012
Matchingspeed:~millions/secfortenprint matchingonasingleserver
26
Face Recognition
FRVT 2012 TAR@FAR =0.1%
Mugshots 96%
Visa photos 99%
http://www.nist.gov/itl/iad/ig/frvt2012.cfm
Visa photos 99%
SearchTime(640kgalleryfaces):0.3seconds
MEDSII TAR@FAR=0.1%
Eigenfaces 9%
Fisherfaces 35%
LBP 34%
COTSA 58%
COTSB 88%
COTSC 97%
http://www.nist.gov/itl/iad/ig/sd32.cfm
IrisRecognition
Irisimagescollectedfrom4.3millioneyes(6.1Mimages)
Overthe95algorithmsevaluated,singleeyeirisfalse
negativeidentication rates(i.e.missrates)areat1.5%or
higher.Fortwoeyes,FNR=0.7%
No.offalsepositivesis25outofatotalof10
13
comparisons
Pupildilation& constrictioncan impactirisrecognition
Templatesizevaries between1KBand20KB
http://www.nist.gov/itl/iad/ig/irexiii.cfm
28
SpeakerRecognition
Givenatargetspeakerandatestspeechsegment,isthe
target speaking in the test segment?
NIST SRE12
targetspeakinginthetestsegment?
Atrialconsistsof:TargetSpeakerID+Testsegment
NIST SRE12
Systemstested 75
90
95
100
TestSegmentDuration 30300s
Number ofTrials
Target Speakers
~2million
2 897
75
80
85
TAR
(%)
TargetSpeakers
KnownNonTarget
UnknownNonTarget
2,897
46,601
61,871
70
75
1 2 3
0.01% 0.1% 1%
FAR
C.Greenbergetal.,The2012NISTSpeakerRecognitionEvaluation,NISTSRE12,December11122012,Orlando,Florida
http://www.nist.gov/itl/iad/mig/sre12results.cfm
29
FAR
FromSolvedtoUnsolved
t
r
a
i
n
e
d
54%TAR@
FAR=0.1%
72%Rank1
accuracy
66.8%TAR@
FAR=10%
U
n
c
o
n
s
t
FVC2004 MBGC CASIA v4 distance NIST SD27 LFW UBIRIS v2
C
o
n
d
i
t
i
o
n
s
FVC2004 MBGC CASIA.v4distance NISTSD27 LFW UBIRIS.v2
I
m
a
g
i
n
g

C
100%TAR
@FAR=0.1%
99.4%TAR
@FAR=0.01%
97.8%TAR
@FAR=0.01%
n
s
t
r
a
i
n
e
d
FRGC,Exp.1 IREXIII FpVTE 2003 FERET IIITDalcoholiciris
Userdistortedimage
Users
Cooperative
C
o
n
Uncooperative
30
FVC2006
UnsolvedProblems
Fundamentalproblems p
Uniqueness(individuality)
Permanence (persistence) Permanence(persistence)
Applicationdrivenproblems
Unconstrained sensing environment (surveillance) Unconstrainedsensingenvironment(surveillance)
Systemsecurity&userprivacy
Template security Templatesecurity
Antispoofing
31
Uniqueness
Givena10digitPIN,no.ofuniqueidentities
thatcanberesolved=10billion
But,whatcanwesayaboutabiometrictrait?
How many traits to identify 7 billion individuals? Howmanytraitstoidentify7billionindividuals?
Bodytraitvs.sensedimage
No.ofmonozygotictwinbirthrateisaboutthreeinevery1,000birthsworldwide
32
Persistence
Humanbody(hencebiometrictraits)willageovertime
Can we devise an age invariant template? Canwedeviseanageinvarianttemplate?
Score=0.84Score=0.76Score=0.71Score=0.58
COTSA
COTSB
33
ImagesfromPCSOmugshot database;CourtesyScottMcCallum
CamerasEverywhere
1M CCTV cameras in London & 4M in U.K.; average Briton is seen by 300 cameras/day;
400K cameras in Beijing provide 100% coverage of public places; 150K cameras in Seoul
34
FaceRecognitioninVideo
35
Howtodetectpersonsofinterestinavideoandthenidentifythem?
36
TopRetrievalRanksforTsarnaev Brothers
(with demographic filtering) (withdemographicfiltering)
37
BiometricSystemVulnerabilities
Insider Attacks
Enrollment Fraud
Collusion
Coercion
Enrollment
S d
Coercion
Exception Abuse
Feature
Extraction
Matcher
Enrollment
Verification
Stored
Templates
Matcher
Hill
Climbing, Re
play &
Man-in-the-
Middle
Spoofing &
Obfuscation
Attacks
Trojan
Horse
Attack
Trojan
Horse
Attack
Template
Attacks
Attacks
External Adversary Attacks
Fake
Iris
Altered
Fingerprints
Minutiae Reconstructed
Fingerprint
BiometricTemplateProtection
Application A
Feature
Extraction
Enrollment
Stored
Templates
Template
Protection
Matcher
Verification
Non-linkable
(Enhances Privacy
Non-invertible
(Enhances System Security
X
X
Application B
(Enhances Privacy
by preventing
cross-matching)
(Enhances System Security
by preventing intrusion attacks)
X
Stored
Templates
Can we generate a non-invertible AND non-linkable biometric template
without compromising the matching accuracy?
BiometricKeyGeneration
Courtesy: Prof. Tyfun Agkul, Istanbul Technical University
40
Circumvention
FaceLiveness
42
WhereisBiometricsGoing? g
Processor,memory&sensors , y
Ubiquitousbiometrics
C Context
Privacy
Biometricsforsocietalgood
P li ti Personalization
Biometrics&forensics
43
Processor,MemoryandSensingTechnology
Microprocessorperformancevs.cost RAMcapacityvs.cost
1892
Inkandpaper
1990s
Opticalsensor
1990s
Capacitivesensor
1997
First swipesensor
2007USVISIT
Slapsensor
2010SAFRAN
Touchless swipesensor
UbiquitousBiometrics q
Biometrics will become more
h l h l b h d holistic, where location, behavior and
recent interaction history fuse with
multimodal biometric ID (strong and
weak traits) to provide a strong
assurance of identity. This degree of
integration is both inevitable and g
necessary for ubiquitous biometrics.
CourtesyRobRowe,Lumidigm
45
46
GoogleGlasses
47
Privacy
Cafes ban Google
glasses to protect
customers' privacy: customers privacy:
Fears users of futuristic
eyewear can record
without permission.
http://www.dailymail.co.uk/sciencetech/article
2323578/CafesbanGoogleglassesprotectcustomers
privacyFearsusersfuturisticeyewearrecord
permission.html#ixzz2UgEfuN8F
48
YourDigitalFootprintDefinesYou
Browsing History
Bookmarks
Preferences
Your Favorite
Web Services
Your Family
& Friends
e-Transactions
Online Interaction Patterns
++
h What
you do?
Home & Office
Mobile
Where
you are?
What
you are?
WHO YOU ARE
GPSFingerprint:
Identification Without Biometric Data IdentificationWithoutBiometricData
Withjusttheanonymouslocationdata,onecanfigureoutwhoyouareby
trackingyoursmartphone.Patternsofourmovements,whentracedona
DeMontjoye,Hidalgo,Verleysen &Blondel,UniqueintheCrowd:ThePrivacyBoundsofHumanMobility,ScientificReports,vol.3,2013
map,createsomethingakintoafingerprint,unique toeveryperson.
50
AffectiveBiometrics
CourtesyProf.Tayfun Akgul,ITU
CellPhoneScience
(ByBillGates,Nov09,2010)
Most of us think of cell phones primarily as
a convenient tool to stay in touch with
people and store information. But
increasingly, scientists are exploring ways to
use cell phones to deliver critical health care
to people in developing countries.
Over1.7billionmobilephonesweresoldworldwidein2012alone
http://www.gartner.com/newsroom/id/2335616
http://www.thegatesnotes.com/Topics/Health/CellPhoneScience
52
MobilePhonebasedVaccinationRegistry
Use fingerprint scans to track children who have received immunizations The goal is to reduce
16monthsoldboy,rightringfinger
Usefingerprintscanstotrackchildrenwhohavereceivedimmunizations.Thegoalistoreduce
redundantdosesandincreasecoveragelevelsindevelopingcountries(MarkThomas,VaxTrak)
53
http://vaxtrac.com/about
Locards ExchangePrinciple
Edmond Locard (18771966), a pioneer in forensic
science, stated that the perpetrator of a crime will bring
something into the crime scene and leave with something from
it, and that both can be used as forensic evidence.
.. Not only his fingerprints or his footprints, but his hair, the
fibers from his clothes, the glass he breaks, the tool mark he
l th i t h t h th bl d h d it leaves, the paint he scratches, the blood or semen he deposits
or collects. All of these and more, bear mute witness against
him. This is evidence that does not forget. ..Only human
f il t fi d it t d d d t d it di i i h it failure to find it, study and understand it, can diminish its
value.
PaulKirk,Crimeinvestigation:physicalevidenceandthepolicelaboratory.Interscience Publishers,NY1953
http://en.wikipedia.org/wiki/Locard%27s_exchange_principle
54
ABriefHistoryofDNATesting
BeforeDNA:
Blood Typing
PCR,STRsandtheOnset
of Modern DNA Testing
Mitochondrial
Based Tests
Rapid
DNA
Testsresultsin
<2hours
Matchesactualsamplesratherthantemplates
BloodTyping
1916
1986
1988
1992
1996
ofModernDNATesting BasedTests DNA
2010s 1953
1986 1992
RFLP:ThefirstDNA
Fingerprinting
YChromosome
BasedTests
NonHuman
DNAForensics
Touch
DNA
Doublehelixstructureof
DNAfirstdiscovered
BloodTyping YChromosomeTests TouchDNA DoublehelixstructureofDNA
http://www.forensicmag.com/article/touchdnacrimescenecrimelaboratory http://www.newenglandinnocence.org/knowledgecenter/resources/dna/
55
Summary&Conclusions
BiometricsRecognitionisheretostay
h h l Researchvs.Technology
Driversforacademicresearchers:
Errorrates(butwhatsthebaseline?),new
biometric traits, fusion, biometrictraits,fusion,
Driversfortechnologyproviders:
i ( l Requirements (errorrates,template
size,processor,throughput),seamless
integration usability return on investment integration,usability,returnoninvestment
56
MyObservations
1. BiometricSystem:Almostalwaysembeddedinanapplication
2 Bi t i T it N ti l b t b tt th th 2. BiometricTrait:Nooptimalone,butsomearebetterthanothers
3. MatcherAccuracy:Zeroerrorisneitherrequirednorguaranteed
4 System Evaluation: Error rates in lab test are lower than field test 4. SystemEvaluation:Errorratesinlabtestarelowerthanfieldtest
5. Baseline:Improperbaselineprovidesfalsesenseofprogress
6 Security: Biometrics is an effective tool only if implemented well 6. Security:Biometricsisaneffectivetoolonlyifimplementedwell
7. BiometricTemplate:Featureextractionisnotaonewayfunction
8 Fusion: Does not guarantee better performance and security 8. Fusion:Doesnotguaranteebetterperformanceandsecurity
9. MatchScore:Gaussiandensityisnotadvisable.Tailsarecritical
10 Impact: Not without a perspective on application & technology 10. Impact:Notwithoutaperspectiveonapplication&technology
57
Acknowledgement
58