By Sunpreet Singh Arora Submitted to Michigan State University in partial fulfillment of the ...

207 Pages·2016·36.77 MB·English

FINGERPRINT RECOGNITION: CONTRIBUTIONS TO LATENT
MATCHING AND 3D FINGERPRINT TARGET GENERATION
By
Sunpreet Singh Arora
A DISSERTATION
Submitted to
Michigan State University
in partial fulfillment of the requirements
for the degree of
Computer Science – Doctor of Philosophy
2016
ABSTRACT
FINGERPRINT RECOGNITION: CONTRIBUTIONS TO LATENT MATCHING AND
3D FINGERPRINT TARGET GENERATION
By
Sunpreet Singh Arora
Automatic fingerprint capture and comparison methods have led to the ubiquitous use of
fingerprint-basedpersonrecognitioninapplicationsrangingfromlawenforcementandbordercon-
troltonationalidentificationandsmartphoneunlock. However,despitetremendousadvancements
inthestate-of-the-art,improvementsarestillneededincaseofsomechallengingapplications,e.g,
to recognize poor quality and distorted fingerprints acquired from non-cooperative users, improve
fingerprint reader fidelity, and determine anti-spoofing capability of different fingerprint readers.
In this thesis, we address two such impending challenges: (i) comparison of latent prints found at
crime scenes to large collections of reference prints (rolled tenprints or slap fingerprints) in law
enforcement databases, and (ii) operational evaluation of fingerprint recognition systems prior to
largescaledeployment.
Wedevelopafeedbackparadigmthatusesreferenceprintfeaturestodynamicallyselectlatent
features during matching. The paradigm automatically determines if dynamic latent feature selec-
tion would improve recognition performance using a statistical hypothesis test and qualitatively
decides the regions in latent and reference prints for applying feedback. The paradigm when used
inconjunctionwithastate-of-the-artlatentmatcherdemonstratesmarkedimprovement(0.5-3.5%)
inlatentmatchingaccuracy.
Further, we develop a framework for crowdsourcing latent print feature markup to a pool
of fingerprint examiners. The framework uses a statistical criterion to automatically determine
whencrowdsourcingisrequired,andamethodtodynamicallydeterminethenumberofexaminers
needed for latent feature markup. Significant recognition performance improvements (2.5-11.5%)
areobtainedusingcrowdsourcedmarkupsinconjunctionwithastate-of-the-artlatentmatcher.
Finally,wedesignandfabricatesingle-fingerandwholehand3Dtargetsforoperationalevalu-
ationofopticalandcapacitivefingerprintreadersaswellasforend-to-endevaluationoffingerprint
recognitionsystems. 2Dcalibrationpatternswithknowncharacteristics(e.g. syntheticfingerprints
with known features, sine gratings with known orientation and spacing) are projected onto elec-
tronic3Dfingerandhandsurfacestocreateelectronic3Dsingle-fingerandwholehandtargets. A
high-resolution 3D printer is used to manufacture physical 3D single-finger and whole hand tar-
gets from electronic targets. Other contributions include: (i) a method to chemically clean the 3D
printedtargetswithoutimpactingtheengravedtargetpatterns,(ii)aproceduretoapplyconductive
coating of metal/metal oxides on the surface of 3D targets using DC sputtering, (iii) fidelity mea-
surement techniques using optical microscopy to assess the 3D target generation process, and (iv)
methods to evaluate fingerprint readers using the fabricated 3D targets. We demonstrate that the
2D calibration pattern features are reproduced with high fidelity both on the electronic and phys-
ical 3D single-finger and whole hand targets and that the intra-class variations between images of
the 3D targets do not degrade matching accuracy (at 0.01% false accept rate). We evaluate several
commerciallyavailablesingle-fingerandslapcontact-basedandcontactlessopticalreadersaswell
ascapacitivereadersusingthegenerated3Dtargets.
Copyright by
SUNPREET SINGH ARORA
2016
To all my loved ones.
v
ACKNOWLEDGMENTS
As I sit down to write this dissertation, I would like to take the opportunity to thank all my
teachers, including my parents, primary, middle school and high school teachers, university pro-
fessors and peers, who have taught me wonderful things, and from whom I have assimilated a
majorityofmylifelearnings. Iwillalwaysremainindebtedtoallofyoufornotonlysharingyour
knowledgebutalsofuelingmycuriositytoexploreandlearn.
I am grateful to my advisor, Dr. Anil K. Jain for his encouragement and support at both
professional and personal level. He backed me to the hilt while I was trying to stand on my
own feet as a researcher. Once I did, he let me take my own flight and explore. I am also thankful
toDr. XiaomingLiu,Dr. ArunRossandDr. SelinAviyenteforservingonmyPh.D.committee.
ThePRIPlabprovidedmetherightkindofenvironmentforlearning. Interactionswithpeersin
the lab during the last four years have not only advanced my knowledge but helped me appreciate
thenuancesofresearch. Specifically,Iwouldliketomentionafewwhohaveleftalastingimpres-
sion on me due to their sincerity, hard work, and tenacity: Soweon, Serhat, Radha, Alessandra,
Scott,Lacey,Tarang,Keyur,Charles,Inci,Josh,Debayan,Kai,Hu,andEryun. Iwouldespecially
like to thank Kai for all his help during my formative years. A few special mentions: Scott for
being the one I could exchange ideas with for the first couple of years, Charles and Inci for being
reallygoodfriendsandforlisteningtomyday-to-daygraduatelifeissuespatientlyoverlunch,and
Lacey for being the frequent flyer companion to attend various conferences together around the
world.
I would like to acknowledge the following organizations and individuals for generously sup-
porting the research projects I worked on: Nicholas G. Paulter Jr. from the National Institute of
Standards and Technology (NIST), NIST Measurement Science program grant 60NANB11D155,
the National Science Foundation (NSF) Center for Identification Technology Research (CITeR)
grant 1066197, Ken Warman from the Bill and Melinda Gates Foundation, Mark Thomas and
ShawnSarwarfromVaxTrac,andCaptainGregoryMichaudfromtheForensicScienceDivisionat
vi
MichiganStatePolice. IamalsogratefultoBrianWright,MichiganStateUniversityandMatthew
Staymates, National Institute of Standards and Technology (NIST) for their assistance with 3D
printing fingerprint models, and Anna Song, Michigan State University for her help in cleaning
the3Dprintedmodels. IwouldalsoliketothankBrianWright,ChrisTraverseandLarsHaubold,
Michigan State University for their help in sputter coating various materials on 3D fingerprint
targets.
I also want to thank the administrative staff of the Computer Science Department at MSU,
includingNorma,Linda,Cathy,Debbie,KatyandCourtneyfortheirhelpwithvariousthingssuch
ascourseregistration,conferencetravelandreimbursementsduringthecourseofmystudy.
IwouldliketoexpressmysinceregratitudetoDr. SalilPrabhakarforgivingmetheopportunity
to intern at Delta ID and extending all possible help to make my stay comfortable in the bay area.
It was a wonderful experience working at Delta ID, particularly with Alex Ivanisov and Dr. Yi
Chen.
I owe a lot to my parents and brother for being by my side while I was going through the
ups and downs of my journey as a graduate student. Without their moral support, writing this
dissertationwouldnothavebeenpossible.
Last but not the least, I would like to mention some special friends I made during my time
here in East Lansing. They provided me immense support in different ways to make my stay
worthwhile, Tridip Das, Girish Kasat, Shantanu Kelkar, Abhishek Santra, Sabyasachi Halder,
Siddhartha Dutta, Shailendra, Rahul Deshpande, Prajakta Deshpande Kulkarni, Maya Patel, Tias
Maiti, Rajib Mandal, Shreya Nad, Soumen Ghosh, Chetan Tambe, Oishi Sanyal, Saptarshi Das,
Saptarshi Mukherjee, Portia Banerjee, Piku Mandal, Arkaprabha Konar, Preetam Giri, and Aritra
Chakroborty. Ialsothankmyfriendsbackhome,particularly,VidushiChaudharyandKajalJoneja
forkeepingmemotivatedwhenthechipsweredown.
vii
TABLE OF CONTENTS
LISTOFTABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii
LISTOFFIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi
Chapter1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 FingerprintFormation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 FundamentalTenets: UniquenessandPermanence . . . . . . . . . . . . . . . . . . 4
1.3 FingerprintMilestones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3.1 MajorScientificStudies . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3.2 ApplicationsinLawEnforcement . . . . . . . . . . . . . . . . . . . . . . 7
1.3.3 OtherApplications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4 ComparisonwithOtherTraits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.5 DesignofFingerprintRecognitionSystems . . . . . . . . . . . . . . . . . . . . . 11
1.5.1 FingerprintAcquisition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.5.1.1 FingerprintSensingTechnologies . . . . . . . . . . . . . . . . . 15
1.5.2 FeatureExtraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.5.3 FingerprintMatching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
1.5.3.1 Exemplar-to-Exemplarmatching . . . . . . . . . . . . . . . . . 20
1.5.3.2 Latent-to-Exemplarmatching . . . . . . . . . . . . . . . . . . . 22
1.6 EvaluationofFingerprintRecognitionSystems . . . . . . . . . . . . . . . . . . . 23
1.6.1 SensingTechnologyCertification . . . . . . . . . . . . . . . . . . . . . . . 23
1.6.2 FeatureExtractionandMatchingEvaluation . . . . . . . . . . . . . . . . . 24
1.7 ChallengesinFingerprintRecognition . . . . . . . . . . . . . . . . . . . . . . . . 26
1.7.1 OpenResearchIssuesandChallenges . . . . . . . . . . . . . . . . . . . . 26
1.7.1.1 Automaticlatentfingerprintmatching . . . . . . . . . . . . . . . 26
1.7.1.2 Interoperabilityoffingerprintreaders . . . . . . . . . . . . . . . 26
1.7.1.3 Operationalevaluationoffingerprintsystems . . . . . . . . . . . 27
1.7.1.4 Fingerprintlivenessdetection . . . . . . . . . . . . . . . . . . . 27
1.7.1.5 Fingerprinttemplatesecurity . . . . . . . . . . . . . . . . . . . . 28
1.7.1.6 Matchingnon-idealfingerprintimages . . . . . . . . . . . . . . 29
1.7.2 AutomaticLatentFingerprintMatching . . . . . . . . . . . . . . . . . . . 29
1.7.3 OperationalEvaluationofFingerprintSystems . . . . . . . . . . . . . . . 30
1.8 DissertationContributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Chapter2 LatentFingerprintMatching: PerformanceGainviaFeedbackfromEx-
emplarPrints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.1.1 ManualLatentMatching . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.1.2 Bottom-upLatentMatchingSystems . . . . . . . . . . . . . . . . . . . . . 40
2.1.3 ProposedTop-DownLatentMatchingFramework . . . . . . . . . . . . . . 40
2.2 FeedbackParadigmforLatentMatching . . . . . . . . . . . . . . . . . . . . . . . 42
viii
2.3 Re-sortingCandidateListbasedonFeedback . . . . . . . . . . . . . . . . . . . . 43
2.3.1 InitialMatchingandAlignment . . . . . . . . . . . . . . . . . . . . . . . 45
2.3.2 ExemplarFeatureExtraction . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.3.3 LatentFeatureExtractionandRefinement . . . . . . . . . . . . . . . . . . 49
2.3.4 MatchScoreComputation . . . . . . . . . . . . . . . . . . . . . . . . . . 51
2.4 TheAdequacyofFeedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
2.4.1 GlobalCriterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
2.4.1.1 ModellingtheMatchScoreDistribution . . . . . . . . . . . . . . 52
2.4.1.2 Testforthepresenceofanupperoutlier . . . . . . . . . . . . . . 53
2.4.2 LocalCriterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
2.5 ExperimentalEvaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
2.5.1 Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
2.5.1.1 NISTSD27 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
2.5.1.2 WVU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
2.5.2 SizeoftheCandidateList(K) . . . . . . . . . . . . . . . . . . . . . . . . 62
2.5.3 EffectivenessoftheGlobalCriterionforFeedback . . . . . . . . . . . . . 62
2.5.4 PerformanceonNISTSD27Database . . . . . . . . . . . . . . . . . . . . 65
2.5.5 PerformanceonWVUDatabase . . . . . . . . . . . . . . . . . . . . . . . 66
2.5.6 ComputationalComplexity . . . . . . . . . . . . . . . . . . . . . . . . . . 68
2.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Chapter3 CrowdPoweredLatentFingerprintMatching: FusingAFISwithExam-
inerMarkups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.1.1 Semi-automaticLatentMatching: AdvantagesandDisadvantages . . . . . 71
3.1.2 ProposedCrowdPoweredLatentMatchingFramework . . . . . . . . . . . 72
3.2 CollectiveWisdomofMultipleExaminers . . . . . . . . . . . . . . . . . . . . . . 74
3.2.1 Expertcrowdsourcingframework . . . . . . . . . . . . . . . . . . . . . . 74
3.2.2 Whentocrowdsource? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.2.3 Howmanyexpertsareenough? . . . . . . . . . . . . . . . . . . . . . . . . 77
3.3 ExperimentalDetails . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.3.1 Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
3.3.2 LatentMarkup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
3.3.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
3.3.3.1 Lights-outMatching . . . . . . . . . . . . . . . . . . . . . . . . 81
3.3.3.2 MatchingIndividualExaminerMarkups . . . . . . . . . . . . . . 81
3.3.3.3 FusingMultipleExaminerMarkups . . . . . . . . . . . . . . . . 84
3.3.3.4 Fusinglights-outAFISwithMultipleMarkups . . . . . . . . . . 84
3.3.3.5 Determiningtheneedforcrowdsourcing . . . . . . . . . . . . . 85
3.3.3.6 Greedycrowdsourcing . . . . . . . . . . . . . . . . . . . . . . . 86
3.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
Chapter4 DesignandFabricationof3DSingle-FingerTargets . . . . . . . . . . . . 89
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
4.1.1 StructuralEvaluationofFingerprintReaders . . . . . . . . . . . . . . . . . 91
ix
4.1.2 BehavioralEvaluationofFingerprintReaders . . . . . . . . . . . . . . . . 92
4.1.3 3DTargetsforBehavioralEvaluation . . . . . . . . . . . . . . . . . . . . 93
4.2 Generating3DTargets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
4.2.1 Preprocessing3Dfingersurface . . . . . . . . . . . . . . . . . . . . . . . 99
4.2.1.1 Alignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
4.2.1.2 Remeshing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
4.2.1.3 Subdivision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
4.2.1.4 Creatingoutersurface . . . . . . . . . . . . . . . . . . . . . . . 101
4.2.1.5 Separatingfrontandrearportions . . . . . . . . . . . . . . . . . 102
4.2.2 Preprocessing2Dcalibrationpattern . . . . . . . . . . . . . . . . . . . . . 102
4.2.3 Mapping2Dcalibrationpatternto3Dsurface . . . . . . . . . . . . . . . . 103
4.2.4 Engraving2Dcalibrationpatternon3Dsurface . . . . . . . . . . . . . . . 106
4.2.5 Postprocessing3Dfingersurface . . . . . . . . . . . . . . . . . . . . . . . 107
4.2.6 3Dprinting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
4.2.7 Chemicalcleaning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
4.3 Fidelityof3DTargetGeneration . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
4.3.1 2Dto3DProjectionError . . . . . . . . . . . . . . . . . . . . . . . . . . 109
4.3.2 3DprintingFabricationError . . . . . . . . . . . . . . . . . . . . . . . . . 110
4.3.3 Fidelityof2Dpatternfeaturesduring3Dtargetcreation . . . . . . . . . . 111
4.3.3.1 Fidelityof2Dpatternfeaturesafterprojectionto3Dsurface . . . 112
4.3.3.2 Fidelityoftheengravedfeaturesonthe3Dsurfaceafter3Dprinting113
4.3.3.3 End-to-end fidelity of 2D calibration pattern features after 3D
printing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
4.3.3.4 Intra-classvariabilitybetween3Dtargetimpressions . . . . . . . 117
4.4 BehavioralEvaluationofFingerprintReadersusing3DTargets . . . . . . . . . . . 118
4.4.1 ExperimentI:SyntheticSineGratingTargets . . . . . . . . . . . . . . . . 118
4.4.2 ExperimentII:FingerprintTargets . . . . . . . . . . . . . . . . . . . . . . 120
4.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
Chapter5 3DWholeHandTargets: EvaluatingSlapandContactlessReaders . . . . 125
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
5.2 GeneratingWholeHandTarget . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
5.3 Fidelityof3DWholeHandTargetGeneration . . . . . . . . . . . . . . . . . . . . 135
5.3.1 Replicationof2Dcalibrationpatternfeaturesonelectronic3Dhandtarget . 136
5.3.2 Replicationofelectronic3Dhandtargetfeaturesonphysical3Dhandtarget 137
5.3.3 Replicationof2Dcalibrationpatternfeaturesonphysical3Dhandtarget . . 138
5.3.4 Consistencybetweendifferentimpressionsofthephysical3Dhandtarget . 139
5.4 EvaluatingContact-basedSlapFingerprintReaders . . . . . . . . . . . . . . . . . 140
5.5 EvaluatingContactlessSlapFingerprintReader . . . . . . . . . . . . . . . . . . . 142
5.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
Chapter6 Generating3DConductiveFingerprintTargets . . . . . . . . . . . . . . . 146
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
6.2 SputterCoating3DTargets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
6.2.1 DCSputteringProcess . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
x