Handbook of Statistics: Machine Learning: Theory and Applications

551 Pages·2013·19.312 MB·2-518\551

by  C.R. Rao and Venu Govindaraju (Eds.)

HANDBOOK OF STATISTICS
VOLUME 31
Handbook of Statistics
VOLUME 31
General Editor
C.R. Rao
C.R. RaoAIMSCS, Universityof HyderabadCampus, Hyderabad, India
Amsterdam Boston Heidelberg London NewYork Oxford
d d d d d
Paris SanDiego SanFrancisco Singapore Sydney Tokyo
d d d d d
Volume 31
Machine Learning: Theory and Applications
Edited by
Venu Govindaraju
Universityat Buffalo, Buffalo, USA
C.R. Rao
C.R. RaoAIMSCS, University of HyderabadCampus, Hyderabad,India
Amsterdam Boston Heidelberg London NewYork Oxford
d d d d d
Paris SanDiego SanFrancisco Singapore Sydney Tokyo
d d d d d
North-HollandisanimprintofElsevier
North-HollandisanimprintofElsevier
TheBoulevard,LangfordLane,Kidlington,Oxford,OX51GB,UK
Radarweg29,POBox211,1000AEAmsterdam,TheNetherlands
Firstedition2013
Copyright(cid:2)2013ElsevierB.V.Allrightsreserved.
Nopartofthispublicationmaybereproduced,storedinaretrievalsystemortransmitted
inanyformorbyanymeanselectronic,mechanical,photocopying,recordingorotherwise
withoutthepriorwrittenpermissionofthepublisher.
Permissions may be sought directly from Elsevier’s Science & Technology Rights
Department in Oxford, UK: phone (+44) (0) 1865 843830; fax (+44) (0) 1865 853333;
email: [e-mail]. Alternatively you can submit your request online by
visiting the Elsevier web site at http://elsevier.com/locate/permissions, and selecting
ObtainingpermissiontouseElseviermaterial.
Notice
Noresponsibilityisassumedbythepublisherforanyinjuryand/ordamagetopersonsor
property as a matter of products liability, negligence or otherwise, or from any use or
operationofanymethods,products,instructionsorideascontainedinthematerialherein.
Becauseofrapidadvancesinthemedicalsciences,inparticular,independentverificationof
diagnosesanddrugdosagesshouldbemade.
BritishLibraryCataloguinginPublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary.
LibraryofCongressCataloging-in-PublicationData
AcatalogrecordforthisbookisavailablefromtheLibraryofCongress.
ISBN:978-0-444-53859-8
ISSN:0169-7161
ForinformationonallNorth-Hollandpublications
visitourwebsiteatstore.elsevier.com
PrintedandboundinGreatBritain
13141516 10 9 8 7 6 5 4 3 2 1
Table of Contents
Volume 31 Handbook of Statistics
Contributors: Vol. 31 xi
Preface to Handbook Volume – 31 xv
Introduction xvii
Part I: Theoretical Aspects 1
Ch. 1. The Sequential Bootstrap 3
P.K. Pathak and C.R. Rao
1. Introduction 4
2. Asequentialbootstrapresamplingscheme 7
3. Bootstrappingempiricalmeasureswitharandomsamplesize 9
4. Convergenceratesforthesequentialbootstrap 12
5. Second-ordercorrectnessofthesequentialbootstrap 14
6. Concludingremarks 17
Acknowledgments 17
References 18
Ch. 2. The Cross-Entropy Method for Estimation 19
Dirk P. Kroese, Reuven Y. Rubinstein, and Peter W. Glynn
1. Introduction 19
2. Estimationsetting 20
3. Extensions 26
Acknowledgment 33
References 33
v
vi TableofContents
Ch. 3. The Cross-Entropy Method for Optimization 35
Zdravko I. Botev, Dirk P. Kroese, Reuven Y. Rubinstein,
and Pierre L’Ecuyer
1. Introduction 35
2. Fromestimationtooptimization 36
3. Applicationstocombinatorialoptimization 40
4. Continuousoptimization 50
5. Summary 57
References 57
Ch. 4. Probability Collectives in Optimization 61
David H. Wolpert, Stefan R. Bieniawski, and Dev G. Rajnarayan
1. Introduction 61
2. Delayedsamplingtheory 64
3. Delayedsamplingexperiments 70
4. Immediatesamplingtheory 82
5. Immediatesamplingexperiments 85
6. Conclusion 97
References 98
Ch. 5. Bagging, Boosting, and Random Forests Using R 101
Hansen Bannerman-Thompson, M. Bhaskara Rao,
and Subramanyam Kasala
1. Introduction 101
2. Datasetsandrationale 103
3. Bagging 107
4. Boosting 113
5. DoBaggingandBoostingreallywork? 115
6. Whatisaclassificationtree? 116
7. Classificationtreeversuslogisticregression 126
8. Randomforest 127
9. Randomforest,genetics,andcross-validation 134
10. Regressiontrees 139
11. BoostingusingtheRpackage,ada 144
12. Epilog 149
References 149
Ch. 6. Matching Score Fusion Methods 151
Sergey Tulyakov and Venu Govindaraju
1. Introduction 151
2. Matchingsystems 156
3. Selectedapproachestofusioninmatchingsystems 158
4. Operatingmodesofmatchingsystems 162
5. Complexitytypesofclassifiercombinationmethods 163
6. Modelingmatchingscoredependencies 165
7. Scorecombinationapplications 167
8. Conclusion 169
Appendices 169
TableofContents vii
A. ProofofClaim4 169
B. ProofofClaim5 171
References 173
Part II: Object Recognition 177
Ch. 7. Statistical Methods on Special Manifolds for Image and Video
Understanding 179
Pavan Turaga, Rama Chellappa, and Anuj Srivastava
1. Introduction 179
2. Somemotivatingexamples 180
3. Differentialgeometrictools 181
4. Commonmanifoldsarisinginimageanalysis 185
5. Applicationsinimageanalysis 187
6. Summaryanddiscussion 198
Acknowledgments 198
References 198
Ch. 8. Dictionary-Based Methods for Object Recognition 203
Vishal M. Patel and Rama Chellappa
1. Introduction 203
2. Sparserepresentation 204
3. Dictionarylearning 210
4. Concludingremarks 222
References 222
Ch. 9. Conditional Random Fields for Scene Labeling 227
Ifeoma Nwogu and Venu Govindaraju
1. Introduction 227
2. OverviewofCRF 229
3. Sceneparsing 236
4. MorerecentimplementationsofCRFscenelabelings 242
5. Conclusionandfuturedirections 245
References 245
Ch. 10.Shape-Based Image Classification and Retrieval 249
N. Mohanty, A. Lee-St. John, R. Manmatha,
and T.M. Rath
1. Introduction 250
2. Priorwork 251
3. Classificationandretrievalmodels 252
4. Features 257
5. Classificationexperiments 261
6. Retrieval 262
viii TableofContents
7. Multipleclasslabels 265
8. Summaryandconclusions 266
References 266
Ch. 11.Visual Search: A Large-Scale Perspective 269
Robinson Piramuthu, Anurag Bhardwaj, Wei Di,
and Neel Sundaresan
1. Introduction 269
2. Whenisbigdataimportant? 272
3. Informationextractionandrepresentation 273
4. Matchingimages 282
5. Practicalconsiderations:memoryfootprintandspeed 285
6. Benchmarkdatasets 289
7. Closingremarks 291
References 293
Part III: Biometric Systems 299
Ch. 12.VideoActivityRecognitionbyLuminanceDifferentialTrajectoryandAligned
Projection Distance 301
Haomian Zheng, Zhu Li, Yun Fu, Aggelos K. Katsaggelos, and Jane You
1. Introduction 302
2. Relatedwork 303
3. Problemformulation 305
4. DLFTandLAPDsolutions 312
5. Experiments 314
6. Conclusion 323
References 324
Ch. 13.Soft Biometrics for Surveillance: An Overview 327
D.A. Reid, S. Samangooei, C. Chen, M.S. Nixon, and A. Ross
1. Introduction 327
2. Performancemetrics 329
3. Incorporatingsoftbiometricsinafusionframework 330
4. Humanidentificationusingsoftbiometrics 333
5. Predictinggenderfromfaceimages 344
6. Applications 346
7. Conclusion 349
References 350
Ch. 14.A User Behavior Monitoring and Profiling Scheme for Masquerade
Detection 353
Ashish Garg, Shambhu Upadhyaya, and Kevin Kwiat
1. Introduction 354
2. Relatedwork 356
3. SupportVectorMachines(SVMs) 358
TableofContents ix
4. Datacollection,featureextraction,andfeaturevectorgeneration 361
5. Experimentaldesign 367
6. Discussionandconclusion 376
Acknowledgments 376
References 377
Ch. 15.Application of Bayesian Graphical Models to Iris Recognition 381
B.V.K. Vijaya Kumar, Vishnu Naresh Boddeti, Jonathon M. Smereka,
Jason Thornton, and Marios Savvides
1. Introduction 381
2. Gaborwavelet-basedmatching 383
3. Correlationfilter-basedirismatching 386
4. Bayesiangraphicalmodelforirisrecognition 390
5. Summary 397
Acknowledgments 397
References 397
Part IV: Document Analysis 399
Ch. 16.Learning Algorithms for Document Layout Analysis 401
Simone Marinai
1. Introduction 401
2. Pixelclassification 405
3. Zoneclassification 406
4. Connectedcomponentclassification 409
5. Textregionsegmentation 411
6. Regionclassification 412
7. Functionallabeling 415
8. Conclusion 416
References 416
Ch. 17.Hidden Markov Models for Off-Line Cursive Handwriting
Recognition 421
Andreas Fischer, Volkmar Frinken, and Horst Bunke
1. Introduction 421
2. Serializationofhandwritingimages 423
3. HMM-basedtextlinerecognition 426
4. Outlookandconclusions 438
Acknowledgment 439
References 439
Ch. 18.Machine Learning in Handwritten Arabic Text Recognition 443
Utkarsh Porwal, Zhixin Shi, and Srirangaraj Setlur
1. Introduction 443
2. Arabicscript—challengesforrecognition 444
3. Learningparadigms 447
4. Featuresfortextrecognition 455