Table Of ContentOBJECT DETECTION
AND RECOGNITION
IN DIGITAL IMAGES
OBJECT DETECTION
AND RECOGNITION
IN DIGITAL IMAGES
THEORY AND PRACTICE
Bogusław Cyganek
AGHUniversityofScienceandTechnology,Poland
A John Wiley & Sons, Ltd., Publication
Thiseditionfirstpublished2013
(cid:2)C 2013JohnWiley&Sons,Ltd
Registeredoffice
JohnWiley&SonsLtd,TheAtrium,SouthernGate,Chichester,WestSussex,PO198SQ,UnitedKingdom
Fordetailsofourglobaleditorialoffices,forcustomerservicesandforinformationabouthowtoapplyfor
permissiontoreusethecopyrightmaterialinthisbookpleaseseeourwebsiteatwww.wiley.com.
TherightoftheauthortobeidentifiedastheauthorofthisworkhasbeenassertedinaccordancewiththeCopyright,
DesignsandPatentsAct1988.
Allrightsreserved.Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,ortransmitted,inany
formorbyanymeans,electronic,mechanical,photocopying,recordingorotherwise,exceptaspermittedbytheUK
Copyright,DesignsandPatentsAct1988,withoutthepriorpermissionofthepublisher.
Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthatappearsinprintmaynotbe
availableinelectronicbooks.
Designationsusedbycompaniestodistinguishtheirproductsareoftenclaimedastrademarks.Allbrandnamesand
productnamesusedinthisbookaretradenames,servicemarks,trademarksorregisteredtrademarksoftheir
respectiveowners.Thepublisherisnotassociatedwithanyproductorvendormentionedinthisbook.This
publicationisdesignedtoprovideaccurateandauthoritativeinformationinregardtothesubjectmattercovered.Itis
soldontheunderstandingthatthepublisherisnotengagedinrenderingprofessionalservices.Ifprofessionaladvice
orotherexpertassistanceisrequired,theservicesofacompetentprofessionalshouldbesought.
MATLAB(cid:2)R isatrademarkofTheMathWorks,Inc.andisusedwithpermission.TheMathWorksdoesnotwarrant
theaccuracyofthetextorexercisesinthisbook.ThisbooksuseordiscussionofMATLAB(cid:2)R softwareorrelated
productsdoesnotconstituteendorsementorsponsorshipbyTheMathWorksofaparticularpedagogicalapproachor
particularuseoftheMATLAB(cid:2)R software.
LibraryofCongressCataloging-in-PublicationData
Cyganek,Boguslaw.
Objectdetectionandrecognitionindigitalimages:theoryandpractice/BoguslawCyganek.
pagescm
Includesbibliographicalreferencesandindex.
ISBN978-0-470-97637-1(cloth)
1.Patternrecognitionsystems. 2.Imageprocessing–Digitaltechniques. 3.Computervision. I.Title.
TK7882.P3C942013
621.39(cid:3)94–dc23
2012050754
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
ISBN:978-0-470-97637-1
Typesetin10/12ptTimesbyAptaraInc.,NewDelhi,India
Tomyfamilywithlove
Contents
Preface xiii
Acknowledgements xv
NotationsandAbbreviations xvii
1 Introduction 1
1.1 ASampleofComputerVision 3
1.2 OverviewofBookContents 6
References 8
2 TensorMethodsinComputerVision 9
2.1 Abstract 9
2.2 Tensor–AMathematicalObject 10
2.2.1 MainPropertiesofLinearSpaces 10
2.2.2 ConceptofaTensor 11
2.3 Tensor–ADataObject 13
2.4 BasicPropertiesofTensors 15
2.4.1 NotationofTensorIndicesandComponents 16
2.4.2 TensorProducts 18
2.5 TensorDistanceMeasures 20
2.5.1 OverviewofTensorDistances 22
2.5.1.1 ComputationofMatrixExponentandLogarithmFunctions 24
2.5.2 EuclideanImageDistanceandStandardizingTransform 29
2.6 FilteringofTensorFields 33
2.6.1 OrderStatisticFilteringofTensorData 33
2.6.2 AnisotropicDiffusionFiltering 36
2.6.3 IMPLEMENTATIONofDiffusionProcesses 40
2.7 LookingintoImageswiththeStructuralTensor 44
2.7.1 StructuralTensorinTwo-DimensionalImageSpace 47
2.7.2 Spatio-TemporalStructuralTensor 50
2.7.3 MultichannelandScale-SpaceStructuralTensor 52
2.7.4 ExtendedStructuralTensor 54
2.7.4.1 IMPLEMENTATIONoftheLinearandNonlinear
StructuralTensor 57
viii Contents
2.8 ObjectRepresentationwithTensorofInertiaandMoments 62
2.8.1 IMPLEMENTATIONofMomentsandtheirInvariants 65
2.9 EigendecompositionandRepresentationofTensors 68
2.10 TensorInvariants 72
2.11 GeometryofMultipleViews:TheMultifocalTensor 72
2.12 MultilinearTensorMethods 75
2.12.1 BasicConceptsofMultilinearAlgebra 78
2.12.1.1 TensorFlattening 78
2.12.1.2 IMPLEMENTATIONTensorRepresentation 84
2.12.1.3 Thek-modeProductofaTensorandaMatrix 95
2.12.1.4 RanksofaTensor 100
2.12.1.5 IMPLEMENTATIONofBasicOperationsonTensors 101
2.12.2 Higher-OrderSingularValueDecomposition(HOSVD) 112
2.12.3 ComputationoftheHOSVD 114
2.12.3.1 ImplementationoftheHOSVDDecomposition 119
2.12.4 HOSVDInducedBases 121
2.12.5 TensorBestRank-1Approximation 123
2.12.6 Rank-1DecompositionofTensors 126
2.12.7 BestRank-(R ,R , ... ,R )Approximation 131
1 2 P
2.12.8 ComputationoftheBestRank-(R ,R , ... ,R )Approximations 134
1 2 P
2.12.8.1 IMPLEMENTATION–RankTensorDecompositions 137
2.12.8.2 CASESTUDY–DataDimensionalityReduction 145
2.12.9 SubspaceDataRepresentation 149
2.12.10NonnegativeMatrixFactorization 151
2.12.11ComputationoftheNonnegativeMatrixFactorization 155
2.12.12ImageRepresentationwithNMF 160
2.12.13ImplementationoftheNonnegativeMatrixFactorization 162
2.12.14NonnegativeTensorFactorization 169
2.12.15MultilinearMethodsofObjectRecognition 173
2.13 Closure 179
2.13.1 ChapterSummary 179
2.13.2 FurtherReading 180
2.13.3 ProblemsandExercises 181
References 182
3 ClassificationMethodsandAlgorithms 189
3.1 Abstract 189
3.2 ClassificationFramework 190
3.2.1 IMPLEMENTATIONComputerRepresentationofFeatures 191
3.3 SubspaceMethodsforObjectRecognition 194
3.3.1 PrincipalComponentAnalysis 195
3.3.1.1 ComputationofthePCA 199
3.3.1.2 PCAforMulti-ChannelImageProcessing 210
3.3.1.3 PCAforBackgroundSubtraction 214
3.3.2 SubspacePatternClassification 215
Contents ix
3.4 StatisticalFormulationoftheObjectRecognition 222
3.4.1 ParametricandNonparametricMethods 222
3.4.2 ProbabilisticFramework 222
3.4.3 BayesDecisionRule 223
3.4.4 MaximumaposterioriClassificationScheme 224
3.4.5 BinaryClassificationProblem 226
3.5 ParametricMethods–MixtureofGaussians 227
3.6 TheKalmanFilter 233
3.7 NonparametricMethods 236
3.7.1 HistogramBasedTechniques 236
3.7.2 ComparingHistograms 239
3.7.3 IMPLEMENTATION–MultidimensionalHistograms 243
3.7.4 ParzenMethod 246
3.7.4.1 KernelBasedMethods 248
3.7.4.2 Nearest-NeighborMethod 250
3.8 TheMeanShiftMethod 251
3.8.1 IntroductiontotheMeanShift 251
3.8.2 ContinuouslyAdaptiveMeanShiftMethod(CamShift) 257
3.8.3 AlgorithmicAspectsoftheMeanShiftTracking 259
3.8.3.1 TrackingofMultipleFeatures 259
3.8.3.2 TrackingofMultipleObjects 260
3.8.3.3 FuzzyApproachtotheCamShift 261
3.8.3.4 DiscriminationwithBackgroundInformation 262
3.8.3.5 AdaptiveUpdateoftheClassifiers 263
3.8.4 IMPLEMENTATIONoftheCamShiftMethod 264
3.9 NeuralNetworks 267
3.9.1 ProbabilisticNeuralNetwork 267
3.9.2 IMPLEMENTATION–ProbabilisticNeuralNetwork 270
3.9.3 HammingNeuralNetwork 274
3.9.4 IMPLEMENTATIONoftheHammingNeuralNetwork 278
3.9.5 MorphologicalNeuralNetwork 282
3.9.5.1 IMPLEMENTATIONoftheMorphologicalNeuralNetwork 285
3.10 KernelsinVisionPatternRecognition 291
3.10.1 KernelFunctions 296
3.10.2 IMPLEMENTATION–Kernels 301
3.11 DataClustering 306
3.11.1 Thek-MeansAlgorithm 308
3.11.2 Fuzzyc-Means 311
3.11.3 KernelFuzzyc-Means 313
3.11.4 MeasuresofClusterQuality 315
3.11.5 IMPLEMENTATIONIssues 317
3.12 SupportVectorDomainDescription 327
3.12.1 ImplementationofSupportVectorMachines 333
3.12.2 ArchitectureoftheEnsembleofOne-ClassClassifiers 334
3.13 Appendix–MATLAB(cid:2)R andotherPackagesforPatternClassification 336
3.14 Closure 336
x Contents
3.14.1 ChapterSummary 336
3.14.2 FurtherReading 337
ProblemsandExercises 338
References 339
4 ObjectDetectionandTracking 346
4.1 Introduction 346
4.2 DirectPixelClassification 346
4.2.1 Ground-TruthDataCollection 347
4.2.2 CASESTUDY–HumanSkinDetection 348
4.2.3 CASESTUDY–PixelBasedRoadSignsDetection 352
4.2.3.1 FuzzyApproach 353
4.2.3.2 SVMBasedApproach 353
4.2.4 PixelBasedImageSegmentationwithEnsembleofClassifiers 361
4.3 DetectionofBasicShapes 364
4.3.1 DetectionofLineSegments 366
4.3.2 UpWriteDetectionofConvexShapes 367
4.4 FigureDetection 370
4.4.1 DetectionofRegularShapesfromCharacteristicPoints 371
4.4.2 ClusteringoftheSalientPoints 375
4.4.3 AdaptiveWindowGrowingMethod 376
4.4.4 FigureVerification 378
4.4.5 CASESTUDY–RoadSignsDetectionSystem 380
4.5 CASESTUDY–RoadSignsTrackingandRecognition 385
4.6 CASESTUDY–FrameworkforObjectTracking 389
4.7 PedestrianDetection 395
4.8 Closure 402
4.8.1 ChapterSummary 402
4.8.2 FurtherReading 402
ProblemsandExercises 403
References 403
5 ObjectRecognition 408
5.1 Abstract 408
5.2 RecognitionfromTensorPhaseHistogramsandMorphologicalScaleSpace 409
5.2.1 ComputationoftheTensorPhaseHistogramsinMorphologicalScale 411
5.2.2 MatchingoftheTensorPhaseHistograms 413
5.2.3 CASESTUDY–ObjectRecognitionwithTensorPhaseHistogramsin
MorphologicalScaleSpace 415
5.3 InvariantBasedRecognition 420
5.3.1 CASESTUDY–PictogramRecognitionwithAffineMoment
Invariants 421
5.4 TemplateBasedRecognition 424
5.4.1 TemplateMatchingforRoadSignsRecognition 425
5.4.2 SpecialDistancesforTemplateMatching 428
5.4.3 RecognitionwiththeLog-PolarandScale-Spaces 429
Contents xi
5.5 RecognitionfromDeformableModels 436
5.6 EnsemblesofClassifiers 438
5.7 CASESTUDY–EnsembleofClassifiersforRoadSignRecognitionfrom
DeformedPrototypes 440
5.7.1 ArchitectureoftheRoadSignsRecognitionSystem 442
5.7.2 ModuleforRecognitionofWarningSigns 446
5.7.3 TheArbitrationUnit 452
5.8 RecognitionBasedonTensorDecompositions 453
5.8.1 PatternRecognitioninSubSpacesSpannedbytheHOSVD
DecompositionofPatternTensors 453
5.8.2 CASESTUDY–RoadSignRecognitionSystemBasedon
DecompositionofTensorswithDeformablePatternPrototypes 455
5.8.3 CASESTUDY–HandwrittenDigitRecognitionwithTensor
DecompositionMethod 462
5.8.4 IMPLEMENTATIONoftheTensorSubspaceClassifiers 465
5.9 EyeRecognitionforDriver’sStateMonitoring 470
5.10 ObjectCategoryRecognition 476
5.10.1 Part-BasedObjectRecognition 476
5.10.2 RecognitionwithBag-of-Visual-Words 477
5.11 Closure 480
5.11.1 ChapterSummary 480
5.11.2 FurtherReading 481
ProblemsandExercises 482
Reference 483
A Appendix 487
A.1 Abstract 487
A.2 MorphologicalScale-Space 487
A.3 MorphologicalTensorOperators 490
A.4 GeometryofQuadraticForms 491
A.5 TestingClassifiers 492
A.5.1 ImplementationoftheConfusionMatrixandTestingObject
DetectioninImages 496
A.6 CodeAccelerationwithOpenMP 499
A.6.1 RecipesforObject-OrientedCodeDesignwithOpenMP 501
A.6.2 HintsonUsingandCodePortingtoOpenMP 507
A.6.3 PerformanceAnalysis 511
A.7 UsefulMATLAB(cid:2)R FunctionsforMatrixandTensorProcessing 512
A.8 ShortGuidetotheAttachedSoftware 513
A.9 Closure 516
A.9.1 ChapterSummary 516
A.9.2 FurtherReading 519
ProblemsandExercises 520
References 520
Index 523
Preface
Weliveinaneraoftechnologicalrevolutioninwhichdevelopmentsinonedomainfrequently
entailbreakthroughsinanother.Similartothenineteenthcenturyindustrialrevolution,thelast
decadescanbetermedanepochofcomputerrevolution.Foryearswehavebeenwitnessing
therapiddevelopmentofmicrochiptechnologieswhichhasresultedinacontinuousgrowth
of computational power at ever decreasing costs. This has been underpinned by the recent
developments of parallel computational systems of graphics processing units and field pro-
grammablegatearrays.Allthesehardwareachievementsalsoopenupnewapplicationareas
and possibilities in the quest of making a computer see and understand what it sees – is a
primarygoalinthedomainofcomputervision.However,althoughfastcomputersareofgreat
helpinthisrespect,whatreallymakesadifferencearenewandbetterprocessingmethodsand
theirimplementations.
Thebookpresentsselectedmethodsofobjectdetectionandrecognitionwithspecialstress
on statistical and – relatively new to this domain – tensor based approaches. However, the
numberofinterestingandimportantmethodsisgrowingrapidly,makingitdifficulttooffera
completecoverageofthesemethodsinonebook.Thereforethegoalofthisbookisslightly
different, namely the methods chosen here have been used by myself and my colleagues
in many projects and proved to be useful in practice. Our main areas concern automotive
applications in which we try to develop vision systems for road sign recognition or driver
monitoring. When starting this book my main purpose was to not only give an overview of
thesemethods,butalsotoprovidethenecessary,thoughconcise,mathematicalbackground.
However,justasimportantareimplementationsofthediscussedmethods.I’mconvincedthat
the connection of detailed theory and its implementation is a prerequisite for the in-depth
understandingofthesubject.Inthisrespectthechoiceoftheimplementationplatformisalso
notasurprise.TheC++programminglanguageusedthroughoutthisbookandintheattached
softwarelibraryisofworldwideindustrystandard.Thisdoesnotmeanthatimplementations
cannotbedoneusingdifferentprogrammingplatforms,forwhichtheprovidedcodeexamples
canbeusedasaguideorfordirectporting.Thebookisaccompaniedbyacompanionwebsite
atwww.wiley.com/go/cyganekobjectwhichcontainsthecode,colorfigures,aswellasslides,
errataandotherusefullinks.
Thisbookgrewasaresultofmyfascinationwithmoderncomputervisionmethodsandalso
after writing my previous book, co-authored with J. Paul Siebert and devoted mostly to the
