Advanced Methods and Deep Learning in Computer Vision (Computer Vision and Pattern Recognition)

584 Pages·2021·25.674 MB·English

by  E. R. Davies (editor), Matthew Turk (editor)

ADVANCED METHODS AND DEEP
LEARNING IN COMPUTER VISION
This page intentionally left blank
Computer Vision and Pattern Recognition
ADVANCED
METHODS AND
DEEP LEARNING
IN
COMPUTER
VISION
Editedby
E.R. Davies
Matthew A. Turk
AcademicPressisanimprintofElsevier
125LondonWall,LondonEC2Y5AS,UnitedKingdom
525BStreet,Suite1650,SanDiego,CA92101,UnitedStates
50HampshireStreet,5thFloor,Cambridge,MA02139,UnitedStates
TheBoulevard,LangfordLane,Kidlington,OxfordOX51GB,UnitedKingdom
Copyright©2022ElsevierInc.Allrightsreserved.
Nopartofthispublicationmaybereproducedortransmittedinanyformorbyanymeans,electronicormechanical,
includingphotocopying,recording,oranyinformationstorageandretrievalsystem,withoutpermissioninwritingfromthe
publisher.Detailsonhowtoseekpermission,furtherinformationaboutthePublisher’spermissionspoliciesandour
arrangementswithorganizationssuchastheCopyrightClearanceCenterandtheCopyrightLicensingAgency,canbefound
atourwebsite:www.elsevier.com/permissions.
ThisbookandtheindividualcontributionscontainedinitareprotectedundercopyrightbythePublisher(otherthanasmay
benotedherein).
Notices
Knowledgeandbestpracticeinthisfieldareconstantlychanging.Asnewresearchandexperiencebroadenour
understanding,changesinresearchmethods,professionalpractices,ormedicaltreatmentmaybecomenecessary.
Practitionersandresearchersmustalwaysrelyontheirownexperienceandknowledgeinevaluatingandusingany
information,methods,compounds,orexperimentsdescribedherein.Inusingsuchinformationormethodstheyshouldbe
mindfuloftheirownsafetyandthesafetyofothers,includingpartiesforwhomtheyhaveaprofessionalresponsibility.
Tothefullestextentofthelaw,neitherthePublishernortheauthors,contributors,oreditors,assumeanyliabilityforany
injuryand/ordamagetopersonsorpropertyasamatterofproductsliability,negligenceorotherwise,orfromanyuseor
operationofanymethods,products,instructions,orideascontainedinthematerialherein.
LibraryofCongressCataloging-in-PublicationData
AcatalogrecordforthisbookisavailablefromtheLibraryofCongress
BritishLibraryCataloguing-in-PublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
ISBN:978-0-12-822109-9
ForinformationonallAcademicPresspublications
visitourwebsiteathttps://www.elsevier.com/books-and-journals
Publisher:MaraConner
AcquisitionsEditor:TimPitts
EditorialProjectManager:IsabellaC.Silva
ProductionProjectManager:SojanP.Pazhayattil
Designer:GregHarris
TypesetbyVTeX
Thisbookisdedicatedtomyfamily.
Tocherishedmemoriesofmyparents,ArthurandMaryDavies.
Tomywife,Joan,forlove,patience,supportandinspiration.
Tomychildren,Elizabeth,SarahandMarion,andgrandchildren,Jasper,Jerome,Eva,TaraandPia,
forbringingendlessjoyintomylife!
RoyDavies
Thisbookisdedicatedtothestudents,colleagues,friends,andfamilywhohavemotivated,guided,
andsupportedmeinwaystoonumeroustomention.
Tomywife,Kelly,andmychildren,HannahandMatt–specialthanksandappreciationforyourlove
andinspiration.
MatthewTurk
This page intentionally left blank
Contents
List of contributors xi 2.13 Conclusion 115
About the editors xiii References 116
Preface xv
3. Learningwithlimitedsupervision
1. Thedramaticallychangingfaceof
SUJOYPAULANDAMITK.ROY-CHOWDHURY
computervision
3.1 Introduction 119
E.R.DAVIES
3.2 Context-awareactivelearning 120
3.3 Weaklysupervisedevent
1.1 Introduction–computervisionandits
localization 129
origins 1
3.4 Domainadaptationofsemantic
1.2 PartA–Understandinglow-levelimage
segmentationusingweaklabels 137
processingoperators 4
3.5 Weakly-supervisedreinforcementlearning
1.3 PartB–2-Dobjectlocationand
fordynamicaltasks 144
recognition 15
3.6 Conclusions 151
1.4 PartC–3-Dobjectlocationandthe
References 153
importanceofinvariance 29
1.5 PartD–Trackingmovingobjects 55
4. Efficientmethodsfordeeplearning
1.6 PartE–Textureanalysis 61
1.7 PartF–Fromartificialneuralnetworksto HANCAI,JILIN,ANDSONGHAN
deeplearningmethods 68
1.8 PartG–Summary 86 4.1 Modelcompression 159
References 87 4.2 Efficientneuralnetwork
architectures 170
2. Advancedmethodsforrobustobject 4.3 Conclusion 185
detection References 185
ZHAOWEICAIANDNUNOVASCONCELOS
5. Deepconditionalimagegeneration
2.1 Introduction 93 GANGHUAANDDONGDONGCHEN
2.2 Preliminaries 95
2.3 R-CNN 96 5.1 Introduction 191
2.4 SPP-Net 97 5.2 Visualpatternlearning:abrief
2.5 FastR-CNN 98 review 194
2.6 FasterR-CNN 101 5.3 Classicalgenerativemodels 195
2.7 CascadeR-CNN 103 5.4 Deepgenerativemodels 197
2.8 Multiscalefeaturerepresentation 106 5.5 Deepconditionalimagegeneration 200
2.9 YOLO 110 5.6 Disentanglementforcontrollable
2.10 SSD 112 synthesis 201
2.11 RetinaNet 113 5.7 Conclusionanddiscussions 216
2.12 Detectionperformances 115 References 216
vii
viii
Contents
6. Deepfacerecognitionusingfulland 9.3 Online-learning-basedmethods 314
partialfaceimages 9.4 Deeplearning-basedmethods 323
9.5 Thetransitionfromtrackingto
HASSANUGAIL
segmentation 327
9.6 Conclusions 331
6.1 Introduction 221
References 332
6.2 Componentsofdeepface
recognition 227
10. Long-termdeepobjecttracking
6.3 Facerecognitionusingfullface
images 231 EFSTRATIOSGAVVESANDDEEPAKGUPTA
6.4 Deepfacerecognitionusingpartialface
data 233 10.1 Introduction 337
6.5 Specificmodeltrainingforfullandpartial 10.2 Short-termvisualobjecttracking 341
faces 237 10.3 Long-termvisualobjecttracking 345
6.6 Discussionandconclusions 239 10.4 Discussion 367
References 240 References 368
7. Unsuperviseddomainadaptationusing 11. Learningforaction-basedscene
shallowanddeeprepresentations understanding
YOGESHBALAJI,HIENNGUYEN,ANDRAMACHELLAPPA CORNELIAFERMÜLLERANDMICHAELMAYNORD
7.1 Introduction 243 11.1 Introduction 373
7.2 Unsuperviseddomainadaptationusing 11.2 Affordancesofobjects 375
manifolds 244 11.3 Functionalparsingofmanipulation
7.3 Unsuperviseddomainadaptationusing actions 383
dictionaries 247 11.4 Functionalsceneunderstandingthrough
7.4 Unsuperviseddomainadaptationusingdeep deeplearningwithlanguageand
networks 258 vision 390
7.5 Summary 270 11.5 Futuredirections 397
References 270 11.6 Conclusions 399
References 399
8. Domainadaptationandcontinual
learninginsemanticsegmentation 12. Self-supervisedtemporalevent
segmentationinspiredbycognitivetheories
UMBERTOMICHIELI,MARCOTOLDO,AND
PIETROZANUTTIGH
RAMYMOUNIR,SATHYANARAYANANAAKUR,AND
SUDEEPSARKAR
8.1 Introduction 275
8.2 Unsuperviseddomainadaptation 277 12.1 Introduction 406
8.3 Continuallearning 291 12.2 Theeventsegmentationtheoryfrom
8.4 Conclusion 298 cognitivescience 408
References 299 12.3 Version1:single-passtemporalsegmentation
usingprediction 410
9. Visualtracking 12.4 Version2:segmentationusing
attention-basedeventmodels 421
MICHAELFELSBERG
12.5 Version3:spatio-temporallocalizationusing
predictionlossmap 428
9.1 Introduction 305
12.6 Othereventsegmentationapproachesin
9.2 Template-basedmethods 308
computervision 440
ix
Contents
12.7 Conclusions 443 14.3 Deepplug-and-playimage
References 444 restoration 485
14.4 Deepunfoldingimagerestoration 492
13. Probabilisticanomalydetection 14.5 Experiments 495
14.6 Discussionandconclusions 504
methodsusinglearnedmodelsfrom
References 505
time-seriesdataformultimediaself-aware
systems
15. Visualadversarialattacksanddefenses
CARLOREGAZZONI,ALIKRAYANI,GIULIASLAVIC,AND
LUCIOMARCENARO CHANGJAEOH,ALESSIOXOMPERO,AND
ANDREACAVALLARO
13.1 Introduction 450
15.1 Introduction 511
13.2 Baseconceptsandstateoftheart 451
15.2 Problemdefinition 512
13.3 Frameworkforcomputinganomalyin
15.3 Propertiesofanadversarialattack 514
self-awaresystems 458
15.4 Typesofperturbations 515
13.4 Casestudyresults:anomalydetectionon
15.5 Attackscenarios 515
multisensorydatafromaself-aware
15.6 Imageprocessing 522
vehicle 467
15.7 Imageclassification 523
13.5 Conclusions 476
15.8 Semanticsegmentationandobject
References 477
detection 529
15.9 Objecttracking 529
14. Deepplug-and-playanddeepunfolding
15.10 Videoclassification 531
methodsforimagerestoration 15.11 Defensesagainstadversarialattacks 533
15.12 Conclusions 537
KAIZHANGANDRADUTIMOFTE
References 538
14.1 Introduction 481
Index 545
14.2 Halfquadraticsplitting(HQS)
algorithm 484