Surface Representations and Automatic Feature Learning for 3D Object Recognition Syed Afaq Ali ...

179 Pages·2012·3.08 MB·English

Surface Representations and Automatic
Feature Learning for 3D Object
Recognition
Syed Afaq Ali Shah
Thisthesisispresentedforthedegreeof
DoctorofPhilosophy
ofTheUniversityofWesternAustralia
SchoolofComputerScienceandSoftwareEngineering.
January,2016
a
(cid:2)
c Copyright 2016 by Syed Afaq Ali Shah
a
a
a Dedicated to my father,
a Syed Zahoor Ahmed Shah,
a And my wife,
a Maleeha
a
a
Abstract
Vision-based object recognition is a popular area that has gained a significant pop-
ularity in the last decade because of its many applications including robotics, medical,
manufacturingandvideosurveillance. Theaimofobjectrecognitionistoidentifyobjects
in a scene and estimate their pose. An important paradigm of object recognition is to
first define suitable surface representations, offline, for 3D objects and save those repre-
sentations in a database. During online recognition, a similar representation of a scene
is matched with the representations stored in the database to recognize objects which are
present in the scene. The main challenges associated with surface representation and 3D
objectrecognitionareocclusionscausedbythepresenceofmultipleobjectsinthescene,
clutterduetounwantedobjects,androbustnesstonoiseandresolution.
This dissertation addresses the aforementioned challenges and investigates novel sur-
facerepresentations,matchingandautomaticfeaturelearningtechniquesforobjectrecog-
nition. Inthisthesis,threedifferentsurfacerepresentations,akeypointdetector,automatic
feature matching and recognition algorithms as well as two different automatic feature
learningtechniquesarepresented.
The first part of the thesis presents two different local surface representations based
on the analysis of the 3D vector field. The divergence and vorticity of the vector field
have been exploited to construct two different features named 3D-Div and 3D-Vor. To
achieve invariance to rigid transformations, each representation is defined on an object
centeredlocalreferenceframe. Inadditiontotheaforementionedsurfacerepresentations,
this dissertation also presents a keypoint-based feature-free representation for 3D mod-
eling and object recognition. Instead of computing the local feature around a given 3D
keypoint,theproposedtechniquemeasuresthegeometricalrelationshipsbetweenthede-
tected keypoints for surface representation. The proposed representation is found to be
computationally efficient compared to existing feature based methods. This thesis also
presentsnovelalgorithmsforrangeimageregistration,3Dkeypointdetectionand3Dob-
jectrecognitionincomplexscenesinthepresenceofocclusionsandclutter. Thepresented
methods achieve superior performance compared to existing techniques when tested on
publiclyavailabledatasetsincludingthelowresolutionWashingtonRGB-Dobject,UWA,
Bologna,Stanford3DmodelandCa’Foscaridatasets.
Thelastpartofthisdissertationinvestigatestwonovelautomaticfeaturelearningtech-
niquesthroughextensiveexperimentsusingpubliclyavailableobject/facedatasets. Inthe
first technique, an Iterative Deep Learning Model (IDLM) is presented. IDLM consists
of pooled convolutional layer followed by artificial neural networks applied in a hierar-
chical fashion to automatically learn discriminative representations from raw face and
object images. The proposed deep learning framework is extensively tested on four pub-
licly available object and face datasets and achieves superior performance compared to
existingmethods. Inthesecondproposedtechnique,dubbedEvolutionaryFeatureLearn-
ing (EFL), evolutionary algorithms are exploited for 3D object recognition. In the EFL
process, irrelevant and redundant features are omitted: only the features that describe the
object in the most discriminative manner are selected during the evolutionary process.
The proposed technique automatically optimizes the candidate solution based on the fit-
ness function and selects the best feature for superior object recognition. The proposed
approach was extensively evaluated on four challenging object datasets achieving supe-
riorperformancecomparedtoexistingtechniquesincludingdeeplearninganddictionary
learningbasedmethods.
Acknowledgements
I am thankful to God for giving me the opportunity and strength to complete my
Ph.D. I am also grateful to many people whose help and support made this journey pos-
sible. First and foremost, my sincere gratitude to my parents for their love and support
throughout my life. All I have and will accomplish is only possible due to their prayers
and sacrifices. I would also like to thank my wife and kids for their patience during the
courseofmyPh.D.Withoutmywife’ssupport,Iwouldnothavehadthepeaceofmindre-
quiredforresearch. Iamalsogratefultomysisterwhokeptmemotivatedbyreinforcing
myenthusiasm.
I would like to express my deepest gratitude to my supervisors Mohammed Ben-
namoun and Farid Boussaid who taught me great skills of conducting hight quality re-
search. Their continuous guidance and support were always a source of inspiration for
me. They created a motivating, enthusiastic and friendly environment which is ideal for
research. Theirinsightfulfeedbackandvaluablesuggestionsgreatlyimprovedthequality
ofmywork. TheirquickresponseonmydraftswasthekeytofinishingmyPhDontime.
I am also grateful to Amar A. El-Sallam for his co-supervision in the first two years of
myPhD.
IamalsogratefultoAjmalSaeedMianfortheusefuldiscussionswehadregarding3D
mesh processing and object recognition. I also had useful discussions with other Ph.D.
candidates and research fellows including Yulan Guo, Senjian An and Arif Mehmood.
I would also like to thank the anonymous reviewers whose constructive criticism and
feedbackhelpedmeimprovemypapersandsubsequentwork.
IamthankfultoRyan,LaurieMcKeaig,SamuelandotherstaffmembersintheCom-
puter Science Support Group for their technical assistance. I especially thank Yvette
Harrapformakingsomanydifficultadministrativetasksveryeasyforme. Thankstomy
lab-matesZohaib,Uzair,Naveed,ZulqarnainandHassanforthefunandsupport.
I would like to acknowledge the contribution of the external research groups and uni-
versities for making their data publicly available for research, including University of
Washington(RGB-D object dataset), Universityof Bologna (3D object dataset) and Uni-
versity Ca’ Foscari Venezia (3D object dataset). I would like to thank Bogdan Alexe,
KevinLai,TalDaromandJanKnoppforsharingthesourcecodeoftheiralgorithms.
In the end, I would acknowledge the financial and logistic supports obtained through
the grants DP110102166 and DP150100294 of Australian Research Council (ARC) and
Scholarship for International Research Fee (SIRF), UWA Top-Up Scholarship and Re-
search Training Scheme (RTS) offered to the candidates by the University of Western
Australia.

i
Contents
Abstract
ListofTables vii
ListofFigures ix
PublicationsPresentedinthisThesis xi
ContributionofCandidatetoPublishedPapers xiii
1 Introduction 1
1.1 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 ResearchContributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Structureofthethesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3.1 3D-Div: ANovelLocalSurfaceFeature(Chapter2) . . . . . . . 6
1.3.2 A Novel Representation and Automatic Correspondence Algo-
rithm(Chapter3) . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3.3 Novel3DKeypointDetectionandObjectRecognitionAlgorithm
(Chapter4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3.4 Keypoints-basedFeature-FreeRepresentation(KFFR)(Chapter5) 8
1.3.5 IterativeDeepLearningforObjectRecognition(Chapter6) . . . 8
1.3.6 LearningEvolutionaryFeaturesfor3DObjectRecognition(Chap-
ter7) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.3.7 Conclusion(Chapter8) . . . . . . . . . . . . . . . . . . . . . . . 9
2 A Novel Local Surface Description for Automatic 3D Object Recognition in
LowResolutionClutteredScenes 11
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.3 ProposedMethodology . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.3.1 3D-DivRepresentation . . . . . . . . . . . . . . . . . . . . . . . 14
2.3.2 LRFConstruction . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.3.3 3DVectorField . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.3.4 3D-DivDescriptor . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4 3DObjectRecognition . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.4.1 OfflinePhase . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
ii
2.4.2 OnlineRecognitionPhase . . . . . . . . . . . . . . . . . . . . . 17
2.4.3 3DObjectDetection . . . . . . . . . . . . . . . . . . . . . . . . 17
2.4.4 FeatureExtractionandMatching . . . . . . . . . . . . . . . . . . 19
2.5 ExperimentalResults . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.6 PairwiseRangeImageRegistration . . . . . . . . . . . . . . . . . . . . . 21
2.7 ExperimentalResults . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3 ANovel3DVorticitybasedApproachforAutomaticRegistrationofLowRes-
olutionRangeImages 25
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.1.1 IntuitiveInterpretationofVorticity . . . . . . . . . . . . . . . . . 27
3.1.2 PaperContributions . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.1.3 Organizationofthepaper . . . . . . . . . . . . . . . . . . . . . . 29
3.2 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.3 LocalSurfaceDescription . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.3.1 VectorField . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.3.2 3D-VorDescriptor . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.3.3 3D-VorSelectionParameter . . . . . . . . . . . . . . . . . . . . 35
3.4 AutomaticPairwiseCorrespondenceandRegistration . . . . . . . . . . . 36
3.4.1 FeatureCorrespondence . . . . . . . . . . . . . . . . . . . . . . 37
3.4.2 LocalValidation . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.5 ExperimentalResults . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.5.1 QualitativeAnalysis . . . . . . . . . . . . . . . . . . . . . . . . 40
3.5.2 QuantitativeAnalysis . . . . . . . . . . . . . . . . . . . . . . . . 40
3.6 ComparisonwithState-of-the-art . . . . . . . . . . . . . . . . . . . . . . 46
3.7 MultiviewRangeImageRegistration(3DModeling) . . . . . . . . . . . 48
3.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4 A Novel Feature Representation for Automatic 3D Object Recognition in
ClutteredScenes 51
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.1.1 PaperContributions . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.1.2 Organizationofthepaper . . . . . . . . . . . . . . . . . . . . . . 55
4.2 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
4.2.1 KeypointDetection . . . . . . . . . . . . . . . . . . . . . . . . . 56
4.2.2 LocalFeatureDescriptors . . . . . . . . . . . . . . . . . . . . . 57