Fast and accurate image registration. Applications to on-board satellite imaging.

261 Pages·2017·22.93 MB·French

by  Martin Rais

Fast and accurate image registration. Applications to
on-board satellite imaging.
Martin Rais
To cite this version:
Martin Rais. Fast and accurate image registration. Applications to on-board satellite imaging..
General Mathematics [math.GM]. Université Paris Saclay (COmUE); Universitat de les Illes Balears
(@Université des iles Baléares), 2016. English. ￿NNT: 2016SACLN077￿. ￿tel-01485321￿
HAL Id: tel-01485321
https://theses.hal.science/tel-01485321
Submitted on 8 Mar 2017
HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est
archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents
entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non,
lished or not. The documents may come from émanant des établissements d’enseignement et de
teaching and research institutions in France or recherche français ou étrangers, des laboratoires
abroad, or from public or private research centers. publics ou privés.
NNT : 2016SACLN077
T D
HESE DE OCTORAT
DE
UNIVERSITAT DE LES ILLES BALEARS
et de
L’UNIVERSITÉ PARIS-SACLAY
PRÉPARÉE A
L’ÉCOLE NORMALE SUPÉRIEURE DE CACHAN
(ÉCOLE NORMALE SUPÉRIEURE PARIS-SACLAY)
ÉCOLE DOCTORALE N°574
Ecole doctorale de mathématiques Hadamard
Spécialité de doctorat : Mathématiques appliquées
Par
M. Martin Emilio Rais
FAST AND ACCURATE IMAGE REGISTRATION.
APPLICATIONS TO ON-BOARD SATELLITE IMAGING.
Thèse présentée et soutenue à Palma de Majorque, le 9 décembre 2016 :
Composition du Jury :
Mme C. Ballester, Professeure, Universitat Pompeu Fabra, Présidente du Jury
M.Pascal Monasse, Professeur, Ecole des Ponts ParisTech, Rapporteur
M.Frédéric Falzon, Directeur de Recherche, Thales Alenia Space, Rapporteur
M.Vasileios Argyriou, Professeur, Kingston University, Examinateur
M.Vincent Michau, Directeur de Recherche, ONERA, Examinateur
M.J.-M. Morel, Professeur, ENS Paris-Saclay, Directeur de thèse
M.Bartomeu Coll Vicens, Professeur, UIB, Co-directeur de thèse

i
AmihermanoAndrés.
i

Acknowledgements
Mymaster’sadvisorfromBuenosAires, MartaMejail, togetherwithsomegoodfriends
ofmine,NorbertoGoussiesandFranciscoGomezFernandez,introducedmetotheworld
of image processing. It was when I met Jean-Michel Morel that my passion for research
started. Hishardwork,motivation,geniusandmostlytheloveforwhathedoeshasbeen
a constant source of inspiration all along my PhD. I don’t have enough words to thank
himforwhathedidduringthelastyears. ThesamegoestomyotheradvisorBartomeu
Coll. Apart from everything he has helped me from an academical point of view, his
warmth,humanvalues,simplicityandhumilityhavebeenveryimportanttome. Ihave
tomentionaswelltheexcellentbeerhebrews.
IwouldalsoliketothanktheimagequalitygroupfromCNES,inparticularPhilippe
Kubik, Gwendoline Blanchet, Jean-Marc Delvit, Christophe Latry and Carole Thiebaut,
forhavingproposedseveraltopicsofthisPhDthesisaswellashavingcollaboratedwith
fruitfuldiscussionsandprovidingimagesimulationsanddatasets.
MygratitudegoesaswelltothethesisreviewersFrédéricFalzonandPascalMonasse,
fortheirfruitfulremarksandfortakingthetimetoreadthisextensivemanuscript. Also,
for the members of the jury Vincent Michau, Vasileios Argyriou, Jirˇí Matas, Coloma
Ballester,JoseLuisLisaniandCatalinaSbertforhavingacceptedtobepartoftheevalu-
ationboard.
In the CMLA, I met some wonderful people. I begin by thanking the secretaries
Micheline, Veronique and Virginie for all the help they gave me. I would also like to
thankmyfriendsatwork: Ives,elMatu,CarlitosandRafa,withwhomIspentlotsofdis-
cussions,funnytalks,beersandgoodtimestogether. IwouldliketothankaswellCarlo,
Miguel,AxelandMauricio,someamazingresearchersIfoundhere. Andfinally,Iwantto
thankparticularlytwoguysthatstronglycontributedwiththisdissertation: Gabrieleand
Enric. Their guidance, patience, methodology and motivation have strongly influenced
mealongmyscientificcareer,andtogetherwithJM,wereconstantsourcesofmotivation.
Yourhelpwasinvaluabletome.
In the TAMI group, I also met excellent researchers. I would like to begin by thank-
ing Toni for all the help he gave me during this thesis. His perseverance as well as his
expertise in the research community have undoubtedly contributed with my academic
growth. I would also like to thank the rest of the people in the group: Ana Belen, Joan
andJulia.
IwouldhaveneverbeenabletofinishthisPhDwithoutthesupportofmyfriends. In
France I met some amazing people. Hernan was my partner during my first years here,
and a great friend. Aleksandra turned fast into my sister, and together with Claudia
andHernan,weremybackboneinthefirstyearsofthisjourneyandmademyeveryday
life much happier. I also have to include my Argentinean friends I made here: Alito, la
chiqui,lachueka,lapetiandLala.
Talkingaboutfriends,ImettwootherparticularindividualswithwhomIsharedmy
every day life and that will stay in my heart forever. Larucha is the best that Uruguay
haseverproduced(thankstolaPetruzi,thesecondbest). Indeed,ittooklessthanayear
iii
iv
for her to become “my wife”. We have spent so much time together that the nickname
seemed just right. She was also part of my backbone, an excellent friend and researcher
andprobably,oneofthereasonsIsucceededtolivefiveyearsabroad. Thislistwouldn’t
becompletewithoutNicolino. WebegantoknoweachotherwhenwetravelledtoAus-
tralia for ICIP, and since then he became more and more important in my life, together
with Sandra. Besides being the best ever, he is an amazing person and a great friend. It
seemsthatthecakewasnotalieafterall.
I would also like to thank my friends from Argentina. This includes Fede, el negro,
elblanco, elpelado, camisa, Jr, Jinkis, JonyandZaina. Iregretnotbeingabletobethere
for some of you in some situations, but even without communicating so often, you are
amazing, and own a really important part of my heart. This obviously extends to “las
chicas”: brujisina,veci,caskis,queipusiandlaprimita,andincludesSofi,AyeandAgata
as well. You are the best company this manga de boludos will ever find. What can I
say about the newcomers? Of course I am dying to share more time with la mumita
and malenuchi, I wish I could see them often. Seeing a simple picture of them growing
brightensupmyday.
Iwouldalsoliketothankthepeoplethathavebeenwithmeandmyfamilyallalong
thisjourney: mitíaHilda,mitíoGo,AxiyLean,MiguelySarita,Diego,BrendayErnest,
eltíoEduardoylatíaElida,Luciana,Hernan,NatiyPablo,eltioJorgeylatíaDiana.
Finally, I have to thank the five most important people in my life. First is my girl-
friendVictoria. Notonlyfortoleratingmysnoring,myarrivinglatebecauseofthethesis,
amongmanyother,butalsobecauseofalltheloveandsupportshegivesmeeveryday.
Also,tomygrandmalaNani,whichImissedalotduringalltheseyears. MyparentsRita
andJorgewerethemainreasonIwasabletostartandtofinishthisthesis. Theirsupport,
education, constant caring, ethical values, strong commitment to work, and their love
made me what I am today. At last, my brother Andrés, who has been my main inspi-
ration to fight and overcome all difficulties. I miss you a lot. I love you with all my
heart.
Lastbutnotleast,IwouldliketothanktheMinisteriodeEconomíayCompetitividad
fortheFPIscholarshipassociatedwiththeprojectTIN2011-27539.
iv
Contents
1 Introduction 15
1.1 Chapter2: ReviewofGlobalSubpixelShiftEstimationMethods . . . . . . 16
1.2 Chapter3: ImprovingwavefrontsensingwithaShackHartmanndevice . 19
1.3 Chapter4: Stab-Active: stabilizingonboardimageaccumulation . . . . . 21
1.4 Chapter5: RANSAAC:RANdomSAmpledAggregatedConsensus . . . . 22
1.5 Summaryofcontributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
1.6 Listofpublications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2 ReviewofGlobalSubpixelShiftEstimationMethods 29
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.1.1 Subpixelshiftestimationapproaches . . . . . . . . . . . . . . . . . 30
2.1.2 Chaptersummary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.2 Gradient-BasedShiftEstimationmethods . . . . . . . . . . . . . . . . . . . 32
2.2.1 OpticalFlowequationwithLeastSquaresminimization . . . . . . 33
2.2.2 OpticalFlowEquationwithTotalLeastSquaresMinimization . . . 33
2.2.3 Biasminimizationthroughiterativeandmultiscalegradient-based
shiftestimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.2.4 Minimizingthebiasthroughcorrectedgradientestimation . . . . . 38
2.2.5 BidirectionalbiascorrectionforGradient-BasedShiftEstimation . 38
2.2.6 Gradientcomputationandimageprefiltering. . . . . . . . . . . . . 41
2.2.7 Interpolationmethodsforimageresampling . . . . . . . . . . . . . 43
2.2.8 Shiftestimationbycentroidofinterpolatingkernel . . . . . . . . . 45
2.3 Phase-correlationmethods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.3.1 Localfunctionfittinginthespatialdomain . . . . . . . . . . . . . . 47
2.3.2 Analyzingthephasedifferencematrixinthefrequencydomain . . 51
2.3.3 Subspacephasecorrelationmethods . . . . . . . . . . . . . . . . . . 53
2.3.4 Gradientcorrelationmethods . . . . . . . . . . . . . . . . . . . . . . 56
2.4 Methodevaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
2.4.1 InfluenceofgradientestimationonGBSEmethods . . . . . . . . . 62
2.4.2 Robustnesstonoise . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
2.4.3 Robustnesstoviolationofthebrightnessconstancyconstraint . . . 76
2.4.4 Robustnesstoaliasedsampleprocesses . . . . . . . . . . . . . . . . 77
2.4.5 Computationalcostcomparison . . . . . . . . . . . . . . . . . . . . 79
2.4.6 EvaluationonrealMRIimages . . . . . . . . . . . . . . . . . . . . . 81
2.5 ConcludingRemarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
3 ImprovingwavefrontsensingwithaShack-Hartmanndevice 87
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
3.1.1 Wavefrontsensingfromearth-observationsatellites . . . . . . . . . 89
3.2 State-of-the-artreview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
v
vi CONTENTS
3.2.1 Correlation-basedMethods . . . . . . . . . . . . . . . . . . . . . . . 90
3.2.2 PhaseCorrelation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
3.2.3 IteratedEstimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
3.2.4 MaximumLikelihoodEstimator . . . . . . . . . . . . . . . . . . . . 95
3.3 AccurateshiftestimationusingopticalflowinthecontextofaSHWFS . . 95
3.3.1 Iterativegradient-basedshiftestimator . . . . . . . . . . . . . . . . 96
3.3.2 Gradientcomputationandimageprefiltering. . . . . . . . . . . . . 97
3.3.3 Interpolationmethodsforimageresampling . . . . . . . . . . . . . 98
3.3.4 Imageintensitiesequalization . . . . . . . . . . . . . . . . . . . . . . 98
3.3.5 Multiscaleimplementation . . . . . . . . . . . . . . . . . . . . . . . 100
3.4 Scenepreselectionandrobustnessestimation . . . . . . . . . . . . . . . . . 100
3.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
3.5.1 Scenepre-selectionevaluation . . . . . . . . . . . . . . . . . . . . . 103
3.5.2 Comparisonwithstate-of-the-artmethods . . . . . . . . . . . . . . 103
3.6 ConcludingRemarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
4 Stab-Active: stabilizingonboardimageaccumulation 109
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
4.2 Methodanalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
4.3 InfluenceofNoiseontheRegistration . . . . . . . . . . . . . . . . . . . . . 111
4.3.1 Noiseestimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
4.3.2 Predictingtheerrorontheestimatedshiftcausedbythenoise . . . 112
4.3.3 Cramer-Raolowerboundsforimageregistrationofseveralimages
withuniformtranslation . . . . . . . . . . . . . . . . . . . . . . . . . 114
4.3.4 ConditionsbasedontheSNRoftheimages . . . . . . . . . . . . . . 116
4.4 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
4.4.1 Gradient-basedshiftestimationalgorithmfornoisyimages . . . . 117
4.4.2 Automaticcalculationofpforeachline . . . . . . . . . . . . . . . . 120
4.4.3 Complexityoftheanalyzedalgorithms . . . . . . . . . . . . . . . . 121
4.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
4.5.1 Impactoftemporalconvolutiononimagenoise . . . . . . . . . . . 123
4.5.2 NumericalResults . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
4.5.3 VisualResults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
4.5.4 ProposedAlgorithms . . . . . . . . . . . . . . . . . . . . . . . . . . 139
4.6 ConcludingRemarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
5 RANSAAC:RANdomSAmpleAggregatedConsensus 149
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
5.1.1 RANSACiterationsandmodelerror . . . . . . . . . . . . . . . . . . 150
5.1.2 Distanceparameter . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
5.1.3 Finalrefinementstep . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
5.1.4 RANSACweaknesses . . . . . . . . . . . . . . . . . . . . . . . . . . 152
5.2 RANSACAlternativesorImprovements . . . . . . . . . . . . . . . . . . . . 153
5.3 RandomSampleAggregatedConsensus . . . . . . . . . . . . . . . . . . . . 155
5.3.1 Aggregationof2Dparametrictransformations . . . . . . . . . . . . 156
5.3.2 AggregationofEstimates . . . . . . . . . . . . . . . . . . . . . . . . 159
5.3.3 LocalOptimizationanditsvariants . . . . . . . . . . . . . . . . . . 160
5.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
5.4.1 QualitativeEvaluation: PaintingsRegistrationwithProjectiveTrans-
formations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
5.4.2 QuantitativeEvaluationbyaSimulatedGroundTruth. Application
1: EstimatingProjectiveTransforms. . . . . . . . . . . . . . . . . . . 164
vi
CONTENTS vii
5.4.3 Application2: EstimatingHomography+Distortion . . . . . . . . . 182
5.5 ConcludingRemarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
6 Conclusions 187
A Fast Interpolation Methods. Focus on Image Resampling for Shift Estimation
onSatelliteImages. 189
A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
A.1.1 Problemstatement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
A.1.2 Interpolationfunctionandinterpolationkernels . . . . . . . . . . . 190
A.1.3 Interpolationkernelcharacteristics . . . . . . . . . . . . . . . . . . . 190
A.2 InterpolationMethods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
A.2.1 IdealInterpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
A.2.2 NearestNeighborInterpolation . . . . . . . . . . . . . . . . . . . . . 195
A.2.3 BilinearInterpolation. . . . . . . . . . . . . . . . . . . . . . . . . . . 195
A.2.4 QuadraticInterpolation . . . . . . . . . . . . . . . . . . . . . . . . . 195
A.2.5 BicubicInterpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
A.2.6 WindowedSincApproximations . . . . . . . . . . . . . . . . . . . . 197
A.2.7 B-Splines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
A.2.8 o-Moms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
A.2.9 Schaum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
A.2.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
A.3 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
A.3.1 Separablekernel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
A.3.2 Imagetranslationalgorithm . . . . . . . . . . . . . . . . . . . . . . . 203
A.3.3 ImagePrefiltering. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
A.3.4 FourierInterpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
A.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
A.4.1 Quantitativeresults. . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
A.4.2 Qualitativeresults . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
A.4.3 ExecutionTime . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
A.4.4 Combiningtimeandapproximationerror . . . . . . . . . . . . . . . 225
B FastAnscombeVSTimplementationforfastonboardnoisestabilization 229
C Algorithmsforgradient-basedshiftestimationusedintheStab-Activeapplica-
tion 231
vii