Monte Carlo methods

433 Pages·2020·8.625 MB·English

by  Barbu A., Zhu S.-C

Adrian Barbu · Song-Chun Zhu
Monte Carlo
Methods
Monte Carlo Methods
Adrian Barbu • Song-Chun Zhu
Monte Carlo Methods
AdrianBarbu Song-ChunZhu
DepartmentofStatistics DepartmentsofStatisticsandComputerScience
FloridaStateUniversity UniversityofCalifornia,LosAngeles
Tallahassee,FL,USA LosAngeles,CA,USA
ISBN978-981-13-2970-8 ISBN978-981-13-2971-5 (eBook)
https://doi.org/10.1007/978-981-13-2971-5
©SpringerNatureSingaporePteLtd.2020
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformationstorage
andretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodologynowknownor
hereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublicationdoes
notimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevantprotective
lawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthors,andtheeditorsaresafetoassumethattheadviceandinformationinthisbookare
believedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsortheeditors
giveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforanyerrorsoromissions
thatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictionalclaimsinpublishedmaps
andinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSingaporePteLtd.
Theregisteredcompanyaddressis:152BeachRoad,#21-01/04GatewayEast,Singapore189721,Singapore
Preface
Real-world systems studied in sciences (e.g., physics, chemistry, and biology) and
engineering (e.g., vision, graphics, machine learning, and robotics) involve complex
interactionsbetweenlargenumbersofcomponents.Therepresentationsforsuchsystems
areprobabilisticmodelsdefinedongraphsinhigh-dimensionalspaces,towhichanalytic
solutions are often unavailable. As a result, Monte Carlo methods have been used
as a common tool for simulation, estimation, inference, and learning in science and
engineering. It is no surprise that the Metropolis algorithm topped the list of ten most
frequently used algorithms for practice of sciences in the twentieth century (Dongarra
andSullivan2000).Withtheever-growingcomputingcapacities,researchersaretackling
morecomplexproblemsandadoptingmoreadvancedmodels.MonteCarlomethodswill
continue to play important roles in the development of sciences and engineering in the
twenty-first century. The recent use of Hamiltonian Monte Carlo and Langevin Monte
Carlo in developing stochastic gradient descent algorithm in deep learning is another
exampleofthistrend.
In history, several communities have contributed to the evolution of Monte Carlo
methods.
• Physicsandchemistry:theearlyMetropolisalgorithm(Metropolis,Rosenbluth,Rosen-
bluth, Teller, and Teller 1953), simulated annealing (Kirkpatrick, Gelatt, and Vecchi
1983), cluster sampling (Swendsen and Wang, 1987; Edwards and Sokal 1988), and
recent work on disconnectivity graphs (Becker and Karpus 1997) for visualizing the
landscapeofspin-glassmodels.
• Probabilityandstatistics:stochasticgradient(RobinandMonro1951,Younes1988),
Hastings dynamics (Hastings 1970), data augmentation (Tanner and Wong 1987),
reversible jumps (Green 1995), and dynamic weighting (Wong and Liang 1997) for
studying bioinformatics and numerous analyses for bounding the convergence of
MarkovchainMonteCarlo(Diaconis1988;DiaconisandStroock1991;andLiu1991).
• Theoreticalcomputerscience:theconvergencerateofclusteringsampling(Jerrumand
Sinclair1989,CooperandFrieze1999).
• Computer vision and pattern theory: Gibbs sampler (Geman and Geman, 1984) for
image processing, jump diffusion (Miller and Grenander 1994) for segmentation,
v
vi Preface
the condensation or particle filtering algorithm for object tracking (Isard and Blake
1996),recentworkondata-drivenMarkovchainMonteCarlo(TuandZhu2002),and
generalized Swendsen-Wang cuts (Barbu and Zhu 2005) for image segmentation and
parsing.
Consideringthatthesediverseareasspeakdifferentlanguages,interdisciplinarycommu-
nicationhasbeenrare.Thisposesaconsiderablechallengeforpractitionersincomputer
scienceandengineeringwhodesiretouseMonteCarlomethods.
Inonerespect,effectiveMonteCarloalgorithmsmustexploretheunderlyingstructures
ofaproblemdomain,andtheyarethusdomainorproblemspecificandhardlyaccessible
to outsiders. For example, many important works in physics, like Swendsen and Wang
(1987),areonly2–3pageslongandincludenobackgroundorintroduction,causingthem
toappearutterlymysterioustocomputerscientistsandengineers.
On the other hand, general or domain agnostic Monte Carlo algorithms invented
by statisticians are well-explained but are usually found not very effective when they
are implemented in generic ways by engineers without utilizing the structures of the
underlying models and representations. As a result, there is a widespread misconception
amongscientistsandgraduatestudentsthatthesemethodsaretooslowandusuallydonot
work.ThisisunfairtotheMonteCarlomethodsandunfortunatefortheinnocentstudents.
This textbook is written for researchers and graduate students in statistics, computer
science,andengineering,basedonmaterialsanddraftversionstaughtbytheseniorauthor
intheDepartmentofStatisticsandComputerScienceattheUniversityofCalifornia,Los
Angeles,inthepast10years.ItcoversabroadrangeoftopicsinMonteCarlocomputing
withboththeoreticalfoundationsandintuitiveconceptsdevelopedinthefourcommunities
above while omitting small tricks which are less applicable or do not work in practice.
It illustrates the art of Monte Carlo design using classical problems in computer vision,
graphics, and machine learning and thus can be used as a reference book by researchers
in computer vision and pattern recognition, machine learning, graphics, robotics, and
artificialintelligence.
The authors would like to thank many current and former PhD students at the UCLA
for their contributions to this book. Mitchell Hill contributed Chaps.9, 10, and 11 based
on his thesis work which has enriched the contents of the book. Zachary Stokes worked
on polishing many details in the manuscript. Maria Pavlovskaia, Kewei Tu, Zhuowen
Tu, Jacob Porway, Tianfu Wu, Craig Yu, Ruiqi Gao, and Erik Nijkamp have contributed
materials and figures as examples. Two UCLA colleagues, Professors Yingnian Wu and
QingZhou,havegiveninvaluablecommentsforimprovingthebook.
TheauthorswouldalsoliketoacknowledgethesupportofDARPA,ONRMURIgrants,
andNSFduringthedevelopmentofthisbook.
Tallahassee,FL,USA AdrianBarbu
LosAngeles,CA,USA Song-ChunZhu
September,2018
Contents
1 IntroductiontoMonteCarloMethods ......................................... 1
1.1 Introduction................................................................. 1
1.2 MotivationandObjectives.................................................. 2
1.3 TasksinMonteCarloComputing.......................................... 3
1.3.1 Task1:SamplingandSimulation ................................ 4
1.3.2 Task2:EstimatingQuantitiesbyMonteCarloSimulation ..... 7
1.3.3 Task3:OptimizationandBayesianInference................... 9
1.3.4 Task4:LearningandModelEstimation......................... 11
1.3.5 Task5:VisualizingtheLandscape .............................. 12
References.......................................................................... 16
2 SequentialMonteCarlo .......................................................... 19
2.1 Introduction................................................................. 19
2.2 Samplinga1-DimensionalDensity........................................ 19
2.3 ImportanceSamplingandWeightedSamples............................. 20
2.4 SequentialImportanceSampling(SIS).................................... 24
2.4.1 Application:TheNumberofSelf-AvoidingWalks.............. 25
2.4.2 Application:ParticleFilteringforTrackingObjects
inaVideo.......................................................... 28
2.4.3 SummaryoftheSMCFramework ............................... 31
2.5 Application:RayTracingbySMC......................................... 33
2.5.1 Example:GlossyHighlights...................................... 34
2.6 PreservingSampleDiversityinImportanceSampling.................... 35
2.6.1 ParzenWindowDiscussion....................................... 39
2.7 MonteCarloTreeSearch................................................... 41
2.7.1 PureMonteCarloTreeSearch ................................... 42
2.7.2 AlphaGo........................................................... 44
2.8 Exercises .................................................................... 46
References.......................................................................... 48
vii
viii Contents
3 MarkovChainMonteCarlo:TheBasics ....................................... 49
3.1 Introduction................................................................. 50
3.2 MarkovChainBasics....................................................... 50
3.3 TopologyofTransitionMatrix:CommunicationandPeriod............. 53
3.4 ThePerron-FrobeniusTheorem............................................ 56
3.5 ConvergenceMeasures..................................................... 58
3.6 MarkovChainsinContinuousorHeterogeneousStateSpaces .......... 61
3.7 ErgodicityTheorem ........................................................ 62
3.8 MCMCforOptimizationbySimulatedAnnealing ....................... 62
3.8.1 PageRankExample............................................... 65
3.9 Exercises .................................................................... 67
References.......................................................................... 70
4 MetropolisMethodsandVariants ............................................... 71
4.1 Introduction................................................................. 71
4.2 TheMetropolis-HastingsAlgorithm....................................... 72
4.2.1 TheOriginalMetropolis-HastingsAlgorithm ................... 72
4.2.2 AnotherVersionoftheMetropolis-HastingsAlgorithm........ 74
4.2.3 OtherAcceptanceProbabilityDesigns........................... 74
4.2.4 KeyIssuesinMetropolisDesign................................. 75
4.3 TheIndependenceMetropolisSampler.................................... 75
4.3.1 TheEigenstructureoftheIMS ................................... 77
4.3.2 GeneralFirstHittingTimeforFiniteSpaces ................... 78
4.3.3 HittingTimeAnalysisfortheIMS .............................. 78
4.4 ReversibleJumpsandTrans-DimensionalMCMC ....................... 80
4.4.1 ReversibleJumps.................................................. 80
4.4.2 ToyExample:1DRangeImageSegmentation .................. 81
4.5 Application:CountingPeople.............................................. 85
4.5.1 MarkedPointProcessModel..................................... 85
4.5.2 InferencebyMCMC.............................................. 86
4.5.3 Results............................................................. 87
4.6 Application:FurnitureArrangement ...................................... 88
4.7 Application:SceneSynthesis .............................................. 90
4.8 Exercises .................................................................... 95
References.......................................................................... 96
5 GibbsSamplerandItsVariants.................................................. 97
5.1 Introduction................................................................. 97
5.2 GibbsSampler .............................................................. 99
5.2.1 AMajorProblemwiththeGibbsSampler....................... 101
Contents ix
5.3 GibbsSamplerGeneralizations ............................................ 102
5.3.1 Hit-and-Run....................................................... 103
5.3.2 GeneralizedGibbsSampler ...................................... 103
5.3.3 GeneralizedHit-and-Run......................................... 104
5.3.4 SamplingwithAuxiliaryVariables............................... 105
5.3.5 SimulatedTempering ............................................. 105
5.3.6 SliceSampling .................................................... 106
5.3.7 DataAugmentation ............................................... 107
5.3.8 MetropolizedGibbsSampler..................................... 108
5.4 DataAssociationandDataAugmentation ................................ 110
5.5 JuleszEnsembleandMCMCSamplingofTexture....................... 112
5.5.1 TheJuleszEnsemble:AMathematicalDefinition
ofTexture.......................................................... 112
5.5.2 TheGibbsEnsembleandEnsembleEquivalence ............... 115
5.5.3 SamplingtheJuleszEnsemble ................................... 116
5.5.4 Experiment:SamplingtheJuleszEnsemble..................... 117
5.6 Exercises .................................................................... 119
References.......................................................................... 120
6 ClusterSamplingMethods ....................................................... 123
6.1 Introduction................................................................. 123
6.2 PottsModelandSwendsen-Wang ......................................... 124
6.3 InterpretationsoftheSWAlgorithm ...................................... 127
6.3.1 Interpretation1:Metropolis-HastingsPerspective .............. 128
6.3.2 Interpretation2:DataAugmentation............................. 131
6.4 SomeTheoreticalResults .................................................. 135
6.5 Swendsen-WangCutsforArbitraryProbabilities ........................ 137
6.5.1 Step1:Data-DrivenClustering .................................. 138
6.5.2 Step2:ColorFlipping ............................................ 139
6.5.3 Step3:AcceptingtheFlip........................................ 140
6.5.4 ComplexityAnalysis.............................................. 142
6.6 VariantsoftheClusterSamplingMethod ................................. 142
6.6.1 ClusterGibbsSampling:The“Hit-and-Run”Perspective ...... 143
6.6.2 TheMultipleFlippingScheme................................... 144
6.7 Application:ImageSegmentation ......................................... 145
6.8 MultigridandMulti-levelSW-cut ......................................... 148
6.8.1 SW-CutsatMultigrid ............................................ 150
6.8.2 SW-cutsatMulti-level ........................................... 152
6.9 SubspaceClustering........................................................ 153
6.9.1 SubspaceClusteringbySwendsen-WangCuts.................. 155
6.9.2 Application:SparseMotionSegmentation ...................... 158
x Contents
6.10 C4:ClusteringCooperativeandCompetitiveConstraints................ 163
6.10.1 OverviewoftheC4Algorithm................................... 165
6.10.2 Graphs,Coupling,andClustering ............................... 166
6.10.3 C4AlgorithmonFlatGraphs .................................... 172
6.10.4 ExperimentsonFlatGraphs ..................................... 175
6.10.5 CheckerboardIsingModel ....................................... 176
6.10.6 C4onHierarchicalGraphs ...................................... 181
6.10.7 ExperimentsonHierarchicalC4 ................................. 183
6.11 Exercises .................................................................... 184
References.......................................................................... 186
7 ConvergenceAnalysisofMCMC................................................ 189
7.1 Introduction................................................................. 189
7.2 KeyConvergenceTopics ................................................... 189
7.3 PracticalMethodsforMonitoring ......................................... 191
7.4 CouplingMethodsforCardShuffling..................................... 193
7.4.1 ShufflingtotheTop............................................... 193
7.4.2 RiffleShuffling.................................................... 194
7.5 GeometricBounds,BottleneckandConductance......................... 196
7.5.1 GeometricConvergence .......................................... 196
7.6 Peskun’sOrderingandErgodicityTheorem............................... 200
7.7 PathCouplingandExactSampling........................................ 201
7.7.1 CouplingFromthePast........................................... 202
7.7.2 Application:SamplingtheIsingModel.......................... 203
7.8 Exercises .................................................................... 205
References.......................................................................... 209
8 DataDrivenMarkovChainMonteCarlo ...................................... 211
8.1 Introduction................................................................. 211
8.2 IssueswithSegmentationandIntroductiontoDDMCMC ............... 211
8.3 SimpleIllustrationoftheDDMCMC...................................... 213
8.3.1 DesigningMCMC:TheBasicIssues ............................ 216
8.3.2 ComputingProposalProbabilitiesintheAtomicSpaces:
AtomicParticles................................................... 217
8.3.3 ComputingProposalProbabilitiesinObjectSpaces:
ObjectParticles ................................................... 219
8.3.4 ComputingMultiple,DistinctSolutions:SceneParticles....... 220
8.3.5 TheΨ-WorldExperiment ........................................ 221
8.4 ProblemFormulationandImageModels ................................. 223
8.4.1 BayesianFormulationforSegmentation ........................ 223
8.4.2 ThePriorProbability ............................................. 224
8.4.3 TheLikelihoodforGreyLevelImages .......................... 224