Thinking Probabilistically: Stochastic Processes, Disordered Systems, and Their Applications

243 Pages·2021·7.941 MB·English

by  Ariel Amir

ThinkingProbabilistically
Probability theory has diverse applications in a plethora of fields, including physics,
engineering, computer science, chemistry, biology, and economics. This book will
familiarizestudentswithvariousapplicationsofprobabilitytheory,stochasticmodel-
ing,andrandomprocesses,usingexamplesfromallthesedisciplinesandmore.
Thereaderlearnsviacasestudiesandbeginstorecognizethesortofproblemsthat
are best tackled probabilistically. The emphasis is on conceptual understanding, the
development of intuition, and gaining insight, keeping technicalities to a minimum.
Nevertheless, a glimpse into the depth of the topics is provided, preparing students
for more specialized texts while assuming only an undergraduate-level background
in mathematics. The wide range of areas covered – never before discussed together
in a unified fashion – includes Markov processes and random walks, Langevin and
Fokker–Planckequations,noise,generalizedcentrallimittheoremandextremevalues
statistics,randommatrixtheory,andpercolationtheory.
ArielAmirisaProfessoratHarvardUniversity.Hisresearchcentersonthetheoryof
complexsystems.

Thinking Probabilistically
Stochastic Processes, Disordered Systems,
and Their Applications
ARIEL AMIR
HarvardUniversity,Massachusetts
UniversityPrintingHouse,CambridgeCB28BS,UnitedKingdom
OneLibertyPlaza,20thFloor,NewYork,NY10006,USA
477WilliamstownRoad,PortMelbourne,VIC3207,Australia
314–321,3rdFloor,Plot3,SplendorForum,JasolaDistrictCentre,NewDelhi–110025,India
79AnsonRoad,#06–04/06,Singapore079906
CambridgeUniversityPressispartoftheUniversityofCambridge.
ItfurtherstheUniversity’smissionbydisseminatingknowledgeinthepursuitof
education,learning,andresearchatthehighestinternationallevelsofexcellence.
www.cambridge.org
Informationonthistitle:www.cambridge.org/9781108479523
DOI:10.1017/9781108855259
©ArielAmir2021
Thispublicationisincopyright.Subjecttostatutoryexception
andtotheprovisionsofrelevantcollectivelicensingagreements,
noreproductionofanypartmaytakeplacewithoutthewritten
permissionofCambridgeUniversityPress.
Firstpublished2021
AcataloguerecordforthispublicationisavailablefromtheBritishLibrary.
LibraryofCongressCataloging-in-PublicationData
Names:Amir,Ariel,1981–author.
Title:Thinkingprobabilistically:stochasticprocesses,disorderedsystems,
andtheirapplications/ArielAmir.
Description:Cambridge,UnitedKingdom;NewYork,NY:CambridgeUniversity
Press,2021.|Includesbibliographicalreferencesandindex.
Identifiers:LCCN2020019651(print)|LCCN2020019652(ebook)|
ISBN9781108479523(hardback)|ISBN9781108789981(paperback)|
ISBN9781108855259(epub)
Subjects:LCSH:Probabilities–Textbooks.|Stochasticprocesses–Textbooks.|
Order-disordermodels–Textbooks.
Classification:LCCQA273.A5482021(print)|LCCQA273(ebook)|DDC519.2–dc23
LCrecordavailableathttps://lccn.loc.gov/2020019651
LCebookrecordavailableathttps://lccn.loc.gov/2020019652
ISBN978-1-108-47952-3Hardback
ISBN978-1-108-78998-1Paperback
CambridgeUniversityPresshasnoresponsibilityforthepersistenceoraccuracy
ofURLsforexternalorthird-partyinternetwebsitesreferredtointhispublication
anddoesnotguaranteethatanycontentonsuchwebsitesis,orwillremain,
accurateorappropriate.
Contents
Acknowledgments pageviii
1 Introduction 1
1.1 ProbabilisticSurprises 5
1.2 Summary 12
1.3 Exercises 13
2 RandomWalks 15
2.1 RandomWalksin1D 16
2.2 DerivationoftheDiffusionEquationforRandomWalksinArbitrary
SpatialDimension 18
2.3 MarkovProcessesandMarkovChains 24
2.4 Google PageRank: Random Walks on Networks as an Example
ofaUsefulMarkovChain 25
2.5 RelationbetweenMarkovChainsandtheDiffusionEquation 30
2.6 Summary 32
2.7 Exercises 32
3 LangevinandFokker–PlanckEquationsandTheirApplications 39
3.1 ApplicationofaDiscreteLangevinEquationtoaBiologicalProblem 45
3.2 TheBlack–ScholesEquation:PricingOptions 51
3.3 AnotherExample:The“WellFunction”inHydrology 60
3.4 Summary 62
3.5 Exercises 63
4 EscapeOveraBarrier 67
4.1 SettingUptheEscape-Over-a-BarrierProblem 70
4.2 Applicationtothe1DEscapeProblem 71
4.3 DerivingLanger’sFormulaforEscape-Over-a-BarrierinAny
SpatialDimension 73
4.4 Summary 79
4.5 Exercises 79
vi Contents
5 Noise 81
5.1 TelegraphNoise:PowerSpectrumAssociatedwithaTwo-Level-System 83
5.2 FromTelegraphNoiseto1/f NoiseviatheSuperpositionofManyTwo-
Level-Systems 88
5.3 PowerSpectrumofaSignalGeneratedbyaLangevinEquation 89
5.4 Parseval’sTheorem:RelatingEnergyintheTimeandFrequencyDomain 90
5.5 Summary 92
5.6 Exercises 92
6 GeneralizedCentralLimitTheoremandExtremeValueStatistics 96
6.1 ProbabilityDistributionofSums:IntroducingtheCharacteristicFunction 98
6.2 Approximating the Characteristic Function at Small Frequencies
forDistributionswithFiniteVariance 99
6.3 CentralRegionofCLT:WheretheGaussianApproximationIsValid 100
6.4 Sum of a Large Number of Positive Random Variables: Universal
DescriptioninLaplaceSpace 103
6.5 ApplicationtoSlowRelaxations:StretchedExponentials 106
6.6 ExampleofaStableDistribution:CauchyDistribution 108
6.7 Self-SimilarityofRunningSums 109
6.8 GeneralizedCLTviaanRG-InspiredApproach 110
6.9 ExploringtheStableDistributionsNumerically 118
6.10 RG-InspiredApproachforExtremeValueDistributions 120
6.11 Summary 127
6.12 Exercises 128
7 AnomalousDiffusion 133
7.1 ContinuousTimeRandomWalks 134
7.2 LévyFlights:WhentheVarianceDiverges 137
7.3 PropagatorforAnomalousDiffusion 138
7.4 BacktoNormalDiffusion 139
7.5 Ergodicity Breaking: When the Time Average and the Ensemble
AverageGiveDifferentResults 139
7.6 Summary 140
7.7 Exercises 141
8 RandomMatrixTheory 145
8.1 LevelRepulsionbetweenEigenvalues:TheBirthofRMT 145
8.2 Wigner’sSemicircleLawfortheDistributionofEigenvalues 149
8.3 JointProbabilityDistributionofEigenvalues 155
8.4 EnsemblesofNon-HermitianMatricesandtheCircularLaw 162
8.5 Summary 179
8.6 Exercises 180
Contents vii
9 PercolationTheory 184
9.1 PercolationandEmergentPhenomena 184
9.2 PercolationonTrees–andthePowerofRecursion 193
9.3 PercolationCorrelationLengthandtheSizeoftheLargestCluster 195
9.4 UsingPercolationTheorytoStudyRandomResistorNetworks 197
9.5 Summary 202
9.6 Exercises 203
AppendixA ReviewofBasicProbabilityConceptsandCommonDistributions 207
A.1 SomeImportantDistributions 208
A.2 CentralLimitTheorem 210
AppendixB ABriefLinearAlgebraReminder,andSomeGaussianIntegrals 211
B.1 BasicLinearAlgebraFacts 211
B.2 GaussianIntegrals 212
AppendixC ContourIntegrationandFourierTransformRefresher 214
C.1 ContourIntegralsandtheResidueTheorem 214
C.2 FourierTransforms 214
AppendixD ReviewofNewtonianMechanics,BasicStatisticalMechanics,andHessians 217
D.1 BasicResultsinClassicalMechanics 217
D.2 TheBoltzmannDistributionandthePartitionFunction 218
D.3 Hessians 218
AppendixE MinimizingFunctionals,theDivergenceTheorem,andSaddle-Point
Approximations 220
E.1 FunctionalDerivatives 220
E.2 LagrangeMultipliers 220
E.3 TheDivergenceTheorem(Gauss’sLaw) 220
E.4 Saddle-PointApproximations 221
AppendixF Notation,Notation… 222
References 225
Index 232
Acknowledgments
I am indebted to the students of Harvard course APMTH 203 for their patience and
perseveranceasthecoursematerialsweredeveloped,andIamhugelygratefultoall
ofmyteachingfellowsalongtheyearsfortheirhardwork:SarahKostinksi,Po-YiHo,
FelixWong,SihengChen,PéturRafnBryde,andJiseonMin.Muchofthecontentsof
theAppendicesdrawsontheirhelpfulnotes.
IthankEliBarkai,StasBurov,OriHirschberg,YipeiGuo,JieLin,DavidNelson,
Efi Shahmoon, Pierpaolo Vivo, and Ahmad Zareei for numerous useful discussions
andcommentsonthenotes.ChristopherBergevinhadexcellentsuggestionsforChap-
ter 2, and Julien Tailleur for Chapter 3. Ori Hirschberg and Eli Barkai had many
importantcommentsandusefulsuggestionsregardingChapter6.IthankGraceZhang,
Satya Majumdar, and Fernando L. Metz for a careful reading of Chapter 8. I am
grateful to Martin Z. Bazant and Bertrand I. Halperin for important comments on
Chapter9.IthankTerryTaoforallowingmetoadaptthediscussionofBlack–Scholes
in Chapter 3 from his insightful blog. I also thank Dr. Yasmine Meroz and Dr. Ben
Golubforgivingguestlecturesinearlyversionsofthecourse.Finally,Iamgrateful
toSimon’scoffeeshopforprovidingreliablyexcellentcoffee,whichwasinstrumental
tothewritingofthisbook.
I dedicate this book to Lindy, Maayan, Tal, and Ella, who keep my feet on the
groundandasmileonmyface.
1 Introduction
Iknowtoowellthattheseargumentsfromprobabilitiesareimpostors,andunless
greatcautionisobservedintheuseofthemtheyareapttobedeceptive–in
geometry,andinotherthingstoo
(fromPlato’sPhaedo)
The purposes of this book are to familiarize you with a broad range of examples
where randomness plays a key role, develop an intuition for it, and get to the level
whereyoumayreadarecentresearchpaperonthesubjectandbeabletounderstand
the terminology, the context, and the tools used. This is in a sense the “organizing
principle” behind the various chapters: In all of them we are driven by applications
whereprobabilityplaysafundamentalrole,andleadstoexcitingandoftenintriguing
phenomena. There are many relations between the chapters, both in terms of the
mathematical tools and in some cases in terms of the physical processes involved,
but one chapter does not follow from the previous one by necessity or hinge on it –
rather, the idea is to present a rich repertoire of problems involving randomness,
giving the reader a good basis in a broad range of fields ... and to have fun
alongtheway.
Randomnessleadstonewphenomena.Inaclassicpaper,Anderson(1972)coined
thephrase“moreisdifferent”.Itisalsotruethat“stochasticisdifferent”...Thebook
willgiveyousometoolstounderstandphenomenaassociatedwithdisorderedsystems
and stochastic processes. These will include percolation (relevant for polymers,
gels, social networks, epidemic spreading); random matrix theory (relevant for
understanding the statistics of nuclear and atomic levels, model certain properties
of ecological systems and more); random walks and Langevin equations (pertinent
to understanding numerous applications in physics, chemistry, cell biology as well
as finance). The emphasis will be on understanding the phenomena and quantifying
them. Note that while all of the applications considered here build on random-
ness in a fundamental way, the collection of topics covered is far from repre-
sentative of the vast realm of applications that hinge on probability theory. (For
instance, two important fields not touched on here are statistical inference and
chemicalkinetics).