Probability: Theory and Examples

433 Pages·2019·3.456 MB·English

by  Rick Durrett

Probability
Thislivelyintroductiontomeasure-theoreticprobabilitytheorycoverslawsoflarge
numbers,centrallimittheorems,randomwalks,martingales,Markovchains,ergodic
theorems,andBrownianmotion.Concentratingontheresultsthatarethemostuseful
for applications, this comprehensive treatment is a rigorous, measure theory–based
graduatetextandreference.Operatingunderthephilosophythatthebestwaytolearn
probabilityistoseeitinaction,thebookcontainsmanyextendedexamplesthatapply
thetheorytoconcreteapplications.Readerslearntorecognizewhenamethodworks
and,moreimportant,whenitdoesnot.
This fifth edition contains a new chapter on multidimensional Brownian motion
and its relationship to PDEs, an advanced topic that is finding new applications.
Setting the foundation for this expansion, Chapter 7 now features a proof of Itoˆs
formula. Key exercises that previously were simply proofs left to the reader have
beendirectlyinsertedintothetextaslemmas.Theneweditionalsoreinstatesdiscus-
sionaboutthecentrallimittheoremformartingalesandstationarysequences.
RICK DURRETT is a James B. Duke professor in the mathematics department
of Duke University. He received his Ph.D. in Operations Research from Stanford
Universityin1976.AfternineyearsatUCLAandtwenty-fiveatCornellUniversity,
he moved to Duke in 2010. He is the author of 8 books and more than 220 journal
articlesonawidevarietyoftopics,andhassupervisedmorethan45Ph.D.students.
He is a member of National Academy of Science, American Academy of Arts and
Sciences,andafellowoftheInstituteofMathematicalStatisticsandoftheAmerican
MathematicalSociety.
CAMBRIDGE SERIES IN STATISTICAL AND
PROBABILISTIC MATHEMATICS
EditorialBoard
Z.Ghahramani(DepartmentofEngineering,UniversityofCambridge)
R.Gill(MathematicalInstitute,LeidenUniversity)
F.P.Kelly(DepartmentofPureMathematicsandMathematicalStatistics,UniversityofCambridge)
B.D.Ripley(DepartmentofStatistics,UniversityofOxford)
S.Ross(DepartmentofIndustrialandSystemsEngineering,
UniversityofSouthernCalifornia)
M.Stein(DepartmentofStatistics,UniversityofChicago)
This series of high-quality upper-division textbooks and expository monographs covers all
aspectsofstochasticapplicablemathematics.Thetopicsrangefrompureandappliedstatistics
toprobabilitytheory,operationsresearch,optimization,andmathematicalprogramming.The
books contain clear presentations of new developments in the field and also of the state of
theartinclassicalmethods.Whileemphasizingrigoroustreatmentoftheoreticalmethods,the
booksalsocontainapplicationsanddiscussionsofnewtechniquesmadepossiblebyadvances
incomputationalpractice.
Acompletelistofbooksintheseriescanbefoundatwww.cambridge.org/statistics.
Recenttitlesincludethefollowing:
22. SaddlepointApproximationswithApplications,byRonaldW.Butler
23. AppliedAsymptotics,byA.R.Brazzale,A.C.DavisonandN.Reid
24. RandomNetworksforCommunication,byMassimoFranceschettiandRonaldMeester
25. DesignofComparativeExperiments,byR.A.Bailey
26. SymmetryStudies,byMarlosA.G.Viana
27. ModelSelectionandModelAveraging,byGerdaClaeskensandNilsLidHjort
28. BayesianNonparametrics,editedbyNilsLidHjortetal.
29. FromFiniteSampletoAsymptoticMethodsinStatistics,byPranabK.Sen,
JulioM.SingerandAntonioC.PedrosadeLima
30. BrownianMotion,byPeterMo¨rtersandYuvalPeres
31. Probability(FourthEdition),byRickDurrett
33. StochasticProcesses,byRichardF.Bass
34. RegressionforCategoricalData,byGerhardTutz
35. ExercisesinProbability(SecondEdition),byLo¨ıcChaumontandMarcYor
36. StatisticalPrinciplesfortheDesignofExperiments,byR.Mead,S.G.Gilmourand
A.Mead
37. QuantumStochastics,byMou-HsiungChang
38. NonparametricEstimationunderShapeConstraints,byPietGroeneboomand
GeurtJongbloed
39. LargeSampleCovarianceMatricesandHigh-DimensionalDataAnalysis,by
JianfengYao,ShurongZhengandZhidongBai
40. MathematicalFoundationsofInfinite-DimensionalStatisticalModels,byEvaristGine´
andRichardNickl
41. Confidence,Likelihood,Probability,byToreSchwederandNilsLidHjort
42. ProbabilityonTreesandNetworks,byRussellLyonsandYuvalPeres
43. RandomGraphsandComplexNetworks(Volume1),byRemcovanderHofstad
44. FundamentalsofNonparametricBayesianInference,bySubhashisGhosalandAadvan
derVaart
45. Long-RangeDependenceandSelf-Similarity,byVladasPipirasandMuradS.Taqqu
46. PredictiveStatistics,byBertrandS.ClarkeandJenniferL.Clarke
47. High-DimensionalProbability,byRomanVershynin
48. High-DimensionalStatistics,byMartinJ.Wainwright
49. Probability:TheoryandExamples(FifthEdition),byRickDurrett

Probability
Theory and Examples
FIFTH EDITION
Rick Durrett
DukeUniversity
UniversityPrintingHouse,CambridgeCB28BS,UnitedKingdom
OneLibertyPlaza,20thFloor,NewYork,NY10006,USA
477WilliamstownRoad,PortMelbourne,VIC3207,Australia
314–321,3rdFloor,Plot3,SplendorForum,JasolaDistrictCentre,NewDelhi–110025,India
79AnsonRoad,#06–04/06,Singapore079906
CambridgeUniversityPressispartoftheUniversityofCambridge.
ItfurtherstheUniversity’smissionbydisseminatingknowledgeinthepursuitof
education,learning,andresearchatthehighestinternationallevelsofexcellence.
www.cambridge.org
Informationonthistitle:www.cambridge.org/9781108473682
DOI:10.1017/9781108591034
©RickDurrett2019
Thispublicationisincopyright.Subjecttostatutoryexception
andtotheprovisionsofrelevantcollectivelicensingagreements,
noreproductionofanypartmaytakeplacewithoutthewritten
permissionofCambridgeUniversityPress.
Firstpublished2019
PrintedintheUnitedKingdombyTJInternationalLtd.,Padstow,Cornwall
AcataloguerecordforthispublicationisavailablefromtheBritishLibrary.
LibraryofCongressCataloging-in-PublicationData
Names:Durrett,Richard,1951–author.
Title:Probability:theoryandexamples/RickDurrett(DukeUniversity,
Durham,NorthCarolina).
Description:Fifthedition.|Cambridge;NewYork,NY:CambridgeUniversity
Press,2019.|Series:Cambridgeseriesinstatisticalandprobabilistic
mathematics;49|Includesbibliographicalreferencesandindex.
Identifiers:LCCN2018047195|ISBN9781108473682(hardback:alk.paper)
Subjects:LCSH:Probabilities.|Probabilities–Textbooks.
Classification:LCCQA273.D8652019|DDC519.2–dc23
LCrecordavailableathttps://lccn.loc.gov/2018047195
ISBN978-1-108-47368-2Hardback
CambridgeUniversityPresshasnoresponsibilityforthepersistenceoraccuracy
ofURLsforexternalorthird-partyinternetwebsitesreferredtointhispublication
anddoesnotguaranteethatanycontentonsuchwebsitesis,orwillremain,
accurateorappropriate.
Contents
Preface pagexi
1 MeasureTheory 1
1.1 ProbabilitySpaces 1
1.2 Distributions 8
1.3 RandomVariables 13
1.4 Integration 15
1.5 PropertiesoftheIntegral 21
1.6 ExpectedValue 25
1.6.1 Inequalities 25
1.6.2 IntegrationtotheLimit 26
1.6.3 ComputingExpectedValues 28
1.7 ProductMeasures,Fubini’sTheorem 33
2 LawsofLargeNumbers 37
2.1 Independence 37
2.1.1 SufficientConditionsforIndependence 39
2.1.2 Independence,Distribution,andExpectation 41
2.1.3 SumsofIndependentRandomVariables 43
2.1.4 ConstructingIndependentRandomVariables 45
2.2 WeakLawsofLargeNumbers 48
2.2.1 L2WeakLaws 48
2.2.2 TriangularArrays 51
2.2.3 Truncation 53
2.3 Borel-CantelliLemmas 58
2.4 StrongLawofLargeNumbers 65
2.5 ConvergenceofRandomSeries* 69
2.5.1 RatesofConvergence 75
2.5.2 InfiniteMean 76
2.6 RenewalTheory* 78
2.7 LargeDeviations* 90
3 CentralLimitTheorems 98
3.1 TheDeMoivre-LaplaceTheorem 98
3.2 WeakConvergence 100
vii
viii Contents
3.2.1 Examples 100
3.2.2 Theory 102
3.3 CharacteristicFunctions 108
3.3.1 Definition,InversionFormula 108
3.3.2 WeakConvergence 114
3.3.3 MomentsandDerivatives 116
3.3.4 Polya’sCriterion* 119
3.3.5 TheMomentProblem* 121
3.4 CentralLimitTheorems 125
3.4.1 i.i.d.Sequences 125
3.4.2 TriangularArrays 128
3.4.3 PrimeDivisors(Erdo¨s-Kac)* 132
3.4.4 RatesofConvergence(Berry-Esseen)* 136
3.5 LocalLimitTheorems* 140
3.6 PoissonConvergence 145
3.6.1 TheBasicLimitTheorem 145
3.6.2 TwoExampleswithDependence 149
3.7 PoissonProcesses 151
3.7.1 CompoundPoissonProcesses 154
3.7.2 Thinning 155
3.7.3 Conditioning 157
3.8 StableLaws* 159
3.9 InfinitelyDivisibleDistributions* 168
3.10 LimitTheoremsinRd 171
4 Martingales 178
4.1 ConditionalExpectation 178
4.1.1 Examples 180
4.1.2 Properties 182
4.1.3 RegularConditionalProbabilities* 185
4.2 Martingales,AlmostSureConvergence 188
4.3 Examples 194
4.3.1 BoundedIncrements 194
4.3.2 Polya’sUrnScheme 196
4.3.3 Radon-NikodymDerivatives 197
4.3.4 BranchingProcesses 200
4.4 Doob’sInequality,ConvergenceinLp,p >1 203
4.5 SquareIntegrableMartingales* 208
4.6 UniformIntegrability,ConvergenceinL1 211
4.7 BackwardsMartingales 216
4.8 OptionalStoppingTheorems 221
4.8.1 ApplicationstoRandomWalks 223
4.9 CombinatoricsofSimpleRandomWalk* 227
5 MarkovChains 232
5.1 Examples 232
5.2 Construction,MarkovProperties 235
Contents ix
5.3 RecurrenceandTransience 243
5.4 RecurrenceofRandomWalksStararredSection 248
5.5 StationaryMeasures 259
5.6 AsymptoticBehavior 268
5.7 Periodicity,Tailσ-Field* 274
5.8 GeneralStateSpace* 278
5.8.1 RecurrenceandTransience 281
5.8.2 StationaryMeasures 281
5.8.3 ConvergenceTheorem 282
5.8.4 GI/G/1Queue 283
6 ErgodicTheorems 286
6.1 DefinitionsandExamples 286
6.2 Birkhoff’sErgodicTheorem 289
6.3 Recurrence 293
6.4 ASubadditiveErgodicTheorem 296
6.5 Applications 300
7 BrownianMotion 305
7.1 DefinitionandConstruction 305
7.2 MarkovProperty,Blumenthal’s0-1Law 311
7.3 StoppingTimes,StrongMarkovProperty 316
7.4 PathProperties 320
7.4.1 ZerosofBrownianMotion 320
7.4.2 HittingTimes 321
7.5 Martingales 325
7.6 Itoˆ’sFormula* 328
8 ApplicationstoRandomWalk 336
8.1 Donsker’sTheorem 336
8.2 CLTsforMartingales 342
8.3 CLTsforStationarySequences 347
8.3.1 MixingProperties 351
8.4 EmpiricalDistributions,BrownianBridge 354
8.5 LawsoftheIteratedLogarithm 360
9 MultidimensionalBrownianMotion 364
9.1 Martingales 364
9.2 HeatEquation 366
9.3 InhomogeneousHeatEquation 368
9.4 Feynman-KacFormula 370
9.5 DirichletProblem 373
9.5.1 ExitDistributions 377
9.6 Green’sFunctionsandPotentialKernels 379
9.7 Poisson’sEquation 382
9.7.1 OccupationTimes 385
9.8 Schro¨dingerEquation 387