Probability and Stochastic Processes for Physicists

372 Pages·2020·5.641 MB·English

by  Cufaro Petroni, Nicola

UNITEXT for Physics
Nicola Cufaro Petroni
Probability
and Stochastic
Processes
for Physicists
UNITEXT for Physics
Series Editors
Michele Cini, University of Rome Tor Vergata, Roma, Italy
Attilio Ferrari, University of Turin, Turin, Italy
Stefano Forte, University of Milan, Milan, Italy
Guido Montagna, University of Pavia, Pavia, Italy
Oreste Nicrosini, University of Pavia, Pavia, Italy
Luca Peliti, University of Napoli, Naples, Italy
Alberto Rotondi, Pavia, Italy
Paolo Biscari, Politecnico di Milano, Milan, Italy
Nicola Manini, University of Milan, Milan, Italy
Morten Hjorth-Jensen, University of Oslo, Oslo, Norway
UNITEXTforPhysicsseries,formerly UNITEXT CollanadiFisicae Astronomia,
publishestextbooksandmonographsinPhysicsandAstronomy,mainlyinEnglish
language, characterized of a didactic style and comprehensiveness. The books
publishedinUNITEXTforPhysicsseriesareaddressedtoupperundergraduateand
graduate students, but also to scientists and researchers as important resources for
their education, knowledge and teaching.
More information about this series at http://www.springer.com/series/13351
Nicola Cufaro Petroni
Probability
and Stochastic Processes
for Physicists
123
NicolaCufaro Petroni
Department ofPhysics
University of Bari
Bari, Italy
ISSN 2198-7882 ISSN 2198-7890 (electronic)
UNITEXTfor Physics
ISBN978-3-030-48407-1 ISBN978-3-030-48408-8 (eBook)
https://doi.org/10.1007/978-3-030-48408-8
©TheEditor(s)(ifapplicable)andTheAuthor(s),underexclusivelicensetoSpringerNature
SwitzerlandAG2020
Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether
thewholeorpartofthematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseof
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and
transmissionorinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilar
ordissimilarmethodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authorsortheeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinor
for any errors or omissions that may have been made. The publisher remains neutral with regard to
jurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
… Iuvat integros accedere fontis
atquehaurire, iuvatquenovosdecerpereflores1
Lucretius, I 927-8
1TranslationbyA.E.Stallings(Penguin,London,2007):
Ithrilltocomeuponuntastedspringsandslakemythirst.
Ijoytopluckstrangeflowers…
Preface
Thisbookoriginatesfromthefull-semesterlessonsheldyearlywithinthemaster’s
degree course in Physics at the University of Bari. Part I about Probability essen-
tiallystartsfromscratchinChap.1asifnothinghadbeenknownbeforeaboutthis
topic,anditquicklygoesontoconceptuallyrelevant,butlessfamiliar,issues.The
keyfeatureofthispartnodoubtisthenotionofrandomvariables(andthenalsoof
stochastic processes) which is introduced in Chap. 3 as a paramount concept that
differs from that of distribution previously elucidated in Chap. 2 . The interplay
betweentherandomvariablesandtheirdistributionsisamainstapleofthepresent
volume. Chapter 4 finally deals with a panoply of limit theorems that in any case
represent the main gate to the world of the subsequent sections.
PartIIisthendevotedtoStochasticProcessesthatarepresentedinChap.5from
arathergeneralstandpointwithparticularattentiontostationarityandergodicity.In
Chap. 6, the first two examples of classical processes (Poisson and Wiener) are
heuristically introduced with a discussion of the familiar procedures used to man-
ufacturetheirrandomtrajectories;thefirstpresentationofboththewhitenoiseand
theBrownianmotionisalsoaddedin.ThevasttopicofMarkovianityistackledin
Chap. 7 with a discussion of the equations ruling the distributions of the
jump-diffusionprocesses:thisgivesalsotheopportunityofintroducingtheCauchy
and Ornstein-Uhlenbeck processes, and of including a quick reference to the gen-
eraltopicoftheLévyprocesses.Theequationsrulinginsteadoftheevolutionofthe
trajectories of the diffusion processes, namely the Itō stochastic differential equa-
tions, are presentedinChap. 8in theframework of thestochastic calculus: several
examples of solutions of stochastic differential equations are also produced.
ThetwochaptersofPartIIIdealfinallywiththemodelingofparticularphysical
problemsbymeansofthetoolsacquiredintheprevioussections.Chapter9givesa
broad presentationof adynamical theory ofthe Brownian motion modeled mainly
by the Ornstein-Uhlenbeck process and is completed with a discussion of the
Smoluchowski approximation and of the Boltzmann equilibrium distributions. In
Chap.10anaccountisgivenofthestochasticmechanics,amodelleadingtoadeep
connection between the time-reversal invariant diffusion processes and the
Schrödingerequation:atopicapparentlyrelevantforphysicistsandmathematicians
vii
viii Preface
alike.Ontheonehandindeeditshows,forinstance,thatthestatisticalBornruleof
quantum mechanics could not be just a postulate added by hand to its formal
apparatus,andon theother italso hintstowider than usual possibledevelopments
of the stochastic analysis.
A number of appendices conclude the book: a few of them are simple com-
plements, lengthy proofs and useful reminders, but others are short discussions
about particular relevant topics moved over there to avoid interrupting the main
text.Forinstance,AppendixAdealswiththepossibleincompatibilityoflegitimate
joint distributions; Appendix C discusses the celebrated Bertrand paradox;
Appendix H shows the baffling outcomes of a naive application of calculus to
stochastic processes; and Appendix I presents the case of a process satisfying the
Chapman-Kolmogorov conditions despite it being non-Markovian.
Besides its traditional purposes, one of the aims of this book is to bridge a gap
often existing between physicists and mathematicians in the subject area of prob-
ability and stochastic processes: a difference usually ranging from notions to
notations,fromtechniquestonames.This,however,wouldarguablybeanoutcome
hardtobeachievedwithinthescopeeitherofonecourseorevenofonebook,and
thereforewewillbesatisfiedifweonlyapproachedthegoalof—atleastpartially—
restoring an intellectual osmosis that has always been fruitful along the centuries.
While even this limited prize would not come in any case without a price—it
usually requires some effort from the physics students to go beyond the most
familiarwaysofthinking—itisimportanttoremarkthatthefocusofthefollowing
chapters will always be on concepts and models, rather than on methods and
mathematical rigor. Most important theorems, for instance, will not be skipped
away, but they will be presented and clarified in their role without proofs, and the
interested reader will be referred for further details tothe existing literature quoted
in Bibliography.
Bari, Italy Nicola Cufaro Petroni
April 2020
Contents
Part I Probability
1 Probability Spaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1 Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.4 Conditional Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.5 Independent Events. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Reference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2 Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.1 Distributions on N . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.1.1 Finite and Countable Spaces . . . . . . . . . . . . . . . . . . . . 19
2.1.2 Bernoulli Trials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.2 Distributions on R. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.2.1 Cumulative Distribution Functions . . . . . . . . . . . . . . . 26
2.2.2 Discrete Distributions . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.2.3 Absolutely Continuous Distributions: Density . . . . . . . 30
2.2.4 Singular Distributions. . . . . . . . . . . . . . . . . . . . . . . . . 34
2.2.5 Mixtures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.3 Distributions on Rn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.3.1 Multivariate Distribution Functions . . . . . . . . . . . . . . . 36
2.3.2 Multivariate Densities. . . . . . . . . . . . . . . . . . . . . . . . . 38
2.3.3 Marginal Distributions . . . . . . . . . . . . . . . . . . . . . . . . 40
2.3.4 Copulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.4 Distributions on R1 and RT . . . . . . . . . . . . . . . . . . . . . . . . . . 45
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
ix
x Contents
3 Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.1 Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.1.1 Measurability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.1.2 Laws and Distributions. . . . . . . . . . . . . . . . . . . . . . . . 51
3.1.3 Generating New rv’s . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.2 Random Vectors and Stochastic Processes . . . . . . . . . . . . . . . . 55
3.2.1 Random Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.2.2 Joint and Marginal Distributions and Densities. . . . . . . 57
3.2.3 Independence of rv’s . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.2.4 Decomposition of Binomial rv’s . . . . . . . . . . . . . . . . . 63
3.3 Expectation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.3.1 Integration and Expectation. . . . . . . . . . . . . . . . . . . . . 66
3.3.2 Change of Variables. . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.3.3 Variance and Covariance . . . . . . . . . . . . . . . . . . . . . . 75
3.4 Conditioning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
3.4.1 Conditional Distributions . . . . . . . . . . . . . . . . . . . . . . 81
3.4.2 Conditional Expectation . . . . . . . . . . . . . . . . . . . . . . . 84
3.4.3 Optimal Mean Square Estimation . . . . . . . . . . . . . . . . 90
3.5 Combinations of rv’s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
3.5.1 Functions of rv’s . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
3.5.2 Sums of Independent rv’s. . . . . . . . . . . . . . . . . . . . . . 95
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
4 Limit Theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
4.1 Convergence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
4.2 Characteristic Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
4.2.1 Definition and Properties . . . . . . . . . . . . . . . . . . . . . . 102
4.2.2 Gaussian Laws. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
4.2.3 Composition and Decomposition of Laws . . . . . . . . . . 111
4.3 Laws of Large Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
4.4 Gaussian Theorems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
4.5 Poisson Theorems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
4.6 Where the Classical Limit Theorems Fail. . . . . . . . . . . . . . . . . 124
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
Part II Stochastic Processes
5 General Notions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
5.1 Identification and Distribution . . . . . . . . . . . . . . . . . . . . . . . . . 129
5.2 Expectations and Correlations . . . . . . . . . . . . . . . . . . . . . . . . . 132
5.3 Convergence and Continuity . . . . . . . . . . . . . . . . . . . . . . . . . . 133
5.4 Differentiation and Integration in ms . . . . . . . . . . . . . . . . . . . . 134