A Basic Course in Probability Theory (Universitext)

277 Pages·2017·3.3 MB·english

by  Bhattacharya

Universitext
Editorial Board
(North America):
S.Axler
K.A.Ribet
Universitext
Editors (North America):S.Axler and K.A.Ribet
Aguilar/Gitler/Prieto:Algebraic Topology from a Homotopical Viewpoint
Aksoy/Khamsi:Nonstandard Methods in Fixed Point Theory
Andersson:Topics in Complex Analysis
Aupetit:A Primer on Spectral Theory
Bachman/Narici/Beckenstein:Fourier and Wavelet Analysis
Badescu:Algebraic Surfaces
Balakrishnan/Ranganathan:A Textbook of Graph Theory
Balser:Formal Power Series and Linear Systems of Meromorphic Ordinary
Differential Equations
Bapat:Linear Algebra and Linear Models (2nd ed.)
Berberian:Fundamentals of Real Analysis
Bhattacharya/Waymire:A Basic Course in Probability Theory
Blyth:Lattices and Ordered Algebraic Structures
Boltyanskii/Efremovich:Intuitive Combinatorial Topology (Shenitzer,trans.)
Booss/Bleecker:Topology and Analysis
Borkar:Probability Theory:An Advanced Course
Böttcher/Silbermann:Introduction to Large Truncated Toeplitz Matrices
Carleson/Gamelin:Complex Dynamics
Cecil:Lie Sphere Geometry:With Applications to Submanifolds
Chae:Lebesgue Integration (2nd ed.)
Charlap:Bieberbach Groups and Flat Manifolds
Chern:Complex Manifolds Without Potential Theory
Cohn:A Classical Invitation to Algebraic Numbers and Class Fields
Curtis:Abstract Linear Algebra
Curtis:Matrix Groups
Debarre:Higher-Dimensional Algebraic Geometry
Deitmar:A First Course in Harmonic Analysis (2nd ed.)
DiBenedetto:Degenerate Parabolic Equations
Dimca:Singularities and Topology of Hypersurfaces
Edwards:A Formal Background to Mathematics I a/b
Edwards:A Formal Background to Mathematics II a/b
Engel/Nagel:A Short Course on Operator Semigroups
Farenick:Algebras of Linear Transformations
Foulds:Graph Theory Applications
Friedman:Algebraic Surfaces and Holomorphic Vector Bundles
Fuhrmann:A Polynomial Approach to Linear Algebra
Gardiner:A First Course in Group Theory
Gårding/Tambour:Algebra for Computer Science
Goldblatt:Orthogonality and Spacetime Geometry
Gustafson/Rao:Numerical Range:The Field of Values of Linear Operators
and Matrices
Hahn:Quadratic Algebras,Clifford Algebras,and Arithmetic Witt Groups
Heinonen:Lectures on Analysis on Metric Spaces
Holmgren:A First Course in Discrete Dynamical Systems
Howe/Tan:Non-Abelian Harmonic Analysis:Applications of SL(2,R)
Howes:Modern Analysis and Topology
Hsieh/Sibuya:Basic Theory of Ordinary Differential Equations
Humi/Miller:Second Course in Ordinary Differential Equations
Hurwitz/Kritikos:Lectures on Number Theory
(continued after index)
Rabi Bhattacharya
Edward C. Waymire
A Basic Course
in Probability Theory
Rabi Bhattacharya Edward C. Waymire
Department ofMathematics Department ofMathematics
University of Arizona Oregon State University
Tucson, AZ 85750 Corvallis, OR 97331
USA USA
[email protected] [email protected]
Editorial Board
(North America):
S.Axler K.A.Ribet
Mathematics Department Mathematics Department
San Francisco State University University of California at Berkeley
San Francisco,CA 94132 Berkeley,CA 94720-3840
USA USA
[email protected] [email protected]
Mathematics Subject Classification (2000):60-xx, 60Jxx
Library ofCongress Control Number: 2007928731
ISBN 978-0-387-71938-2 e-ISBN 978-0-387-71939-9
Printed on acid-free paper.
© 2007 Springer Science+Business Media,Inc.
All rights reserved.This work may not be translated or copied in whole or in part without the
written permission ofthe publisher (Springer Science+Business Media,Inc.,233 Spring Street,
New York,NY 10013,USA),except for brief excerpts in connection with reviews or scholarly
analysis. Use in connection with any form of information storage and retrieval, electronic
adaptation,computer software,or by similar or dissimilar methodology now known or hereafter
developed is forbidden.
The use in this publication oftrade names,trademarks,service marks,and similar terms,even if
they are not identified as such,is not to be taken as an expression ofopinion as to whether or
not they are subject to proprietary rights.
9 8 7 6 5 4 3 2 1
springer.com
To Gowri and Linda
PREFACE
In1937,A.N.Kolmogorovintroducedameasure-theoreticmathematicalframework
forprobabilitytheoryinresponsetoDavidHilbert’sSixthProblem.Thistextprovides
the basic elements of probability within this framework. It may be used for a one-
semester course in probability, or as a reference to prerequisite material in a course
on stochastic processes. Our pedagogical view is that the subsequent applications to
stochasticprocessesprovideacontinuedopportunitytomotivateandreinforcethese
importantmathematicalfoundations.Thebookisbestsuitedforstudentswithsome
prior, or at least concurrent, exposure to measure theory and analysis. But it also
provides a fairly detailed overview, with proofs given in appendices, of the measure
theory and analysis used.
The selection of material presented in this text grew out of our effort to provide
aself-containedreferencetofoundationalmaterialthatwouldfacilitateacompanion
treatiseonstochasticprocessesthatwehavebeendeveloping.1 Whiletherearemany
excellenttextbooksavailablethatprovidetheprobabilitybackgroundforvariouscon-
tinuedstudiesofstochasticprocesses,thepresenttreatmentwasdesignedwiththisas
anexplicitgoal.Thisledtosomeuniquefeaturesfromtheperspectiveoftheordering
and selection of material.
WebeginwithChapterIonvariousmeasure-theoreticconceptsandresultsrequired
for the proper mathematical formulation of a probability space, random maps, dis-
tributions,andexpectedvalues.Standardresultsfrommeasuretheoryaremotivated
and explained with detailed proofs left to an appendix.
Chapter II is devoted to two of the most fundamental concepts in probability
theory: independence and conditional expectation (and/or conditional probability).
This continues to build upon, reinforce, and motivate basic ideas from real analy-
sis and measure theory that are regularly employed in probability theory, such as
Carath´eodory constructions, the Radon–Nikodym theorem, and the Fubini–Tonelli
theorem. A careful proof of the Markov property is given for discrete-parameter
random walks on Rk to illustrate conditional probability calculations in some
generality.
Chapter III provides some basic elements of martingale theory that have evolved
tooccupyasignificantfoundationalroleinprobabilitytheory.Inparticular,optional
stopping and maximal inequalities are cornerstone elements. This chapter provides
sufficient martingale background, for example, to take up a course in stochastic dif-
ferentialequationsdevelopedinachapterofourtextonstochasticprocesses.Amore
comprehensivetreatmentofmartingaletheoryisdeferredtostochasticprocesseswith
further applications there as well.
The various laws of large numbers and elements of large deviation theory are
developed in Chapter IV. This includes the classical 0-1 laws of Kolmogorov and
1Bhattacharya, R. and E. Waymire (2007): Theory and Applications of Stochastic
Processes,Springer-Verlag,GraduateTextsinMathematics.
viii PREFACE
Hewitt–Savage.Someemphasisisgiventosize-biasinginlargedeviationcalculations
which are of contemporary interest.
Chapter V analyzes in detail the topology of weak convergence of probabilities
defined on metric spaces, culminating in the notion of tightness and a proof of
Prohorov’s theorem.
ThecharacteristicfunctionisintroducedinChapterVIviaafirst-principlesdevel-
opment of Fourier series and the Fourier transform. In addition to the operational
calculus and inversion theorem, Herglotz’s theorem, Bochner’s theorem, and the
Cram´er–L´evy continuity theorem are given. Probabilistic applications include the
Chung–Fuchs criterion for recurrence of random walks on Rk, and the classical cen-
tral limit theorem for i.i.d. random vectors with finite second moments. The law of
rare events (i.e., Poisson approximation to binomial) is also included as a simple
illustration of the continuity theorem, although simple direct calculations are also
possible.
In Chapter VII, central limit theorems of Lindeberg and Lyapounov are derived.
Although there is some mention of stable and infinitely divisible laws, the full treat-
ment of infinite divisibility and L´evy–Khinchine representation is more properly
deferred to a study of stochastic processes with independent increments.
The Laplace transform is developed in Chapter VIII with Karamata’s Tauberian
theorem as the main goal. This includes a heavy dose of exponential size-biasing
techniques to go from probabilistic considerations to general Radon measures. The
standard operational calculus for the Laplace transform is developed along the way.
Random series of independent summands are treated in Chapter IX. This
includes the mean square summability criterion and Kolmogorov’s three series cri-
teria based on Kolmogorov’s maximal inequality. An alternative proof to that
presented in Chapter IV for Kolmogorov’s strong law of large numbers is given, to-
gether with the Marcinkiewicz and Zygmund extension, based on these criteria and
Kronecker’s lemma. The equivalence of a.s. convergence, convergence in probability,
and convergence in distribution for seriesof independent summands is also included.
In Chapter X, Kolmogorov’s consistency conditions lead to the construction of
probabilitymeasuresontheCartesianproductofinfinitelymanyspaces.Applications
includeaconstructionofGaussianrandomfieldsanddiscreteparameterMarkovpro-
cesses. The deficiency of Kolmogorov’s construction of a model for Brownian motion
is described, and the L´evy–Ciesielski “wavelet” construction is provided.
Basic properties of Brownian motion are taken up in Chapter XI. Included are
variousrescalingsandtime-inversionproperties,togetherwiththefine-scalestructure
embodied in the law of the iterated logarithm for Brownian motion.
InChapterXIImanyofthebasicnotionsintroducedinthetextaretiedtogethervia
further considerations of Brownian motion. In particular, this chapter revisits condi-
tionalprobabilitiesintermsoftheMarkovandstrongMarkovpropertiesforBrownian
motion, stopping times, and the optional stopping and/or sampling theorems for
Brownianmotionandrelatedmartingales,andleadstoweakconvergenceofrescaled
randomwalkswithfinitesecondmomentstoBrownianmotion,i.e.,Donsker’sinvari-
ance principle or the functional central limit theorem, via the Skorokhod embedding
theorem.
PREFACE ix
Thetextisconcludedwithahistoricaloverview,ChapterXIII,onBrownianmotion
anditsfundamentalroleinapplicationstophysics,financialmathematics,andpartial
differential equations, which inspired its creation.
Mostofthematerialinthisbookhasbeenusedbyusingraduateprobabilitycourses
taughtattheUniversityofArizona,IndianaUniversity,andOregonStateUniversity.
TheauthorsaregratefultoVirginiaJonesforsuperbword-processingskillsthatwent
intothepreparationofthistext.Also,twoOregonStateUniversitygraduatestudents,
Jorge Ramirez and David Wing, did an outstanding job in uncovering and reporting
variousbugsinearlierdraftsofthistext.ThanksgotoProfessorAnirbanDasgupta,
the editorial staff at Springer and anonymous referees for their insightful remarks.
Finally, the authors gratefully acknowledge partial support from NSF grants DMS
04-06143 and CMG 03-27705, which facilitated the writing of this book.
Rabi Bhattacharya
Edward C. Waymire
March 2007
Contents
Preface vii
I Random Maps, Distribution, and Mathematical Expectation 1
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
II Independence, Conditional Expectation 19
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
III Martingales and Stopping Times 37
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
IV Classical Zero–One Laws, Laws of Large Numbers and Deviations 49
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
V Weak Convergence of Probability Measures 59
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
VI Fourier Series, Fourier Transform, and Characteristic Functions 73
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
VII Classical Central Limit Theorems 99
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
VIII Laplace Transforms and Tauberian Theorem 107
Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118