Essentials of Stochastic Processes

268 Pages·2012·1.734 MB·English

by  Richard Durrett (auth.)

Springer Texts in Statistics
SeriesEditors:
G.Casella
S.Fienberg
I.Olkin
Forfurthervolumes:
http://www.springer.com/series/417

Richard Durrett
Essentials of Stochastic
Processes
Second Edition
123
RichardDurrett
DukeUniversity
DepartmentofMathematics
Box90320
Durham
NorthCarolina
USA
ISSN1431-875X
ISBN978-1-4614-3614-0 ISBN978-1-4614-3615-7(eBook)
DOI10.1007/978-1-4614-3615-7
SpringerNewYorkHeidelbergDordrechtLondon
LibraryofCongressControlNumber:2012937472
©SpringerScience+BusinessMedia,LLC1999,2012
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’slocation,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.
PermissionsforusemaybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violations
areliabletoprosecutionundertherespectiveCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
While the advice and information in this book are believed to be true and accurate at the date of
publication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityfor
anyerrorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,with
respecttothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
70
10-Sep
60
10-Jun
10-May
50
at expiry
40
30
20
10
0
500 520 540 560 580 600 620 640 660 680 700

Preface
Betweenthefirstundergraduatecourseinprobabilityandthefirstgraduatecourse
that uses measure theory, there are a number of courses that teach Stochastic
Processes to students with many different interests and with varying degrees of
mathematicalsophistication.Toallowreaders(andinstructors)tochoosetheirown
levelofdetail,manyoftheproofsbeginwithanonrigorousanswertothequestion
“Why is this true?” followed by a Proof that fills in the missing details. As it is
possibletodriveacarwithoutknowingabouttheworkingoftheinternalcombustion
engine, it is also possible to apply the theory of Markov chains without knowing
the details of the proofs.It is my personalphilosophythat probabilitytheory was
developed to solve problems, so most of our effort will be spent on analyzing
examples. Readers who want to master the subject will have to do more than a
fewofthe20dozencarefullychosenexercises.
ThisbookbeganasnotesItypedinthespringof1997asIwasteachingORIE
361 at Cornell for the second time. In Spring 2009, the mathematics department
there introduced its own version of this course, MATH 474. This started me on
the task of preparing the second edition. The plan was to have this finished in
Spring2010afterthesecondtimeItaughtthecourse,butwhenMayrolledaround
completing the book lost out to getting ready to move to Durham after 25 years
in Ithaca. In the Fall of 2011,I taughtDuke’s versionof the course, Math 216,to
20 undergradsand 12 graduate students and over the Christmas break the second
editionwascompleted.
Thesecondeditiondifferssubstantiallyfromthefirst,thoughcuriouslythelength
andthenumberofproblemshasremainedroughlyconstant.Throughoutthe book
there are many new examples and problems, with solutions that use the TI-83 to
eliminate the tediousdetails of solving linear equationsby hand.My studentstell
meIshouldjustuseMATLABandmaybeIwillforthenextedition.
The Markov chains chapter has been reorganized. The chapter on Poisson
processeshasmovedupfromthirdtosecond,andisnowfollowedbyatreatmentof
thecloselyrelatedtopicofrenewaltheory.ContinuoustimeMarkovchainsremain
fourth, with a new section on exit distributions and hitting times, and reduced
coverageofqueueingnetworks.Martingales,a difficultsubjectforstudentsatthis
vii
viii Preface
level,nowcomesfifth,inordertosetthestagefortheiruseinanewsixthchapter
onmathematicalfinance.Thetreatmentoffinanceexpandsthetwosectionsofthe
previous treatment to include American options and the the capital asset pricing
model.BrownianmotionmakesacameoappearanceinthediscussionoftheBlack-
Scholestheorem,butincontrasttothepreviousedition,isnotdiscussedindetail.
As usual the second edition has profited from people who have told me about
typosoverthelastdozenyears.Ifyoufindnewones,email:[email protected].
RickDurrett
Contents
1 MarkovChains............................................................... 1
1.1 DefinitionsandExamples............................................. 1
1.2 MultistepTransitionProbabilities .................................... 8
1.3 ClassificationofStates ................................................ 12
1.4 StationaryDistributions ............................................... 20
1.5 LimitBehavior......................................................... 26
1.6 SpecialExamples...................................................... 34
1.6.1 DoublyStochasticChains..................................... 34
1.6.2 DetailedBalanceCondition................................... 37
1.6.3 Reversibility ................................................... 42
1.6.4 TheMetropolis-HastingsAlgorithm ......................... 43
1.7 ProofsoftheMainTheorems(cid:2)........................................ 46
1.8 ExitDistributions...................................................... 52
1.9 ExitTimes.............................................................. 58
1.10 InfiniteStateSpaces(cid:2).................................................. 64
1.11 ChapterSummary...................................................... 71
1.12 Exercises............................................................... 75
2 PoissonProcesses............................................................. 93
2.1 ExponentialDistribution .............................................. 93
2.2 DefiningthePoissonProcess ......................................... 97
2.3 CompoundPoissonProcesses......................................... 103
2.4 Transformations........................................................ 106
2.4.1 Thinning ....................................................... 106
2.4.2 Superposition.................................................. 107
2.4.3 Conditioning................................................... 108
2.5 ChapterSummary...................................................... 110
2.6 Exercises............................................................... 111
3 RenewalProcesses ........................................................... 119
3.1 LawsofLargeNumbers............................................... 119
ix
x Contents
3.2 ApplicationstoQueueingTheory..................................... 124
3.2.1 GI/G/1Queue.................................................. 125
3.2.2 CostEquations................................................. 126
3.2.3 M/G/1Queue.................................................. 127
3.3 AgeandResidualLife* ............................................... 129
3.3.1 DiscreteCase.................................................. 129
3.3.2 GeneralCase................................................... 131
3.4 ChapterSummary...................................................... 133
3.5 Exercises............................................................... 134
4 ContinuousTimeMarkovChains.......................................... 139
4.1 DefinitionsandExamples............................................. 139
4.2 ComputingtheTransitionProbability ................................ 144
4.3 LimitingBehavior ..................................................... 149
4.4 ExitDistributionsandHittingTimes ................................. 156
4.5 MarkovianQueues..................................................... 160
4.6 QueueingNetworks*.................................................. 167
4.7 ChapterSummary...................................................... 174
4.8 Exercises............................................................... 175
5 Martingales................................................................... 185
5.1 ConditionalExpectation............................................... 185
5.2 Examples,BasicProperties ........................................... 188
5.3 GamblingStrategies,StoppingTimes................................ 192
5.4 Applications............................................................ 195
5.5 Convergence ........................................................... 200
5.6 Exercises............................................................... 204
6 MathematicalFinance....................................................... 209
6.1 TwoSimpleExamples................................................. 209
6.2 BinomialModel ....................................................... 213
6.3 ConcreteExamples.................................................... 218
6.4 CapitalAssetPricingModel .......................................... 223
6.5 AmericanOptions ..................................................... 226
6.6 Black-ScholesFormula................................................ 230
6.7 CallsandPuts.......................................................... 234
6.8 Exercises............................................................... 236
A ReviewofProbability........................................................ 241
A.1 Probabilities,Independence........................................... 241
A.2 RandomVariables,Distributions...................................... 245
A.3 ExpectedValue,Moments............................................. 251
References......................................................................... 259
Index............................................................................... 261