Understanding probability

573 Pages·2012·3.4 MB·English

by  H C Tijms

UnderstandingProbability
UnderstandingProbabilityisauniqueandstimulatingapproachtoafirstcoursein
probability.Thefirstpartofthebookdemystifiesprobabilityandusesmanywonderful
probabilityapplicationsfromeverydaylifetohelpthereaderdevelopafeelfor
probabilities.Thesecondpart,coveringawiderangeoftopics,teachesclearlyand
simplythebasicsofprobability.
ThisfullyrevisedThirdEditionhasbeenpackedwithevenmoreexercisesand
examples,anditincludesnewsectionsonBayesianinference,MarkovchainMonte
Carlosimulation,hittingprobabilitiesinrandomwalksandBrownianmotion,anda
newchapteroncontinuous-timeMarkovchainswithapplications.Hereyouwillfind
allthematerialtaughtinanintroductoryprobabilitycourse.Thefirstpartofthebook,
withitseasy-goingstyle,canbereadbyanybodywithareasonablebackgroundinhigh
schoolmathematics.Thesecondpartofthebookrequiresabasiccourseincalculus.
Henk Tijms isEmeritusProfessoratVrijeUniversityinAmsterdam.Heisthe
authorofseveraltextbooksandnumerouspapersonappliedprobabilityandstochastic
optimization.In2008HenkTijmsreceivedtheprestigiousINFORMSExpository
WritingAwardforhispublicationsandbooks.

Understanding Probability
ThirdEdition
HENK TIJMS
VrijeUniversity,Amsterdam
cambridge university press
Cambridge,NewYork,Melbourne,Madrid,CapeTown,
Singapore,Sa˜oPaulo,Delhi,MexicoCity
CambridgeUniversityPress
TheEdinburghBuilding,CambridgeCB28RU,UK
PublishedintheUnitedStatesofAmericabyCambridgeUniversityPress,NewYork
www.cambridge.org
Informationonthistitle:www.cambridge.org/9781107658561
(cid:2)C H.Tijms2004,2007,2012
Thispublicationisincopyright.Subjecttostatutoryexception
andtotheprovisionsofrelevantcollectivelicensingagreements,
noreproductionofanypartmaytakeplacewithoutthewritten
permissionofCambridgeUniversityPress.
Firstpublished2004
Secondedition2007
Thirdedition2012
PrintedintheUnitedKingdomattheUniversityPress,Cambridge
AcatalogrecordforthispublicationisavailablefromtheBritishLibrary
LibraryofCongressCataloginginPublicationdata
Tijms,H.C.
Understandingprobability/HenkTijms.–3rded.
p. cm.
Includesbibliographicalreferencesandindex.
ISBN978-1-107-65856-1(pbk.)
1.Probabilities. 2.Mathematicalanalysis. 3.Chance. I.Title.
QA273.T48 2012
519.2–dc23 2012010536
ISBN978-1-107-65856-1Paperback
CambridgeUniversityPresshasnoresponsibilityforthepersistenceor
accuracyofURLsforexternalorthird-partyinternetwebsitesreferredto
inthispublication,anddoesnotguaranteethatanycontentonsuch
websitesis,orwillremain,accurateorappropriate.
Contents
Preface pageix
Introduction 1
PARTONE:PROBABILITYINACTION 9
1 Probabilityquestions 11
2 Lawoflargenumbersandsimulation 18
2.1 Lawoflargenumbersforprobabilities 19
2.2 Basicprobabilityconcepts 28
2.3 Expectedvalueandthelawoflargenumbers 34
2.4 Drunkard’swalk 38
2.5 St.Petersburgparadox 41
2.6 Rouletteandthelawoflargenumbers 44
2.7 Kellybettingsystem 46
2.8 Random-numbergenerator 52
2.9 Simulatingfromprobabilitydistributions 57
2.10 Problems 65
3 Probabilitiesineverydaylife 75
3.1 Birthdayproblem 76
3.2 Couponcollector’sproblem 83
3.3 Craps 86
3.4 Gamblingsystemsforroulette 90
3.5 Gambler’sruinproblem 93
3.6 Optimalstopping 95
v
vi Contents
3.7 The1970draftlottery 98
3.8 Problems 102
4 Rareeventsandlotteries 108
4.1 Binomialdistribution 109
4.2 Poissondistribution 111
4.3 Hypergeometricdistribution 129
4.4 Problems 136
5 Probabilityandstatistics 142
5.1 Normalcurve 144
5.2 Conceptofstandarddeviation 152
5.3 Square-rootlaw 160
5.4 Centrallimittheorem 162
5.5 Graphicalillustrationofthecentrallimittheorem 166
5.6 Statisticalapplications 168
5.7 Confidenceintervalsforsimulations 171
5.8 Centrallimittheoremandrandomwalks 177
5.9 Brownianmotion 186
5.10 FalsifieddataandBenford’slaw 196
5.11 Normaldistributionstrikesagain 202
5.12 Statisticsandprobabilitytheory 203
5.13 Problems 206
6 ChancetreesandBayes’rule 212
6.1 MontyHalldilemma 213
6.2 Testparadox 218
6.3 Problems 222
PARTTWO:ESSENTIALSOFPROBABILITY 227
7 Foundationsofprobabilitytheory 229
7.1 Probabilisticfoundations 230
7.2 Compoundchanceexperiments 239
7.3 Somebasicrules 243
8 ConditionalprobabilityandBayes 256
8.1 Conditionalprobability 256
8.2 Lawofconditionalprobability 266
Contents vii
8.3 Bayes’ruleinoddsform 270
8.4 Bayesianstatistics−discretecase 278
9 Basicrulesfordiscreterandomvariables 283
9.1 Randomvariables 283
9.2 Expectedvalue 286
9.3 Expectedvalueofsumsofrandomvariables 290
9.4 Substitutionruleandvariance 293
9.5 Independenceofrandomvariables 299
9.6 Importantdiscreterandomvariables 303
10 Continuousrandomvariables 318
10.1 Conceptofprobabilitydensity 319
10.2 Expectedvalueofacontinuousrandom
variable 326
10.3 Substitutionruleandthevariance 328
10.4 Importantprobabilitydensities 333
10.5 Transformationofrandomvariables 353
10.6 Failureratefunction 357
11 Jointlydistributedrandomvariables 360
11.1 Jointprobabilitymassfunction 360
11.2 Jointprobabilitydensityfunction 362
11.3 Marginalprobabilitydensities 367
11.4 Transformationofrandomvariables 374
11.5 Covarianceandcorrelationcoefficient 377
12 Multivariatenormaldistribution 382
12.1 Bivariatenormaldistribution 382
12.2 Multivariatenormaldistribution 391
12.3 Multidimensionalcentrallimittheorem 393
12.4 Chi-squaretest 399
13 Conditioningbyrandomvariables 404
13.1 Conditionaldistributions 404
13.2 Lawofconditionalprobabilityforrandomvariables 411
13.3 Lawofconditionalexpectation 418
13.4 Conditionalexpectationasacomputationaltool 422
13.5 Bayesianstatistics−continuouscase 428
viii Contents
14 Generatingfunctions 435
14.1 Generatingfunctions 435
14.2 Moment-generatingfunctions 443
14.3 Chernoffbound 448
14.4 Stronglawoflargenumbersrevisited 450
14.5 Centrallimittheoremrevisited 454
14.6 Lawoftheiteratedlogarithm 456
15 Discrete-timeMarkovchains 459
15.1 Markovchainmodel 460
15.2 Time-dependentanalysisofMarkovchains 468
15.3 AbsorbingMarkovchains 472
15.4 Long-runanalysisofMarkovchains 481
15.5 MarkovchainMonteCarlosimulation 490
16 Continuous-timeMarkovchains 507
16.1 Markovchainmodel 507
16.2 Time-dependentprobabilities 516
16.3 Limitingprobabilities 520
Appendix Countingmethodsandex 532
Recommendedreading 538
Answerstoodd-numberedproblems 539
Bibliography 556
Index 558
Preface
Why do so many students find probability difficult? Could it be the way the
subjectistaughtinsomanytextbooks?WhenIwasastudent,aclassintopology
madeagreatimpressiononme.Theteacheraskedusnottotakenotesduring
thefirsthourofhislectures.Inthathour,heexplainedideasandconceptsfrom
topologyinanon-rigorous,intuitiveway.Allwehadtodowaslisteninorder
tograsptheconceptsbeingintroduced.Inthesecondhourofthelecture,the
materialfromthefirsthourwastreatedinamathematicallyrigorouswayand
thestudentswereallowedtotakenotes.Ilearnedalotfromthisapproachof
interweavingintuitionandformalmathematics.
Thisbookiswrittenverymuchinthesamespirit.Itfirsthelpsyoudevelop
a“feelforprobabilities”beforepresentingthemoreformalmathematics.The
bookisnotwritteninatheorem–proofstyle.Instead,itaimstoteachthenovice
theconceptsofprobabilitythroughtheuseofmotivatingandinsightfulexam-
ples.Nomathematicsareintroducedwithoutspecificexamplesandapplications
to motivate the theory. Instruction is driven by the need to answer questions
aboutprobabilityproblemsthataredrawnfromreal-worldcontexts.Thebook
isorganizedintotwoparts.PartOneisinformal,usingmanythought-provoking
examplesandproblemsfromtherealworldtohelpthereaderunderstandwhat
probabilityreallymeans.Probabilitycanbefunandengaging,butthisbeauti-
fulbranchofmathematicsisalsoindispensabletomodernscience.Thebasic
theoryofprobability–includingtopicsthatareusuallynotfoundinintroduc-
toryprobabilitybooks–iscoveredinPartTwo.Designedforanintroductory
probabilitycourse,thiscanbereadindependentlyofPartOne.Thebookcan
be used in a variety of disciplines ranging from finance and engineering to
mathematics and statistics. As well as for introductory courses, the book is
alsosuitableforself-study.TheprerequisiteknowledgeforPartTwoisabasic
course in calculus, but Part One can be read by anybody with a reasonable
backgroundinhighschoolmathematics.
This book distinguishes itself from other introductory probability texts by
its emphasis on why probability works and how to apply it. Simulation in
ix