Probability: a lively introduction

547 Pages·2018·2.908 MB·English

by  Henk Tijms

Probability:ALivelyIntroduction
Probabilityhasapplicationsinmanyareasofmodernscience,nottomentioninour
dailylife,anditsimportanceasamathematicaldisciplinecannotbeoverrated.This
engagingbook,withitseasytofollowwritingstyle,providesacomprehensive,yet
concise,introductiontothesubject.Itcoversallofthestandardmaterialfor
undergraduateandfirst-year-graduate-levelcourses,aswellasmanytopicsthatare
usuallynotfoundinstandardtexts–suchasBayesianinference,MarkovchainMonte
Carlosimulation,andChernoffbounds.
Thestudent-friendlytexthasthefollowingadditionalfeatures:
• Istheresultofmanyyearsofteachingandfeedbackfromstudents
• Stresseswhyprobabilityissorelevantandhowtoapplyit
• Offersmanyreal-worldexamplestosupportthetheory
• Includesmorethan750problemswithdetailedsolutionsoftheodd-numbered
problems
• Givesstudentsconfidenceintheirownproblem-solvingskills
henk tijmsisProfessorEmeritusattheVrijeUniversityinAmsterdam.Heisthe
authorofseveraltextbooksandnumerouspapersonappliedprobabilityandstochastic
optimization.In2008HenkreceivedtheprestigiousINFORMSExpositoryWriting
Awardforhispublicationsandbooks.Hisactivitiesalsoincludethepopularizationof
probabilityforhigh-schoolstudentsandthegeneralpublic.

Probability: A Lively Introduction
HENK TIJMS
VrijeUniversiteit,Amsterdam
UniversityPrintingHouse,CambridgeCB28BS,UnitedKingdom
OneLibertyPlaza,20thFloor,NewYork,NY10006,USA
477WilliamstownRoad,PortMelbourne,VIC3207,Australia
4843/24,2ndFloor,AnsariRoad,Daryaganj,Delhi–110002,India
79AnsonRoad,#06–04/06,Singapore079906
CambridgeUniversityPressispartoftheUniversityofCambridge.
ItfurtherstheUniversity’smissionbydisseminatingknowledgeinthepursuitof
education,learning,andresearchatthehighestinternationallevelsofexcellence.
www.cambridge.org
Informationonthistitle:www.cambridge.org/9781108418744
DOI:10.1017/9781108291361
©HenkTijms2018
Thispublicationisincopyright.Subjecttostatutoryexception
andtotheprovisionsofrelevantcollectivelicensingagreements,
noreproductionofanypartmaytakeplacewithoutthewritten
permissionofCambridgeUniversityPress.
Firstpublished2018
PrintedintheUnitedKingdombyClays,StIvesplc
AcataloguerecordforthispublicationisavailablefromtheBritishLibrary.
LibraryofCongressCataloging-in-PublicationData
Names:Tijms,H.
Title:Probability:alivelyintroduction/HenkTijms,VrijeUniversiteit,Amsterdam.
Description:Cambridge:CambridgeUniversityPress,2017.|Includesindex.
Identifiers:LCCN2017014908|ISBN9781108418744(Hardback:alk.paper)|
ISBN9781108407847(pbk.:alk.paper)
Subjects:LCSH:Probabilities–Textbooks.
Classification:LCCQA273.2.T552017|DDC519.2–dc23LCrecord
availableathttps://lccn.loc.gov/2017014908
ISBN978-1-108-41874-4Hardback
ISBN978-1-108-40784-7Paperback
Additionalresourcesforthispublicationatwww.cambridge.org/TijmsProbability
CambridgeUniversityPresshasnoresponsibilityforthepersistenceoraccuracyof
URLsforexternalorthird-partyinternetwebsitesreferredtointhispublication
anddoesnotguaranteethatanycontentonsuchwebsitesis,orwillremain,
accurateorappropriate.
(cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1) (cid:1)
Contents
Preface pageix
1 FoundationsofProbabilityTheory 1
1.1 ProbabilisticFoundations 3
1.2 ClassicalProbabilityModel 7
1.3 GeometricProbabilityModel 15
1.4 CompoundChanceExperiments 19
1.5 SomeBasicRules 25
1.6 Inclusion–ExclusionRule 36
2 ConditionalProbability 42
2.1 ConceptofConditionalProbability 42
2.2 ChainRuleforConditionalProbabilities 47
2.3 LawofConditionalProbability 54
2.4 Bayes’RuleinOddsForm 67
2.5 BayesianInference−DiscreteCase 77
3 DiscreteRandomVariables 85
3.1 ConceptofaRandomVariable 85
3.2 ExpectedValue 89
3.3 ExpectedValueofSumsofRandomVariables 99
3.4 SubstitutionRuleandVariance 106
3.5 IndependenceofRandomVariables 113
3.6 BinomialDistribution 118
3.7 PoissonDistribution 124
3.8 HypergeometricDistribution 135
3.9 OtherDiscreteDistributions 140
v
vi Contents
4 ContinuousRandomVariables 146
4.1 ConceptofProbabilityDensity 147
4.2 ExpectedValueofaContinuousRandomVariable 156
4.3 SubstitutionRuleandtheVariance 160
4.4 UniformandTriangularDistributions 164
4.5 ExponentialDistribution 167
4.6 Gamma,Weibull,andBetaDistributions 177
4.7 NormalDistribution 180
4.8 OtherContinuousDistributions 193
4.9 Inverse-TransformationMethodandSimulation 198
4.10 Failure-RateFunction 202
4.11 ProbabilityDistributionsandEntropy 205
5 JointlyDistributedRandomVariables 209
5.1 JointProbabilityMassFunction 209
5.2 JointProbabilityDensityFunction 212
5.3 MarginalProbabilityDensities 219
5.4 TransformationofRandomVariables 228
5.5 CovarianceandCorrelationCoefficient 233
6 MultivariateNormalDistribution 239
6.1 BivariateNormalDistribution 239
6.2 MultivariateNormalDistribution 248
6.3 MultidimensionalCentralLimitTheorem 250
6.4 Chi-SquareTest 257
7 ConditioningbyRandomVariables 261
7.1 ConditionalDistributions 262
7.2 LawofConditionalProbabilityforRandomVariables 269
7.3 LawofConditionalExpectation 276
7.4 ConditionalExpectationasaComputationalTool 283
7.5 BayesianInference−ContinuousCase 294
8 GeneratingFunctions 302
8.1 GeneratingFunctions 302
8.2 BranchingProcessesandGeneratingFunctions 311
8.3 Moment-GeneratingFunctions 313
8.4 CentralLimitTheoremRevisited 318
9 AdditionalTopicsinProbability 321
9.1 BoundsandInequalities 321
9.2 StrongLawofLargeNumbers 327
Contents vii
9.3 KellyBettingSystem 335
9.4 Renewal–RewardProcesses 339
10 Discrete-TimeMarkovChains 348
10.1 MarkovChainModel 349
10.2 Time-DependentAnalysisofMarkovChains 357
10.3 AbsorbingMarkovChains 362
10.4 Long-RunAnalysisofMarkovChains 373
10.5 MarkovChainMonteCarloSimulation 386
11 Continuous-TimeMarkovChains 403
11.1 MarkovChainModel 403
11.2 Time-DependentProbabilities 414
11.3 LimitingProbabilities 420
AppendixA:CountingMethods 438
AppendixB:BasicsofSetTheory 443
AppendixC:SomeBasicResultsfromCalculus 447
AppendixD:BasicsofMonteCarloSimulation 451
AnswerstoOdd-NumberedProblems 463
Index 532

Preface
Why do so many students find probability difficult? Even the most mathe-
matically competent often find probability a subject that is difficult to use
and understand. The difficulty stems from the fact that most problems in
probability, even ones that are easy to understand, cannot be solved by using
cookbook recipes as is sometimes the case in other areas of mathematics.
Instead, each new problem often requires imagination and creative thinking.
That is why probability is difficult, but also why it is fun and engaging.
Probability is a fascinating subject and I hope, in this book, to share my
enthusiasmforthesubject.
Probabilityisbesttaughttobeginning students byusingmotivating exam-
plesandproblems,andasolutionapproachthatgivesthestudentsconfidence
tosolveproblemsontheirown.Theexamplesandproblemsshouldberelevant,
clear, and instructive. This book is not written in a theorem–proof style, but
proofsflowwiththesubsequenttextandnomathematicsisintroducedwithout
specificexamplesandapplicationstomotivatethetheory.Itdistinguishesitself
from other introductory probability texts by its emphasis on why probability
is so relevant and how to apply it. Every attempt has been made to create a
student-friendlybookandtohelpstudentsunderstandwhattheyarelearning,
notjusttolearnit.
Thistextbookisdesignedforafirstcourseinprobabilityatanundergraduate
level or first-year graduate level. It covers all of the standard material for
such courses, but it also contains many topics that are usually not found in
introductoryprobabilitybooks–suchasstochasticsimulation.Theemphasis
throughoutthebookisonprobability,butattentionisalsogiventostatistics.In
particular,Bayesianinferenceisdiscussedatlengthandillustratedwithseveral
illuminatingexamples.Thebookcanbeusedinavarietyofdisciplines,rang-
ing from applied mathematics and statistics to computer science, operations
ix