The Probability Lifesaver - All the tools you need to understand chance

737 Pages·2017·7.833 MB·english

by  Steven J. Miller

b a
b
o
i
r
l
i
p
t
y
r
l
e
i
v
f
a
es
All the tools you need to understand chance
Steven J. Miller
PRINCETON UNIVERSITY PRESS
Princeton and Oxford
Copyright(cid:2)c 2017byPrincetonUniversityPress
PublishedbyPrincetonUniversityPress,41WilliamStreet,
Princeton,NewJersey08540
IntheUnitedKingdom:PrincetonUniversityPress,6OxfordStreet,
Woodstock,OxfordshireOX201TR
press.princeton.edu
LibraryofCongressCataloging-in-PublicationData
Names:Miller,StevenJ.,1974–
Title:Theprobabilitylifesaver:allthetoolsyouneedtounderstand
chance/StevenJ.Miller.
Description:Princeton:PrincetonUniversityPress,[2017]|Series:A
Princetonlifesaverstudyguide|Includesbibliographicalreferencesandindex.
Identifiers:LCCN2016040785|ISBN9780691149547(hardcover:alk.paper)|
ISBN9780691149554(pbk.:alk.paper)
Subjects:LCSH:Probabilities.|Chance.|Gamesofchance(Mathematics)|
Randomvariables.
Classification:LCCQA273.M551852017|DDC519.2-dc23LCrecord
availableathttps://lccn.loc.gov/2016040785
BritishLibraryCataloging-in-PublicationDataisavailable
ThisbookhasbeencomposedinTimesNewRomanwithStencilandAvantGarde
TypesetbyNovaTechsetPvtLtd,Bangalore,India
PrintedintheUnitedStatesofAmerica
Contents
NotetoReaders xv
HowtoUseThisBook xix
I GeneralTheory 1
1 Introduction 3
1.1 BirthdayProblem 4
1.1.1 StatingtheProblem 4
1.1.2 SolvingtheProblem 6
1.1.3 GeneralizingtheProblemandSolution:Efficiencies 11
1.1.4 NumericalTest 14
1.2 FromShootingHoopstotheGeometricSeries 16
1.2.1 TheProblemandItsSolution 16
1.2.2 RelatedProblems 21
1.2.3 GeneralProblemSolvingTips 25
1.3 Gambling 27
1.3.1 The2008SuperBowlWager 28
1.3.2 ExpectedReturns 28
1.3.3 TheValueofHedging 29
1.3.4 Consequences 31
1.4 Summary 31
1.5 Exercises 34
2 BasicProbabilityLaws 40
2.1 Paradoxes 41
2.2 SetTheoryReview 43
2.2.1 CodingDigression 47
2.2.2 SizesofInfinityandProbabilities 48
2.2.3 OpenandClosedSets 50
2.3 OutcomeSpaces,Events,andtheAxiomsofProbability 52
2.4 AxiomsofProbability 57
2.5 BasicProbabilityRules 59
2.5.1 LawofTotalProbability 60
2.5.2 ProbabilitiesofUnions 61
2.5.3 ProbabilitiesofInclusions 64
2.6 ProbabilitySpacesandσ-algebras 65
2.7 Appendix:ExperimentallyFindingFormulas 70
2.7.1 ProductRuleforDerivatives 71
2.7.2 ProbabilityofaUnion 72
2.8 Summary 73
2.9 Exercises 73
3 CountingI:Cards 78
3.1 FactorialsandBinomialCoefficients 79
3.1.1 TheFactorialFunction 79
3.1.2 BinomialCoefficients 82
3.1.3 Summary 87
3.2 Poker 88
3.2.1 Rules 88
3.2.2 Nothing 90
3.2.3 Pair 92
3.2.4 TwoPair 95
3.2.5 ThreeofaKind 96
3.2.6 Straights,Flushes,andStraightFlushes 96
3.2.7 FullHouseandFourofaKind 97
3.2.8 PracticePokerHand:I 98
3.2.9 PracticePokerHand:II 100
3.3 Solitaire 101
3.3.1 Klondike 102
3.3.2 AcesUp 105
3.3.3 FreeCell 107
3.4 Bridge 108
3.4.1 Tic-tac-toe 109
3.4.2 NumberofBridgeDeals 111
3.4.3 TrumpSplits 117
3.5 Appendix:CodingtoComputeProbabilities 120
3.5.1 TrumpSplitandCode 120
3.5.2 PokerHandCodes 121
3.6 Summary 124
3.7 Exercises 124
4 ConditionalProbability,Independence,and
Bayes’Theorem 128
4.1 ConditionalProbabilities 129
4.1.1 GuessingtheConditionalProbabilityFormula 131
4.1.2 ExpectedCountsApproach 132
4.1.3 VennDiagramApproach 133
4.1.4 TheMontyHallProblem 135
4.2 TheGeneralMultiplicationRule 136
4.2.1 Statement 136
4.2.2 PokerExample 136
4.2.3 HatProblemandErrorCorrectingCodes 138
4.2.4 AdvancedRemark:DefinitionofConditionalProbability 138
4.3 Independence 139
4.4 Bayes’Theorem 142
4.5 PartitionsandtheLawofTotalProbability 147
4.6 Bayes’TheoremRevisited 150
4.7 Summary 151
4.8 Exercises 152
5 CountingII:Inclusion-Exclusion 156
5.1 FactorialandBinomialProblems 157
5.1.1 “Howmany”versus“What’stheprobability” 157
5.1.2 ChoosingGroups 159
5.1.3 CircularOrderings 160
5.1.4 ChoosingEnsembles 162
5.2 TheMethodofInclusion-Exclusion 163
5.2.1 SpecialCasesoftheInclusion-ExclusionPrinciple 164
5.2.2 StatementoftheInclusion-ExclusionPrinciple 167
5.2.3 JustificationoftheInclusion-ExclusionFormula 168
5.2.4 UsingInclusion-Exclusion:SuitedHand 171
5.2.5 TheAtLeasttoExactlyMethod 173
5.3 Derangements 176
5.3.1 CountingDerangements 176
5.3.2 TheProbabilityofaDerangement 178
5.3.3 CodingDerangementExperiments 178
5.3.4 ApplicationsofDerangements 179
5.4 Summary 181
5.5 Exercises 182
6 CountingIII:AdvancedCombinatorics 186
6.1 BasicCounting 187
6.1.1 EnumeratingCases:I 187
6.1.2 EnumeratingCases:II 188
6.1.3 SamplingWithandWithoutReplacement 192
6.2 WordOrderings 199
6.2.1 CountingOrderings 200
6.2.2 MultinomialCoefficients 202
6.3 Partitions 205
6.3.1 TheCookieProblem 205
6.3.2 Lotteries 207
6.3.3 AdditionalPartitions 212
6.4 Summary 214
6.5 Exercises 215
II IntroductiontoRandomVariables 219
7 IntroductiontoDiscreteRandomVariables 221
7.1 DiscreteRandomVariables:Definition 221
7.2 DiscreteRandomVariables:PDFs 223
7.3 DiscreteRandomVariables:CDFs 226
7.4 Summary 233
7.5 Exercises 235
8 IntroductiontoContinuousRandomVariables 238
8.1 FundamentalTheoremofCalculus 239
8.2 PDFsandCDFs:Definitions 241
8.3 PDFsandCDFs:Examples 243
8.4 ProbabilitiesofSingletonEvents 248
8.5 Summary 250
8.6 Exercises 250
9 Tools:Expectation 254
9.1 CalculusMotivation 255
9.2 ExpectedValuesandMoments 257
9.3 MeanandVariance 261
9.4 JointDistributions 265
9.5 LinearityofExpectation 269
9.6 PropertiesoftheMeanandtheVariance 274
9.7 SkewnessandKurtosis 279
9.8 Covariances 280
9.9 Summary 281
9.10 Exercises 281
10 Tools:ConvolutionsandChangingVariables 285
10.1 Convolutions:DefinitionsandProperties 286
10.2 Convolutions:DieExample 289
10.2.1 TheoreticalCalculation 289
10.2.2 ConvolutionCode 290
10.3 ConvolutionsofSeveralVariables 291
10.4 ChangeofVariableFormula:Statement 294
10.5 ChangeofVariablesFormula:Proof 297
10.6 Appendix:ProductsandQuotients
ofRandomVariables 302
10.6.1 DensityofaProduct 302
10.6.2 DensityofaQuotient 303
10.6.3 Example:QuotientofExponentials 304
10.7 Summary 305
10.8 Exercises 305
11 Tools:DifferentiatingIdentities 309
11.1 GeometricSeriesExample 310
11.2 MethodofDifferentiatingIdentities 313
11.3 ApplicationstoBinomialRandomVariables 314
11.4 ApplicationstoNormalRandomVariables 317
11.5 ApplicationstoExponential
RandomVariables 320
11.6 Summary 322
11.7 Exercises 323
III SpecialDistributions 325
12 DiscreteDistributions 327
12.1 TheBernoulliDistribution 328
12.2 TheBinomialDistribution 328
12.3 TheMultinomialDistribution 332
12.4 TheGeometricDistribution 335
12.5 TheNegativeBinomialDistribution 336
12.6 ThePoissonDistribution 340
12.7 TheDiscreteUniformDistribution 343
12.8 Exercises 346
13 ContinuousRandomVariables:
UniformandExponential 349
13.1 TheUniformDistribution 349
13.1.1 MeanandVariance 350
13.1.2 SumsofUniformRandomVariables 352
13.1.3 Examples 354
13.1.4 GeneratingRandomNumbersUniformly 356
13.2 TheExponentialDistribution 357
13.2.1 MeanandVariance 357
13.2.2 SumsofExponentialRandomVariables 361
13.2.3 ExamplesandApplicationsofExponentialRandom
Variables 364
13.2.4 GeneratingRandomNumbersfrom
ExponentialDistributions 365
13.3 Exercises 367
14 ContinuousRandomVariables:TheNormalDistribution 371
14.1 DeterminingtheNormalizationConstant 372
14.2 MeanandVariance 375
14.3 SumsofNormalRandomVariables 379
14.3.1 Case1:μ =μ =0andσ2 =σ2=1 380
X Y X Y
14.3.2 Case2:Generalμ ,μ andσ2,σ2 383
X Y X Y
14.3.3 SumsofTwoNormals:FasterAlgebra 385
14.4 GeneratingRandomNumbersfrom
NormalDistributions 386
14.5 ExamplesandtheCentralLimitTheorem 392
14.6 Exercises 393
15 TheGammaFunctionandRelatedDistributions 398
15.1 Existenceof(cid:3)(s) 398
15.2 TheFunctionalEquationof(cid:3)(s) 400
15.3 TheFactorialFunctionand(cid:3)(s) 404
15.4 SpecialValuesof(cid:3)(s) 405
15.5 TheBetaFunctionandtheGammaFunction 407
15.5.1 ProofoftheFundamentalRelation 408
15.5.2 TheFundamentalRelationand(cid:3)(1/2) 410
15.6 TheNormalDistributionandtheGammaFunction 411
15.7 FamiliesofRandomVariables 412
15.8 Appendix:CosecantIdentityProofs 413
15.8.1 TheCosecantIdentity:FirstProof 414
15.8.2 TheCosecantIdentity:SecondProof 418
15.8.3 TheCosecantIdentity:SpecialCases=1/2 421
15.9 CauchyDistribution 423
15.10 Exercises 424
16 TheChi-squareDistribution 427
16.1 OriginoftheChi-squareDistribution 429
16.2 MeanandVarianceofX∼χ2(1) 430
16.3 Chi-squareDistributionsandSumsofNormalRandom
Variables 432
16.3.1 SumsofSquaresbyDirectIntegration 434
16.3.2 SumsofSquaresbytheChangeofVariablesTheorem 434
16.3.3 SumsofSquaresbyConvolution 439
16.3.4 SumsofChi-squareRandomVariables 441
16.4 Summary 442
16.5 Exercises 443
IV LimitTheorems 447
17 InequalitiesandLawsofLargeNumbers 449
17.1 Inequalities 449
17.2 Markov’sInequality 451
17.3 Chebyshev’sInequality 453
17.3.1 Statement 453
17.3.2 Proof 455
17.3.3 NormalandUniformExamples 457
17.3.4 ExponentialExample 458
17.4 TheBooleandBonferroniInequalities 459
17.5 TypesofConvergence 461
17.5.1 ConvergenceinDistribution 461
17.5.2 ConvergenceinProbability 463
17.5.3 AlmostSureandSureConvergence 463
17.6 WeakandStrongLawsofLargeNumbers 464
17.7 Exercises 465
18 Stirling’sFormula 469
18.1 Stirling’sFormulaandProbabilities 471
18.2 Stirling’sFormulaandConvergenceofSeries 473
18.3 FromStirlingtotheCentralLimitTheorem 474
18.4 IntegralTestandthePoorMan’sStirling 478
18.5 ElementaryApproachestowards
Stirling’sFormula 482
18.5.1 DyadicDecompositions 482
18.5.2 LowerBoundstowardsStirling:I 484
18.5.3 LowerBoundstowardStirlingII 486
18.5.4 LowerBoundstowardsStirling:III 487
18.6 StationaryPhaseandStirling 488
18.7 TheCentralLimitTheoremandStirling 490
18.8 Exercises 491
19 GeneratingFunctionsandConvolutions 494
19.1 Motivation 494
19.2 Definition 496
19.3 UniquenessandConvergenceof
GeneratingFunctions 501
19.4 ConvolutionsI:DiscreteRandomVariables 503
19.5 ConvolutionsII:ContinuousRandomVariables 507
19.6 DefinitionandPropertiesofMomentGenerating
Functions 512
19.7 ApplicationsofMomentGeneratingFunctions 520
19.8 Exercises 524
20 ProofoftheCentralLimitTheorem 527
20.1 KeyIdeasoftheProof 527
20.2 StatementoftheCentralLimitTheorem 529
20.3 Means,Variances,andStandardDeviations 531
20.4 Standardization 533
20.5 NeededMomentGeneratingFunctionResults 536
20.6 SpecialCase:SumsofPoisson
RandomVariables 539
20.7 ProofoftheCLTforGeneralSumsviaMGF 542
20.8 UsingtheCentralLimitTheorem 544
20.9 TheCentralLimitTheoremand
MonteCarloIntegration 545
20.10 Summary 546
20.11 Exercises 548
21 FourierAnalysisandtheCentralLimitTheorem 553
21.1 IntegralTransforms 554
21.2 ConvolutionsandProbabilityTheory 558
21.3 ProofoftheCentralLimitTheorem 562
21.4 Summary 565
21.5 Exercises 565
V AdditionalTopics 567
22 HypothesisTesting 569
22.1 Z-tests 570
22.1.1 NullandAlternativeHypotheses 570
22.1.2 SignificanceLevels 571
22.1.3 TestStatistics 573
22.1.4 One-sidedversusTwo-sidedTests 576
22.2 Onp-values 579
22.2.1 ExtraordinaryClaimsandp-values 580
22.2.2 Largep-values 580
22.2.3 Misconceptionsaboutp-values 581
22.3 Ont-tests 583
22.3.1 EstimatingtheSampleVariance 583
22.3.2 Fromz-teststot-tests 584
22.4 ProblemswithHypothesisTesting 587
22.4.1 TypeIErrors 587
22.4.2 TypeIIErrors 588
22.4.3 ErrorRatesandtheJusticeSystem 588
22.4.4 Power 590
22.4.5 EffectSize 590
22.5 Chi-squareDistributions,GoodnessofFit 590
22.5.1 Chi-squareDistributionsandTestsofVariance 591
22.5.2 Chi-squareDistributionsandt-distributions 595
22.5.3 GoodnessofFitforListData 595
22.6 TwoSampleTests 598
22.6.1 Two-samplez-test:KnownVariances 598
22.6.2 Two-samplet-test:UnknownbutSameVariances 600
22.6.3 UnknownandDifferentVariances 602
22.7 Summary 604
22.8 Exercises 605
23 DifferenceEquations,MarkovProcesses,
andProbability 607
23.1 FromtheFibonacciNumberstoRoulette 607
23.1.1 TheDouble-plus-oneStrategy 607
23.1.2 AQuickReviewoftheFibonacciNumbers 609
23.1.3 RecurrenceRelationsandProbability 610
23.1.4 DiscussionandGeneralizations 612
23.1.5 CodeforRouletteProblem 613
23.2 GeneralTheoryofRecurrenceRelations 614
23.2.1 Notation 614
23.2.2 TheCharacteristicEquation 615