Probability, Statistics, and Stochastic Processes, Second Edition

574 Pages·2012·6.12 MB·English

by  Peter Olofsson

PROBABILITY,
STATISTICS, AND
STOCHASTIC
PROCESSES

PROBABILITY,
STATISTICS, AND
STOCHASTIC
PROCESSES
Second Edition
PETEROLOFSSON
MIKAELANDERSSON
Copyright©2012byJohnWiley&Sons,Inc.Allrightsreserved
PublishedbyJohnWiley&Sons,Inc.,Hoboken,NewJersey
PublishedsimultaneouslyinCanada
Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,ortransmittedinanyformor
byanymeans,electronic,mechanical,photocopying,recording,scanning,orotherwise,exceptas
permittedunderSection107or108ofthe1976UnitedStatesCopyrightAct,withouteithertheprior
writtenpermissionofthePublisher,orauthorizationthroughpaymentoftheappropriateper-copyfeeto
theCopyrightClearanceCenter,Inc.,222RosewoodDrive,Danvers,MA01923,[phone],fax
[phone], or on the web at www.copyright.com. Requests to the Publisher for permission should
beaddressedtothePermissionsDepartment,JohnWiley&Sons,Inc.,111RiverStreet,Hoboken,NJ
07030, [phone], fax [phone], or online at http://www.wiley.com/go/permission.
LimitofLiability/DisclaimerofWarranty:Whilethepublisherandauthorhaveusedtheirbesteffortsin
preparingthisbook,theymakenorepresentationsorwarrantieswithrespecttotheaccuracyor
completenessofthecontentsofthisbookandspecificallydisclaimanyimpliedwarrantiesof
merchantabilityorfitnessforaparticularpurpose.Nowarrantymaybecreatedorextendedbysales
representativesorwrittensalesmaterials.Theadviceandstrategiescontainedhereinmaynotbesuitable
foryoursituation.Youshouldconsultwithaprofessionalwhereappropriate.Neitherthepublishernor
authorshallbeliableforanylossofprofitoranyothercommercialdamages,includingbutnotlimitedto
special,incidental,consequential,orotherdamages.
Forgeneralinformationonourotherproductsandservicesorfortechnicalsupport,pleasecontactour
CustomerCareDepartmentwithintheUnitedStatesat(800)762-2974,outsidetheUnitedStatesat(317)
572-3993orfax(317)572-4002.
Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthatappearsinprintmay
notbeavailableinelectronicformats.FormoreinformationaboutWileyproducts,visitourwebsiteat
www.wiley.com.
LibraryofCongressCataloging-in-PublicationData:
Olofsson,Peter,1963–
Probability,statistics,andstochasticprocesses/PeterOlofsson,Mikael
Andersson.–2nded.
p.cm.
ISBN978-0-470-88974-9(hardback)
1. Stochasticprocesses–Textbooks. 2. Probabilities–Textbooks. 3. Mathematical
statistics–Textbooks.I.Andersson,Mikael.II.Title.
QA274.O462012
519.2(cid:2)3–dc23
2011040205
PrintedintheUnitedStatesofAmerica
ISBN:9780470889749
10 9 8 7 6 5 4 3 2 1
CONTENTS
Preface xi
PrefacetotheFirstEdition xiii
1 BasicProbabilityTheory 1
1.1 Introduction 1
1.2 SampleSpacesandEvents 3
1.3 TheAxiomsofProbability 7
1.4 FiniteSampleSpacesandCombinatorics 15
1.4.1 Combinatorics 17
1.5 ConditionalProbabilityandIndependence 27
1.5.1 IndependentEvents 33
1.6 TheLawofTotalProbabilityandBayes’Formula 41
1.6.1 Bayes’Formula 47
1.6.2 GeneticsandProbability 54
1.6.3 RecursiveMethods 55
Problems 63
2 RandomVariables 76
2.1 Introduction 76
2.2 DiscreteRandomVariables 77
2.3 ContinuousRandomVariables 82
2.3.1 TheUniformDistribution 90
2.3.2 FunctionsofRandomVariables 92
2.4 ExpectedValueandVariance 95
2.4.1 TheExpectedValueofaFunctionofaRandomVariable 100
2.4.2 VarianceofaRandomVariable 104
v
vi CONTENTS
2.5 SpecialDiscreteDistributions 111
2.5.1 Indicators 111
2.5.2 TheBinomialDistribution 112
2.5.3 TheGeometricDistribution 116
2.5.4 ThePoissonDistribution 117
2.5.5 TheHypergeometricDistribution 121
2.5.6 DescribingDataSets 121
2.6 TheExponentialDistribution 123
2.7 TheNormalDistribution 127
2.8 OtherDistributions 131
2.8.1 TheLognormalDistribution 131
2.8.2 TheGammaDistribution 133
2.8.3 TheCauchyDistribution 134
2.8.4 MixedDistributions 135
2.9 LocationParameters 137
2.10 TheFailureRateFunction 139
2.10.1 UniquenessoftheFailureRateFunction 141
Problems 144
3 JointDistributions 156
3.1 Introduction 156
3.2 TheJointDistributionFunction 156
3.3 DiscreteRandomVectors 158
3.4 JointlyContinuousRandomVectors 160
3.5 ConditionalDistributionsandIndependence 164
3.5.1 IndependentRandomVariables 168
3.6 FunctionsofRandomVectors 172
3.6.1 Real-ValuedFunctionsofRandomVectors 172
3.6.2 TheExpectedValueandVarianceofaSum 176
3.6.3 Vector-ValuedFunctionsofRandomVectors 182
3.7 ConditionalExpectation 185
3.7.1 ConditionalExpectationasaRandomVariable 189
3.7.2 ConditionalExpectationandPrediction 191
3.7.3 ConditionalVariance 192
3.7.4 RecursiveMethods 193
3.8 CovarianceandCorrelation 196
3.8.1 TheCorrelationCoefficient 201
3.9 TheBivariateNormalDistribution 209
3.10 MultidimensionalRandomVectors 216
3.10.1 OrderStatistics 218
3.10.2 ReliabilityTheory 223
3.10.3 TheMultinomialDistribution 225
3.10.4 TheMultivariateNormalDistribution 226
3.10.5 Convolution 227
CONTENTS vii
3.11 GeneratingFunctions 231
3.11.1 TheProbabilityGeneratingFunction 231
3.11.2 TheMomentGeneratingFunction 237
3.12 ThePoissonProcess 240
3.12.1 ThinningandSuperposition 244
Problems 247
4 LimitTheorems 263
4.1 Introduction 263
4.2 TheLawofLargeNumbers 264
4.3 TheCentralLimitTheorem 268
4.3.1 TheDeltaMethod 273
4.4 ConvergenceinDistribution 275
4.4.1 DiscreteLimits 275
4.4.2 ContinuousLimits 277
Problems 278
5 Simulation 281
5.1 Introduction 281
5.2 RandomNumberGeneration 282
5.3 SimulationofDiscreteDistributions 283
5.4 SimulationofContinuousDistributions 285
5.5 Miscellaneous 290
Problems 292
6 StatisticalInference 294
6.1 Introduction 294
6.2 PointEstimators 294
6.2.1 EstimatingtheVariance 302
6.3 ConfidenceIntervals 304
6.3.1 ConfidenceIntervalfortheMeanintheNormalDistribution
withKnownVariance 307
6.3.2 ConfidenceIntervalforanUnknownProbability 308
6.3.3 One-SidedConfidenceIntervals 312
6.4 EstimationMethods 312
6.4.1 TheMethodofMoments 312
6.4.2 MaximumLikelihood 315
6.4.3 EvaluationofEstimatorswithSimulation 322
6.4.4 BootstrapSimulation 324
6.5 HypothesisTesting 327
6.5.1 LargeSampleTests 332
6.5.2 TestforanUnknownProbability 333
viii CONTENTS
6.6 FurtherTopicsinHypothesisTesting 334
6.6.1 P-Values 334
6.6.2 DataSnooping 335
6.6.3 ThePowerofaTest 336
6.6.4 MultipleHypothesisTesting 338
6.7 GoodnessofFit 339
6.7.1 Goodness-of-FitTestforIndependence 346
6.7.2 Fisher’sExactTest 349
6.8 BayesianStatistics 351
6.8.1 Noninformativepriors 359
6.8.2 CredibilityIntervals 362
6.9 NonparametricMethods 363
6.9.1 NonparametricHypothesisTesting 363
6.9.2 ComparingTwoSamples 370
6.9.3 NonparametricConfidenceIntervals 375
Problems 378
7 LinearModels 391
7.1 Introduction 391
7.2 SamplingDistributions 392
7.3 SingleSampleInference 395
7.3.1 InferencefortheVariance 396
7.3.2 InferencefortheMean 399
7.4 ComparingTwoSamples 402
7.4.1 InferenceaboutMeans 402
7.4.2 InferenceaboutVariances 407
7.5 AnalysisofVariance 409
7.5.1 One-WayAnalysisofVariance 409
7.5.2 MultipleComparisons:Tukey’sMethod 412
7.5.3 Kruskal–WallisTest 413
7.6 LinearRegression 415
7.6.1 Prediction 422
7.6.2 GoodnessofFit 424
7.6.3 TheSampleCorrelationCoefficient 425
7.6.4 Spearman’sCorrelationCoefficient 429
7.7 TheGeneralLinearModel 431
Problems 436
8 StochasticProcesses 444
8.1 Introduction 444
8.2 Discrete-TimeMarkovChains 445
8.2.1 TimeDynamicsofaMarkovChain 447
8.2.2 ClassificationofStates 450