Probability, Markov Chains, Queues, and Simulation: The Mathematical Basis of Performance Modeling

777 Pages·2009·6.27 MB·English

by  William J. Stewart

PROBABILITY, MARKOV CHAINS,
QUEUES, AND SIMULATION
This page intentionally left blank

Copyright(cid:2)c2009byPrincetonUniversityPress
PublishedbyPrincetonUniversityPress,41Williamstreet,Princeton,NewJersey08540
IntheUnitedKingdom:PrincetonUniversityPress,6OxfordStreet,
Woodstock,OxfordshireOX201TW
AllRightsReserved
LibraryofCongressCataloging-in-PublicationData
Stewart,WilliamJ.,1946–
Probability,Markovchains,queuesandsimulation:themathematicalbasisof
performancemodeling/WilliamJ.Stewart.–1sted.
p.cm.
ISBN978-0-691-14062-9(cloth:alk.paper)1.Probability–Computersimulation.
2.Markovprocesses.3.Queueingtheory.I.Title.
QA273.S75322009
519.201(cid:2)13–dc22 2008041122
BritishLibraryCataloging-in-PublicationDataisavailable
ThisbookhasbeencomposedinTimes
Printedonacid-freepaper.∞
press.princeton.edu
MATLABisaregisteredtrademarkofTheMathWorks,Inc.
TypesetbySRNovaPvtLtd,Bangalore,India
PrintedintheUnitedStatesofAmerica
10 9 8 7 6 5 4 3 2 1
Thisbookisdedicatedtoallthose
whomIlove,especially
Mydearwife, Kathie,
andmywonderfulchildren
Nicola,Stephanie,Kathryn,andWilliam
Myfather,William J.Stewartand
the memoryofmymother,Mary(Marshall)Stewart
This page intentionally left blank
Contents
PrefaceandAcknowledgments xv
I PROBABILITY 1
1 Probability 3
1.1 Trials,SampleSpaces,andEvents 3
1.2 ProbabilityAxiomsandProbabilitySpace 9
1.3 ConditionalProbability 12
1.4 IndependentEvents 15
1.5 LawofTotalProbability 18
1.6 Bayes’Rule 20
1.7 Exercises 21
2 Combinatorics—TheArtofCounting 25
2.1 Permutations 25
2.2 PermutationswithReplacements 26
2.3 PermutationswithoutReplacement 27
2.4 CombinationswithoutReplacement 29
2.5 CombinationswithReplacements 31
2.6 Bernoulli(Independent)Trials 33
2.7 Exercises 36
3 RandomVariablesandDistributionFunctions 40
3.1 DiscreteandContinuousRandomVariables 40
3.2 TheProbabilityMassFunctionforaDiscreteRandomVariable 43
3.3 TheCumulativeDistributionFunction 46
3.4 TheProbabilityDensityFunctionforaContinuousRandomVariable 51
3.5 FunctionsofaRandomVariable 53
3.6 ConditionedRandomVariables 58
3.7 Exercises 60
4 JointandConditionalDistributions 64
4.1 JointDistributions 64
4.2 JointCumulativeDistributionFunctions 64
4.3 JointProbabilityMassFunctions 68
4.4 JointProbabilityDensityFunctions 71
4.5 ConditionalDistributions 77
4.6 ConvolutionsandtheSumofTwoRandomVariables 80
4.7 Exercises 82
5 ExpectationsandMore 87
5.1 Definitions 87
5.2 ExpectationofFunctionsandJointRandomVariables 92
5.3 ProbabilityGeneratingFunctionsforDiscreteRandomVariables 100
viii Contents
5.4 MomentGeneratingFunctions 103
5.5 MaximaandMinimaofIndependentRandomVariables 108
5.6 Exercises 110
6 DiscreteDistributionFunctions 115
6.1 TheDiscreteUniformDistribution 115
6.2 TheBernoulliDistribution 116
6.3 TheBinomialDistribution 117
6.4 GeometricandNegativeBinomialDistributions 120
6.5 ThePoissonDistribution 124
6.6 TheHypergeometricDistribution 127
6.7 TheMultinomialDistribution 128
6.8 Exercises 130
7 ContinuousDistributionFunctions 134
7.1 TheUniformDistribution 134
7.2 TheExponentialDistribution 136
7.3 TheNormalorGaussianDistribution 141
7.4 TheGammaDistribution 145
7.5 ReliabilityModelingandtheWeibullDistribution 149
7.6 Phase-TypeDistributions 155
7.6.1 TheErlang-2Distribution 155
7.6.2 TheErlang-r Distribution 158
7.6.3 TheHypoexponentialDistribution 162
7.6.4 TheHyperexponentialDistribution 164
7.6.5 TheCoxianDistribution 166
7.6.6 GeneralPhase-TypeDistributions 168
7.6.7 FittingPhase-TypeDistributionstoMeansandVariances 171
7.7 Exercises 176
8 BoundsandLimitTheorems 180
8.1 TheMarkovInequality 180
8.2 TheChebychevInequality 181
8.3 TheChernoffBound 182
8.4 TheLawsofLargeNumbers 182
8.5 TheCentralLimitTheorem 184
8.6 Exercises 187
II MARKOVCHAINS 191
9 Discrete-andContinuous-TimeMarkovChains 193
9.1 StochasticProcessesandMarkovChains 193
9.2 Discrete-TimeMarkovChains:Definitions 195
9.3 TheChapman-KolmogorovEquations 202
9.4 ClassificationofStates 206
9.5 Irreducibility 214
9.6 ThePotential,Fundamental,andReachabilityMatrices 218
9.6.1 PotentialandFundamentalMatricesandMeanTimetoAbsorption 219
9.6.2 TheReachabilityMatrixandAbsorptionProbabilities 223
Contents ix
9.7 RandomWalkProblems 228
9.8 ProbabilityDistributions 235
9.9 Reversibility 248
9.10 Continuous-TimeMarkovChains 253
9.10.1 TransitionProbabilitiesandTransitionRates 254
9.10.2 TheChapman-KolmogorovEquations 257
9.10.3 TheEmbeddedMarkovChainandStateProperties 259
9.10.4 ProbabilityDistributions 262
9.10.5 Reversibility 265
9.11 Semi-MarkovProcesses 265
9.12 RenewalProcesses 267
9.13 Exercises 275
10 NumericalSolutionofMarkovChains 285
10.1 Introduction 285
10.1.1 SettingtheStage 285
10.1.2 StochasticMatrices 287
10.1.3 TheEffectofDiscretization 289
10.2 DirectMethodsforStationaryDistributions 290
10.2.1 IterativeversusDirectSolutionMethods 290
10.2.2 GaussianEliminationandLUFactorizations 291
10.3 BasicIterativeMethodsforStationaryDistributions 301
10.3.1 ThePowerMethod 301
10.3.2 TheIterativeMethodsofJacobiandGauss–Seidel 305
10.3.3 TheMethodofSuccessiveOverrelaxation 311
10.3.4 DataStructuresforLargeSparseMatrices 313
10.3.5 InitialApproximations,Normalization,andConvergence 316
10.4 BlockIterativeMethods 319
10.5 DecompositionandAggregationMethods 324
10.6 TheMatrixGeometric/AnalyticMethodsforStructuredMarkovChains 332
10.6.1 TheQuasi-Birth-DeathCase 333
10.6.2 BlockLowerHessenbergMarkovChains 340
10.6.3 BlockUpperHessenbergMarkovChains 345
10.7 TransientDistributions 354
10.7.1 MatrixScalingandPoweringMethodsforSmallStateSpaces 357
10.7.2 TheUniformizationMethodforLargeStateSpaces 361
10.7.3 OrdinaryDifferentialEquationSolvers 365
10.8 Exercises 375
III QUEUEINGMODELS 383
11 ElementaryQueueingTheory 385
11.1 IntroductionandBasicDefinitions 385
11.1.1 ArrivalsandService 386
11.1.2 SchedulingDisciplines 395
11.1.3 Kendall’sNotation 396
11.1.4 GraphicalRepresentationsofQueues 397
11.1.5 PerformanceMeasures—MeasuresofEffectiveness 398
11.1.6 Little’sLaw 400