Basics of Probability and Stochastic Processes

303 Pages·2019·5.594 MB·English

by  Esra Bas

Esra Bas
Basics
of Probability
and Stochastic
Processes
Basics of Probability and Stochastic Processes
Esra Bas
Basics of Probability
and Stochastic Processes
123
EsraBas
Industrial Engineering
Istanbul TechnicalUniversity
Istanbul,Turkey
ISBN978-3-030-32322-6 ISBN978-3-030-32323-3 (eBook)
https://doi.org/10.1007/978-3-030-32323-3
©SpringerNatureSwitzerlandAG2019
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
orinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, expressed or implied, with respect to the material contained
hereinorforanyerrorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregard
tojurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Preface
This book is aimed as a textbook for one-semester course in Introduction to
Probability and Stochastic Processes to be taught at engineering schools at the
undergraduate level. Since I teach this topic to students with different engineering
majors, I thought a book at a very basic level without theoretical details would be
beneficial. It is organized so that the students with no prior knowledge can learn
about the basic concepts of probability and stochastic processes in a step-by-step
manner and get insights by reading numerous remarks and warnings. All the
examplesandproblemsaresolvedstep-by-stepbyassumingthatthestudentshave
only basic calculus knowledge. The chapters include some basic examples, which
are revisited as a new concept is introduced. Since I believe that engineering
students can acquire knowledge by visual means more easily, I added several
figures and diagrams to facilitate the comprehension of the basic concepts and the
solutionsoftheexamplesandproblems.Ialsousedatableformatwhererelevantso
that the concepts and formulae can be understood in comparison with each other.
Thistableformatisalsointendedtoserveasasummaryofcrucialformulae.Itried
tokeepeachchaptersimplewithafewsub-chaptersandindependentfromtheother
chapters.
Thisbookhastwomainparts.Inthefirstmainpartofthisbook,thereaderscan
get familiar with the basics of probability including combinatorial analysis, con-
ditional probability, discrete and continuous random variables, and other selected
topics in probability including jointly distributed random variables, while in the
second main part of this book, they learn the basics of stochastic processes
including point process, counting process, renewal process, regenerative process,
Poissonprocess,Markovchains,queueingmodelsandreliabilitytheory.Thetopics
are presented from broad to detailed levels. As an example, Chap. 4 is devoted to
“IntroductiontoRandomVariables”,whichprovidesthebasicsthatarerelevantto
both discrete and continuous random variables. However, the readers can learn
more about “Discrete Random Variables” and “Continuous Random Variables” in
Chaps. 5 and 6, respectively. As another example, Chap. 9 is devoted to “A Brief
Introduction to Point Process, Counting Process, Renewal Process, Regenerative
Process, Poisson Process”. In this chapter, the readers can understand the basic
v
vi Preface
relations between these basic subjects. Afterwards, the interested readers can also
learn more about Poisson Process and Renewal Process in Chaps. 10 and 11,
respectively. I deliberately included only basic concepts in each chapter. As an
example,onlybirthanddeathqueueingmodelsareprovidedinChap. 15,sincethe
emphasis of this book is on the concepts at a very basic level.
Although the primary audience of this book is all engineering students at the
undergraduate level, the graduate students who want to refresh their basic knowl-
edge about probability and stochastic processes can also use this book as a review
before starting with the course stochastic processes at the graduate level. Since I
havealsobeenteachingthistopicatthegraduatelevel,theimportanceofrecalling
the basic concepts has been very clear to me based on my teaching experiences.
Istanbul, Turkey Esra Bas
Contents
Part I Basics of Probability
1 Combinatorial Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1 The Basic Principle of Counting and the Generalized Basic
Principle of Counting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Combinatorial Analysis (Combinatorics) . . . . . . . . . . . . . . . . . 5
2 Basic Concepts, Axioms and Operations in Probability . . . . . . . . . 15
2.1 Basic Concepts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2 Axioms of Probability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3 Basic Operations in Set Theory Versus Basic Operations
in Probability Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3 Conditional Probability, Bayes’ Formula, Independent Events. . . . 27
3.1 Conditional Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.2 Bayes’ Formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.3 Independent Events. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4 Introduction to Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.1 Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.2 Basic Parameters for the Discrete and Continuous Random
Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
5 Discrete Random Variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
5.1 Special Discrete Random Variables . . . . . . . . . . . . . . . . . . . . . 55
5.2 Basic Parameters for Special Discrete Random Variables . . . . . 59
6 Continuous Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
6.1 Special Continuous Random Variables. . . . . . . . . . . . . . . . . . . 71
6.2 Basic Parameters for Special Continuous Random Variables . . . 83
vii
viii Contents
7 Other Selected Topics in Basic Probability. . . . . . . . . . . . . . . . . . . 95
7.1 Jointly Distributed Random Variables . . . . . . . . . . . . . . . . . . . 96
7.2 Conditional Distribution, Conditional Expected Value,
Conditional Variance, Expected Value by Conditioning,
Variance by Conditioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
7.3 Moment Generating Function and Characteristic Function. . . . . 108
7.4 Limit Theorems in Probability. . . . . . . . . . . . . . . . . . . . . . . . . 110
Part II Basics of Stochastic Processes
8 A Brief Introduction to Stochastic Processes. . . . . . . . . . . . . . . . . . 125
9 A Brief Introduction to Point Process, Counting Process,
Renewal Process, Regenerative Process, Poisson Process . . . . . . . . 131
9.1 Point Process, Counting Process . . . . . . . . . . . . . . . . . . . . . . . 132
9.2 Renewal Process, Regenerative Process . . . . . . . . . . . . . . . . . . 134
9.3 Poisson Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
10 Poisson Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
10.1 Homogeneous Versus Nonhomogeneous Poisson Process . . . . . 149
10.2 Additional Properties of a Homogeneous Poisson Process. . . . . 153
11 Renewal Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
11.1 Basic Terms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
11.2 Limit Theorem, Elementary Renewal Theorem,
Renewal Reward Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
11.3 Regenerative Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
12 An Introduction to Markov Chains . . . . . . . . . . . . . . . . . . . . . . . . 179
12.1 Basic Concepts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
12.2 TransitionProbabilityMatrix,MatrixofTransitionProbability
Functions, Chapman-Kolmogorov Equations . . . . . . . . . . . . . . 181
12.3 Communication Classes, Irreducible Markov Chain,
Recurrent Versus Transient States, Period of a State,
Ergodic State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
12.4 Limiting Probability of a State of a Markov Chain . . . . . . . . . . 189
13 Special Discrete-Time Markov Chains . . . . . . . . . . . . . . . . . . . . . . 199
13.1 Random Walk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
13.2 Branching Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
13.3 Hidden Markov Chains. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
13.4 Time-Reversible Discrete-Time Markov Chains . . . . . . . . . . . . 205
13.5 Markov Decision Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
14 Continuous-Time Markov Chains. . . . . . . . . . . . . . . . . . . . . . . . . . 217
14.1 Continuous-Time Markov Chain and Birth & Death Process. . . 218
14.2 Birth & Death Queueing Models. . . . . . . . . . . . . . . . . . . . . . . 221
Contents ix
14.3 Kolmogorov’s Backward/Forward Equations, Infinitesimal
Generator Matrix of a CTMC and Time-Reversibility
of a CTMC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
15 An Introduction to Queueing Models . . . . . . . . . . . . . . . . . . . . . . . 233
15.1 Basic Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
15.2 Balance Equations and Little’s Law Equations for B&D
Queueing Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
16 Introduction to Brownian Motion. . . . . . . . . . . . . . . . . . . . . . . . . . 253
16.1 Basic Properties of a Brownian Motion . . . . . . . . . . . . . . . . . . 253
16.2 Other Properties of a Brownian Motion . . . . . . . . . . . . . . . . . . 256
17 Basics of Martingales . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
17.1 Martingale, Submartingale, Supermartingale,
Doob Type Martingale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
17.2 Azuma-Hoeffding Inequality, Kolmogorov’s Inequality,
the Martingale Convergence Theorem . . . . . . . . . . . . . . . . . . . 269
18 Basics of Reliability Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
18.1 Basic Definitions for a Nonrepairable Item. . . . . . . . . . . . . . . . 274
18.2 Basic Definitions for a Repairable Item . . . . . . . . . . . . . . . . . . 278
18.3 Systems with Independent Components . . . . . . . . . . . . . . . . . . 280
18.4 Systems with Dependent Components . . . . . . . . . . . . . . . . . . . 284
Area Under the Standard Normal Curve to the Left of z ... ..... .... 293
References.... .... .... .... ..... .... .... .... .... .... ..... .... 297
Index .... .... .... .... .... ..... .... .... .... .... .... ..... .... 299
Part I
Basics of Probability