Table Of ContentStochastic Processes: An Introduction
Solutions Manual
Third Edition
(cid:13)c Peter W Jones and Peter Smith
School of Computing and Mathematics, Keele University, UK
Preface
The website includes answers and solutions of all the end-of-chapter problems in the textbook
Stochastic Processes: An Introduction, third edition. We hope that they will prove helpful to
lecturersindesigningcourses,andtostudentsasasourceofmodelexamples. Theoriginalproblems
as numbered in the text are also included.
There are obviously references to results and examples from the textbook, and the manual
shouldbe viewedas a supplement to the book. To help identify the sections andchapters,the full
contents of Stochastic Processes follow this preface.
Every effort has been made to eliminate misprints or errors (or worse), and the authors, who
were responsible for the LaTeX code, apologise in advance for any which occur.
Peter W. Jones
Peter Smith Keele, 2017
1
Contents of Stochastic Processes
Chapter 1: Some Background in Probability
1.1 Introduction
1.2 Probability
1.3 Conditional probability and independence
1.4 Discrete random variables
1.5 Continuous random variables
1.6 Mean and variance
1.7 Some standard discrete probability distributions
1.8 Some standard continuous probability distributions
1.9 Generating functions
1.10 Conditional expectation
Problems
Chapter 2: Some Gambling Problems
2.1 Gambler’s ruin
2.2 Probability of ruin
2.3 Some numerical simulations
2.4 Expected duration of the game
2.5 Some variations of gambler’s ruin
2.5.1 The infinitely rich opponent
2.5.2 The generous gambler
2.5.3 Changing the stakes
Problems
Chapter 3: Random Walks
3.1 Introduction
3.2 Unrestricted random walks
3.3 Probability distribution after n steps
3.4 First returns of the symmetric random walk
Problems
Chapter 4: Markov Chains
4.1 States and transitions
4.2 Transition probabilities
4.3 General two-state Markov chain
4.4 Powers of the transition matrix for the m-state chain
4.5 Gambler’s ruin as a Markov chain
4.6 Classification of states
4.7 Classification of chains
4.8 A wildlife Markov chain model
Problems
Chapter 5: Poisson Processes
5.1 Introduction
5.2 The Poisson process
5.3 Partition theorem approach
5.4 Iterative method
5.5 The generating function
5.6 Variance for the Poissonprocess
2
5.7 Arrival times
5.8 Summary of the Poisson process
Problems
Chapter 6: Birth and Death Processes
6.1 Introduction
6.2 The birth process
6.3 Birth process: generating function equation
6.4 The death process
6.5 The combined birth and death process
6.6 General population processes
Problems
Chapter 7: Queues
7.1 Introduction
7.2 The single server queue
7.3 The stationary process
7.4 Queues with multiple servers
7.5 Queues with fixed service times
7.6 Classification of queues
Problems
Chapter 8: Reliability and Renewal
8.1 Introduction
8.2 The reliability function
8.3 The exponential distribution and reliability
8.4 Mean time to failure
8.5 Reliability of series and parallel systems
8.6 Renewal processes
8.7 Expected number of renewals
Problems
Chapter 9: Branching and Other Random Processes
9.1 Introduction
9.2 Generational growth
9.3 Mean and variance
9.4 Probability of extinction
9.5 Branching processes and martingales
9.6 Stopping rules
9.7 A continuous time epidemic
9.8 A discrete time epidemic model
9.9 Deterministic epidemic models
9.10 An iterative scheme for the simple epidemic
Problems
Chapter 10: Brownian Motion: Wiener Process
10.1 Introduction
10.2 Brownian motion
10.3 Wiener process as a limit of a random walk
10.4 Brownian motion with drift
10.5 Scaling
10.6 First visit times
3
10.7 Other Brownian motions in one dimension
10.8 Brownian motion in more than one dimension
Problems
Chapter 11: Computer Simulations and Projects
4
Contents of the Solutions Manual
Chapter 1: Some Background in Probability 6
Chapter 2: Some Gambling Problems 20
Chapter 3: Random Walks 34
Chapter 4: Markov Chains 49
Chapter 5: Poisson Processes 74
Chapter 6: Birth and Death Processes 80
Chapter 7: Queues 105
Chapter 8: Reliability and Renewal 121
Chapter 9: Branching and Other Random Processes 130
Chapter 10: Brownian Motion: Wiener Process 150
5
Chapter 1
Some background in probability
1.1. The Venn diagram of three events is shown in Figure 1.5(in the text). Indicate on the diagram the
following events:
(a) A B; (b) A (B C); (c) A (B C); (d) (A C)c; (e) (A B) Cc.
∪ ∪ ∪ ∩ ∪ ∩ ∩ ∪
S S
A B A B
C C
(a) (b)
S S
A B A B
C C
(c) (d)
S
A B
C
(e)
Figure 1.1:
The eventsare shaded in Figure 1.1.
1.2. In a random experiment, A, B, C are three events. In set notation write down expressions for the
events:
(a) only A occurs;
(b) all three events A, B, C occur;
(c) A and B occur but C does not;
(d) at least one of the events A, B, C occurs;
(e) exactly one of the events A, B, C occurs;
(f) not more than two of the events occur.
(a) A (B C)c; (b) A (B C)=A B C; (c) (A B) Cc; (d) A B C;
∩ ∪ ∩ ∩ ∩ ∩ ∩ ∩ ∪ ∪
6
(e) A (B C)c representsan event in A butnot in either B norC: therefore theanswer is
∩ ∪
(A (B C)c) (B (A C)c) (C (A B)c).
∩ ∪ ∪ ∩ ∪ ∪ ∩ ∪
(f) A B C
∩ ∩
1.3. For two events A and B, P(A)=0.4, P(B)=0.5 and P(A B)=0.3. Calculate
∩
(a) P(A B); (b) P(A Bc); (c) P(Ac Bc).
∪ ∩ ∪
(a) From (1.1) P(A B)=P(A)+P(B) P(A B), it follows that
∪ − ∩
P(A B)=0.4+0.5 0.3=0.6.
∪ −
(b) SinceA=(A Bc) (A B) and A Bc, and A B are mutually exclusive, then,
∩ ∪ ∩ ∩ ∩
P(A)=P[(A Bc) (A B)]=P(A Bc)+P(A B),
∩ ∪ ∩ ∩ ∩
so that
P(A Bc)=P(A) P(A B)=0.4 0.3=0.1.
∩ − ∩ −
(c) Since Ac Bc =(A B)c, then
∪ ∩
P(Ac Bc)=P[(A B)c]=1 P(A B)=1 0.3=0.7.
∪ ∩ − ∩ −
1.4. Two distinguishable fair dice a and b are rolled. What are the elements of the sample space? What is
the probability that the sum of the face values of the two dice is9? What isthe probability that at least one
5 or at least one 3 appears?
The 36 elements of the sample space are listed in Example 1.1. The event A , that the sum is 9, is
1
given by
A = (3,6),(4,5),(5,4),(6,3) .
1
{ }
HenceP= 4 = 1.
36 9
Let A be the event that at least one 5 or at least one 3 appears. Then by counting the elements in
2
thesample space in Example 1.1, P(A )= 20 = 5.
2 36 9
1.5. Two distinguishable fair dice a and b are rolled. What is the probability that the sum of the faces is
not more than 6?
LettherandomvariableX bethesumofthefaces. BycountingeventsinthesamplespaceinExample
1.1, P(X)= 15 = 5.
36 12
1.6. For the probability generating function
1
G(s)=(2 s)−2
−
find p and its mean.
n
{ }
Note that G(1)=1. Usingthe binomial theorem (see the Appendix)
1 1 1 1 ∞ 1 s n 1
G(s)= (1 s)−2 = −2 where −2 =1.
√2 − 2 √2 n 2 0
Xn=0(cid:18) (cid:19)(cid:16) (cid:17) (cid:18) (cid:19)
Hence
1 1 1 1
p0 = √2, pn= √22n −n2 , (n=1,2,...).
(cid:18) (cid:19)
The mean is
µ=G′(1)= 21(2−s)−32 = 12.
s=1
(cid:12)
(cid:12)
(cid:12)
7
1.7. Findtheprobabilitygenerating functionG(s)ofthePoissondistribution(see Section1.7)withparam-
eter α given by
e−ααn
p = , n=0,1,2,....
n n!
Determine the mean and variance of p from the generating function.
n
{ }
Given pn =e−ααn/n!, thegenerating function is given by
G(s)= ∞ pnsn = ∞ e−αnα!nsn =e−α ∞ (αns!)n =eα(s−1).
n=0 n=0 n=0
X X X
The mean and variance are given by
d
µ=G′(1)= eα(s−1) =α,
ds s=1
(cid:2) (cid:3)
σ2 =G′′(1)+G′(1) [G′(1)]2 =[α2eα(s−1)+αeα(s−1) α2e2α(s−1)]s=1 =α.
− −
1.8. A panel contains n warning lights. The times to failure of the lights are the independent random
variables T ,T ,...,T which have exponential distributions with parameters α ,α ,...,α respectively.
1 2 n 1 2 n
Let T be the random variable of the time to first failure, that is
T =min T ,T ,...,T .
1 2 n
{ }
Show that T has an exponential distribution with parameter n α .
j=1 j
P
The probability that no warning light has failed by time t is
P(T t) = P(T t T t T t)
1 2 n
≥ ≥ ∩ ≥ ∩···∩ ≥
= P(T t)P(T t) P(T t)
1 2 n
≥ ≥ ··· ≥
= e−α1te−α2t e−αnt =e−(α1+α2+···+αn)t.
···
1.9. The geometric distribution with parameter p is given by
p(x)=qx−1p, x=1,2,...
whereq=1 p(seeSection1.7). Finditsprobabilitygeneratingfunction. Calculatethemeanandvariance
−
of the geometric distribution from its pgf.
The generating function is given by
∞ p ∞ p qs ps
G(s)= qx−1psx = (qs)x = = ,
q q1 qs 1 qs
x=1 x=1 − −
X X
usingthe formula for thesum of a geometric series.
The mean is given by
d ps p pqs 1
µ=G′(1)= = + = .
ds 1 qs 1 qs (1 qs)2 p
(cid:20) − (cid:21)s=1 (cid:20) − − (cid:21)s=1
Forthe variance,
d p 2pq
G′′(s)= = .
ds (1 qs)2 (1 qs)3
(cid:20) − (cid:21) −
is required. Hence
2q 1 1 q
σ2 =G′′(1)+G′(1) [G′(1)]2 = + = .
− p2 p − p2 p2
8
1.10. Two distinguishable fair dice a and b are rolled. What are the probabilities that:
(a) at least one 4 appears;
(b) only one 4 appears;
(c) the sum of the face values is 6;
(d) the sum of the face values is 5 and one 3 is shown;
(e) the sum of the face values is 5 or only one 3 is shown?
From theTable in Example 1.1:
(a) If A is theevent that at least one 4 appears, then P(A )= 11.
1 1 36
(b) IfA is the eventthat only one 4 appears, then P(A )= 10 = 5.
2 2 36 18
(c) If A is theevent that thesum of the faces is 6, then P(A )= 5 .
3 3 36
(d) IfA is the eventthat theface valuesis 5 and one 3 is shown, then P(A )= 2 = 1 .
4 4 36 18
(e) If A is theevent that thesum of thefaces is 5 or only one3 is shown, then P(A )= 10 = 5 .
5 5 36 18
1.11. Two distinguishable fair dice a and b are rolled. What is the expected sum of the face values? What
is the variance of the sum of the face values?
Let N be the random variable representing the sum x+y, where x and y are face values of the two
dice. Then
6 6 6 6
1 1
E(N)= (x+y)= 6 x+6 y =7.
36 36
" #
x=1y=1 x=1 y=1
XX X X
and
6 6
1
V(N) = E(N2) E(N)2 = (x+y)2 72
− 36 −
Xx=1Xy=1
6 6 2
1
= 12 x2+2 x 49
36 −
" ! #
x=1 x=1
X X
329 35
= 49= =5.833....
6 − 6
1.12. Three distinguishable fair dice a, b and c are rolled. How many possible outcomes are there for the
faces shown? When the dice are rolled, what is the probability that just two dice show the same face values
and the third one is different?
The sample space contains 63 =216 elements of theform, (in theorder a,b,c),
S = (i,j,k) , (i=1,...,6;j =1,...,6;k=1,...,6).
{ }
Let A be therequired event. Supposethat a and b have thesame face values, which can occur in 6 ways,
andthatchasadifferentfacevaluewhichcanoccursin5ways. Hencethetotalnumberofwaysinwhich
a and b are the same but c is different is 6 5= 30 ways. The faces b and c, and c and a could also be
×
the same so that the total number of ways for the possible outcome is 3 30 = 90 ways. Therefore the
×
required probability is
90 5
P(A)= = .
216 12
1.13. In a sample space S, the events B and C are mutually exclusive, but A and B, are not. Show that
P(A (B C))=P(A)+P(B)+P(C) P(A (B C)).
∪ ∪ − ∩ ∪
From awell-shuffled packof52playingcards asinglecard israndomlydrawn. Findtheprobabilitythat
it is a club or an ace or the king of hearts.
From (1.1) (in thebook)
P(A (B C))=P(A)+P(B C) P(A (B C)) (i).
∪ ∪ ∪ − ∩ ∪
9
