1 Stochastic Processes, Theory for Applications Solutions to Exercises Updated and corrected as of October 16, 2014 R.G.Gallager These solutions are available to instructors from Cambridge Press at www.Cambridge.org. If any errors or typos are discovered, please notify Robert Gallager at gallager@mit.edu so that they can be corrected. The author gratefully acknowleges the help of Shan-Yuan Ho, who has edited many of these solutions, and of a number of teaching assistants, particulartly Natasha Blitvic and Mina Karzand, who wrote earlier drafts of some solutions. Thanks are also due to Kluwer Academic Press for permission to use a number of exercises that also appeared in Gallager, ‘Discrete Stochastic Processes,’ Kluwer, 1995. The original solutions to those exercises were prepared by Shan-Yuan Ho, but changed substantially here by the author both to be consistent with the new text and occasionally for added clarity. 2 APPENDIX A. SOLUTIONS TO EXERCISES A.1 Solutions for Chapter 1 Exercise 1.1: LetA andA bearbitraryeventsandshowthatPr A A +Pr A A =Pr A + 1 2 1 2 1 2 1 { } { } { } Pr A . Explain which parts of the sample space are being double-counted on both sides of this equation { 2} S and which partsare being counted once. Solution: As shown in the figure below, A A is part of A A and is thus being double 1 2 1 2 counted on the left side of the equation. It is also being double counted on the right (and S is in fact the meaning of A A as those sample points that are both in A and in A ). 1 2 1 2 A A 1 2 ⇥ B ⇥� ?SBN ��⇠⇠ ⇤⇤⌫�� 6AKACOC ⇤� AC ⇢A ⇢A A⇡A⇡ 1 1 2 2 Exercise 1.2: Thisexercisederivestheprobabilityofanarbitrary(non-disjoint)unionofevents,derives the unionbound,and derives some usefullimit expressions. a) For 2 arbitraryevents A and A , show that 1 2 A A =A (A A ), (A.1) 1 2 1 2 1 � [ [ where A A = A Ac. Show that A and A A are disjoint. Hint: This is what Venn diagrams were 2� 1 2 1 1 2� 1 invented for. Solution: Note that each sample point ! is in A or Ac, but not both. Thus each ! is in 1 1 exactly one of A , AcA or AcAc. In the first two cases, ! is in both sides of (A.1) and in 1 1 2 1 2 the last case it is in neither. Thus the two sides of (A.1) are identical. Also, as pointed out above, A and A A are disjoint. These results are intuitively obvious from the Venn 1 2 1 � diagram, A Ac A A A Ac =A A 1 2 1 2 2 1 2� 1 A � AU ?�↵ ��⇠⇠ 6⇥⇥� BMB 6 ⇥ B ⇢A⇢⇡A ⇡ 1 2 b) For any n � 2 and arbitrary events A1,... ,An, define Bn = An� ni=�11Ai. Show that B1,B2,... are disjoint events and show that for each n 2, n A = n B . Hint: Use induction. � i=1 i i=1 i S Solution: Let B = A . From (a) B Sand B aSre disjoint and (from (A.1)), A A = 1 1 1 2 1 2 B B . Let C = n A . We use induction to prove that C = n B and that the 1 2 n i=1 i n i=1 i S B are disjoint. We have seen that C =B B , which forms the basis for the induction. n 2 1 2 WeSassume that Cn�S1 = ni=�11Bi and prove Sthat Cn = ni=1Bi. S CS = C A = C SA Cc n n 1 n n 1 n n 1 � � � n = C [B = [B . n 1 n i � i 1 � [ [ In the second equality, we used (A.1), letting C play the role of A and A play the role n 1 1 n � A.1. SOLUTIONS FOR CHAPTER 1 3 of A . From this same application of (A.1), we also see that C and B =A C are 2 n 1 n n n 1 disjoint. Since Cn 1 = ni=�11Bi, this also shows that Bn is disjo�int from B1,...�,Bn�1. � � c) Show that S Pr 1 A = Pr 1 B = 1 Pr B . n n n n=1 n=1 n=1 { } Solution: If ! 2 1n=1nA[n, thenoit is innA[n for soome nX� 1. Thus ! 2 ni=1Bi, and thus ! 2 1n=1Bn. TheSsame argument works the other way, so 1n=1An =S 1n=1Bn. This establishes the first equality above, and the second is the third axiom of probability. S S S d) Show that for each n, Pr B Pr A . Use this to show that n n { } { } Pr 1 A 1 Pr A . n n n=1  n=1 { } n[ o X Solution: Since Bn = An � ni=�11Ai, we see that ! 2 Bn implies that ! 2 An, i.e., that B A . From (1.5), this implies that Pr B Pr A for each n. Thus n n n n ✓ S { }  { } Pr 1 A = 1 Pr B 1 Pr A . n n n n=1 n=1 { }  n=1 { } n[ o X X e) Show that Pr 1n=1An =limn!1Pr ni=1Ai . Hint: Combine (c) and (b). Note that this says that the probability of a limit is equal to the limit of the probabilities. This might well appear to be obvious �S �S withouta proof, but you will see situationslater wheresimilarappearinginterchangescannotbe made. Solution: From (c), k Pr 1 A = 1 Pr B = lim Pr B . n n n n=1 n=1 { } k n=1 { } n[ o X !1X From (b), however, k k k Pr B = Pr B = Pr A . n n n n=1 { } ( ) ( ) n=1 n=1 X [ [ Combining the first equation with the limit in k of the second yields the desired result. f) Show that Pr 1n=1An =limn!1Pr ni=1Ai . Hint: Remember De Morgan’sequalities. Solution: Usi�nTg De M organs equal�itTies, k 1 1 Pr A = 1 Pr Ac = 1 lim Pr Ac ( n) � ( n) �k ( n) n=1 n=1 !1 n=1 \ [ [ k = lim Pr A . n k ( ) !1 n=1 \ Exercise 1.3: Find the probability that a five card poker hand, chosen randomly from a 52 card deck, contains4aces. Thatis,ifall52!arrangementsofadeckofcardsareequallylikely,whatistheprobability that all 4 aces are in the first 5 cardsof the deck. Solution: The ace of spades can be in any of the first 5 positions, the ace of hearts in any of the 4 remaining positions out of the first 5, and so forth for the other two aces. The 4 APPENDIX A. SOLUTIONS TO EXERCISES remaining48cardscanbeinanyoftheremaining48positions. Thusthereare(5 4 3 2)48! · · · permutations of the 52 cards for which the first 5 cards contain 4 aces. Thus 5!48! 5 4 3 2 Pr 4 aces = = = · · · =1.847 10�5. { } 52! 52 51 50 49 ⇥ · · · Exercise 1.4: Consider a sample space of 8 equiprobable sample points and let A ,A ,A be three 1 2 3 events each of probability 1/2 such that Pr A A A =Pr A Pr A Pr A . 1 2 3 1 2 3 { } { } { } { } a) Create an example where Pr A A = Pr A A = 1 but Pr A A = 1. Hint: Make a table with a { 1 2} { 1 3} 4 { 2 3} 8 row for each sample point and a column for each of the above 3 events and try di↵erent ways of assigning samplepoints to events (the answer is not unique). Solution: Note that exactly one sample point must be in A ,A , and A in order to make 1 2 3 Pr A A A = 1/8. In order to make Pr A A = 1/4, there must be one additional 1 2 3 1 2 { } { } sample point that contains A and A but not A . Similarly, there must be one sample 1 2 3 point that contains A and A but not A . These points give rise to the first three rows 1 3 2 in the table below. There can be no additional sample point containing A and A since 2 3 Pr A A = 1/8. Thus each remaining sample point can be in at most 1 of the events 2 3 { } A ,A , and A . Since Pr A = 1/2 for 1 i 3 two sample points must contain A 1 2 3 i 2 { }   alone, two must contain A alone, and a single sample point must contain A alone. This 3 1 uniquely specifies the table below except for which sample point lies in each event. Sample points A A A 1 2 3 1 1 1 1 2 1 1 0 3 1 0 1 4 1 0 0 5 0 1 0 6 0 1 0 7 0 0 1 8 0 0 1 b)Showthat,foryourexample,A andA arenotindependent. Notethatthedefinitionofstatisticalinde- 2 3 pendencewouldbe verystrangeif it allowedA ,A ,A tobe independent whileA andA aredependent. 1 2 3 2 3 This illustrateswhy the definitionof independencerequires(1.14)ratherthanjust (1.15). Solution: Note that Pr A A = 1/8 = Pr A Pr A , so A and A are dependent. 2 3 2 3 2 3 { } 6 { } { } We also note that Pr AcAcAc = 0 = Pr Ac Pr Ac Pr Ac , further reinforcing the { 1 2 3} 6 { 1} { 2} { 3} conclusion that A ,A ,A are not statistically independent. Although the definition in 1 2 3 (1.14) of statistical independence of more than 2 events looks strange, it is clear from this example that (1.15) is insu�cient in the sense that it only specifies part of the above table. Exercise 1.5: This exerciseshows that for all rv’s X, F (x) is continuous from the right. X a) For any given rv X, any real number x, and each integer n 1, let A = ! : X > x+1/n , n � { } and show that A A . Use this and the corollaries to the axioms of probability to show that 1 2 ✓ ✓ ··· Pr A =lim Pr A . n�1 n n!1 { n} n o S A.1. SOLUTIONS FOR CHAPTER 1 5 Solution: If X(!) > x+ 1, then (since 1 > 1 ), we also have X(!) > x+ 1 . Thus n n n+1 n+1 A A for all n 1. Thus from (1.9), Pr A =lim Pr A . n ✓ n+1 � n�1 n n!1 { n} n o S b) Show that Pr A =Pr X >x and that Pr X >x =lim Pr X >x+1/n . n�1 n { } { } n!1 { } n o Solution: If XS(!) > x, then there must be an n su�ciently large that X(!) > x+1/n. Thus ! : X > x A . The subset inequality goes the other way also since { } ✓ n 1 n} X(!) > x + 1/n for any n� 1 implies that X(!) > x. Since these represent the same S � events, they have the same probability and Pr A = Pr X >x . Then from (a) n 1 n { } � we also have n o S Pr X >x = lim Pr A = lim Pr X >x+1/n . n { } n { } n { } !1 !1 c) Show that for ✏>0, lim Pr X x+✏ =Pr X x . ✏ 0 ! {  } {  } Solution: Taking the complement of both sides of the above equation, Pr X x = {  } lim Pr X x+1/n . Since Pr X x+✏ is non-decreasing in ✏, it also follows that n !1 {  } {  } for ✏ >0, Pr X x =lim Pr X x+✏ . ✏ 0 {  } ! {  } Exercise 1.6: Show that for a continuous nonnegative rv X, 1 1 Pr X >x dx= xf (x)dx. (A.2) X { } Z0 Z0 Hint1: FirstrewritePr{X >x}ontheleftsideof(A.2)as x1fX(y)dy. Thenthinkthrough,toyourlevel of comfort,how and why the orderof integrationcan be interchngedin the resultingexpression. R SWoelulotoiokna:t WE[eXh]a=ve 0P1rP{Xr{X>x>}x=} dRxx1asfXli(my)a!dy1fr0oamFc(txh)eddxefisinnicteiotnheoflima ictoinngtinoupoeurastirovn. a is where the interesting issue is. !1 R R a a Fc(x)dx = 1f (y)dydx X Z0 Z0 Zx a a a 1 = f (y)dydx+ f (y)dydx X X Z0 Zx Z0 Za a y = f (y)dxdy+aFc (a). X X Z0 Z0 Wefirstbroketheintegralontherightintotwoparts,onefory 0, then for any ✏ (0,a), aF (a) >� ✏ for all su�ciently large a. For X 2 � 6 APPENDIX A. SOLUTIONS TO EXERCISES all such a, then Fc (a) > (� ✏)/a. This would imply that X = 1Fc (x)dx = , which X � 0 X 1 is a contradiction. Thus � = 0, i.e., lim aFc (a) = 0, establishing (A.2) for the case a!1 X R where E[X] is finite. The case where E[X] is infinite is a minor perturbation. The result that lim aFc(a) = 0 is also important and can be seen intuitively from a X !1 Figure 1.3. Hint 2: As an alternateapproach, derive (A.2)usingintegrationby parts. Solution: Using integration by parts and being less careful, 1d xFc (x) = 1xf (x)dx+ 1Fc (x)dx. X � X X Z0 Z0 Z0 � � The left side is lim aFc (a) 0F (0) so this shows the same thing, again requiring the fact that lim aaF!c1(a) =X0 w�hen EX[X] exists. a X !1 Exercise 1.7: Suppose X and Y are discrete rv’s with the PMF p (x ,y ). Show (a picture will XY i j help)that this is relatedto the joint CDFby p (x ,y )= lim [F(x ,y ) F(x �,y ) F(x ,y �)+F(x �,y �)]. XY i j i j i j i j i j �>0,� 0 � � � � � � ! Solution: The picture below makes this equation obvious. Note that F(x ,y ) is the i j probability of the quadrant of joint sample values (x,y) that satisfy (x,y) (x ,y ). The i j  term F (x �,y ) is the probability of the hatched region on the left and F (x ,y �) XY i j XY i j � � � F (x �,y �) is the probability of the hatched region on the right. Thus the expres- XY i j � � sion on the right of the equation is the probability of the � by � square to the left and below (x ,y ). When this square becomes too small to include any other sample point, the i j probability of the square is p (x ,y ). XY i j (x ,y ) i j �� @s F (x ,y �) ��@ XY i j � F (x �,y �) FXY(xi �,yj) ��@ � XY i� j� � ��@ �� Exercise 1.8: AvariationofExample1.5.1istoletM bearandomvariablethattakesonbothpositive and negative values with the PMF 1 p (m)= . M 2m (m +1) | | | | Inotherwords,M is symmetricaround0 and M hasthesamePMFas thenonnegative rv N of Example | | 1.5.1. a)Showthat mp (m)= and mp (m)= . (ThusshowthattheexpectationofM not m�0 M 1 m<0 M �1 only does not exist but is undefinedeven in the extendedreal number system.) P P A.1. SOLUTIONS FOR CHAPTER 1 7 Solution: 1 mp (m) = = M m 0 m 0 2(m +1) 1 � � | | X X 1 mp (m) = � = . M m<0 m<0 2(m +1) �1 | | X X b)Supposethatthetermsin 1m= mpM(m)aresummedintheorderof2positivetermsforeachnegative term (i.e., in the order 1,2, 1,3,�41, 2,5, ). Find the limiting value of the partial sums in this series. �P � ··· Hint: You may find it helpfulto know that n 1 n 1 lim dx =�, n!1 i=1 i �Z1 x � X where� is the Euler-Mascheroniconstant, � =0.57721 . ··· Solution: The sum after 3n terms is 2n n 2n+1 1 1 1 1 = . 2(m+1) � 2(m+1) 2 i m=1 m=1 i=n+2 X X X Taking the limit as n , the Euler-Mascheroni constant cancels out and !1 1 2n+1 1 1 2n+1 1 1 lim = lim dx = = ln2. n 2 i n 2 x 2 !1 i=n+2 !1 Zn+2 X c)Repeat(b)where,foranygivenintegerk>0,theorderofsummationiskpositivetermsforeachnegative term. Solution: Thisisdonethesameway,andtheansweris 1lnk. Whattheexerciseessentially 2 shows is that in a sum for which both the positive terms sum to infinity and the negative termssumto , onecangetanydesiredlimitbysummingtermsinanappropriateorder. �1 Infact, toreachanydesiredlimit, onealternatesbetweenpositivetermsuntilexceedingthe desired limit, then negative terms until falling below the desired limit, then positive terms again, etc. Exercise 1.9: (Proof of Theorem 1.4.1)The boundson the binomialin this theoremare basedon the Stirling bounds. Thesesay that for all n 1, n! is upper and lower boundedby � n n n n p2⇡n 1 . k! r2⇡k(n�k) kk(n�k)n�k  � 12k(n�k)� Solution: Use the factorial lower bound on n! and the upper bound on k and (n k)!. � This yields n n nn 1 1 > exp k 2⇡k(n k) kk(n k)n k �12k � 12(n k) ✓ ◆ r � � � ✓ � ◆ n nn n > 1 , 2⇡k(n k) kk(n k)n k � 12k(n k) r � � �  � � where the latter equation comes from combining the two terms in the exponent and then using the bound e x >1 x. � � d) Derive the lower boundin Theorem1.4.1. Solution: This follows by substituting pñ for k in the solution to c) and substituting this in the binomial formula. e) Show that �(p,p˜)=p˜ln(pp˜)+(1�p˜)ln(11�pp˜) is 0 at p˜=p and nonnegative elsewhere. � Solution: Itisobviousthat�(p,pˆ) =0 forp˜=p. Takingthefirsttwo derivativesof�(p,p˜) with respect to p˜, @�(p,p˜) p(1 p˜) @f2(p,p˜) 1 = ln � = . @p˜ � p˜(1 p) @p˜2 p˜(1 p˜) ✓ � ◆ � A.1. SOLUTIONS FOR CHAPTER 1 9 Since the second derivative is positive for 0 0 for p˜= p. 6 Furthermore, �(p,p˜) increases as p˜moves in either direction away from p. Exercise 1.10: Let X be a ternary rv taking on the 3 values 0, 1, 2 with probabilities p ,p ,p 0 1 2 respectively. Find the medianof X for each of the cases below. a) p =0.2, p =0.4, p =0.4. 0 1 2 b) p =0.2, p =0.2, p =0.6. 0 1 2 c) p =0.2, p =0.3, p =0.5. 0 1 2 Note1: Themedianisnotuniquein(c). Findtheintervalofvaluesthataremedians. Note2: Somepeople force the median to be distinct by defining it as the midpoint of the interval satisfying the definition given here. Solution: The median of X is 1 for (a), 2 for (b), and the interval [1,2) for (c). d) Now suppose that X is nonnegative and continuous with the density f (x) = 1 for 0 x 0.5 and X   f (x)=0 for 0.51, but it is otherwiseunknown. Find X X  the medianor interval of medians. Themedianis sometimes(incorrectly)definedas that↵ forwhich Pr X >↵ =Pr X <↵ . Show thatit { } { } is possiblefor no such ↵ to exist. Hint: Look at the examplesabove. Solution: The interval of medians is [0.5,1]. In particular, Pr X x = 1/2 for all x in {  } this interval and Pr X x =1/2 in this interval. { � } For each of the first 3 examples, there is no ↵ for which Pr X <↵ = Pr X >↵ . { } { } One should then ask why there must always be an x such that Pr X x 1/2 and { � } � Pr X x 1/2. To see this, let x =inf x : F (x) 1/2 . We must have F (x ) 1/2 o X X o {  } � { � } � sinceF iscontinuousontheright. Becauseoftheinfimum, wemusthaveF (x ✏) <1/2 X X o � for all ✏ > 0, and therefore Pr X x ✏ 1/2. But Pr X x is continuous on the o { � � } � { � } left for the same reason that F (x) is continuous on the right, and thus x is a median of X o X. This is the kind of argument that makes many people hate analysis. Exercise 1.11: a)ForanygivenrvY,expressE[Y ]intermsof F (y)dyand Fc (y)dy. Hint: | | y<0 Y y 0 Y � Reviewthe argument in Figure1.4. R R Solution: We have seen in (1.34) that E[Y] = F (y)dy+ Fc (y)dy. Y � Y Zy<0 Zy�0 Since all negative values of Y become positive in Y , | | E[ Y ] =+ F (y)dy+ Fc (y)dy. Y | | Y Zy<0 Zy�0 To spell this out in greater detail, let Y = Y+ +Y where Y+ = max 0,Y and Y = � � { } min Y,0 . Then Y = Y+ + Y and Y = Y+ Y = Y+ + Y . Since E[Y+] = � � � { } | | � | | Fc (y)dy and E[Y ] = F (y)dy, the above results follow. y 0 Y � � y<0 Y � R R 10 APPENDIX A. SOLUTIONS TO EXERCISES b) For some given rv X with E[X ]< , let Y =X ↵. Using(a), show that | | 1 � ↵ E[X ↵]= F (x)dx+ 1Fc (x)dx. | � | X X Z�1 Z↵ Solution: This follows by changing the variable of integration in (a). That is, E[ X ↵ ] = E[ Y ] =+ F (y)dy+ Fc (y)dy Y | � | | | Y Zy<0 Zy�0 ↵ = F (x)dx+ 1Fc (x)dx, X X Z�1 Z↵ where in the last step, we have changed the variable of integration from y to x ↵. � c) Show thatE[X ↵] is minimizedover ↵ by choosing↵ to be a medianof X. Hint: Boththe easy way | � | and the most instructive way to do this is to use a graphical argument illustrating the above two integrals Be carefulto show that whenthe medianis an interval, all points in this interval achieve the minimum. Solution: As illustrated in the picture, we are minimizing an integral for which the integrand changes from F (x) to Fc (x) at x = ↵. If F (x) is strictly increasing in x, then X X X Fc =1 F is strictly decreasing. We then minimize the integrand over all x by choosing X � X ↵ to be the point where the curves cross, i.e., where F (x) = .5. Since the integrand has X been minimized at each point, the integral must also be minimized. 1 F (x) X ↵ 0.5 Fc (x) X 0 If F is continuous but not strictly increasing, then there might be an interval over which X F (x) =.5; all points on this interval are medians and also minimize the integral; Exercise X 1.10 (c) gives an example where F (x) =0.5 over the interval [1,2). Finally, if F (↵) 0.5 X X � and F (↵ ✏) < 0.5 for some ↵ and all ✏ > 0 (as in parts (a) and (b) of Exercise 1.10), X � then the integral is minimized at that ↵ and that ↵ is also the median. Exercise 1.12: Let X be a rv with CDFF (x). Find the CDFof the following rv’s. X a) The maximum of n IID rv’s, each with CDFF (x). X Solution: Let M be the maximum of the n rv’s X ,... ,X . Note that for any real x, + 1 n M is less than or equal to x if and only if X x for each j, 1 j n. Thus + j    n Pr M x =Pr X x,X x,... ,X x = Pr X x , + 1 2 n j {  } {    } {  } j=1 Y where we have used the independence of the X ’s. Finally, since Pr X x = F (x) for j j X n {  } each j, we have F (x) =Pr M x = F (x) . M+ { +  } X b) The minimum of n IID rv’s, each with CDFF �(x). � X

Stochastic Processes: Theory for Applications (Instructor's Solution Manulal) (Solutions)

Similar Stochastic Processes: Theory for Applications (Instructor's Solution Manulal) (Solutions)