Table Of Content

An Intermediate Course in Probability Allan Gut An Intermediate Course in Probability ~ Springer Alian Gut Department of Mathematics Uppsala University S-75106 Uppsala Sweden With 5 figures. Mathematics Subject Classification (1991): 60-01 Library of Congress Cataloging-in-Publication Data Out, Alian, 1944- An intermediate course in probability/A lian Out. p cm. Includes bibliographical references (pp. 253-256) and index. ISBN 978-0-387-94507-1 ISBN 978-1-4757-2431-8 (eBook) DOI 10.1007/978-1-4757-2431-8 1. Probabilities. 1. Title. QA273.08685 1995 519.2-dc20 95-11995 Printed on acid-free paper. © 1995 Springer Science+Business Media New York Original1y published by Springer Science+Business Media, Inc. in 1995 All rights reserved. This work may not be translated or copied in whole or in part without the written permis sion of the publisher, Springer Science+Business Media, LLC. except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodo1ogy now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. 98765 springeronline.com Preface The purpose of this book is to provide the reader with a solid background and understanding of the basic results and methods in probability the ory before entering into more advanced courses (in probability and/or statistics). The presentation is fairly thorough and detailed with many solved examples. Several examples are solved with different methods in order to illustrate their different levels of sophistication, their pros, and their cons. The motivation for this style of exposition is that experi ence has proved that the hard part in courses of this kind usually in the application of the results and methods; to know how, when, and where to apply what; and then, technically, to solve a given problem once one knows how to proceed. Exercises are spread out along the way, and every chapter ends with a large selection of problems. Chapters I through VI focus on some central areas of what might be called pure probability theory: multivariate random variables, condi tioning, transforms, order variables, the multivariate normal distribution, and convergence. A final chapter is devoted to the Poisson process be cause of its fundamental role in the theory of stochastic processes, but also because it provides an excellent application of the results and meth ods acquired earlier in the book. As an extra bonus, several facts about this process, which are frequently more or less taken for granted, are thereby properly verified. The book concludes with three appendixes: In the first we provide some suggestions for further reading and in the second we provide a list of abbreviations and useful facts concerning some standard distributions. The third appendix contains answers to the problems given at the end of each chapter. The level of the book is between the first undergraduate course in probability and the first graduate course. In particular, no knowledge of measure theory is assumed. The prerequisites (beyond a first course in probability) are basic analysis and some linear algebra. vi Preface Chapter V is, essentially, a revision of a handout by professor Carl Gustav Esseen. I am most grateful to him for allowing me to include the material in the book. The readability of a book is not only a function of its content and how (well) the material is presented. Very important are layout, fonts, and other aesthetical aspects. My heartfelt thanks to Anders Vretblad for his ideas, views, and suggestions, for his design and creation of the allan. sty file, and for his otherwise most generous help. I am also very grateful to Svante Janson for providing me with various index-making devices and to Lennart Norell for creating Fig ure 111.6.1. Ola Hossjer and Pontus Andersson have gone through the manuscript with great care at different stages in a search for misprints, slips, and other obscurities; I wish to thank them so much for everyone of their discoveries as well as for many other remarks (unfortunately, I am responsible for possible remaining inadvertencies). I also wish to thank my students from a second course in probability theory in Uppsala and Jan Ohlin and his students from a similar course at the Stockholm University for sending me a list of corrections on an earlier version of this book. Finally, I wish to thank Svante Janson and Dietrich von Rosen for several helpful suggestions and moral support, and Martin Gilchrist of Springer-Verlag for the care and understanding he has shown me and my manuscript. Up psala, Sweden Allan Gut Contents Preface............................................... v Notation and Symbols .... , .. .. . . .. . . . . . . . . . . . . . . . . .. . . . xi Introduction ......................................... 1 1. Models.. . .. . . .. . . . . . . . . . . . . . . . . . . . . .. . . .. . .. . . . . . . . 1 2. The Probability Space ............................... 3 3. Independence and Conditional Probabilities ............ 5 4. Random Variables ................................... 6 5. Expectation, Variance, and Moments .................. 8 6. Joint Distributions and Independence .................. 9 7. Sums of Random Variables, Covariance, and Correlation ...................................... 10 8. Limit Theorems. . . .. . .. . . . . . . .. ... . . .. . . .. . .. . . .. . .. 12 9. Stochastic Processes ................................. 13 10. The Contents of the Book ............................ 13 I. Multivariate Random Variables ...................... 17 1. Introduction ........................................ 17 2. Functions of Random Variables ....................... 21 3. Problems ........................................... 27 II. Conditioning ...................................... 32 1. Conditional Distributions ............................ 32 2. Conditional Expectation and Conditional Variance .... . . . 34 3. Distributions with Random Parameters ................ 40 4. The Bayesian Approach .............................. 45 5. Regression and Prediction ............................ 49 6. Martingales ........................................ 53 7. Problems. . . . . . . . . . . . . . . . .. . . .. . . . . . . . . . .. . .. . . .. . . . 55 III. Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 1. Introduction ........................................ 60 2. The Probability Generating Function .................. 62 viii Contents 3. The Moment Generating Function ..................... 66 4. The Characteristic Function .......................... 73 5. Sums of a Random Number of Random Variables ....... 80 6. Branching Processes ................................. 87 7. Distributions with Random Parameters ................ 94 8. Problems ........................................... 96 IV. Order Statistics ................................... 102 1. One-Dimensional Results ............................. 102 2. The Joint Distribution of the Extremes ................ 107 3. The Joint Distribution of the Order Statistic. . . . . . . . . . .. 111 4. Problems... . . . . .. .. . . . . . .. . . . . . .. . . .. . . .. . . . . . .. . .. 115 V. The Multivariate Normal Distribution ................ 119 1. Preliminaries from Linear Algebra ..................... 119 2. The Covariance Matrix .............................. 122 3. A First Definition ................................... 123 4. The Characteristic Function: Another Definition ........ 125 5. The Density: A Third Definition ...................... 127 6. Conditional Distributions ............................ 129 7. Independence ....................................... 133 8. Linear Transformations .............................. 135 9. Quadratic Forms and Cochran's Theorem .............. 139 10. Problems ........................................... 143 VI. Convergence 149 1. Definitions 149 2. Uniqueness ........................................ . 152 3. Relations Between the Convergence Concepts .......... . 155 4. The Borel-Cantelli Lemmas and Complete Convergence .. 163 5. Convergence via Transforms ......................... . 169 6. The Law of Large Numbers and the Central Limit Theorem .......................... . 172 7. Convergence of Sums of Sequences of Random Variables .. 177 8. Problems .......................................... . 186 VII. The Poisson Process. . . . . . . . .. . . . . . . . .. . . . . . . . . . .. 195 1. Introduction and Definitions .......................... 195 2. Restarted Poisson Processes .......................... 207 3. Conditioning on the Number of Occurrences in an Interval ............................ 215 4. Conditioning on Occurrence Times .................... 220 5. Several Independent Poisson Processes ................. 222 Contents ix 6. Thinning of Poisson Processes ........................ 231 7. The Compound Poisson Process ....................... 236 8. Some Further Generalizations and Remarks ............. 238 9. Problems.... . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . .. 246 Appendixes ........... " . . . . . . . .. . . . . . . . . . . . . . . . . . . .. 253 1. Suggestions for Further Reading ....................... 253 2. Some Distributions and Their Characteristics ........... 257 3. Answers to Problems ................................ 263 Index ............................................... 271 Notation and Symbols sample space w elementary event :F collection of events I{A} indicator function of (the set) A #{A} number of elements in (cardinality of) (the set) A complement of the set A probability of A X,Y,Z, ... random variables F(x), Fx(x) distribution function (of X) XEF X has distribution (function) F C(Fx) the continuity set of Fx p(x), px(x) probability function (of X) I(x), Ix(x) density (function) (of X) ~(x) standard normal distribution function 4J(x) standard normal density (function) Be(p) Bernoulli distribution (J(r, s) beta distribution Bin(n,p) binomial distribution C(m,a) Cauchy distribution x 2(n) chi-square distribution o(a) one-point distribution Exp(a) exponential distribution F(m,n) (Fisher's) F~distribution Fs(p) first success distribution r(p,a) gamma distribution Ge(p) geometric distribution H(N,n,p) hypergeometric distribution L(a) Laplace distribution LN(Il,0'2) log-normal distribution N(Il,0'2) normal distribution