A Probability Path

460 Pages·2005·22.458 MB·English

by  Sidney Resnick

A Probability Path
Sidney I. Resnick
A Probability Path
Springer Science+Business Media, LLC
Sidney I. Resnick
School of Operations Research and
Industrial Engineering
Ithaca, NY 14853
USA
AMS Subject Classifications: 60-XX, 60-01, 60A10, 60Exx, 60El0, 60E15, 60Fxx, 60040, 60048
Library of Congress Cataloging-in-Publication Data
Resnick, Sidney I.
A probability path I Sidney Resnick.
p. em.
Includes bibliographical references and index.
I. Probabilities. I. Title.
QA273.R437 1998
519'.24-dc21 98-21749
CIP
printed on acid-free paper.
ISBN 978-1-4612-7297-7 ISBN 978-14612-1818-0 (eBook)
DOI 10.1007/978-14612-1818-0
©2005 Springer Science+Business Media New York
Originally published by Birkhiiuser Boston in 2005
Softcover reprint of the hardcover 1st edition 2005
All rights reserved. This work may not be translated or copied in whole or in part without the
written permission of the publisher (Springer Science+Business Media, LLC), except for brief
excerpts in connection with reviews or scholarly analysis. Use in connection with any form of
information storage and retrieval, electronic adaptation, computer software, or by similar or
dissimilar methodology now known or hereafter developed is forbidden.
The use in this publication of trade names, trademarks, service marks and similar terms, even if they
are not identified as such, is not to be taken as an expression of opinion as to whether or not they are
subject to proprietary rights.
Cover design by Minna Resnick.
Typeset by the author in U\ TEX.
(HP)
9 8 7 6 5 SPIN 11361374
www.birkhauser.com
Contents
Preface xi
1 Sets and Events 1
1.1 Introduction . . . . . . . . 1
1.2 Basic Set Theory . . . . . 2
1.2.1 Indicator functions 5
1.3 Limits of Sets . . . . . . . 6
1.4 Monotone Sequences . . . 8
1.5 Set Operations and Closure . 11
1.5.1 Examples .......... . 13
1.6 The a-field Generated by a Given Class C 15
1. 7 Borel Sets on the Real Line . 16
1.8 Comparing Borel Sets . 18
1.9 Exercises ......... . 20
2 Probability Spaces 29
2.1 Basic Definitions and Properties 29
2.2 More on Closure . . . . . . . . 35
2.2.1 Dynkin's theorem .... 36
2.2.2 Proof of Dynkin's theorem . 38
2.3 Two Constructions . . . . . . . . . 40
2.4 Constructions of Probability Spaces 42
2.4.1 General Construction of a Probability Model 43
2.4.2 Proof of the Second Extension Theorem . . . 49
0 ••• 0 • 0
vi Contents
2.5 Measure Constructions . . . . . . . . . . . . . . . . . . . . . . . 57
2.5.1 Lebesgue Measure on (0, 1] . . . . . . . . . . . . . . . . 57
2.5.2 Construction of a Probability Measure on lR with Given
Distribution Function F (x) . 61
2.6 Exercises ..... ... ... .. . .. ... . . . . . . . . . . . 63
3 Random Variables, Elements, and Measurable Maps 71
3.1 Inverse Maps . . . . . . . . . . . . . 71
3.2 Measurable Maps, Random Elements,
Induced Probability Measures . . . . 74
3.2.1 Composition ..... .. . . 77
3.2.2 Random Elements of Metric Spaces 78
3.2.3 Measurability and Continuity . 80
3.2.4 Measurability and Limits . 81
3.3 a-Fields Generated by Maps 83
3.4 Exercises . . . . . . . . . . . . . 85
4 Independence 91
4.1 Basic Definitions .... . . . . 91
4.2 Independent Random Variables . 93
4.3 Two Examples of Independence 95
4.3.1 Records, Ranks, Renyi Theorem . . 95
4.3.2 Dyadic Expansions of Uniform Random Numbers 98
4.4 More on Independence: Groupings . . . . . . . . . . . . . . 100
4.5 Independence, Zero-One Laws, Borel-Cantelli Lemma ... 102
4.5.1 Borel-Cantelli Lemma .. . ... 102
4.5.2 Borel Zero-One Law . . . . . 103
4.5.3 Kolmogorov Zero-One Law . 107
4.6 Exercises ... ...... . .. . . . 110
5 Integration and Expectation 117
5.1 Preparation for Integration . ...... . . .. . . 117
5.1.1 Simple Functions ....... . .... . . 117
5.1.2 Measurability and Simple Functions . . . . 118
5.2 Expectation and Integration . . . . . . . . . 119
5.2.1 Expectation of Simple Functions . . 119
5.2.2 Extension of the Definition . . . . 122
5.2.3 Basic Properties of Expectation . 123
5.3 Limits and Integrals ......... . . . . . 131
5.4 Indefinite Integrals . . . . . . . . . . . . . . 134
5.5 The Transformation Theorem and Densities ... 135
5.5.1 Expectation is Always an Integral on lR . 137
5.5.2 Densities ........ . . . 139
5.6 The Riemann vs Lebesgue Integral . . 139
5.7 Product Spaces .......... . . 143
Contents vii
5.8 Probability Measures on Product Spaces . 147
5.9 Fubini's theorem . 149
5.10 Exercises ..... . 155
6 Convergence Concepts 167
6.1 Almost Sure Convergence . 167
6.2 Convergence in Probability . 169
6.2.1 Statistical Terminology . . 170
6.3 Connections Between a.s. and i.p. Convergence . 171
6.4 Quantile Estimation . . . . . . . 178
6.5 L P Convergence . . . . . . . . . . . . . . . . 180
6.5.1 Uniform Integrability ....... . . 182
6.5.2 Interlude: A Review of Inequalities . 186
6.6 More on L P Convergence . . 189
6. 7 Exercises . . . . . . . . . . . . . . . . . . . 195
7 Laws of Large Numbers and Sums
of Independent Random Variables 203
7.1 Truncation and Equivalence . . . . . . . . . 203
7.2 A General Weak Law of Large Numbers . . 204
7.3 Almost Sure Convergence of Sums
of Independent Random Variables . 209
7.4 Strong Laws of Large Numbers . . . . 213
7.4.1 Two Examples . . . . . . . . 215
7.5 The Strong Law of Large Numbers for liD Sequences . . 219
7.5.1 Two Applications of the SLLN . . . . . . . . . . 222
7.6 The Kolmogorov Three Series Theorem . . . . . . . . . 226
7.6.1 Necessity of the Kolmogorov Three Series Theorem . 230
7. 7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
8 Convergence in Distribution 247
8.1 Basic Definitions . . . . . . . . . . . . . . . . 247
8.2 Scheffe's lemma . . . . . . . . . . . . . . . . . 252
8.2.1 Scheffe's lemma and Order Statistics . 255
8.3 The Baby Skorohod Theorem . . . . . . . . . . 258
8.3.1 The Delta Method ......... . . 261
8.4 Weak Convergence Equivalences; Portmanteau Theorem . 263
8.5 More Relations Among Modes of Convergence . . . . . . 267
8.6 New Convergences from Old . . . . . . . . . . . . . . . . 268
8.6.1 Example: The Central Limit Theorem form-Dependent
Random Variables . . . . . . . . . . . . . . . . . . . . . 270
8.7 The Convergence to Types Theorem ................ 274
8. 7.1 Application of Convergence to Types: Limit Distributions
for Extremes . 278
8.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
viii Contents
9 Characteristic Functions and the Central Limit Theorem 293
9.1 Review of Moment Generating Functions
and the Central Limit Theorem . . . . . . . . . . . . . . 294
9.2 Characteristic Functions: Definition and First Properties . . 295
9.3 Expansions .. . .... . . 297
9.3.1 Expansion of eix ••• • ••• • ••.• . 297
9.4 Moments and Derivatives .......... . . . 301
9.5 Two Big Theorems: Uniqueness and Continuity . 302
9.6 The Selection Theorem, Tightness, and
Prohorov's theorem .. . ........ . . 307
9.6.1 The Selection Theorem .. .. . . . 307
9.6.2 Tightness, Relative Compactness,
and Prohorov's theorem . . . . . . 309
9.6.3 Proof of the Continuity Theorem . . 311
9.7 The Classical CLT for iid Random Variables . . 312
9.8 The Lindeberg-Feller CLT . 314
9.9 Exercises ... ...... . .... .. .. . . 321
10 Martingales 333
10.1 Prelude to Conditional Expectation:
The Radon-Nikodym Theorem . . . . 333
10.2 Definition of Conditional Expectation . . 339
10.3 Properties of Conditional Expectation . . .. 344
10.4 Martingales .. . . .. ....... . . 353
10.5 Examples of Martingales ..... . . . 356
10.6 Connections between Martingales and Submartingales . . 360
10.6.1 Doob's Decomposition . 360
10.7 Stopping Times . . . . . . . . . . . . . . . 363
10.8 Positive Super Martingales . .. .. . . . . 366
10.8.1 Operations on Supermartingales . . 367
10.8.2 Upcrossings . . . . . . . . . . . . 369
10.8.3 Boundedness Properties .. . . . . 369
10.8.4 Convergence of Positive Super Martingales . . 371
10.8.5 Closure . . . . . . . . . . . . 374
10.8.6 Stopping Supermartingales . 377
10.9 Examples .... . . .... . . . 379
10.9.1 Gambler's Ruin .. ... . . 379
10.9.2 Branching Processes . ... . . 380
10.9.3 Some Differentiation Theory . 382
10.10 Martingale and Submartingale Convergence . . . 386
10.10.1 Krickeberg Decomposition .. ... . . 386
10.10.2 Doob's (Sub)martingale Convergence Theorem . . .. 387
10.11 Regularity and Closure . . . . 388
10.12 Regularity and Stopping . . . . 390
10.13 Stopping Theorems . .. . . . 392
Contents ix
10.14 Wald's Identity and Random Walks . . . . . . . ... .. . 398
10.14.1 The Basic Martingales . . . . . . . . . ... ... 400
10.14.2 Regular Stopping Times . . . . . . . . . .. 402
10.14.3 Examples of Integrable Stopping Times .. . ...... 407
10.14.4 The Simple Random Walk . ... ..... . . . . ... 409
10.15 Reversed Martingales . ... . .. . .... .. . . . .. .. . . 412
10.16 Fundamental Theorems of Mathematical Finance .. . . . .. . 416
10.16.1 A Simple Market Model ..... . . . .. .. ... . . 416
10.16.2 Admissible Strategies and Arbitrage . . . .. .. 419
10.16.3 Arbitrage and Martingales . . . . . . . . . . . .. 420
10.16.4 Complete Markets . . . . . . . ...... .. .. ... 425
10.16.5 Option Pricing .. . . ... . . ..... ... ... .. 428
10.17 Exercises . ... . . .. . . .. .... . ... . . . . .. .... 429
References 443
Index 445
Preface
There are several good current probability books-Billingsley (1995), Durrett
(1991), Port (1994), Fristedt and Gray (1997), and I still have great affection
for the books I was weaned on-Breiman (1992), Chung (1974), Feller (1968,
1971) and even Loeve (1977). The books by Neveu (1965, 1975) are educational
and models of good organization. So why publish another? Many of the exist
ing books are encyclopedic in scope and seem intended as reference works, with
navigation problems for the beginner. Some neglect to teach any measure theory,
assuming students have already learned all the foundations elsewhere. Most are
written by mathematicians and have the built in bias that the reader is assumed to
be a mathematician who is coming to the material for its beauty. Most books do
not clearly indicate a one-semester syllabus which will offer the essentials.
I and my students have consequently found difficulties using currently avail
able probability texts. There is a large market for measure theoretic probability by
students whose primary focus is not mathematics for its own sake. Rather, such
students are motivated by examples and problems in statistics, engineering, biol
ogy and finance to study probability with the expectation that it will be useful to
them in their research work. Sometimes it is not clear where their work will take
them, but it is obvious they need a deep understanding of advanced probability in
order to read the literature, understand current methodology, and prove that the
new technique or method they are dreaming up is superior to standard practice.
So the clientele for an advanced or measure theoretic probability course that is
primarily motivated by applications outnumbers the clientele deeply embedded in
pure mathematics. Thus, I have tried to show links to statistics and operations re
search. The pace is quick and disciplined. The course is designed for one semester
with an overstuffed curriculum that leaves little time for interesting excursions or