Probability Theory and Stochastic Processes

717 Pages·2020·8.299 MB·English

by  Pierre Brémaud

Universitext
Pierre Brémaud
Probability
Theory and
Stochastic
Processes
Universitext
Universitext
Series Editors
Sheldon Axler
San Francisco State University
Carles Casacuberta
Universitat de Barcelona
John Greenlees
University of Warwick, Coventry
Angus MacIntyre
Queen Mary University of London
Kenneth Ribet
University of California, Berkeley
Claude Sabbah
École Polytechnique, CNRS, Université Paris-Saclay, Palaiseau
Endre Süli
University of Oxford
Wojbor A. Woyczyński,
Case Western Reserve University
Universitext is a series of textbooks that presents material from a wide variety of
mathematical disciplines at master’s level and beyond. The books, often well
class-tested by their author, may have an informal, personal even experimental
approachtotheirsubjectmatter.Someofthemostsuccessfulandestablishedbooks
intheserieshaveevolvedthroughseveraleditions,alwaysfollowingtheevolution
of teaching curricula, to very polished texts.
Thus as research topics trickle down into graduate-level teaching, first textbooks
written for new, cutting-edge courses may make their way into Universitext.
More information about this series at http://www.springer.com/series/223
Pierre Brémaud
Probability Theory
and Stochastic Processes
Pierre Brémaud
Département d’Informatique
INRIA, École Normale Supérieure
Paris CX 5, France
ISSN 0172-5939 ISSN 2191-6675 (electronic)
Universitext
ISBN 978-3-030-40182-5 ISBN 978-3-030-40183-2 (eBook)
https://doi.org/10.1007/978-3-030-40183-2

© Springer Nature Switzerland AG 2020
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the
material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors, and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the
editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors
or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims
in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Pour Marion
Contents
Introduction xv
Part One: Probability Theory 1
1 Warming Up 3
1.1 Sample Space, Events and Probability . . . . . . . . . . . . . . . . 3
1.1.1 Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.1.2 Probability of Events . . . . . . . . . . . . . . . . . . . . . . 6
1.2 Independence and Conditioning . . . . . . . . . . . . . . . . . . . . 9
1.2.1 Independent Events . . . . . . . . . . . . . . . . . . . . . . . 9
1.2.2 Bayes’ Calculus . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.2.3 Conditional Independence . . . . . . . . . . . . . . . . . . . 13
1.3 Discrete Random Variables . . . . . . . . . . . . . . . . . . . . . . . 14
1.3.1 Probability Distributions and Expectation . . . . . . . . . . 14
1.3.2 Famous Discrete Probability Distributions . . . . . . . . . . 22
1.3.3 Conditional Expectation . . . . . . . . . . . . . . . . . . . . 29
1.4 The Branching Process . . . . . . . . . . . . . . . . . . . . . . . . . 32
1.4.1 Generating Functions . . . . . . . . . . . . . . . . . . . . . . 32
1.4.2 Probability of Extinction . . . . . . . . . . . . . . . . . . . . 38
1.5 Borel’s Strong Law of Large Numbers . . . . . . . . . . . . . . . . . 40
1.5.1 The Borel–Cantelli Lemma . . . . . . . . . . . . . . . . . . . 40
1.5.2 Markov’s Inequality . . . . . . . . . . . . . . . . . . . . . . . 41
1.5.3 Proof of Borel’s Strong Law . . . . . . . . . . . . . . . . . . 42
1.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2 Integration 51
2.1 Measurability and Measure . . . . . . . . . . . . . . . . . . . . . . . 52
2.1.1 Measurable Functions. . . . . . . . . . . . . . . . . . . . . . 52
2.1.2 Measure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
2.2 The Lebesgue Integral . . . . . . . . . . . . . . . . . . . . . . . . . 66
2.2.1 Construction of the Integral . . . . . . . . . . . . . . . . . . 66
2.2.2 Elementary Properties of the Integral . . . . . . . . . . . . . 71
2.2.3 Beppo Levi, Fatou and Lebesgue . . . . . . . . . . . . . . . 73
2.3 The Other Big Theorems . . . . . . . . . . . . . . . . . . . . . . . . 75
2.3.1 The Image Measure Theorem . . . . . . . . . . . . . . . . . 76
2.3.2 The Fubini–Tonelli Theorem . . . . . . . . . . . . . . . . . . 76
2.3.3 The Riesz–Fischer Theorem . . . . . . . . . . . . . . . . . . 83
vii
viii CONTENTS
2.3.4 The Radon–Nikody´m Theorem . . . . . . . . . . . . . . . . 88
2.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
3 Probability and Expectation 95
3.1 From Integral to Expectation . . . . . . . . . . . . . . . . . . . . . 95
3.1.1 Translation . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
3.1.2 Probability Distributions . . . . . . . . . . . . . . . . . . . . 97
3.1.3 Independence and the Product Formula. . . . . . . . . . . . 108
3.1.4 Characteristic Functions . . . . . . . . . . . . . . . . . . . . 114
3.1.5 Laplace Transforms . . . . . . . . . . . . . . . . . . . . . . . 118
3.2 Gaussian vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
3.2.1 Two Equivalent Definitions . . . . . . . . . . . . . . . . . . 119
3.2.2 Independence and Non-correlation . . . . . . . . . . . . . . . 121
3.2.3 The pdf of a Non-degenerate Gaussian Vector . . . . . . . . 123
3.3 Conditional Expectation . . . . . . . . . . . . . . . . . . . . . . . . 125
3.3.1 The Intermediate Theory . . . . . . . . . . . . . . . . . . . . 125
3.3.2 The General Theory . . . . . . . . . . . . . . . . . . . . . . 131
3.3.3 The Doubly Stochastic Framework . . . . . . . . . . . . . . 135
3.3.4 The L2-theory of Conditional Expectation . . . . . . . . . . 136
3.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
4 Convergences 145
4.1 Almost-sure Convergence . . . . . . . . . . . . . . . . . . . . . . . . 145
4.1.1 A Sufficient Condition and a Criterion . . . . . . . . . . . . 145
4.1.2 Beppo Levi, Fatou and Lebesgue . . . . . . . . . . . . . . . 148
4.1.3 The Strong Law of Large Numbers . . . . . . . . . . . . . . 149
4.2 Two Other Types of Convergence . . . . . . . . . . . . . . . . . . . 156
4.2.1 Convergence in Probability . . . . . . . . . . . . . . . . . . . 156
4.2.2 Convergence in Lp . . . . . . . . . . . . . . . . . . . . . . . 158
4.2.3 Uniform Integrability . . . . . . . . . . . . . . . . . . . . . . 160
4.3 Zero-one Laws . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
4.3.1 Kolmogorov’s Zero-one Law . . . . . . . . . . . . . . . . . . 162
4.3.2 The Hewitt–Savage Zero-one Law . . . . . . . . . . . . . . . 163
4.4 Convergence in Distribution and in Variation . . . . . . . . . . . . . 166
4.4.1 The Role of Characteristic Functions . . . . . . . . . . . . . 166
4.4.2 The Central Limit Theorem . . . . . . . . . . . . . . . . . . 172
4.4.3 Convergence in Variation . . . . . . . . . . . . . . . . . . . . 176
4.4.4 Proof of Paul L´evy’s Criterion . . . . . . . . . . . . . . . . . 182
4.5 The Hierarchy of Convergences . . . . . . . . . . . . . . . . . . . . 188
4.5.1 Almost-sure vs in Probability . . . . . . . . . . . . . . . . . 188
4.5.2 The Rank of Convergence in Distribution . . . . . . . . . . . 189
4.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
CONTENTS ix
Part Two: Standard Stochastic Processes 197
5 Generalities on Random Processes 199
5.1 The Distribution of a Random Process . . . . . . . . . . . . . . . . 199
5.1.1 Kolmogorov’s Theorem on Distributions . . . . . . . . . . . 199
5.1.2 Second-order Stochastic Processes . . . . . . . . . . . . . . . 203
5.1.3 Gaussian Processes . . . . . . . . . . . . . . . . . . . . . . . 206
5.2 Random Processes as Random Functions . . . . . . . . . . . . . . . 208
5.2.1 Versions and Modifications . . . . . . . . . . . . . . . . . . . 208
5.2.2 Kolmogorov’s Continuity Condition . . . . . . . . . . . . . . 209
5.3 Measurability Issues . . . . . . . . . . . . . . . . . . . . . . . . . . 212
5.3.1 Measurable Processes and their Integrals . . . . . . . . . . . 212
5.3.2 Histories and Stopping Times . . . . . . . . . . . . . . . . . 213
5.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
6 Markov Chains, Discrete Time 221
6.1 The Markov Property . . . . . . . . . . . . . . . . . . . . . . . . . . 221
6.1.1 The Markov Property on the Integers . . . . . . . . . . . . . 221
6.1.2 The Markov Property on a Graph . . . . . . . . . . . . . . . 227
6.2 The Transition Matrix . . . . . . . . . . . . . . . . . . . . . . . . . 235
6.2.1 Topological Notions . . . . . . . . . . . . . . . . . . . . . . . 235
6.2.2 Stationary Distributions and Reversibility . . . . . . . . . . 237
6.2.3 The Strong Markov Property . . . . . . . . . . . . . . . . . 242
6.3 Recurrence and Transience . . . . . . . . . . . . . . . . . . . . . . . 245
6.3.1 Classification of States . . . . . . . . . . . . . . . . . . . . . 245
6.3.2 The Stationary Distribution Criterion . . . . . . . . . . . . . 250
6.3.3 Foster’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . 257
6.4 Long-run Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
6.4.1 The Markov Chain Ergodic Theorem . . . . . . . . . . . . . 260
6.4.2 Convergence in Variation to Steady State . . . . . . . . . . . 262
6.4.3 Null Recurrent Case: Orey’s Theorem . . . . . . . . . . . . . 265
6.4.4 Absorption . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
6.5 Monte Carlo Markov Chain Simulation . . . . . . . . . . . . . . . . 272
6.5.1 Basic Principle and Algorithms . . . . . . . . . . . . . . . . 272
6.5.2 Exact Sampling . . . . . . . . . . . . . . . . . . . . . . . . . 275
6.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
7 Markov Chains, Continuous Time 289
7.1 Homogeneous Poisson Processes on the Line . . . . . . . . . . . . . 289
7.1.1 The Counting Process and the Interval Sequence. . . . . . . 289
7.1.2 Stochastic Calculus of hpps . . . . . . . . . . . . . . . . . . 294
7.2 The Transition Semigroup . . . . . . . . . . . . . . . . . . . . . . . 298
7.2.1 The Infinitesimal Generator . . . . . . . . . . . . . . . . . . 298
7.2.2 The Local Characteristics . . . . . . . . . . . . . . . . . . . 302
7.2.3 hmcs from hpps . . . . . . . . . . . . . . . . . . . . . . . . 306
7.3 Regenerative Structure . . . . . . . . . . . . . . . . . . . . . . . . . 310
7.3.1 The Strong Markov Property . . . . . . . . . . . . . . . . . 310
7.3.2 Imbedded Chain . . . . . . . . . . . . . . . . . . . . . . . . 311
7.3.3 Conditions for Regularity . . . . . . . . . . . . . . . . . . . 314
x CONTENTS
7.4 Long-run Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
7.4.1 Recurrence . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
7.4.2 Convergence to Equilibrium . . . . . . . . . . . . . . . . . . 324
7.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
8 Spatial Poisson Processes 329
8.1 Generalities on Point Processes . . . . . . . . . . . . . . . . . . . . 329
8.1.1 Point Processes as Random Measures . . . . . . . . . . . . . 329
8.1.2 Point Process Integrals and the Intensity Measure . . . . . . 334
8.1.3 The Distribution of a Point Process . . . . . . . . . . . . . . 338
8.2 Unmarked Spatial Poisson Processes . . . . . . . . . . . . . . . . . 345
8.2.1 Construction . . . . . . . . . . . . . . . . . . . . . . . . . . 345
8.2.2 Poisson Process Integrals . . . . . . . . . . . . . . . . . . . . 347
8.3 Marked Spatial Poisson Processes . . . . . . . . . . . . . . . . . . . 351
8.3.1 As Unmarked Poisson Processes . . . . . . . . . . . . . . . . 351
8.3.2 Operations on Poisson Processes. . . . . . . . . . . . . . . . 354
8.3.3 Change of Probability Measure . . . . . . . . . . . . . . . . 357
8.4 The Boolean Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 360
8.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365
9 Queueing Processes 371
9.1 Discrete-time Markovian Queues. . . . . . . . . . . . . . . . . . . . 371
9.1.1 The Basic Example . . . . . . . . . . . . . . . . . . . . . . . 371
9.1.2 Multiple Access Communication . . . . . . . . . . . . . . . . 372
9.1.3 The Stack Algorithm . . . . . . . . . . . . . . . . . . . . . . 375
9.2 Continuous-time Markovian Queues . . . . . . . . . . . . . . . . . . 378
9.2.1 Isolated Markovian Queues. . . . . . . . . . . . . . . . . . . 378
9.2.2 Markovian Networks . . . . . . . . . . . . . . . . . . . . . . 385
9.3 Non-exponential Models . . . . . . . . . . . . . . . . . . . . . . . . 391
9.3.1 M/GI/∞ . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392
9.3.2 M/GI/1/∞/fifo . . . . . . . . . . . . . . . . . . . . . . . . 394
9.3.3 GI/M/1/∞/fifo . . . . . . . . . . . . . . . . . . . . . . . . 396
9.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399
10 Renewal and Regenerative Processes 403
10.1 Renewal Point processes . . . . . . . . . . . . . . . . . . . . . . . . 403
10.1.1 The Renewal Measure . . . . . . . . . . . . . . . . . . . . . 403
10.1.2 The Renewal Equation . . . . . . . . . . . . . . . . . . . . . 407
10.1.3 Stationary Renewal Processes . . . . . . . . . . . . . . . . . 413
10.2 The Renewal Theorem . . . . . . . . . . . . . . . . . . . . . . . . . 416
10.2.1 The Key Renewal Theorem . . . . . . . . . . . . . . . . . . 416
10.2.2 The Coupling Proof of Blackwell’s Theorem . . . . . . . . . 424
10.2.3 Defective and Excessive Renewal Equations . . . . . . . . . 428
10.3 Regenerative Processes . . . . . . . . . . . . . . . . . . . . . . . . . 430
10.3.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 430
10.3.2 The Limit Distribution . . . . . . . . . . . . . . . . . . . . . 431
10.4 Semi-Markov Processes . . . . . . . . . . . . . . . . . . . . . . . . . 435