Probability Approximations via the Poisson Clumping Heuristic

288 Pages·1989·4.23 MB·English

by  David Aldous (auth.)

11prp" D. Aldous
r
Applied Probability
Mathematical
Approximations
Sciences
77 via the Poisson
Clumping Heuristic
Springer-Verlag
Applied Mathematical Sciences
Volume 77
Editors
F. John J. E. Marsden L. Sirovich
Advisors
M. Ghil J.K. Hale J. Keller
K. Kirchgassner B.J. Matkowsky
J.T. Stuart A. Weinstein
Applied Mathematical Sciences
1. John: Partial Differential Equations, 4th ed.
2. Sirovich: Techniques of Asymptotic Analysis.
3. Hale: Theory of Functional Differential Equations, 2nd ed.
4. Percus: Combinatorial Methods.
5. von Mises/Friedrichs: Fluid Dynamics.
6. Freiberger/Grenander: A Short Course in Computational Probability and Statistics.
7. Pipkin: Lectures on Viscoelasticity Theory.
9. Friedrichs: Spectral Theory of Operators in Hilbert Space.
11. Wolovich: Linear Multivariable Systems.
12. Berkovitz: Optimal Control Theory.
13. Bluman/Cole:Similarity Methods for Differential Equations.
14. Yoshizawa: Stability Theory and the Existence of Periodic Solution and Almost Periodic Solutions.
15. Braun: Differential Equations and Their Applications, 3rd ed.
16. Lefschetz: Applications of Algebraic Topology.
17. Collatz/Wetterling: Optimization Problems.
18. Grenander: Pattern Synthesis: Lectures in Pattern Theory, Vol I.
20. Driver:Ordinary and Delay Differential Equations.
21. Courant/Friedrichs: Supersonic Flow and Shock Waves.
22. Rouche/Habets/Lalov:Stability Theory by Liapunov's Direct Method.
23. Lamperti: Stochastic Processes: A Survey of the Mathematical Theory.
24. Grenander: Pattern Analysis: Lectures in Pattern Theory, Vol. 11.
25. Davies: Integral Transforms and Their Applications, 2nd ed.
26. Kushner/Clark: Stochastic Approximation Methods for Constrained and Unconstrained Systems
27. de Boor: A Practical Guide to Splines.
28. Keilson: Markov Chain Models-Rarity and Exponentiality.
29. de Veubeke: A Course in Elasticity.
30. Sniarvcki: Geometric Quantization and Quantum Mechanics.
31. Reid:Sturmian Theory for Ordinary Differential Equations.
32. Meis/Markowitz: Numerical Solution of Partial Differential Equations.
33. Grenander: Regular Structures: Lectures in Pattern Theory, Vol. III.
34. Kevorkian/Cole; Perturbation methods in Applied Mathematics.
35. Carr: Applications of Centre Manifold Theory.
36. Bengtsson/Ghil/Kallen: Dynamic Meteorology: Data Assimilation Methods.
37. Saperstone: Semidynamical Systems in Infinite Dimensional Spaces.
38. Lichtenberg/Lieberman: Regular and Stochastic Motion.
39. Piccini/Stampacchia/Vidossich: Ordinary Differential Equations in R.
40. Navlor/Sell: Linear Operator Theory in Engineering and Science.
41. Sparrow: The Lorenz Equations: Bifurcations, Chaos, and Strange Attractors.
42. Guckenheimer/Holmes: Nonlinear Oscillations, Dynamical Systems and Bifurcations of Vector Fields.
43. Ockendon/Tavler: Inviscid Fluid Flows.
44. Po_v: Semigroups of Linear Operators and Applications to Partial Differential Equations.
45. Glashoff/Gustalson: Linear Optimization and Approximation: An Introduction to the Theoretical Analysis
and Numerical Treatment of Semi-Infinite Programs.
46. Wilcox: Scattering Theory for Diffraction Gratings.
47. Hale et al.: An Introduction to Infinite Dimensional Dynamical Systems-Geometric Theory.
48. Murray; Asymptotic Analysis.
49. Ladv_henskava: The Boundary-Value Problems of Mathematical Physics.
50. Wilcox: Sound Propagation in Stratified Fluids.
SL Gohtbink-/Schaef/er: Bifurcation and Groups in Bifurcation Theory, Vol. I.
52. Chipot: Variation al Inequalities and Flow in Porous Media.
53. Majc/a: Compressible Fluid Flow and Systems of Conservation Laws in Several Space Variables.
54. Wasow: Linear Turning Point Theory.
(( ontinued/ollowin,ti index)
David Aldous
Probability Approximations via
the Poisson Clumping Heuristic
With 25 Illustrations
Springer-Verlag
New York Berlin Heidelberg
London Paris Tokyo
David Aldous
Department of Statistics
University of California-Berkeley
Berkeley, CA 94720
USA
Editors
F. John J.E. Marsden L. Sirovich
Courant Institute of Department of Division of
Mathematical Sciences Mathematics Applied Mathematics
New York University University of California Brown University
New York, NY 10012 Berkeley, CA 94720 Providence, RI 02912
USA USA USA
Mathematics Subject Classification (1980): 60C05, 60D05, 60F05, 60J20, 60K99
Library of Congress Cataloging-in-Publication Data
Aldous, D. J. (David J.)
Probability approximations via the Poisson clumping
heuristic.
(Applied mathematical sciences ; v. 77)
Bibliography: p.
1. Combinatorial probabilities. 2. Stochastic
geometry. 3. Markov processes. I. Title. II. Title:
Poisson clumping heuristic. III. Series: Applied
mathematical sciences (Springer-Verlag New York Inc.)
v. 77.
QA1.A647 vol. 77 [QA273.45] 510 s [519.2] 88-29453
Printed on acid-free paper
© 1989 by Springer-Verlag New York Inc.
All rights reserved. This work may not be translated or copied in whole or in part without the written
permission of the publisher (Springer-Verlag, 175 Fifth Avenue, New York, NY 10010, USA),
except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with
any form of information storage and retrieval, electronic adaptation, computer software, or by similar
or dissimilar methodology now known or hereafter developed is forbidden.
The use of general descriptive names, trade names, trademarks, etc. in this publication, even if the
former are not especially identified, is not to be taken as a sign that such names, as understood by
the Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone.
Camera-ready copy prepared using LaTEX.
Printed and bound by R.R. Donnelley and Sons, Harrisonburg, Virginia.
Printed in the United States of America.
9 8 7 6 5 4 3 2
1
ISBN 0-387-96899-7 Springer-Verlag NewYork Berlin Heidelberg
ISBN 3-540-96899-7 Springer-Verlag Berlin Heidelberg NewYork
Preface
If you place a large number of points randomly in the unit square, what
is the distribution of the radius of the largest circle containing no points?
Of the smallest circle containing 4 points? Why do Brownian sample paths
have local maxima but not points of increase, and how nearly do they have
points of increase? Given two long strings of letters drawn i.i.d. from a
finite alphabet, how long is the longest consecutive (resp. non-consecutive)
substring appearing in both strings? If an imaginary particle performs a
simple random walk on the vertices of a high-dimensional cube, how long
does it take to visit every vertex? If a particle moves under the influence
of a potential field and random perturbations of velocity, how long does it
take to escape from a deep potential well? If cars on a freeway move with
constant speed (random from car to car), what is the longest stretch of
empty road you will see during a long journey? If you take a large i.i.d.
sample from a 2-dimensional rotationally-invariant distribution, what is the
maximum over all half-spaces of the deviation between the empirical and
true distributions?
These questions cover a wide cross-section of theoretical and applied
probability. The common theme is that they all deal with maxima or min-
ima, in some sense. The purpose of this book is to explain a simple idea
which enables one to write down, with little effort, approximate solutions
to such questions. Let us try to say this idea in one paragraph.
(a) Problems about random extrema can often be translated into prob-
lems about sparse random sets in d > 1 dimensions.
(b) Sparse random sets often resemble i.i.d. random clumps thrown down
randomly (i.e., centered at points of a Poisson process).
(c) The problem of interest reduces to estimating mean clump size.
(d) This mean clump size can be estimated by approximating the under-
lying random process locally by a simpler, known process for which
explicit calculations are possible.
(Part (b) explains the name Poisson clumping heuristic).
N
This idea is known, though rarely explicitly stated, in several specific
settings, but its power and range seems not to be appreciated. I assert that
this idea provides the correct way to look at extrema and rare events in
a wide range of probabilistic settings: to demonstrate this assertion, this
book treats over 100 examples. Our arguments are informal, and we are not
going to prove anything. This is a rather eccentric format for a mathematics
book some reasons for this format are indicated later.
The opening list of problems was intended to persuade every probabilist
to read the book! I hope it will appeal to graduate students as well as
experts, to the applied workers as well as theoreticians. Much of it should
be comprehensible to the reader with a knowledge of stochastic processes at
the non-measure-theoretic level (Markov chains, Poisson process, renewal
theory, introduction to Brownian motion), as provided by the books of
Karlin and Taylor (1975; 1982) and Ross (1983). Different chapters are
somewhat independent, and the level of sophistication varies.
Although the book ranges over many fields of probability theory, in each
field we focus narrowly on examples where the heuristic is applicable, so
this work does not constitute a complete account of any field. I have tried
to maintain an honest "lecture notes" style through the main part of each
chapter. At the end of each chapter is a "Commentary" giving references
to background material and rigorous results. In giving references I try to
give a book or survey article on the field in question, supplemented by
recent research papers where appropriate: I do not attempt to attribute
results to their discoverers. Almost all the examples are natural (rather
than invented to show off the technique), though I haven't always given a
thorough explanation of how they arise.
The arguments in examples are sometimes deliberately concise. Most
results depend on one key calculation, and it is easier to see this in a
half-page argument than in a three-page argument. In rigorous treatments
it is often necessary to spend much effort in showing that certain effects
are ultimately negligible; we simply omit this effort, relying on intuition
to see what the dominant effect is. No doubt one or two of our heuristic
conclusions are wrong: if heuristic arguments always gave the right answer,
then there wouldn't be any point in ever doing rigorous arguments, would
there? Various problems which seem interesting and unsolved are noted as
"thesis projects", though actually some are too easy, and others too hard,
for an average Ph.D. thesis.
The most-studied field of application of the heuristic is to extremes of 1-
parameter stationary processes. The standard reference work on this field,
Leadbetter et al. (1983), gives theoretical results covering perhaps 10 of our
examples. One could write ten similar books, each covering 10 examples
from another field. But I don't have the energy or inclination to do so,
which is one reason why this book gives only heuristics. Another reason is
that connections between examples in different fields are much clearer in
the heuristic treatment than in a complete technical treatment, and 1 hope
Vii
this book will make these connections more visible.
At the risk of boring some readers and annoying others, here is a para-
graph on the philosophy of approximations, heuristic and limit theorems.
The proper business of probabilists is calculating probabilities. Often exact
calculations are tedious or impossible, so we resort to approximations. A
limit theorem is an assertion of the form: "the error in a certain approxi-
mation tends to 0 as (say) N -+ oc" . Call such limit theorem naive if there
is no explicit error bound in terms of N and the parameters of the under-
lying process. Such theorems are so prevalent in theoretical and applied
probability that people seldom stop to ask their purpose. Given a serious
applied problem involving specific parameters, the natural first steps are
to seek rough analytic approximations and to run computer simulations;
the next step is to do careful numerical analysis. It is hard to give any
argument for the relevance of a proof of a naive limit theorem, except as a
vague reassurance that your approximation is sensible, and a good heuris-
tic argument seems equally reassuring. For the theoretician, the defense of
naive limit theorems is "I want to prove something, and that's all f can
prove". There are fields which are sufficiently hard that this is a reasonable
attitude (some of the areas in Chapters G, I, J for example), but in most
of the fields in this book the theoretical tools for proving naive limit the-
orems have been sufficiently developed that such theorems are no longer
of serious theoretical research interest (although a few books consolidating
the techniques would be useful).
Most of our approximations in particular examples correspond to known
naive limit theorems, mentioned in the Commentaries. I deliberately de-
emphasize this aspect, since as argued above I regard the naive limit theory
as irrelevant for applications and mostly trite as theory. On the other hand,
explicit error bounds are plainly relevant for applications and interesting
as theory (because they are difficult, for a start!). In most of our examples,
explicit error bounds are not know: I regard this as an important area
for future research. Stein's method is a powerful modern tool for getting
explicit bounds in "combinatorial" type examples, whose potential is not
widely realized. Hopefully other tools will become available in the future.
Acknowledgements: As someone unable to recollect what I had for dinner
last night, I am even more unable to recollect the many people who (con-
sciously or unconsciously) provided sources of the examples; but I thank
them. Course based on partial early drafts of the book were given in Berke-
ley in 1984 and Cornell in 1986, and I thank the audiences for their feed-
back. In particular, I thank Persi Diaconis, Rick Durrett, Harry Kesten,
V. Anantharam and Jim Pitman for helpful comments. I also thank Pilar
Fresnedo for drawing the diagrams, and Ed Sznyter for a great job con-
verting my haphazard two-fingered typing into this elegant LATEX book.

Contents
A The Heuristic
1
Al The M/M/1 queue ..
. . . . . . . . . . . . . . . . . . . . . . 1
A2 Mosaic processes on R2. . . . . . . . . . . . . . . .... . . 2
A3 Mosaic processes on other spaces .. . . . . . . . . . . . . 5
A4 The heuristic
. . . . . . . . . . . . . . . . . . . . . . . . . . . 5
A5 Estimating clump sizes . . . . . . . . . . . . . . . . . . . . 7
A6 The harmonic mean formula. . . . . . . . . . . . . . . . . . 8
A7 Conditioning on semi-local maxima . . . . . . . . . . . . . . 10
A8 The renewal-sojourn method . . . . . . . . . . . . . . . . . . 11
A9 The ergodic-exit method . . . . . . . . . . . . . . . . . . . . 12
A10 Limit assertions . . . . . . . .... . . . . . . . . . . . . . . . . 14
A11-A21 Commentary . . . . . . . . . . . . . . . . . . . . . 15
B Markov Chain Hitting Times 23
. ...
B l Introduction. . . . . . . . . . . . . . . . . . . . . . 23
B2 The heuristic for Markov hitting times .. . . . . . . . . . . . 24
B3 Example: Basic single server queue .. . . . . . . . . . . . . . 25
B4 Example: Birth-and-death processes . . . . . . . . . . . . . . 25
B5 Example: Patterns in coin-tossing 26
. . . . . . . . . . . . . . .
B6 Example: Card-shuffling.. 27
B7 Example: Random walk on Zd mod N . . . . . . . . . . . 28
B8 Example: Random trapping on Zd.. . . . . . . . . . . . 28
B9 Example: Two M/M/1 queues in series. 28
. . . . . . . . . . .
B10 Example: Large density of heads in coin-tossing. 29
. . . . . .
B l l Counter-example .. . . . . . . . . . . . . . . . . . . . . . . . 30
B12 Hitting small subsets. . . . . . . . . . . . . . . . . . . . 30
B13 Example: Patterns in coin-tossing, continued. . . . . . . . . 31
B14 Example: Runs in biased die-throwing .. . . . . . . . . . . . 31
B15 Example: Random walk on Zd mod N, continued. . . . . . 32
B16 Hitting sizable subsets .. . . . . . . . . . . . . . . . . . . . . 32
B17 The ergodic-exit form of the heuristic for Markov hitting times. 32
B18 Example: A simple reliability model . . . . . . . . . . . . . . 33
B19 Example: Timesharing computer. . . . . . . . . . . . . . . . 33