Reversible Markov Chains and Random Walks on Graphs

516 Pages·2014·1.78 MB·English

Reversible Markov Chains and Random Walks on Graphs
David Aldous and James Allen Fill
Unfinished monograph, 2002 (this is recompiled version, 2014)
2
Contents
1 Introduction (July 20, 1999) 13
1.1 Word problems . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.1.1 Random knight moves . . . . . . . . . . . . . . . . . . 13
1.1.2 The white screen problem . . . . . . . . . . . . . . . . 13
1.1.3 Universal traversal sequences . . . . . . . . . . . . . . 14
1.1.4 How long does it take to shuffle a deck of cards? . . . 15
1.1.5 Samplingfromhigh-dimensionaldistributions: Markov
chain Monte Carlo . . . . . . . . . . . . . . . . . . . . 15
1.1.6 Approximate counting of self-avoiding walks . . . . . . 16
1.1.7 Simulating a uniform random spanning tree . . . . . . 17
1.1.8 Voter model on a finite graph . . . . . . . . . . . . . . 17
1.1.9 Are you related to your ancestors? . . . . . . . . . . . 17
1.2 So what’s in the book? . . . . . . . . . . . . . . . . . . . . . . 18
1.2.1 Conceptual themes . . . . . . . . . . . . . . . . . . . . 18
1.2.2 Prerequisites . . . . . . . . . . . . . . . . . . . . . . . 19
1.2.3 Contents and alternate reading . . . . . . . . . . . . . 19
2 General Markov Chains (September 10, 1999) 23
2.1 Notation and reminders of fundamental results . . . . . . . . 23
2.1.1 Stationary distribution and asymptotics . . . . . . . . 24
2.1.2 Continuous-time chains . . . . . . . . . . . . . . . . . 25
2.2 Identities for mean hitting times and occupation times . . . . 27
2.2.1 Occupation measures and stopping times . . . . . . . 27
2.2.2 Mean hitting time and related formulas . . . . . . . . 29
2.2.3 Continuous-time versions . . . . . . . . . . . . . . . . 34
2.3 Variances of sums . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.4 Two metrics on distributions . . . . . . . . . . . . . . . . . . 36
2.4.1 Variation distance . . . . . . . . . . . . . . . . . . . . 36
2.4.2 L2 distance . . . . . . . . . . . . . . . . . . . . . . . . 39
3
4 CONTENTS
2.4.3 Exponential tails of hitting times . . . . . . . . . . . . 41
2.5 Distributional identities . . . . . . . . . . . . . . . . . . . . . 42
2.5.1 Stationarity consequences . . . . . . . . . . . . . . . . 42
2.5.2 A generating function identity . . . . . . . . . . . . . 43
2.5.3 Distributions and continuization . . . . . . . . . . . . 44
2.6 Matthews’ method for cover times . . . . . . . . . . . . . . . 45
2.7 New chains from old . . . . . . . . . . . . . . . . . . . . . . . 47
2.7.1 The chain watched only on A . . . . . . . . . . . . . . 47
2.7.2 The chain restricted to A . . . . . . . . . . . . . . . . 48
2.7.3 The collapsed chain . . . . . . . . . . . . . . . . . . . 48
2.8 Miscellaneous methods . . . . . . . . . . . . . . . . . . . . . . 49
2.8.1 Martingale methods . . . . . . . . . . . . . . . . . . . 49
2.8.2 A comparison argument . . . . . . . . . . . . . . . . . 51
2.8.3 Wald equations . . . . . . . . . . . . . . . . . . . . . . 52
2.9 Notes on Chapter 2. . . . . . . . . . . . . . . . . . . . . . . . 52
2.10 Move to other chapters . . . . . . . . . . . . . . . . . . . . . . 55
2.10.1 Attaining distributions at stopping times . . . . . . . 55
2.10.2 Differentiating stationary distributions . . . . . . . . . 55
3 Reversible Markov Chains (September 10, 2002) 57
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.1.1 Time-reversals and cat-and-mouse games . . . . . . . 59
3.1.2 Entrywise ordered transition matrices . . . . . . . . . 62
3.2 Reversible chains and weighted graphs . . . . . . . . . . . . . 63
3.2.1 The fluid model . . . . . . . . . . . . . . . . . . . . . . 66
3.3 Electrical networks . . . . . . . . . . . . . . . . . . . . . . . . 67
3.3.1 Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.3.2 The analogy . . . . . . . . . . . . . . . . . . . . . . . . 68
3.3.3 Mean commute times . . . . . . . . . . . . . . . . . . 70
3.3.4 Foster’s theorem . . . . . . . . . . . . . . . . . . . . . 71
3.4 The spectral representation . . . . . . . . . . . . . . . . . . . 72
3.4.1 Mean hitting times and reversible chains . . . . . . . . 75
3.5 Complete monotonicity . . . . . . . . . . . . . . . . . . . . . 77
3.5.1 Lower bounds on mean hitting times . . . . . . . . . . 79
3.5.2 Smoothness of convergence . . . . . . . . . . . . . . . 81
3.5.3 Inequalities for hitting time distributions on subsets . 83
3.5.4 Approximate exponentiality of hitting times . . . . . . 85
3.6 Extremal characterizations of eigenvalues . . . . . . . . . . . 87
3.6.1 The Dirichlet formalism . . . . . . . . . . . . . . . . . 87
3.6.2 Summary of extremal characterizations . . . . . . . . 89
CONTENTS 5
3.6.3 The extremal characterization of relaxation time . . . 89
3.6.4 Simple applications . . . . . . . . . . . . . . . . . . . . 91
3.6.5 Quasistationarity . . . . . . . . . . . . . . . . . . . . . 95
3.7 Extremal characterizations and mean hitting times . . . . . . 98
3.7.1 Thompson’s principle and leveling networks . . . . . . 100
3.7.2 Hitting times and Thompson’s principle . . . . . . . . 102
3.8 Notes on Chapter 3 . . . . . . . . . . . . . . . . . . . . . . . . 108
4 Hitting and Convergence Time, and Flow Rate, Parameters
for Reversible Markov Chains (October 11, 1994) 113
4.1 The maximal mean commute time τ∗ . . . . . . . . . . . . . . 115
4.2 The average hitting time τ . . . . . . . . . . . . . . . . . . . 117
0
4.3 The variation threshold τ . . . . . . . . . . . . . . . . . . . . 119
1
4.3.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . 119
4.3.2 Proof of Theorem 6 . . . . . . . . . . . . . . . . . . . 123
4.3.3 τ in discrete time, and algorithmic issues . . . . . . . 126
1
4.3.4 τ and mean hitting times . . . . . . . . . . . . . . . . 128
1
4.3.5 τ and flows . . . . . . . . . . . . . . . . . . . . . . . . 130
1
4.4 The relaxation time τ . . . . . . . . . . . . . . . . . . . . . . 131
2
4.4.1 Correlations and variances for the stationary chain . . 134
4.4.2 Algorithmic issues . . . . . . . . . . . . . . . . . . . . 137
4.4.3 τ and distinguished paths . . . . . . . . . . . . . . . . 139
2
4.5 The flow parameter τ . . . . . . . . . . . . . . . . . . . . . . 142
c
4.5.1 Definition and easy inequalities . . . . . . . . . . . . . 142
4.5.2 Cheeger-type inequalities . . . . . . . . . . . . . . . . 145
4.5.3 τ and hitting times . . . . . . . . . . . . . . . . . . . 146
c
4.6 Induced and product chains . . . . . . . . . . . . . . . . . . . 148
4.6.1 Induced chains . . . . . . . . . . . . . . . . . . . . . . 148
4.6.2 Product chains . . . . . . . . . . . . . . . . . . . . . . 149
4.6.3 Efron-Stein inequalities . . . . . . . . . . . . . . . . . 152
4.6.4 Why these parameters? . . . . . . . . . . . . . . . . . 153
4.7 Notes on Chapter 4 . . . . . . . . . . . . . . . . . . . . . . . . 154
5 Examples: Special Graphs and Trees (April 23 1996) 159
5.1 One-dimensional chains . . . . . . . . . . . . . . . . . . . . . 160
5.1.1 Simple symmetric random walk on the integers . . . . 160
5.1.2 Weighted linear graphs. . . . . . . . . . . . . . . . . . 162
5.1.3 Useful examples of one-dimensional chains . . . . . . . 165
5.2 Special graphs . . . . . . . . . . . . . . . . . . . . . . . . . . 168
5.2.1 Biased walk on a balanced tree . . . . . . . . . . . . . 195
6 CONTENTS
5.3 Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
5.3.1 Parameters for trees . . . . . . . . . . . . . . . . . . . 200
5.3.2 Extremal trees . . . . . . . . . . . . . . . . . . . . . . 203
5.4 Notes on Chapter 5 . . . . . . . . . . . . . . . . . . . . . . . . 205
6 Cover Times (October 31, 1994) 207
6.1 The spanning tree argument . . . . . . . . . . . . . . . . . . . 208
6.2 Simple examples of cover times . . . . . . . . . . . . . . . . . 212
6.3 More upper bounds . . . . . . . . . . . . . . . . . . . . . . . . 214
6.3.1 Simple upper bounds for mean hitting times. . . . . . 215
6.3.2 Known and conjectured upper bounds . . . . . . . . . 216
6.4 Short-time bounds . . . . . . . . . . . . . . . . . . . . . . . . 217
6.4.1 Covering by multiple walks . . . . . . . . . . . . . . . 219
6.4.2 Bounding point probabilities . . . . . . . . . . . . . . 221
6.4.3 A cat and mouse game . . . . . . . . . . . . . . . . . . 222
6.5 Hitting time bounds and connectivity . . . . . . . . . . . . . 223
6.5.1 Edge-connectivity . . . . . . . . . . . . . . . . . . . . 224
6.5.2 Equivalence of mean cover time parameters . . . . . . 226
6.6 Lower bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
6.6.1 Matthews’ method . . . . . . . . . . . . . . . . . . . . 227
6.6.2 Balanced trees . . . . . . . . . . . . . . . . . . . . . . 227
6.6.3 A resistance lower bound . . . . . . . . . . . . . . . . 228
6.6.4 General lower bounds . . . . . . . . . . . . . . . . . . 229
6.7 Distributional aspects . . . . . . . . . . . . . . . . . . . . . . 231
6.8 Algorithmic aspects . . . . . . . . . . . . . . . . . . . . . . . 232
6.8.1 Universal traversal sequences . . . . . . . . . . . . . . 232
6.8.2 Graph connectivity algorithms . . . . . . . . . . . . . 233
6.8.3 A computational question . . . . . . . . . . . . . . . . 233
6.9 Notes on Chapter 6 . . . . . . . . . . . . . . . . . . . . . . . . 233
7 Symmetric Graphs and Chains (January 31, 1994) 237
7.1 Symmetric reversible chains . . . . . . . . . . . . . . . . . . . 238
7.1.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . 238
7.1.2 This section goes into Chapter 3 . . . . . . . . . . . . 240
7.1.3 Elementary properties . . . . . . . . . . . . . . . . . . 240
7.1.4 Hitting times . . . . . . . . . . . . . . . . . . . . . . . 241
7.1.5 Cover times . . . . . . . . . . . . . . . . . . . . . . . . 242
7.1.6 Product chains . . . . . . . . . . . . . . . . . . . . . . 246
7.1.7 The cutoff phenomenon and the upper bound lemma . 248
7.1.8 Vertex-transitive graphs and Cayley graphs . . . . . . 249
CONTENTS 7
7.1.9 Comparison arguments for eigenvalues . . . . . . . . . 252
7.2 Arc-transitivity . . . . . . . . . . . . . . . . . . . . . . . . . . 254
7.2.1 Card-shuffling examples . . . . . . . . . . . . . . . . . 255
7.2.2 Cover times for the d-dimensional torus Zd. . . . . . . 257
N
7.2.3 Bounds for the parameters . . . . . . . . . . . . . . . 259
7.2.4 Group-theory set-up . . . . . . . . . . . . . . . . . . . 259
7.3 Distance-regular graphs . . . . . . . . . . . . . . . . . . . . . 259
7.3.1 Exact formulas . . . . . . . . . . . . . . . . . . . . . . 260
7.3.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . 262
7.3.3 Monotonicity properties . . . . . . . . . . . . . . . . . 262
7.3.4 Extremal distance-regular graphs . . . . . . . . . . . . 263
7.3.5 Gelfand pairs and isotropic flights . . . . . . . . . . . 263
7.4 Notes on Chapter 7 . . . . . . . . . . . . . . . . . . . . . . . . 263
8 Advanced L2 Techniques for Bounding Mixing Times (May
19 1999) 267
8.1 The comparison method for eigenvalues . . . . . . . . . . . . 270
8.2 Improved bounds on L2 distance . . . . . . . . . . . . . . . . 278
8.2.1 Lq norms and operator norms . . . . . . . . . . . . . . 278
8.2.2 A more general bound on L2 distance . . . . . . . . . 280
8.2.3 Exact computation of N(s) . . . . . . . . . . . . . . . 284
8.3 Nash inequalities . . . . . . . . . . . . . . . . . . . . . . . . . 287
8.3.1 Nash inequalities and mixing times . . . . . . . . . . . 288
8.3.2 The comparison method for bounding N(·) . . . . . . 290
8.4 Logarithmic Sobolev inequalities . . . . . . . . . . . . . . . . 292
8.4.1 The log-Sobolev time τ . . . . . . . . . . . . . . . . . 292
l
8.4.2 τ , mixing times, and hypercontractivity . . . . . . . . 294
l
8.4.3 Exact computation of τ . . . . . . . . . . . . . . . . . 298
l
8.4.4 τ and product chains . . . . . . . . . . . . . . . . . . 302
l
8.4.5 The comparison method for bounding τ . . . . . . . . 304
l
8.5 Combining the techniques . . . . . . . . . . . . . . . . . . . . 306
8.6 Notes on Chapter 8 . . . . . . . . . . . . . . . . . . . . . . . . 307
9 A Second Look at General Markov Chains (April 21, 1995)309
9.1 Minimal constructions and mixing times . . . . . . . . . . . . 309
9.1.1 Strong stationary times . . . . . . . . . . . . . . . . . 311
9.1.2 Stopping times attaining a specified distribution . . . 312
9.1.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . 314
9.2 Markov chains and spanning trees . . . . . . . . . . . . . . . 316
9.2.1 General Chains and Directed Weighted Graphs . . . . 316
8 CONTENTS
9.2.2 Electrical network theory . . . . . . . . . . . . . . . . 319
9.3 Self-verifying algorithms for sampling from a stationary dis-
tribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
9.3.1 Exact sampling via the Markov chain tree theorem . . 322
9.3.2 Approximate sampling via coalescing paths . . . . . . 323
9.3.3 Exact sampling via backwards coupling . . . . . . . . 324
9.4 Making reversible chains from irreversible chains . . . . . . . 326
9.4.1 Mixing times . . . . . . . . . . . . . . . . . . . . . . . 326
9.4.2 Hitting times . . . . . . . . . . . . . . . . . . . . . . . 327
9.5 An example concerning eigenvalues and mixing times . . . . . 329
9.6 Miscellany . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331
9.6.1 Mixing times for irreversible chains . . . . . . . . . . . 331
9.6.2 Balanced directed graphs . . . . . . . . . . . . . . . . 331
9.6.3 An absorption time problem. . . . . . . . . . . . . . . 332
9.7 Notes on Chapter 9 . . . . . . . . . . . . . . . . . . . . . . . . 332
10 Some Graph Theory and Randomized Algorithms (Septem-
ber 1 1999) 335
10.1 Expanders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
10.1.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . 336
10.1.2 Random walk on expanders . . . . . . . . . . . . . . . 337
10.1.3 Counter-example constructions . . . . . . . . . . . . . 338
10.2 Eigenvalues and graph theory . . . . . . . . . . . . . . . . . . 339
10.2.1 Diameter of a graph . . . . . . . . . . . . . . . . . . . 339
10.2.2 Paths avoiding congestion . . . . . . . . . . . . . . . . 340
10.3 Randomized algorithms . . . . . . . . . . . . . . . . . . . . . 342
10.3.1 Background . . . . . . . . . . . . . . . . . . . . . . . . 342
10.3.2 Overviewofrandomizedalgorithmsusingrandomwalks
or Markov chains . . . . . . . . . . . . . . . . . . . . . 344
10.4 Miscellaneous graph algorithms . . . . . . . . . . . . . . . . . 344
10.4.1 Amplification of randomness . . . . . . . . . . . . . . 344
10.4.2 Using random walk to define an objective function . . 346
10.4.3 Embedding trees into the d-cube . . . . . . . . . . . . 347
10.4.4 Comparing on-line and off-line algorithms . . . . . . . 349
10.5 Approximate counting via Markov chains . . . . . . . . . . . 351
10.5.1 Volume of a convex set. . . . . . . . . . . . . . . . . . 353
10.5.2 Matchings in a graph . . . . . . . . . . . . . . . . . . 353
10.5.3 Simulating self-avoiding walks . . . . . . . . . . . . . . 354
10.6 Notes on Chapter 9 . . . . . . . . . . . . . . . . . . . . . . . . 355
10.7 Material belonging in other chapters . . . . . . . . . . . . . . 358
CONTENTS 9
10.7.1 Large deviation bounds . . . . . . . . . . . . . . . . . 358
10.7.2 The probabilistic method in combinatorics . . . . . . . 358
10.7.3 copied to Chapter 4 section 6.5 . . . . . . . . . . . . . 358
11 Markov Chain Monte Carlo (January 8 2001) 361
11.1 Overview of Applied MCMC . . . . . . . . . . . . . . . . . . 361
11.1.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 361
11.1.2 Further aspects of applied MCMC . . . . . . . . . . . 366
11.2 The two basic schemes . . . . . . . . . . . . . . . . . . . . . . 369
11.2.1 Metropolis schemes . . . . . . . . . . . . . . . . . . . . 369
11.2.2 Line-sampling schemes . . . . . . . . . . . . . . . . . . 370
11.3 Variants of basic MCMC . . . . . . . . . . . . . . . . . . . . . 371
11.3.1 Metropolized line sampling . . . . . . . . . . . . . . . 371
11.3.2 Multiple-try Metropolis . . . . . . . . . . . . . . . . . 372
11.3.3 Multilevel sampling . . . . . . . . . . . . . . . . . . . 373
11.3.4 Multiparticle MCMC. . . . . . . . . . . . . . . . . . . 375
11.4 A little theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
11.4.1 Comparison methods . . . . . . . . . . . . . . . . . . . 376
11.4.2 Metropolis with independent proposals . . . . . . . . . 377
11.5 Thediffusionheuristicforoptimalscalingofhighdimensional
Metropolis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378
11.5.1 Optimal scaling for high-dimensional product distri-
bution sampling . . . . . . . . . . . . . . . . . . . . . 378
11.5.2 The diffusion heuristic.. . . . . . . . . . . . . . . . . . 380
11.5.3 Sketch proof of Theorem . . . . . . . . . . . . . . . . . 381
11.6 Other theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 382
11.6.1 Sampling from log-concave densities . . . . . . . . . . 382
11.6.2 Combining MCMC with slow exact sampling . . . . . 383
11.7 Notes on Chapter MCMC . . . . . . . . . . . . . . . . . . . . 383
11.8 Belongs in other chapters . . . . . . . . . . . . . . . . . . . . 385
11.8.1 Pointwise ordered transition matrices . . . . . . . . . 385
12 Coupling Theory and Examples (October 11, 1999) 387
12.1 Using coupling to bound variation distance . . . . . . . . . . 387
12.1.1 The coupling inequality . . . . . . . . . . . . . . . . . 388
12.1.2 Comments on coupling methodology . . . . . . . . . . 388
12.1.3 Random walk on a dense regular graph . . . . . . . . 390
12.1.4 Continuous-time random walk on the d-cube . . . . . 391
12.1.5 The graph-coloring chain . . . . . . . . . . . . . . . . 392
12.1.6 Permutations and words . . . . . . . . . . . . . . . . . 393
10 CONTENTS
12.1.7 Card-shuffling by random transpositions . . . . . . . . 395
12.1.8 Reflection coupling on the n-cycle . . . . . . . . . . . 396
12.1.9 Card-shuffling by random adjacent transpositions . . . 397
12.1.10Independent sets . . . . . . . . . . . . . . . . . . . . . 398
12.1.11Two base chains for genetic algorithms . . . . . . . . . 400
12.1.12Path coupling . . . . . . . . . . . . . . . . . . . . . . . 402
12.1.13Extensions of a partial order . . . . . . . . . . . . . . 404
12.2 Notes on Chapter 4-3 . . . . . . . . . . . . . . . . . . . . . . 405
13 Continuous State, Infinite State and Random Environment
(June 23, 2001) 409
13.1 Continuous state space . . . . . . . . . . . . . . . . . . . . . . 409
13.1.1 One-dimensional Brownian motion and variants . . . . 409
13.1.2 d-dimensional Brownian motion . . . . . . . . . . . . . 413
13.1.3 Brownian motion in a convex set . . . . . . . . . . . . 413
13.1.4 Discrete-time chains: an example on the simplex . . . 416
13.1.5 Compact groups . . . . . . . . . . . . . . . . . . . . . 419
13.1.6 Brownian motion on a fractal set . . . . . . . . . . . . 420
13.2 Infinite graphs . . . . . . . . . . . . . . . . . . . . . . . . . . 421
13.2.1 Set-up . . . . . . . . . . . . . . . . . . . . . . . . . . . 422
13.2.2 Recurrence and Transience . . . . . . . . . . . . . . . 423
13.2.3 The finite analog of transience . . . . . . . . . . . . . 425
13.2.4 Random walk on Zd . . . . . . . . . . . . . . . . . . . 425
13.2.5 The torus Zd . . . . . . . . . . . . . . . . . . . . . . . 427
m
13.2.6 The infinite degree-r tree . . . . . . . . . . . . . . . . 431
13.2.7 Generating function arguments . . . . . . . . . . . . . 432
13.2.8 Comparison arguments. . . . . . . . . . . . . . . . . . 433
13.2.9 The hierarchical tree . . . . . . . . . . . . . . . . . . . 435
13.2.10Towards a classification theory for sequences of finite
chains . . . . . . . . . . . . . . . . . . . . . . . . . . . 437
13.3 Random Walks in Random Environments . . . . . . . . . . . 441
13.3.1 Mixing times for some random regular graphs . . . . . 441
13.3.2 Randomizing infinite trees . . . . . . . . . . . . . . . . 444
13.3.3 Bias and speed . . . . . . . . . . . . . . . . . . . . . . 446
13.3.4 Finite random trees . . . . . . . . . . . . . . . . . . . 447
13.3.5 Randomly-weighted random graphs. . . . . . . . . . . 449
13.3.6 Random environments in d dimensions . . . . . . . . . 450
13.4 Notes on Chapter 13 . . . . . . . . . . . . . . . . . . . . . . . 451