Graph Algorithms in the Language of Linear Algebra

373 Pages·2011·5.06 MB·English

by  Jeremy Kepner and John Gilbert

Graph Algorithms in the
Language of Linear Algebra
SE22_Kepner_FM-04D-2o8w-1n1lo.ainddedd 019 Dec 2011 to 129.174.55.245. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php5/12/2011 10:14:49 AM
The SIAM series on Software, Environments, and Tools focuses on the practical implementation of
computational methods and the high performance aspects of scientific computation by emphasizing
in-demand software, computing environments, and tools for computing. Software technology development
issues such as current status, applications and algorithms, mathematical software, software tools, languages
and compilers, computing environments, and visualization are presented.
Editor-in-Chief
Jack J. Dongarra
University of Tennessee and Oak Ridge National Laboratory
Editorial Board
James W. Demmel, University of California, Berkeley
Dennis Gannon, Indiana University
Eric Grosse, AT&T Bell Laboratories
Jorge J. Moré, Argonne National Laboratory
Software, Environments, and Tools
Jeremy Kepner and John Gilbert, editors, Graph Algorithms in the Language of Linear Algebra
Jeremy Kepner, Parallel MATLAB for Multicore and Multinode Computers
Michael A. Heroux, Padma Raghavan, and Horst D. Simon, editors, Parallel Processing for Scientific Computing
Gérard Meurant, The Lanczos and Conjugate Gradient Algorithms: From Theory to Finite Precision Computations
Bo Einarsson, editor, Accuracy and Reliability in Scientific Computing
Michael W. Berry and Murray Browne, Understanding Search Engines: Mathematical Modeling and Text
Retrieval, Second Edition
Craig C. Douglas, Gundolf Haase, and Ulrich Langer, A Tutorial on Elliptic PDE Solvers and Their Parallelization
Louis Komzsik, The Lanczos Method: Evolution and Application
Bard Ermentrout, Simulating, Analyzing, and Animating Dynamical Systems: A Guide to XP´PAUT for Researchers
and Students
V. A. Barker, L. S. Blackford, J. Dongarra, J. Du Croz, S. Hammarling, M. Marinova, J. Wasniewski, and
P. Yalamov, LAPACK95 Users’ Guide
Stefan Goedecker and Adolfy Hoisie, Performance Optimization of Numerically Intensive Codes
Zhaojun Bai, James Demmel, Jack Dongarra, Axel Ruhe, and Henk van der Vorst, Templates for the Solution
of Algebraic Eigenvalue Problems: A Practical Guide
Lloyd N. Trefethen, Spectral Methods in MATLAB
E. Anderson, Z. Bai, C. Bischof, S. Blackford, J. Demmel, J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling,
A. McKenney, and D. Sorensen, LAPACK Users’ Guide, Third Edition
Michael W. Berry and Murray Browne, Understanding Search Engines: Mathematical Modeling and Text Retrieval
Jack J. Dongarra, Iain S. Duff, Danny C. Sorensen, and Henk A. van der Vorst, Numerical Linear Algebra
for High-Performance Computers
R. B. Lehoucq, D. C. Sorensen, and C. Yang, ARPACK Users’ Guide: Solution of Large-Scale Eigenvalue Problems
with Implicitly Restarted Arnoldi Methods
Randolph E. Bank, PLTMG: A Software Package for Solving Elliptic Partial Differential Equations, Users’ Guide 8.0
L. S. Blackford, J. Choi, A. Cleary, E. D’Azevedo, J. Demmel, I. Dhillon, J. Dongarra, S. Hammarling,
G. Henry, A. Petitet, K. Stanley, D. Walker, and R. C. Whaley, ScaLAPACK Users’ Guide
Greg Astfalk, editor, Applications on Advanced Architecture Computers
Roger W. Hockney, The Science of Computer Benchmarking
Françoise Chaitin-Chatelin and Valérie Frayssé, Lectures on Finite Precision Computations
SE22_Kepner_FM-04D-2o8w-1n1lo.ainddedd 029 Dec 2011 to 129.174.55.245. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php5/12/2011 10:14:49 AM
Graph Algorithms in the
Language of Linear Algebra
Edited by
Jeremy Kepner
MIT Lincoln Laboratories
Lexington, Massachusetts
John Gilbert
University of California at Santa Barbara
Santa Barbara, California
Society for Industrial and Applied Mathematics
Philadelphia
SE22_Kepner_FM-04D-2o8w-1n1lo.ainddedd 039 Dec 2011 to 129.174.55.245. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php5/12/2011 10:14:49 AM
Copyright © 2011 by the Society for Industrial and Applied Mathematics
10 9 8 7 6 5 4 3 2 1
All rights reserved. Printed in the United States of America. No part of this book may be reproduced,
stored, or transmitted in any manner without the written permission of the publisher. For information,
write to the Society for Industrial and Applied Mathematics, 3600 Market Street, 6th Floor, Philadelphia,
PA 19104-2688 USA.
Trademarked names may be used in this book without the inclusion of a trademark symbol. These
names are used in an editorial context only; no infringement of trademark is intended.
MATLAB is a registered trademark of The MathWorks, Inc. For MATLAB product information, please
contact The MathWorks, Inc., 3 Apple Hill Drive, Natick, MA 01760-2098 USA, 508-647-7000,
Fax: [phone], [email protected], www.mathworks.com.
This work is sponsored by the Department of the Air Force under Air Force Contract FA8721-
05-C-0002. Opinions, interpretations, conclusions, and recommendations are those of the authors
and are not necessarily endorsed by the United States Government.

Library of Congress Cataloging-in-Publication Data
Kepner, Jeremy V., 1969-
Graph algorithms in the language of linear algebra / Jeremy Kepner, John Gilbert.
p. cm. -- (Software, environments, and tools)
Includes bibliographical references and index.
ISBN 978-0-898719-90-1
1. Graph algorithms. 2. Algebras, Linear. I. Gilbert, J. R. (John R.), 1953- II. Title.
QA166.245.K47 2011
511’.6--dc22 2011003774
is a registered trademark.
SE22_Kepner_FM-04D-2o8w-1n1lo.ainddedd 049 Dec 2011 to 129.174.55.245. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php5/12/2011 10:14:49 AM
for
Dennis Healy
whose vision
allowed us all
to see further
SE22_Kepner_FM-04D-2o8w-1n1lo.ainddedd 059 Dec 2011 to 129.174.55.245. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php5/12/2011 10:14:50 AM
List of Contributors
David A. Bader John Gilbert Sanjeev Mohindra
College of Computing Computer Science Department MIT Lincoln Laboratory
Georgia Institute of Technology University of California at Santa 244 Wood Street
Atlanta, GA 30332 Barbara Lexington, MA 02420
[email protected] Santa Barbara, CA 93106 [email protected]
[email protected]
Nadya Bliss Huy Nguyen
MIT Lincoln Laboratory Christine E. Heitsch MIT Computer Science and
244 Wood Street School of Mathematics Artificial Intelligence Laboratory
Lexington, MA 02420 Georgia Institute of Technology (CSAIL)
[email protected] Atlanta, GA 30332 32 Vassar Street
[email protected] Cambridge, MA 02139
Robert Bond [email protected]
MIT Lincoln Laboratory Bruce Hendrickson
244 Wood Street Discrete Algorithms and Charles M. Rader
Lexington, MA 02420 Mathematics Department MIT Lincoln Laboratory
[email protected] Sandia National Laboratories 244 Wood Street
Albuquerque, NM 87185 Lexington, MA 02420
Aydın Buluç [email protected] [email protected]
High Performance Computing
Research W. Philip Kegelmeyer Steve Reinhardt
Lawrence Berkeley National Informatics and Decision Sciences Microsoft Corporation
Laboratory Department 716 Bridle Ridge Road
1 Cyclotron Road Sandia National Laboratories Eagan, MN 55123
Berkeley, CA 94720 Livermore, CA 94551 [email protected]
[email protected] [email protected]
Eric Robinson
Daniel M. Dunlavy Jeremy Kepner MIT Lincoln Laboratory
Computer Science and Informatics MIT Lincoln Laboratory 244 Wood Street
Department 244 Wood Street Lexington, MA 02420
Sandia National Laboratories Lexington, MA 02420 [email protected]
Albuquerque, NM 87185 [email protected]
[email protected] Viral B. Shah
Tamara G. Kolda 82 E Marine Drive
Alan Edelman Informatics and Decision Sciences Badrikeshwar, Flat No. 25
Mathematics Department Department Mumbai 400 002
MIT Sandia National Laboratories India
77 Massachusetts Avenue Livermore, CA 94551 [email protected]
Cambridge, MA 02139 [email protected]
[email protected]
Jure Leskovec
Christos Faloutsos Computer Science Department
School of Computer Science Stanford University
Carnegie Mellon University Stanford, CA 94305
Pittsburgh, PA 15213 [email protected]
[email protected]
Kamesh Madduri
Jeremy T. Fineman Computational Research Division
School of Computer Science Lawrence Berkeley National
Carnegie Mellon University Laboratory
Pittsburgh, PA 15213 Berkeley, CA 94720
[email protected] [email protected]
vii
SE22_Kepner_FM-04D-2o8w-1n1lo.ainddedd 069 Dec 2011 to 129.174.55.245. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php5/12/2011 10:14:50 AM
Contents
List of Figures xvii
List of Tables xxi
List of Algorithms xxiii
Preface xxv
Acknowledgments xxvii
I Algorithms 1
1 Graphs and Matrices 3
J. Kepner
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2.1 Graph adjacency matrix duality . . . . . . . . . . . . . . 4
1.2.2 Graph algorithms as semirings . . . . . . . . . . . . . . . 5
1.2.3 Tensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3.1 Simulating power law graphs . . . . . . . . . . . . . . . . 6
1.3.2 Kronecker theory . . . . . . . . . . . . . . . . . . . . . . . 7
1.4 Computation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.4.1 Graph analysis metrics . . . . . . . . . . . . . . . . . . . . 7
1.4.2 Sparse matrix storage . . . . . . . . . . . . . . . . . . . . 8
1.4.3 Sparse matrix multiply. . . . . . . . . . . . . . . . . . . . 9
1.4.4 Parallel programming . . . . . . . . . . . . . . . . . . . . 9
1.4.5 Parallel matrix multiply performance. . . . . . . . . . . . 10
1.5 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2 Linear Algebraic Notation and Definitions 13
E. Robinson, J. Kepner, and J. Gilbert
2.1 Graph notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
ix
Downloaded 09 Dec 2011 to 129.174.55.245. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
x Contents
2.2 Array notation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3 Algebraic notation . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3.1 Semirings and related structures . . . . . . . . . . . . . . 14
2.3.2 Scalar operations . . . . . . . . . . . . . . . . . . . . . . . 15
2.3.3 Vector operations . . . . . . . . . . . . . . . . . . . . . . . 15
2.3.4 Matrix operations . . . . . . . . . . . . . . . . . . . . . . 16
2.4 Array storage and decomposition . . . . . . . . . . . . . . . . . . 16
2.4.1 Sparse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4.2 Parallel . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3 Connected Components and Minimum Paths 19
C. M. Rader
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.2 Strongly connected components . . . . . . . . . . . . . . . . . . . 20
3.2.1 Nondirected links . . . . . . . . . . . . . . . . . . . . . . . 21
3.2.2 Computing C quickly . . . . . . . . . . . . . . . . . . . . 22
3.3 Dynamic programming, minimum paths, and matrix exponentia-
tion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.3.1 Matrix powers . . . . . . . . . . . . . . . . . . . . . . . . 25
3.4 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4 Some Graph Algorithms in an Array-Based Language 29
V. B. Shah, J. Gilbert, and S. Reinhardt
4.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.2 Sparse matrices and graphs . . . . . . . . . . . . . . . . . . . . . 30
4.2.1 Sparse matrix multiplication . . . . . . . . . . . . . . . . 31
4.3 Graph algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.3.1 Breadth-first search . . . . . . . . . . . . . . . . . . . . . 32
4.3.2 Strongly connected components . . . . . . . . . . . . . . . 33
4.3.3 Connected components. . . . . . . . . . . . . . . . . . . . 34
4.3.4 Maximal independent set . . . . . . . . . . . . . . . . . . 35
4.3.5 Graph contraction . . . . . . . . . . . . . . . . . . . . . . 35
4.3.6 Graph partitioning . . . . . . . . . . . . . . . . . . . . . . 37
4.4 Graph generators . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.4.1 Uniform random graphs . . . . . . . . . . . . . . . . . . . 39
4.4.2 Power law graphs . . . . . . . . . . . . . . . . . . . . . . . 39
4.4.3 Regular geometric grids . . . . . . . . . . . . . . . . . . . 39
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
5 Fundamental Graph Algorithms 45
J. T. Fineman and E. Robinson
5.1 Shortest paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
5.1.1 Bellman–Ford . . . . . . . . . . . . . . . . . . . . . . . . . 46
5.1.2 Computing the shortest path tree (for Bellman–Ford) . . 48
5.1.3 Floyd–Warshall . . . . . . . . . . . . . . . . . . . . . . . . 53
Downloaded 09 Dec 2011 to 129.174.55.245. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
Contents xi
5.2 Minimum spanning tree. . . . . . . . . . . . . . . . . . . . . . . . 55
5.2.1 Prim’s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
6 Complex Graph Algorithms 59
E. Robinson
6.1 Graph clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
6.1.1 Peer pressure clustering . . . . . . . . . . . . . . . . . . . 59
6.1.2 Matrix formulation . . . . . . . . . . . . . . . . . . . . . . 66
6.1.3 Other approaches . . . . . . . . . . . . . . . . . . . . . . . 67
6.2 Vertex betweenness centrality . . . . . . . . . . . . . . . . . . . . 68
6.2.1 History . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
6.2.2 Brandes’ algorithm . . . . . . . . . . . . . . . . . . . . . . 69
6.2.3 Batch algorithm . . . . . . . . . . . . . . . . . . . . . . . 75
6.2.4 Algorithm for weighted graphs . . . . . . . . . . . . . . . 78
6.3 Edge betweenness centrality . . . . . . . . . . . . . . . . . . . . . 78
6.3.1 Brandes’ algorithm . . . . . . . . . . . . . . . . . . . . . . 78
6.3.2 Block algorithm. . . . . . . . . . . . . . . . . . . . . . . . 83
6.3.3 Algorithm for weighted graphs . . . . . . . . . . . . . . . 84
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
7 Multilinear Algebra for Analyzing Data with Multiple Linkages 85
D. Dunlavy, T. Kolda, and W. P. Kegelmeyer
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
7.2 Tensors and the CANDECOMP/PARAFAC decomposition. . . . 87
7.2.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
7.2.2 Vector and matrix preliminaries. . . . . . . . . . . . . . . 88
7.2.3 Tensor preliminaries . . . . . . . . . . . . . . . . . . . . . 88
7.2.4 The CP tensor decomposition . . . . . . . . . . . . . . . . 89
7.2.5 CP-ALS algorithm . . . . . . . . . . . . . . . . . . . . . . 89
7.3 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
7.3.1 Data as a tensor . . . . . . . . . . . . . . . . . . . . . . . 91
7.3.2 Quantitative measurements on the data . . . . . . . . . . 93
7.4 Numerical results . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
7.4.1 Community identification . . . . . . . . . . . . . . . . . . 94
7.4.2 Latent document similarity . . . . . . . . . . . . . . . . . 95
7.4.3 Analyzing a body of work via centroids . . . . . . . . . . 97
7.4.4 Author disambiguation. . . . . . . . . . . . . . . . . . . . 98
7.4.5 Journal prediction via ensembles of tree classifiers . . . . 103
7.5 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
7.5.1 Analysis of publication data . . . . . . . . . . . . . . . . . 106
7.5.2 Higher order analysis in data mining . . . . . . . . . . . . 107
7.5.3 Other related work . . . . . . . . . . . . . . . . . . . . . . 108
7.6 Conclusions and future work . . . . . . . . . . . . . . . . . . . . . 108
7.7 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
Downloaded 09 Dec 2011 to 129.174.55.245. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php