Table Of ContentA L G O R I T H M
D E S I G N
, ,
Foundations Analysis
and Internet Examples
M I C H A E L T. G O O D R I C H I R O B E R T O T A M A S S I A
ORITHM DESIGN
idatiolis, Analysis and Internet Examples
nd Edition
jright © 1999, 2000, 2003, 2004, 2005, 2006, byJohn Wiley & Sons Inc. All
s reserved
orized. reprint by Wiley India (P.) Ltd., 4435/7, Ansari .Road, Daryagan], New
1110002.
ghts reserved. AUTHORIZED REPRINT OF THE EDITION PUBLISHED BY
N WILEY & SONS, INC., U.K No part of this book may be reproduced in
form without the written permission of the publisher.
ntennial logo designer: Richard J. Pacifico
publishr and the author make no
Ls of Liability/Disclaimer of Warranty: The
esentation or warranties with respect to the accuracy orcompleteness of thé
ents ofthis work and specifically diâclaim all wàrranties, including without
ation warranties of fitness for a particular purpose. No warranty may be
ted or extended by sales or promotional materials. The advice and strategies
ained herein may not be suitable for everysituation. This work is sold with
understanding that the publisher is not engaged in rendering legal,
)unting, or other professiohal services. If professional assistance is required,
services of a competent professional person should be sought. Neither the
isher nor the author shall be liable for damages arising herefrom. The fact
an organization or Website if referred to in this work as a citation and/or a
ntial source of further information does not mean that the author or the
lisher endorses the information the organization or Website may provide or
mmendations it may make. Further, readers should be aware that Internet
)sites listed in this work may have changed or disappeared between when
work was written and when it is read.
i
y also publishes its books in a variety of electronic formats. Some content
--appéars in print may not be avail'able in electronic books. For more
rmation abOut wiley products, visit our website at www.wiley.com.
Drint: 2011
nted at: Ashim Printline, Sahibabad
3N : 978-81-265-0986-7
Tb my ¿hiidren, Paul, Anna, and Jack
-
Michael T Goodrich.
To Isabel
-
Ròberto Tamassia
Preface
This book is designed to provide a comprehensive introduction to the design and
analysis of computer algorithms and data structures. In terms M the computer sci-
ence and computer engineering curricula, we have written this book to be primarily
focused on the Junior-Senior level Algorithms (CS7) course, which is taught as a
first-year graduate course in some schools.
Topics
The topics covered in this book are taken from a broad spectrum of discrete algo-
rithm design and analysis, including the following:
Design and analysis of algorithms, including asymptotic notation; worst-
case analysis, amortization, randomization and experimental analysis
Algorithmic design patterns, including greedy method, divide-arid-conquer,
dynamic programming, backtracking and branch-and-bound
Algorithmic frameworks, including NP-completeness, approximation algo-
rithms, on-line algorithms, external-memory algorithms, distributed algo-
rithms, and parallel algorithms
Data structures, including lists, vectors, trees, priority queues, AYL trees, 2-
4 trees, red-black trees, splay trees, B-trees, hash tables, skip-lists, union-find
trees
Combinatorial algorithms, including heap-sort, quick-sort, merge-sort, se-
lection, parallel list ranking, parallel sorting
* Graph algorithms, including traversals (DFS and BFS), topological sorting,
shortest paths (all-pairs and single-source), minimum spanning tree, maxi-
mum flow, minimum-cost flow, and matching
Geometric algorithms, including range searching, convex hulls, segment in-
tersection, and closest pairs
Numerical algorithms, including integér, matrix, and polynomial multipli-
cation, the Fast Fourier Transform (FF1'), extended Euclid's algorithm, mod-
ular exponentiation, and primality testing
Internet algorithms, including packet routing, multicasting, leader election,
encryption, digital signatures, text pattern matching, information retrieval,
data compression, Web caching, and Web auctions
Preface
About the Authors
Professors Öoodrich and Tamassia are well-recognized researchers in data struc-
tures and àlgorithms, having published many papers in this field, with applications
to Internet computing, information visualization, geographic information systems,
and computer security. They have an extensive record of research collaboration and
have served as principal investigators in several joint projects sponsored by the Na-
tional Science Foundation, the Army Research Office, and:the Defense Advanced
Research Projects Agency. They are also active in educational technology research,
with special emphasis on algorithm visualization systems and infrastructure support
for distance learning.
Michael Goodrich received his Ph.D. in Computer Science' from Purdue Uni-
versity in 1987. He is currently a professor in the Dçpartment of Information and
Computer Science at University of California, Irvine. Prior to this service we was
Professor of Compùter Science at Jorns Hopkins University, and director of the
Hopkins Center for Algorithm Engineering. He is an editor for the International
Journal of Computational Geometry & Applications, Journal of Computational and
System Sciences, and Journal of Graph Algorithms and Applications.
Roberto Tamassia received his Ph.D. in Electrical and Computer Engineering
froth the University of Illinois at Urbana-Champaign in 1988. He is currently a
professOr in the Department of Computer Science and the director of the Center
for Geometric Computing at Brown University. He is an editor for Computational
Geometry: Thçory and. Applications and the Journal of Graph Algorithms and Ap-
plications, and he has previously served on the editonal board of IEEE Transactions
on Computers.
In addition to their research accomplishments, the authors also have extensive
experience in the classroom. For example, Dr. Goodrich has taught data struc-
tures and algorithms courses since 1987, including Data Structures as a freshman-
sophomore level coursé and Introduction to Algorithms as a upper level coùrse.
He has earned several teaching awards in this capacity. His teaching style is to
involve the students in lively interactive classroom sessions that bring out the in-
tuition and insights behind data structuring and algorithmic techniques, as well as
in formulating solutions whose analyis is mathematically rigorous; Dr. Tamas-
sia has taught Data Structures and Algorithms as an introduètory freshman-level
course since 1988. He has also attracted, many students (including several under-
graduates) to his advanced course on Computational Geometry, which is à popular
graduate-level CS course at Brown. One thing that has set his teaching style. 'apart
is his effective use of interactive hypennedia presentations, continUing the tradi
tion of.Brown's "electronic classçoom?' The carefully designed Web pages of the
courses taught by Dr. Tamassia have bçen used as reference material by students
and professionals worldwide.
Preface
For the Instructor
Algorithms (CS7)
primarily as a textbook for a Junior-Senior
This book is intended
. graduate course. in some schools. This
course, which is also taught as a first-year
reinforcemént exercises,
which are divided between
book containsmany exercises,
Certain aspects of this book were
creativity exercises, and implementation projects.
specifically designed with theinstructor in mind, including:
make mathçmatical ar-
is, picture proofs), which
Visual justifications (that
appealing to visual learners. An
guments more understandable for students,
bottom-up heap constfuc-
is. our analysis of
example of visual justifications
for students to understand;
tion. This topic has traditionally been difficult
explain. The included visual proof
for instructors to
hence, time coñsuming
is intuitive, ngorous, and quick
techniques for design-
Algorithmic design patterns, which provide general
divide-and-conquer, dy-
algonthms Examples include
ing and implementing
namic programming, the decorator pattern, and the template method pattern
in an al-
ofrandomization, which takes advantage of random choices
Use
and analysis. Such usage replaces complex
gorithin to simplify its dsign
with intuitive analy-
of sophisticated data structures
average-case analysis
incluçie skip lists,
and algorithths. Examples
sis of simple data structures
quick-select, and randomized primality
randomized uick-sort, randomized
testing.
traditional algonthniic
which either motivate
Internet algorithmics topics,
algorithms that are
viewpoint or highlight new
topics from a new Intérnet.
Examples include information retrieval,
derived from Internet applications.
algorithms, and Web caching
routing, Web auction
Web crawling, päcket
algorithms. We have found that motivating algorithms topics by their Inter-
interest in. the study of algo-
significantly improves, student
net applications
rithms.
.
design methods, object-
examples, which cover software
Java implementation
of algorithms.,
and experimental analysis
oriented implementation issues,
sections of various
examples, provided in separate
These implementation.
insíructors can either cover them in their. lec-
chapters, are Optional, so that
additional reading, or skip them altogether.
tures, assign them. as
N
allow the instructor a great deal of freedom in
This book is alsd structured to
the dependence between chap-
how to organize and present the material. Likewise,
customize an algorithms course to
allowing the instructor to
ters is rather flexible,
highlight the topics that he or she feels are most important. We .have extensively
quite interesting to stu-
topics, which should prove
disèussed Internet Algorithmics
application of traditional
inclúded examples of Internet
dents. In addition, we have
algorithms topics in several nlaces as well.
Preface
could be used for a traditional Introduc-
0.1 how this book
We show in Table
motivated from the
with some new topics
(CS7) course, albeit
tion to Algorithms
Internet.
Option
Topics
Ch.
Experimental analysis -
Algorithm analysis
Heap Java example
Dáta structures
Include one of § 3.2-3.5
Searching
3
In-place quick-sort
Sorting
The IPT
Algorithmic techniques
5
DFS Java example
Graph algorithms
Dijkstra Java example
Weighted graphs
Include at end of course
Matching and flow
Tries
Text processing (at least one section)
9
Include at end of course
computational geometry
.12
Backtracking
NP-completeness
13
Include at end of course
Frameworks (at least one)
14
traditional Introduction to Algorithms
syllabus sche4ule for a
Table 0.1: Exámple.
for each chapter.
(CS7) course, including optional choices.
Algorithmics course,
'be used for a specialized Internet
This book can also.
topics, but in .a new internet-motivated
traditional àlgorithms
which r views some
derived from Internet ap-
algorithmic topics that are
light, while also covering new
book could be used for a such a course.
plications. We show in Table 0.2 how this
Option
Ch. Topics
Experimental analysis
i Algorithm analysis
Quickly review
2 Data structures (inc. hashing)
lists) Search tree Java example
Searching (inc. § 3.5, skip
In-place quick-sort
Sorting
The WI
Algorithmic techniques
5
DFS Java. example
Graph algorithms
6
Skip one MST alg.
Weighted graphs
Matching algorithms
Matching and flowS
Pattern matching
Text processing
Java examples.
Security & cryptography
19
Multi-cásting
Network algorithms
11
include at end of course
NP-completeness
13
Include at end of course
Frameworks (at least two)
14
includ-
Internet AlgorithmiCs course,
syllabus schedule for an
Table 0.2: Example
for each chapter.
ing optional choices
.
Preface
Of course, other optiòns are also possible, including a course that is a mixture of
a traditional Introduction to Algorithms (CS7) course and an Internet Algorithmics
course. We do not belabor this point, however, leaving such creative arrangements
to the interested instructor.
Web Added-Value Education
This book comes accompanied by an extensive Web site:
http://www.wiley.com/college/goodrichi
This Web site includes an extensive collection of educational aids that augment the
topics of this book. Specifically for students we include:
Presentation handouts (four-per-page format) for most topics in this book
A database of hints on selected assignments, indexed by problem number
Interactive applets that animate fundamentl data structures and algorithms
.. Source code for the Java examples in this book
We feel that the hint server should be of particular iñterest, particularly for èreativity
problems that can be quite challenging for some students.
For instructors i.ising this book, there is a dedicated portion of the Web site just
for them, which includes the following additional teaching aids
Solutions to seleòted exercises in this book
A database of additional exercises and their solutions
Presentations (one-per-page format) for most topics covered in this book
Readers interested in the implementation of algorithms and data structures can
download JDSL the Data Structures Library in Java, from
http://www.jdsl.org/
Prerequisites
Wehave written this book assuming that the reader comes to it with cértain knowl-
edge In particular, we assume that the reader has a basic understanding of elemen-
tary data structures, such as arrays and linked lists, and is at least vaguely familiar
with a high-level programming language, such as C, C++, or .Java. Even so, all
álgorithms are described in a high-level "pseudo-code," and specific programming
langùage ëonstructs are only used in the optional lava implementation example
sections.
In terms of mathematical background, we assume the reader is familiar with
topiÇs from first-year college mathematics, including exponents, logarithms, sum-
mations, limits, and elementary probability. Even so, we review most of these
fads in Chapter 1, iñcluding exponénts, logarithms, and summations, and we give
¡
t .suthmary of other useful mathematical facts, including elementary probability, in
Appendix A.
i
Fundamental Tools
I
I Algorithm Analysis
3
1.1 Methodologies for Analyzing Algorithms 5
.
1.2 Asymptotic Notation 13
1.3 AQuick Mathematical Review
21
1.4 Case Studies in Algorithm Analysis.
.. 31
1.5 Amortization 34
. ..
1.6 Experimentation 42
1.7 Exercises 47
.
2 Basic Data Structurés 55
2.1 Stacks and Queues 57
2.2 Vectors Lists, and Sequences 65
2.3 Trees 75
. .
2.4 Priority Queues and Heaps 94
2.5 Dictionaries and Hash Tables 114
2.6 Java Example: Heap 128
.
2.7 Exercises 131
.
3 Search Trees and Skip Lists 139
3.1 Ordered Dictionaries and Binary Search Trees 141
3.2 AVL Trees 152
.
3.3 Bounded-Depth Search Trees 159
3.4 Splay Trees. 185
3.5 Skip Lists 195
.
3.6 Java Example: AVL and Red-Black Trees 202
3.7 Exercises 212
. .
4 Sorting, Sets, and Selection 217
4.1 Merge-Sôrt 219
.
4.2 The Set Abstract Data Type 225
.
4.3 Quick-Sort 235
. .
4.4 A Lower Bound on Comparison-Based Sorting 239
4.5 Bucket-Sort and Radix-Sort 241
.
Comparison of Sorting Algorithms ..........
4.6 244
. ., .
4.7 Selection 245
.
4.8 Java Example: In-Place Quick-Sort 248
. .
49 Exercises . 251
Contents
257
5 Fundamental Techniques
259
5.1 The Greedy Method
5.2 Divide-and-Conquer 263
5.3 Dynamk Programming 274
282
5.4 Exercises
285
Graph Algorithms
6 Graphs 287
6.1 The Graph Abstract Data Type 289
6.2 Data Structures for Graphs 296
303
6.3 Graph Traversal
316
6.4 Directed. Graphs
6.5 Java Example: Depth-First Search 329
335
6.6 Exercises
7 Weighted Graphs .339
7.1 Single-Source Shortest Paths 341
7.2 Ail-Pairs Shortest Paths 354
Minimum Spanning Trees ......... 360
7.3
.
7.4 Java Example: Dijkstra's Algorithm 373
376
7.5 Exercises
8 Network Flow and Matching 381
.
8.1 Flows and Cuts 383
8.2 Maximum Flow 387
8.3 Maximum Bipartite Matching . 396
8.4 Minimum-Cost Flow: 398
. .
8.5 Java Example: Minimum-Cost Flow 1.05
/ .
/ 412
8.6 Exercises
.
III Internet Atgorithmics 415
9 Text Processing 417
9.1 Strings and Pattern Matching Algorithms 419
9.2: Tries 429
«o
9.3 Text Compression
.
9.4 Text Similarity Testing 443
9.5 Exercises .
. .
10 Number Theory and Cryptography 451
10.1 Fundamental Algorithms Involving Numbers 453
10.2 Cryptographic Cômputations 471
10.3 Information Security Algorithms and Protocols 481
488
10.4 The Fast Fourier Transform
.
10.5 Java Example: FFT 500
.
508
10.6 Exercises
