Table Of Content

A L G O R I T H M D E S I G N , , Foundations Analysis and Internet Examples M I C H A E L T. G O O D R I C H I R O B E R T O T A M A S S I A ORITHM DESIGN idatiolis, Analysis and Internet Examples nd Edition jright © 1999, 2000, 2003, 2004, 2005, 2006, byJohn Wiley & Sons Inc. All s reserved orized. reprint by Wiley India (P.) Ltd., 4435/7, Ansari .Road, Daryagan], New 1110002. ghts reserved. AUTHORIZED REPRINT OF THE EDITION PUBLISHED BY N WILEY & SONS, INC., U.K No part of this book may be reproduced in form without the written permission of the publisher. ntennial logo designer: Richard J. Pacifico publishr and the author make no Ls of Liability/Disclaimer of Warranty: The esentation or warranties with respect to the accuracy orcompleteness of thé ents ofthis work and specifically diâclaim all wàrranties, including without ation warranties of fitness for a particular purpose. No warranty may be ted or extended by sales or promotional materials. The advice and strategies ained herein may not be suitable for everysituation. This work is sold with understanding that the publisher is not engaged in rendering legal, )unting, or other professiohal services. If professional assistance is required, services of a competent professional person should be sought. Neither the isher nor the author shall be liable for damages arising herefrom. The fact an organization or Website if referred to in this work as a citation and/or a ntial source of further information does not mean that the author or the lisher endorses the information the organization or Website may provide or mmendations it may make. Further, readers should be aware that Internet )sites listed in this work may have changed or disappeared between when work was written and when it is read. i y also publishes its books in a variety of electronic formats. Some content --appéars in print may not be avail'able in electronic books. For more rmation abOut wiley products, visit our website at www.wiley.com. Drint: 2011 nted at: Ashim Printline, Sahibabad 3N : 978-81-265-0986-7 Tb my ¿hiidren, Paul, Anna, and Jack - Michael T Goodrich. To Isabel - Ròberto Tamassia Preface This book is designed to provide a comprehensive introduction to the design and analysis of computer algorithms and data structures. In terms M the computer science and computer engineering curricula, we have written this book to be primarily focused on the Junior-Senior level Algorithms (CS7) course, which is taught as a first-year graduate course in some schools. Topics The topics covered in this book are taken from a broad spectrum of discrete algorithm design and analysis, including the following: Design and analysis of algorithms, including asymptotic notation; worst- case analysis, amortization, randomization and experimental analysis Algorithmic design patterns, including greedy method, divide-arid-conquer, dynamic programming, backtracking and branch-and-bound Algorithmic frameworks, including NP-completeness, approximation algorithms, on-line algorithms, external-memory algorithms, distributed algorithms, and parallel algorithms Data structures, including lists, vectors, trees, priority queues, AYL trees, 2- 4 trees, red-black trees, splay trees, B-trees, hash tables, skip-lists, union-find trees Combinatorial algorithms, including heap-sort, quick-sort, merge-sort, selection, parallel list ranking, parallel sorting * Graph algorithms, including traversals (DFS and BFS), topological sorting, shortest paths (all-pairs and single-source), minimum spanning tree, maximum flow, minimum-cost flow, and matching Geometric algorithms, including range searching, convex hulls, segment in- tersection, and closest pairs Numerical algorithms, including integér, matrix, and polynomial multipli- cation, the Fast Fourier Transform (FF1'), extended Euclid's algorithm, mod- ular exponentiation, and primality testing Internet algorithms, including packet routing, multicasting, leader election, encryption, digital signatures, text pattern matching, information retrieval, data compression, Web caching, and Web auctions Preface About the Authors Professors Öoodrich and Tamassia are well-recognized researchers in data structures and àlgorithms, having published many papers in this field, with applications to Internet computing, information visualization, geographic information systems, and computer security. They have an extensive record of research collaboration and have served as principal investigators in several joint projects sponsored by the Na- tional Science Foundation, the Army Research Office, and:the Defense Advanced Research Projects Agency. They are also active in educational technology research, with special emphasis on algorithm visualization systems and infrastructure support for distance learning. Michael Goodrich received his Ph.D. in Computer Science' from Purdue Uni- versity in 1987. He is currently a professor in the Dçpartment of Information and Computer Science at University of California, Irvine. Prior to this service we was Professor of Compùter Science at Jorns Hopkins University, and director of the Hopkins Center for Algorithm Engineering. He is an editor for the International Journal of Computational Geometry & Applications, Journal of Computational and System Sciences, and Journal of Graph Algorithms and Applications. Roberto Tamassia received his Ph.D. in Electrical and Computer Engineering froth the University of Illinois at Urbana-Champaign in 1988. He is currently a professOr in the Department of Computer Science and the director of the Center for Geometric Computing at Brown University. He is an editor for Computational Geometry: Thçory and. Applications and the Journal of Graph Algorithms and Ap- plications, and he has previously served on the editonal board of IEEE Transactions on Computers. In addition to their research accomplishments, the authors also have extensive experience in the classroom. For example, Dr. Goodrich has taught data structures and algorithms courses since 1987, including Data Structures as a freshman- sophomore level coursé and Introduction to Algorithms as a upper level coùrse. He has earned several teaching awards in this capacity. His teaching style is to involve the students in lively interactive classroom sessions that bring out the in- tuition and insights behind data structuring and algorithmic techniques, as well as in formulating solutions whose analyis is mathematically rigorous; Dr. Tamas- sia has taught Data Structures and Algorithms as an introduètory freshman-level course since 1988. He has also attracted, many students (including several under- graduates) to his advanced course on Computational Geometry, which is à popular graduate-level CS course at Brown. One thing that has set his teaching style. 'apart is his effective use of interactive hypennedia presentations, continUing the tradi tion of.Brown's "electronic classçoom?' The carefully designed Web pages of the courses taught by Dr. Tamassia have bçen used as reference material by students and professionals worldwide. Preface For the Instructor Algorithms (CS7) primarily as a textbook for a Junior-Senior This book is intended . graduate course. in some schools. This course, which is also taught as a first-year reinforcemént exercises, which are divided between book containsmany exercises, Certain aspects of this book were creativity exercises, and implementation projects. specifically designed with theinstructor in mind, including: make mathçmatical ar- is, picture proofs), which Visual justifications (that appealing to visual learners. An guments more understandable for students, bottom-up heap constfuc- is. our analysis of example of visual justifications for students to understand; tion. This topic has traditionally been difficult explain. The included visual proof for instructors to hence, time coñsuming is intuitive, ngorous, and quick techniques for design- Algorithmic design patterns, which provide general divide-and-conquer, dy- algonthms Examples include ing and implementing namic programming, the decorator pattern, and the template method pattern in an al- ofrandomization, which takes advantage of random choices Use and analysis. Such usage replaces complex gorithin to simplify its dsign with intuitive analy- of sophisticated data structures average-case analysis incluçie skip lists, and algorithths. Examples sis of simple data structures quick-select, and randomized primality randomized uick-sort, randomized testing. traditional algonthniic which either motivate Internet algorithmics topics, algorithms that are viewpoint or highlight new topics from a new Intérnet. Examples include information retrieval, derived from Internet applications. algorithms, and Web caching routing, Web auction Web crawling, päcket algorithms. We have found that motivating algorithms topics by their Inter- interest in. the study of algo- significantly improves, student net applications rithms. . design methods, object- examples, which cover software Java implementation of algorithms., and experimental analysis oriented implementation issues, sections of various examples, provided in separate These implementation. insíructors can either cover them in their. lec- chapters, are Optional, so that additional reading, or skip them altogether. tures, assign them. as N allow the instructor a great deal of freedom in This book is alsd structured to the dependence between chap- how to organize and present the material. Likewise, customize an algorithms course to allowing the instructor to ters is rather flexible, highlight the topics that he or she feels are most important. We .have extensively quite interesting to stu- topics, which should prove disèussed Internet Algorithmics application of traditional inclúded examples of Internet dents. In addition, we have algorithms topics in several nlaces as well. Preface could be used for a traditional Introduc- 0.1 how this book We show in Table motivated from the with some new topics (CS7) course, albeit tion to Algorithms Internet. Option Topics Ch. Experimental analysis - Algorithm analysis Heap Java example Dáta structures Include one of § 3.2-3.5 Searching 3 In-place quick-sort Sorting The IPT Algorithmic techniques 5 DFS Java example Graph algorithms Dijkstra Java example Weighted graphs Include at end of course Matching and flow Tries Text processing (at least one section) 9 Include at end of course computational geometry .12 Backtracking NP-completeness 13 Include at end of course Frameworks (at least one) 14 traditional Introduction to Algorithms syllabus sche4ule for a Table 0.1: Exámple. for each chapter. (CS7) course, including optional choices. Algorithmics course, 'be used for a specialized Internet This book can also. topics, but in .a new internet-motivated traditional àlgorithms which r views some derived from Internet ap- algorithmic topics that are light, while also covering new book could be used for a such a course. plications. We show in Table 0.2 how this Option Ch. Topics Experimental analysis i Algorithm analysis Quickly review 2 Data structures (inc. hashing) lists) Search tree Java example Searching (inc. § 3.5, skip In-place quick-sort Sorting The WI Algorithmic techniques 5 DFS Java. example Graph algorithms 6 Skip one MST alg. Weighted graphs Matching algorithms Matching and flowS Pattern matching Text processing Java examples. Security & cryptography 19 Multi-cásting Network algorithms 11 include at end of course NP-completeness 13 Include at end of course Frameworks (at least two) 14 includ- Internet AlgorithmiCs course, syllabus schedule for an Table 0.2: Example for each chapter. ing optional choices . Preface Of course, other optiòns are also possible, including a course that is a mixture of a traditional Introduction to Algorithms (CS7) course and an Internet Algorithmics course. We do not belabor this point, however, leaving such creative arrangements to the interested instructor. Web Added-Value Education This book comes accompanied by an extensive Web site: http://www.wiley.com/college/goodrichi This Web site includes an extensive collection of educational aids that augment the topics of this book. Specifically for students we include: Presentation handouts (four-per-page format) for most topics in this book A database of hints on selected assignments, indexed by problem number Interactive applets that animate fundamentl data structures and algorithms .. Source code for the Java examples in this book We feel that the hint server should be of particular iñterest, particularly for èreativity problems that can be quite challenging for some students. For instructors i.ising this book, there is a dedicated portion of the Web site just for them, which includes the following additional teaching aids Solutions to seleòted exercises in this book A database of additional exercises and their solutions Presentations (one-per-page format) for most topics covered in this book Readers interested in the implementation of algorithms and data structures can download JDSL the Data Structures Library in Java, from http://www.jdsl.org/ Prerequisites Wehave written this book assuming that the reader comes to it with cértain knowl- edge In particular, we assume that the reader has a basic understanding of elementary data structures, such as arrays and linked lists, and is at least vaguely familiar with a high-level programming language, such as C, C++, or .Java. Even so, all álgorithms are described in a high-level "pseudo-code," and specific programming langùage ëonstructs are only used in the optional lava implementation example sections. In terms of mathematical background, we assume the reader is familiar with topiÇs from first-year college mathematics, including exponents, logarithms, summations, limits, and elementary probability. Even so, we review most of these fads in Chapter 1, iñcluding exponénts, logarithms, and summations, and we give ¡ t .suthmary of other useful mathematical facts, including elementary probability, in Appendix A. i Fundamental Tools I I Algorithm Analysis 3 1.1 Methodologies for Analyzing Algorithms 5 . 1.2 Asymptotic Notation 13 1.3 AQuick Mathematical Review 21 1.4 Case Studies in Algorithm Analysis. .. 31 1.5 Amortization 34 . .. 1.6 Experimentation 42 1.7 Exercises 47 . 2 Basic Data Structurés 55 2.1 Stacks and Queues 57 2.2 Vectors Lists, and Sequences 65 2.3 Trees 75 . . 2.4 Priority Queues and Heaps 94 2.5 Dictionaries and Hash Tables 114 2.6 Java Example: Heap 128 . 2.7 Exercises 131 . 3 Search Trees and Skip Lists 139 3.1 Ordered Dictionaries and Binary Search Trees 141 3.2 AVL Trees 152 . 3.3 Bounded-Depth Search Trees 159 3.4 Splay Trees. 185 3.5 Skip Lists 195 . 3.6 Java Example: AVL and Red-Black Trees 202 3.7 Exercises 212 . . 4 Sorting, Sets, and Selection 217 4.1 Merge-Sôrt 219 . 4.2 The Set Abstract Data Type 225 . 4.3 Quick-Sort 235 . . 4.4 A Lower Bound on Comparison-Based Sorting 239 4.5 Bucket-Sort and Radix-Sort 241 . Comparison of Sorting Algorithms .......... 4.6 244 . ., . 4.7 Selection 245 . 4.8 Java Example: In-Place Quick-Sort 248 . . 49 Exercises . 251 Contents 257 5 Fundamental Techniques 259 5.1 The Greedy Method 5.2 Divide-and-Conquer 263 5.3 Dynamk Programming 274 282 5.4 Exercises 285 Graph Algorithms 6 Graphs 287 6.1 The Graph Abstract Data Type 289 6.2 Data Structures for Graphs 296 303 6.3 Graph Traversal 316 6.4 Directed. Graphs 6.5 Java Example: Depth-First Search 329 335 6.6 Exercises 7 Weighted Graphs .339 7.1 Single-Source Shortest Paths 341 7.2 Ail-Pairs Shortest Paths 354 Minimum Spanning Trees ......... 360 7.3 . 7.4 Java Example: Dijkstra's Algorithm 373 376 7.5 Exercises 8 Network Flow and Matching 381 . 8.1 Flows and Cuts 383 8.2 Maximum Flow 387 8.3 Maximum Bipartite Matching . 396 8.4 Minimum-Cost Flow: 398 . . 8.5 Java Example: Minimum-Cost Flow 1.05 / . / 412 8.6 Exercises . III Internet Atgorithmics 415 9 Text Processing 417 9.1 Strings and Pattern Matching Algorithms 419 9.2: Tries 429 «o 9.3 Text Compression . 9.4 Text Similarity Testing 443 9.5 Exercises . . . 10 Number Theory and Cryptography 451 10.1 Fundamental Algorithms Involving Numbers 453 10.2 Cryptographic Cômputations 471 10.3 Information Security Algorithms and Protocols 481 488 10.4 The Fast Fourier Transform . 10.5 Java Example: FFT 500 . 508 10.6 Exercises