An Introduction to the Analysis of Algorithms

593 Pages·2013·6.64 MB·english

by  Robert Sedgewick

AN INTRODUCTION
TO THE
ANALYSIS OF ALGORITHMS
Second Edition
http://avaxho.me/blogs/ChrisRedfield
This page intentionally left blank
AN INTRODUCTION
TO THE
ANALYSIS OF ALGORITHMS
Second Edition
Robert Sedgewick
Princeton University
Philippe Flajolet
INRIA Rocquencourt
UpperSaddleRiver,NJ(cid:15)Boston(cid:15)Indianapolis(cid:15)SanFrancisco
NewYork(cid:15)Toronto(cid:15)Montreal(cid:15)London(cid:15)Munich(cid:15)Paris(cid:15)Madrid
Capetown(cid:15)Sydney(cid:15)Tokyo(cid:15)Singapore(cid:15)MexicoCity
Manyofthedesignationsusedbymanufacturersandsellerstodistinguishtheirprod-
uctsareclaimedastrademarks. Wherethosedesignationsappearinthisbook,and
the publisher was aware of a trademark claim, the designations have been printed
withinitialcapitallettersorinallcapitals.
(cid:315)eauthorsandpublisherhavetakencareinthepreparationofthisbook,butmake
no expressed or implied warranty of any kind and assume no responsibility for er-
rorsoromissions. Noliabilityisassumedforincidentalorconsequentialdamagesin
connectionwithorarisingoutoftheuseoftheinformationorprogramscontained
herein.
(cid:315)e publisher offers excellent discounts on this book when ordered in quantity for
bulkpurchasesorspecialsales,whichmayincludeelectronicversionsand/orcustom
coversandcontentparticulartoyourbusiness, traininggoals, marketingfocus, and
brandinginterests. Formoreinformation,pleasecontact:
U.S.CorporateandGovernmentSales
[phone]
[email protected]
ForsalesoutsidetheUnitedStates,pleasecontact:
InternationalSales
[email protected]
VisitusontheWeb: informit.com/aw
LibraryofCongressControlNumber: 2012955493
Copyright⃝c 2013PearsonEducation,Inc.
All rights reserved. Printed in the United States of America. (cid:315)is publication is
protectedbycopyright,andpermissionmustbeobtainedfromthepublisherpriorto
anyprohibitedreproduction,storageinaretrievalsystem,ortransmissioninanyform
or by any means, electronic, mechanical, photocopying, recording, or likewise. To
obtainpermissiontousematerialfromthiswork,pleasesubmitawrittenrequestto
PearsonEducation,Inc.,PermissionsDepartment,OneLakeStreet,UpperSaddle
River,NewJersey07458,oryoumayfaxyourrequestto(201)236-3290.
ISBN-13: 978-0-321-90575-8
ISBN-10: 0-321-90575-X
TextprintedintheUnitedStatesonrecycledpaperatCourierinWestford,Massachusetts.
Firstprinting,January2013
FOREWORD
PEOPLEwhoanalyzealgorithmshavedoublehappiness. Firstofallthey
experience the sheer beauty of elegant mathematical patterns that sur-
roundelegantcomputationalprocedures. (cid:315)entheyreceiveapracticalpayoff
whentheirtheoriesmakeitpossibletogetotherjobsdonemorequicklyand
moreeconomically.
Mathematical models have been a crucial inspiration for all scienti(cid:320)c
activity, even though they are only approximate idealizations of real-world
phenomena. Insideacomputer,suchmodelsaremorerelevantthaneverbe-
fore,becausecomputerprogramscreatearti(cid:320)cialworldsinwhichmathemat-
icalmodelsoftenapplyprecisely. Ithinkthat’swhyIgothookedonanalysis
of algorithms when I was a graduate student, and why the subject has been
mymainlife’sworkeversince.
Untilrecently,however,analysisofalgorithmshaslargelyremainedthe
preserveofgraduatestudentsandpost-graduateresearchers. Itsconceptsare
notreallyesotericordifficult,buttheyarerelativelynew,soithastakenawhile
tosortoutthebestwaysoflearningthemandusingthem.
Now,aftermorethan40yearsofdevelopment,algorithmicanalysishas
matured to the point where it is ready to take its place in the standard com-
puter science curriculum. (cid:315)e appearance of this long-awaited textbook by
Sedgewick and Flajolet is therefore most welcome. Its authors are not only
worldwideleadersofthe(cid:320)eld,theyalsoaremastersofexposition. Iamsure
that every serious computer scientist will (cid:320)nd this book rewarding in many
ways.
D.E.Knuth
This page intentionally left blank
PREFACE
THIS book is intended to be a thorough overview of the primary tech-
niques used in the mathematical analysis of algorithms. (cid:315)e material
covereddraws fromclassicalmathematical topics, including discretemathe-
matics, elementary real analysis, and combinatorics, as well as from classical
computersciencetopics,includingalgorithmsanddatastructures. (cid:315)efocus
ison“average-case”or“probabilistic”analysis,thoughthebasicmathematical
toolsrequiredfor“worst-case”or“complexity”analysisarecoveredaswell.
We assume that the reader has some familiarity with basic concepts in
both computer science and real analysis. In a nutshell, the reader should be
able to both write programs and prove theorems. Otherwise, the book is
intendedtobeself-contained.
(cid:315)ebookismeanttobeusedasatextbookinanupper-levelcourseon
analysisofalgorithms. Itcanalsobeusedinacourseindiscretemathematics
forcomputerscientists,sinceitcoversbasictechniquesindiscretemathemat-
icsaswellascombinatoricsandbasicpropertiesofimportantdiscretestruc-
tureswithinafamiliarcontextforcomputersciencestudents. Itistraditional
tohavesomewhatbroadercoverageinsuchcourses,butmanyinstructorsmay
(cid:320)ndtheapproachheretobeausefulwaytoengagestudentsinasubstantial
portion of the material. (cid:315)e book also can be used to introduce students in
mathematics and applied mathematics to principles from computer science
relatedtoalgorithmsanddatastructures.
Despite the large amount of literature on the mathematical analysis of
algorithms,basicinformationonmethodsandmodelsinwidespreadusehas
notbeendirectlyaccessibletostudentsandresearchersinthe(cid:320)eld. (cid:315)isbook
aimstoaddressthissituation,bringingtogetherabodyofmaterialintended
toprovidereaderswithbothanappreciationforthechallengesofthe(cid:320)eldand
thebackgroundneededtolearntheadvancedtoolsbeingdevelopedtomeet
these challenges. Supplemented by papers from the literature, the book can
serveasthebasisforanintroductorygraduatecourseontheanalysisofalgo-
rithms,orasareferenceorbasisforself-studybyresearchersinmathematics
orcomputersciencewhowantaccesstotheliteratureinthis(cid:320)eld.
Preparation. Mathematical maturity equivalent to one or two years’ study
at the college level is assumed. Basic courses in combinatorics and discrete
mathematics may provide useful background (and may overlap with some
viii P(cid:350)(cid:337)(cid:338)(cid:333)(cid:335)(cid:337)
material in the book), as would courses in real analysis, numerical methods,
orelementarynumbertheory. Wedrawonalloftheseareas,butsummarize
the necessary material here, with reference to standard texts for people who
wantmoreinformation.
Programming experience equivalent to one or two semesters’ study at
the college level, including elementary data structures, is assumed. We do
not dwell on programming and implementation issues, but algorithms and
data structures are the central object of our studies. Again, our treatment is
complete in the sense that we summarize basic information, with reference
tostandardtextsandprimarysources.
Relatedbooks. Related texts include (cid:315)e Art of Computer Programming by
Knuth; Algorithms, Fourth Edition, by Sedgewick and Wayne; Introduction
toAlgorithmsbyCormen,Leiserson,Rivest,andStein;andourownAnalytic
Combinatorics. (cid:315)isbookcouldbeconsideredsupplementarytoeachofthese.
Inspirit,thisbookisclosesttothepioneeringbooksbyKnuth. Ourfo-
cusisonmathematicaltechniquesofanalysis,though,whereasKnuth’sbooks
arebroadandencyclopedicinscope,withpropertiesofalgorithmsplayinga
primaryroleandmethodsofanalysisasecondaryrole. (cid:315)isbookcanserveas
basicpreparationfortheadvancedresultscoveredandreferredtoinKnuth’s
books. Wealsocoverapproachesandresultsintheanalysisofalgorithmsthat
havebeendevelopedsincepublicationofKnuth’sbooks.
We also strive to keep the focus on covering algorithms of fundamen-
talimportanceandinterest,suchasthosedescribedinSedgewick’sAlgorithms
(nowinitsfourthedition,coauthoredbyK.Wayne). (cid:315)atbooksurveysclassic
algorithms for sorting and searching, and for processing graphs and strings.
Ouremphasisisonmathematicsneededtosupportscienti(cid:320)cstudiesthatcan
serveasthebasisofpredictingperformanceofsuchalgorithmsandforcom-
paringdifferentalgorithmsonthebasisofperformance.
Cormen, Leiserson, Rivest, and Stein’s Introduction to Algorithms has
emergedasthestandardtextbookthatprovidesaccesstotheresearchlitera-
tureonalgorithmdesign. (cid:315)ebook(andrelatedliterature)focusesondesign
and the theory of algorithms, usually on the basis of worst-case performance
bounds. Inthisbook,wecomplementthisapproachbyfocusingontheanal-
ysis of algorithms, especially on techniques that can be used as the basis for
scienti(cid:320)c studies (as opposed to theoretical studies). Chapter 1 is devoted
entirelytodevelopingthiscontext.
P(cid:350)(cid:337)(cid:338)(cid:333)(cid:335)(cid:337) ix
(cid:315)is book also lays the groundwork for our Analytic Combinatorics, a
general treatment that places the material here in a broader perspective and
develops advanced methods and models that can serve as the basis for new
research,notonlyintheanalysisofalgorithmsbutalsoincombinatoricsand
scienti(cid:320)c applications more broadly. A higher level of mathematical matu-
rity is assumed for that volume, perhaps at the senior or beginning graduate
student level. Of course, careful study of this book is adequate preparation.
It certainly has been our goal to make it sufficiently interesting that some
readerswillbeinspiredtotacklemoreadvancedmaterial!
Howtousethisbook. Readersofthisbookarelikelytohaveratherdiverse
backgrounds in discrete mathematics and computer science. With this in
mind,itisusefultobeawareoftheimplicitstructureofthebook: ninechap-
ters in all, an introductory chapter followed by four chapters emphasizing
mathematicalmethods,thenfourchaptersemphasizingcombinatorialstruc-
tureswithapplicationsintheanalysisofalgorithms,asfollows:
INTRODUCTION
ONE ANALYSIS OF ALGORITHMS
DISCRETE MATHEMATICAL METHODS
TWO RECURRENCE RELATIONS
THREE GENERATING FUNCTIONS
FOUR ASYMPTOTIC APPROXIMATIONS
FIVE ANALYTIC COMBINATORICS
ALGORITHMS AND COMBINATORIAL STRUCTURES
SIX TREES
SEVEN PERMUTATIONS
EIGHT STRINGS AND TRIES
NINE WORDS AND MAPPINGS
Chapter 1 puts the material in the book into perspective, and will help all
readers understand the basic objectives of the book and the role of the re-
maining chapters in meeting those objectives. Chapters 2 through 4 cover