Intro to Algorithms

1339 Pages·2002·13.61 MB·English

PREFACE
CHAPTER1:INTRODUCTION
PARTI:MathematicalFoundations
CHAPTER2:GROWTHOF FUNCTIONS
CHAPTER3:SUMMATIONS
CHAPTER4:RECURRENCES
CHAPTER5:SETS,ETC.
CHAPTER6:COUNTINGAND PROBABILITY
PARTII:Sorting and OrderStatistics
CHAPTER7:HEAPSORT
CHAPTER8:QUICKSORT
CHAPTER9:SORTINGINLINEARTIME
CHAPTER10:MEDIANS AND ORDERSTATISTICS
PARTIII:Data Structures
CHAPTER11:ELEMENTARYDATA STRUCTURES
CHAPTER12:HASHTABLES
CHAPTER13:BINARYSEARCHTREES
CHAPTER14:RED-BLACK TREES
CHAPTER15:AUGMENTINGDATA STRUCTURES
PARTIV:AdvancedDesignand AnalysisTechniques
CHAPTER16:DYNAMIC PROGRAMMING
CHAPTER17:GREEDYALGORITHMS
CHAPTER18:AMORTIZEDANALYSIS
PARTV:AdvancedData Structures
CHAPTER19:B-TREES
CHAPTER20:BINOMIALHEAPS
CHAPTER21:FIBONACCIHEAPS
CHAPTER22:DATA STRUCTURESFORDISJOINTSETS
PARTVI: GraphAlgorithms
CHAPTER23:ELEMENTARYGRAPHALGORITHMS
CHAPTER24:MINIMUMSPANNINGTREES
1
CHAPTER25:SINGLE-SOURCESHORTESTPATHS
CHAPTER26:ALL-PAIRS SHORTESTPATHS
CHAPTER27:MAXIMUMFLOW
PARTVII:SelectedTopics
CHAPTER28:SORTINGNETWORKS
CHAPTER29:ARITHMETIC CIRCUITS
CHAPTER30:ALGORITHMS FORPARALLEL COMPUTERS
CHAPTER31:MATRIX OPERATIONS
CHAPTER32:POLYNOMIALSAND THE FFT
CHAPTER33:NUMBER-THEORETIC ALGORITHMS
CHAPTER34:STRINGMATCHING
CHAPTER35:COMPUTATIONALGEOMETRY
CHAPTER36:NPCOMPLETENESS
CHAPTER37:APPROXIMATIONALGORITHMS
BIBLIOGRAPHY
2
PPRREEFFAACCEE
Thisbookprovidesacomprehensive introductiontothe modernstudyofcomputeralgorithms.Itpresentsmany
algorithms and coverstheminconsiderable depth, yetmakestheir designand analysis accessible toalllevels of
readers.Wehave triedtokeepexplanations elementarywithout sacrificingdepthofcoverage ormathematicalrigor.
Eachchapterpresentsanalgorithm, adesigntechnique,anapplicationarea,orarelatedtopic.Algorithms are
describedinEnglishand ina"pseudocode"designedtobereadable byanyone who hasdone alittleprogramming.
The bookcontains over260figuresillustratinghowthe algorithms work.Sinceweemphasize efficiencyasadesign
criterion, weincludecarefulanalysesofthe runningtimesofallour algorithms.
The text isintendedprimarilyforuseinundergraduateorgraduatecoursesinalgorithms ordatastructures.
Becauseitdiscussesengineeringissuesinalgorithmdesign, aswellasmathematicalaspects,itisequallywellsuited
forself-studybytechnicalprofessionals.
Totheteacher
Tothestudent
Totheprofessional
Errors
Acknowledgments
1
To the teacher
Thisbookisdesignedtobebothversatile and complete.Youwillfinditusefulforavarietyofcourses,froman
undergraduatecourseindatastructuresupthroughagraduatecourseinalgorithms.Becausewehave provided
considerablymorematerialthancanfitinatypicalone-termcourse,youshould think ofthe bookasa"buffet"or
"smorgasbord"fromwhichyoucanpickand choosethe materialthatbestsupportsthe courseyouwishtoteach.
Youshould finditeasytoorganize your coursearound justthe chaptersyouneed.Wehave madechapters
relativelyself-contained,sothatyouneednotworryabout anunexpectedand unnecessarydependenceofone
chapteronanother.Eachchapterpresentsthe easiermaterialfirstand the moredifficultmateriallater,withsection
boundariesmarkingnaturalstoppingpoints.Inanundergraduatecourse,youmightuseonlythe earliersections from
achapter;inagraduatecourse,youmightcoverthe entirechapter.
Wehave includedover900exercisesand over120problems.Eachsectionendswithexercises,and each
chapterendswithproblems.The exercisesaregenerallyshortquestions thattestbasic masteryofthe material. Some
aresimple self-checkthought exercises,whereasothersaresuitable asassignedhomework.The problems aremore
elaboratecasestudiesthatoftenintroducenewmaterial; theytypicallyconsistofseveralquestions thatleadthe
student throughthe stepsrequiredtoarrive atasolution.
Wehave starred(*)the sections and exercisesthataremoresuitable forgraduatestudentsthanfor
undergraduates.Astarredsectionisnotnecessarilymoredifficultthananunstarredone,but itmayrequirean
understandingofmoreadvancedmathematics.Likewise,starredexercisesmayrequireanadvancedbackground or
morethanaverage creativity.
1
To the student
Wehopethatthistextbookprovidesyouwithanenjoyable introductiontothe fieldofalgorithms.Wehave
attemptedtomakeeveryalgorithmaccessible and interesting. To help youwhenyouencounterunfamiliarordifficult
algorithms,wedescribeeachone inastep-by-stepmanner.Wealsoprovidecarefulexplanations ofthe mathematics
neededtounderstand the analysis ofthe algorithms.Ifyoualreadyhave some familiaritywithatopic,youwillfind
the chaptersorganizedsothatyoucanskimintroductorysections and proceedquicklytothe moreadvanced
material.
Thisisalarge book,and your classwillprobablycoveronlyaportionofitsmaterial. Wehave tried,however,to
makethisabookthatwillbeusefultoyounowasacoursetextbookand alsolaterinyour careerasamathematical
deskreferenceoranengineeringhandbook.
Whatarethe prerequisitesforreadingthisbook?
• Youshould have some programmingexperience.Inparticular,youshould understand recursive procedures
and simple datastructuressuchasarrays and linkedlists.
• Youshould have some facilitywithproofs bymathematicalinduction. Afewportions ofthe bookrelyonsome
knowledge ofelementarycalculus.Beyond that,PartIofthisbookteachesyouallthe mathematicaltechniques
youwillneed.
1
To the professional
The widerange oftopicsinthisbookmakesitanexcellent handbookonalgorithms.Becauseeachchapteris
relativelyself-contained,youcanfocus inonthe topicsthatmostinterestyou.
Mostofthe algorithms wediscusshave greatpracticalutility.Wethereforeaddressimplementationconcerns and
otherengineeringissues.Wegenerallyprovidepracticalalternativestothe fewalgorithms thatareprimarilyof
theoreticalinterest.
Ifyouwishtoimplement anyofthe algorithms,youwillfindthe translationofour pseudocodeintoyour favorite
programminglanguage afairlystraightforwardtask.The pseudocodeisdesignedtopresent eachalgorithmclearly
and succinctly. Consequently, wedonotaddresserror-handlingand othersoftware-engineeringissuesthatrequire
specific assumptions about your programmingenvironment.Weattempttopresent eachalgorithmsimplyand
directlywithout allowingthe idiosyncrasiesofaparticularprogramminglanguage toobscureitsessence.
1
Errors
Abookofthislengthiscertaintocontainerrorsand omissions.Ifyoufindanyerrorsorhave otherconstructive
suggestions,wewould appreciatehearingfromyou. Weparticularlywelcome suggestions fornewexercisesand
problems,but pleaseincludesolutions.Youcanmailyour commentsto
IntroductiontoAlgorithms
MITLaboratoryforComputerScience
545TechnologySquare
Cambridge,Massachusetts02139
Alternatively, youcanuseInternetelectronic mailtosubmit bugreports,requestalistofknownerrors,ormake
constructive suggestions.To receive instructions,send electronic mailtoalgorithms@theory. lcs.mit.eduwith
"Subject:help"inthe message header.Weregretthatwecannotpersonallyrespond toallmail.
1
Acknowledgments
Manyfriendsand colleagueshave contributedgreatlytothe qualityofthisbook.Wethank allofyouforyour
help and constructive criticisms.
MIT'sLaboratoryforComputerSciencehasprovidedanidealworkingenvironment.Our colleaguesinthe
laboratory'sTheoryofComputationGroup have beenparticularlysupportive and tolerant ofour incessant
requestsforcriticalappraisalofchapters.Wespecificallythank BaruchAwerbuch, ShafiGoldwasser,Leo
Guibas,TomLeighton, AlbertMeyer,David Shmoys,and Eva Tardos.ThankstoWilliamAng, SallyBemus,
RayHirschfeld,and MarkReinhold forkeepingour machines(DEC Microvaxes,Apple Macintoshes,and Sun
Sparcstations)runningand forrecompilingT Xwheneverweexceededacompile-time limit.ThinkingMachines
E
CorporationprovidedpartialsupportforCharlesLeisersontoworkonthisbookduringaleave ofabsencefrom
MIT.
Manycolleagueshave useddraftsofthistext incoursesatotherschools.Theyhave suggestednumerous
corrections and revisions.Weparticularlywishtothank RichardBeigel(Yale),AndrewGoldberg(Stanford),
JoanLucas(Rutgers),MarkOvermars(Utrecht),AlanSherman(Tuftsand Maryland),and Diane Souvaine
(Rutgers).
Manyteachingassistantsinour courseshave madesignificant contributions tothe development ofthismaterial.
Weespeciallythank AlanBaratz, Bonnie Berger,AditiDhagat,BurtKaliski, Arthur Lent,AndrewMoulton,
MariosPapaefthymiou, CindyPhillips,MarkReinhold,PhilRogaway, Flavio Rose,Arie Rudich, AlanSherman,
CliffStein, SusmitaSur,GregoryTroxel, and MargaretTuttle.
Additionalvaluable technicalassistancewasprovidedbymanyindividuals.DeniseSergent spent manyhours
inthe MITlibrariesresearchingbibliographic references.Maria Sensale,the librarianofour readingroom, was
always cheerfuland helpful. AccesstoAlbertMeyer'spersonallibrarysavedmanyhoursoflibrarytimein
preparingthe chapternotes.Shlomo Kipnis,BillNiehaus,and David Wilsonproofreadold exercises,developed
newones,and wrotenotesontheir solutions.MariosPapaefthymiouand GregoryTroxelcontributedtothe
indexing. Overthe years,our secretariesInna Radzihovsky, DeniseSergent,Gayle Sherman, and especiallyBe
Hubbardprovidedendlesssupportinthisproject,forwhichwethank them.
Manyerrorsinthe earlydraftswerereportedbystudents.Weparticularlythank BobbyBlumofe,Bonnie
Eisenberg, Raymond Johnson, JohnKeen, RichardLethin, MarkLillibridge,JohnPezaris,Steve Ponzio,and
MargaretTuttle fortheir carefulreadings.
Colleagueshave alsoprovidedcriticalreviewsofspecific chapters,orinformationonspecific algorithms,for
whichwearegrateful. Weespeciallythank BillAiello,AlokAggarwal, Eric Bach, VaÜekChvátal, RichardCole,
JohanHastad,AlexIshii, David Johnson, JoeKilian, DinaKravets,BruceMaggs,JimOrlin, JamesPark,Thane
Plambeck,HerschelSafer,JeffShallit,CliffStein, GilStrang, BobTarjan, and PaulWang. Severalofour
colleaguesalsograciouslysupplieduswithproblems;weparticularlythank AndrewGoldberg, DannySleator,
and UmeshVazirani.
1
ThisbookwastypesetwithLAT X, amacropackage forT X. The figuresweredrawnonanApple
E E
MacintoshusingMacDrawII;thankstoJoanna TerryofClaris Corporationand MichaelMahoneyofAdvanced
ComputerGraphicsfortimelycustomersupport.The indexwascompiledusingWindex, aC programwrittenby
the authors.The bibliographywaspreparedusingBIBT X. The bookwastypesetatthe AmericanMathematical
E
SocietywithanAutologic phototypesetter;thankstoRalphYoungenofAMSforhishelp.The coverforthe
bookwasdesignedbyJeannetLeendertse.The bookdesignwascreatedbyRebeccaDaw, and Amy
Hendricksonimplementedthe designinLAT X.
E
IthasbeenapleasureworkingwithThe MITPressand McGraw-Hillinthe development ofthistext.We
especiallythank Frank Satlow, TerryEhling,LarryCohen, and Lorrie Lejeune ofThe MITPressand David
ShapiroofMcGraw-Hillfortheir encouragement,support,and patience.WeareparticularlygratefultoLarry
Cohenforhisoutstandingcopyediting.
Finally, wethank our wives—Nicole Cormen, LindaLue Leiserson, and GailRivest—and our
children—Ricky, William,and DebbyLeisersonand Alexand ChristopherRivest—fortheir love and support
duringthe writingofthisbook.(AlexRivestalsohelpedwiththe "Martianbirthdayparadox.") The love,patience,
and encouragement ofour familiesmadethisprojectpossible.Weaffectionatelydedicatethisbooktothem.
Cambridge,Massachusetts
March1990
THOMAS H. CORMEN
CHARLES E. LEISERSON
RONALDL. RIVEST
2