The Art of Computer Programming. Volume 4A. Combinatorial Algorithms. Part 1

2011·33.9 MB·English

by  Donald E. Knuth

the classic work
EXTENDED AND REFINED
The
Art of
Computer
Programming
VOLUME 4A
Combinatorial Algorithms
Part 1
DONALD KNUTH
E.
VolumeIIFundamentalAlgorithms
ThirdEdition(0-201-89683-4)
Thisfirstvolumebeginswithbasicprogramming
conceptsandtechnique—s,thenfocuseson
informationstructures therepresentation
ofinformationinsideacomputer,thestructural
relationshipsbetweendataelementsandhow
todealwiththemefficiently.Elementary
applicationsaregiventosimulation,numerical
methods,symboliccomputing,softwareand
systemdesign.
Volume2/SeminumericalAlgorithms
ThirdEdition(0-201-89684-2)
Thesecondvolumeoffersacomplete
introductiontothefieldofseminumerical
algorithms,withseparatechaptersonrandom
numbersandarithmetic.Thebooksummarizes
themajorparadigmsandbasictheoryofsuch
algorithms,therebyprovidingacomprehensive
interfacebetweencomputerprogramming
andnumericalanalysis.
Volume3/SortingandSearching
SecondEdition(0-201-89685-0)
Thethirdvolumecomprisesthemost
comprehensivesurveyofclassicalcomputer
techniquesforsortingandsearching.Itextends
ttohecotnrseiadtemrenbtotohfdlaartgaesatnrductsmuarlelsdinatVaoblasuemseanId
internalandexternalmemories.
Volume4A/CombinatorialAlgorithms,
Part1(0-201-03804-8)
Thisvolumeintroducestechniquesthatallow
computerstodealefficientlywithgigantic
problems.ItscoveragebeginswithBoolean
functionsandbitwisetricksandtechniques,
thentreatsindepththegenerationofall
tuplesandpermutations,allcombinations
andpartitions,andalltrees.

DONALDE.KNUTH StanfordUniversity
ADDISON-WESLEY
Volume4A / Combinatorial Algorithms, Part 1
THE ART OF
COMPUTER PROGRAMMING
UNCpaeppweertYooSrwaknddl••eTSRoyirdvonenret,yoNJ••TMooknBytoorsetao•lnSi••ngLIaonpndoidraoennap•o•lMiMesxuin•ciocShaCnit•FyrPaanrciissco• Madrid
.
TRhaendpoomemHoounsep,ag1e98463),7cisopqyuroitgehdtf©ro1m98T6hebyGoVlidkernamGaSteethb.yVikramSeth(NewYork:
Theauthorandpublisherhavetakencareinthepreparationofthisbook,butmakeno
eowmxiiptsrhseiosorsnesad.riosNrionigmlipoalubitieloidtfywtiahsrearsausnsetuymoefodfthfaeonryiinnkfcioinrddmenatatainlodnoraosrcsopunrsmeoeqgurneaonmtrsieasclpoodnntasamiibanigleeidtsyhiefnroecrionen.rnreocrtsioonr
Tpaunhrdepocpsouenbstleionsrthesprpaeorcftifieacrluslsaearxlcetseo,llyweonhutircdhibsumcsoaiunyenstissn,coltnuradtiehnieislnbegcotgoorkaolnwsi,chemvnaerroskrieodtneisrnegadnfdion/couqsru,acnautnisdttyobmrfoacrnodbviuenlrgks
interests.Formoreinformation,pleasecontact:
U.S.CorporateandGovernmentSales [phone]
corpsalesSpearsontechgroup.com
ForsalesoutsidetheU.S.,pleasecontact:
InternationalSales [e-mail]
VisitusontheWeb:informit.com/aw
LibraryofCongressCataloging-in-PublicationData
Knuth, DonaldErvin, 1938-
Txhvei,8a8r3tpo.fc24ompcmu.terprogramming/DonaldErvinKnuth.
algCIoonrncitlteuhndmtesss.:biv—.bliv1..ogrF3.aupnhdSiaocmraetlnitnagrlefaenardlegnoscreeiatsrhcmhasin.ndg.—ind—evx..v.2.4aS.emiCnoummbeirniactaolrial
algCIIII1.ooSSSSnBrBBBtNNiNNEetlnhe9999tmc7777sst8888:r----,o0000n----pvi.2222acr0000t11114-a---d.8880i19993g6668iC8880to3a454m-l---b1850incao((((tmvvvv....opruit3421,ae,a,)rls332—rrnadddlPgroeeeordddg...i)))rtahmmmsi,ngp.art2.1.Computer
algorithms. I. Title.
0Q0A57.61.6—.KD6C241 1997 97-2147
Internetpagehttp://www-cs-faculty.stanford.edu/~knuth/taocp.htmlcontains
currentinformationaboutthisbookandrelatedbooks.
tSahebeeougatrlsaTophhhesttSupts:ae/ndf/iownrwdwm-aGcnrsay-pfhoafBcatushleet,yexi.nascmtlpaulndefison.grdd.oewdnul/o~akdnaubtlhe/ssogfbt.whatremlfofrordeianlfionrgmwatiitohn
mAantdiosneeahbtotupt:t/h/ewwMwM-IcXsc-ofmapcuutletry..stanford.edu/~knuth/mmix.htmlforbasicinfor-
Copyright©2011byPearsonEducation,Inc.
aoAprlrnloytberpyicrgtoahehntdisybbirmtyeeesdaceonrrpvseye,prdri.eogldheutPc,ctrtriiaononntnidec,d,psetimrnoemrcitahshgaseeniiioUcnnnaailm,tureesdptthrioSebttveoaatcloeobssptyyasoiitfnengmeA,,dmerofrerricocotmarrd.aitnnhsgemT,ihpsiuossrbilopilnusibkhielenwiriacsapnerty.iioofrnoFrotimros
informationregardingpermissions,writeto:
PearsonEducation,Inc.
RightsandContractsDepartment
5B0o1stBono,ylMstAon0S2t1r1ee6t,SuiFtaex9:00(617)671-3447
ISBN-13 978-0-201-03804-0
ISBN-10 0-201-03804-8
TextprintedintheUnitedStatesatCourierWestfordinWest.ford,Massachusetts.
Secondprinting,March2011
PREFACE
Toputallthegoodstuffintoonebookispatentlyimpossible,
andattemptingeventobereasonablycomprehensive
aboutcertainasp—ectsGEofRAthLeDsuBb.jeFctOLisLlAikNeDly,to“Eldeiatdort’osrCuonranwear"yg(r2o0w0t5h.)
ThetitleofVolume4isCombinatorialAlgorithms,andwhenIproposedit
Iwasstronglyinclinedtoaddasubtitle:TheKindofProgrammingILikeBest.
Myeditorshavedecidedtotonedownsuchexuberance,butthefactremains
thatprogramswithacombinatorialflavorhavealwaysbeenmyfavorites.
OntheotherhandI’veoftenbeensurprisedtofindthat,inmanypeople’s
minds,theword“combinatorial”islinkedwithcomputationaldifficulty.Indeed,
SamuelJohnson,inhisfamousdictionaryoftheEnglishlanguage(1755),said
thatthecorrespondingnoun“isnowgenerallyusedinanillsense.” Colleagues
tellmetalesofwoe,inwhichtheyreportthat “thecombinatoricsofthesit-
uationdefeatedus.” Whyisitthat,forme,combinatoricsarousesfeelingsof
purepleasure,yetformanyothersitevokespurepanic?
It’struethatcombinatorialproblemsareoftenassociatedwithhumongously
largenumbers. Johnson’sdictionaryentryalsoincludedaquotefromEphraim
Chambers,whohadstatedthatthetotalnumberofwordsoflength24orless,
ina24-letteralphabet,is1,391,724,288,887,252,999,425,128,493,402,200. The
correspondingnumberfora10-letteralphabetis11,111,111,110;andit’sonly
3905whenthenumberoflettersis5.Thusa“combinatorialexplosion”certainly
doesoccurasthesizeoftheproblemgrowsfrom5to10to24andbeyond.
Computingmachineshavebecometremendouslymorepowerfulthroughout
mylife.AsIwritethesewords,Iknowthattheyarebeingprocessedbya“lap-
top”whosespeedismorethan100,000timesfasterthanthetrustyIBMType650
computertowhichI’vededicatedthesebooks;mycurrentmachine’smemory
capacityisalsomorethan100,000timesgreater.Tomorrow’scomputerswillbe
evenfasterandmorecapacious.Buttheseamazingadvanceshavenotdiminished
people’scravingforanswerstocombinatorialquestions;quitethecontrary.Our
once-unimaginableabilitytocomp—utesorapidlyhasraisedourexpectations,
andwhettedourappetiteformore because,infact,thesizeofacombinatorial
problemcanincreasemorethan100,000-foldwhennsimplyincreasesby1.
Combinatorialalgorithmscanbedefinedinformallyastechniquesforthe
high-speedmanipulationofcombinatorialobjectssuchaspermutationsorgraphs.
Wetypicallytrytofindpatternsorarrangementsthatarethebestpossibleways
tosatisfycertainconstraints.Thenumberofsuchproblemsisvast,andtheart
VI PREFACE
ofwritingsuchprogramsisespeciallyimportantandappealingbecauseasingle
goodideacansaveyearsorevencenturiesofcomputertime.
Indeed,thefactthatgoodalgorithmsforcombinatorialproblemscanhavea
terrificpayoffhasledtoterrificadvancesinthestateoftheart.Manyproblems
thatoncewerethoughttobeintractablecannowbepolishedoffwithease,and
manyalgorithmsthatoncewereknowntobegoodhavenowbecomebetter.
Startingabout1970,computerscientistsbegantoexperienceaphenomenon
thatwecalled“Floyd’sLemma”:Problemsthatseemedtoneedn3operations
couldactuallybesolvedin()(n2):problemsthatseemedtorequiren2couldbe
handledinO(nlogn);andnlognwasoftenreducibleto0(n). Moredifficult
problemssawareductioninrunningtimefrom0(2”)to0(1.5”)to0(1.3"),
etc. Otherproblemsremaineddifficultingeneral,buttheywerefoundtohave
importantspecialcasesthataremuchsimpler. Manycombinatorialquestions
thatIoncethoughtwouldneverbeansweredduringmylifetimehavenowbeen
resolved,andthosebreakthroughshavebeenduemainlytoimprovementsin
algorithmsratherthantoimprovementsinprocessorspeeds.
By1975,suchresearchwasadvancingsorapidlythatasubstantialfraction
ofthepaperspublishedinleadingjournalsofcomputerscienceweredevoted
tocombinatorialalgorithms. Andtheadvancesweren’tbeingmadeonlyby
peopleinthecoreofcomputerscience;significantcontributionswerecoming
fromworkersinelectricalengineering,artificialintelligence,operationsresearch,
mathematics, physics, statistics, andotherfields. Iwastryingtocomplete
Volume4ofTheArtofComputerProgramming,butinsteadIfeltlikeIwas
sittingonthelidofaboilingkettle: Iwasconfrontedwithacombinatorial
explosionofanotherkind,aprodigiousexplosionofnewideas!
Thisseriesofbookswasbornatthebeginningof1962, whenInaively
wroteoutalistoftentativechaptertitlesfora12-chapterbook. Atthattime
Idecidedtoincludeabriefchapteraboutcombinatorialalgorithms,justfor
fun. “Heylook,mostpeopleusecomputerstodealwithnumbers,butwecan
alsowriteprogramsthatdealwithpatterns.” Inthosedaysitwaseasytogive
afairlycompletedescriptionofjustabouteverycombinatorialalgorithmthat
wasknown. Andevenby1966,whenI’dfinishedafirstdraftofabout3000
handwrittenpagesforthatalready-overgrownbook,fewerthan100ofthose
pagesbelongedtoChapter7.IhadabsolutelynoideathatwhatI’dforeseenas
asortof“saladcourse”wouldeventuallyturnouttobethemaindish.
Thegreatcombinatorialfermentationof1975hascontinuedtochurn,as
moreandmorepeoplehavebeguntoparticipate.Newideasimproveuponthe
olderones,butrarelyreplacethemormakethemobsolete. SoofcourseI’ve
hadtoabandonanyhopesthatIoncehadofbeingabletosurroundthefield,
towriteadefinitivebookthatsetseverythinginorderandprovidesone-stop
shoppingforeveryonewhohascombinatorialproblemstosolve. Thearrayof
applicabletechniqueshasmushroomedtothepointwhereIcanalmostnever
dmiussctussresatrsiucbttompyiscelafntdoseaxy,pla“iHnerien’gstthheemfoisnatlisomlpuotritoan:ntenprdinocfipslteosry.t”hatInssteeeamd,toI
underliealloftheefficientcombinatorialmethodsthatI’veencounteredsofar.