Table Of Content

the classic work EXTENDED AND REFINED The Art of Computer Programming VOLUME 4A Combinatorial Algorithms Part 1 DONALD KNUTH E. VolumeIIFundamentalAlgorithms ThirdEdition(0-201-89683-4) Thisfirstvolumebeginswithbasicprogramming conceptsandtechnique—s,thenfocuseson informationstructures therepresentation ofinformationinsideacomputer,thestructural relationshipsbetweendataelementsandhow todealwiththemefficiently.Elementary applicationsaregiventosimulation,numerical methods,symboliccomputing,softwareand systemdesign. Volume2/SeminumericalAlgorithms ThirdEdition(0-201-89684-2) Thesecondvolumeoffersacomplete introductiontothefieldofseminumerical algorithms,withseparatechaptersonrandom numbersandarithmetic.Thebooksummarizes themajorparadigmsandbasictheoryofsuch algorithms,therebyprovidingacomprehensive interfacebetweencomputerprogramming andnumericalanalysis. Volume3/SortingandSearching SecondEdition(0-201-89685-0) Thethirdvolumecomprisesthemost comprehensivesurveyofclassicalcomputer techniquesforsortingandsearching.Itextends ttohecotnrseiadtemrenbtotohfdlaartgaesatnrductsmuarlelsdinatVaoblasuemseanId internalandexternalmemories. Volume4A/CombinatorialAlgorithms, Part1(0-201-03804-8) Thisvolumeintroducestechniquesthatallow computerstodealefficientlywithgigantic problems.ItscoveragebeginswithBoolean functionsandbitwisetricksandtechniques, thentreatsindepththegenerationofall tuplesandpermutations,allcombinations andpartitions,andalltrees. DONALDE.KNUTH StanfordUniversity ADDISON-WESLEY Volume4A / Combinatorial Algorithms, Part 1 THE ART OF COMPUTER PROGRAMMING UNCpaeppweertYooSrwaknddl••eTSRoyirdvonenret,yoNJ••TMooknBytoorsetao•lnSi••ngLIaonpndoidraoennap•o•lMiMesxuin•ciocShaCnit•FyrPaanrciissco• Madrid . TRhaendpoomemHoounsep,ag1e98463),7cisopqyuroitgehdtf©ro1m98T6hebyGoVlidkernamGaSteethb.yVikramSeth(NewYork: Theauthorandpublisherhavetakencareinthepreparationofthisbook,butmakeno eowmxiiptsrhseiosorsnesad.riosNrionigmlipoalubitieloidtfywtiahsrearsausnsetuymoefodfthfaeonryiinnkfcioinrddmenatatainlodnoraosrcsopunrsmeoeqgurneaonmtrsieasclpoodnntasamiibanigleeidtsyhiefnroecrionen.rnreocrtsioonr Tpaunhrdepocpsouenbstleionsrthesprpaeorcftifieacrluslsaearxlcetseo,llyweonhutircdhibsumcsoaiunyenstissn,coltnuradtiehnieislnbegcotgoorkaolnwsi,chemvnaerroskrieodtneisrnegadnfdion/couqsru,acnautnisdttyobmrfoacrnodbviuenlrgks interests.Formoreinformation,pleasecontact: U.S.CorporateandGovernmentSales (800)382-3419 corpsalesSpearsontechgroup.com ForsalesoutsidetheU.S.,pleasecontact: InternationalSales [email protected] VisitusontheWeb:informit.com/aw LibraryofCongressCataloging-in-PublicationData Knuth, DonaldErvin, 1938- Txhvei,8a8r3tpo.fc24ompcmu.terprogramming/DonaldErvinKnuth. algCIoonrncitlteuhndmtesss.:biv—.bliv1..ogrF3.aupnhdSiaocmraetlnitnagrlefaenardlegnoscreeiatsrhcmhasin.ndg.—ind—evx..v.2.4aS.emiCnoummbeirniactaolrial algCIIII1.ooSSSSnBrBBBtNNiNNEetlnhe9999tmc7777sst8888:r----,o0000n----pvi.2222acr0000t11114-a---d.8880i19993g6668iC8880to3a454m-l---b1850incao((((tmvvvv....opruit3421,ae,a,)rls332—rrnadddlPgroeeeordddg...i)))rtahmmmsi,ngp.art2.1.Computer algorithms. I. Title. 0Q0A57.61.6—.KD6C241 1997 97-2147 Internetpagehttp://www-cs-faculty.stanford.edu/~knuth/taocp.htmlcontains currentinformationaboutthisbookandrelatedbooks. tSahebeeougatrlsaTophhhesttSupts:ae/ndf/iownrwdwm-aGcnrsay-pfhoafBcatushleet,yexi.nascmtlpaulndefison.grdd.oewdnul/o~akdnaubtlhe/ssogfbt.whatremlfofrordeianlfionrgmwatiitohn mAantdiosneeahbtotupt:t/h/ewwMwM-IcXsc-ofmapcuutletry..stanford.edu/~knuth/mmix.htmlforbasicinfor- Copyright©2011byPearsonEducation,Inc. aoAprlrnloytberpyicrgtoahehntdisybbirmtyeeesdaceonrrpvseye,prdri.eogldheutPc,ctrtriiaononntnidec,d,psetimrnoemrcitahshgaseeniiioUcnnnaailm,tureesdptthrioSebttveoaatcloeobssptyyasoiitfnengmeA,,dmerofrerricocotmarrd.aitnnhsgemT,ihpsiuossrbilopilnusibkhielenwiriacsapnerty.iioofrnoFrotimros informationregardingpermissions,writeto: PearsonEducation,Inc. RightsandContractsDepartment 5B0o1stBono,ylMstAon0S2t1r1ee6t,SuiFtaex9:00(617)671-3447 ISBN-13 978-0-201-03804-0 ISBN-10 0-201-03804-8 TextprintedintheUnitedStatesatCourierWestfordinWest.ford,Massachusetts. Secondprinting,March2011 PREFACE Toputallthegoodstuffintoonebookispatentlyimpossible, andattemptingeventobereasonablycomprehensive aboutcertainasp—ectsGEofRAthLeDsuBb.jeFctOLisLlAikNeDly,to“Eldeiatdort’osrCuonranwear"yg(r2o0w0t5h.) ThetitleofVolume4isCombinatorialAlgorithms,andwhenIproposedit Iwasstronglyinclinedtoaddasubtitle:TheKindofProgrammingILikeBest. Myeditorshavedecidedtotonedownsuchexuberance,butthefactremains thatprogramswithacombinatorialflavorhavealwaysbeenmyfavorites. OntheotherhandI’veoftenbeensurprisedtofindthat,inmanypeople’s minds,theword“combinatorial”islinkedwithcomputationaldifficulty.Indeed, SamuelJohnson,inhisfamousdictionaryoftheEnglishlanguage(1755),said thatthecorrespondingnoun“isnowgenerallyusedinanillsense.” Colleagues tellmetalesofwoe,inwhichtheyreportthat “thecombinatoricsofthesit- uationdefeatedus.” Whyisitthat,forme,combinatoricsarousesfeelingsof purepleasure,yetformanyothersitevokespurepanic? It’struethatcombinatorialproblemsareoftenassociatedwithhumongously largenumbers. Johnson’sdictionaryentryalsoincludedaquotefromEphraim Chambers,whohadstatedthatthetotalnumberofwordsoflength24orless, ina24-letteralphabet,is1,391,724,288,887,252,999,425,128,493,402,200. The correspondingnumberfora10-letteralphabetis11,111,111,110;andit’sonly 3905whenthenumberoflettersis5.Thusa“combinatorialexplosion”certainly doesoccurasthesizeoftheproblemgrowsfrom5to10to24andbeyond. Computingmachineshavebecometremendouslymorepowerfulthroughout mylife.AsIwritethesewords,Iknowthattheyarebeingprocessedbya“lap- top”whosespeedismorethan100,000timesfasterthanthetrustyIBMType650 computertowhichI’vededicatedthesebooks;mycurrentmachine’smemory capacityisalsomorethan100,000timesgreater.Tomorrow’scomputerswillbe evenfasterandmorecapacious.Buttheseamazingadvanceshavenotdiminished people’scravingforanswerstocombinatorialquestions;quitethecontrary.Our once-unimaginableabilitytocomp—utesorapidlyhasraisedourexpectations, andwhettedourappetiteformore because,infact,thesizeofacombinatorial problemcanincreasemorethan100,000-foldwhennsimplyincreasesby1. Combinatorialalgorithmscanbedefinedinformallyastechniquesforthe high-speedmanipulationofcombinatorialobjectssuchaspermutationsorgraphs. Wetypicallytrytofindpatternsorarrangementsthatarethebestpossibleways tosatisfycertainconstraints.Thenumberofsuchproblemsisvast,andtheart VI PREFACE ofwritingsuchprogramsisespeciallyimportantandappealingbecauseasingle goodideacansaveyearsorevencenturiesofcomputertime. Indeed,thefactthatgoodalgorithmsforcombinatorialproblemscanhavea terrificpayoffhasledtoterrificadvancesinthestateoftheart.Manyproblems thatoncewerethoughttobeintractablecannowbepolishedoffwithease,and manyalgorithmsthatoncewereknowntobegoodhavenowbecomebetter. Startingabout1970,computerscientistsbegantoexperienceaphenomenon thatwecalled“Floyd’sLemma”:Problemsthatseemedtoneedn3operations couldactuallybesolvedin()(n2):problemsthatseemedtorequiren2couldbe handledinO(nlogn);andnlognwasoftenreducibleto0(n). Moredifficult problemssawareductioninrunningtimefrom0(2”)to0(1.5”)to0(1.3"), etc. Otherproblemsremaineddifficultingeneral,buttheywerefoundtohave importantspecialcasesthataremuchsimpler. Manycombinatorialquestions thatIoncethoughtwouldneverbeansweredduringmylifetimehavenowbeen resolved,andthosebreakthroughshavebeenduemainlytoimprovementsin algorithmsratherthantoimprovementsinprocessorspeeds. By1975,suchresearchwasadvancingsorapidlythatasubstantialfraction ofthepaperspublishedinleadingjournalsofcomputerscienceweredevoted tocombinatorialalgorithms. Andtheadvancesweren’tbeingmadeonlyby peopleinthecoreofcomputerscience;significantcontributionswerecoming fromworkersinelectricalengineering,artificialintelligence,operationsresearch, mathematics, physics, statistics, andotherfields. Iwastryingtocomplete Volume4ofTheArtofComputerProgramming,butinsteadIfeltlikeIwas sittingonthelidofaboilingkettle: Iwasconfrontedwithacombinatorial explosionofanotherkind,aprodigiousexplosionofnewideas! Thisseriesofbookswasbornatthebeginningof1962, whenInaively wroteoutalistoftentativechaptertitlesfora12-chapterbook. Atthattime Idecidedtoincludeabriefchapteraboutcombinatorialalgorithms,justfor fun. “Heylook,mostpeopleusecomputerstodealwithnumbers,butwecan alsowriteprogramsthatdealwithpatterns.” Inthosedaysitwaseasytogive afairlycompletedescriptionofjustabouteverycombinatorialalgorithmthat wasknown. Andevenby1966,whenI’dfinishedafirstdraftofabout3000 handwrittenpagesforthatalready-overgrownbook,fewerthan100ofthose pagesbelongedtoChapter7.IhadabsolutelynoideathatwhatI’dforeseenas asortof“saladcourse”wouldeventuallyturnouttobethemaindish. Thegreatcombinatorialfermentationof1975hascontinuedtochurn,as moreandmorepeoplehavebeguntoparticipate.Newideasimproveuponthe olderones,butrarelyreplacethemormakethemobsolete. SoofcourseI’ve hadtoabandonanyhopesthatIoncehadofbeingabletosurroundthefield, towriteadefinitivebookthatsetseverythinginorderandprovidesone-stop shoppingforeveryonewhohascombinatorialproblemstosolve. Thearrayof applicabletechniqueshasmushroomedtothepointwhereIcanalmostnever dmiussctussresatrsiucbttompyiscelafntdoseaxy,pla“iHnerien’gstthheemfoisnatlisomlpuotritoan:ntenprdinocfipslteosry.t”hatInssteeeamd,toI underliealloftheefficientcombinatorialmethodsthatI’veencounteredsofar.