Sequential and Parallel Algorithms and Data Structures: The Basic Toolbox PDF
Preview Sequential and Parallel Algorithms and Data Structures: The Basic Toolbox
Peter Sanders Kurt Mehlhorn Martin Dietzfelbinger Roman Dementiev Sequential and Parallel Algorithms and Data Structures The Basic Toolbox Sequential and Parallel Algorithms and Data Structures Peter Sanders • Kurt Mehlhorn Martin Dietzfelbinger • Roman Dementiev Sequential and Parallel Algorithms and Data Structures The Basic Toolbox Peter Sanders Kurt Mehlhorn Karlsruhe Institute of Technology Max Planck Institute for Informatics Karlsruhe, Germany Saarbrücken, Germany M artin Dietzfelbinger Roman Dementiev Technische Universität Ilmenau Intel Deutschland GmbH Ilmenau, Germany Feldkirchen, Germany Intel material (including descriptions of algorithms and their implementations, text, figures, and tables describing their experimental evaluation and experimental results on Intel processors, and sample code) is reproduced by permission of Intel Corporation. ISBN 978-3-030-25208-3 ISBN 978-3-030-25209-0 (eBook) https://doi.org/10.1007/978-3-030-25209-0 © Springer Nature Switzerland AG 2019 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland To all algorithmicists Preface An algorithmis a preciseand unambiguousrecipefor solvinga classof problems. If formulated as programs, they can be executed by a machine. Algorithms are at the heart of every nontrivial computer application. Therefore, every computer sci- entistandeveryprofessionalprogrammershouldknowaboutthebasic algorithmic toolbox:structures thatallow efficient organizationand retrievalof data, key algo- rithmsforproblemsongraphs,andgenerictechniquesformodeling,understanding, andsolvingalgorithmicproblems.Thisbookisa conciseintroductiontothisbasic toolbox,intendedforstudentsandprofessionalsfamiliarwithprogrammingandba- sic mathematicallanguage.Althoughthe presentationisconcise,wehaveincluded manyexamples,pictures,informalexplanations,andexercises,andsomelinkageto the real world.The bookcoversthe materialtypicallytaughtin undergraduateand first-yeargraduatecoursesonalgorithms. Mostchaptershavethesamebasicstructure.Webeginbydiscussingaproblem asitoccursinareal-lifesituation.Weillustratethemostimportantapplicationsand thenintroducesimplesolutionsasinformallyaspossibleandasformallyasneces- sary toreallyunderstandtheissuesathand.Whenwemovetomoreadvancedand optionalissues,thisapproachgraduallyleadstoamoremathematicaltreatment,in- cluding theorems and proofs. Thus, the book should work for readers with a wide range of mathematical expertise. There are also advanced sections (marked with a *) where we recommend that readers should skip them on first reading. Exercises provideadditionalexamples,alternativeapproachesandopportunitiestothinkabout theproblems.Itishighlyrecommendedtotakealookattheexercisesevenifthere isnotimetosolvethemduringthefirstreading.Inordertobeabletoconcentrateon ideasratherthanprogrammingdetails,weusepictures,words,andhigh-levelpseu- docodetoexplainouralgorithms.Asection“ImplementationNotes”linkstheseab- stractideastoclean,efficientimplementationsinrealprogramminglanguagessuch asC++andJava.Eachchapterendswithasectiononfurtherfindingsthatprovides aglimpseatthestateoftheart,generalizations,andadvancedsolutions. Thefirst edition ofthis booktreatedonly sequentialalgorithmsand data struc- tures. Today,almosteverycomputer,be it a desktop,a notebook,or a smartphone, has multiple cores, and sequential machines have become an exotic species. The viii Preface reasonforthischangeis thatsequentialprocessorshaveceased togetproportional performanceimprovementsfromincreasedcircuitcomplexity.Althoughthenumber of transistorsin an integratedcircuit(still) doubleseverytwo years(Moore’slaw), the only reasonable way to use this transistor budget is to put multiple processor coresonachip.Theconsequenceisthatnowadayseveryperformance-criticalappli- cationhastobeparallelized.Moreover,bigdata–theexplosionofdatasetsizesin manyapplications–hasproducedanenormousdemandforalgorithmsthatscaleto alargenumberofprocessors.Thisparadigmshifthasprofoundeffectsonteaching algorithms.Parallelalgorithmsarenolongeraspecializedtopicreservedforasmall percentageofstudents.Rather,everystudentneedssomeexposuretoparallelalgo- rithms,andparallelsolutionparadigmsneedtobetaughtearlyon.Asaconsequence, parallelalgorithmsshouldbeintegratedtightlyandearlyintoalgorithmscourses.We therefore decided to include parallel algorithms in the second edition of the book. Each chapter now has some sections on parallel algorithms. The goals remain the same as for the first edition: a careful balance between simplicity and efficiency, between theory and practice, and between classical results and the forefrontof re- search.We use a slightlydifferentstyle forthesectionson parallelcomputing.We includeconcreteprogrammingexamplesbecauseparallelprogrammingisstillmore difficultthansequentialprogramming(theprogramsareavailableatgithub.com/ basic-toolbox-sample-code/basic-toolbox-sample-code/).We also referenceoriginalwork directlyin the textinstead of in the sectionon history becausetheparallelpartisclosertocurrentresearch. Algorithmicsisamodernandactiveareaofcomputerscience,evenatthelevel of the basic toolbox. We have made sure that we present algorithms in a modern way, including explicitly formulated invariants. We also discuss important further aspects,suchasalgorithmengineering,memoryhierarchies,algorithmlibraries,and certifyingalgorithms. Wehavechosentoarrangemostofthematerialbyproblemdomainandnotby solutiontechnique.Thechapteronoptimizationtechniquesisanexception.Wefind thatan organizationby problemdomainallowsa moreconcise presentation.How- ever,itisalsoimportantthatreadersandstudentsobtainagoodgraspoftheavailable techniques. Therefore, we have structured the optimization chapter by techniques, and an extensive index providescross-referencesbetween differentapplicationsof thesametechnique.Boldpagenumbersintheindexindicatethepageswherecon- ceptsaredefined. Thisbookcanbeusedinmultiplewaysinteaching.Wehaveusedthefirstedition ofthebookandadraftversionofthesecondeditioninundergraduateandgraduate coursesonalgorithmics.Inafirstyearundergraduatecourse,weconcentratedonthe sequentialpartofthe book.Ina secondalgorithmscourseatan advancedbachelor levelormaster’slevelandwithstudentswithsomeexperienceinparallelprogram- ming we made most of the sequential part of the book a prerequisite and concen- trated on the more advancedmaterial such as the starred sections and the parts on externalmemoryandparallelalgorithms.Ifa firstalgorithmscourseis taughtlater intheundergraduatecurriculum,thebookcanbeusedtoteachparallelandsequen- tialalgorithmsinanintegratedway.Althoughwehaveincludedsomematerialabout Preface ix parallelprogrammingandseveralconcreteprogrammingexamples,theparallelpart of the book works best for readers who already have some backgroundin parallel programming.Anotherapproachistousethebooktoprovideconcretealgorithmic contentfora parallelprogrammingcoursethatusesanotherbookforthe program- ming part. Last but notleast, the book should be usefulfor independentstudy and toprofessionalswhohaveabasicknowledgeofalgorithmsbutlessexperiencewith parallelism. Followusonanexcitingtourthroughtheworldofalgorithms. Ilmenau,Saarbrücken,Karlsruhe,Heidelberg MartinDietzfelbinger August2018 KurtMehlhorn PeterSanders RomanDementiev Afirsteditionofthisbookwaspublishedin2008.Sincethenthebookhasbeen translatedintoChinese,Greek,Japanese,andGerman.MartinDietzfelbingertrans- latedthebookintoGerman.Actually,hedidmuchmorethanatranslation.Hethor- oughlyrevisedthebookandimprovedthepresentationatmanyplaces.Healsocor- rectedanumberofmistakes.Thus,thebookgainedthroughthetranslation,andwe decidedtomaketheGermaneditionthereferenceforanyfutureeditions.Itisonly natural that we asked Martin to become an author of the German edition and any futureeditionsofthebook. Saarbrücken,Karlsruhe KurtMehlhorn March2014 PeterSanders SoonafterthepublicationoftheGermanedition,westartedworkingonthere- visedEnglishedition.Wedecidedtoexpandthebookintotheparallelworldforthe reasons indicated in the preface. Twenty years ago, parallel machines were exotic, nowadays,sequentialmachinesareexotic.However,theparallelworldismuchmore diverseandcomplexthanthesequentialworld,andthereforealgorithm-engineering issues becomemoreimportant.We concludedthatwe hadto goallthe wayto im- plementationsandexperimentalevaluationsforsomeoftheparallelalgorithms.We invitedRomanDementievto workwithusonthealgorithmengineeringaspectsof parallelcomputing.RomanreceivedhisPhDin2006forathesison“AlgorithmEn- gineeringforLargeDataSets”.HenowworksforIntel,whereheisresponsiblefor performanceengineeringofamajordatabasesystem. Ilmenau,Saarbrücken,Karlsruhe MartinDietzfelbinger November2017 KurtMehlhorn PeterSanders Contents 1 Appetizer:IntegerArithmetic ................................... 1 1.1 Addition................................................. 2 1.2 Multiplication:TheSchoolMethod .......................... 6 1.3 ResultChecking .......................................... 10 1.4 ARecursiveVersionoftheSchoolMethod .................... 11 1.5 KaratsubaMultiplication ................................... 13 1.6 ParallelMultiplication ..................................... 15 1.7 AlgorithmEngineering..................................... 17 1.8 ThePrograms ............................................ 18 1.9 ProofsofLemma1.7andTheorem1.9 ....................... 21 1.10 ImplementationNotes...................................... 23 1.11 HistoricalNotesandFurtherFindings ........................ 23 2 Introduction................................................... 25 2.1 AsymptoticNotation....................................... 26 2.2 TheSequentialMachineModel.............................. 29 2.3 Pseudocode .............................................. 32 2.4 ParallelMachineModels ................................... 38 2.5 ParallelPseudocode ....................................... 44 2.6 DesigningCorrectAlgorithmsandPrograms .................. 46 2.7 AnExample–BinarySearch................................ 49 2.8 BasicAlgorithmAnalysis .................................. 52 2.9 Average-CaseAnalysis..................................... 57 2.10 Parallel-AlgorithmAnalysis ................................ 62 2.11 RandomizedAlgorithms.................................... 63 2.12 Graphs .................................................. 68 2.13 PandNP ................................................ 73 2.14 ImplementationNotes...................................... 77 2.15 HistoricalNotesandFurtherFindings ........................ 79 xii Contents 3 RepresentingSequencesbyArraysandLinkedLists ............... 81 3.1 ProcessingArraysinParallel................................ 82 3.2 LinkedLists.............................................. 86 3.3 ProcessingLinkedListsinParallel........................... 91 3.4 UnboundedArrays ........................................ 97 3.5 *AmortizedAnalysis ...................................... 102 3.6 StacksandQueues ........................................ 106 3.7 ParallelQueue-LikeDataStructures.......................... 109 3.8 ListsversusArrays ........................................ 113 3.9 ImplementationNotes...................................... 114 3.10 HistoricalNotesandFurtherFindings ........................ 116 4 HashTablesandAssociativeArrays.............................. 117 4.1 HashingwithChaining..................................... 120 4.2 UniversalHashing......................................... 122 4.3 HashingwithLinearProbing................................ 128 4.4 ChainingversusLinearProbing ............................. 130 4.5 *PerfectHashing.......................................... 131 4.6 ParallelHashing .......................................... 134 4.7 ImplementationNotes...................................... 147 4.8 HistoricalNotesandFurtherFindings ........................ 149 5 SortingandSelection ........................................... 153 5.1 SimpleSorters............................................ 156 5.2 Simple,Fast,andInefficientParallelSorting................... 158 5.3 Mergesort–anO(nlogn)SortingAlgorithm .................. 160 5.4 ParallelMergesort......................................... 162 5.5 ALowerBound........................................... 165 5.6 Quicksort ................................................ 168 5.7 ParallelQuicksort ......................................... 173 5.8 Selection................................................. 178 5.9 ParallelSelection.......................................... 180 5.10 BreakingtheLowerBound ................................. 182 5.11 *ParallelBucketSortandRadixSort ......................... 186 5.12 *ExternalSorting ......................................... 187 5.13 ParallelSampleSortwithImplementations.................... 191 5.14 *ParallelMultiwayMergesort............................... 201 5.15 ParallelSortingwithLogarithmicLatency..................... 205 5.16 ImplementationNotes...................................... 207 5.17 HistoricalNotesandFurtherFindings ........................ 208
The list of books you might like
