Algorithm Design: A Methodological Approach - 150 problems and detailed solutions

820 Pages·2023·17.364 MB·English

by  Patrick Bosc, Marc Guyomard, Laurent Miclet

(cid:105) (cid:105)
(cid:105) (cid:105)
Algorithm Design
A bestseller in its French edition, this book is original in its construction and
its success in the French market demonstrates its appeal. It is based on three
principles: (1) An organization of the chapters by families of algorithms:
exhaustive search, divide and conquer, etc. On the contrary, there is no chapter
devoted only to a systematic exposure of, say, algorithms on strings. Some of
these will be found in different chapters. (2) For each family of algorithms, an
introduction is given to the mathematical principles and the issues of a rigorous
design, with one or two pedagogical examples. (3) For the most part, the book
details 150 problems, spanning seven families of algorithms. For each problem, a
precise and progressive statement is given. More importantly, a complete solution
is detailed, with respect to the design principles that have been presented; often,
some classical errors are pointed out. Roughly speaking, two-thirds of the book
is devoted to the detailed rational construction of the solutions.
(cid:105) (cid:105)
(cid:105) (cid:105)
Taylor & Francis
Taylor & Francis Group
http://taylorandfrancis.com
i i
i i
Algorithm Design
A Methodological Approach
150 Problems and Detailed Solutions
Patrick Bosc, Marc Guyomard and Laurent Miclet
i i
i i
(cid:105) (cid:105)
(cid:105) (cid:105)
First edition published 2023
by CRC Press
6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742
and by CRC Press
4 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN
© 2023 Patrick Bosc, Marc Guyomard and Laurent Miclet
CRC Press is an imprint of Taylor and Francis Group, LLC
Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot
assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have
attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders
if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please
write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized
in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying,
microfilming, and recording, or in any information storage or retrieval system, without written permission from the
publishers.
For permission to photocopy or use material electronically from this work, access www.copyright.com or contact the
Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. For works that are
not available on CCC please contact [email protected]
Trademark notice: Product or corporate names may be trademarks or registered trademarks and are used only for
identification and explanation without intent to infringe.
ISBN: 978-1-032-36941-9 (hbk)
ISBN: 978-1-032-36939-6 (pbk)
ISBN: 978-1-003-33459-0 (ebk)
DOI: 10.1201/b23251
Typeset in URWPalladio font
by KnowledgeWorks Global Ltd.
(cid:105) (cid:105)
(cid:105) (cid:105)
(cid:105) (cid:105)
(cid:105) (cid:105)
Contents
Preface ix
1 Mathematicsandcomputerscience:someusefulnotions 1
1.1 Reasoningandproving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.1 Propositionalandpredicatecalculus . . . . . . . . . . . . . . . . . . . 1
1.1.2 Proofbycontradiction . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.3 Proofbyinductionwithoneindex . . . . . . . . . . . . . . . . . . . . 4
1.1.3.1 Proofbysimpleinduction . . . . . . . . . . . . . . . . . . . 4
1.1.3.2 Proofbypartialinduction . . . . . . . . . . . . . . . . . . . 6
1.1.3.3 Proofbystronginduction. . . . . . . . . . . . . . . . . . . . 6
1.1.4 Partitioninduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.1.5 Foundationoftheproofbyinduction . . . . . . . . . . . . . . . . . . 9
1.1.6 Proofbyinductionwithseveralindexes . . . . . . . . . . . . . . . . . 9
1.2 Recurrencerelations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.2.1 Generalities,examplesandclosedforms . . . . . . . . . . . . . . . . 10
1.2.2 Establishingandcomputingarecurrencerelation . . . . . . . . . . . 11
1.2.2.1 Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.2.2.2 Someexamples. . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.2.2.3 Designingthealgorithmassociatedwitharecurrence . . . 14
1.3 Recurrence,induction,recursion,etc. . . . . . . . . . . . . . . . . . . . . . . 15
1.4 Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.4.1 Basicnotations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.4.2 Definitionofsets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.4.3 Operationsonsets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.4.4 Specialsets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.4.5 Relations,functionsandarrays . . . . . . . . . . . . . . . . . . . . . . 18
1.4.5.1 Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.4.5.2 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.4.5.3 Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.4.6 Bags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
1.4.7 Cartesianproductandinductivestructures . . . . . . . . . . . . . . . 20
1.4.8 Stringsandsequences . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
1.5 Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
1.5.1 Directedgraphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
1.5.2 Undirectedgraphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
1.5.3 Weightedgraphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
1.6 Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
1.7 Priorityqueues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
1.7.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
1.7.2 Implementations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
1.8 FIFOandLIFOqueues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
1.9 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
v
(cid:105) (cid:105)
(cid:105) (cid:105)
(cid:105) (cid:105)
(cid:105) (cid:105)
vi Contents
1.10 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2 Complexityofanalgorithm 71
2.1 Reminders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
2.1.1 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
2.1.2 Algorithmics,complexityofanalgorithm . . . . . . . . . . . . . . . . 71
2.1.3 Minimumandmaximumcomplexityofanalgorithm . . . . . . . . . 73
2.1.4 Ordersofgrowth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
2.1.5 Someexamplesofcomplexity . . . . . . . . . . . . . . . . . . . . . . . 76
2.1.6 Aboutelementaryoperations . . . . . . . . . . . . . . . . . . . . . . . 77
2.1.7 Practicalcomputingtime . . . . . . . . . . . . . . . . . . . . . . . . . 77
2.1.8 Pseudo-polynomialproblems . . . . . . . . . . . . . . . . . . . . . . . 78
2.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
2.3 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
3 Specification,invariants,anditeration 93
3.1 Principlefortheconstructionofloopsbyinvariant . . . . . . . . . . . . . . 93
3.2 Anintroductoryexample:theEuclidiandivision . . . . . . . . . . . . . . . . 95
3.3 Usefultechniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
3.3.1 Sequentialcomposition . . . . . . . . . . . . . . . . . . . . . . . . . . 96
3.3.2 Predicatestrengtheningandweakening . . . . . . . . . . . . . . . . . 98
3.3.3 Strengtheningbytheintroductionofprogrammingvariables . . . . 98
3.4 Heuristicsforthediscoveryofinvariants . . . . . . . . . . . . . . . . . . . . 99
3.4.1 Breakupofthepostcondition . . . . . . . . . . . . . . . . . . . . . . . 99
3.4.2 Hypothesisoftheworkcarriedoutpartly . . . . . . . . . . . . . . . . 101
3.4.3 Strengtheningtheinvariant . . . . . . . . . . . . . . . . . . . . . . . . 103
3.5 Aboutboundedlinearsearch . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
3.5.1 Anexample . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
3.5.2 Specialcaseandrelatedpattern . . . . . . . . . . . . . . . . . . . . . . 109
3.6 Somekeypointstodevelopaloop . . . . . . . . . . . . . . . . . . . . . . . . 110
3.7 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
3.8 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
4 Reduceandconquer,recursion 157
4.1 Afewremindersaboutrecursion . . . . . . . . . . . . . . . . . . . . . . . . . 157
4.2 Recurrencerelationandrecursion . . . . . . . . . . . . . . . . . . . . . . . . 158
4.3 “Reduceandconquer”anditscomplexity . . . . . . . . . . . . . . . . . . . 159
4.3.1 Presentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
4.3.2 Example1:thespiralpattern . . . . . . . . . . . . . . . . . . . . . . . 162
4.3.3 Example2:Hanoitowers . . . . . . . . . . . . . . . . . . . . . . . . . 164
4.4 Whatshouldberemindedfor“Reduceandconquer” . . . . . . . . . . . . . 166
4.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
4.6 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
5 Generateandtest 197
5.1 Fundamentals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
5.1.1 Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
5.1.2 Functionsandrelatedframes . . . . . . . . . . . . . . . . . . . . . . . 202
5.1.2.1 Totalfunctions . . . . . . . . . . . . . . . . . . . . . . . . . . 202
5.1.2.2 Thecaseofpartialfunctions . . . . . . . . . . . . . . . . . . 207
5.1.3 Patternsfor“Generateandtest” . . . . . . . . . . . . . . . . . . . . . 210
(cid:105) (cid:105)
(cid:105) (cid:105)
(cid:105) (cid:105)
(cid:105) (cid:105)
Contents vii
5.1.3.1 Patternsderivedfrom“AllSolutions” . . . . . . . . . . . . . 210
5.1.3.2 Patternsderivedfrom“OptimalSolution” . . . . . . . . . . 212
5.1.3.3 Patternsderivedfrom“OneSolution” . . . . . . . . . . . . . 212
5.1.3.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
5.2 Anexample:theoptimalpartitionofanarray . . . . . . . . . . . . . . . . . 215
5.2.1 Thebasicproblem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
5.2.2 Theinitialproblemwithastrengthenedpreconditionandastrength-
enedpostcondition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
5.2.3 Theinitialproblemwithastrengthenedprecondition . . . . . . . . . 221
5.3 Whatshouldberememberedfrom“Generateandtest” . . . . . . . . . . . . 223
5.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
5.5 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
6 Branchandbound 307
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
6.1.1 Branchandbound:theprinciple . . . . . . . . . . . . . . . . . . . . . 307
6.1.1.1 Theseparationphase . . . . . . . . . . . . . . . . . . . . . . 308
6.1.1.2 Theselectionphase . . . . . . . . . . . . . . . . . . . . . . . 308
6.1.1.3 Theevaluationphase . . . . . . . . . . . . . . . . . . . . . . 309
6.1.2 Thegeneric“Branchandbound”algorithm. . . . . . . . . . . . . . . 310
6.1.2.1 Thepriorityqueue . . . . . . . . . . . . . . . . . . . . . . . . 310
6.1.2.2 Constructionofthegeneric“Branchandbound”algorithm 310
6.1.2.3 Thegeneric“Branchandbound”algorithmitself . . . . . . 311
6.1.3 Aninterestingspecialcaseforthefunctionsf andf. . . . . . . . . . 312
∗
6.1.4 Anexample:thetravelingsalesman . . . . . . . . . . . . . . . . . . . 314
6.1.4.1 Initialchoices . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
6.1.4.2 Thealgorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 314
6.1.4.3 Firstcasestudy:thezeroheuristicfunction . . . . . . . . . 316
6.1.4.4 Secondcasestudy:uniformcost . . . . . . . . . . . . . . . . 318
6.2 Whatshouldberememberedaboutthe“Branchandbound”approach . . . 320
6.3 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
6.4 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
7 Greedyalgorithms 335
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
7.1.1 Presentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
7.1.2 Provingthatagreedyalgorithmisexactoroptimal . . . . . . . . . . 336
7.1.3 Anexample:taskschedulingonaphotocopier . . . . . . . . . . . . . 336
7.1.4 The“leadrun”method . . . . . . . . . . . . . . . . . . . . . . . . . . 336
7.1.5 Proofaposteriori:thetransformationtechnique . . . . . . . . . . . . . 340
7.2 Whattorememberaboutgreedymethods . . . . . . . . . . . . . . . . . . . 341
7.3 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342
7.4 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
8 Divideandconquer 415
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
8.1.1 Presentationandprinciple . . . . . . . . . . . . . . . . . . . . . . . . . 415
8.1.2 Anexample:merge-sort . . . . . . . . . . . . . . . . . . . . . . . . . . 416
8.1.3 Generalpatternfor“Divideandconquer” . . . . . . . . . . . . . . . 418
8.1.4 Atypologyof“Divideandconquer”algorithms . . . . . . . . . . . . 419
8.1.5 Complexityof“Divideandconquer” . . . . . . . . . . . . . . . . . . 420
(cid:105) (cid:105)
(cid:105) (cid:105)
i i
i i
viii Contents
8.1.5.1 Introductiontothemastertheorem . . . . . . . . . . . . . . 420
8.1.5.2 Othertypesofrecurrenceequations . . . . . . . . . . . . . . 423
8.1.6 Aboutthesizeoftheproblem . . . . . . . . . . . . . . . . . . . . . . . 424
8.2 WhattorememberabouttheDaCapproach . . . . . . . . . . . . . . . . . . 424
8.3 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424
8.4 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491
9 Dynamicprogramming 631
9.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 631
9.2 Anexample:thelongestsub-sequencecommontotwosequences . . . . . . 635
9.3 Whatshouldberememberedtoapplydynamicprogramming . . . . . . . . 639
9.4 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 640
9.4.1 Cuttingandsharing:one-dimensionalproblems . . . . . . . . . . . . 640
9.4.2 Cuttingandsharing:two-dimensionalproblems . . . . . . . . . . . . 648
9.4.3 Graphsandtrees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 660
9.4.4 Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 673
9.4.4.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 674
9.4.4.2 Theproblem . . . . . . . . . . . . . . . . . . . . . . . . . . . 676
9.4.5 Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 681
9.4.6 Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 686
9.4.7 Pseudo-polynomialproblems . . . . . . . . . . . . . . . . . . . . . . 689
9.5 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 690
Notations 791
Listofproblems 793
Bibliography 797
Index 799
i i
i i
(cid:105) (cid:105)
(cid:105) (cid:105)
Preface
Whatthisbookis
This is a book about algorithmics, and its goal is to teach this field and to help instruc-
tors to teach it. Algorithmics is, as generally admitted, “the science and art of designing
algorithmstosolveagivenproblem,preferablyinminimaltime”.Consequently,thisbook
aimsatpresentingefficientmethodstodesignalgorithms—mainlythroughexamples.
Itisstructuredaccordingtotworules:
• Eachchapterisdevotedtoamethodologyofalgorithmdesign,whichcanbeapplied
to different data structures. For example, the chapter “Dynamic Programming” pro-
posesproblemson2Darrays,ontrees,onstrings,aswellasongraphs.Asamatterof
fact,someproblemsareproposedintwodifferentchapterssincetheycanbesolvedin
differentways—whichisparticularlyinterestingtostudy.Wehavetakencareforeach
problemtohighlightdifferentaspectsofthemethodologies:noneofthe148problems
is just a rewriting of another one. The particular interest of each problem is shortly
describedatitsbeginning.
• Eachchapterbeginsbyaninformalpresentationonanexample,thengivesthetechni-
calbasesofthemethodology,andexposesacompleteexample.Thenfollowsthemain
part,composedofdetailedproblemswiththeircompletesolution.Somewhatcomplex
problems are tackled in a progressive manner in order to highlight the constructive
process of the analysis leading to the solution. This solution is itself exhaustively de-
tailed, concluding in programs written in pseudo-code. We do not give the coding in
a specific programming language since the transcription from this pseudo-code into
anyusualoneisgenerallystraightforward.Thecorrectnessofallalgorithmshasbeen
testedthroughcodewrittenbytheauthors.
Whatthisbookisnot
This book is not a “data structures and algorithms” textbook. There are some necessary
reminders about data structures in the first chapter, and they are intensively used in the
book,buttheiractualimplementationisnotdescribed.Thereaderissupposedtobeaware
of the good packages and classes she or he can use in her or his favorite programming
language.
Itisnotatextbookonalgorithmseither,sinceitdoesnotdescribesystematicallyhow
tosolveageneralproblem,suchassortingorsubstringmatching.Itindeedexposesand
solves a great variety of problems, but all the examples have been designed to help the
reader to understand what is the efficient methodology that can be applied—not to give
anexhaustivelist.
Letusqualifyitas“anexample-basedbookonalgorithmdesignmethodologies”.
ix
(cid:105) (cid:105)
(cid:105) (cid:105)