Concise Algorithmics: The Basic Toolbox

124 Pages·1.048 MB·English

by  Mehlhorn K., Sanders P.

Entwurf-Entwurf-Entwurf-Entwurf-Entwurf-Entwurf-Entwurf-Entwurf
Concise Algorithmics
or
Algorithms and Data Structures —
The Basic Toolbox
or...
Kurt Mehlhorn and Peter Sanders
Entwurf-Entwurf-Entwurf-Entwurf-Entwurf-Entwurf-Entwurf-Entwurf
iii
Mehlhorn,SandersJune11,2005
Foreword
Buymenot[25].
iv v
Mehlhorn,SandersJune11,2005
Contents
1 AmuseGeule: IntegerArithmetics 3
1.1 Addition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Multiplication: TheSchoolMethod . . . . . . . . . . . . . . . . . . 4
1.3 ARecursiveVersionoftheSchoolMethod. . . . . . . . . . . . . . . 6
1.4 KaratsubaMultiplication . . . . . . . . . . . . . . . . . . . . . . . . 8
1.5 ImplementationNotes. . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.6 FurtherFindings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2 Introduction 13
2.1 AsymptoticNotation . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2 MachineModel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3 Pseudocode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.4 DesigningCorrectPrograms . . . . . . . . . . . . . . . . . . . . . . 23
2.5 BasicProgramAnalysis. . . . . . . . . . . . . . . . . . . . . . . . . 25
2.6 AverageCaseAnalysisandRandomizedAlgorithms . . . . . . . . . 29
2.7 DataStructuresforSetsandSequences. . . . . . . . . . . . . . . . . 32
2.8 Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.9 ImplementationNotes. . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.10 FurtherFindings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3 RepresentingSequencesbyArraysandLinkedLists 39
3.1 UnboundedArrays . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.2 LinkedLists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.3 StacksandQueues . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.4 ListsversusArrays . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.5 ImplementationNotes. . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.6 FurtherFindings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
vi CONTENTS CONTENTS vii
4 HashTables 59 9 GraphTraversal 157
4.1 HashingwithChaining . . . . . . . . . . . . . . . . . . . . . . . . . 62 9.1 BreadthFirstSearch . . . . . . . . . . . . . . . . . . . . . . . . . . 158
4.2 UniversalHashFunctions . . . . . . . . . . . . . . . . . . . . . . . . 63 9.2 DepthFirstSearch . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
4.3 HashingwithLinearProbing . . . . . . . . . . . . . . . . . . . . . . 67
9.3 ImplementationNotes. . . . . . . . . . . . . . . . . . . . . . . . . . 165
4.4 ChainingVersusLinearProbing . . . . . . . . . . . . . . . . . . . . 70
9.4 FurtherFindings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
4.5 ImplementationNotes. . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.6 FurtherFindings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
10 ShortestPaths 167
5 SortingandSelection 75 10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
5.1 SimpleSorters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 10.2 ArbitraryEdgeCosts(Bellman-FordAlgorithm). . . . . . . . . . . . 171
5.2 Mergesort—anO(nlogn)Algorithm . . . . . . . . . . . . . . . . . 80 10.3 AcyclicGraphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
5.3 ALowerBound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 10.4 Non-NegativeEdgeCosts(Dijkstra’sAlgorithm) . . . . . . . . . . . 173
5.4 Quicksort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 10.5 MonotoneIntegerPriorityQueues . . . . . . . . . . . . . . . . . . . 176
5.5 Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
10.6 AllPairsShortestPathsandPotentialFunctions . . . . . . . . . . . . 181
5.6 BreakingtheLowerBound . . . . . . . . . . . . . . . . . . . . . . . 92
10.7 ImplementationNotes. . . . . . . . . . . . . . . . . . . . . . . . . . 182
5.7 ExternalSorting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
10.8 FurtherFindings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
5.8 ImplementationNotes. . . . . . . . . . . . . . . . . . . . . . . . . . 98
5.9 FurtherFindings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
11 MinimumSpanningTrees 185
6 PriorityQueues 105 11.1 SelectingandDiscardingMSTEdges . . . . . . . . . . . . . . . . . 186
6.1 BinaryHeaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 11.2 TheJarn´ık-PrimAlgorithm . . . . . . . . . . . . . . . . . . . . . . . 187
6.2 AddressablePriorityQueues . . . . . . . . . . . . . . . . . . . . . . 112 11.3 Kruskal’sAlgorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 188
6.3 ImplementationNotes. . . . . . . . . . . . . . . . . . . . . . . . . . 120 11.4 TheUnion-FindDataStructure . . . . . . . . . . . . . . . . . . . . . 190
6.4 FurtherFindings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
11.5 ImplementationNotes. . . . . . . . . . . . . . . . . . . . . . . . . . 191
11.6 FurtherFindings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
7 SortedSequences 123
7.1 BinarySearchTrees . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
12 GenericApproachestoOptimization 195
7.2 Implementationby(a;b)-Trees . . . . . . . . . . . . . . . . . . . . . 128
7.3 MoreOperations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 12.1 LinearProgramming—ABlackBoxSolver. . . . . . . . . . . . . . 196
7.4 AugmentingSearchTrees. . . . . . . . . . . . . . . . . . . . . . . . 138 12.2 GreedyAlgorithms—NeverLookBack . . . . . . . . . . . . . . . . 199
7.5 ImplementationNotes. . . . . . . . . . . . . . . . . . . . . . . . . . 140 12.3 DynamicProgramming—BuildingitPiecebyPiece . . . . . . . . . 201
7.6 FurtherFindings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 12.4 SystematicSearch—IfinDoubt,UseBruteForce . . . . . . . . . . 204
12.5 LocalSearch—ThinkGlobally,ActLocally . . . . . . . . . . . . . 207
8 GraphRepresentation 147
12.6 EvolutionaryAlgorithms . . . . . . . . . . . . . . . . . . . . . . . . 214
8.1 EdgeSequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
12.7 ImplementationNotes. . . . . . . . . . . . . . . . . . . . . . . . . . 217
8.2 AdjacencyArrays—StaticGraphs. . . . . . . . . . . . . . . . . . . 149
12.8 FurtherFindings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
8.3 AdjacencyLists—DynamicGraphs . . . . . . . . . . . . . . . . . . 150
8.4 AdjacencyMatrixRepresentation . . . . . . . . . . . . . . . . . . . 151
8.5 ImplicitRepresentation . . . . . . . . . . . . . . . . . . . . . . . . . 152 13 Summary: ToolsandTechniquesforAlgorithmDesign 219
8.6 ImplementationNotes. . . . . . . . . . . . . . . . . . . . . . . . . . 153 13.1 GenericTechniques . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
8.7 FurtherFindings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 13.2 DataStructuresforSets . . . . . . . . . . . . . . . . . . . . . . . . . 220
viii CONTENTS CONTENTS 1
A Notation 225 [amusegeulearithmetik. BildvonAlChawarizmi] =
(
A.1 GeneralMathematicalNotation. . . . . . . . . . . . . . . . . . . . . 225
A.2 SomeProbabilityTheory . . . . . . . . . . . . . . . . . . . . . . . . 227
A.3 UsefulFormulas. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
Bibliography 230
2 CONTENTS 3
Mehlhorn,SandersJune11,2005
Chapter 1
Amuse Geule: Integer
Arithmetics
Weintroduceourreadersintothedesign,analysis,andimplementationofalgorithms
by studying algorithms for basic arithmetic operations on large integers. We treat
addition and multiplication in the text and leave division and square roots for the
exercises.
Integerarithmeticisinterestingformanyreasons:
Arithmetic on long integers is needed in applications like cryptography, geo-
(cid:15)
metriccomputing,andcomputeralgebra.
Wearefamiliarwiththeproblemandknowalgorithmsforadditionandmulti-
(cid:15)
plication. Wewillseethatthehighschoolalgorithmforintegermultiplication
isfarfromoptimalandthatmuchbetteralgorithmsexist.
Wewilllearnbasicanalysistechniquesinasimplesetting.
(cid:15)
Wewilllearnbasicalgorithmengineeringtechniquesinasimplesetting.
(cid:15)
Wewillseetheinterplaybetweentheoryandexperimentinasimplesetting.
(cid:15)
Weassumethatintegersarerepresentedasdigit-strings(digitszeroandoneinour
theoretical analysis and larger digits in our programs) and that two primitive opera-
tionsareavailable:theadditionofthreedigitswithatwodigitresult(thisissometimes
calledafulladder)andthemultiplicationoftwodigitswithaonedigitresult.Wewill
measurethe efficiencyof our algorithmsbythe numberofprimitiveoperationsexe-
cuted.
4 AmuseGeule: IntegerArithmetics 1.2Multiplication: TheSchoolMethod 5
We assume throughout this section that a andb are n-digitintegers. We refer to n T
thedigitsofaasa toa witha beingthemostsignificant(alsocalledleading) 40000 0.3
n 1 0 n 1
digitanda beingt(cid:0)heleastsignifica(cid:0)ntdigit. [consistentlyreplaced bit (was used 80000 1.18
0
= mixedwithdigit)bydigit] 160000 4.8
)
320000 20.34
1.1 Addition
Table1.1: Therunningtimeoftheschoolmethodforthemultiplicationofn-bitinte-
We allknowhowtoaddtwointegersaandb. We simplywritethemontopofeach gers. Therunningtimegrowsquadratically.
otherwiththeleastsignificantdigitsalignedandsumdigit-wise,carryingasinglebit
= fromonepositiontothenext.[picture!]
)
= [todo: properalignment of numbers in tables]Table1.1showstheexecution
c=0 : Digit // Variableforthecarrydigit )
time of the school method using a C++ implementation and 32 bit digits. The time
fori:=0ton 1do adda,b,andctoforms andanewcarryc
(cid:0) i i i givenistheaverageexecutiontimeover??? manyrandominputsona??? machine.
s =c
n The quadratic growth of the running time is clearly visible: Doubling n leads to a
four-foldincreaseinrunningtime. Wecaninterpretthetableindifferentways:
Weneedoneprimitiveoperationforeachpositionandhenceatotalofnprimitive
operations. (1) We can take the table to confirm our theoretical analysis. Our analysis pre-
dicts quadratic growth and we are measuring quadratic growth. However, we ana-
Lemma1.1 Twon-digitintegerscanbeaddedwithnprimitiveoperations. lyzed the number of primitive operations and we measured running time on a ???
computer.Ouranalysisconcentratesonprimitiveoperationsondigitsandcompletely
ignores all book keeping operations and all questions of storage. The experiments
1.2 Multiplication: The School Method
showthat thisabstraction isausefulone. We willfrequentlyonlyanalyze thenum-
berof“representativeoperations”. Ofcourse,thechoiceofrepresentativeoperations
= [picture!] We allknowhowtomultiplytwointegers. Inthissectionwe willreview requiresinsightandknowledge.InSection2.2wewillintroduceamorerealisticcom-
)
themethodfamiliartoallofus,inlatersectionswewillgettoknowamethodwhich putermodeltohaveabasisforabstraction. Wewilldeveloptoolstoanalyzerunning
issignificantlyfasterforlargeintegers. timeofalgorithmsonthismodel. Wewillalsoconnectourmodeltorealmachines,so
Theschoolmethodforintegermultiplicationworksasfollows:Wefirstformpar- thatwecantakeouranalysisasapredictorofactualperformance. Wewillinvestigate
tialproducts pi bymultiplyingawiththei-thdigitbi ofbandthensumthesuitably the limits of our theory. Under what circumstances arewe goingto concede that an
alignedproducts pi 2itoobtaintheproductofaandb. experimentcontradictstheoreticalanalysis?
(cid:1)
(2) We can use the table to strengthen our theoretical analysis. Our theoretical
p=0 : (cid:0)
fori:=0tondo p:=a b 2i+p analysis tells us that the running time grows quadratically in n. From our table we
i
(cid:1) (cid:1) may conclude that the running time on a ??? is approximately??? n2 seconds. We
(cid:1)
Letusanalyzethenumberofprimitiveoperationsrequiredbytheschoolmethod. can use thisknowledgeto predict the runningtime for our program onother inputs.
Weneednprimitivemultiplicationstomultiplyabybiandhenceatotalofn n=n2 [todo: redonumbers.] Herearethreesampleoutputs. Forn=100000,therunning =
(cid:1) (
primitive multiplications. All intermediate sums are at most 2n-digit integers and timeis1:85secondsandtheratiois1:005,forn=1000,therunningtimeis0:0005147
henceeachiterationsneedsatmost2nprimitiveadditions. Thusthereareatmost2n2 secondsandtheratiois2:797,forn=200,therunningtimeis3:3 10 5secondsand
(cid:0)
(cid:1)
primitiveadditions. theratiois4:5,andforn=1000000,therunningtimeis263:6secondsandtheratio
is1:433. Weseethatourpredictionscanbefaroff. Wesimplyweretoocareless. We
Lemma1.2 Theschoolmethodmultipliestwon-digitintegerswithnomorethan3n2 started withthe assumptionthat therunningis cn2 for someconstantc, estimatedc,
primitiveoperations. andthenpredicted.Startingfromtheassumptionthattherunningtimeiscn2+dn+e
6 AmuseGeule: IntegerArithmetics 1.3ARecursiveVersionoftheSchoolMethod 7
wouldhaveleadustodifferentconclusions.Weneedtodoourtheorymorecarefully1
in order to be able to predict. Also, when we made the prediction that the running
time is approximately ??? 10 10 n2 seconds, we did not make any restrictions on Figure1.1: todo: visuzlizationoftheschoolmethodanditsrecursivevariant.
(cid:0)
(cid:1) (cid:1)
n. However, my computer has a finite memory (albeit large) and this memory is
organized into a complex hierarchy of registers, first and second level cache, main
andhence
memory, and disk memory. The access times to the different levels of the memory
a b=a b 22k+(a b +a b ) 2k+a b :
differwidelyandthiswillhaveaneffectonrunningtimes. Noneoftheexperiments (cid:1) 1(cid:1) 1(cid:1) 1(cid:1) 0 0(cid:1) 1 (cid:1) 0(cid:1) 0
reportedsofarrequiresourprogramtousediskmemory.Weshouldanalyzethespace Thisformulasuggeststhefollowingalgorithmforcomputinga b:
(cid:1)
requirement ofour programin order to beable topredict for howlargea valueofn
the program is able “to run in core2”. In our example, we ran into both traps. The a) Splitaandbintoa1,a0,b1,andb0.
predictionis offfor smalln becausewe ignoredthe linearandconstantterms in the
b) Computethefourproductsa b ,a b ,a b ,anda b .
1 1 1 0 0 1 0 0
running time and the predictionis offfor largen becauseour predictionignores the (cid:1) (cid:1) (cid:1) (cid:1)
effectsofthememoryhierarchy. c) Addthesuitablyalignedproductstoobtaina b.
(cid:1)
(3) We can use the table to conjecture quadratic growth of the running time of
Observe that the numbers a , a , b , and b are n=2 -bit numbers and hence the
our algorithm. Again, we need to be careful. What are we actually conjecturing? 1 0 1 0 d e
multiplications in step (2) are simpler than the original multiplication if n=2 <n,
Thattherunningtimegrowsquadraticallyonrandomnumbers? Afterall,weranthe d e
i.e.,n>1. Thecompletealgorithmisnowasfollows:Tomultiply1-bitnumbers,use
algorithmonlyonrandomnumbers(andevenonveryfewofthem). Thattherunning
our multiplication primitive, and to multiply n-bit numbers for n 2, use the three
timegrowsquadraticallyintheworstcase,i.e.,thattherearenoinstancesthatleadto (cid:21)
stepapproachabove.[picture!] =
higherthanquadraticgrowth? Thattherunninggrowsquadraticallyinthebestcase, (
Itisclearwhythisapproachiscalleddivide-and-conquer.Wereducetheproblem
i.e.,thattherearenoinstancesthatleadtolessthanquadraticgrowth?Weseethatwe
ofmultiplyinga bintosomenumberofsimplerproblemsofthesamekind.Adivide
needtodevelopmoreconceptsandaricherlanguage. (cid:1)
and conquer algorithm always consists of three parts: In the first part, we split the
[droppedcheckingfornow. Ifweuseitweshoulddomoreinthatdirection
originalproblemintosimplerproblemsofthesamekind(ourstep(1)),inthesecond
= later,e.g.,pqs,flows,sorting]
) partwe solvethesimplerproblemsusingthe samemethod(ourstep(2)), andin the
third part, we obtain the solution to the original problem from the solutions to the
subproblems. The following program implements the divide-and-conquer approach
1.3 A Recursive Version of the School Method
tointegermultiplication.
Whatistheconnectionofourrecursiveintegermultiplicationtotheschoolmethod?
We derivea recursiveversionofthe schoolmethod. This willbeour firstencounter Itisreallythesame.Figure??showsthattheproductsa b ,a b ,a b ,anda b
1 1 1 0 0 1 0 0
(cid:1) (cid:1) (cid:1) (cid:1)
ofthedivide-and-conquerparadigm,oneofthefundamentalparadigmsinalgorithm arealsocomputedbytheschoolmethod.Knowingthatourrecursiveintegermultipli-
design. Wewillalsolearninthissectionthatconstantfactorscanmakeasignificant cationisjusttheschoolmethodindisguisetellsusthattherecursivealgorithmsuses
difference. a quadraticnumberofprimitiveoperations. Let us also derivethisfrom first princi-
Letaandbbeourtwonbit-integerswhichwewanttomultiply. Letk= n=2 . ples. Thiswillallowustointroducerecurrencerelations,apowerfulconceptforthe
b c
We splitaintotwo numbersa anda ; a consistsofthek leastsignificantbitsand analysisofrecursivealgorithm.
1 0 0
a consistsofthen kmostsignificantbits. Then
1
(cid:0)
Lemma1.3 LetT(n)bethemaximalnumberofprimitiveoperationsrequiredbyour
a=a 2k+a and b=b 2k+b recursivemultiplicationalgorithmwhenappliedton-bitintegers.Then
1 0 1 0
(cid:1) (cid:1)
1Maybethisisnotwhatyouwantedtoread,butitisthetruth. 1 ifn=1;
2Mainmemorywascalledcorememoryinancienttimes(whenoneoftheauthorsstudiedcomputer T(n)(cid:20)(4 T( n=2 )+3 2 n ifn 2:
science). (cid:1) d e (cid:1) (cid:1) (cid:21)
8 AmuseGeule: IntegerArithmetics 1.4KaratsubaMultiplication 9
Proof: Multiplying two 1-bit numbers requires one primitive multiplication. This compared to multiplications and hence saving a multiplication more than outweighs
justifies the casen=1. So assume n 2. Splittinga andb intothe four pieces a , threeadditionaladditions. Weobtainthefollowingalgorithmforcomputinga b:
1
a , b , and b requires no primitive op(cid:21)erations3. Each piece has at most n=2 bits (cid:1)
0 1 0 a) Splitaandbintoa ,a ,b ,andb .
d e 1 0 1 0
and hence the four recursive multiplications require at most 4 T( n=2 ) primitive
operations. Finally, we needthreeadditions to assemble thefina(cid:1)l redsult.eEachaddi- b) Computethethreeproductsp2=a1 b1, p0=a0 b0,and p1=(a1+a0) (b1+
(cid:1) (cid:1) (cid:1)
tioninvolvestwonumbersofatmost2nbitsandhencerequiresatmost2nprimitive b0).
operations. Thisjustifiestheinequalityforn 2.
c) Addthesuitablyalignedproductstoobtaina b,i.e.,computea baccordingto
(cid:21)
theformulaa b=p 22k+(p p p ) 2(cid:1)k+p . (cid:1)
2 1 2 0 0
InSection2.5wewilllearnthatsuchrecurrencesareeasytosolveandyieldthe (cid:1) (cid:1) (cid:0) (cid:0) (cid:1)
Thenumbersa ,a ,b ,b ,a +a ,andb +b are n=2 +1-bitnumbersandhence
alreadyconjecturedquadraticexecutiontimeoftherecursivealgorithm. Atleastifn 1 0 1 0 1 0 1 0 d e
themultiplicationsinstep(2)aresimplerastheoriginalmultiplicationif n=2 +1<
isapoweroftwowegeteventhesameconstantfactors. [exercise: inductionproof d e
n,i.e.,n 4. Thecompletealgorithmisnowasfollows: Tomultiply3-bitnumbers,
= fornpoweroftwo?] [someexplanationhowthisintroducestheconceptforthe (cid:21)
) use the school method, and to multiply n-bit numbers for n 4, use the three step
= nextsection?] (cid:21)
) approachabove.
Table??showstherunningtimeoftheKaratsubamethodincomparisonwiththe
1.4 Karatsuba Multiplication school method. We also show the ratio of the running times (last column) and the
ratiototherunningtimeintheprecedingiteration,i.e.,theratiobetweentherunning
timefor2n-bitintegersandtherunningtimeonn-bitintegers.Weseethattheratiois
= [checknamesandcitations]
) aroundthreeincaseoftheKaratsubamethod(doublingthesizeofthenumbersabout
In1962theSovietmathematicianKaratsuba[51]discoveredafasterwayofmulti-
plyinglargeintegers.Therunningtimeofhisalgorithmgrowslikenlog3 n1:58. The triplestherunningtime)andisaboutfourincaseoftheschoolmethod(doublingthe
(cid:25) sizeofthenumbersaboutquadruplestherunningtime). Thelatterstatementrequires
method is surprisingly simple. Karatsuba observed that a simple algebraic identity
some phantasy. We also see that the Karatsuba-method looses on short integers but
allowsonetosaveonemultiplicationinthedivide-and-conquerimplementation,i.e.,
winsonverylargeintegers.
one can multiply n-bit numbers using only three(!!) multiplications of integers half
Thelessonstorememberare:
thesize.
Thedetailsareasfollows.Letaandbbeourtwonbit-integerswhichwewantto Betterasymptoticsultimatelywins. However,itmaylooseonsmallinputs.
(cid:15)
multiply. Letk= n=2 . Wesplitaintotwonumbersa anda ;a consistsofthek
b c 1 0 0 An asymptotically slower algorithm can be faster on small inputs. You may
leastsignificantbitsanda1consistsofthen(cid:0)kmostsignificantbits. Then (cid:15) onlybeinterestedinsmallinputs.
a=a 2k+a and b=b 2k+b ItistimetoderivetheasymptoticsoftheKaratsubamethod.
1 0 1 0
(cid:1) (cid:1)
andhence(themagicisinthesecondequality) Lemma1.4 LetT(n)bethemaximalnumberofprimitiveoperationsrequiredbythe
Karatsubaalgorithmwhenappliedton-bitintegers.Then
a b = a b 22k+(a b +a b ) 2k+a b
(cid:1) 1(cid:1) 1(cid:1) 1(cid:1) 0 0(cid:1) 1 (cid:1) 0(cid:1) 0 3n2 ifn 3;
= a b 22k+((a +a ) (b +b ) a b a b ) 2k+a b T(n) (cid:20)
1 1 1 0 1 0 1 1 0 0 0 0
(cid:1) (cid:1) (cid:1) (cid:0) (cid:1) (cid:0) (cid:1) (cid:1) (cid:1) (cid:20)(3 T( n=2 +1)+6 2 n ifn 4:
(cid:1) d e (cid:1) (cid:1) (cid:21)
At first sight, we have only made things more complicated. A second look shows Proof: Multiplying two n-bit numbers with the school method requires no more
thatthelastformulacanbeevaluatedwithonlythreemultiplications,namely,a1 b1, than3n2 primitiveoperationsbyLemma 1.2. Thisjustifies thefirstline. So assume
(cid:1)
tah1e(cid:1)rbe0c,uarnsdiv(eai1m+plae0m)(cid:1)e(nbta1t+iobn0o)f.tWheesaclhsooonlemedetshixoda.dTdhiteioknesy.iTshtahtaitsatdhdrieteiomnsoarerethcahneaipn nop(cid:21)era4t.ioSnpsl4i.ttinEgacahapnidecbeianntodtthheefsouumrspiaece+saa1,aan0d,bb1,+anbdbh0avreeqautirmesosntopnr=i2mi+tiv1e
0 1 0 1
d e
3Itwillrequirework,butitisworkthatwedonotaccountforinouranalysis. 4Itwillrequirework,butitisworkthatwedonotaccountforinouranalysis.
10 AmuseGeule: IntegerArithmetics 1.6FurtherFindings 11
n TK TK=TK0 TS TS=TS0 TK=TS [reportonswitchingtimeinLEDA]NotethatthisnumberofbitsmakestheKarat- =
(
80000 5.85 – 1.19 – 4.916 suba algorithm useful to applications in crytography where multiplying numbers up
160000 17.51 2.993 4.73 3.975 3.702 to2048bitsisthemosttimeconsumingoperationinsomeapproaches[?].
320000 52.54 3.001 19.77 4.18 2.658
640000 161 3.065 99.97 5.057 1.611
1.6 Further Findings
1280000 494 3.068 469.6 4.698 1.052
2560000 1457 2.95 1907 4.061 0.7641
IstheKaratsubamethodthefastestknownmethodforintegermultiplication? Much
5120000 4310 2.957 7803 4.091 0.5523
faster methods are known. The asymptotically most efficient algorithm is due to
Scho¨nhage and Strassen[84]. It multiplies n-bit integers in time O(nlognloglogn).
Table1.2: TherunningtimeoftheKaratsubaandtheschoolmethodforintegermul- Their method is beyond the scope of this book. [more? the generalization of
tiplication: T istherunningtimeoftheKaratsubamethodandT istherunningtime Karatzubaisabitheavy.] =
K S (
of theschoolmethod. T andT denote therunning time ofthe preceding iteration.
K0 S0
a a
Foran algorithmwith runningtimeT(n)=cn we haveT=T =T(2n)=T(n)=2 .
0
Fora =log3wehave2a =3andfora =2wehave2a =4. Thetablewasproduced
ona300MHzSUNULTRA-SPARC.
bits and hence the three recursive multiplications require at most 3 T( n=2 +1)
(cid:1) d e
primitiveoperations. Finally,weneedtwoadditionstoforma +a andb +b and
0 1 0 1
four additions to assemble the final result. Each addition involves two numbers of
atmost2nbitsandhencerequiresatmost2nprimitiveoperations. Thisjustifiesthe
inequalityforn 4.
(cid:21)
ThetechniquesintroducedinSection2.5allowustoconcludethatT(n) ???nlog3).
(cid:20)
= [fillinconstantfactorandlowerorderterms!]
)
1.5 Implementation Notes
Karatsubaintegermultiplicationissuperiortotheschoolmethodforlargeinputs. In
our implementation the superiority only shows for integers with more than several
millionbits. However,a simplerefinementcanimprovethatsignificantly. Since the
schoolmethodissuperiortoKaratsubaforshortintegersweshouldstoptherecursion
earlier and switchto the schoolmethod for numberswhich have atn bits for some
0
yet to be determinedn . Inthis way we obtain a methodwhich isneverworse than
0
eithertheschoolmethodorthenaiveKaratsubaalgorithm.
Whatisagoodchoiceforn ? Wecananswerthisquestionanalyticallyandexper-
0
imentally. Theexperimentalapproachiseasierhere: wecansimplytimetherevised
Karatsubaalgorithmfordifferentvaluesofn andthentakethevaluewhichgivesthe
0
= smallestrunningtime. Ona??? thebestresultswereobtainedforn 512.[check]
0
) (cid:25)