Data Structures and Algorithm Analysis

601 Pages·2013·3.43 MB·English

Data Structures and Algorithm
Analysis
Edition 3.2 (Java Version)
Clifford A. Shaffer
DepartmentofComputerScience
VirginiaTech
Blacksburg,VA24061
March28,2013
Update3.2.0.10
Foralistofchanges,see
http://people.cs.vt.edu/˜shaffer/Book/errata.html
Copyright©2009-2012byCliffordA.Shaffer.
ThisdocumentismadefreelyavailableinPDFformforeducationaland
othernon-commercialuse. Youmaymakecopiesofthisfileand
redistributeinelectronicformwithoutcharge. Youmayextractportionsof
thisdocumentprovidedthatthefrontpage,includingthetitle,author,and
thisnoticeareincluded. Anycommercialuseofthisdocumentrequiresthe
writtenconsentoftheauthor. Theauthorcanbereachedat
[email protected].
Ifyouwishtohaveaprintedversionofthisdocument,printcopiesare
publishedbyDoverPublications
(seehttp://store.doverpublications.com/0486485811.html).
Furtherinformationaboutthistextisavailableat
http://people.cs.vt.edu/˜shaffer/Book/.
Contents
Preface xiii
I Preliminaries 1
1 DataStructuresandAlgorithms 3
1.1 APhilosophyofDataStructures 4
1.1.1 TheNeedforDataStructures 4
1.1.2 CostsandBenefits 6
1.2 AbstractDataTypesandDataStructures 8
1.3 DesignPatterns 12
1.3.1 Flyweight 13
1.3.2 Visitor 13
1.3.3 Composite 14
1.3.4 Strategy 15
1.4 Problems,Algorithms,andPrograms 16
1.5 FurtherReading 18
1.6 Exercises 20
2 MathematicalPreliminaries 23
2.1 SetsandRelations 23
2.2 MiscellaneousNotation 27
2.3 Logarithms 29
2.4 SummationsandRecurrences 30
2.5 Recursion 34
2.6 MathematicalProofTechniques 36
iii
iv Contents
2.6.1 DirectProof 37
2.6.2 ProofbyContradiction 37
2.6.3 ProofbyMathematicalInduction 38
2.7 Estimation 44
2.8 FurtherReading 45
2.9 Exercises 46
3 AlgorithmAnalysis 53
3.1 Introduction 53
3.2 Best,Worst,andAverageCases 59
3.3 AFasterComputer,oraFasterAlgorithm? 60
3.4 AsymptoticAnalysis 63
3.4.1 UpperBounds 63
3.4.2 LowerBounds 65
3.4.3 ΘNotation 66
3.4.4 SimplifyingRules 67
3.4.5 ClassifyingFunctions 68
3.5 CalculatingtheRunningTimeforaProgram 69
3.6 AnalyzingProblems 74
3.7 CommonMisunderstandings 75
3.8 MultipleParameters 77
3.9 SpaceBounds 78
3.10 SpeedingUpYourPrograms 80
3.11 EmpiricalAnalysis 83
3.12 FurtherReading 84
3.13 Exercises 85
3.14 Projects 89
II FundamentalDataStructures 91
4 Lists,Stacks,andQueues 93
4.1 Lists 94
4.1.1 Array-BasedListImplementation 97
4.1.2 LinkedLists 100
4.1.3 ComparisonofListImplementations 108
Contents v
4.1.4 ElementImplementations 111
4.1.5 DoublyLinkedLists 112
4.2 Stacks 117
4.2.1 Array-BasedStacks 117
4.2.2 LinkedStacks 120
4.2.3 ComparisonofArray-BasedandLinkedStacks 121
4.2.4 ImplementingRecursion 121
4.3 Queues 125
4.3.1 Array-BasedQueues 125
4.3.2 LinkedQueues 128
4.3.3 ComparisonofArray-BasedandLinkedQueues 131
4.4 Dictionaries 131
4.5 FurtherReading 138
4.6 Exercises 138
4.7 Projects 141
5 BinaryTrees 145
5.1 DefinitionsandProperties 145
5.1.1 TheFullBinaryTreeTheorem 147
5.1.2 ABinaryTreeNodeADT 149
5.2 BinaryTreeTraversals 149
5.3 BinaryTreeNodeImplementations 154
5.3.1 Pointer-BasedNodeImplementations 154
5.3.2 SpaceRequirements 160
5.3.3 ArrayImplementationforCompleteBinaryTrees 161
5.4 BinarySearchTrees 163
5.5 HeapsandPriorityQueues 170
5.6 HuffmanCodingTrees 178
5.6.1 BuildingHuffmanCodingTrees 179
5.6.2 AssigningandUsingHuffmanCodes 185
5.6.3 SearchinHuffmanTrees 188
5.7 FurtherReading 188
5.8 Exercises 189
5.9 Projects 192
6 Non-BinaryTrees 195
vi Contents
6.1 GeneralTreeDefinitionsandTerminology 195
6.1.1 AnADTforGeneralTreeNodes 196
6.1.2 GeneralTreeTraversals 197
6.2 TheParentPointerImplementation 199
6.3 GeneralTreeImplementations 206
6.3.1 ListofChildren 206
6.3.2 TheLeft-Child/Right-SiblingImplementation 206
6.3.3 DynamicNodeImplementations 207
6.3.4 Dynamic“Left-Child/Right-Sibling”Implementation 210
6.4 K-aryTrees 210
6.5 SequentialTreeImplementations 212
6.6 FurtherReading 215
6.7 Exercises 215
6.8 Projects 218
III SortingandSearching 221
7 InternalSorting 223
7.1 SortingTerminologyandNotation 224
7.2 ThreeΘ(n2)SortingAlgorithms 225
7.2.1 InsertionSort 225
7.2.2 BubbleSort 227
7.2.3 SelectionSort 229
7.2.4 TheCostofExchangeSorting 230
7.3 Shellsort 231
7.4 Mergesort 233
7.5 Quicksort 236
7.6 Heapsort 243
7.7 BinsortandRadixSort 244
7.8 AnEmpiricalComparisonofSortingAlgorithms 251
7.9 LowerBoundsforSorting 253
7.10 FurtherReading 257
7.11 Exercises 257
7.12 Projects 261
Contents vii
8 FileProcessingandExternalSorting 265
8.1 PrimaryversusSecondaryStorage 265
8.2 DiskDrives 268
8.2.1 DiskDriveArchitecture 268
8.2.2 DiskAccessCosts 272
8.3 BuffersandBufferPools 274
8.4 TheProgrammer’sViewofFiles 282
8.5 ExternalSorting 283
8.5.1 SimpleApproachestoExternalSorting 285
8.5.2 ReplacementSelection 288
8.5.3 MultiwayMerging 290
8.6 FurtherReading 295
8.7 Exercises 295
8.8 Projects 299
9 Searching 301
9.1 SearchingUnsortedandSortedArrays 302
9.2 Self-OrganizingLists 307
9.3 BitVectorsforRepresentingSets 313
9.4 Hashing 314
9.4.1 HashFunctions 315
9.4.2 OpenHashing 320
9.4.3 ClosedHashing 321
9.4.4 AnalysisofClosedHashing 331
9.4.5 Deletion 334
9.5 FurtherReading 335
9.6 Exercises 336
9.7 Projects 338
10 Indexing 341
10.1 LinearIndexing 343
10.2 ISAM 346
10.3 Tree-basedIndexing 348
10.4 2-3Trees 350
10.5 B-Trees 355
10.5.1 B+-Trees 358
viii Contents
10.5.2 B-TreeAnalysis 364
10.6 FurtherReading 365
10.7 Exercises 365
10.8 Projects 367
IV AdvancedDataStructures 369
11 Graphs 371
11.1 TerminologyandRepresentations 372
11.2 GraphImplementations 376
11.3 GraphTraversals 380
11.3.1 Depth-FirstSearch 383
11.3.2 Breadth-FirstSearch 384
11.3.3 TopologicalSort 384
11.4 Shortest-PathsProblems 388
11.4.1 Single-SourceShortestPaths 389
11.5 Minimum-CostSpanningTrees 393
11.5.1 Prim’sAlgorithm 393
11.5.2 Kruskal’sAlgorithm 397
11.6 FurtherReading 399
11.7 Exercises 399
11.8 Projects 402
12 ListsandArraysRevisited 405
12.1 Multilists 405
12.2 MatrixRepresentations 408
12.3 MemoryManagement 412
12.3.1 DynamicStorageAllocation 414
12.3.2 FailurePoliciesandGarbageCollection 421
12.4 FurtherReading 425
12.5 Exercises 426
12.6 Projects 427
13 AdvancedTreeStructures 429
13.1 Tries 429
Contents ix
13.2 BalancedTrees 434
13.2.1 TheAVLTree 435
13.2.2 TheSplayTree 437
13.3 SpatialDataStructures 440
13.3.1 TheK-DTree 442
13.3.2 ThePRquadtree 447
13.3.3 OtherPointDataStructures 451
13.3.4 OtherSpatialDataStructures 453
13.4 FurtherReading 453
13.5 Exercises 454
13.6 Projects 455
V TheoryofAlgorithms 459
14 AnalysisTechniques 461
14.1 SummationTechniques 462
14.2 RecurrenceRelations 467
14.2.1 EstimatingUpperandLowerBounds 467
14.2.2 ExpandingRecurrences 470
14.2.3 DivideandConquerRecurrences 472
14.2.4 Average-CaseAnalysisofQuicksort 474
14.3 AmortizedAnalysis 476
14.4 FurtherReading 479
14.5 Exercises 479
14.6 Projects 483
15 LowerBounds 485
15.1 IntroductiontoLowerBoundsProofs 486
15.2 LowerBoundsonSearchingLists 488
15.2.1 SearchinginUnsortedLists 488
15.2.2 SearchinginSortedLists 490
15.3 FindingtheMaximumValue 491
15.4 AdversarialLowerBoundsProofs 493
15.5 StateSpaceLowerBoundsProofs 496
15.6 FindingtheithBestElement 499
x Contents
15.7 OptimalSorting 501
15.8 FurtherReading 504
15.9 Exercises 504
15.10Projects 507
16 PatternsofAlgorithms 509
16.1 DynamicProgramming 509
16.1.1 TheKnapsackProblem 511
16.1.2 All-PairsShortestPaths 513
16.2 RandomizedAlgorithms 515
16.2.1 Randomizedalgorithmsforfindinglargevalues 515
16.2.2 SkipLists 516
16.3 NumericalAlgorithms 522
16.3.1 Exponentiation 523
16.3.2 LargestCommonFactor 523
16.3.3 MatrixMultiplication 524
16.3.4 RandomNumbers 526
16.3.5 TheFastFourierTransform 527
16.4 FurtherReading 532
16.5 Exercises 532
16.6 Projects 533
17 LimitstoComputation 535
17.1 Reductions 536
17.2 HardProblems 541
17.2.1 TheTheoryofNP-Completeness 543
17.2.2 NP-CompletenessProofs 547
17.2.3 CopingwithNP-CompleteProblems 552
17.3 ImpossibleProblems 555
17.3.1 Uncountability 556
17.3.2 TheHaltingProblemIsUnsolvable 559
17.4 FurtherReading 561
17.5 Exercises 562
17.6 Projects 564
Bibliography 567
Contents xi
Index 573