Table Of Content

BeyondtheWorst-Case Analysis of Algorithms There are no silver bullets in algorithm design, and no single algorithmic idea is powerful and flexible enough to solve every computational problem. Nor are there silver bullets in algorithm analysis, as the most enlightening method for analyzing an algorithm often depends on the problem and the application. However, typical algorithmscoursesrelyalmostentirelyonasingleanalysisframework,thatof worst- caseanalysis,whereinanalgorithmisassessedbyitsworstperformanceonanyinput of agivensize. The purpose of this book is to popularize several alternatives to worst-case analysis and their most notable algorithmic applications, from clustering to linear programmingtoneuralnetworktraining.Fortyleadingresearchershavecontributed introductionstodifferentfacetsof thisfield,emphasizingthemostimportantmodels andresults,manyof whichcanbetaughtinlecturestobeginninggraduatestudents intheoreticalcomputerscienceandmachinelearning. Tim Roughgarden is a professor of computer science at Columbia University. For his research, he has been awarded the ACM Grace Murray Hopper Award, the Presidential Early Career Award for Scientists and Engineers (PECASE), the KalaiPrizeinComputerScienceandGameTheory,theSocialChoiceandWelfare Prize, the Mathematical Programming Society’s Tucker Prize, and the EATCS- SIGACTGödelPrize.Hewasaninvitedspeakeratthe2006InternationalCongress of Mathematicians, the Shapley Lecturer at the 2008 World Congress of the Game TheorySociety,andaGuggenheimFellowin2017.HisotherbooksincludeTwenty Lectures on Algorithmic Game Theory (2016) and the Algorithms Illuminated book series(2017–2020). Beyond the Worst-Case Analysis of Algorithms Editedby Tim Roughgarden ColumbiaUniversity,NewYork UniversityPrintingHouse,CambridgeCB28BS,UnitedKingdom OneLibertyPlaza,20thFloor,NewYork,NY10006,USA 477WilliamstownRoad,PortMelbourne,VIC3207,Australia 314–321,3rdFloor,Plot3,SplendorForum,JasolaDistrictCentre,NewDelhi–110025,India 79AnsonRoad,#06–04/06,Singapore079906 CambridgeUniversityPressispartoftheUniversityofCambridge. ItfurtherstheUniversity’smissionbydisseminatingknowledgeinthepursuitof education,learning,andresearchatthehighestinternationallevelsofexcellence. www.cambridge.org Informationonthistitle:www.cambridge.org/9781108494311 DOI:10.1017/9781108637435 ©CambridgeUniversityPress2021 Thispublicationisincopyright.Subjecttostatutoryexception andtotheprovisionsofrelevantcollectivelicensingagreements, noreproductionofanypartmaytakeplacewithoutthewritten permissionofCambridgeUniversityPress. Firstpublished2021 PrintedintheUnitedKingdombyTJBooksLimited,PadstowCornwall AcataloguerecordforthispublicationisavailablefromtheBritishLibrary. ISBN978-1-108-49431-1Hardback CambridgeUniversityPresshasnoresponsibilityforthepersistenceoraccuracyof URLsforexternalorthird-partyinternetwebsitesreferredtointhispublication anddoesnotguaranteethatanycontentonsuchwebsitesis,orwillremain, accurateorappropriate. Contents Preface pagexiii Listof Contributors xv 1 Introduction 1 TimRoughgarden 1.1 TheWorst-CaseAnalysisof Algorithms 1 1.2 FamousFailuresandtheNeedforAlternatives 3 1.3 Example:ParameterizedBoundsinOnlinePaging 8 1.4 Overviewof theBook 12 1.5 Notes 20 PARTONE REFINEMENTSOFWORST-CASEANALYSIS 2 ParameterizedAlgorithms 27 FedorV.Fomin,DanielLokshtanov,SaketSaurabh,andMeiravZehavi 2.1 Introduction 27 2.2 Randomization 31 2.3 StructuralParameterizations 34 2.4 Kernelization 35 2.5 HardnessandOptimality 39 2.6 Outlook:NewParadigmsandApplicationDomains 42 2.7 TheBigPicture 46 2.8 Notes 47 3 FromAdaptiveAnalysistoInstanceOptimality 52 JérémyBarbay 3.1 CaseStudy1:MaximaSets 52 3.2 CaseStudy2:Instance-OptimalAggregationAlgorithms 60 3.3 Surveyof AdditionalResultsandTechniques 64 3.4 Discussion 65 3.5 SelectedOpenProblems 66 3.6 KeyTakeaways 67 3.7 Notes 68 v CONTENTS 4 ResourceAugmentation 72 TimRoughgarden 4.1 OnlinePagingRevisited 72 4.2 Discussion 75 4.3 SelfishRouting 77 4.4 SpeedScalinginScheduling 81 4.5 LooselyCompetitiveAlgorithms 86 4.6 Notes 89 PARTTWO DETERMINISTICMODELSOFDATA 5 PerturbationResilience 95 KonstantinMakarychevandYuryMakarychev 5.1 Introduction 95 5.2 CombinatorialOptimizationProblems 98 5.3 DesigningCertifiedAlgorithms 101 5.4 Examplesof CertifiedAlgorithms 106 5.5 Perturbation-ResilientClusteringProblems 108 5.6 Algorithmfor2-Perturbation-ResilientInstances 111 5.7 (3+ε)-CertifiedLocalSearchAlgorithmfork-Medians 113 5.8 Notes 115 6 ApproximationStabilityandProxyObjectives 120 AvrimBlum 6.1 IntroductionandMotivation 120 6.2 DefinitionsandDiscussion 121 6.3 Thek-MedianProblem 125 6.4 k-Means,Min-Sum,andOtherClusteringObjectives 132 6.5 ClusteringApplications 133 6.6 NashEquilibria 134 6.7 TheBigPicture 135 6.8 OpenQuestions 136 6.9 Relaxations 137 6.10 Notes 137 7 SparseRecovery 140 EricPrice 7.1 SparseRecovery 140 7.2 ASimpleInsertion-OnlyStreamingAlgorithm 142 7.3 HandlingDeletions:LinearSketchingAlgorithms 143 7.4 UniformAlgorithms 148 7.5 LowerBound 154 7.6 DifferentMeasurementModels 155 7.7 MatrixRecovery 158 7.8 Notes 160 vi CONTENTS PARTTHREE SEMIRANDOMMODELS 8 DistributionalAnalysis 167 TimRoughgarden 8.1 Introduction 167 8.2 Average-CaseJustificationsof ClassicalAlgorithms 171 8.3 Good-on-AverageAlgorithmsforEuclideanProblems 175 8.4 RandomGraphsandPlantedModels 179 8.5 RobustDistributionalAnalysis 183 8.6 Notes 184 9 IntroductiontoSemirandomModels 189 UrielFeige 9.1 Introduction 189 9.2 WhyStudySemirandomModels? 192 9.3 SomeRepresentativeWork 196 9.4 OpenProblems 209 10 SemirandomStochasticBlockModels 212 AnkurMoitra 10.1 Introduction 212 10.2 RecoveryviaSemidefiniteProgramming 215 10.3 RobustnessAgainstaMonotoneAdversary 218 10.4 InformationTheoreticLimitsof ExactRecovery 219 10.5 PartialRecoveryandBelief Propagation 221 10.6 RandomversusSemirandomSeparations 223 10.7 AboveAverage-CaseAnalysis 226 10.8 SemirandomMixtureModels 230 11 Random-OrderModels 234 AnupamGuptaandSahilSingla 11.1 Motivation:PickingaLargeElement 234 11.2 TheSecretaryProblem 237 11.3 Multiple-SecretaryandOtherMaximizationProblems 238 11.4 MinimizationProblems 247 11.5 RelatedModelsandExtensions 250 11.6 Notes 254 12 Self-ImprovingAlgorithms 259 C.Seshadhri 12.1 Introduction 259 12.2 InformationTheoryBasics 263 12.3 TheSelf-ImprovingSorter 266 12.4 Self-ImprovingAlgorithmsfor2DMaxima 272 12.5 MoreSelf-ImprovingAlgorithms 277 12.6 Critiqueof theSelf-ImprovingModel 278 vii CONTENTS PARTFOUR SMOOTHEDANALYSIS 13 SmoothedAnalysisof LocalSearch 285 BodoManthey 13.1 Introduction 285 13.2 SmoothedAnalysisof theRunningTime 286 13.3 SmoothedAnalysisof theApproximationRatio 301 13.4 DiscussionandOpenProblems 304 13.5 Notes 305 14 SmoothedAnalysisof theSimplexMethod 309 DanielDadushandSophieHuiberts 14.1 Introduction 309 14.2 TheShadowVertexSimplexMethod 310 14.3 TheSuccessiveShortestPathAlgorithm 315 14.4 LPswithGaussianConstraints 319 14.5 Discussion 329 14.6 Notes 330 15 SmoothedAnalysisof ParetoCurvesinMultiobjectiveOptimization 334 HeikoRöglin 15.1 AlgorithmsforComputingParetoCurves 334 15.2 Numberof Pareto-optimalSolutions 342 15.3 SmoothedComplexityof BinaryOptimizationProblems 352 15.4 Conclusions 354 15.5 Notes 355 PARTFIVE APPLICATIONSINMACHINELEARNING ANDSTATISTICS 16 NoiseinClassification 361 Maria-FlorinaBalcanandNikaHaghtalab 16.1 Introduction 361 16.2 Model 362 16.3 TheBestCaseandtheWorstCase 363 16.4 Benefitsof AssumptionsontheMarginalDistribution 365 16.5 Benefitsof AssumptionsontheNoise 374 16.6 FinalRemarksandCurrentResearchDirections 378 17 RobustHigh-DimensionalStatistics 382 IliasDiakonikolasandDanielM.Kane 17.1 Introduction 382 17.2 RobustMeanEstimation 384 17.3 BeyondRobustMeanEstimation 396 17.4 Notes 399 viii