Eliminating Adverse Control Plane Interactions in

129 Pages·2017·8.67 MB·English

Eliminating Adverse Control Plane
Interactions in Independent
Network Systems
Matthew K. Mukerjee
May 2018
CMU-CS-18-106
ComputerScienceDepartment
SchoolofComputerScience
CarnegieMellonUniversity
Pittsburgh,PA
ThesisCommittee:
SrinivasanSeshan,Chair
VyasSekar
PeterSteenkiste
BruceMaggs(Duke;Akamai)
Submittedinpartialfulfillmentoftherequirements
forthedegreeofDoctorofPhilosophy.
Copyright©2018MatthewK.Mukerjee.
ThisworkwassupportedbytheNationalScienceFoundationundergrantsnumberedCNS-1040801,CNS-1314721,CNS-1345305,andCNS-1565343.
Theviewsandconclusionscontainedinthisdocumentarethoseoftheauthorandshouldnotbeinterpretedasrepresentingtheofficialpolicies,
eitherexpressedorimplied,ofanysponsoringinstitution,theU.S.government,oranyotherentity.
Keywords: networks,controlplane,splitcontrol,controlcompetition,informationsharing,
interfaces, reconfigurable datacenters, datacenter optics, content brokering, CDNs, live video
delivery
ToSophie
forputtingupwith
thislong,crazyride.
iv
Abstract
Networksystemoperationistypicallydividedintocontrolanddataplanes—while
thedataplaneisresponsibleforprocessingindividualmessagesorpackets,thecontrol
planecomputestheconfigurationofdevicesandoptimizessystem-wideperformance.
Unfortunately,thecontrolplaneofeachnetworksystemorprotocollayertypically
operatesindependently(e.g.,CDNsselectingserverswithoutcoordinatingwithISPs),
resultinginpoorinteractionsbetweencontrolplanesacrosssystems.Wecancategorize
existingsystemsintooneoffourgeneralcontrolplanecoordinationmechanismsthat
overcometheseproblems,basedonhowmuchinformationcanbesharedbetween
controlplanes. Ifnoinformationcanbeshared,controlplanessimplyreacttodata
planechangesasarudimentaryformofcoordination(e.g.,CDNserverselection+ISP
trafficengineering).Ifallinformationcanbeshared,transparencyindecisionmaking
can remove most poor interactions (e.g., Coflow datacenter flow scheduling). In
manyscenarios,however,onlysomeinformationcanbeshared(e.g.,betweencontrol
planes running in different companies). Coordination in these scenarios is more
specialized;controlplaneswithseparatedataplaneresourcescanusepriorityranking
(i.e.,providingalistofpreferencesforresourceswithoutneedingtoshowhowthese
preferencesweredecided;e.g.,BGProutingbetweenISPs),andcontrolplaneswith
shared data plane resources can use hierarchicalpartitioning (i.e., making coarse-
graineddecisionsglobally,andfine-graineddecisionslocally;e.g.,internet-wideBGP
+OSPFrouting). Whilesystemsutilizingcontrolplanecoordinationexisttoday,they
havebeendesignedadhoc. Weproposeasetofrecipesthatshowwhenit’sappropriate
to use different coordination mechanisms, based on key properties (information
sharingandsharedresources)invariedscenarios(layering,administrativeseparation,
andinternet-scalesystems).
Weusetheserecipestoguidesystemdesigninavarietyofcontexts,asacasestudy
incontrolplanecoordination. First,manysystemsuselayerseparationformodularity,
butindoingsotradeperformanceforgenerality. Aslayeredsystemshavenoinfor-
mationsharingconstraints,wearguethattransparency(i.e.,cross-layeroptimization;
allowinglayerstorunspecializedcodetobetteruseotherlayers)isthecorrecttech-
niqueforregainingperformance. WeexplorethiswithouremulatorEtalon[116],in
thecontextofreconfigurabledatacenters. Second,somesystemsareadministratively
separate(i.e.,aresplitacrossdifferentcompanies),limitinginformationsharingfor
business reasons. These systems may overcome adverse interactions using priority
ranking. We explore this with VDX [114, 115], in the context of content brokering.
Finally,systemsneedinginternetscalabilityareincreasinglycombiningaslowcentral-
izedcontrolplanewithafastdistributedcontrolplane. Completeinformationsharing
(andthus,transparency)wouldappearpossible,buttimescaleseparationmakesthis
fundamentallyimpractical. Instead,hierarchicalpartitioningcanovercomethechal-
lengespresentinthisscenario. WeexplorethiswithVDN[117],inthecontextoflive
videodelivery. Throughthiscasestudywefindthatthesecoordinationmechanism
notonlysolveavarietyofproblems,butcanbeefficientlyimplemented.
vi
Acknowledgments
FirstandforemostIwouldliketothankmygirlfriendSophieforalwaysencourag-
ingmetopushforwardonwhatIwanttoworkonandprovidinginfiniteemotional
support. Iwouldalsoliketothankmyfamily(Mom,Dad,Amanda,andAustin)for
theirloveandemotionalsupport.
Iwouldliketothankmyadvisor,SriniSeshan,forsevenyearsoffuntacklinga
largevarietyofinterestingproblems,aswellasgivingmetheneededencouragement
tokeepsubmittingpapersuntiltheygetin. Iwouldalsoliketothankthefacultyon
mycommittee(PeterSteenkiste,VyasSekar,andBruceMaggs)forallthehelpand
feedbackthatwentintothisdissertation,aswellasfuncollaborationsthroughoutthe
years.
I’dliketothankmyundergradandmastersadvisors,AndrewCampbell,Tanzeem
Choudhury,andDanielFreedman,forpushingmetowardsresearchandeventuallya
PhD.
I’dliketothankmymentorsatGoogle, BenGreenstein, MikePiatek,andMatt
Welsh,forafunSummerinternship.
I’dliketothanktheresearchandsupportstaffinCSDandXIA,DebCavlovich,
Angie Miller, Kathy McNiff, Jenn Landefeld, Angy Malloy, Dan Barrett, and Nitin
Gupta, for making sure everything just works and minimizing a lot of the pain of
gettingaPhD.
I would like to thank all my friends, collaborators, and mentors: David Naylor,
NicoFeltman,LiliEhrlich,SusuandRayKanemoto,EvanShimizu,AlexLloydand
Dana Daugherty, Lauren and Logan Plath, Ravi Choudhuri, Elizabeth, Matt, and
ArgusGriffin,KrisSalada,MitchCairns,andMichaelaNachtigall,YuZhao,Lindsey
andVinceSlaugh,BrianRussman,RichardWangandYvonneYu,YuchenWu,Alex
Poms, John Wright, Hyeontaek Lim, David Witmer, Junchen Jiang, Bruno Vavala,
YingJungChen,EunjinLee,NadiBozkurt,ConglongLi,DevdeepRay,RanyshaWare,
AthulaBalachandran,GeorgeNychis,WolfRichter,RajaSambasivan,DongsuHan,
JustineSherry,HuiZhang,DaveAndersen,MichaelKaminsky,GeorgePorter,Alex
Snoeren, other people working under XIA, and the CMU systems seminar folks. I
apologizeifIforgotanyone.
Finally,I’dliketothanktheanonymousreviewersforSIGCOMM,NSDI,CoNEXT,
andHotNetsfortheirfeedbackondraftsoftheincludedwork.
The work presented in this dissertation is based on work in submission to
SIGCOMM 2018 [116], appearing in CoNEXT 2017 [115], HotNets 2016 [114], and
SIGCOMM 2015 [117], and builds on work appearing in CoNEXT 2015 [98] and
ANCS2017 [95].
viii
Contents
1 Introduction 1
1.1 CoordinationMechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Understandingdesignsforsplitcontrolplanesystems . . . . . . . . . . . . . . . . . 8
1.2.1 Salientfeatures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.2.2 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.2.3 DesignSpace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.3 CommonScenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.4 Recipesforcontrolplanecoordination . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.5 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.6 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.7 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2 ControlPlanesinDifferentLayers: ACaseStudyinReconfigurableDatacenters 19
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.2 Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.2.1 NetworkModel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.2.2 ComputingSchedules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.2.3 ScheduleExecution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.2.4 Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.3 Etalon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.3.2 SoftwareSwitch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.3.3 TimeDilation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.3.4 Testbed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.3.5 Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.4 Overcomingrapidbandwidthfluctuationwithdynamicbufferresizing . . . . . . 27
2.4.1 Understandingtheproblem . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.4.2 Dynamicbufferresizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.4.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.4.4 Incorporatingexplicitnetworkfeedback . . . . . . . . . . . . . . . . . . . . 31
2.4.5 Delaysensitivityanalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.5 OvercomingpoordemandestimationwithADUs . . . . . . . . . . . . . . . . . . . 32
2.5.1 Understandingproblemswithdemandestimation . . . . . . . . . . . . . . 32
ix
2.5.2 UsingADUs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.5.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.5.4 Cumulativeresults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.6 Overcomingdifficult-to-scheduleworkloadswith
application-specificchanges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.6.1 HDFSwriteplacementdifficultiesandsolutions . . . . . . . . . . . . . . . 37
2.6.2 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.7 Relatedwork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3 ControlPlanesinDifferentBusinesses: ACaseStudyinContentBrokering 43
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.2 ContentDelivery: ThePastandthePresent . . . . . . . . . . . . . . . . . . . . . . . 46
3.2.1 TraditionalContentDelivery . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.2.2 BrokersandDeliveryToday . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.3 PotentialProblemsandOpportunities . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.3.1 Traces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.3.2 PotentialProblemsforaCDN . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.3.3 PotentialProblemsforaBroker . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.4 ExploringtheDesignSpace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.4.1 GeneralizingDesigns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.4.2 DesignSpace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.5 NarrowingtheDesignSpace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.5.1 SimulationOverview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.5.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.6 VDXinDetail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.6.1 DecisionProtocolDetails . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.6.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.6.3 ChallengesandLimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.7 EvaluatingaMarketplaceDesign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.7.1 DataDriven . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.7.2 Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.7.3 Microbenchmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.8 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.9 Relatedwork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4 ControlPlanesinInternet-ScaleSystems: ACaseStudyinLiveVideo 65
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4.2.1 Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
4.2.2 Designgoals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.2.3 Caseforcentralizedoptimization . . . . . . . . . . . . . . . . . . . . . . . . 71
4.2.4 Caseforhybridcontrol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
x