OBSERVING AND IMPROVING THE RELIABILITY OF INTERNET LAST-MILE LINKS Aaron David ...

144 Pages·2013·7.33 MB·English

ABSTRACT
TitleofDocument: OBSERVING AND IMPROVING THE RELIABILITY OF
INTERNET LAST-MILE LINKS
Aaron David Schulman, Doctor of Philosophy, 2013
DirectedBy: Professor Neil Spring
Department of Computer Science
People rely on having persistent Internet connectivity from their homes and mobile
devices. However, unlike links in the core of the Internet, the links that connect people’s
homesandmobiledevices,knownas“last-mile”links,arenotredundant. Asaresult,the
reliabilityofanygivenlinkisofparamountconcern: whenlast-milelinksfail,peoplecan
becompletelydisconnectedfromtheInternet.
In addition to lacking redundancy, Internet last-mile links are vulnerable to failure.
Such links can fail because the cables and equipment that make up last-mile links are
exposed to the elements; for example, weather can cause tree limbs to fall on overhead
cables, and flooding can destroy underground equipment. They can also fail, eventually,
because cellular last-mile links can drain a smartphone’s battery if an application tries to
communicatewhensignalstrengthisweak.
In this dissertation, I defend the following thesis: By building on existing infrastruc-
ture, it is possible to (1) observe the reliability of Internet last-mile links across different
weather conditions and link types; (2) improve the energy efficiency of cellular Inter-
netlast-milelinks; and(3)provideanincrementallydeployable, energy-efficientInternet
last-mile downlink that is highly resilient to weather-related failures. I defend this thesis
bydesigning,implementing,andevaluatingsystems.
First, I study the reliability of last-mile links during weather events. To observe fail-
ures of last-mile links, I develop ThunderPing—a system that monitors a geographically
diverse set of last-mile links without participation from providers or customers. So far,
ThunderPinghascollected4billionpingsfrom3.5millionIPaddressesover400daysof
probing from PlanetLab hosts. Because pings may fail to solicit a response even when
a last-mile link has not failed, losses must be analyzed to determine if they constitute
last-mile link failures. Among other challenges I encountered in this project, I found that
determining the connectivity state from noisy pings is similar to finding the edges in a
noisy picture. As such, I use algorithmic edge detection to find when a host transitions
between connectivity states. By matching these connectivity states with weather reports
fromweather stationsat airports,I observehow weatheraffectslast-mile linkfailure rate
andfailureduration.
Second,Iimprovethereliabilityofcellularlinksbyreducingwastedenergy. Todoso,
IdevelopBartendr,asystemthatpredictswhenamovingsmartphonewillexperiencehigh
signalstrength. Akeychallengeistopredicthighsignalstrengthwithoutconsumingmore
energy than exploiting it would save. I also develop energy-aware scheduling algorithms
fordifferentapplicationworkloads—syncingandstreaming—basedonthesepredictions.
I evaluate the scheduling algorithms with a simulation driven by traces obtained during
actualdrives.
Third,Idesignareliablebroadcastsystemthatisinexpensivetodeploytomanyusers
and is energy-efficient to receive. I adapt reliable FM Radio Data System (RDS) broad-
casts to act as an Internet last-mile link. To accomplish this, I design and implement an
over-the-air protocol, receiver software, and a hardware bridge for incremental deploy-
ment. I implement the full end-to-endsystem, deploy it on a 3kW commercial FM radio
station in a metropolitan area, and evaluate the loss rate, energy consumption, and syn-
chronization on either a smartphone or on my new hardware bridge. The results indicate
that the full end-to-end system can be reliable, a smartphone receiver can sleep between
desired broadcasts, and two receivers tend to deliver the same broadcast within about
5ms.
OBSERVING AND IMPROVING THE RELIABILITY OF INTERNET
LAST-MILE LINKS
by
Aaron David Schulman
DissertationsubmittedtotheFacultyoftheGraduateSchoolofthe
UniversityofMaryland,CollegeParkinpartialfulfillment
oftherequirementsforthedegreeof
DoctorofPhilosophy
2013
AdvisoryCommittee:
ProfessorNeilSpring,Chair/Advisor
ProfessorBobbyBhattarcharjee
ProfessorMichaelHicks
ProfessorPrabalDutta
ProfessorMarkShayman
(cid:13)c Copyright by
Aaron David Schulman
2013
Acknowledgments
Iamtrulygratefultoallthosewhohelpedmecompletethisdissertation. IfIneglectedto
mentionyou,Iapologize. PleaseknowthatIgreatlyappreciateyoursupport.
Over the course of undergraduate and graduate school, my advisor, Neil Spring, in-
spired me to pursue an academic career doing networking research. Along the way, he
developed my thinking and gave me the skills to succeed in doing such research. Neil
significantlyraisedmyexpectationsofwritingandpresentationquality: hehelpedmere-
alize what I was capable of and gave me a strong set of criteria to evaluate my own work
and that of others. He gave me the confidence to investigate hypotheses that are difficult
to test. He taught me how to create graphs that present data in a novel way, but are still
intuitive and clear. Throughout, Neil was himself willing to learn new skills in order to
workwithmeontheresearchproblemsthatIwaspassionateabout.
Ever since I met him when I started graduate school, Dave Levin has been one of my
closest mentors. Any time I had a question, day or night, I knew Dave would eagerly
answerhisphoneandworkthroughtheproblemwithme. Asacollaborator,Davetaught
me skills in every area of graduate school. For example, he taught me how to write
explicitly,diagrambeautifully,andcookdeliciously.
BobbyBhattacharjeehasbeenaninspiringcollaborator. Hetaughtmetheimportance
ofcolorselectioninmypresentations. Healsogavemetheopportunitytoteachembedded
systemsdesigntoanamazinggroupofundergrads.
From the first moment I heard him speak, Prabal Dutta always asked the tough ques-
tions. Prabalgavemetheopportunitytospendayearandahalfwithhimandhisexcellent
students in Michigan learn electrical engineering while spending a year and half of grad-
uate school with my (now) wife. He also introduced me to Thomas Schmid. In the few
months I spent with Thomas, he taught basic electrical engineering skills such as solder-
ingcircuitboardsandadvancedskillssuchasdesigningembeddedsystems.
My wife, Ophira Vishkin, taught me how to be precise and never let me doubt my
ability. Throughout graduate school, she has been a source of serenity and brilliant edit-
ing.
I thank Mark Shayman and Michael Hicks for giving me detailed feedback on this
dissertation. Theircommentssignificantlyimprovedthequalityofit.
The Lab : Randy Baden, Adam Bender, Dave Levin, Cristian Lumezanu, Rob Sher-
1
wood, Bo Han, and Justin McCann patiently taught me the ropes of being a graduate
student. EverybodyknowsthatIwillneverforgetthelessonstheytaughtme.
TheLab : YunusBasagalar,VassiliosLekakis,MattLentz,YoundoLee,andRamakr-
2
ishnaPadmanabhanhavebeengreatfriendsandevenbettercollaborators. Theyprovided
ii
supportandencouragementduringmyproposalanddefense.
One weekend, Michelle Hugue (Meesh) asked me to drive out from Annapolis to
College Park so she could convince me to go to graduate school for a PhD (rather than a
Master’s). Shealsotoldmetoemailthetalentedyoungprofessorwhoeventuallybecame
myadvisor.
During my summer at Microsoft Research India, my collaborators, Vishnu Navda,
RamachandranRamjee,andVenkatPadmanabhan,weregreatmentorsandcollaborators.
TheygavemetheopportunitytodoresearchwhilelearningaboutIndianculture.
Wyn Bennett taught me how to program in Ms. Jellison’s computer science class
and worked with me on the two science fair projects that whet my appetite for research.
Brandi Adams provided me with excellent writing advice and gave me a reason to play
with microphones. Katrina LaCurts has been a great collaborator and is the world’s best
overseas conference roommate. My friends Dan Noble, Aaron Silverman, Chris Testa,
JoshHandelman,andPatrickShoemakeralwaysmadesureIwasenjoyingresearch. Iam
continuallyinspiredbytheircreativityandworkethic.
Myparentsgavemeunconditionalsupport,love,andencouragement. Inaddition,my
father gave me a passion for learning how things work and my mother taught me how to
speakinfrontofanaudience.
This dissertation was supported by NSF Awards CNS-0917098, CNS-0643443, and
CNS-0626629.
Finally, I would like to explicitly acknowledge the people that I collaborated with on
theworkinthisdissertation.
Chapter2: MycollaboratorswereYoundoLee,RamakrishnaPadmanabhan,andNeil
Spring. Additionally, Patrick Shoemaker and Dave Levin provided helpful discussions.
Also thanks to the anonymous reviewers and our shepherd Fabia`n Bustamante for their
comments.
Chapter3: MycollaboratorswereVishnuNavda,RamachandranRamjee,NeilSpring,
Pralhad Deshpande, Calvin Grunewald, Kamal Jain, and Venkata N. Padmanabhan. Ad-
ditionally, our shepherd Prasun Sinha, the anonymous reviewers, and the summer 2009
inters of MSR India provided insightful comments and discussion. I extend my gratitude
tothecabdriversatMicrosoftResearchIndiafortheirassistancewiththedatacollection.
Chapter 4: My collaborators were Dave Levin, Neil Spring. Additionally, Prabal
DuttaandThomasSchmidprovidedmanyusefuldiscussions.
iii
Contents
1 Introduction 1
1.1 Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 Insights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2 BackgroundandRelatedWork 13
2.1 Observinglast-milelinkfailures . . . . . . . . . . . . . . . . . . . . . . 14
2.1.1 Withprivilegeddata . . . . . . . . . . . . . . . . . . . . . . . . 14
2.1.2 Withoutprivilegeddata . . . . . . . . . . . . . . . . . . . . . . . 15
2.2 Improvingtheenergyefficiencyofsmartphonecommunication . . . . . . 17
2.2.1 Radioenergy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.2.2 Processingandpositioningenergy . . . . . . . . . . . . . . . . . 22
2.2.3 Energy-efficientcellulardatascheduling . . . . . . . . . . . . . . 23
2.3 Improvingreliabilitywithdatabroadcasting . . . . . . . . . . . . . . . . 24
2.3.1 Datacastingsystems . . . . . . . . . . . . . . . . . . . . . . . . 25
2.3.2 Broadcasttechnologies . . . . . . . . . . . . . . . . . . . . . . . 27
2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3 Weather-relatedLast-mileLinkFailures 31
3.1 MeasuringtheresponsivenessofInternethostsduringweather . . . . . . 33
3.1.1 FindingIPaddressessubjecttoweather . . . . . . . . . . . . . . 33
3.1.2 Pinging(last-milelinks)intherain . . . . . . . . . . . . . . . . 36
3.1.3 Potentialsourcesoferror . . . . . . . . . . . . . . . . . . . . . . 37
3.2 Inferringlink-statefrompings . . . . . . . . . . . . . . . . . . . . . . . 37
3.2.1 FilteringoutintermittentPlanetLabnodefailures . . . . . . . . . 39
3.2.2 Detectingfailureswithconditionalprobabilities . . . . . . . . . . 41
3.2.3 Detectingchangesinprolongedlossrate . . . . . . . . . . . . . . 44
3.2.4 UnderstandingtheHOSEDstate . . . . . . . . . . . . . . . . . . 48
3.3 Weatherhistoryandlinktype . . . . . . . . . . . . . . . . . . . . . . . . 50
3.3.1 Theweatheratahostduringaping . . . . . . . . . . . . . . . . 50
3.3.2 IdentifyingthelinktypeofanIPaddress . . . . . . . . . . . . . 51
3.4 Failurerate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.4.1 UPtoDOWNfailures . . . . . . . . . . . . . . . . . . . . . . . 53
3.4.2 UPtoHOSEDfailures . . . . . . . . . . . . . . . . . . . . . . . 57
iv
3.5 Failureduration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.5.1 DoesThunderPingobservethedurationoffailures? . . . . . . . . 58
3.5.2 HowlongdoUP→DOWNfailureslast? . . . . . . . . . . . . . . 62
3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4 Energy-awareCellularDataScheduling 67
4.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.1.1 Signalvariesbylocation . . . . . . . . . . . . . . . . . . . . . . 70
4.2 Suitableapplications . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.2.1 Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.2.2 Streaming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.3 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.3.1 Predictingsignalwithsignaltracks . . . . . . . . . . . . . . . . 77
4.3.2 Schedulingsync . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.3.3 Schedulingstreaming . . . . . . . . . . . . . . . . . . . . . . . . 82
4.4 Simulation-basedevaluation . . . . . . . . . . . . . . . . . . . . . . . . 85
4.4.1 Syncing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
4.4.2 Streaming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
4.5 Relatedwork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
4.5.1 Predictingwirelessnetworkquality . . . . . . . . . . . . . . . . 90
4.5.2 Stabilityofcellularsignals . . . . . . . . . . . . . . . . . . . . . 91
4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5 ReliableBroadcastLast-mileLink 93
5.1 ThereliabilityofFMRDSmetropolitanradiobroadcasting . . . . . . . . 95
5.1.1 FMRDSlossrateislow . . . . . . . . . . . . . . . . . . . . . . 97
5.1.2 Wheretherearepeople,thereareFMtowers . . . . . . . . . . . 99
5.1.3 Everytransmissioncanbestored . . . . . . . . . . . . . . . . . . 102
5.1.4 FMReceiversaresmall . . . . . . . . . . . . . . . . . . . . . . . 102
5.2 DesignandimplementationofAbbie . . . . . . . . . . . . . . . . . . . . 103
5.2.1 Towersharingprotocol . . . . . . . . . . . . . . . . . . . . . . . 104
5.2.2 Over-the-airprotocol . . . . . . . . . . . . . . . . . . . . . . . . 104
5.2.3 RDS-to-LANbridge . . . . . . . . . . . . . . . . . . . . . . . . 107
5.2.4 Receiversoftware . . . . . . . . . . . . . . . . . . . . . . . . . . 108
5.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
5.3.1 Push: DNScacheupdates . . . . . . . . . . . . . . . . . . . . . 110
5.3.2 Anonymousandsynchronous: massreset . . . . . . . . . . . . . 112
5.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
5.4.1 Metropolitandeployment . . . . . . . . . . . . . . . . . . . . . . 113
5.4.2 Abbie’senergyconsumptiononanAndroidphone . . . . . . . . 115
5.4.3 Abbieend-to-endsynchronization . . . . . . . . . . . . . . . . . 116
5.4.4 RDSreceivercoldboot . . . . . . . . . . . . . . . . . . . . . . . 117
5.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
v
6 ConclusionsandOpenQuestions 119
6.1 Longevity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
6.2 Openquestions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
vi