MON: Mission-optimized Overlay Networks

0.39 MB·

by  Bruce Spang

MON: Mission-optimized Overlay Networks
B. Spang, A. Sabnis, R. Sitaraman, D. Towsley B. DeCleene
College of Information & Computer Sciences BAE Systems & Technology Solutions
U. Massachusetts - Amherst Burlington, MA 01803
Amherst, MA 01003 [email protected]
bspang,asabnis,ramesh,towsley @cs.umass.edu
{ }
Abstract—Large organizations often have users in multiple
sites which are connected over the Internet. Since resources are ☺ Mission PFO M MONtra
7 limited,communicationbetweenthesesitesneedstobecarefully Utility Mapping
1 orchestratedforthemostbenefittotheorganization.Wepresent Uplink
Overlay
0 a Mission-optimized Overlay Network (MON), a hybrid overlay
Operator
2 network architecture for maximizing utility to the organization.
Wecombineanofflineandanonlinesystemtosolvenon-concave
n
utility maximization problems. The offline tier, the Predictive
Ja Flow Optimizer (PFO), creates plans for routing traffic using a M M Site
Site
modelofnetworkconditions.Theonlinetier,MONtra,isawareof
7 Public
2 thepreciselocalnetworkconditionsandisabletoreactquicklyto Internet M
problemswithinthenetwork.Eithertieraloneisinsufficient.The

PFOmaytaketoolongtoreacttonetworkchanges.MONtraonly
]
I has local information and cannot optimize non-concave mission Site
N utilities.However,bycombiningthetwosystems,MONisrobust
andachievesnear-optimalutilityunderawiderangeofnetwork
.
s conditions. While best-effort overlay networks are well studied, Fig.1. MONArchitectureDiagram
c
our work is the first to design overlays that are optimized for
[
mission utility.

1 In this paper, in contrast to best-effort overlays studied in
v I. INTRODUCTION
prior work, we propose and study overlays that are driven by
8 Large organizations have users in multiple sites that are
explicitly stated mission goals. For instance, consider a multi-
9
connected over the Internet. A business may have multiple
0 site defense organization. The goals for the overlay are set by
officesaroundtheworldwhichneedtocommunicatewitheach
8 an operator who dictates the relative utility of various types
0 other.Adefenseorganizationmayhavepersonneldeployedat of communication that occur between the different sites. Note
. multiple sites, who need to communicate and fulfill specific
1 that the mission goals may vary with time, e.g., an urgent
0 mission goals. A retailer may have multiple shops, warehouse all-hands video conference watched by users in all the sites
7 locations, and offices. One traditional approach to facilitating
may take higher precedence than downloads, VOIP and other
1 communication between distributed sites of an organization
traffic classes. A Mission-optimized Overlay Network (MON)
:
v is to deploy a private enterprise network with dedicated dynamicallyallocatesavailableoverlayresourcestothetraffic
i infrastructure to fulfill the organization’s communication re-
X betweensitestomaximizeoverallmissionutility,enablingthe
quirements. However, an alternate approach is to build an
goals of the organization to be met.
r
a overlay on top of the public Internet, avoiding the need for
dedicated infrastructure.
A. MON Functionality
Overlays have been studied and built for the past 25 years
[1], [9], [24], [26], [29]. Large CDNs such as Akamai [21] MON takes as input the (time-varying) mission goals set
have built overlays for delivering web and video content by the operator and routes traffic on the overlay network to
since the late 1990’s [25]. These overlays are “best-effort”, meet these goals (see Figure 1). Each site is connected to the
in that they are concerned with providing higher reliability public Internet through a transport controller which performs
and performance than what the native Internet can offer for overlay routing.
all traffic using the overlay. Such best-effort overlays include We group the traffic between sites into a set of classes K,
caching overlays for Web content [7], routing overlays for where each traffic class represents a set of end-user sessions
reliablytransportinglivevideostreams[3],[17],P2Poverlays of a specific type (such as video, downloads, VOIP, etc...)
for downloads [24], [26], [29], and security overlays for pre- between a specific source and destination site. Each traffic
venting DDoS attacks [25]. However, best-effort overlays do class may use a set of overlay routes. The mission goals are
notexplicitlyoptimizethe“missiongoals”oftheorganization captured by mission utility functions specified by the overlay
that operates the overlay. operator. MON determines a set of sessions from each class
Symbol Meaning
and a rate for each session so as to maximize the cumulative
mission utility. K Set of traffic classes
MON is designed to continually adapt to change. The Nk Maximum number of sessions for class k
mission utility functions can change as mission goals change. ρk Set of routes usable by k
The number of sessions in each class that need to be routed L Set of underlay links
can change with user demand. The underlying Internet could Lˆ PFO’s estimate of L
suffer from failures that require traffic to be rerouted. To deal C Set of underlay link capacities
with these changes, MON continually adapts the number of Cˆ PFO’s estimate of C
sessionsnk andratesxk foreachclassk ∈K tooptimizethe Uk(xk) Per-class mission utility
mission utility. n Number of admitted sessions for class k
k
x Rate assigned to a flow
B. Our Contributions f
(cid:80)
x Aggregate class rate (x = x )
We propose a novel two-tiered overlay architecture for wk MONtra’s weight for clkass k ofn∈ρflkowff
f
MON that combines offline and online tiers. The offline
V (x ) MONtra’s utility for class k on flow f
f f
tier is called the Predictive Flow Optimizer (PFO), and it
γ MONtra’s stability constant
periodically performs a global optimization of the cumulative
missionutility.TheoutputofPFOis“mapped”toalower-level Fig.2. TableofNotation
online network transport mechanism called MONtra which
performs the actual routing of traffic in the network using
proportionally-fair utility functions (see Figure 1). We prove
VOIP) between a specific source and destination site. Each
that MON’s two-tiered architecture converges to an optimal
class uses a set of overlay routes ρ . Mission goals are
cumulative mission utility. An interesting aspect of our work k
captured by mission utility functions specified by the overlay
is a mapping process that allows us to implement arbitrary
operator. Specifically, associated with each class k K is a
non-decreasing mission utility functions using logarithmic ∈
function U (x ) that corresponds to the value of one session
transport utilities which are well-studied and have desirable k k
of class k receiving a rate of x . MON chooses a number of
properties such as proportional fairness. k
sessions n for each class k K and routes each chosen
To establish the real-world feasibility of MON, we im- k ∈
session of class k at a rate of x so as to maximize the
plement a prototype within the Deterlab [19] testbed. We k (cid:80)
cumulative mission utility expressed as n U (x ).
show that the PFO implementation using a bilinear global k k k k
optimizer, in combination with MONtra implemented on the MON has a two-tiered architecture which combines a non-
Deterlab nodes, is able to send traffic at rates that converge real-time global optimizer (PFO) with a distributed real-time
to a solution that achieves optimal mission utility. We also transport protocol (MONtra) to optimize mission utility. It
showthatthesystemisrobusttochangesinthenetwork,such is a novel application of the divide-and-conquer principle
as those caused by network partitions or congestion. Further, in network design. We use predicted global knowledge to
we show that the system is robust to changes in the number periodically “push” the overlay network into an optimized
of sessions, such as those caused by flash crowd events. We state. We maintain the network in a near-optimal state, even
also empirically evaluate MON when the network and traffic in the presence of sudden network changes (such as partitions
demands are not precisely known. In this case, we show that or congestion), using a mission-aware distributed transport
MON degrades gracefully and still provides a near-optimal protocol.Inthissection,wedescribethefollowingthreemajor
mission utility. aspects of the architecture (see Figure 1):
C. Roadmap The Predictive Flow Optimizer (Section II-A) solves an
•
WegiveanoverviewoftheMONarchitectureandadetailed optimization problem to come up with a plan for routing
description of each component in Section II. We describe our traffic in the MON. It solves a non-concave bilinear
prototype and experimental setup in detail in Section III. We optimization problem periodically using the predicted
implement MON and present our empirical results in Section network state and projected future traffic conditions.
IV. We compare MON to prior work in Section V and then MONtra (Section II-B) solves an online optimization
•
conclude in Section VI. problem to react to changes in the network. Using ideas
from network utility maximization [12], it adjusts the
II. THEMONARCHITECTURE sending rates of each site to solve a convex optimization
MON dynamically allocates available overlay resources to problem.
traffic between sites to maximize overall mission utility. In AmappingbetweenPFOandMONtra(SectionII-C)en-
•
order to do this, we need a precise definition of the mission sures that when PFO has full knowledge of the network,
utility maximization problem. We group the traffic between MONtra will converge to PFO’s target rates. Our main
sites into a set of classes K, where each traffic class k K result is that this convergence happens if MONtra has
represents a set of sessions of a specific type (e.g. vide∈o or the same gradient as PFO at the target rates.
A. Predictive Flow Optimizer amountoftrafficMONsendsforasinglesessionalongflowf.
The Predictive Flow Optimizer (PFO) outputs a routing To do so, PFO solves the following non-concave optimization
plan for the network that maximizes mission utility. It runs problem:
periodically using a prediction of future network conditions (cid:88) (cid:88)
max n U ( x ) (1)
k k f
and traffic demands. n,x
PFO performs “call admission” by choosing a number of k(cid:88)∈K (cid:88)f∈ρk
subject to n x Cˆ l Lˆ (2)
sessions n to admit in each class k K. It can decide k f l
k ≤ ∀ ∈
to admit no sessions at all for a given∈class k by setting k∈Kf∈ρk;l(cid:51)f
nk to zero. In addition, PFO chooses a per-flow rate xf to nk ≤Nk ∀k ∈K
providetoeachadmittedsessionofclassk K alongaroute x 0 k K,f ρ
f k
∈ ≥ ∀ ∈ ∈
f ρk in the network. The output of PFO is then used to n Z k K
∈ k
set the parameters of the MONtra controllers, a process we ∈ ∀ ∈
call “mapping”. Thus, PFO solves a hard global optimization Solving the Optimization Problem: The above optimiza-
problem,albeitinanon-realtimefashionusingpredictedtraffic tion problem is NP-Hard, since the number of sessions must
and network states. be an integer and the mission utility functions are not con-
The Optimization problem: PFO runs periodically and cave. However, for certain mission utility functions there are
solvesthefollowingoptimizationproblemtorouteapredicted optimization techniques which make solving this problem
setofsessionsontheoverlay.PFOtakesasinputasetoftraffic more feasible. For instance, if the mission utility functions
classes K. Each traffic class k K has a set of N sessions are piecewise linear, the problem becomes a bilinear program
k
that need to be routed from a sp∈ecific source site to a specific that can be solved efficiently using the ANTIGONE solver
destination site. For instance, a traffic class could be all the [20].Otherapproachesforsolvingnon-concavenetworkutility
VOIP phone calls made from a given site to another given maximization problems offline are described in the literature
site. (e.g. [10]).
PFOhastheoptiontosendtrafficalongdifferentpathsinthe
B. MON Transport Control (MONtra)
network. For example, it might send traffic directly from one
MONtraworksatthetransportlayerofMON,andisrespon-
site to another, or send it indirectly via a number of enclaves.
sibleforreactingrapidlytochangesintheunderlyingnetwork.
We say that a flow corresponds to the unique pair of a traffic
MONtra consists of weighted proportionally-fair congestion
class and a route through the network. For each class k, PFO
controllers,whichroutesessiontraffictomatchtheratechosen
has a set of ρ of possible flows. Each flow f ρ starts
k ∈ k by PFO.
at the source and ends at the destination associated with the
For each overlay route, MONtra’s controllers optimize the
class, using zero or more sites as intermediate nodes. Let ρ
transport-layer utility function V (x ) = w logx (which
denote the set of flows for all classes. f f f f
should not be confused with the mission utility function
PFO uses a model of the underlying network to pick a
U (x)). In Section II-C, we will describe how to choose
feasible set of rates for the flows in each class. Let L be the k
weightsforthesecontrollerssothattheyprovablyconvergeto
setofunderlaynetworklinks,andC thesetoflinkcapacities.
PFO’stargetrates.Inanattempttomakethemappingeasierto
The capacity of link l L is C . For convenience, we will
∈ l understand,wewillmodelMONtraasusingonecontrollerper
write l f to denote the links used by flow f and l f to
∈ (cid:51) sessiononaflow.It’spossibletoextendthemodeltocombine
denote the flows that use link l. PFO uses an estimate of the
all the sessions on a flow into one controller. MONtra solves
set of links Lˆ, and an estimate Cˆ of the link capacities.
PFOissaidtohavefullknowledgeofthenetworkifL=Lˆ thefollowingoptimizationproblem,whereListhesetoflinks
and C =Cˆ. At a minimum, PFO knows the uplinks for each in the network and Cl is the capacity of a link:
site and their capacities, i.e., Lˆ is the set of uplinks from the max (cid:88)n V (x )
k(f) f f
MONtra nodes to the public Internet (see Figure 1). Between x
f ρ
thesetwoextremes,PFOmayincorporatepartialknowledgeof (cid:88)∈ (cid:88)
subject to n x C l L
k f l
the links and capacities using tools from network tomography ≤ ∀ ∈
(e.g. [5]). k∈Kf∈ρk;l(cid:51)f
x 0 k K,f ρ
Inadditiontotheabove,theoverlayoperatorprovidesPFO f ≥ ∀ ∈ ∈ k
a mission utility function U (x ), for each traffic class k, We solve this optimization problem using techniques from
k k
representing the value of giving one session of class k a Network Utility Maximization [13]. We initialize the con-
rate of x . We assume that the mission utility functions are trollers to the the rate selected by PFO, then adapt the rate
k
fromR+ R,aresubdifferentiableeverywhere,andarenon- based on network feedback. After each success/loss signal,
→
decreasing. We do not assume that they are concave, in order weadjustratesaccordingtothefollowingupdaterules,where
to incorporate mission utilities for inelastic traffic [10]. γ is a constant chosen for stability:
For each traffic class k K, PFO outputs a number of
allowed sessions nk. For ea∈ch class k and possible flow f xf ←xf +γ·wf (after each successful packet)
∈ x γ x (for each loss)
ρ , PFO outputs a target rate x which corresponds to the f f
k f ← − ·
Note that unlike existing multipath TCP research (e.g. [11],
A
[13], [15], [22], [28]), MONtra uses uncoupled controllers.
We would like our controllers to exactly match the target
rateschosenbyPFO,whichimpliesthetransportoptimization
problem should have a unique optima. Unfortunately, the B C
coupled controllers in the multipath literature allow multiple
(a)TriangleTopology
optima.
Instead of using a window-based controller, we use a rate Site
r15
based controller which sends packets at a rate of xf. To do Router
this,wegeneratedelaysbetweeneachpacketsothatthepacket r16
sending process is Poisson with rate x .
f s30 r17
C. Mapping
r19 r18 s31
The mapping layer is responsible for ensuring that MONtra
r20 s33 s32 r0 r21
converges to the set of rates chosen by PFO. Intuitively, we
wouldlikethetransportutilityfunctiontoactlikethemission s34 r2 r1 r4 r22
utility functions in the vicinity of the target rates selected by
r3 s25 r5 r23 r10
PFO. If we could ensure that MONtra sends at the same rate
as PFO and has the same derivative at PFO’s target rates, r8 r6 s26 s35 s27 r24 r11 s29
MONtra might behave in the same way as PFO even if there r9 r7 s28 r12 r14
were slight changes to the network. The following theorem
r13
uses similar intuition and allows us to prove that MONtra
converges to PFO’s target rates. (b)AT&TTopology
Theorem 1. Suppose PFO has full knowledge of the network Fig.3. ExperimentalTopologies
and selects a set of rates A and a number of sessions n for
k
eachclassk thatmaximizesthemissionutilityfunctionU(A).
Foreachlinkl,letλ bethedualvariableassociatedwiththe III. EVALUATIONMETHODOLOGY
l
capacity constraint for link l. Fix the number of sessions for To show how MON performs in a realistic setting, we
each class in the transport layer to nk. Using the following implemented MON and ran it on a set of network topologies,
transport utility functions for a given flow f with class k, traffic scenarios, and mission utility functions as outlined
MONtra’s rates will converge to A: below.
V (x )=w logx NetworkTestbed.WerantheexperimentsonDeterlab[19],
f f f f
(cid:0)(cid:88) (cid:1) whichallowedustoallocatephysicallinuxmachinesforeach
w =n λ A
f k l f site and router in the network. We used Linux’s traffic control
l f system to set the network bandwidth. We used token bucket
∈
Proof: See Appendix A filterswithaburstsizeof100kbandamaximumqueuelatency
Note that if the only active constraint in the PFO solution of 5ms, which provides stable throughput when we transfer
is constraint (2), this mapping confirms the earlier intuition files between the hosts.
that we should match the PFO gradient at the target operating NetworkTopologies.Weemulatedthesmalltriangletopol-
point. Since only the rate-related constraints are active for ogy shown in Figure 3(a) to illustrate MON behavior in an
PFO,bythePFOKKTconditions, ∂ U(A)=((cid:80) n λ ). easier to understand context. We also emulated several large
Therefore, our mapping simplifies t∂oxfw = ∂ U(lA∈f)Ak. Al t topologies from [16] (AT&T USA, Bell Canada, BTN, and
thetargetoperatingpoint,thepartialderifvative∂xofftheMOfNtra Abilene).WepresenttheresultsfortheAT&Ttopologyshown
utility function with respect to a flow f is wf = ∂ U(A). in Figure 3(b) in most of our experiments.
Soinadditiontomatchingtherate,inthiscasAefwew∂oxufldalso Mission Utility Functions. We use the following two mis-
expect the MONtra utility functions to approximate the PFO sionutilityfunctionstoillustratethebehaviorofMON,though
utility functions close to the operating point. our system works for arbitrary mission utility functions:
Thismappingtheoremalsoworksforanyimplementationof (cid:40)
0 if x 0.8
MONtraandotherformulationsofPFOoptimizationproblem. U (x)= ≤
A
For instance, MONtra could use other classes of concave min(0.1x,0.005x+0.114) otherwise
transportutilityfunctions,oranothermethodofdistributedop- U (x)=0.2x
B
timizationsuchasbackpressurerouting.ThePFOoptimization
problem could use another way of combining per-flow rates, The type A mission utility function has non-concavity and
e.g. by summing the weighted rates across all paths instead of monotonicallyincreasingutilitywithdiminishingreturns.The
summing the rates across all paths. type B mission utility function increases linearly with rate
0.2
Utility Utility
Path Target Actual
0.32
0.15
A C 10.0 9.71
0.24 →
0.1 A B C 5.0 4.84
0.16 → →
5·10−2 8·10−2 (a)
Rate Rate Path Target Actual
0.4 0.8 1.2 1.6 2 0.2 0.4 0.6 0.8 1
(a)ClassA (b)ClassB s35 s26 10.0 9.24
→
Fig.4. MissionUtilityFunctionsforTrafficClassAandB s29 s31 10.0 9.24
→
s33 s35 10.0 9.24
→
s31 s33 10.0 9.23
→
s34 s32 6.0 5.9
→
s26 s34 6.0 5.68
without a point of diminishing returns. These functions are
→
shown in Figure 4. s32 s29 6.0 4.4
→
s32 s34 4.0 3.55
PFOImplementation.SinceourapproachreducesthePFO
→
s26 s32 4.0 3.4
optimization to a bilinear program, we used the ANTIGONE
→
solver [20] to efficiently solve the optimization problem. The s34 s29 4.0 3.24
→
solver uses branch-and-bound techniques and convex relax- (b)
ations such as McCormick’s envelopes which allow bilinear Fig.5. MONtraratesconvergedtotheoptimaltargetratessetbyPFOfor
optimization problems to be solved efficiently. boththetriangletopologyandtheAT&Ttopology
MONtra Implementation.TheMONtraimplementationis
based on multipath network utility maximization theory from
[13]. We used the controllers described in Section II-B with
per-packet acks to detect congestion. We gave each host an 100
ionffinseitsesiboancsklcohgosoefndabtya tPoFsOen.dW. We eseatlwthaeyssutasbedilittyhecnounmstbanert Utility 80 123HHHoooppp
n
γ = 0.001 and further improved stability by dividing γ by Missio 60 4Hop
the largest weight (i.e. maxr∈ρwr). malized 2400
or
N
IV. EVALUATIONRESULTS 0
ATT Bell-Canada BTN Abilene
Network
A. Does the overlay optimize mission utility?
Fig.6. Missionutilityincreaseswiththenumberofallowablehops,though
Our first experiment shows that MON optimizes mission
twohopsaresufficienttoobtainmostofthebenefits.
utility for simple scenarios. We give PFO full knowledge of
the network topology and capacities, and show that MONtra
convergestoPFO’stargetrates.Weshowthisforbothasimple
topology and a more realistic one.
For the simple topology, we set up three nodes on Deterlab
100
using the triangle topology shown in Figure 3(a). We set the
capacity between Node B and Node C to 5Mbps and set the
95
capacity of all other links to 10Mbps. We have two traffic y
classes, one between Node A and Node C and one between Utilit 90
Node B and Node C. PFO assigns a rate of 10 Mbps between on
Node A and Node C and a rate of 5 Mbps from Node A Missi 85
to Node C via Node B. Table 5(a) shows the rates MONtra d
e
z
converged to for the two flows. In this simple example, there ali 80
m
arenosharedlinksbetweentheflow.Notethattheactualrates or
N
achieved by MONtra is close to the target rate set by PFO. 75
RandomPaths
We also ran this experiment on the more realistic AT&T BestPaths
network topology. We set the capacity of all links to 10Mbps, 70
1 2 3 4 5 6 7 8 9 10
and ran PFO using the actual topology and capacities. Unlike Numberofpaths
thetriangleexperiment,PFOsharedlinksbetweenflows.Table
Fig.7. Theimpactofusingadditionalindirectpathsonmissionutility.
3(b) shows the rates MONtra converged to, which are close
to the rates computed by PFO.
1.2
4.0
1.0 3.5
3.0
0.8
y
Utilit bps)2.5
n0.6 M
Missio dput(2.0
0.4 Optimal Goo1.5
MONtra ActualA
0.2 FixedRate 1.0 ActualB
TCP 0.5 TargetA
0.0 TargetB
1 2 3 4 5 6 7 8 9
0.0
A Bcapacity 5 10 15 20
→ Numberofhighpriorityconnections
Fig.8. HowMONtrareactswhenPFOhasincorrecttopologyknowledge. Fig.9. HowMONtrareactswhendemandchanges.PFOchose11sessions
WeranMONtrawithPFOoutputsforanA→C capacityof3Mbps,and forthehighpriorityflow,butwevariedthenumberofsessionsfrom1to20.
plottedtheratesitconvergedtoforvariousvaluesofA→C capacities.
even when the network is slightly different from what PFO
B. Are overlay paths useful? If yes, how many? How does
expects. We compare MONtra to a transport layer which uses
random path selection compare with choosing the best paths?
Scalable TCP [14] without trying to match PFO’s rates, and a
Two sites can communicate directly with each other, or transport layer which always sends at PFO’s target rates.
they can communicate indirectly via a series of other sites.
This experiment used a mathematical model of the triangle
The impact of overlay routing on reliability and performance
network. We consider flows over links A B and B C.
for traditional best-effort overlays are well-known [1], [2], → →
We set link B C to 5Mbps, and run PFO with the capacity
[23]. Here we ask analogous questions for mission-optimized →
on A B set to 3Mbps. We then used those PFO rates for
overlays by ascertaining the benefits of overlay paths for →
differentcapacitiesonA B.Wetestedcapacitiesof1Mbps
mission utility maximization. →
to 10Mbps in intervals of 1Mbps.
To study the impact of overlay paths, we ran PFO on a
Figure 8 shows the result of this experiment. When the
varietyofrealnetworktopologiesfrom[16](AT&TUSA,Bell
capacity is 3Mbps and PFO has correct knowledge of the
Canada,BTN,andAbilene).Wesetuplinkcapacityto30Mbps
network, MONtra and the fixed-rate implementation match
and the capacity between routers to 10Mbps. To simulate a
PFO’s target rates. When link A B’s capacity is between
partially loaded overlay, we created traffic classes between →
1 and 4 Mbps, MONtra was also able to match PFO’s
half the sites. We first restricted PFO to use only direct, one-
rates. Although MONtra had different rates between 5 and
hop paths, then to using one-hop and two-hop paths, and then
10Mbps, it had a higher mission utility than the other two
to three- and four-hop paths. Note that the optimal mission
implementations.
utility cannot decrease when we allow a greater number of
hops, since more hops corresponds to a larger feasible region. D. What if demand changes?
Figure 6 shows the results of this experiment. In all networks,
In our previous experiments, we allowed PFO to choose
there was more than a 20% increase in mission utility from
the number of sessions used by MONtra. In a real system,
using two hop paths over just the one-hop direct path. In
the number of sessions might be less predictable; PFO may
this particular experiment, there was no benefit to using paths
choose to admit n sessions and then many more sessions
longer than two hops. k
could arrive. In this experiment, we show how MONtra reacts
Next,weconsidertheimpactofthenumberofallowedpaths
when the number of sessions is more or less than what PFO
(i.e. ρ ) on mission utility. It may be useful to not have to
k
| | chooses.
consider all overlay paths when running PFO. As shown in
Thisexperimentusesthetrianglenetwork.Wesetcapacities
Figure 7, adding just one well-chosen indirect path for each
A C to3Mbps,B C to5Mbps,andA C to10Mbps.
class can be sufficient to obtain the maximum mission utility.
→ → →
PFO split the capacity of the B C link between a flow of
However,wegenerallydonotknowthesinglebestpathbefore
→
class A and a flow of class B. PFO gave the flow of class A
running PFO. Including a few random paths is also sufficient
3Mbps for eleven sessions, and gave the flow of class B the
to improve mission utility. Just one randomly picked indirect
remaining 2Mbps for one session.
path gave 80% of the optimal mission utility, and adding four
WethenvarythenumberofsessionsfortheflowofclassA
random indirect paths gave 95% of optimal.
from1to20.Figure9showstheratesthatMONtraconverged
C. How does MONtra react to slight changes?
to. For 7 sessions and up, MONtra converged to PFO’s target
OurnextexperimentshowsthatMONtraoptimizesmission rate.Below7sessions,thetwosystemsdivergedandMONtra
utility better than existing transport layer implementations, sent more of flow B than PFO.
18
14 A→B→C
A C 16
→
12 B C
→ 14
(Mbps)180 Utility1102
Goodput 6 Mission 68
4
4
2 2
0 0
50 100 150 200 50 100 150 200
Time(sec) Time(sec)
(a)Ratesontriangletopology (b)Missionutilityontriangletopology
Fig. 10. MONtra rates and utilities with a link failure. We started MONtra with the correct PFO outputs. At 60 seconds, we reduced the capacity of the
A→B link.WeallowedMONtratoadapt,thenat140seconds,were-ranPFOtocorrecttherates.Theoverallutilitywentdownafterthefailure,though
PFOwasabletoimprovetheperformanceoveronlyMONtra.
new set of rates that increase the overall mission utility. The
140
dropinmissionutilityat140secondsisbecauseourprototype
120
doesn’t gracefully switch flows after PFO runs.
100 In the AT&T network, we set the capacity on all links to
y 10Mbps.At40seconds,weremovedalllinksconnectedtothe
Utilit 80 two routers with the highest degree (r2 and r13). Just before
n
Missio 60 1F5ig0usreec1o1ndssh,owwestrhee-ramnisPsFioOntuotislietlyecotvaernteiwmes.etAogfaitnar,gMetOraNtetrsa.
40
reacts to the network failure, and PFOselects new target rates
20 that increase mission utility. This shows that MON is resilient
to network failures.
0
50 100 150
Time(sec) V. RELATEDWORK
Overlays have been studied and built for the past 25 years.
Fig.11. MissionutilityovertimeforthepartitionexperimentontheAT&T
Large CDNs such as Akamai [21] have built overlays for
network. At 40 seconds, we removed all links connected to the two most
connected routers. At 150 seconds, we re-ran PFO which increased mission delivering web and video content since the late 1990’s [25].
utility. However,theseoverlaysare“best-effort”overlaysthatattempt
to provide higher reliability and performance than what the
native Internet can offer. Best-effort overlays come in many
E. How does MON react to failures?
flavors, including caching overlays for Web content [7], rout-
This experiment shows that MON adapts to sudden, large ing overlays for reliably transporting live video streams [3],
changes in the network. At the start of the experiment, we [17],P2Poverlaysfordownloads[24],[26],[29],andsecurity
providePFOfullknowledgeofthenetworkandfeeditsoutput overlays for preventing DDoS attacks [25]. However, such
to MONtra. We then cause a link failure, and allow MONtra overlays do not attempt to explicitly optimize the “mission
toreact.Wefinallyre-runPFOwiththeupdatedtopology,and goals” of the enterprise that operates the overlay, the focus of
it picks a new, more optimal set of rates to route around the our work.
failure. There has been prior work on overlay networks driven by
In the triangle network, we set the capacity between nodes quality-of-service (QoS). Networks that focus on QoS tend to
B and C to 5Mbps, and set all other capacities to 10Mbps. guarantee each flow a particular performance metric such as
MONtrabeginswiththeratesselectedbyPFO.At60seconds, rate,packetloss,jitteretc[8],[27].Incontrast,MONworksby
wesetthecapacitybetweennodesAandBto1Mbps.At140 optimizingthecumulativemissionutilityoftheoverlaytraffic.
seconds, we re-run PFO and update MONtra to use the new In particular, MON might sacrifice the QoS of some (lower-
flowsandrates.Figure10(a)showstheratesovertimeforthe priority) traffic flows to enhance the QoS of other (higher-
experiment, and Figure 10(b) shows the mission utility over priority) flows.
time. When the network failure occurs, MONtra adjusts its MONusesthenetworkutilitymaximization(NUM)frame-
sending rate to compensate. When PFO is re-run, it selects a work, first pioneered by Kelly [12] who described dis-
tributed algorithms for optimizing concave utilities. Non- [5] R. Castro, M. Coates, G. Liang, R. Nowak, and B. Yu. Network
concave NUM problems are one of the major open problems tomography: recent developments. Statistical science, pages 499–517,
2004.
in the field (e.g. [6]; sec. V-E). The general approaches are to
[6] M. Chiang, S. H. Low, A. R. Calderbank, and J. C. Doyle. Layering
solve the problem offline (e.g. [10]), which means the system as optimization decomposition: A mathematical theory of network
adapts slowlyat bestor not atall atworst to networkfailures, architectures. ProceedingsoftheIEEE,95(1):255–312,Jan2007.
[7] J.Dilley,B.Maggs,J.Parikh,H.Prokop,R.Sitaraman,andB.Weihl.
or to approximate the problem with a distributed algorithm
Globally distributed content delivery. Internet Computing, IEEE,
(e.g. [18]), which may not converge to a globally optimal 6(5):50–58,2002.
point. Our two-tiered approach of periodically performing a [8] Z.Duan,Z.-L.Zhang,andY.T.Hou. Serviceoverlaynetworks:Slas,
qos,andbandwidthprovisioning. IEEE/ACMTrans.Netw.,11(6):870–
global non-concave optimization to drive real-time transport
883,Dec.2003.
controllers with logarithmic utilities is a novel approach to [9] H.Eriksson. Mbone:Themulticastbackbone. Communicationsofthe
this classical problem. ACM,37(8),1994.
[10] M.FazelandM.Chiang.Networkutilitymaximizationwithnonconcave
utilitiesusingsum-of-squaresmethod. InProceedingsofthe44thIEEE
VI. CONCLUSIONS ConferenceonDecisionandControl,pages1867–1874,Dec2005.
[11] H.Han,S.Shakkottai,C.V.Hollot,R.Srikant,andD.Towsley. Multi-
In this paper, we proposed a Mission-optimized Overlay
pathtcp:ajointcongestioncontrolandroutingschemetoexploitpath
Network which routes traffic to explicitly optimize the goals diversityintheinternet.IEEE/ACMTransactionsonNetworking(TON),
of an organization. By incorporating utility into its routing 14(6):1260–1271,2006.
[12] F. Kelly. Charging and rate control for elastic traffic. European
decisions, MON’s decisions are more useful than prior, best-
TransactionsonTelecommunications,1997.
effort overlays. We proposed a novel, two-tiered architecture [13] F.KellyandT.Voice.Stabilityofend-to-endalgorithmsforjointrouting
for this overlay. The higher-tier PFO periodically performs a andratecontrol. SIGCOMMComput.Commun.Rev.,35(2):5–12,Apr.
2005.
global optimization of cumulative mission utility, allowing it
[14] T.Kelly. Scalabletcp:Improvingperformanceinhighspeedwidearea
to maximize the complex, non-concave utility functions that networks. SIGCOMMComput.Commun.Rev.,33(2):83–91,Apr.2003.
occur in practice. The lower-tieredMONtra transport protocol [15] R. Khalili, N. Gast, M. Popovic, U. Upadhyay, and J.-Y. Le Boudec.
uses PFO’s target rates to keep the overlay network near Mptcp is not pareto-optimal: performance issues and a possible solu-
tion. In Proceedings of the 8th international conference on Emerging
the optimal operating point by responding to network and networkingexperimentsandtechnologies,pages1–12.ACM,2012.
traffic events in real-time. This architecture also opens up a [16] S. Knight, H. X. Nguyen, N. Falkner, R. Bowden, and M. Roughan.
new approach to investigating non-concave Network Utility Theinternettopologyzoo. IEEEJournalonSelectedAreasinCommu-
nications,29(9):1765–1775,October2011.
Maximization (NUM) problems which has not been studied
[17] L. Kontothanassis, R. Sitaraman, J. Wein, D. Hong, R. Kleinberg,
in the prior literature. B. Mancuso, D. Shaw, and D. Stodolsky. A transport layer for live
Weimplementedaprototypeofourarchitectureonphysical streaming in a content delivery network. Proceedings of the IEEE,
92(9):1408–1419,2004.
hardware, and showed that MON converges quickly to a state
[18] J.W.Lee,R.R.Mazumdar,andN.B.Shroff.Non-convexoptimization
where the cumulative mission utility is maximized. By using and rate control for multi-class services in the internet. IEEE/ACM
mission utility information, MON can respond gracefully to TransactionsonNetworking,13(4):827–840,Aug2005.
[19] J. Mirkovic, T. V. Benzel, T. Faber, R. Braden, J. T. Wroclawski, and
network failures and changes in demand.
S.Schwab. Thedeterproject:Advancingthescienceofcybersecurity
experimentation and test. In In Technologies for Homeland Security
ACKNOWLEDGMENTS (HST),2010IEEEInternationalConferenceon,page7,2010.
[20] R.MisenerandC.A.Floudas.ANTIGONE:AlgorithmsforcoNTinuous
The research reported in this paper was sponsored by /IntegerGlobalOptimizationofNonlinearEquations.JournalofGlobal
DARPA under Contract No. N66001-15-C-4045. The views Optimization,2014. DOI:10.1007/s10898-014-0166-2.
[21] E.Nygren,R.K.Sitaraman,andJ.Sun.Theakamainetwork:aplatform
and conclusions contained in this document are those of the
for high-performance internet applications. SIGOPS Oper. Syst. Rev.,
authors and should not be interpreted as representing the 44:2–19,August2010.
official policies of DARPA and the the U.S. Government. The [22] Q.Peng,A.Walid,J.Hwang,andS.H.Low. Multipathtcp:Analysis,
design, and implementation. IEEE/ACM Transactions on Networking
U.S. Government is authorized to reproduce and distribute
(ToN),24(1):596–609,2016.
reprints for Government purposes notwithstanding any copy- [23] H.Rahul,M.Kasbekar,R.Sitaraman,andA.Berger. Towardsrealizing
right notation hereon. Approved for public release; unlimited the performance and availability benefits of a global overlay network.
InProc.ofPassiveandActiveMeasurementConference.Citeseer,2006.
distribution.
[24] S. Ratnasamy, P. Francis, M. Handley, R. Karp, and S. Shenker. A
scalable content-addressable network. In Proceedings of the 2001
REFERENCES ConferenceonApplications,Technologies,Architectures,andProtocols
for Computer Communications, SIGCOMM ’01, pages 161–172, New
[1] D. Andersen, H. Balakrishnan, F. Kaashoek, and R. Morris. Resilient York,NY,USA,2001.ACM.
overlaynetworks. SIGOPSOper.Syst.Rev.,35(5):131–145,Oct.2001. [25] R. K. Sitaraman, M. Kasbekar, W. Lichtenstein, and M. Jain. Overlay
[2] D. G. Andersen, A. C. Snoeren, and H. Balakrishnan. Best-path vs. networks:Anakamaiperspective. AdvancedContentDelivery,Stream-
multi-pathoverlayrouting. InProceedingsofthe3rdACMSIGCOMM ing,andCloudServices,pages305–328,2014.
Conference on Internet Measurement, IMC ’03, pages 91–100, New [26] I. Stoica, R. Morris, D. Liben-Nowell, D. R. Karger, M. F. Kaashoek,
York,NY,USA,2003.ACM. F.Dabek,andH.Balakrishnan. Chord:Ascalablepeer-to-peerlookup
[3] K.Andreev,B.M.Maggs,A.Meyerson,andR.K.Sitaraman.Designing protocolforinternetapplications.IEEE/ACMTrans.Netw.,11(1):17–32,
overlaymulticastnetworksforstreaming.InProceedingsofthefifteenth Feb.2003.
annualACMsymposiumonParallelalgorithmsandarchitectures,pages [27] L. Subramanian, I. Stoica, H. Balakrishnan, and R. Katz. OverQoS:
149–158.ACM,2003. An Overlay Based Architecture for Enhancing Internet QoS. In 1st
[4] S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge SymposiumonNetworkedSystemsDesignandImplementation(NSDI),
UniversityPress,NewYork,NY,USA,2004. SanFrancisco,CA,March2004.
[28] D. Wischik, C. Raiciu, A. Greenhalgh, and M. Handley. Design, ∂ (cid:88)
implementationandevaluationofcongestioncontrolformultipathtcp. U(A)= nkλl (7)
∂x
InNSDI,volume11,pages8–8,2011. f
l f
[29] B. Y. Zhao, J. D. Kubiatowicz, and A. D. Joseph. Tapestry: A ∈
∂ (cid:88)
fault-tolerantwide-areaapplicationinfrastructure. SIGCOMMComput. U(A)= n λ (8)
k k
Commun.Rev.,32(1):81–81,Jan.2002. ∂n
k
k k
∈
APPENDIXA For a set of target rates A and link dual variables λ to
(cid:48) (cid:48)l
PROOFOFMAPPINGTHEOREM be optimal for the MONtra problem, they must satisfy the
following KKT conditions for the MONtra problem:
Theorem 1. Suppose PFO has full knowledge of the network
and selects a set of rates A and a number of sessions n for (cid:88) (cid:88)
k n A C l L (9)
eachclassk thatmaximizesthemissionutilityfunctionU(A). k (cid:48)f ≤ l ∀ ∈
Foreachlinkl,letλ bethedualvariableassociatedwiththe k∈Kf∈ρk;l(cid:51)f
l A 0 k K,f ρ (10)
capacity constraint for link l. Fix the number of sessions for (cid:48)f ≥ ∀ ∈ ∈ k
each class in the transport layer to nk. Using the following λ(cid:48)l ≥0, ∀l∈L (11)
(cid:88) (cid:88)
transport utility functions for a given flow f with class k, λ ( n A C )=0 l L (12)
MONtra’s rates will converge to A: (cid:48)l k (cid:48)f − l ∀ ∈
k∈Kf∈ρk;l(cid:51)f
d (cid:88)
Vf(xf)=wflogxf dx Vf(xf)= nkλ(cid:48)l (13)
wf =nk(cid:0)(cid:88)λl(cid:1)Af f l∈f
l∈f Since the MONtra problem is concave, if some point A(cid:48),λ(cid:48)l
satisfy the MONtra KKT conditions then it is optimal and
Proof: Recall the PFO solves the following optimization
MONtra will converge to that point. We will show that PFO’s
problem:
optimal rates A and link dual variables λ satisfy these KKT
l
(cid:88) (cid:88)
max n U ( x ) conditions,soMONtrawillconvergetothePFOoptimalpoint.
k k f
n,x TheMONtraKKTconditions(9)and(10)aresatisfiedsince
k(cid:88)∈K (cid:88)f∈ρk they are identical to the PFO KKT conditions (3) and (4)
subject to n x Cˆ l Lˆ
k f ≤ l ∀ ∈ because we’ve fixed the number of flows in the transport to
k∈Kf∈ρk;l(cid:51)f the number of flows chosen by PFO. MONtra KKT condition
n N k K
k ≤ k ∀ ∈ (11) is satisfied by PFO KKT condition (5). MONtra KKT
xf 0 k K,f ρk condition (12) is satisfied by the PFO KKT condition (6).
≥ ∀ ∈ ∈
nk Z k K MONtra KKT condition (13) is satisfied by our choice of wf
∈ ∀ ∈
since for flow f with class k:
And MONtra solves the following optimization problem:
d w
f
(cid:88) Vf(xf)=
max nk(f)Vf(xf) dxf Af
x (cid:88)
f ρ = n λ
(cid:88)∈ (cid:88) k l
subject to nkxf Cl l L l f
≤ ∀ ∈ ∈
k∈Kf∈ρk;l(cid:51)f Therefore A,λl satisfy the KKT conditions for MONtra.
xf ≥0 ∀k ∈K,f ∈ρk Since MONtra is concave, it will converge to A,λl.
Since we assume PFO has perfect information, Lˆ =L and
Cˆ =C.
SincethesetofratesA,thenumberofadmittedsessionsn ,
k
and the dual variables for each link λ and for each constraint
l
onthenumberofsessionsλ areoptimalforPFO,theysatisfy
k
the following KKT conditions (See [4]; Sec. 5.5.3):
(cid:88) (cid:88)
n A C l L (3)
k f l
≤ ∀ ∈
k∈Kf∈ρk;l(cid:51)f
A 0 k K,f ρ (4)
f k
≥ ∀ ∈ ∈
n N k K
k k
≤ ∀ ∈
n Z k K
k
∈ ∀ ∈
λ 0, l L (5)
l
≥ ∀ ∈
(cid:88) (cid:88)
λ ( n A C )=0 l L (6)
l k f l
− ∀ ∈
k∈Kf∈ρk;l(cid:51)f